Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[Fixed Thx] dllcall(readfile) and FilePointerEx help


  • Please log in to reply
8 replies to this topic
Razlin
  • Members
  • 454 posts
  • Last active: Feb 05 2014 06:21 PM
  • Joined: 05 Nov 2007
Hi guys,

I'm new to the dll business I hope to get it one day its painfully slow for some odd reason.

I'm trying to read from a file starting at offset character 50 for example.

I got the reading working fine from help file.

DllCall("ReadFile", UInt, hFile, str, TestString, UInt, BytesToRead, UIntP, BytesActuallyRead, UInt, 0)

info from msdn site http://msdn.microsof...467(VS.85).aspx

BOOL WINAPI ReadFile(
__in HANDLE hFile,
__out LPVOID lpBuffer,
__in DWORD nNumberOfBytesToRead,
__out_opt LPDWORD lpNumberOfBytesRead,
__inout_opt LPOVERLAPPED lpOverlapped
);


what I need help is with overlapped.

If hFile is opened with FILE_FLAG_OVERLAPPED and lpOverlapped is not NULL, the read operation starts at the offset that is specified in the OVERLAPPED structure, and ReadFile may return before the read operation is complete.


overlapped stuct from msdn

typedef struct _OVERLAPPED {
ULONG_PTR Internal;
ULONG_PTR InternalHigh;
union {
struct {
DWORD Offset;
DWORD OffsetHigh;
};
DVOID Pointer;
};
HANDLE hEvent;
} OVERLAPPED,
*LPOVERLAPPED;


This means nothing to me.

how do I configure Offset in the overlapped stuct to be 50.

I probably aint even explainning this right

[ Moderator!: MSDN link fixed ]
-=Raz=-

poetbox
  • Members
  • 113 posts
  • Last active: Sep 11 2013 08:05 AM
  • Joined: 07 Jan 2007
FileRead, Contents, C:\Address List.txt

StringRight, OutputVar, Contents, StrLen(Contents)-50

msgbox %OutputVar%


Razlin
  • Members
  • 454 posts
  • Last active: Feb 05 2014 06:21 PM
  • Joined: 05 Nov 2007
@Poetbox
the idea is to not use readfile and use the dllcall. reason bellow.


I have written an app already that uses readfile and does work but its not fast enough to read large files.

I was testing my app that I wrote on a 190mg file it worked great locally like less then 10 seconds to read but the logs are on network drives and using fileread was slow on network as it has to read in the entire file like 5-10 minutes.

if I use a dllcall I can "only" read the end of the file therefore reading will be done exceptionaly faster.
Hence why I need dllcall help.
thanks for your time and effort though
-=Raz=-

Zippo()
  • Guests
  • Last active:
  • Joined: --
Here is how you calculate the low byte/high byte of a file size:

a := 0xfffffff
b := a&0xffffffff
c := Floor(a/0xffffffff)
MsgBox % "LowByte = "b . " HighByte = " . c

Subtract your file pointer from the file size then run the math on it.

Zippo()
  • Guests
  • Last active:
  • Joined: --
Sorry I should explain that better. It's been a long week :(

To read the last 50 bytes of a file, fill in the structure like this:
FileGetSize, filesize, filename.ext
filesize := filesize-50 ;Read the last 50 bytes of the file
filepointerlb := filesize&0xffffffff
filepointerhb := Floor(filesize/0xffffffff)

NumPut(0, OVERLAPPED, 0, "UInt")
NumPut(0, OVERLAPPED, 4, "UInt")
NumPut(filepointerlb, OVERLAPPED, 8, "UInt")
NumPut(filepointerhb, OVERLAPPED, 12, "UInt")
NumPut(0, OVERLAPPED, 16, "UInt")
NumPut(hEvent, OVERLAPPED, 20, "UInt") ;hEvent = handle returned from CreateEvent

I don't know what kind of problems you'll have if you don't sync the fileIO by waiting on an event, but you can specify 0 for hEvent. I've never had a use for this structure.

Like I said, it's been a long week. If I made any mistakes hopefully someone will correct me :)

Lexikos
  • Administrators
  • 9844 posts
  • AutoHotkey Foundation
  • Last active:
  • Joined: 17 Oct 2006
OVERLAPPED structures are only 20 bytes - the union means that Offset and Pointer share the same memory, and hEvent is at 16, not 20.

a := 0xfffffff
b := a&0xffffffff
c := Floor(a/0xffffffff)
MsgBox % "LowByte = "b . " HighByte = " . c

Technically that should be:
a := 0xfffffff
b := a&0xffffffff
c := [color=red]a >> 32[/color]
MsgBox % "Low[color=red]Dword[/color] = " . b . " High[color=red]Dword[/color] = " . c
You may find it simpler to set the file pointer with SetFilePointerEx, and simply read however many bytes you need. For instance, to read the last 50 bytes, set the file pointer to the file size minus 50 bytes, then read 50 bytes. OVERLAPPED structures are typically only used for overlapped I/O (i.e. the program continues whatever it was doing while the system reads the file into memory.)

Razlin
  • Members
  • 454 posts
  • Last active: Feb 05 2014 06:21 PM
  • Joined: 05 Nov 2007
This is really advanced for me at this point but hopefully I can get a hand at figuring this out.
Here is my attempt with filepointerEx

DO NOT RUN THIS CODE
As I dont know what it will do.
In theory it shouldnt do anything but its playing with file handles and with my luck something I did is wrong.


am I going in the right direction with this?
and what values do I put where its Orange?

/*
SetFilePointerEx(
  __in       HANDLE hFile,
  __in       LARGE_INTEGER liDistanceToMove,
  __out_opt  PLARGE_INTEGER lpNewFilePointer,
  __in       DWORD dwMoveMethod
);

dwMoveMethod
0 FILE_BEGIN
1 FILE_CURRENT
2 FILE_END
*/

;### Get file handle    this section should be fine.   "should be"
GENERIC_READ = 0x80000000  ; Open the file for reading rather than writing.
OPEN_EXISTING = 3  ; This mode indicates that the file to be opened must already exist.
FILE_SHARE_READ = 0x1 ; This and the next are whether other processes can open the file while we have it open.
FILE_SHARE_WRITE = 0x2
filename = c:\test.txt
hFile := DllCall("CreateFile", str, FileName, UInt, GENERIC_READ, UInt, FILE_SHARE_READ|FILE_SHARE_WRITE, UInt, 0, UInt, OPEN_EXISTING, Uint, 0, UInt, 0)
;###


;###Numput stuff
VarsetCapacity(NewFilePointer, 16, 0)    ;I think it's 16.

;This is where I'm the most confused / lost / helpless / ... ...
;Whats right here and whats wrong here.. or is there anything right lol.
numput(hfile, 	NewFilePointer, 0, "Int")
numput(10, 		NewFilePointer, 4, "UInt")
numput([color=orange]DWORD[/color], 	NewFilePointer, 8, "UInt")
numput(3, 		NewFilePointer, 12, "UInt")

ReturnValue := DllCall("SetFilePointerEx", UInt, &NewFilePointer)

DllCall("CloseHandle", UInt, hFile)  ; Close the file.

Thanks again to all whom have learned this and can shed some light.
-=Raz=-

Lexikos
  • Administrators
  • 9844 posts
  • AutoHotkey Foundation
  • Last active:
  • Joined: 17 Oct 2006
SetFilePointerEx accepts four parameters, not a structure with four fields.
BOOL SetFilePointerEx(
  HANDLE hFile,                    ; UInt
  LARGE_INTEGER liDistanceToMove,  ; Int64
  PLARGE_INTEGER lpNewFilePointer, ; Int64P
  DWORD dwMoveMethod               ; UInt
);


Razlin
  • Members
  • 454 posts
  • Last active: Feb 05 2014 06:21 PM
  • Joined: 05 Nov 2007
@Lexikos
Thank you.
Thank you.
Thank you.


I dont know why I was trying to make it a struct.. I guess the overlapped got me all mixed up.
then I had to use numput and that got me all mixed up.

and then boom... you saved the day.

Thank you, thank you.. I got it working.

ReturnValue := DllCall("SetFilePointerEx", UInt, hfile, int64, -charcount, Int64P, newval, Uint, 2)

Woot.
-=Raz=-