[Lib] ObjDump / ObjLoad - Object Backup

Post your working scripts, libraries and tools for AHK v1.1 and older
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

[Lib] ObjDump / ObjLoad - Object Backup

18 May 2014, 05:48

Note!
  • Only standard objects are supported, ComObject, RegExMatchObject, FileObject, Func and DynaCall (AutoHotkey_H only) objects are not supported.
  • Struct (AutoHotkey_H only) object can be dumped but ObjLoad will restore a normal object instead of Struct object.
  • When value is of type string or raw data, ObjDump will dump only the string by default and always for keys, so StrPut(key)*(A_IsUnicode?2:1) is used. Additional parameter DumpBuffer can be used to force dumping obj.GetCapacity(value) instead of StrPut(value)*(A_IsUnicode?2:1) for values.
  • Numbers consume only required bytes using NumSize/NumType.
A small drawback is that resulting file might get quite big, especially when using DumpBuffer=true.
Dump object to variable/memory:
size := ObjDump(Object,Variable,DumpBuffer)
Load object from variable/memory:
Obj := ObjLoad(Address)
Dump object to file:
size := ObjDump(FilePath,Object,DumpBuffer)
Load object from file:
Obj := ObjLoad(FilePath)
AutoHotkey v1
AutoHotkey v2
UPDATE:
  • *16.11.2017
  • Fix to handle properly keys and values containing numbers and spaces or tabs
  • Update latest v2 syntax
    *25.02.2016
  • Fix ObjLoad bug.
  • ObjDump and ObjLoad can now handle object reference to itself or another object within dumped object.
    *02.11.2015
  • Fix saving binary memory of key (use obj.GetAddress(k) instead of &v)
    *20.08.2015
  • Include size of dumped object for simplified use
    Note wrote:New ObjLoad() function is not compatible to the old functions and cannot load a file dumped with old ObjDump function.
    You can rename the old ObjLoad function to ObjLoad2 and run this with new ObjDump function:
    ObjDump(ObjLoad2(File),File)
  • 20.05.2014
  • Implemented obj-64 -> file -> obj-32 and the other way round.
  • Use only the required memory for numbers.
  • Additional parameter DumpBuffer, when true obj.GetCapacity(key) is used else StrPut(value)*(A_IsUnicode?2:1)
  • 18.05.2014
  • Fixed writing large objects
Old_Versions
User avatar
cyruz
Posts: 346
Joined: 30 Sep 2013, 13:31

Re: [Lib] ObjDump / ObjLoad - Object Backup

18 May 2014, 10:01

This is seriously good stuff. I asked for this as a builtin some time ago... Thank you for your work!
ABCza on the old forum.
My GitHub.
Guest10
Posts: 578
Joined: 01 Oct 2013, 02:50

Re: [Lib] ObjDump / ObjLoad - Object Backup

18 May 2014, 16:28

this script is way over my head. any examples a newbie can use this? :geek:
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

18 May 2014, 16:39

Save_Load_Object_to_file
Save_Load_Object_to_variable
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

19 May 2014, 05:23

Interesting attempt, but obviously failed.

Code: Select all

ObjDump(A_ScriptDir "\1.b",AhkDllThread())
AhkThread := ObjLoad(A_ScriptDir "\1.b")
AhkThread.ahktextdll("MsgBox,Test")
MsgBox
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

19 May 2014, 05:42

There are 2 things why this would not work:
- when ObjDump finishes the reference to the AhkDllThread object is deleted and MemoryFreeLibrary is called so there is no code anymore.
- Only standard objects are supported, RegEx, DynaCall and Struct objects are not standard objects and cannot be dumped/loaded properly.
strobo
Posts: 125
Joined: 30 Sep 2013, 15:24

Re: [Lib] ObjDump / ObjLoad - Object Backup

19 May 2014, 09:28

Nice Lib.
Time (encode, decode) and Space benchmark vs JSON implementations (and/or other (de)serializations) would be interesting.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

19 May 2014, 12:04

Thanks ;)

Time to decode encode depends on size of the object, in avarage ObjDump/ObjLoad will be faster.
I have written ObjDump to support high speed dump for huge objects and dumping any content, so even raw data is supported.

My tests shown, the larger the object, the faster ObjLoad and ObjDump would be, for example even here ObjDump is faster:

Code: Select all

; SetBatchLines, -1 ; uncomment when using AHK v1
ObjSize:=1000000
VarToUse:="A_TickCount" ; use A_TickCount, A_ScriptFullPath, A_Now ...
TimesToRun:=1
obj:=[]
Loop % ObjSize
  obj[A_Index]:=%VarToUse%
s:=A_TickCount
Loop % TimesToRun
{
  out:=""
  for k,v in obj
    out.=k "`t" v "`n" 
}
forLoop:=A_TickCount-s
s:=A_TickCount
Loop % TimesToRun
{
  out:=""
  ObjDump(obj,var) 
}
objDump:=A_TickCount-s
MsgBox % "For loop: " forLoop "`nObjDump: " objDump
ExitApp
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

20 May 2014, 16:14

Update:
  • Implemented obj-64 -> file -> obj-32 and the other way round.
  • Use only the required memory for numbers(char,short,integer,int64,double).
  • Changed to consume only one byte for data type.
  • Additional parameter DumpBuffer, when 0/false (default) StrPut(value)*(A_IsUnicode?2:1) is used, eslse obj.GetCapacity(key) is used to dump all data for values.
Note wrote:New ObjLoad() function is not compatible to the old functions and cannot load a file dumped with old ObjDump function.
You can rename the old ObjLoad function to ObjLoad2 and run this with new ObjDump function:
ObjDump(ObjLoad2(File),File)
User avatar
joedf
Posts: 8940
Joined: 29 Sep 2013, 17:08
Location: Canada
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

20 May 2014, 20:44

Finally! I wanted this for quite a while, thanks HotkeyIt ;)
Image Image Image Image Image
Windows 10 x64 Professional, Intel i5-8500, NVIDIA GTX 1060 6GB, 2x16GB Kingston FURY Beast - DDR4 3200 MHz | [About Me] | [About the AHK Foundation] | [Courses on AutoHotkey]
[ASPDM - StdLib Distribution] | [Qonsole - Quake-like console emulator] | [LibCon - Autohotkey Console Library]
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

15 Aug 2014, 03:49

hi HotKeyIt

I try to use this function to pass objects between processes,Use Help on OnMessage examples:
The script stagnation, where the reason for this?

Receive

Code: Select all

#SingleInstance
OnMessage(0x4a, "Receive_WM_COPYDATA")  ; 0x4a 为 WM_COPYDATA
WinSetTitle, ahk_id %A_ScriptHwnd%, ,Receive
return

Receive_WM_COPYDATA(wParam, lParam)
{
ToolTip,Receive_WM_COPYDATA
j := ObjLoad(lParam,wParam)
MsgBox % j.A 
}

Code: Select all

TargetScriptTitle = Receive ahk_class AutoHotkey

F1:: 
obj:={a:1,b:2,c:[1,2,"test"],d:{a:0}}
Size := ObjDump(obj,StringToSend)
result := Send_WM_COPYDATA(StringToSend,Size, TargetScriptTitle)

return

Send_WM_COPYDATA(ByRef StringToSend,Size, ByRef TargetScriptTitle){
    Prev_DetectHiddenWindows := A_DetectHiddenWindows
    Prev_TitleMatchMode := A_TitleMatchMode
    DetectHiddenWindows On
    SetTitleMatchMode 2
    SendMessage, 0x4a, Size, &CopyDataStruct,, %TargetScriptTitle%  
    DetectHiddenWindows %Prev_DetectHiddenWindows%
    SetTitleMatchMode %Prev_TitleMatchMode%       
    return ErrorLevel 
}
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

15 Aug 2014, 05:25

Enjoy ;)
DynaRun()

Code: Select all

PID:=DynaRun("
(
#SingleInstance
OnMessage(0x4a, ""ON_WM_COPYOBJECT"")  ; 0x4a 为 WM_COPYDATA
DetectHiddenWindows,On
WinSetTitle, ahk_id `%A_ScriptHwnd`%, ,Receive
return
ON_WM_COPYOBJECT(wParam, lParam) {
  j := ObjLoad(NumGet(lParam+A_PtrSize*2,""PTR""),NumGet(lParam + A_PtrSize,""PTR""))
  MsgBox `% j.A ""``n"" j.c.3
}
)")

TargetScriptTitle = Receive ahk_class AutoHotkey
F1:: 
obj:={a:1,b:2,c:[1,2,"test"],d:{a:0}},Size := ObjDump(obj,DataToSend)
result := WM_COPYOBJECT(TargetScriptTitle,&DataToSend,Size)
return
Esc::
Process,Close,%PID%
ExitApp

WM_COPYOBJECT(Title, dwData, cbData) {
	static WM_COPYDATA := 0x4A
    Prev_DetectHiddenWindows := A_DetectHiddenWindows
    ,Prev_TitleMatchMode := A_TitleMatchMode
    DetectHiddenWindows On
    SetTitleMatchMode 2
	VarSetCapacity(COPYDATASTRUCT, 3*A_PtrSize, 0)
	,NumPut(dwData,  &COPYDATASTRUCT, 2*A_PtrSize,"PTR")
	,NumPut(cbData,  &COPYDATASTRUCT, 1*A_PtrSize,"PTR")
	SendMessage, %WM_COPYDATA%, 0, &COPYDATASTRUCT,,%title%
    DetectHiddenWindows %Prev_DetectHiddenWindows%
    ,SetTitleMatchMode %Prev_TitleMatchMode%    
	return ErrorLevel == "FAIL" ? false : true
}
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

15 Aug 2014, 08:59

Thanks. :D

Incidentally:
erratic Crash disappeared.
expect H v1.1.15

----------------------------
I try to use an object to pass parameters, but why not correct?

Code: Select all




obj:={a:A_TickCount}
PID:=DynaRun("MsgBox % Parameters.a",1,obj)
MsgBox,Test

DynaRun(s, pn:="", pr:=""){ ; s=Script,pn=PipeName,n:=,pr=Parameters,p1+p2=hPipes,P=PID
   static AhkPath:="""" A_AhkPath """" (A_IsCompiled||(A_IsDll&&DllCall(A_AhkPath "\ahkgetvar","Str","A_IsCompiled"))?" /E":"") " """
   if (-1=p1 := DllCall("CreateNamedPipe","str",pf:="\\.\pipe\" (pn!=""?pn:"AHK" A_TickCount),"uint",2,"uint",0,"uint",255,"uint",0,"uint",0,"Ptr",0,"Ptr",0))
      || (-1=p2 := DllCall("CreateNamedPipe","str",pf,"uint",2,"uint",0,"uint",255,"uint",0,"uint",0,"Ptr",0,"Ptr",0))
      Return 0
   ; allow compiled executable to execute dynamic scripts. Requires AHK_H
   Run, % AhkPath pf """ " (IsObject(pr)?1:pr),,UseErrorLevel HIDE, P

if IsObject(pr)
{

Size := ObjDump(pr,Data)
    VarSetCapacity(COPYDATASTRUCT, 3*A_PtrSize, 0)
    ,NumPut(&Data,  &COPYDATASTRUCT, 2*A_PtrSize,"PTR")
    ,NumPut(Size,  &COPYDATASTRUCT, 1*A_PtrSize,"PTR")
s:="Parameters := ObjLoad(NumGet(" &COPYDATASTRUCT "+A_PtrSize*2,""PTR""),NumGet(" &COPYDATASTRUCT " + A_PtrSize,""PTR""))`n" s
}
  
   If ErrorLevel
      return 0,DllCall("CloseHandle","Ptr",p1),DllCall("CloseHandle","Ptr",p2)
   DllCall("ConnectNamedPipe","Ptr",p1,"Ptr",0),DllCall("CloseHandle","Ptr",p1),DllCall("ConnectNamedPipe","Ptr",p2,"Ptr",0)
   if !DllCall("WriteFile","Ptr",p2,"Wstr",s:=(A_IsUnicode?chr(0xfeff):"") s,"UInt",StrLen(s)*(A_IsUnicode ? 2 : 1)+(A_IsUnicode?4:6),"uint*",0,"Ptr",0)
      P:=0
   DllCall("CloseHandle","Ptr",p2)
   Return P
}

arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

15 Aug 2014, 16:30

我稍加修改,使之便于阅读,也许会提升些许效率。
I slightly modified to make it easier to read, and perhaps will increase a little efficiency.

Code: Select all

ObjDump(obj,ByRef var,mode:=0){
	If IsObject(var)
	{ ; FileAppend mode
		If FileExist(obj)
		{
			FileDelete,%obj%
			If ErrorLevel
				return
		}
		f:=FileOpen(obj,"rw-rwd"),VarSetCapacity(v,sz:=RawObjectSize(var,mode),0)
			,RawObject(var,&v,mode),count:=sz//65536,ptr:=&v
		Loop % count
			f.RawWrite(ptr+0,65536),ptr+=65536
		return sz,f.RawWrite(ptr+0,Mod(sz,65536)),f.Close()
	} 
	else 
		return sz,VarSetCapacity(var,sz:=RawObjectSize(obj,mode),0),RawObject(obj,&var,mode)
}

RawObject(obj,addr,buf:=0){
	; Type.Enum:    Char.1 UChar.2 Short.3 UShort.4 Int.5 UInt.6 Int64.7 UInt64.8 Double.9 String.10 Object.11
	for k,v in obj
	{ ; 9 = Int64 for size and Char for type
		(IsObject(k)
			? (NumPut(-11,addr+0,"Char"),NumPut(sz:=RawObjectSize(k,buf),addr+1,"UInt64"),RawObject(k,addr+9),addr+=sz+9)
			: (k+0="") 
				? (NumPut(-10,addr+0,"Char"),NumPut(StrPut(k,addr+9)*(A_IsUnicode?2:1),addr+1,"UInt64"),addr+=NumGet(addr+1,"UInt64")+9) 
				: (NumPut( InStr(k,".")?-9:k>4294967295?-8:k>65535?-6:k>255?-4:k>-1?-2:k>-129?-1:k>-32769?-3:k>-2147483649?-5:-7,addr+0,"Char")
					,NumPut(k,addr+1,InStr(k,".")?"Double":k>4294967295?"UInt64":k>65535?"UInt":k>255?"UShort":k>-1?"UChar":k>-129?"Char":k>-32769?"Short":k>-2147483649?"Int":"Int64")
					,addr+=InStr(k,".")||k>4294967295?9:k>65535?5:k>255?3:k>-129?2:k>-32769?3:k>-2147483649?5:9))

		(IsObject(v) 
			? (NumPut( 11,addr+0,"Char"),NumPut(sz:=RawObjectSize(v,buf),addr+1,"UInt64"),RawObject(v,addr+9),addr+=sz+9)
			: (v+0="") 
				? (NumPut( 10,addr+0,"Char"),NumPut(sz:=buf?obj.GetCapacity(k):StrPut(v)*(A_IsUnicode?2:1),addr+1,"Int64"),DllCall("RtlMoveMemory","PTR",addr+9,"PTR",&v,"PTR",sz),addr+=sz+9) 
				: (NumPut( InStr(v,".")?9:v>4294967295?8:v>65535?6:v>255?4:v>-1?2:v>-129?1:v>-32769?3:v>-2147483649?5:7,addr+0,"Char")
					,NumPut(v,addr+1,InStr(v,".")?"Double":v>4294967295?"UInt64":v>65535?"UInt":v>255?"UShort":v>-1?"UChar":v>-129?"Char":v>-32769?"Short":v>-2147483649?"Int":"Int64")
					,addr+=InStr(v,".")||v>4294967295?9:v>65535?5:v>255?3:v>-129?2:v>-32769?3:v>-2147483649?5:9))	


	}
}

RawObjectSize(obj,buf:=0,sz:=0){
	; 9 = Int64 for size and Char for type
	for k,v in obj
	{ 
		sz +=(IsObject(k)
			? (RawObjectSize(k,buf)+9) : ((k+0="")
				? (StrPut(k)*(A_IsUnicode?2:1)+9) 
				: InStr(k,".")||k>4294967295?9:k>65535?5:k>255?3:k>-129?2:k>-32769?3:k>-2147483649?5:9))
		+(IsObject(v)
			?  RawObjectSize(v,buf)+9 : ((v+0="") 
				? (buf?obj.GetCapacity(k):StrPut(v)*(A_IsUnicode?2:1))+9
				: InStr(v,".")||v>4294967295?9:v>65535?5:v>255?3:v>-129?2:v>-32769?3:v>-2147483649?5:9))
	}
	return sz
}

Code: Select all

ObjLoad(addr,sz:=""){
	; Arrays to retrieve type and size from number type
	static type:=["Char","UChar","Short","UShort","Int","UInt","Int64","UInt64","Double"],num:=[1,1,2,2,4,4,8,8,8]
	end:=addr+sz, obj:=[]
	While % addr<end ; 9 = Int64 for size and Char for type
		k:=(NumGet(addr+0,"Char")=-11 
			? (ObjLoad(addr+9,sz:=NumGet(addr+1,"UInt64")),addr+=sz+9) 
			: NumGet(addr+0,"Char")=-10 
				? (StrGet(addr+9),addr+=NumGet(addr+1,"UInt64")+9) 
				: (NumGet(addr+1,type[sz:=-1*NumGet(addr+0,"Char")]),addr+=num[sz]+1))

		,(NumGet(addr+0,"Char")= 11 
			? (obj[k]:=ObjLoad(addr+9,sz:=NumGet(addr+1,"PTR")),addr+=sz+9)  
			: NumGet(addr+0,"Char")= 10 
				? (obj.SetCapacity(k,sz:=NumGet(addr+1,"UInt64")),DllCall("RtlMoveMemory","PTR",obj.GetAddress(k),"PTR",addr+9,"PTR",sz),addr+=sz+9) 
				: (obj[k]:=NumGet(addr+1,type[sz:=NumGet(addr+0,"Char")]),addr+=num[sz]+1))

	return obj
}

ObjFileLoad(File){
	If !FileExist(File)
		return
	FileGetSize,sz,% File
	FileRead,v,*c %File%
	If ErrorLevel||!sz
		return
	return ObjLoad(&v,sz)
}

In this example, about 500 milliseconds faster.

Code: Select all

; SetBatchLines, -1 ; uncomment when using AHK v1
ObjSize:=1000000
VarToUse:="A_TickCount" ; use A_TickCount, A_ScriptFullPath, A_Now ...
TimesToRun:=1
obj:=[]
Loop % ObjSize
  obj[A_Index]:=%VarToUse%
s:=A_TickCount
Loop % TimesToRun
{
  out:=""
  for k,v in obj
    out.=k "`t" v "`n"
}
forLoop:=A_TickCount-s
s:=A_TickCount
Loop % TimesToRun
{
  out:=""
  ObjDump(obj,var)
}
objDump:=A_TickCount-s
MsgBox % "For loop: " forLoop "`nObjDump: " objDump
ExitApp
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 08:11

arcticir wrote:I try to use an object to pass parameters, but why not correct?
Following DynaRun() versions will automatically convert obj to hex and create global A_Argv in new process containing a copy of the object:
DynaRun_for_AutoHotkey_v1
DynaRun_for_AutoHotkey_v2
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 08:27

HotKeyIt these spoilers make it impossible to read anything.
So I changed them.
Recommends AHK Studio
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 09:23

My fault, thanks nnnik :)

For reference: ObjToHex() and HexToObj().
AutoHotkey_v1
AutoHotkey_v2
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 09:28

Is it necessary?

Code: Select all

Run, % AhkPath pf """ " (IsObject(pr) ? "": pr),,UseErrorLevel HIDE, P

In addition, Is it possible also add this to BinRun, relatively speaking, I am more like BinRun.
Run BinRun Mouse does not appear funnel. ;)
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 12:02

arcticir wrote:Is it necessary?

Code: Select all

Run, % AhkPath pf """ " (IsObject(pr) ? "": pr),,UseErrorLevel HIDE, P
No.
In addition, Is it possible also add this to BinRun, relatively speaking, I am more like BinRun.
Sure :)
BinRun_for_AutoHotkey_v1
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: [Lib] ObjDump / ObjLoad - Object Backup

16 Aug 2014, 12:54

Thanks.

Can I request a more concise BinRun It?
only for AHK_H,I have never used other versions loaded BinRun
only load AHKH. EXE, I only used to load AHKH.EXE, not loaded over other software
and will AHK. EXE stored in the variable, so that the next call.

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: wilkster and 105 guests