Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Run AHK scripts with less (half or even less) memory usage


  • Please log in to reply
26 replies to this topic
heresy
  • Members
  • 291 posts
  • Last active: Sep 26 2008 10:47 PM
  • Joined: 11 Mar 2008
Hello Community.
as I'm kind of memory usage paranoid.
because i'd like to hear that my app is using less memory from users of my scripts (usually i publish my scripts to none-ahk users)
so I was always looking for tricks to reduce the memory usage of ahk scripts that i wrote.
but purposeless minimizing window and restoring was the only solution that i 've found so far.

eventually i've found this article at dotnetSpider while googling.
there were few clues to accomplish my dream. especially SetProcessWorkingSetSize() API
It was not that rough to realize. but the result was much like magic even with less effort.

However someone in IRC devaluated it according to This article.
So i've performed Performance Benchmark hundred times with DerRaphael's support.
Here's the code that i've used to benchmark it. (derRaphael wrote it. Thanks)

SetBatchLines, -1
res := ""
t1 := 0, t2 := 0
_t1 := 0, _t2 := 0
iterations := 25
outerLoops := 20

loop, %outerLoops%
{
	tooltip, currently @ test: %A_index%
	t1 := 0, t2 := 0
	Loop, %iterations%
	{
		VarSetCapacity(Test, 25*1024*1024, 97)

			DllCall("QueryPerformanceFrequency", "Int64 *", Freq)
			DllCall("QueryPerformanceCounter", "Int64 *", Start)
			Loop, 10000
				a := chr(NumGet(Test,A_Index,"UChar"))
			DllCall("QueryPerformanceCounter", "Int64 *", End)
		wo%A_Index% := (End - Start)/Freq
			
			EmptyMem()

			DllCall("QueryPerformanceFrequency", "Int64 *", Freq)
			DllCall("QueryPerformanceCounter", "Int64 *", Start)
			Loop, 10000
				a := chr(NumGet(Test,A_Index,"UChar"))
			DllCall("QueryPerformanceCounter", "Int64 *", End)
		w%A_Index% := (End - Start)/Freq
		
		t1 += wo%A_Index%
		t2 += w%A_Index%
	}
	_t1 += t1/iterations
	_t2 += t2/iterations
	res .= "Run" a_index ":`tnormal - " t1/iterations "sec`temty() - " t2/iterations "sec`n"
	tooltip, currently @ test: A_index
}
	res .= "`nAverage normal " _t1/outerLoops " seconds"
		 . "`nAverage emtpy() " _t2/outerLoops " seconds"
MsgBox,0,Results,%res%
return

EmptyMem(PID="AHK Rocks"){
    pid:=(pid="AHK Rocks") ? DllCall("GetCurrentProcessId") : pid
    h:=DllCall("OpenProcess", "UInt", 0x001F0FFF, "Int", 0, "Int", pid)
    DllCall("SetProcessWorkingSetSize", "UInt", h, "Int", -1, "Int", -1)
    DllCall("CloseHandle", "Int", h)
}

here's the result in short for lazy people
If you don't use SetBatchLine, -1 while calling this function will reduce the performance like only 10ms though i'm still willing to use it.
but if you use SetBatchLine, -1 there were NO PERFORMANCE REDUCE for Script. sometimes it's even slightly faster though i don't know why

eg) Results from the benchmark with SetBatchLine, -1
Normal : Average 0.007232 seconds
With Func : Average 0.007195 seconds

as dotnetSpider says. after calling this function memory usage will get larger gradually as scripts claims more mem.
but this can be easily solved by using SetTimer (call this func periodically)
However i've read some posts that are negative to use This func periodically.
If you care, Use it only once at Script startup

Furthermore, you can even reduce the memory usage of external apps such as 3DsMax, Photoshop, IE, etc (even explore.exe)
you might be interest in writing Memory Optimization app by using this function.
but applying This function to all Processes may potentially degrade the performance of the system. (from MSDN)
and it's not recommendable to use it on Game Processes



however here's the simple but magical function. use at your own risk.
best use of this function is put it at the end of the AutoExecution Area
or call this function depend on event such as guisize:

Download EmptyMem.ahk to /Lib (Required Win32NT)
Omit parameter to reduce scriptself's mem usage else put target's PID



Example
;This example will reduce mem usage from around 5,000k to 500k
Run, taskmgr.exe
MsgBox, Remember current memory usage then compare.
EmptyMem()
Sleep, 5000
Exitapp

EmptyMem(PID="AHK Rocks"){
    pid:=(pid="AHK Rocks") ? DllCall("GetCurrentProcessId") : pid
    h:=DllCall("OpenProcess", "UInt", 0x001F0FFF, "Int", 0, "Int", pid)
    DllCall("SetProcessWorkingSetSize", "UInt", h, "Int", -1, "Int", -1)
    DllCall("CloseHandle", "Int", h)
}

PS. i bet Chris will like it if he's still alive :p
Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007
I know of an app, superb btw, which periodically calls SetProcessWorkingSetSize to reduce its working set. Although I haven't experienced any noticeable performance hit with it, TBH, I don't like the idea of managing the working set directly. And I suppose this API is the artifact of the past, maybe 95/98 era (:i.e. NT4), when memory was precious. Anyway, there may be indirect ways to reduce the working set (:by system).

There exist two interesting functions in my start-up AHK script in this regard. One is increasing constantly the working set, which used to make AHK's mem usage hit over 15MB if let alone. The other is reducing the working set as a side effect, usually reducing the mem usage from 7-8MB to 1-2 MB, even less than 1MB sometimes in my case. As these two functions are frequently used ones, my AHK script's mem usage is usually confined in the range of 1-8MB.

PS. I think I forgot one important thing. Have you checked how much total usage of memory is actually reduced, comparing to the reduction of AHK's working set?

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005
B-e-a utiful.. Many thanks! Your solution solves me a puzzle.

When you load and lock many image resources from a PE, the memory usage shoots up drastically. FreeResource() just decrements a counter and it is Windows OS that decides when to free the unwanted memory usage. To my horror, the memory was freed/reset only when AHK touched the maximum physical memory available.

To experiment the said effect, you may try the code posted:

Creating a JPEG only DLL and retrieving the images posted by SKANWith every change of image, the memory usage grows by 1 MB, even when previously loaded image is loaded again. When EmptyMem() is used after FreeResource(), the usage never exceeds 4MB .. which makes me feel much comfortable.

Thank you Heresy! :)

PS:

Run AHK scripts with less (half or even more) memory usage
I think it should read less
kWo4Lk1.png

heresy
  • Members
  • 291 posts
  • Last active: Sep 26 2008 10:47 PM
  • Joined: 11 Mar 2008
@ Sean
Thanks for your valuable opinion.
as you said i've read few apprehension of using SetProcessWorkingSetSize.
but it was the only solution that i've found so far that reduces mem usage apprantely anyhow.
actually i didn't know deeply about where the reduced mem goes to.
someone said it'll be dumped to VM but as far as i've cheked. it was not. (although MSDN says similar)
i think it's why there's no apparent performance difference.
Maybe it's just AHK. hope Chris will explain for this. (like how ahk interprets the codes)

Have you checked how much total usage of memory is actually reduced, comparing to the reduction of AHK's working set?

Since ahk isn't kind of mammoth app i couldn't get the precise result.
so i've tested with Adobe Photoshop (uses 66MB at startup for me)

;From Process Explorer

;Before
Process        PID   Window Status  Working Set  WS Private  WS Shared  Virtual Size
Photoshop.exe  2252  Running        66,632 K     43,876 K    5,540 K	 207,732 K

;After EmptyMem()
Process        PID   Window Status  Working Set  WS Private  WS Shared  Virtual Size
Photoshop.exe  2252  Running        [color=red]2,236 K      1,492 K     208 K[/color]      207,732 K

Total mem usage doesn't reduced Instantly not like we do Var:=""
but Total mem usage also got reduced gradually. like 500k/sec
I'm not sure if Total mem usage will be reduced precisely same as Process's reduced WS finally.
although i still don't know exactly where the reduced mem gone then back to useable mem.
If i don't get apparent performance reduce, i might use it. cause it's only solution that i've found so far
if you know any other method to get same result please let me know
However, i'll modify 1st post a little to warn people.

@ SKAN
i learnt very much from your tutorials and answers
i'm glad that eventually i could be helpful to you SKAN. :)
and Thanks for correction !
Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007

Since ahk isn't kind of mammoth app i couldn't get the precise result.
so i've tested with Adobe Photoshop (uses 66MB at startup for me)

;From Process Explorer

;Before
Process        PID   Window Status  Working Set  WS Private  WS Shared  Virtual Size
Photoshop.exe  2252  Running        66,632 K     43,876 K    5,540 K	 207,732 K

;After EmptyMem()
Process        PID   Window Status  Working Set  WS Private  WS Shared  Virtual Size
Photoshop.exe  2252  Running        [color=red]2,236 K      1,492 K     208 K[/color]      207,732 K

Total mem usage doesn't reduced Instantly like we do Var:=""
but Total mem usage also got reduced gradually. like 500k/sec
I'm not sure if Total mem usage will be reduced precisely same as Process's reduced WS finally.
although i still don't know exactly where the reduced mem gone then back to useable mem.

They are gone to the usually/loosely called VM, more precisely, the paging file. I can see it certainly reduces the physical memory usage in the case of photoshop. In case of AHK, however, the actual reduction of the physical memory usage depends on the nature of each scripts. For example, as large portion of the working set of many scripts of mine consist of shared memory, the reduction of the working set of the script doesn't lead to the reduction of the (total) physical memory usage.

heresy
  • Members
  • 291 posts
  • Last active: Sep 26 2008 10:47 PM
  • Joined: 11 Mar 2008

the actual reduction of the physical memory usage depends on the nature of each scripts.

Thanks for the clarification.
it's much clear now. so if a script uses lots of dllcall to call WinApi functions. it wouldn't affect magically.
(however it'll still reduce process memory usage apparently though total physical memory usage never been reduced as you said.)
but when script wrtitten of much of native ahk. it would.
whether script uses dllcalls or not, it'll reduce the process memory usage apparently. it's the point.

if anyone experienced any noticeable performance reduction while using this function. report it :)
Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com

  • Guests
  • Last active:
  • Joined: --
hi

if i have taskmanager opened, where exacly can i see the result. Should i look at the memmory use of the process in processes tab or should i see the difference in performance tab in taskmanager.

im talking about windows taskmanager i dont use any 3rd part app and i use this code:


#NoEnv
SendMode Input
SetWorkingDir %A_ScriptDir%

Run, "C:\Program Files\Adobe\Adobe Photoshop CS3\Photoshop.exe"
EmptyMem() 
Sleep, 7000
Exitapp 

EmptyMem(PID="Photoshop"){ 
    pid:=(pid="Photoshop") ? DllCall("GetCurrentProcessId") : pid 
    h:=DllCall("OpenProcess", "UInt", 0x001F0FFF, "Int", 0, "Int", pid) 
    DllCall("SetProcessWorkingSetSize", "UInt", h, "Int", -1, "Int", -1) 
    DllCall("CloseHandle", "Int", h) 
}

I dont know much about this but doesnt applications dump memmory when minimized? If so whats the difference?

Thanks

  • Guests
  • Last active:
  • Joined: --
hmm i didnt put questionmarks after .. but those were questions.

heresy
  • Members
  • 291 posts
  • Last active: Sep 26 2008 10:47 PM
  • Joined: 11 Mar 2008
1. you need to specify target app's pid (process identifier) for function parameter
2. you need to wait for photoshop's startup loading to be completed

below code will work. but use it carefully as Sean warned.

Run, "C:\Program Files\Adobe\Adobe Photoshop CS3\Photoshop.exe",,,PID ;save PID here
WinWait, ahk_class Photoshop ;wait for window existstence
Sleep, 10000 ;photoshop's startup loading must be completed before apply it
EmptyMem(PID) ;put photoshop's PID here as saved in first line
Exitapp

EmptyMem(PID="AHK Rocks"){
    pid:=(pid="AHK Rocks") ? DllCall("GetCurrentProcessId") : pid
    h:=DllCall("OpenProcess", "UInt", 0x001F0FFF, "Int", 0, "Int", pid)
    DllCall("SetProcessWorkingSetSize", "UInt", h, "Int", -1, "Int", -1)
    DllCall("CloseHandle", "Int", h)
}

Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com

  • Guests
  • Last active:
  • Joined: --
thanks ill try taht

but use it carefully as Sean warned.

once, after start or whenever it slows down ... how do you recommend i use it for photoshop?

ABCza
  • Members
  • 132 posts
  • Last active: Jan 04 2015 01:02 AM
  • Joined: 03 Jun 2008
Nice, with an hotkey associated it could be useful to reduce the working set of a particular process. This will help me in freeing memory when the damned outlook fill the memory of my damned work notebook :O
All my scripts/snippets are released under the WTFPL: http://sam.zoy.org/wtfpl/COPYING

heresy
  • Members
  • 291 posts
  • Last active: Sep 26 2008 10:47 PM
  • Joined: 11 Mar 2008

how do you recommend i use it for photoshop?

if you're graphic designer and have excessive use of Photoshop. i'd not recommened to use it. Otherwise. call this function on Photoshop only once.

Nice, with an hotkey associated it could be useful to reduce the working set of a particular process. This will help me in freeing memory when the damned outlook fill the memory of my damned work notebook :O

yeah that's how i'm using it. especially on laptop. these days we have tons of physical memory on PC so this function isn't much worth. but laptop :)
Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com

tank
  • Administrators
  • 4345 posts
  • AutoHotkey Foundation
  • Last active: May 02 2019 09:16 PM
  • Joined: 21 Dec 2007
Dare i to ask about
VirtualFree
; www.msdn.microsoft.com/en-us/library/aa366892(VS.85).aspx

I have an application that i automate the paging file for this app grows indefinately untill it explodes :?
I beleive i could use VirtualFree to empty this on occasion i will be clear the stored data serves no purpose and it is a fault of that applications design
I however cannot for the life of me figure out how to get lpAddress of a running process

Ideas???
Never lose.
WIN or LEARN.

tain
  • Members
  • 62 posts
  • Last active: May 21 2014 10:02 PM
  • Joined: 16 Feb 2010
I find EmptyMem() to be very effective. I use it like this for a few processes that eat up resources:

Process, wait, ProcessThatEatsLotsOfMem.exe ;find the process
NewPID = %ErrorLevel%  ; Save the PID value immediately
EmptyMem(NewPID)

EmptyMem(PIDtoEmpty){   ;http://www.autohotkey.com/forum/topic32876.html (first line removed)
    h:=DllCall("OpenProcess", "UInt", 0x001F0FFF, "Int", 0, "Int", PIDtoEmpty)
    DllCall("SetProcessWorkingSetSize", "UInt", h, "Int", -1, "Int", -1)
    DllCall("CloseHandle", "Int", h)
}

And there is a similar project over here now:
<!-- m -->http://www.autohotke...topic53543.html<!-- m -->


Thanks for the code!

Raule
  • Guests
  • Last active:
  • Joined: --
thank you for posting this.
This has reduce my memory use from 5,988K to 1,976K

can we call this function many times ?
or only one time

after running for a period memory usage increase, calling after each hotkey reduce memory, but cause any issue ?