23 replies to this topic

[edwin]

Hi Guys,

am trying to automate an application called peregrine client. But the spy doesnt give info reg. the class or instance or visible text or anything to make use of. It just gives the window title and mouse coor. Pl help me guys..!! Wat I need to do is.. there is a button there which indicates the arrival of new tickets. I want to check for changes in the number in that button by refreshing and if there is a change it should double click the last arrived ticket.. since spy is not giving any instance or class reg. control.. I cannot proceed further.. Guys please help me out..!! Thanks in advance..!!

[Title edited. Please write descriptive titles for your topics. ~jaco0646]

[edwin]

I tried getting the pixel color.. but since the place am trying contains numbers, the pixel color doesnt differ.. also it is is difficult to take screenshots and compare everytime(cos it has to be refreshed infact for every few seconds).. So guys tell me some other way.. I believe there are lot of experts in scripting here and guys pl help me out with your ideas..!!

[TLM]

Have you tried a UID spy? Try this and mouse over window. No guarantees it'l work but its worth a try..

#SingleInstance ignore

; Simple Window Spy

DetectHiddenWindows, On

Loop
    {
      CoordMode, ToolTip,
	MouseGetPos, X, Y, MWINB, MCTRL, 2
	MouseGetPos, X, Y, MWINA, MCTRLA, 1
	ToolTip, ~~~~~~~~~~~~~~~~`nSimple Window Spy`n~~~~~~~~~~~~~~~~`n`nThe HWND is: %MCTRL%`nWindow ClassNN is: %MCTRLA%`nUnique ID # is: %MWINA%`n`n~~~~~~~~~~~~~~~~`n`nPress F1 to copy info`nPress ESC to exit`n~~~~~~~~~~~~~~~~, 10, 10
  }

Return

f1::

FileDelete, %A_Desktop%\WindowInformation.txt

  infoCopy = The HWND is: %MCTRL%`n`nWindow ClassNN is: %MCTRLA%`n`nUnique ID # is: %MWINA%
  FileAppend, %infoCopy%, %A_Desktop%\WindowInformation.txt
  Run, %A_Desktop%\WindowInformation.txt

Return

ESC::Exitapp

You can also use process or WinGet to get a PID.

..there is a button there which indicates the arrival of new tickets. I want to check for changes in the number in that button by refreshing ..

PLEASE POST A SCREEN SHOT OF THE TARGET WINDOW + BUTTON!

If you can get the coordinates of upper left hand corner you should be able to figure out how far the button is from there. Once you get that, move the coordinates into the button where the text (or number) comes up. Goto Start - Run and type Magnify.exe. Use it to get as close as possible to the edge of a character. Get the background color of the button without characters and then look for changes..



So in this case you'd be looking for either D3D3CB and 030303 or if not D3D3CB (the background).

[sinkfaze]

What type of program is this? Is it a terminal services/remote client application? Is it a web-based application?

[edwin]

Sorry guys for late reply. First thanks for all your responses. Hey TLM, I tried using tat UID spy.. it doesnt work. But Thank you for responding. I need more help from you guys.
And reg. the screen shot, since it is highly confidential, am scared of posting tat. Also, I took the screen shot of tat button alone and tried to post here. but its asking for a link to tat image. I don know were to upload tat and give a link. Becos most of the sites are restricted. So guys tel me a way to do tat too.

Hey sinkfaze, I think it is web based application. Because we access the application thru net. If am wrong correct me.

And I spent four days by just reading the whole help file, but it didnt help me. Guys I doubt whether this can be automated. Pls help me out to automate this, cos this job occupies most of my time..

Thanks in advance.. !!

[HotKeyIt]

I have never seen an app without ahk_class???

I think the only way is to capture image and compare it.
Try this:
:!: you will find Hash here Hash() see page 7 posted by Laszlo
:!: you will find CaptureScreen here ScreenCapture

Loop
{
	CaptureScreen(1,0,0)
	pic1:=ClipboardAll
	CaptureScreen(1,0,0)
	pic2:=ClipboardAll
	If (changed:=(Hash(pic1)!=Hash(pic2)))
		ToolTip % "Last time window changed at " . lastchange:=A_TickCount
	else
		ToolTip % "Last time window changed at " . lastchange
}
ExitApp
Hash( Str, nSz=0 ) { 
  Static HashData
  If ( HashData="" ) {
       HashData := DllCall( "LoadLibrary", Str,"shlwapi.dll")
       HashData := DllCall( "GetProcAddress", UInt,HashData, Str, "HashData" )      
	} IfLess,nSz,1, SetEnv,nSz, % StrLen(Str) 
  DllCall( HashData, UInt,Ptr, UInt,Sz, UIntP,Hash, UInt,4 )
  IfEqual,A_FormatInteger,D, SetFormat,Integer,% "H" ((Flag:="D")+WM_NULL)
  DllCall( "shlwapi.dll\HashData", UInt,&Str, UInt,StrLen(Str), UIntP,Hash, UInt,4 )
  Hash := SubStr(Hash+0x1000000000,-7)
  StringUpper, Hash, Hash
  IfEqual,Flag,D, SetFormat,Integer,D
  Return Hash
}
/* CaptureScreen(aRect, bCursor, sFileTo, nQuality)
1) If the optional parameter bCursor is True, captures the cursor too.
2) If the optional parameter sFileTo is 0, set the image to Clipboard.
   If it is omitted or "", saves to screen.bmp in the script folder,
   otherwise to sFileTo which can be BMP/JPG/PNG/GIF/TIF.
3) The optional parameter nQuality is applicable only when sFileTo is JPG. Set it to the desired quality level of the resulting JPG, an integer between 0 - 100.
4) If aRect is 0/1/2/3, captures the entire desktop/active window/active client area/active monitor.
5) aRect can be comma delimited sequence of coordinates, e.g., "Left, Top, Right, Bottom" or "Left, Top, Right, Bottom, Width_Zoomed, Height_Zoomed".
   In this case, only that portion of the rectangle will be captured. Additionally, in the latter case, zoomed to the new width/height, Width_Zoomed/Height_Zoomed.

Example:
CaptureScreen(0)
CaptureScreen(1)
CaptureScreen(2)
CaptureScreen(3)
CaptureScreen("100, 100, 200, 200")
CaptureScreen("100, 100, 200, 200, 400, 400")   ; Zoomed
*/

/* Convert(sFileFr, sFileTo, nQuality)
Convert("C:\image.bmp", "C:\image.jpg")
Convert("C:\image.bmp", "C:\image.jpg", 95)
Convert(0, "C:\clip.png")   ; Save the bitmap in the clipboard to sFileTo if sFileFr is "" or 0.
*/

;CaptureScreen()
;Return

CaptureScreen(aRect = 0, bCursor = False, sFile = "", nQuality = "")
{
	If	!aRect
	{
		SysGet, nL, 76
		SysGet, nT, 77
		SysGet, nW, 78
		SysGet, nH, 79
	}
	Else If	aRect = 1
		WinGetPos, nL, nT, nW, nH, A
	Else If	aRect = 2
	{
		WinGet, hWnd, ID, A
		VarSetCapacity(rt, 16, 0)
		DllCall("GetClientRect" , "Uint", hWnd, "Uint", &rt)
		DllCall("ClientToScreen", "Uint", hWnd, "Uint", &rt)
		nL := NumGet(rt, 0, "int")
		nT := NumGet(rt, 4, "int")
		nW := NumGet(rt, 8)
		nH := NumGet(rt,12)
	}
	Else If	aRect = 3
	{
		VarSetCapacity(mi, 40, 0)
		DllCall("GetCursorPos", "int64P", pt)
		DllCall("GetMonitorInfo", "Uint", DllCall("MonitorFromPoint", "int64", pt, "Uint", 2), "Uint", NumPut(40,mi)-4)
		nL := NumGet(mi, 4, "int")
		nT := NumGet(mi, 8, "int")
		nW := NumGet(mi,12, "int") - nL
		nH := NumGet(mi,16, "int") - nT
	}
	Else
	{
		StringSplit, rt, aRect, `,, %A_Space%%A_Tab%
		nL := rt1
		nT := rt2
		nW := rt3 - rt1
		nH := rt4 - rt2
		znW := rt5
		znH := rt6
	}

	mDC := DllCall("CreateCompatibleDC", "Uint", 0)
	hBM := CreateDIBSection(mDC, nW, nH)
	oBM := DllCall("SelectObject", "Uint", mDC, "Uint", hBM)
	hDC := DllCall("GetDC", "Uint", 0)
	DllCall("BitBlt", "Uint", mDC, "int", 0, "int", 0, "int", nW, "int", nH, "Uint", hDC, "int", nL, "int", nT, "Uint", 0x40000000 | 0x00CC0020)
	DllCall("ReleaseDC", "Uint", 0, "Uint", hDC)
	If	bCursor
		CaptureCursor(mDC, nL, nT)
	DllCall("SelectObject", "Uint", mDC, "Uint", oBM)
	DllCall("DeleteDC", "Uint", mDC)
	If	znW && znH
		hBM := Zoomer(hBM, nW, nH, znW, znH)
	If	sFile = 0
		SetClipboardData(hBM)
	Else	Convert(hBM, sFile, nQuality), DllCall("DeleteObject", "Uint", hBM)
}

CaptureCursor(hDC, nL, nT)
{
	VarSetCapacity(mi, 20, 0)
	mi := Chr(20)
	DllCall("GetCursorInfo", "Uint", &mi)
	bShow   := NumGet(mi, 4)
	hCursor := NumGet(mi, 8)
	xCursor := NumGet(mi,12)
	yCursor := NumGet(mi,16)

	VarSetCapacity(ni, 20, 0)
	DllCall("GetIconInfo", "Uint", hCursor, "Uint", &ni)
	xHotspot := NumGet(ni, 4)
	yHotspot := NumGet(ni, 8)
	hBMMask  := NumGet(ni,12)
	hBMColor := NumGet(ni,16)

	If	bShow
		DllCall("DrawIcon", "Uint", hDC, "int", xCursor - xHotspot - nL, "int", yCursor - yHotspot - nT, "Uint", hCursor)
	If	hBMMask
		DllCall("DeleteObject", "Uint", hBMMask)
	If	hBMColor
		DllCall("DeleteObject", "Uint", hBMColor)
}

Zoomer(hBM, nW, nH, znW, znH)
{
	mDC1 := DllCall("CreateCompatibleDC", "Uint", 0)
	mDC2 := DllCall("CreateCompatibleDC", "Uint", 0)
	zhBM := CreateDIBSection(mDC2, znW, znH)
	oBM1 := DllCall("SelectObject", "Uint", mDC1, "Uint",  hBM)
	oBM2 := DllCall("SelectObject", "Uint", mDC2, "Uint", zhBM)
	DllCall("SetStretchBltMode", "Uint", mDC2, "int", 4)
	DllCall("StretchBlt", "Uint", mDC2, "int", 0, "int", 0, "int", znW, "int", znH, "Uint", mDC1, "int", 0, "int", 0, "int", nW, "int", nH, "Uint", 0x00CC0020)
	DllCall("SelectObject", "Uint", mDC1, "Uint", oBM1)
	DllCall("SelectObject", "Uint", mDC2, "Uint", oBM2)
	DllCall("DeleteDC", "Uint", mDC1)
	DllCall("DeleteDC", "Uint", mDC2)
	DllCall("DeleteObject", "Uint", hBM)
	Return	zhBM
}

Convert(sFileFr = "", sFileTo = "", nQuality = "")
{
	If	sFileTo  =
		sFileTo := A_ScriptDir . "\screen.bmp"
	SplitPath, sFileTo, , sDirTo, sExtTo, sNameTo

	If Not	hGdiPlus := DllCall("LoadLibrary", "str", "gdiplus.dll")
		Return	sFileFr+0 ? SaveHBITMAPToFile(sFileFr, sDirTo . "\" . sNameTo . ".bmp") : ""
	VarSetCapacity(si, 16, 0), si := Chr(1)
	DllCall("gdiplus\GdiplusStartup", "UintP", pToken, "Uint", &si, "Uint", 0)

	If	!sFileFr
	{
		DllCall("OpenClipboard", "Uint", 0)
		If	 DllCall("IsClipboardFormatAvailable", "Uint", 2) && (hBM:=DllCall("GetClipboardData", "Uint", 2))
		DllCall("gdiplus\GdipCreateBitmapFromHBITMAP", "Uint", hBM, "Uint", 0, "UintP", pImage)
		DllCall("CloseClipboard")
	}
	Else If	sFileFr Is Integer
		DllCall("gdiplus\GdipCreateBitmapFromHBITMAP", "Uint", sFileFr, "Uint", 0, "UintP", pImage)
	Else	DllCall("gdiplus\GdipLoadImageFromFile", "Uint", Unicode4Ansi(wFileFr,sFileFr), "UintP", pImage)

	DllCall("gdiplus\GdipGetImageEncodersSize", "UintP", nCount, "UintP", nSize)
	VarSetCapacity(ci,nSize,0)
	DllCall("gdiplus\GdipGetImageEncoders", "Uint", nCount, "Uint", nSize, "Uint", &ci)
	Loop, %	nCount
		If	InStr(Ansi4Unicode(NumGet(ci,76*(A_Index-1)+44)), "." . sExtTo)
		{
			pCodec := &ci+76*(A_Index-1)
			Break
		}
	If	InStr(".JPG.JPEG.JPE.JFIF", "." . sExtTo) && nQuality<>"" && pImage && pCodec
	{
	DllCall("gdiplus\GdipGetEncoderParameterListSize", "Uint", pImage, "Uint", pCodec, "UintP", nSize)
	VarSetCapacity(pi,nSize,0)
	DllCall("gdiplus\GdipGetEncoderParameterList", "Uint", pImage, "Uint", pCodec, "Uint", nSize, "Uint", &pi)
	Loop, %	NumGet(pi)
		If	NumGet(pi,28*(A_Index-1)+20)=1 && NumGet(pi,28*(A_Index-1)+24)=6
		{
			pParam := &pi+28*(A_Index-1)
			NumPut(nQuality,NumGet(NumPut(4,NumPut(1,pParam+0)+20)))
			Break
		}
	}

	If	pImage
		pCodec	? DllCall("gdiplus\GdipSaveImageToFile", "Uint", pImage, "Uint", Unicode4Ansi(wFileTo,sFileTo), "Uint", pCodec, "Uint", pParam) : DllCall("gdiplus\GdipCreateHBITMAPFromBitmap", "Uint", pImage, "UintP", hBitmap, "Uint", 0) . SetClipboardData(hBitmap), DllCall("gdiplus\GdipDisposeImage", "Uint", pImage)

	DllCall("gdiplus\GdiplusShutdown" , "Uint", pToken)
	DllCall("FreeLibrary", "Uint", hGdiPlus)
}

CreateDIBSection(hDC, nW, nH, bpp = 32, ByRef pBits = "")
{
	NumPut(VarSetCapacity(bi, 40, 0), bi)
	NumPut(nW, bi, 4)
	NumPut(nH, bi, 8)
	NumPut(bpp, NumPut(1, bi, 12, "UShort"), 0, "Ushort")
	NumPut(0,  bi,16)
	Return	DllCall("gdi32\CreateDIBSection", "Uint", hDC, "Uint", &bi, "Uint", 0, "UintP", pBits, "Uint", 0, "Uint", 0)
}

SaveHBITMAPToFile(hBitmap, sFile)
{
	DllCall("GetObject", "Uint", hBitmap, "int", VarSetCapacity(oi,84,0), "Uint", &oi)
	hFile:=	DllCall("CreateFile", "Uint", &sFile, "Uint", 0x40000000, "Uint", 0, "Uint", 0, "Uint", 2, "Uint", 0, "Uint", 0)
	DllCall("WriteFile", "Uint", hFile, "int64P", 0x4D42|14+40+NumGet(oi,44)<<16, "Uint", 6, "UintP", 0, "Uint", 0)
	DllCall("WriteFile", "Uint", hFile, "int64P", 54<<32, "Uint", 8, "UintP", 0, "Uint", 0)
	DllCall("WriteFile", "Uint", hFile, "Uint", &oi+24, "Uint", 40, "UintP", 0, "Uint", 0)
	DllCall("WriteFile", "Uint", hFile, "Uint", NumGet(oi,20), "Uint", NumGet(oi,44), "UintP", 0, "Uint", 0)
	DllCall("CloseHandle", "Uint", hFile)
}

SetClipboardData(hBitmap)
{
	DllCall("GetObject", "Uint", hBitmap, "int", VarSetCapacity(oi,84,0), "Uint", &oi)
	hDIB :=	DllCall("GlobalAlloc", "Uint", 2, "Uint", 40+NumGet(oi,44))
	pDIB :=	DllCall("GlobalLock", "Uint", hDIB)
	DllCall("RtlMoveMemory", "Uint", pDIB, "Uint", &oi+24, "Uint", 40)
	DllCall("RtlMoveMemory", "Uint", pDIB+40, "Uint", NumGet(oi,20), "Uint", NumGet(oi,44))
	DllCall("GlobalUnlock", "Uint", hDIB)
	DllCall("DeleteObject", "Uint", hBitmap)
	DllCall("OpenClipboard", "Uint", 0)
	DllCall("EmptyClipboard")
	DllCall("SetClipboardData", "Uint", 8, "Uint", hDIB)
	DllCall("CloseClipboard")
}

Unicode4Ansi(ByRef wString, sString)
{
	nSize := DllCall("MultiByteToWideChar", "Uint", 0, "Uint", 0, "Uint", &sString, "int", -1, "Uint", 0, "int", 0)
	VarSetCapacity(wString, nSize * 2)
	DllCall("MultiByteToWideChar", "Uint", 0, "Uint", 0, "Uint", &sString, "int", -1, "Uint", &wString, "int", nSize)
	Return	&wString
}

Ansi4Unicode(pString)
{
	nSize := DllCall("WideCharToMultiByte", "Uint", 0, "Uint", 0, "Uint", pString, "int", -1, "Uint", 0, "int",  0, "Uint", 0, "Uint", 0)
	VarSetCapacity(sString, nSize)
	DllCall("WideCharToMultiByte", "Uint", 0, "Uint", 0, "Uint", pString, "int", -1, "str", sString, "int", nSize, "Uint", 0, "Uint", 0)
	Return	sString
}

[edwin]

Hey HotKeyIt.. Thank you so much for your reply. This forum is really helpful. Spy is showing the ahk_class as ahk_class Transparent Windows Client. But what i referred to is it is not showing up the class NN parameter. To be clear, let me put the information which spy shows up.

>>>>>>>>>>( Window Title & Class )<<<<<<<<<<<
ServiceCenter - Update Incident Number 600-02-7667249 - Peregr - \\Remote
ahk_class Transparent Windows Client

>>>>>>>>>>>>( Mouse Position )<<<<<<<<<<<<<
On Screen: 721, 599 (less often used)
In Active Window: 725, 603

>>>>>>>>>( Now Under Mouse Cursor )<<<<<<<<

Color: 0xD8E9EC (Blue=D8 Green=E9 Red=EC)

>>>>>>>>>>( Active Window Position )<<<<<<<<<<
left: -4 top: -4 width: 1288 height: 972

>>>>>>>>>>>( Status Bar Text )<<<<<<<<<<

>>>>>>>>>>>( Visible Window Text )<<<<<<<<<<<

>>>>>>>>>>>( Hidden Window Text )<<<<<<<<<<<

>>>>( TitleMatchMode=slow Visible Text )<<<<

>>>>( TitleMatchMode=slow Hidden Text )<<<<


Am totally new here. And sorry if I have asked any simple question. Kindly help me to automate this. Becos I doubt whether this could be automated.

Thanks

[HotKeyIt]

Have you tried my code? Does it register changes?
I think this is the only way to go.

I want to check for changes in the number in that button by refreshing and if there is a change it should double click the last arrived ticket

How can you get that ticket recognized? Is it a number? If yes, does that window have other numbers in it? If yes, do they have same color?

Can you select and copy any text from that window?

It will be definitely not easy, but I am sure there is a way to automate it

[edwin]

Hey man, Thanks for your reply. I think it's really a challenge to automate this. First let me answer all your questions.

It is a tool which opens upon clicking a link through Citrix platform. * Nothing can be selected and copied in that window. If you right click it has their own options of that tool. It is a tool. I think it makes sense.
*The records are lined up with various fields. (Just like excel)
* for refresh the shortcut is F2, But thr is no shortcut for clicking that button. But thr is a Hyperlink which can be clicked to count the records. It shows a window with title 'Information', under which it will say like " There are 20 records in the list". Again, this window also not recognised by the spy tool.
*The ticket can be opened by double-clicking on it. The tickets can be arranged using various tools. So it can be made like whenever a new ticket has arrived it comes first.

I would love to paste a screenshot over here. But since it is confidential it is difficult to be pasted here.

Once again thanks for responding to me. This forum rocks to the core. I am getting help instantly. Great!!

[HotKeyIt]

So you need to read a number that sit somewhere in that window?
Is the number always same format?
Could you take a screenshot and cut off that numbers to do an ImageSearch for those?

Can you post an screenshot?
You can erase the confidential fields if necessary!
You can sign up on <!-- m -->https://ahknet.autohotkey.com/#file<!-- m -->
Upload your image there.
Link here to the image.

[edwin]

Kindly help me out. Though it seems difficult, I think we can make it.

And I tried that Screencapture and comparing(the idea you told). To my surprise it worked. But its popping up for unnecessary things. But now its not working. I don know y. I used the same script again. My system showed an error becos of dump memory full. Can you also tell me how to store the screenshots in any of the physical drives. Or it would be great if the other screenshots are deleted except the previously taken one and the current one(Jus for comparing purpose alone).???

Thank you once again!!

[HotKeyIt]

Just look closer into ScreenCapture, you can capture the screen/window into file very easy CaptureScreen(1,"","c:\Temp\file.jpg").

Can you break down steps you need to do in that window

[TLM]

edwin just a side question. Are you sure the app does not offload the updates to a log somewhere? You could use FileMon.exe to see if this is so.