IE HTML Element Spy

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

It retrieves the info of a DOM element under the cursor when the cursor is hovering over Internet Explorer_Server.
The hotkey is currently set to ^E.

REQUIRE COM Standard Library.

^E::MsgBox, % IE_HtmlElement()

IE_HtmlElement()
{
	CoordMode, Mouse
	MouseGetPos, xpos, ypos,, hWnd, 3
	Loop
	{
		If Not	hWnd
		Exit
		WinGetClass, sClass, ahk_id %hWnd%
		If Not	sClass == "Internet Explorer_Server"
			hWnd :=	DllCall("GetAncestor", "Uint", hWnd, "Uint", 1)
		Else	Break
	}
	If	pwin := IE_GetWindow(hWnd)
	{
	If	pelt := COM_Invoke(pwin, "document.elementFromPoint", xpos-xorg:=COM_Invoke(pwin,"screenLeft"), ypos-yorg:=COM_Invoke(pwin,"screenTop"))
	{
		While	(type:=COM_Invoke(pelt,"tagName"))="IFRAME" || type="FRAME"
		{
		selt .= "[" type "]" (A_Index==1 ? "":"." A_Index-1) "`n" COM_Invoke(pelt,"src") "`n", COM_Release(pwin)
		pwin := COM_QueryService(pbrt:=COM_Invoke(pelt,"contentWindow"), "{332C4427-26CB-11D0-B483-00C04FD90119}"), COM_Release(pbrt)
		pbrt := COM_Invoke(pwin, "document.elementFromPoint", xpos-xorg+=COM_Invoke(pelt,"getBoundingClientRect.left"), ypos-yorg+=COM_Invoke(pelt,"getBoundingClientRect.top")), COM_Release(pelt), pelt:=pbrt
		}
		pbrt :=	COM_Invoke(pelt, "getBoundingClientRect")
		l  :=	COM_Invoke(pbrt, "left")
		t  :=	COM_Invoke(pbrt, "top")
		r  :=	COM_Invoke(pbrt, "right")
		b  :=	COM_Invoke(pbrt, "bottom")
		COM_Release(pbrt)
		selt .=	"[HTML]`n" COM_Invoke(pelt, "outerHTML") "`n[TEXT]`n" COM_Invoke(pelt, "outerText") "`n[RECTANGLE]`n(" l+xorg "," t+yorg "," r+xorg "," b+yorg ")"
		COM_Release(pelt)
	}
	COM_Release(pwin)
	Return	selt
	}
}

IE_GetWindow(hWnd)
{
	Static	h:=0
	If	h =0
		h:=COM_AccInit()
	If	pacc:=COM_AccessibleObjectFromWindow(hWnd)
	Return	COM_QueryService(pacc, "{332C4427-26CB-11D0-B483-00C04FD90119}") + COM_Release(pacc)*0
}

IE_GetDocument(hWnd)
{
	Static	h:=0,msg,iid
	If	h =0
		h:=COM_AccInit(), msg:=DllCall("user32\RegisterWindowMessage", "str", "WM_HTML_GETOBJECT"), COM_GUID4String(iid, "{332C4425-26CB-11D0-B483-00C04FD90119}")
	If	DllCall("user32\SendMessageTimeout", "Uint", hWnd, "Uint", msg, "Uint", 0, "Uint", 0, "Uint", 2, "Uint", 1000, "UintP", lr) && DllCall("oleacc\ObjectFromLresult", "Uint", lr, "Uint", &iid, "Uint", 0, "UintP", pdoc)=0
	Return	pdoc
}

- Posted 05 September 2009 - 07:41 AM

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

Thanks for this for this totally different approach it will probably be easier for most users to digest than any element parsers i have posted

I tried this out just to see if it resolved an issue i have yet to solve well
delving into elements within frames
example

I even tried something similar to this once based on some functionality Lexikos demonstrated in scrollmomentum
GetDeepestScrollElement which worked exceptionally well but i couldnt seem to have any luck mapping a path out so users could easily get the path i.e. return something like

document.frames[0].contentwindow.document.all.item[blah].innerhtml

in particular the frame index(when no name or id is avail) was the point of dificulty

since your on the IE subject i am curious if you have any thoughts on how to return a frame index

- Posted 07 September 2009 - 05:28 AM

Never lose.
WIN or LEARN.

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

I tried this out just to see if it resolved an issue i have yet to solve well delving into elements within frames

Good question. As a matter of fact, I was a little hesitated about whether should try to recurse into frames or not, however, decided not, to keep it simple.

I believe the key function with (i)frames is contentWindow (or contentDocument if supported) which represents the Window (or Document) object of (i)frames. So, can apply the same method to it as done in the function posted.

- Posted 07 September 2009 - 11:37 AM

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

OK, I updated the function to recurse into (i)frames.

- Posted 07 September 2009 - 09:55 PM

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

Nice try :idea:
Perhaps i explained the issue incorrectly

first i start by what i think i see in your function
[frame] seems to mean the first frame in the recursion

.....
<body> <!-- [frame] -->
<iframe src="blah.html"></iframe>
 .....

blah.html

.....
<body> <!-- [frame].1 -->
<iframe src="blah2.html"></iframe>

where i was asking for a way to get the actual frame index

.....
<body> <!-- [frame] -->
<iframe src="blah.html"></iframe>
<!-- [frame].1 -->
<iframe src="bleh.html"></iframe> .....

blah.html

.....
<body> <!-- [frame] -->
<iframe src="blah2.html"></iframe>
<!-- [frame].1 -->
<iframe src="bleh2.html"></iframe> .....

thus giving the user the information necesary to easily write the following code to get content from bleh2.html

;first frame of the browsers top window
;second frame of the child frames window
COM_Invoke(pwb,document.frames.item[0].contentwindow.frames.item[1].contentwindow.document.body.innerhtml")

I want to be clear the only tactic i have been able to come up with is taking the bodies of each of the frames and comparing with the returned frame body to determin which frame in the collection of frames i have

this is of course heavy on overhead and perhaps (i hope) not the best way

- Posted 07 September 2009 - 11:50 PM

Never lose.
WIN or LEARN.

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

Why don't you just use sourceIndex?

idx0 := COM_Invoke(frame0, "sourceIndex")
idx1 := COM_Invoke(frame1, "sourceIndex")
...
COM_Invoke(pwb,"document.all.item[" idx0 "].contentwindow.document.all.item[" idx1 "].contentwindow.document.body.innerhtml")

- Posted 08 September 2009 - 12:31 AM

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

Why don't you just use sourceIndex?

because some layouts are dynamic and would shift source index often the frame index is more reliable as a point of reference :wink:

either way thanks for this function.

I suggest if there is not a way to reference the frame index, then the delving into frames might be represented as you have suggested instead of the way it is now, but thats just my simple short sited opinion :wink:

- Posted 08 September 2009 - 01:12 AM

Never lose.
WIN or LEARN.

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

I suggest if there is not a way to reference the frame index, then the delving into frames might be represented as you have suggested instead of the way it is now, but thats just my simple short sited opinion :wink:

Yes, I'm not aware of a way to return the frame index. From the standpoint of AHK, firefox may be easier to use when dealing with frames as a frame creates/uses a separate (its own) window in FF, which is not the case with IE. So, I may add also to display the screen coordinates of the bounding rectangle of a frame at next update.

- Posted 08 September 2009 - 02:37 AM

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

Thanks

- Posted 08 September 2009 - 02:54 AM

Never lose.
WIN or LEARN.

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

Since Sean has seen fit to entertain my notions (thanks

)

I feel inclined to share my wrapping of in a gui and some alterations
as usual this link is 90% Sean's genious and 50% my frakking it up but.... hopefully it helps :lol: My screw up

ctrl / to pause or resume

Posted Image

nuthing at all fancy

I hope you dont mind me posting my adaption in your thread but since its based on your above posted code i thought it might be appropriate if not let me know and I'll get it moved off your thread

#10

- Posted 08 September 2009 - 03:23 PM

Never lose.
WIN or LEARN.

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

Of course I don't mind. BTW, I was more interested in providing GUI to highlight the bounding rectangle of the element, however, I haven't decided which one to use yet.

#11

- Posted 08 September 2009 - 11:13 PM

DeWild1

Members
369 posts

Last active: Feb 28 2014 08:15 PM
Joined: 30 Apr 2006

Tank + Sean work of fricken ART

ctrl / to pause or resume

This is very pretty, functional beyond my understanding but my head is full of rocks.... :roll:

Could you make a small example of how to put the different parts into a script.
Most may see google log in as a useful example or craigslist or for fun, make an example script with the differnet things you could automate on like Mafia Wars on myspace.com. 8)

#12

- Posted 01 October 2009 - 03:19 AM

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

Could you make a small example of how to put the different parts into a script.

I'm not sure of what you try to achieve. You can retrieve the whole document too, instead of sneaking into each element via this script.

#13

- Posted 01 October 2009 - 11:01 PM

tank

Administrators
4345 posts

AutoHotkey Foundation

Last active: May 02 2019 09:16 PM
Joined: 21 Dec 2007

I'll explain as this is something i have discussed before. an html equiv of the script recorder records which elements clicked and automatically generates a rudimentary script.
I have neither the time nor interest to take it to this level any time soon

#14

- Posted 02 October 2009 - 12:34 AM

Never lose.
WIN or LEARN.

Sean

Members
2462 posts

Last active: Feb 07 2012 04:00 AM
Joined: 12 Feb 2007

I'll explain as this is something i have discussed before. an html equiv of the script recorder records which elements clicked and automatically generates a rudimentary script.

Oh, I see. The basic idea of the solution is simple: set-up events via COM_ConnectObject. But, the details will be tedious, so, I'll also leave the details to the users.

#15

- Posted 02 October 2009 - 01:17 AM

Sign In