Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

Post your working scripts, libraries and tools
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

21 Mar 2017, 02:58

I created a script to get the element under the cursor in Internet Explorer (regardless of the zoom percentage). It could potentially be incorporated into an update of iWB2 Learner.

The script puts temporary borders around, and gets text from, the element under the cursor.

It seems to always work correctly, however, if anyone notices an error or omission, or script issue, please comment below, because I would want something potentially widely used like this to be perfect. If anyone has written something similar, also, please inform me.

It uses 4 lots of SplashImage to show the borders around the element.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

Last edited by jeeswg on 10 Nov 2017, 16:43, edited 3 times in total.
rommmcek
Posts: 370
Joined: 15 Aug 2014, 15:18

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

23 Mar 2017, 12:56

Very nice! I saw this on StackOverflow, but I tried

Code: [Select all] [Download] GeSHi © Codebox Plus

window.devicePixelRatio
without success!
P.s.: Works a bit slow, I think and it seems to be a problem if multiple IE windows are opened, not sure, made just a quick test, anyway thanks!
Edit: I couldn't confirm reported problems, since relevant IE window was corrupt. I realized that after iWB2_Learner and ACC_Viewer stopped working too. Strangely the window itself still works!
User avatar
tank
Posts: 2326
Joined: 28 Sep 2013, 22:15
Facebook: charlie.simmons.7334
Google: ttnnkkrr
GitHub: ttnnkkrr
Location: Louisville KY
Contact:

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

18 Aug 2017, 09:43

https://autohotkey.com/boards/viewtopic ... 263#p27263 I dunno bout getting the zoom level tho. see if this simple code helps you get started
except for the zoom level what your trying to do has been done win the iwebbrowser learner
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
https://www.facebook.com/ahkscript.org
If you have forum suggestions please submit a pull request
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

18 Aug 2017, 12:05

@tank: Yes, iWB2 Learner has the limitation that it can only work properly if you set the zoom in Internet Explorer to 100%, so the aim of this script was to find some code that could be used to improve iWB2 Learner.

It has not been easy to improve iWB2 Learner, I plan to try and rewrite it from scratch, and at that point, understanding the script better, either create a rewritten version (more likely), or amend the existing version (possible).

A script to get the zoom percentage from Internet Explorer:
Internet Explorer get/set zoom/text size - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=26359&p=165674#p165674
Helgef
Posts: 3147
Joined: 17 Jul 2016, 01:02
Contact:

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

18 Aug 2017, 12:18

jeeswg wrote: I plan to try and rewrite it from scratch

If you do it for v2, it needn't be improved to be valuable. :mrgreen:

Cheers.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

18 Aug 2017, 12:27

I tried to amend the script to work with an old version of AHK v2 that supported the Gui command, but the problems relating to understanding how the script works, and new AHK v2 errors have made that very difficult. Also I'll have to learn the new GUI functions/objects.

I'll be back though, with improved AHK v1/v2 versions of iWB2 Learner and AccViewer at some point:
Acc / AccViewer / iWB2 Learner / Anchor conversion attempts - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=37&t=30606

Well, I'll start off with rewrites (no functionality added), and then improved versions. So 2 original scripts * 2 AHK versions * 2 script versions = 8 scripts in total.
Cheers.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

25 Aug 2017, 22:58

Get the screen coordinates for a specific web element in Internet Explorer. See above for the JEE_Borders function, which puts a temporary coloured border around the specified screen coordinates.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

01 Sep 2017, 02:28

A function to get the element under the cursor.

[WBGet function]
Basic Webpage Controls with JavaScript / COM - Tutorial - Tutorials - AutoHotkey Community
https://autohotkey.com/board/topic/47052-basic-webpage-controls-with-javascript-com-tutorial/

See above for JEE_IntExpEltGetPos and JEE_Borders functions.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

Last edited by jeeswg on 01 Sep 2017, 08:31, edited 3 times in total.
User avatar
tank
Posts: 2326
Joined: 28 Sep 2013, 22:15
Facebook: charlie.simmons.7334
Google: ttnnkkrr
GitHub: ttnnkkrr
Location: Louisville KY
Contact:

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

01 Sep 2017, 08:19

Just a thought. When you post these maybe put links to the library items your using?
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
https://www.facebook.com/ahkscript.org
If you have forum suggestions please submit a pull request
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

02 Sep 2017, 01:42

Get column under cursor.
- If the element under the cursor is the cell (TD), then the parent is the row (TR), and the grandparent is the table (TBODY).
- To know which column to grab: there may be a better way but the script loops through the innerText for each cell in the current row, until it finds a match (i.e. the cell under the cursor.)
- The script finds each row in the body, and grabs the nth cell in each row.

See above for: WBGet and JEE_IntExpGetEltUnderCursor functions.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus



Get table under cursor.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus



Get row under cursor.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

User avatar
TheDewd
Posts: 1017
Joined: 19 Dec 2013, 11:16
Location: USA

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 10:10

How can I use this on a website that uses iFrames. The website has 3 iFrames (Top, Left, Right). The website's contents are displayed inside the frames.

The script only works if I open the URL of the page inside the iFrame in a new tab/window.

I want to create a script that performs different actions based on the element under the mouse cursor.

For example, I would create a right-click menu, but the menu options would be different depending on the item I right-clicked on the website.

To get the data from inside the iFrame, I currently use something like this:

Code: [Select all] [Download] (Untitled.ahk)GeSHi © Codebox Plus

For Window In ComObjCreate("Shell.Application").Windows {
If (Window.LocationName = "Website Title") {
Main := Window.Document.parentWindow.frames["body"].frames["main"].Document

If (Main.Title = "Title of page inside iframe") {
MsgBox, % Main.getElementById("TheElementID").outerHTML
}
}
}
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
User avatar
tank
Posts: 2326
Joined: 28 Sep 2013, 22:15
Facebook: charlie.simmons.7334
Google: ttnnkkrr
GitHub: ttnnkkrr
Location: Louisville KY
Contact:

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 13:46

This script gets frames but coordinates are not suited for zoom
https://autohotkey.com/boards/viewtopic ... 263#p27263
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
https://www.facebook.com/ahkscript.org
If you have forum suggestions please submit a pull request
User avatar
TheDewd
Posts: 1017
Joined: 19 Dec 2013, 11:16
Location: USA

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 13:59

@tank: That's exactly what I needed! This should be fine for me, but if I needed to share with others who are using zoom are there any ways to make it work?
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 14:31

@TheDewd:
- Does my script not work on websites that use frames? What exactly is the problem?
- Please provide me with the names of any websites (that don't require a login) that you know of, that use frames, so I can test my script. (I don't know from memory of any such websites.)
- Even if tank's solution works for you, I think this is worth investigating. Thanks.
User avatar
TheDewd
Posts: 1017
Joined: 19 Dec 2013, 11:16
Location: USA

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 14:57

@jeeswg:

Tank's solution works, but if possible it would be great to not rely on having the zoom at 100%. I tried to combine your solution with tank's, but I've had no success. The coordinates are always offcenter when zoomed in.

The website is not public. It is for my job/work.

Your script only puts the borders around the top & bottom frame -- not the content inside the frames.

It's complicated because it's using iFrames within iFrames (Multi-Level). See below...

Here's the source code of the initial page (TOP & BODY):

Code: [Select all] [Expand] [Download] (Untitled.html)GeSHi © Codebox Plus


Here's the source code of the "Body" frame (MENU & MAIN):

Code: [Select all] [Download] (Untitled.html)GeSHi © Codebox Plus

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<TITLE id="PageTitle">Body Source</TITLE>
</HEAD>
<FRAMESET cols="160,*" id="frameContainer" frameborder="yes" bordercolor="#dbd7d0">
<frame SRC="http://www.google.com/" id="menu" NAME="menu" SCROLLING=no marginheight=0 marginwidth=0></frame>
<frame name="main" src="http://www.google.com/" scrolling="auto" marginheight=0 marginwidth=0></frame>
</FRAMESET>
</HTML>


Image
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 16:37

Preliminary rewrite of the script in the original post to handle frames. So far tested only to one level of depth.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


- The script uses the same logic as tank's script, while trying to account for the zoom level.
- The idea is that if the element under the cursor is a frame, you check the element under the cursor again, but this time relative to the frame, and you repeat this until you retrieve the first element under the cursor that is not a frame.
- There is some fiddly mathematics involved, re. working out the positions of the cursor and the frames, which I haven't fully worked out, but which I've sort of figured out intuitively.
- If you wish to provide some further html examples with nested frames, I could take a look.
- Note: I was concerned that it would be pointless to test your examples, as they would be frames with no elements in them, however, you get an error message when you run the files: 'This content cannot be displayed in a frame', and even a nice red X image element, that you can test the script on. So that was lucky, I don't know whether you were aware of that or not.
- Your examples, horizontal and vertical, were perfect, just what I needed to be able to do some effective tests.
- I searched around for websites that use frames or example webpages with frames. I didn't find any good examples. So I would welcome any.
User avatar
TheDewd
Posts: 1017
Joined: 19 Dec 2013, 11:16
Location: USA

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 16:57

Your updated code works for the top iFrame, but the bottom iFrame is only showing the following:

oElt.tagName
FRAME

oElt.OuterHTML
<FRAME marginHeight=0 src="ThePageForThisFrame.html" name=main marginWidth=0>
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 17:04

- I just tested my script on the 2 html samples again, with them zoomed in, and they worked perfectly. So it's hard to say what the problem is.
- Perhaps you can tweak the script to make it work for you?
- Btw the script might try to handle nested frames, but I don't know if it will handle them correctly. So you could try break after one iteration of the while loop. [EDIT:] Ah, I left the break in there already.
- Btw your test relates to one of the example scripts that you posted ('First Source'), correct? Did it work correctly on the other example ('Body Source')?
- I tried both html examples separately. So if you want to me to test a different html example, please provide it explicitly.
User avatar
TheDewd
Posts: 1017
Joined: 19 Dec 2013, 11:16
Location: USA

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 17:19

The website has a TOP and BOTTOM frame.

The TOP frame loads a page containing no frames.

The BOTTOM frame loads a page creating two more frames.

The BOTTOM frame has the two frames inside it, LEFT AND RIGHT.

Your script works on the TOP frame because there are no more levels.

The BOTTOM frame needs to go down more levels because the source of the BOTTOM frame is just to create the two additional frames for LEFT and RIGHT.

I would then need to go another level into the LEFT and RIGHT frame to get the content of those frames.

Hope that makes sense! The HTML I provided earlier is all I have at the moment.

I'm not sure why it shows that error when I try my examples in Internet Explorer -- Same error here... :-/
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
User avatar
jeeswg
Posts: 4952
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Internet Explorer get element under cursor (show borders, show text) (any zoom percentage)

10 Nov 2017, 17:32

Well if there are more levels, you should remove the line 'break'. But I don't know if it will work correctly. If you provide some html examples that have frames within frames, I could experiment. I might try and create such an example myself, so that ultimately my script can handle frames within frames.

Return to “Scripts and Functions”

Who is online

Users browsing this forum: No registered users and 13 guests