Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post a reply


In an effort to prevent automatic submissions, we require that you type the text displayed into the field underneath.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 28 Aug 2017, 15:06

I tried your code on a page that I use having frames and it did not work. I'm guessing it is something that is going to be different depending on how view page is built. Thank you for sharing

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Visioneer » 28 Aug 2017, 10:19

This gets it right.
myURL := wb.document.parentWindow.document.getElementsByTagName("frame")[1].LocationURL

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Visioneer » 28 Aug 2017, 09:54

Hi,
I want to get the "current" url of the 2nd frame of 2 frame frameset. It displays many different pages, while the address bar shows the original
url you started out with. It never changes. When I grab the location.href or the src, it keeps coming back with the address bar url. I need the
active url of the fames page. I do not see anything in these frame pages source code to indicate their url.

These return the address bar only. Not what I want.
;myURL := wb.document.parentWindow.document.getElementsByTagName("frame")[1].src
;myURL := wb.document.parentWindow.document.getElementsByTagName("frame")[1].document.location.href

I would prefer COM solution.

Thanks

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Cerberus » 29 Jul 2017, 17:40

Code: [Select all]GeSHi © Codebox Plus

pwb.document.getElementsByClassName("x-btn-noicon")[1].click

This works. Apparently, when you try it with the "get ID" method, it gives you the text area on the button, but not the button itself. I think normally "click" on such a text area works, but not on this page. So I used "get elements by class name" to get at one of the layers above the button text: any of the layers that have something with "btn" works on this page, apparently.

Notice the "[1]". The thingy "get elements" gives a collection of results, not just a single results. But you can only act on a single thing or get the content of a single thing (at least in this way). That's why you always need a number when you use a "plural" get/select thingy, if you want to act on it.

Note also that 1 means the second element, and 0 means the first element: that's called "zero indexing", and for some reason that fails to convince me, many computer languages use zero indexing.

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by bobnoob » 26 Jul 2017, 23:49

OK it;s working now.
The fix is to get rid of

Code: [Select all]GeSHi © Codebox Plus

Browser_Forward::Reload
Browser_Back::




Now I can fill in the user name and password for the website I tried to scrap. But I cannot click the button.
Can you please check this website and advise on how to click the logon button??
https://oms.itradenetwork.com/secure/login/logon.cfm

thank you so much for replying

I have tried using iWB2 Learner. it says the login button has ID and index. but it has no name.


I tried this code to click the login. but it does not work. pwb.document.all.getElementByID("ext-gen32").click()

I did some success with this code. I just simulate enter key.
Sleep, 1000
SendInput {Enter 2}

I think I am off to a good start.

I still have a lot of questions. But I will start my own thread.



Thank you very much!!

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 25 Jul 2017, 21:53

Hi bobnoob,

It would be a bit time consuming for me to answer in an post. Do you want to email me at Joe@the-Automator.com and we'll plan a time to have a webex type call where I can walk you through some things?

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by bobnoob » 25 Jul 2017, 17:20

Helgef wrote:
do I just click the green Triangle and it will show message box right?

Press Browser_Back.



sorry I am :crazy: slow... where do I press browser_back?


you mean press with my left mouse click?

this browser back
Image


or this?
Image


thank you

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Helgef » 25 Jul 2017, 16:24

do I just click the green Triangle and it will show message box right?

Press Browser_Back.

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by bobnoob » 25 Jul 2017, 16:11

Sorry Yes, it does have that too.
I just push it down , as I was watching your video and I follow step by step . unfortunately it didn't go as well as the video.

do I just click the green Triangle and it will show message box right?


does it matter that I am running my SciTE.exe from this location: C:\IT\AutoHotKey\SciTE\SciTE.exe

here is some screenshot. I click the green triangle .. but I don't see a message box.
Image


thank you again

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 24 Jul 2017, 21:51

Hi bobnoob,
Yes, definitely comment out the #include default. It is a file on my computer that I use "everywhere" but is not needed.

If you're using the syntax writer you should have had the "WBGet" function also pasted to that page when you used it. Basically you're trying to use the wbGET function when your code shows: pwb :=WBGet()

The syntax writer would have inserted the following :

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by bobnoob » 24 Jul 2017, 18:07

Hi Joe,
thanks for the awesome tutorial.
one simple script , but I cannot get it work,

Here my script
. First I have an error that says
C:\IT\AutoHotKey\SciTE\OMSItrade.ahk (1) : ==> Function library not found.
Specifically: #Include <default_Settings>


Code: [Select all]GeSHi © Codebox Plus

#Include <default_Settings>
Browser_Forward::Reload
Browser_Back::
;***********************************************************
pwb := WBGet()

var:=pwb.LocationURL ;grab current url


MsgBox,,page url, % var




so I tried to just remove #Include <default_Settings>. to get over the error.
I have IE page open to amazon login just like your first video. I press the green triangle button but there is no message box?


any idea? thank you

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 14 May 2017, 20:59

I'm looking forward to our Webinar this Tuesday @ 4 EST on Web scraping with AutoHotkey.
Here's the original post and signup link
:dance:

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 09 Feb 2017, 15:28

Just comment out the lines that call it. You won't have the icon for the menu items that used it but that is the quickest/easiest way to remedy it.

Or you could download it here and put it in these locations:
C:\Windows\System32
C:\Windows\SysWOW64

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by SnowFlake » 09 Feb 2017, 14:43

I dont have the WMPLOC.DLL on my computer so i get errors, im on windows 10

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by Joe Glines » 13 Nov 2016, 16:26

Glad you figured it out

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by IfWinExist » 13 Nov 2016, 13:09

D'oh ! Answered my own question. Of course it does "nothing". It is running in the background. Launch SciTE or other text editor, and then use the background script's shortcuts. Hopefully this will help someone else down the road.

Re: Web Scraping with AutoHotkey & COM Tutorial- GUI syntax writer and demo videos

Post by IfWinExist » 13 Nov 2016, 12:26

Joe - your tutorial videos are really great! I am having trouble launching your web-scraping script-writer. I launch it and nothing seems to happen, but my other scripts (including my scripts, and AHK Studio Developer) are running fine. I have tried two machines: Windows 10 Home with Autohotkey version 1.1.24.02 and Windows Server 2008 R2 with Autohotkey version 1.1.22.09 . I read through all the posts in this thread, and when I encountered icons that wouldn't load in the script, I commented them out, as suggested for Win 10. What versions of Windows and Autohotkey are you using in your tutorial video? Any other help you can provide would be greatly appreciated !!

Re: WebScraping and COM- GUI syntax writer and demo videos

Post by Joe Glines » 14 Jan 2016, 17:23

Yep- no worries. I'd also through in a #SingleInstance, Force command...

Re: WebScraping and COM- GUI syntax writer and demo videos

Post by Bruttosozialprodukt » 14 Jan 2016, 17:22

Thanks. :) Well, the transparency was just a random idea of mine, I'll probably just remove it with the next release.

Re: WebScraping and COM- GUI syntax writer and demo videos

Post by Joe Glines » 14 Jan 2016, 17:18

Thanks Bruttosozialprodukt! I'm playing with it. I love that the zoom level doesn't affect the highlighted area! That was always annoying with the Learner tool. Personally I don't like the transparency but I'll let others talk to that side of things. Thanks for posting it here! I hope it helps some people (and me!)

Top