[Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

Post your working scripts, libraries and tools
User avatar
yawikflame
Posts: 20
Joined: 02 Jan 2017, 06:19

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

24 Apr 2018, 23:54

Xtra wrote:PageInst.Evaluate("document.getElementsByTagName('html')[0].textContent;").Value

thank you! that works partially.
i was hoping to grab the tags as well...
i'm trying to retrieve all the buttons that have a certain value, and i tried several solutions offered at stackOverFlow, but none of them worked.

cheers ;)
yawik
User avatar
Xtra
Posts: 1205
Joined: 02 Oct 2015, 12:15

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

25 Apr 2018, 00:49

With all tags:
PageInst.Evaluate("document.getElementsByTagName('html')[0].outerHTML;").Value
Shadowpheonix
Posts: 1258
Joined: 16 Apr 2015, 09:41

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

26 Apr 2018, 13:04

For some reason, I cannot seem to use Chrome.GetPageByTitle("page title"). When I try, it always fails to retrieve the page. :(

Here is a version of GeekDude's InjectJS.ahk example that I modified to show what I am trying. If I use Chrome.GetPage() instead (as originally written in the example), the script works perfectly.
Spoiler
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

29 Apr 2018, 12:39

After a little reflection, I wrote an a simple extension for the class Chrome, which eliminates the need to change the original code and supports the functionality described earlier.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


I hope, you will find this useful for improvement.
Last edited by KusochekDobra on 29 Apr 2018, 15:01, edited 2 times in total.
User avatar
jeeswg
Posts: 4970
Joined: 19 Dec 2016, 01:58
Location: UK

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

29 Apr 2018, 12:46

Here's a way to get the Chrome path. Cheers.

Code: [Select all] [Download] GeSHi © Codebox Plus

q:: ;Chrome - get path
RegRead, vPath, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\chrome.exe
MsgBox, % vPath
return
User avatar
Xtra
Posts: 1205
Joined: 02 Oct 2015, 12:15

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

30 Apr 2018, 00:55

KusochekDobra wrote:After a little reflection, I wrote an a simple extension for the class Chrome, which eliminates the need to change the original code and supports the functionality described earlier.


You can shorten your class like this:

Code: [Select all] [Download] GeSHi © Codebox Plus

Class ChromeEx Extends Chrome {
__New(ProfilePath:="", URL:="about:blank", Flags:="", ChromePath:="", DebugPort:=9222, funcName := "") {
this.funcName := funcName
base.__New(ProfilePath, URL, Flags, ChromePath, DebugPort)
}
; etc etc...
}

And then you would use:
myChromeEx := new ChromeEx("ChromeProfile")

You also dont need to add repeat code that is in chrome.ahk:
GetPageBy(Key, Value, MatchMode:="exact", Index:=1)
its in the base class and you have access to it already.

Give it a try.

HTH
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

30 Apr 2018, 20:34

Thank you, this is very much noticed! ;)
The method GetPageBy() is overridden, passing to the constructor Class Page two parameters.
And it does not have to be an extension. I would like to see this tool to be in the future part of the library and just to not clutter up the author's original code, it seemed useful, to take it beyond.
Stavencross
Posts: 59
Joined: 24 May 2016, 16:42

grabbing JSON from page

05 May 2018, 09:54

Really been digging in to this Lib, its so helpful! Just want to see if ya'll have a better way on how to scrape the JSON I've been grabbing from these pages. Here is my code:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus




If you check jsonGrabber(), you'll see I'm grabbing the JSON from the page DOM. I grab the innerHTML of the page, and run strReplace to get rid of the <pre> tags. I thought I'd see if ya'll had a better way to accomplish this as it feels dirty. I'm also open to any other code optimizations you could throw my way. I've been working on my coding skills a lot in the last 6 months and I know I really need to start pushing myself to write better & more modular code
maitresin
Posts: 22
Joined: 20 Mar 2018, 19:33

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

05 May 2018, 20:51

Hi,

I got 2 questions:

1-How I could save my profil and cookies? I mean if I log into a website I want to be able to launch this chrome and still be logged in my account.
2-How I could download an .html page like the exemple .pdf but .html. It also need to be save cookies and keep connection active for downloading the .html from my account (which is only downloadable from account).

Thanks!
GeekDude
Posts: 838
Joined: 02 Oct 2013, 22:13

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

08 May 2018, 14:44

Shadowpheonix

Shadowpheonix wrote:For some reason, I cannot seem to use Chrome.GetPageByTitle("page title"). When I try, it always fails to retrieve the page. :(

Completely my fault. I forgot to return the page found in that helper function (also GetPageByURL). It will be fixed in the next release.



KusochekDobra

KusochekDobra, I am going to break down my replies to your posts into a numbered list for easier reference.


# 1

KusochekDobra wrote:If under the control of the script will be several pages and all of them will be send a messages, then we need implement the concept of the event stack for make a manage all of them without transmittance.

I am not sure what you mean by the word "transmittance". From context I think it means something like blocking the event listener thread?

Would you be able to produce some code that demonstrates the problem which is solved by implementing your own event queue? From my tests things act more or less the same both with and without using the queue. (replacing this.e.InsertEvent(msg.text) with this.e.bFunction.Call(msg.text). The code I used to test is shown below.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus



# 2

Code: [Select all] [Download] GeSHi © Codebox Plus

	; Finds a running process chrome.exe with a parameter 'remote-debugging-port'
FindStartedDEV() {
findMe := "--remote-debugging-port=" . this.DebugPort
for Item in ComObjGet( "winmgmts:" ).ExecQuery("Select * from Win32_Process") {
if (Item.Name == "chrome.exe" && InStr(Item.Commandline, findMe))
return true
} return false
}

In what context do you use FindStartedDEV? It looks like it could be useful but I'm not sure how.
I might want to adapt this into a utility function that returns a list of ports for chrome instances running the debug protocol.


# 3

Code: [Select all] [Download] GeSHi © Codebox Plus

			if (funcName) {
this.e := New this.EventQueue(funcName)
this.BoundKeepAlive := this.Call.Bind(this, "Browser.getVersion",, False)
}

Why is BoundedKeepAlive now conditional?


# 4

Code: [Select all] [Download] GeSHi © Codebox Plus

				if (this.e && data.method == "Console.messageAdded" && (msg := data.params.message).level == "info")
this.e.InsertEvent(msg.text)

Why limit callbacks to only console.info() messages? There are other callbacks that could be useful to a user such as Page.frameNavigated or messages generated by console.log.


# 5

Code: [Select all] [Download] GeSHi © Codebox Plus

			InsertEvent(msg) {
this.eventStack.InsertAt(1, msg)
ePop := this.ePop
SetTimer,% ePop, -10
}

Setting a timer for a class method function increases the reference count, meaning that the class cannot be garbage collected until that timer is deleted. Your queue function doesn't appear to have a method for removing this timer, so as the script connects to and disconnects from pages it will slowly leak memory.

My suggestion would be to modify _PopEvent() so that, before the while loop it calls SetTimer,% ePop, Delete. This way the timer will be deleted and the reference count will go down whenever the timer is no longer needed.



Stavencross

Stavencross wrote:Really been digging in to this Lib, its so helpful! Just want to see if ya'll have a better way on how to scrape the JSON I've been grabbing from these pages. Here is my code:

*snip*

If you check jsonGrabber(), you'll see I'm grabbing the JSON from the page DOM. I grab the innerHTML of the page, and run strReplace to get rid of the <pre> tags. I thought I'd see if ya'll had a better way to accomplish this as it feels dirty. I'm also open to any other code optimizations you could throw my way. I've been working on my coding skills a lot in the last 6 months and I know I really need to start pushing myself to write better & more modular code

It's hard to say without access to the actual page, but I would bet that grabbing from document.body.innerText would work better for you than document.body.innerHTML



maitresin

maitresin wrote:Hi,

I got 2 questions:

1-How I could save my profil and cookies? I mean if I log into a website I want to be able to launch this chrome and still be logged in my account.
2-How I could download an .html page like the exemple .pdf but .html. It also need to be save cookies and keep connection active for downloading the .html from my account (which is only downloadable from account).

Thanks!

  1. If you use the same profile folder each time the cookies will persist. You could start a chrome instance using that profile, log in manually, then use the script to do things after you've logged in. Alternatively, you could automate the login process so it logs in automatically. Also, if there isn't already a non-debug chrome instance running under the default profile, you could use this library to start a debug instance with the default profile and it should just work.
  2. If you just need the source of a particular page, such as is shown when you press control-u on a webpage, you should be able to access it with PageInst.Evaluate("document.children[0].outerHTML").value. If you need the entire page plus its resources, such as is downloaded when you press control-s on a webpage, I'm not sure if that's possible from this interface.
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

08 May 2018, 19:00

#1
In fact, it was just a guess witch I did not bother checking, and now I look like a fool. Apparently, somewhere in the implementation, this method has already been created. Thank you for this clarification.
But for greater reliability, it can come in handy. :)

#2
I use for automatic understanding, connect me to an already existing process, or create a new connection. For example:

Code: [Select all] [Download] GeSHi © Codebox Plus

if (Chrome.FindStartedDEV()) {
ChromeInst := Chrome
ChromeInst.DebugPort := 9222
ChromeInst.funcName := funcName
} else
ChromeInst := new Chrome(ProfilePath,,Flags,,,funcName)


#3
Sorry, this code-line got there by accident. Usually I write the code late at night and because of what I make such stupid mistakes.

#4
This decision seemed useful. console.log() I use for debug on the page in browser.

#5
As it turned out in #1, it is not necessary, but thanks for the clarification. I still do not know much, so I can not apply knowledge correctly.

The result:
I really liked your idea, use such a functional approach to automate the browser, instead of COM. And I also wanted to see how this project develops, getting new opportunities, so I just try to be useful in my unindifference. You can use all the examples for your own purposes and describe them at your own discretion. You have more experience and I am sure that you will do it right! :thumbup:
GeekDude
Posts: 838
Joined: 02 Oct 2013, 22:13

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

09 May 2018, 08:48

KusochekDobra wrote:I really liked your idea, use such a functional approach to automate the browser, instead of COM. And I also wanted to see how this project develops, getting new opportunities, so I just try to be useful in my unindifference. You can use all the examples for your own purposes and describe them at your own discretion. You have more experience and I am sure that you will do it right! :thumbup:


Thank you for the response and for the kind words. I appreciate your enthusiasm and hard work! :bravo:

Your code was well designed and shows a good understanding of the concepts, even if a few parts were missed. Thanks to your work and ideas, you can expect to see event handling and callbacks implemented in the next release.
GeekDude
Posts: 838
Joined: 02 Oct 2013, 22:13

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

09 May 2018, 21:25

Chrome.ahk v1.2 has been released

Please see the original post for a link to the release and revision history pages.



Changes


Additions

  • Added support for an array of URLs in class Chrome's constructor.
  • Added support for event callbacks by passing a function to the Chrome.GetPage methods. Thanks KusochekDobra!
  • Added method Chrome.FindInstances to look for preexisting Chrome instances. Thanks KusochekDobra!
  • Added new example script EventCallbacks.ahk demonstrating JS callbacks to AHK.
PoorInRichfield
Posts: 5
Joined: 01 May 2018, 16:50

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

10 May 2018, 08:49

First off, thank you GeekDude for this script! Using a modern browser like Chrome is so much better than struggling with IE and COM.

I have one small suggestion. I've been using the Evaluate(JS) method for most of the work I'm doing. However, when this method call bombs because the JavaScript fails, the exception message isn't very helpful as it's usually describing a line of code in your Chrome.ahk file and not the JavaScript that caused the issue. My solution to this is to simply add the JS variable to the second parameter in the Exception object...

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

GeekDude
Posts: 838
Joined: 02 Oct 2013, 22:13

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

10 May 2018, 12:13

PoorInRichfield wrote:First off, thank you GeekDude for this script! Using a modern browser like Chrome is so much better than struggling with IE and COM.

You're welcome! It's great to hear that people are finding my work useful.

PoorInRichfield wrote:I have one small suggestion. I've been using the Evaluate(JS) method for most of the work I'm doing. However, when this method call bombs because the JavaScript fails, the exception message isn't very helpful as it's usually describing a line of code in your Chrome.ahk file and not the JavaScript that caused the issue. My solution to this is to simply add the JS variable to the second parameter in the Exception object...

Although that's not exactly what that parameter is supposed to be used for, your proof of concept gave me an idea. It's possible to create your own exception objects without using the Exception() function, or by basing them off the function. What if we set the What parameter to -1 (meaning it will show the error for your call to PageInst.Evaluate() rather than my call to throw Exception) and then add our own field Code to hold the appropriate JS code? For example:

Code: [Select all] [Download] GeSHi © Codebox Plus

; Chrome.ahk
if (response.exceptionDetails)
throw {"base": Exception(response.result.description
, -1, Chrome.Jxon_Dump(response.exceptionDetails))
, "Code": JS}

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

The effect of -1 is visible if you don't catch the error and let it go straight to the screen.



Another idea I had, which you may or may not like better, would be to augment the Extra parameter to include the code.

Code: [Select all] [Download] GeSHi © Codebox Plus

if (response.exceptionDetails)
throw Exception(response.result.description, -1
, Chrome.Jxon_Dump({"Code": JS
, "exceptionDetails": response.exceptionDetails}))



Finally, I think it might be nice to add a utility function to Chrome.ahk for interpreting the exceptions it displays it in a nice dialog that shows the line numbers, the line on which the error occurred, etc. like AHK's built in.

Code: [Select all] [Download] GeSHi © Codebox Plus

try
PageInst.Evaluate(JS)
catch e
Chrome.DisplayError(e) ; or, depending on how it is implemented, e.Display()



What do you think?
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

10 May 2018, 12:55

It will be very powerful instrument! :)

In "Source code (zip)" - link => in "Chrome.ahk" file => in 72 code-line have a one missing letter "t" (...\App Pahs\chrome.exe).
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

12 May 2018, 19:50

Can I ask you to add the name of the function to the constructor of class Page by the third parameter, which will be called when the connection is broken?
For example:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


I make a simple wrapper for use in the program and concluded that it would be convenient to remove data about an unused connection from the object of this wrapper because of a disconnection that may occur if the user closes the tab.
Example:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


I hope this request does not look like impudence.
And thank you very much for your hard work!
Beautiful code and maximum efficiency! :bravo:
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

14 May 2018, 08:51

Sometimes, when creating an instance of a class Chrome, passing an address to the constructor, the method WaitForLoad() returns flow control before the page loads:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


This can be circumvented by creating an instance of the class without an address, followed by a transition PageInst.Call("Page.navigate", {"url": address}) and PageInst.WaitForLoad(), but is there any way to do without it?
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

14 May 2018, 10:35

And another question. Is it possible to replace this exception:

Code: [Select all] [Download] GeSHi © Codebox Plus

Call(DomainAndMethod, Params:="", WaitForResponse:=True)
{
if !this.Connected
throw Exception("Not connected to tab")

; ... other code
}

To return a connection to the state "disabled"?

I think that this is logical, if all the tabs are closed, then you need to create a new connection, reinitializing the instance of the class Chrome. Instead, there is a constant call "Runtime.evaluate" without parameters, and the program terminates its work after the exception message.
KusochekDobra
Posts: 32
Joined: 25 Apr 2016, 18:00

Re: [Library] Chrome.ahk - Automate Google Chrome using native AutoHotkey. No Selenium!

15 May 2018, 12:24

Is it possible to connect to events such as Target.targetCreated? This feature would reduce the possibility of errors, telling the ahk-code what him need to create a new connection.

Return to “Scripts and Functions”

Who is online

Users browsing this forum: nwr425 and 25 guests