Google Search function that returns Url of first hit

Get help with using AutoHotkey and its commands and hotkeys
roysubs
Posts: 107
Joined: 29 Sep 2018, 16:37

Google Search function that returns Url of first hit

17 Oct 2018, 05:41

Found these extremely useful functions and something that I really want to do, but it was last updated in 2011 and no longer works.
https://autohotkey.com/board/topic/7331 ... ry-script/

Could anyone advise on a working update, variants, or newer, better methods that supercede this please?

Specific error is in the UnHTM function.: "The following variable name contains an illegal character: '<lots of unicode characters>'"

Code: Select all

; -------------------- 
;
; Google(SearchQuery, StrTitle, StrDescriptionStr), function to return URL of first google search result
;
; Input the google query term or phrase in the first parameter. The function returns the url of the first google result.
; Optionally, you may also enter a variable in the 2nd or 3rd paramters:
;   Title       - stores the page title of the first result
;   Description - stores the google description text for that result.
;
; Url := Google("AutoHotkey", Title, Description)
; MsgBox, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description
;
; Returns:
; Url:           http://www.autohotkey.com/
; Title:         AutoHotkey - Free Mouse and Keyboard Macro Program with ...
; Description:   Free keyboard macro program. Supports hotkeys for keyboard, mouse, and joystick. Can expand abbreviations as you type them (AutoText).
;
; https://autohotkey.com/board/topic/73311-quick-dirty-google-query-script/
;
; -------------------- 

#F1:
    Url := Google("AutoHotkey", Title, Description)
    MsgBox, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description
return 

Google(Phrase, ByRef Title="", ByRef Text="")   ; http://www.autohotkey.com/forum/topic78570.html
{
	If DllCall("Wininet.dll\InternetGetConnectedState", "Str", 0x40, "Int", 0) {
		URLDownloadToFile, % "http://www.google.com/search?q=" EncodeURL(Phrase), %A_Temp%\GoogleSearch
		FileRead, Result, %A_Temp%\GoogleSearch
		FileDelete, %A_Temp%\GoogleSearch
		If (Title := RegExMatch(Result, "i)<h3 class=""r""><a href=""(?:/url\?q=)?\K[^/][^""& ]*", Link)) {
			StringTrimLeft, Result, Result, %Title%
			Title := UnHTM(SubStr(Result, @ := InStr(Result, ">", True, 25) + 1, InStr(Result, "</a>") - @)), RegExMatch(Result, "si)(?:<div class=""?s""?>|<span class=""?st""?>)\K.*?(?=(?:<br>)*\s*</?(?:span|div|table))", Text), Text := RegExReplace(UnHTM(Text), "\s+", " ")
			Return EncodeURL(Link, True)
		} Else
			MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, Failed to find a result!
	} Else
		MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, No internet connectivity!
}

EncodeURL(Text, FromURL=False) ; Uberi
{
	FormatInteger := A_FormatInteger, FoundPos := 0, SearchFor := FromURL ? "%.." : "[^\w-.~% ]"
	SetFormat, IntegerFast, Hex
	If !FromURL
		StringReplace, Text, Text, `%, `%25, All
	While (FoundPos := RegExMatch(Text, SearchFor, Char, FoundPos + 1))
		StringReplace, Text, Text, %Char%, % FromURL ? Chr("0x" SubStr(Char, 2)) : "%" SubStr(0 SubStr(Asc(Char), 3), -1), All
	If !FromURL {
		StringReplace, Text, Text, %A_Space%, +, All
		Text := RegExReplace(Text, "%..", "$U0")
	}
	SetFormat, IntegerFast, %FormatInteger%
	Return Text
}

UnHTM(HTM)   ; www.autohotkey.com/forum/topic51342.html
{
	Static HT
	If (HT = "")
		HT := "ááââ´´ææàà&ååããää&bdquo„¦¦&bull•ç縸¢¢&circˆ©©¤¤&dagger†&dagger‡°°÷÷ééêêèèððëë&euro€&fnofƒ½½¼¼¾¾>>&hellip…ííîî¡¡ìì¿¿ïï««&ldquo“&lsaquo‹&lsquo‘<<¯¯&mdash—µµ··  &ndash–¬¬ññóóôô&oeligœòòªªººøøõõöö¶¶&permil‰±±££"""»»&rdquo”®®&rsaquo›&rsquo’&sbquo‚&scaronš§§­ ¹¹²²³³ßßþþ&tilde˜××&trade™úúûûùù¨¨üüýý¥¥ÿÿ"
	HTM := RegExReplace(HTM, "<[^>]+>")
	Loop, Parse, HTM, &`;
		If !((A_Index & 1) or InStr(R, "&" A_LoopField ";", True))
			R .= "&" A_LoopField ";"
	StringTrimRight, R, R, 1
	Loop, Parse, R, `;
		If InStr(HT, A_LoopField)
			StringReplace, HTM, HTM, %A_LoopField%;, % SubStr(HT, InStr(HT, A_LoopField) + StrLen(A_LoopField), 1), All
		Else If (SubStr(A_LoopField, 2, 1) = "#")
			StringReplace, HTM, HTM, %A_LoopField%;, % Chr(((SubStr(A_LoopField, 3, 1) = "x") ? 0 : "") SubStr(A_LoopField, 3)), All
	Return RegExMatch(HTM, "\S") ? RegExReplace(HTM, "^\s*([\s\S]*\S)\s*$", "$1") : ""
}
CyL0N
Posts: 111
Joined: 27 Sep 2018, 09:58

Re: Google Search function that returns Url of first hit

17 Oct 2018, 06:22

EDIT: Removed other script for clarity...

Code: Select all

; -------------------- 
;
; Google(SearchQuery, StrTitle, StrDescriptionStr), function to return URL of first google search result
;
; Input the google query term or phrase in the first parameter. The function returns the url of the first google result.
; Optionally, you may also enter a variable in the 2nd or 3rd paramters:
;   Title       - stores the page title of the first result
;   Description - stores the google description text for that result.
;
; Url := Google("AutoHotkey", Title, Description)
; MsgBox, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description
;
; Returns:
; Url:           http://www.autohotkey.com/
; Title:         AutoHotkey - Free Mouse and Keyboard Macro Program with ...
; Description:   Free keyboard macro program. Supports hotkeys for keyboard, mouse, and joystick. Can expand abbreviations as you type them (AutoText).
;
; https://autohotkey.com/board/topic/73311-quick-dirty-google-query-script/
;
; -------------------- 

#F1:
    Url := Google("AutoHotkey", Title, Description)
    MsgBox, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description
return 

Google(Phrase, ByRef Title="", ByRef Text="")   ; http://www.autohotkey.com/forum/topic78570.html
{
	If DllCall("Wininet.dll\InternetGetConnectedState", "Str", 0x40, "Int", 0) {
		URLDownloadToFile, % "http://www.google.com/search?q=" EncodeURL(Phrase), %A_Temp%\GoogleSearch
		FileRead, Result, %A_Temp%\GoogleSearch
		FileDelete, %A_Temp%\GoogleSearch
		If (Title := RegExMatch(Result, "i)<h3 class=""r""><a href=""(?:/url\?q=)?\K[^/][^""& ]*", Link)) {
			StringTrimLeft, Result, Result, %Title%
			Title := UnHTM(SubStr(Result, @ := InStr(Result, ">", True, 25) + 1, InStr(Result, "</a>") - @)), RegExMatch(Result, "si)(?:<div class=""?s""?>|<span class=""?st""?>)\K.*?(?=(?:<br>)*\s*</?(?:span|div|table))", Text), Text := RegExReplace(UnHTM(Text), "\s+", " ")
			Return EncodeURL(Link, True)
		} Else
			MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, Failed to find a result!
	} Else
		MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, No internet connectivity!
}

EncodeURL(Text, FromURL=False) ; Uberi
{
	FormatInteger := A_FormatInteger, FoundPos := 0, SearchFor := FromURL ? "%.." : "[^\w-.~% ]"
	SetFormat, IntegerFast, Hex
	If !FromURL
		StringReplace, Text, Text, `%, `%25, All
	While (FoundPos := RegExMatch(Text, SearchFor, Char, FoundPos + 1))
		StringReplace, Text, Text, %Char%, % FromURL ? Chr("0x" SubStr(Char, 2)) : "%" SubStr(0 SubStr(Asc(Char), 3), -1), All
	If !FromURL {
		StringReplace, Text, Text, %A_Space%, +, All
		Text := RegExReplace(Text, "%..", "$U0")
	}
	SetFormat, IntegerFast, %FormatInteger%
	Return Text
}

UnHTM( HTM ) {   ; Remove HTML formatting / Convert to ordinary text   by SKAN 19-Nov-2009
	Static HT,C=";" ; Forum Topic: www.autohotkey.com/forum/topic51342.html  Mod: 16-Sep-2010
	IfEqual,HT,,   SetEnv,HT, % "ááââ´´ææàà&ååãã&au"
		. "mlä&bdquo„¦¦&bull•ç縸¢¢&circˆ©©¤¤&dagger†&dagger‡°"
		. "°÷÷ééêêèèððëë&euro€&fnofƒ½½¼¼¾¾>>&h"
		. "ellip…ííîî¡¡ìì¿¿ïï««&ldquo“&lsaquo‹&lsquo‘<<&m"
		. "acr¯&mdash—µµ··  &ndash–¬¬ññóóôô&oeligœòò&or"
		. "dfªººøøõõöö¶¶&permil‰±±££""»»&rdquo”®"
		. "®&rsaquo›&rsquo’&sbquo‚&scaronš§§­ ¹¹²²³³ßßþþ&tilde˜&tim"
		. "es×&trade™úúûûùù¨¨üüýý¥¥ÿÿ"
		$ := RegExReplace( HTM,"<[^>]+>" )               ; Remove all tags between  "<" and ">"
	Loop, Parse, $, &`;                              ; Create a list of special characters
		L := "&" A_LoopField C, R .= (!(A_Index&1)) ? ( (!InStr(R,L,1)) ? L:"" ) : ""
	StringTrimRight, R, R, 1
	Loop, Parse, R , %C%                               ; Parse Special Characters
		If F := InStr( HT, L := A_LoopField )             ; Lookup HT Data
			StringReplace, $,$, %L%%C%, % SubStr( HT,F+StrLen(L), 1 ), All
	Else If ( SubStr( L,2,1)="#" )
		StringReplace, $, $, %L%%C%, % Chr(((SubStr(L,3,1)="x") ? "0" : "" ) SubStr(L,3)), All
	Return RegExReplace( $, "(^\s*|\s*$)")            ; Remove leading/trailing white spaces
}
roysubs
Posts: 107
Joined: 29 Sep 2018, 16:37

Re: Google Search function that returns Url of first hit

17 Oct 2018, 09:23

Hi CyLON, this doesn't generate any errors for sure, but when I run it, it does nothing, no MsgBox is displayed. Thought I maybe needed to have Edge or Chrome open for it to run, but still nothing when in those.

What needs to change to make it work? (I'm on Windows 10)
tmplinshi
Posts: 1278
Joined: 01 Oct 2013, 14:57

Re: Google Search function that returns Url of first hit

17 Oct 2018, 10:16

try this:

Code: Select all

/*
result := gsearch("autohotkey")

MsgBox, % result.MaxIndex()
MsgBox, % result[1].title "`n"
         . result[1].url "`n"
         . result[1].description "`n"
*/

gsearch(keyword) {
	return gsearch.search(keyword)
}

gsearch_getFirstUrl(keyword) {
	static whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	whr.Open("HEAD", "https://www.google.com/search?btnI&q=" keyword, true)
	whr.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36")
	whr.Option(6) := false ; disable auto redirect
	whr.Send()
	whr.WaitForResponse()

	try
		return whr.GetResponseHeader("Location")
	catch
		return gsearch(keyword)[1].url
}

class gsearch
{
	search(keyword) {
		req := this.httpGet("https://www.google.com/search?q=" keyword)
		doc := this.html2dom(req.responseText)
		return this.parseDom(doc)
	}

	httpGet(ByRef url) {
		static req := ComObjCreate("Msxml2.XMLHTTP")

		req.open("GET", url, true)
		req.setRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36")
		req.SetRequestHeader("Pragma", "no-cache")
		req.SetRequestHeader("Cache-Control", "no-cache")
		req.SetRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
		req.Send()
		while req.readyState != 4
			sleep 100

		return req
	}

	parseDom(doc) {
		out := []

		list_g := doc.getElementByID("search").getElementsByClassName("g")
		Loop, % list_g.length
		{
			if !list_g[A_Index-1].id {
				a := list_g[A_Index-1].getElementsByClassName("r")[0].all.tags("a")[0]
				out.push( { "title": RegExReplace(a.innerText, "`as)\R.*$")
				           , "url": a.href
				           , "description": list_g[A_Index-1].getElementsByClassName("st")[0].innerText } )
			}
		}

		return out
	}

	html2dom(ByRef html, removeHeadAndImg := true) {
		if removeHeadAndImg {
			html := RegExReplace(html, "`as)<head>.*?</head>|<img .*?>")
		}
		html := "<!doctype html><meta http-equiv=""X-UA-Compatible"" content=""IE=edge"">" . html

		static doc := ComObjCreate("HTMLFile")
		doc.open()
		doc.write(html)
		doc.close()
		
		return doc
	}
}
Edit: Added gsearch_getFirstUrl function
Last edited by tmplinshi on 17 Oct 2018, 11:20, edited 1 time in total.
tmplinshi
Posts: 1278
Joined: 01 Oct 2013, 14:57

Re: Google Search function that returns Url of first hit

17 Oct 2018, 11:05

I added gsearch_getFirstUrl function to above post. This function will first attempt to get the url from "I'm Feeling Lucky" search, if failed then it returns the result from gsearch function.
CyL0N
Posts: 111
Joined: 27 Sep 2018, 09:58

Re: Google Search function that returns Url of first hit

18 Oct 2018, 06:18

Elegant as ever, but with one minor flaw as opposed to https://autohotkey.com/board/topic/7331 ... ry-script/ which was referenced by the OP, it uses slightly more bandwidth & so is a little slower,though i much prefer it,given that's not an issue for me.

@roysubs, if you don't see a message box at all,it was because the search function was never run,i reckon the hotkey you bound it to was never picked up or something,in any case it doesn't need any browser open.

Besides @tmplinishi's solution is way better,if not more upto date,and the description is actually retrieved...

Running the script i posted earlier as it is, without activating the hotkey should run just fine,or just remove the hotkey label altogether,and that should clearly show if it there's an issue,i can't say for sure cos i'm running Win7 x64. Or just Here if still inclined to use your OP:

Code: Select all

Url := Google("AutoHotkey", Title, Description)
MsgBox, 0x40040, %A_ScriptName%, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description


; --------------------
;
; Google(SearchQuery, StrTitle, StrDescriptionStr), function to return URL of first google search result
;
; Input the google query term or phrase in the first parameter. The function returns the url of the first google result.
; Optionally, you may also enter a variable in the 2nd or 3rd paramters:
;   Title       - stores the page title of the first result
;   Description - stores the google description text for that result.
;
; Url := Google("AutoHotkey", Title, Description)
; MsgBox, % "Url:`t`t" Url "`nTitle:`t`t" Title "`nDescription:`t" Description
;
; Returns:
; Url:           http://www.autohotkey.com/
; Title:         AutoHotkey - Free Mouse and Keyboard Macro Program with ...
; Description:   Free keyboard macro program. Supports hotkeys for keyboard, mouse, and joystick. Can expand abbreviations as you type them (AutoText).
;
; https://autohotkey.com/board/topic/73311-quick-dirty-google-query-script/
;
; --------------------


Google(Phrase, ByRef Title="", ByRef Text="")   ; http://www.autohotkey.com/forum/topic78570.html
{
	If DllCall("Wininet.dll\InternetGetConnectedState", "Str", 0x40, "Int", 0) {
		URLDownloadToFile, % "http://www.google.com/search?q=" EncodeURL(Phrase), %A_Temp%\GoogleSearch
		FileRead, Result, %A_Temp%\GoogleSearch
		FileDelete, %A_Temp%\GoogleSearch
		If (Title := RegExMatch(Result, "i)<h3 class=""r""><a href=""(?:/url\?q=)?\K[^/][^""& ]*", Link)) {
			StringTrimLeft, Result, Result, %Title%
			Title := UnHTM(SubStr(Result, @ := InStr(Result, ">", True, 25) + 1, InStr(Result, "</a>") - @)), RegExMatch(Result, "si)(?:<div class=""?s""?>|<span class=""?st""?>)\K.*?(?=(?:<br>)*\s*</?(?:span|div|table))", Text), Text := RegExReplace(UnHTM(Text), "\s+", " ")
			Return EncodeURL(Link, True)
		} Else
			MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, Failed to find a result!
	} Else
		MsgBox, 262160, %A_ScriptName% - %A_ThisFunc%(): Error, No internet connectivity!
}

EncodeURL(Text, FromURL=False) ; Uberi
{
	FormatInteger := A_FormatInteger, FoundPos := 0, SearchFor := FromURL ? "%.." : "[^\w-.~% ]"
	SetFormat, IntegerFast, Hex
	If !FromURL
		StringReplace, Text, Text, `%, `%25, All
	While (FoundPos := RegExMatch(Text, SearchFor, Char, FoundPos + 1))
		StringReplace, Text, Text, %Char%, % FromURL ? Chr("0x" SubStr(Char, 2)) : "%" SubStr(0 SubStr(Asc(Char), 3), -1), All
	If !FromURL {
		StringReplace, Text, Text, %A_Space%, +, All
		Text := RegExReplace(Text, "%..", "$U0")
	}
	SetFormat, IntegerFast, %FormatInteger%
	Return Text
}

UnHTM( HTM ) {   ; Remove HTML formatting / Convert to ordinary text   by SKAN 19-Nov-2009
	Static HT,C=";" ; Forum Topic: www.autohotkey.com/forum/topic51342.html  Mod: 16-Sep-2010
	IfEqual,HT,,   SetEnv,HT, % "ááââ´´ææàà&ååãã&au"
		. "mlä&bdquo„¦¦&bull•ç縸¢¢&circˆ©©¤¤&dagger†&dagger‡°"
		. "°÷÷ééêêèèððëë&euro€&fnofƒ½½¼¼¾¾>>&h"
		. "ellip…ííîî¡¡ìì¿¿ïï««&ldquo“&lsaquo‹&lsquo‘<<&m"
		. "acr¯&mdash—µµ··  &ndash–¬¬ññóóôô&oeligœòò&or"
		. "dfªººøøõõöö¶¶&permil‰±±££""»»&rdquo”®"
		. "®&rsaquo›&rsquo’&sbquo‚&scaronš§§­ ¹¹²²³³ßßþþ&tilde˜&tim"
		. "es×&trade™úúûûùù¨¨üüýý¥¥ÿÿ"
		$ := RegExReplace( HTM,"<[^>]+>" )               ; Remove all tags between  "<" and ">"
	Loop, Parse, $, &`;                              ; Create a list of special characters
		L := "&" A_LoopField C, R .= (!(A_Index&1)) ? ( (!InStr(R,L,1)) ? L:"" ) : ""
	StringTrimRight, R, R, 1
	Loop, Parse, R , %C%                               ; Parse Special Characters
		If F := InStr( HT, L := A_LoopField )             ; Lookup HT Data
			StringReplace, $,$, %L%%C%, % SubStr( HT,F+StrLen(L), 1 ), All
	Else If ( SubStr( L,2,1)="#" )
		StringReplace, $, $, %L%%C%, % Chr(((SubStr(L,3,1)="x") ? "0" : "" ) SubStr(L,3)), All
	Return RegExReplace( $, "(^\s*|\s*$)")            ; Remove leading/trailing white spaces
}

roysubs
Posts: 107
Joined: 29 Sep 2018, 16:37

Re: Google Search function that returns Url of first hit

18 Oct 2018, 09:06

These are *incredible*. Thanks guys, so so useful. Got both approaches working; the problem was my end, I had just stupidly declared the hotkey as ^F1: (single colon instead of double-colon...). Coffe-addled brain didn't see it for a few hours!

One possible extension if you think it doable? After searching for something, I'd like to automatically grab the first x number of hits in the list, and then open a new tab and load each url. Is that easy to achieve with the existing functions?
tmplinshi
Posts: 1278
Joined: 01 Oct 2013, 14:57

Re: Google Search function that returns Url of first hit

18 Oct 2018, 09:42

The gsearch function will return all results from first page, usually 10 results. So to open first 3 urls:

Code: Select all

for idx, this in gsearch("autohotkey")
	Run, % this.url
until (idx = 3)
roysubs
Posts: 107
Joined: 29 Sep 2018, 16:37

Re: Google Search function that returns Url of first hit

18 Oct 2018, 09:47

Unbelievably cool... I'm going to have a lot of uses for this... thanks again tmplinshi! :-)
roysubs
Posts: 107
Joined: 29 Sep 2018, 16:37

Re: Google Search function that returns Url of first hit

21 Oct 2018, 04:51

Just posting this here for anyone that might come across this post as a third option for this sort of thing, could also be useful.
https://www.reddit.com/r/AutoHotkey/com ... utube_and/

Return to “Ask For Help”

Who is online

Users browsing this forum: Bing [Bot], Gh0sTG0, Mipha, Scr1pter, SOTE, wryyymuda and 31 guests