JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

Post your working scripts, libraries and tools for AHK v1.1 and older
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

15 Nov 2015, 00:27

evilC wrote:Coco, I submitted a pull request to your repo with a small fix that makes it compatible with AHK_H.
AHK_H supports a null type, so I just renamed your null var to _null
See my comment on your PR
SifJar
Posts: 398
Joined: 11 Jan 2016, 17:52

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

25 Jan 2016, 21:45

I fear I may be asking a silly question, but I have what I think is a fairly straightforward task that I think should be easy enough with this library if I could just figure it out. I am trying to (for the moment) obtain a single value from a JSON file, and I've got the following code, which doesn't work:

Code: Select all

#Include JSON.ahk

FileRead, test, test.json
example := JSON.Load(test)
str1 := JSON.Dump(example, testReplacer)
MsgBox % str1

testReplacer(this, key, value){
	if (key == "download_url")
		return value
	return 0 ;tried undefined and null here also
}
With this, str1 contains the full contents of the JSON file. I want only the value with the key "download_url", and I'm very lost as to how to go about doing this. I appreciate any help!

EDIT: Or would this be easier with VxE's JSON_ToObj?
Coco-guest

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

25 Jan 2016, 23:09

SifJar wrote:I fear I may be asking a silly question, but I have what I think is a fairly straightforward task that I think should be easy enough with this library if I could just figure it out. I am trying to (for the moment) obtain a single value from a JSON file, and I've got the following code, which doesn't work:

Code: Select all

#Include JSON.ahk

FileRead, test, test.json
example := JSON.Load(test)
str1 := JSON.Dump(example, testReplacer)
MsgBox % str1

testReplacer(this, key, value){
	if (key == "download_url")
		return value
	return 0 ;tried undefined and null here also
}
With this, str1 contains the full contents of the JSON file. I want only the value with the key "download_url", and I'm very lost as to how to go about doing this. I appreciate any help!

EDIT: Or would this be easier with VxE's JSON_ToObj?
I discovered a bug, but meanwhile, regarding your sample code - you are trying to pass an empty variable testReplacer. The replacer parameter must be a function object so you would need to pass str1 := JSON.Dump(example, Func("testReplacer")). Also, from within the replacer function, you have to take into account the dummy "" key which contains the root object as the value. Example usage:

Code: Select all

json_obj := {foo:"Foo", bar:"Bar", "download_url": "htpp://download_url.com"}
json_str := JSON.Dump(json_obj, Func("TestReplacer"))
MsgBox, %json_str%

TestReplacer(this, key, value)
{
	; if key is blank(""), value is the root object so return it
	if (key == "" || key == "download_url")
		return value
}
Now back to the bug/issue. As you can see in the output of my example, the keys foo & bar were still included. AutoHotkey doesn't have an undefined type and returning blank or 0 doesn't really tell if the caller wants to exclude those keys as blank("") and 0 can be valid values of properties. I was thinking of using a custom undefined object w/c can be accessed via JSON.undefined, so that the caller can do return JSON.undefined if he/she wants the property to be excluded from the output. Let me think about this for a couple of days. For now you can manually filter the object first before passing it to JSON.Dump(), something like: filtered_obj := { "download_url": json_obj["download_url"] }, output := JSON.Dump(filtered_obj).
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Jan 2016, 12:27

Updated (commit 6aaeb3d):
  • Load() - remove static variable null. This should make it AHK_H-compatible(untested)
  • Dump() - improve skipping of non-serializable objects such as COM, Func, BoundFunc, FileObject, etc.
  • Added JSON.Undefined, a proxy for the JavaScript undefined type for use with reviver/replacer functions.
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Jan 2016, 12:49

@SifJar, I've fixed the issue you can do the following:

Code: Select all

#Include JSON.ahk
 
FileRead, test, test.json
example := JSON.Load(test)
str1 := JSON.Dump(example, Func("testReplacer"))
MsgBox % str1
 
testReplacer(this, key, value*) {
	; Initially 'replacer' gets called with an empty key("") representing the object being stringified. Make sure to return the root object as is.
	if (key == "" || key == "download_url")
		return value[1] ; on v1.1, if var contains a number(cached integer), 'return var' stringifies the return value -> 0 becomes "0"
	return JSON.Undefined ; see line 336 of JSON.ahk
}
Last edited by Coco on 30 Jan 2016, 02:51, edited 2 times in total.
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

30 Jan 2016, 02:45

Updated (commit 43cb649) - Fixed a minor issue
User avatar
evilC
Posts: 4823
Joined: 27 Feb 2014, 12:30

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Feb 2016, 12:21

Hi Coco, is it possible to force this lib to explicitly force keys for an array?

Code: Select all

#include <JSON>

obj := {}
obj[1] := "one"
obj[2] := "two"
obj[10] := "ten"

str := JSON.Dump(obj)
Clipboard := str
MsgBox % str
{"1":"one","2":"two","10":"ten"}

but comment out obj[10] := "ten" and you get: ["one","two"]
I want the indexes to always be present. The array should always be sparse.
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Feb 2016, 13:00

@evilC: Yes it's possible, and I think it should for "objects". The current behavior(when dumping) is to first check if object.IsArray is true. If true, regardless of whether if it's a sparse array or not, it will be stringified with square brackets. For sparse arrays, say if the array is [1, 2,, 4], it will be stringified as [1,2,null,4]. IsArray can be force by overriding Array() and setting the property or using a custom base object. JSON.Load() does this during parsing. It first checks if Array() is overridden and if the returned object has the IsArray property and if the value of the property is true. Otherwise, it uses a custom base object(w/ the IsArray property) to create arrays from JSON texts such as ["foo", "bar"]. So if you dump an array previously returned by JSON.Load it will be stringified w/ square brackets. Back to JSON.Dump, if object.IsArray is false, the function falls back to the the previous behavior of identifying arrays by using a for-loop. By removing the fallback behavior(for-loop), the caller can force index keys to be present by simply using {} or Object() when creating the object. And if they want an "array", they should override Array() with something like:

Code: Select all

Array(fields*)
{
	static array_base := {IsArray: true}
	fields.base := array_base
	return fields
}
I decided to retain the fallback behavior, to not force users to override Array(). I am considering removing it or perhaps adding a flag/parameter to control conformity with the spec(ECMAScript).
Coco
Posts: 771
Joined: 29 Sep 2013, 20:37
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

09 Apr 2016, 13:35

So I decided to run a perf/stress test on the parser. I also decided to match it against other JSON-like parser so as to see on how/what I can improve in the parser. The following JSON libs were used: (credit goes to their corresponding authors) The test script contains 2 tests:
  • First Test - Performance - small JSON text, repetitive calls:
    Input JSON: {"foo":"The quick \"brown\" fox","bar":"jumps over the lazy dog.","baz":["Hello\nWorld","AutoHotkey"]}
    Iterations: 1000000
  • Second Test - Stress - Large-ish JSON text(approx. 1MB, 1000000+ chars), single call:
    Input JSON: [{"foo":"The quick \"brown\" fox","bar":"jumps over the lazy dog.","baz":["Hello\nWorld","AutoHotkey"]}, ... ]
    Iterations: single call
Test script: (Put respective libs above in the same dir)

Code: Select all

#NoEnv
#Include JSON_ToObj.ahk
#Include LSON.ahk
#Include Jxon.ahk

SetBatchLines -1
ListLines Off

elapsed := []

; First test(performance), small input
str := "{""foo"":""The quick \""brown\"" fox"",""bar"":""jumps over the lazy dog."",""baz"":[""Hello\nWorld"",""AutoHotkey""]}"
n := 1000000

start := A_TickCount
Loop % n
	JSON_ToObj(str)
elapsed.Push( ((A_TickCount-start)/n)/1000 )

; LSON_Deserialize()
start := A_TickCount
Loop % n
	LSON_Deserialize(str)
elapsed.Push( ((A_TickCount-start)/n)/1000 )

; Jxon_Load()
start := A_TickCount
Loop % n
	Jxon_Load(str)
elapsed.Push( ((A_TickCount-start)/n)/1000 )

; - - - - - - -

; Second test(stress), large input, approx 1MB JSON
i := 12000 ; if each char is 1 byte, 12K iterations should produce approx. 1MB JSON text
text := "["
Loop % i
	text .= str . (A_Index<i ? "," : "]")

; JSON_ToObj()
start := A_TickCount
JSON_ToObj(text)
elapsed.Push( (A_TickCount-start)/1000 )

; LSON_Deserialize()
start := A_TickCount
LSON_Deserialize(text)
elapsed.Push( (A_TickCount-start)/1000 )

; Jxon_Load()
start := A_TickCount
Jxon_Load(text)
elapsed.Push( (A_TickCount-start)/1000 )

; Show results
MsgBox % Format("
( Join`r`n Comment
Test #1:
JSON text length: {1} chars
{2} iterations;

JSON_ToObj():`t{4}
LSON_Deserialize():`t{5}
Jxon_Load():`t{6}

- - - - - - - - - - - - - - - - - - -

Test #2:
JSON text length: {3} chars

JSON_ToObj():`t{7}
LSON_Deserialize():`t{8}
Jxon_Load():`t{9}
)", StrLen(str), n, StrLen(text), elapsed*)

return
Results:

Code: Select all

---------------------------
*
---------------------------
Test #1:

JSON text length: 102 chars

1000000 iterations;



JSON_ToObj():	0.000192

LSON_Deserialize():	0.000310

Jxon_Load():	0.000205



- - - - - - - - - - - - - - - - - - -



Test #2:

JSON text length: 1236001 chars



JSON_ToObj():	478.205000

LSON_Deserialize():	323.546000

Jxon_Load():	3.105000
---------------------------
OK   
---------------------------
As you can see in the results of the first test, the 3 parsers pretty much perform similarly w/ JSON_ToObj being 2-3 times faster than the other two.

The results for the second test are pretty surprising, I'm not really sure why JSON_ToObj & LSON_Deserialize are performing poorly. LSON uses Regex calls and recursive function calls, which might have been the cause of the drop off. However, I'm not really sure about JSON_ToObj. Based on my understanding of the code, it should be using lesser loops than Jxon_Load since it doesn't really read the input per character. I can only assume that perhaps the StringReplace on line: 56(StringReplace, str, A_LoopField, [, [], A) is causing the slowdown. As for Jxon_Load, the input is passed ByRef and is never altered. Perhaps somebody could perform the tests as well. Plus I'm not really sure if I'm doing it correctly.
reallyisbillyc

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

16 Jun 2016, 12:06

hi, hoping someone can help me. ive been looking for a solution for days as I'm a total noob at any type of code. I was wondering
can an horizontal scrollable list of cover art be made using json

what I need is ahk to read contents of a folder and display in an horizontal list gui
json is my latest find , so please excuse me if I'm totally wrong

looking at other json stuff is this what is needed and can one of the zIndex's be used as a variable to call up and display other media with the same name
ie
horizontal list displays box art
upper right would be video
upper left would be an swf or png, all files would have the same name and clicking on zIndex 3 would trigger an event/application

Code: Select all

{
  "transitionTime": 400,
  "selectPosition": 2,
  "hide": false,
  "hideStart": 1500,
  "hideDuration": 1500,
  "type": "horizontal",
  "points": [
    {"x":0, "y":75, "rotation": 0, "scale":1, "zIndex":1},
    {"x":22, "y":75, "rotation": 0, "scale":1, "zIndex":2},
    {"x":44, "y":75, "rotation": 0, "scale":1.1, "zIndex":3},
    {"x":66, "y":75, "rotation": 0, "scale":1, "zIndex":2},
    {"x":88, "y":75, "rotation": 0, "scale":1, "zIndex":1}
  ]
}
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

20 Oct 2016, 05:10

Very nice! The performance has been greatly improved compare to your first version (back to 2013).
Tested on a 540 KB file, the "2013 version" takes 8 seconds, and the current version takes only 0.28 seconds. Almost as fast as the JScript-based parser ParseJson (it takes 0.21 seconds).
User avatar
jNizM
Posts: 3183
Joined: 30 Sep 2013, 01:33
Contact:

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

20 Oct 2016, 05:19

Here is another one.. (but slow because of recursive)

Code: Select all

json(i)
{
    if (RegExMatch(i, "s)^__chr(A|W):(.*)", m))
    {
        VarSetCapacity(b, 4, 0), NumPut(m2, b, 0, "int")
        return StrGet(&b, 1, (m1 = "A") ? "cp28591" : "utf-16")
    }
    if (RegExMatch(i, "s)^__str:((\\""|[^""])*)", m))
    {
        str := m1
        for p, r in {b:"`b", f:"`f", n:"`n", 0:"", r:"`r", t:"`t", v:"`v", "'":"'", """":"""", "/":"/"}
            str := RegExReplace(str, "\\" p, r)
        while (RegExMatch(str, "s)^(.*?)\\x([0-9a-fA-F]{2})(.*)", m))
            str := m1 json("__chrA:0x" m2) m3
        while (RegExMatch(str, "s)^(.*?)\\u([0-9a-fA-F]{4})(.*)", m))
            str := m1 json("__chrW:0x" m2) m3
        while (RegExMatch(str, "s)^(.*?)\\([0-9]{1,3})(.*)", m))
            str := m1 json("__chrA:" m2) m3
        return RegExReplace(str, "\\\\", "\")
    }
    str := [], obj := []
    while (RegExMatch(i, "s)^(.*?[^\\])""((\\""|[^""])*?[^\\]|)""(.*)$", m))
        str.Push(json("__str:" m2)), i := m1 "__str<" str.MaxIndex() ">" m4
    while (RegExMatch(RegExReplace(i, "\s+", ""), "s)^(.*?)(\{|\[)([^\{\[\]\}]*?)(\}|\])(.*)$", m))
    {
        a := (m2="{") ? 0 : 1, c := m3, i := m1 "__obj<" ((obj.MaxIndex() + 1) ? obj.MaxIndex() + 1 : 1) ">" m5, tmp := []
        while (RegExMatch(c, "^(.*?),(.*)$", m))
            tmp.Push(m1), c := m2
        tmp.Push(c), tmp2 := {}, obj.Push(cobj := {})
        for k, v in tmp
        {
            if (RegExMatch(v, "^(.*?):(.*)$", m))
                tmp2[m1] := m2
            else
                tmp2.Push(v)
        }
        for k, v in tmp2
        {
            for x, y in str
                k := RegExReplace(k, "__str<" x ">", y), v := RegExReplace(v, "__str<" x ">", y)
            for x, y in obj
                v := RegExMatch(v, "^__obj<" x ">$") ? y : v
            cobj[k] := v
        }
    }
    return obj[obj.MaxIndex()]
}
[AHK] v2.0.5 | [WIN] 11 Pro (Version 22H2) | [GitHub] Profile
User avatar
evilC
Posts: 4823
Joined: 27 Feb 2014, 12:30

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

25 Oct 2016, 12:50

Hi, I am seeing an issue when deserializing JSON with a quote in the key name

Code: Select all

j := "{""Hello \"" world"": ""Test""}"
obj := JSON.Load(j)
j conatins the string Hello \" World (Exact characters in the string)
I am expecting the object to contain exactly this as a string.

However, what I get is this: Hello "" World which is how you would format a string in AHK if you wished to output it and have it appear as Hello " World, however i want it to actually contain Hello " World, not a string that would render as such.

Basically my use-case is this:
I am storing a regex in an INI file like so:

Code: Select all

[Regexes]
{"^Regex looking for a plus symbol \+": {"color": "Yellow"}
, "^Regex with \"quotes\" in it": {"color": "Yellow"}}
The keys are the regexes to run on each line of the log, the values are the color to highlight the line if the regex matches.

I read the INI section in using IniRead, j, % SettingsFile, Regexes so I end up with the string {"^Regex looking for a plus symbol \+": {"color": "Yellow"}, "^Regex with \"quotes\" in it": {"color": "Yellow"}}.

I am struggling to find a way to represent these values in the INI file, that JSON also likes.

The "plus regex" syntax in the INI file is verbatim as you would like it passed to the regex: ^Regex looking for a plus symbol \+, however the JSON lib does not like this, so I run all json through a RegExMatch to replace one slash with two - so before JSON.Load, I convert the string to ^Regex looking for a plus symbol \\+, and when the object is created, it contains ^Regex looking for a plus symbol \+ as it should.

The "quote regex" syntax in the INI file is slightly artificial - what actually needs to get passed to RegexMatch is a string containing ^Regex with "quotes" in it (As you do not need to escape quotes for a regex) but I can see how JSON might have a problem parsing that.
Also, with my current backslash-escaping code (Which I have to do because of the "plus regex"), what I end up passing to JSON is ^Regex with \\"quotes\\" in it

So... looking for some guidance here. I would like to keep the INI file format as close to what you would actually feed to RegexMatch
Lemming
Posts: 1
Joined: 18 Dec 2016, 02:23

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

18 Dec 2016, 02:57

Hi am a bit lost when it comes to the JSON format. Could someone provide AHK code for extracting info from this JSON file:

Code: Select all

{"documents":[
{"keyPhrases":["car","Hatyai","kg of heroin","drugs","syndicate members eager","failure","early morning","several hours","Malaysian men","intended destination","hours of driving","Dusadee Choosankij","Penang","Chumphon","methamphetamine","border","bars of heroin","crystal","Thai police","officers","life imprisonment","Thai jail","calls","BANGKOK","long hours","Narcotics Suppression Division acting chief","chilling photos","Malaysians","Chumpon","beatings","consequence","friends","southern Thailand","information","rear audio speaker compartment","authorities","telephone numbers","checkpoints","illegal trade","day","duo","unidentified person","week","night duty","Dec","package","mission","effort","passage","route","According","country","detention"],"id":"lemming"}
],"errors":[]}
Specifically, I would like every keyphrase to be in my own AHK pseudo-array, keyphrase1 = car, keyphrase2 = Hatyai and so on.

BTW these are merely keyphrases from a crime news story, so don't be alarmed ;-P It's the output from a text analytics API.
5ball
Posts: 12
Joined: 28 Feb 2016, 10:04

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

23 Jan 2017, 22:21

Lemming wrote:Hi am a bit lost when it comes to the JSON format. Could someone provide AHK code for extracting info from this JSON file:

Code: Select all

{"documents":[
{"keyPhrases":["car","Hatyai","kg of heroin","drugs","syndicate members eager","failure","early morning","several hours","Malaysian men","intended destination","hours of driving","Dusadee Choosankij","Penang","Chumphon","methamphetamine","border","bars of heroin","crystal","Thai police","officers","life imprisonment","Thai jail","calls","BANGKOK","long hours","Narcotics Suppression Division acting chief","chilling photos","Malaysians","Chumpon","beatings","consequence","friends","southern Thailand","information","rear audio speaker compartment","authorities","telephone numbers","checkpoints","illegal trade","day","duo","unidentified person","week","night duty","Dec","package","mission","effort","passage","route","According","country","detention"],"id":"lemming"}
],"errors":[]}
Specifically, I would like every keyphrase to be in my own AHK pseudo-array, keyphrase1 = car, keyphrase2 = Hatyai and so on.

BTW these are merely keyphrases from a crime news story, so don't be alarmed ;-P It's the output from a text analytics API.

Code: Select all

#include json.ahk

fileread, jsonFile, your_json_file.json
myDocument := json.load(jsonFile)
now you can access keyphrases like this

Code: Select all

myDocument.keyPhrases.1 ; will be "car"
myDocument.keyPhrases.2 ; will be "Hatyai"
; etc, etc. 
if you actually need a pseudo-array

Code: Select all

for i, keyPhrase in myDocument.keyPhrases
{
  keyPhrase%a_index% := keyPhrase
}
egocarib
Posts: 100
Joined: 21 May 2015, 18:21

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

15 Oct 2017, 11:32

Noted an issue on github. Jxon_Load throws an execution error for me on AHKv2. Seems like an easy fix.
json

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Jul 2018, 08:58

Json example:

Code: Select all

{
    "something_1":12345,
    "something_2":67890,
    "something_longer":{
        "random_text_17":5678,
        "random_text_37":1234,
    },
}
With this I get the the value from something_1

Code: Select all

myjson := json.load(json_string)
MsgBox % myjson.something_1    ; -> 12345
But how get I the text and value from something_longer? Because random_text_xxx is everytime something different, so I cannot hardcode it

thanks for every help
gregster
Posts: 9014
Joined: 30 Sep 2013, 06:48

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Jul 2018, 09:26

But how get I the text and value from something_longer? Because random_text_xxx is everytime something different, so I cannot hardcode it
Try this (use your json object instead):

Code: Select all

data := { 	   "something_1":12345 
		  ,    "something_2":67890
		  ,    "something_longer"  :  {  "random_text_17":5678	
							           , "random_text_37":1234       } }

MsgBox % data.something_1
for text, number in data.something_longer		; text and number are just variable numes
	msgbox % text " : " number
ExitApp
User avatar
Tigerlily
Posts: 377
Joined: 04 Oct 2018, 22:31

Re: JSON 2.0 (and Jxon) - JSON lib for AutoHotkey

27 Nov 2018, 17:03

Hello Coco,

I'm wondering if you or anyone else would be able to let me know how to access the collections of items found in JSON format? I have code that works, but I cant dive down into individual items.. I'm pretty noob at describing syntax at this point so plz forgive me if I"m using the wrong terms.

here is my working code using your JXON function:

Code: Select all

FileRead, JSONdata, %WBdir%\URLscores.json

readJSON := Jxon_Load(JSONdata) ; load new 

Xl.ActiveSheet.Range("B" URLcell).Value := (readJSON.categories.performance.score * 100)
Xl.ActiveSheet.Range("C" URLcell).Value := (readJSON.categories.pwa.score * 100)
Xl.ActiveSheet.Range("D" URLcell).Value := (readJSON.categories.accessibility.score * 100)
Xl.ActiveSheet.Range("E" URLcell).Value := (readJSON["categories","best-practices","score"] * 100)
Xl.ActiveSheet.Range("F" URLcell).Value := (readJSON.categories.seo.score * 100)

works great, but say i wanted to grab something like this:

Xl.ActiveSheet.Range("F" URLcell).Value := readJSON.categories.performance.details.items[5].url


I get nothing back as a return value. I'm sure it's just my syntax, as I don't quite fully understand your documentation above and have been working off some examples I found around the AHK forums.. can you point me in the right direction?

Also, another issue I found was that if the JSON object had a "-" in the middle of it's name, I had to use your alternate syntax - is there any other way around this? I think it's an AHK thing, but I can't seem to find anything about escaping the "-" character in a variable. I've tried using " ` and {} next to it, but no luck.

Thank you so much for your libraries, they have been mighty helpful!! I'm using it to scrape Google Lighthouse data through the API into an excel spreadsheet, here is the full code (WIP) if you are interested. I'm still working out a few bugs, but has been super efficient using your libs (:

Code: Select all

#SingleInstance, force ;	Doesn't allow the script to run multiple instances at once.
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases. 
;~ #Warn ; Enable warnings to assist with detecting common errors. 
SendMode Input ; Recommended for new scripts due to its superior speed and reliability. 
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory. 
SetTitleMatchMode, 2
SetKeyDelay, 10, 10
DetectHiddenWindows, On
CoordMode, Screen




;[][][][][][][][][][][][][][][][]				CONNECT TO EXCEL				[][][][][][][][][][][][][][][][];



Xl := ComObjCreate("Excel.Application") ; creates Excel handle


;[][][][][][][][][][][][][][][][]				SELECT URL FILE & WRITE DIRECTORY				[][][][][][][][][][][][][][][][];


MsgBox, 0x1030, Select URL file, Please select the excel file that contains the URLs you need Lighthouse Performance scores from.`n`nThe URLs must be listed downwards starting from cell A2.
		
fileToOpen := Xl.GetOpenFilename(FileFilter := "Excel Files [*.xls* or *.csv], *.xls*; *.csv", FilterIndex := 1, Title := "OPEN URL FILE", MultiSelect := False)
	if fileToOpen = 0 
	{
		SoundPlay *-1
		MsgBox, 0x1010, No FIle Selected,No file was selected.`n`nLighthouse Score Scraper will close.
		ExitApp
	}
		else If fileToOpen <> 0
		Xl.Workbooks.Open(FileName := fileToOpen, UpdateLinks := 0)

		WinActivate, ahk_exe EXCEL.EXE
		
	MsgBox, 0x1034, Run in Background?, Would you like to be able to watch the data load into the spreadsheet?`n`nIf yes`, make sure that you don't edit that spreadsheet while the data is loading or URLs may fail to load into it.`n`nIf no`, your Excel file will load data in the background.	
	IfMsgBox Yes
		{
		Xl.Visible := True
		}
		else
			{
				
			}
		

;[][][][][][][][][][][][][][][][]				GRAB OPENED FILE'S DIRECTORY				[][][][][][][][][][][][][][][][];


SplitPath, fileToOpen, WBFileName, WBdir, WBext, WBnameNoExt, WBdrive

saveToOpenPath := SubStr(WBdir, 4)

#ofURLs := Xl.Workbooks(WBnameNoExt).ActiveSheet.Range("A2:A" Xl.ActiveSheet.UsedRange.Rows.Count).Count ; the range of used cells


;[][][][][][][][][][][][][][][][]				SET TEMPLATE IN EXCEL FILE				[][][][][][][][][][][][][][][][];


Xl.ActiveSheet.Range("A1").Value := "URL"
Xl.ActiveSheet.Range("B1").Value := "Performance"
Xl.ActiveSheet.Range("C1").Value := "PWA"
Xl.ActiveSheet.Range("D1").Value := "Accessibility"
Xl.ActiveSheet.Range("E1").Value := "Best Practices"
Xl.ActiveSheet.Range("F1").Value := "SEO"
Xl.ActiveSheet.Range("G1").Value := "Timestamp"

SoundPlay *-1
MsgBox, 0x1030, Pulling Data thru API, Pulling data from Lighthouse API. Please wait..., 3


;[][][][][][][][][][][][][][][][]				LIGHTHOUSE API CALL - WRITE 2 .JSON FILE & PARSE SCORES				[][][][][][][][][][][][][][][][];

MainLoop:
Loop, %#ofURLs%
{
	
	
;/][//][//][//][//][//][//][//][			Reset Score Variable Values				][//][//][//][//][//][//][//][/;


RepeatScrape:	
readJSON.categories.performance.score :=
readJSON.categories.pwa.score :=
readJSON.categories.accessibility.score :=
readJSON["categories","best-practices","score"] :=
readJSON.categories.seo.score :=


;/][//][//][//][//][//][//][//][			Grab Latest URL Value				][//][//][//][//][//][//][//][/;


URLcell := 1 + (A_Index)
URLaddress := Xl.ActiveSheet.Range("A" URLcell).Value


;/][//][//][//][//][//][//][//][				Start / Update Progress Bar			][//][//][//][//][//][//][//][/;


loaded := (((A_Index - 1) / #ofURLs) * 100) 
loader := SubStr(loaded, 1 , 5)

if (URLcell = 2)
{
progressX := 0
progressY := 500
}

WinGetPos , progressX, progressY, , , Lighthouse Score Scraping in Progress... ahk_class AutoHotkey2

Progress, a m2 t w500 x%progressX% y%progressY% c10 fm8 fs7 wm300 ws200, %loader%`%				||			%URLcell% / %#ofURLs%, Currently scraping:`n`n%URLaddress%, Lighthouse Score Scraping in Progress...
	Progress, %loader%
	if (loader = 100)
		{
		SoundPlay *-1
		MsgBox 0x1030, All Finished, Lighthouse SEO and Performance scores have populated your chosen excel sheet.  Want to open it now??
		Progress, Off
		break
		}

;/][//][//][//][//][//][//][//][			Set  CMD Prompt directory to Opened File's Location				][//][//][//][//][//][//][//][/;


Jump2SaveDirectory := "cd\" saveToOpenPath ; go to main directory


;			|||||||||||||||||||||||||||||||||||				Skip Cell if URL Cell is Nonexistant				|||||||||||||||||||||||||||||||||||			 ;


if (!URLaddress)
	Goto, Skip	


;/][//][//][//][//][//][//][//][			Run Command Line to Call Lighthout API & Write to .JSON file				][//][//][//][//][//][//][//][/;


RunWait, %comspec% /k %Jump2SaveDirectory% && lighthouse %URLaddress% --quiet --chrome-flags=""--headless"" --output=json --output-path=.\URLscores.json && exit  ,,Hide ,CMDpid

		;	*************	remove cd\ command and give full output path


;/][//][//][//][//][//][//][//][			Read and Parse JSON Data				][//][//][//][//][//][//][//][/;


FileRead, JSONdata, %WBdir%\URLscores.json

readJSON := Jxon_Load(JSONdata) ; load new 

Xl.ActiveSheet.Range("B" URLcell).Value := (readJSON.categories.performance.score * 100)
Xl.ActiveSheet.Range("C" URLcell).Value := (readJSON.categories.pwa.score * 100)
Xl.ActiveSheet.Range("D" URLcell).Value := (readJSON.categories.accessibility.score * 100)
Xl.ActiveSheet.Range("E" URLcell).Value := (readJSON["categories","best-practices","score"] * 100)
Xl.ActiveSheet.Range("F" URLcell).Value := (readJSON.categories.seo.score * 100)


;			|||||||||||||||||||||||||||||||||||				Embed Timestamp				|||||||||||||||||||||||||||||||||||			 ;


FormatTime, TimeStamp, , MM-dd-yyyy h:mm:ss tt
Xl.ActiveSheet.Range("G" URLcell).Value := TimeStamp


;			|||||||||||||||||||||||||||||||||||				Delete JSON file				|||||||||||||||||||||||||||||||||||			 ;

FileDelete, %WBdir%\URLscores.json


;/][//][//][//][//][//][//][//][			Ask User If They Want to Repeat Scrape (It Failed)				][//][//][//][//][//][//][//][/;




If (!Xl.ActiveSheet.Range("F" URLcell).Value)
	{
		;~ SoundPlay, A_WorkingDir\cow-moo4.wav
		MsgBox, 0x1010, Failed to get URL scores..., The Lighthouse scores for %URLaddress% were unable to be retrieved.`n`nit's possible the web address was invalid.,6
		;~ SoundPlay, A
		;~ IfMsgBox Yes
		;~ {
		;~ GoTo, RepeatScrape
		;~ }
		;~ else
		;~ {
		Xl.ActiveSheet.Range("B" URLcell).Value := "Unable"
		Xl.ActiveSheet.Range("C" URLcell).Value := "To"
		Xl.ActiveSheet.Range("D" URLcell).Value := "Access"
		Xl.ActiveSheet.Range("E" URLcell).Value := "Website"
		Xl.ActiveSheet.Range("F" URLcell).Value := "Data"
		;~ }
		Process, Priority, %CMDpid%, High
		Process, Close, %CMDpid%	
		
		
		;			|||||||||||||||||||||||||||||||||||				Save Workbook				|||||||||||||||||||||||||||||||||||			 ;
		
		
		
Xl.Workbooks(WBnameNoExt).Save


;			|||||||||||||||||||||||||||||||||||				Get Progress Window's Latest Location				|||||||||||||||||||||||||||||||||||			 ;


WinGetPos , progressX, progressY, , , Lighthouse Score Scraping in Progress... ahk_class AutoHotkey2
}

Skip:
}



;/][//][//][//][//][//][//][//][			Ask User If They Want to Open Their New Report				][//][//][//][//][//][//][//][/;



	;~ SoundPlay, A_WorkingDir\drumroll3.wav
MsgBox, 0x1024, Open Report? ,Your file has successfully saved!`n`nWould you like to open your report?
IfMsgBox Yes
	{
	Xl.Visible := True
		Process, Priority, %CMDpid%, High
		Process, Close, %CMDpid%	
	ExitApp
	}
else
	{
		try
		{
	Xl.Workbooks(WBnameNoExt).Close
			}
			catch e
			{
		
			}
					try
		{
	Xl.Quit
			}
			catch e
			{
		
			}
				
				Process, Priority, %CMDpid%, High
				Process, Close, %CMDpid%
	ExitApp
	}





End::

Xl.DisplayAlerts := False
		try
		{
	Xl.Workbooks(WBnameNoExt).Save
	Xl.Workbooks(WBnameNoExt).Close
			}
			catch e
			{
		
			}
			Xl.DisplayAlerts := True
		try
		{
	Xl.Quit
			}
			catch e
			{
		
			}
	FileDelete, %WBdir%\URLscores.json
	Process, Priority, %CMDpid%, High
	Process, Close, %CMDpid%
MsgBox Lighthouse Score Scraper Closed. The Excel file was saved.
ExitApp
-TL

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: No registered users and 157 guests