Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[AHK_L/v2] Yaml() - Yaml Parser (++JSON)


  • Please log in to reply
116 replies to this topic
HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008

I finally got the Yaml parser ready for release :)

No idea what Yaml is Learn Yaml in 5 minutes 

Save Function as [url="https://github.com/HotKeyIt/Yaml/raw/master/Yaml.ahk"]Yaml.ahk[/url] - or - [url="https://github.com/HotKeyIt/Yaml/raw/ahkV2/Yaml.ahk"]Yaml.ahk (v2 version)[/url] in your lib folder.

Note! You will need latest AutoHotkey_L or AutoHotkey_H to use these functions.

I have tried to keep Yaml 1.2 format but there are some things that I have changed and some do not work:

- Tab character can be used to intend text as well as 2 spaces.
- Keys and sequence are allowed on same level.
- - Sequence is saved in Object[""]
- - That way we can have digit keys: "1": "value"
- - One drawback is that JSON does not support it, so when Yaml is dumped to JSON, key/value map becomes a sequence object

- !!Type is currently not implemented and ignored


Please report any issues or suggestions.

Enjoy wink.png


Yaml() is used to create an object.
yamlObj:=Yaml(file,isfile:=1) ;isfile is set to 1 by default
yamlObj:=Yaml(YamlText,0) ;so we need to use 0 to load text.
- file = Can be a file path or yaml text, to load text set "isfile" to 0

After Yaml is parsed your object will contain the appropriate sequences and maps.
Sequences and documents are saved in separate object, this object is again saved in obj[""].
Tab instead of 2 spaces to indent text is supported as well.
Also maps and seqences are allowed within same object
yamlObj:=Yaml("Key: abc`n1: cde`n- Item1`n- Item2",0)
MsgBox % yamlObj.Key
MsgBox % yamlObj.1 ;key named 1
MsgBox % yamlObj.(1) ;sequence item (same as yamlObj[""].1)
MsgBox % yamlObj.() ;count items saved in yamlObj[""] (same as yamlObj[""].MaxIndex())

Dump can be used to parse object back to Yaml text.
Yaml text can be returned in JSON syntax, threfore parameter need to be 0 or higher, see example.
MsgBox % yamlObj.Dump(JSON_LEVEL)
;Yaml_Dump can be used to dump other (non Yaml) objects as well
variable:="MyVariable"
aKey:="I'm a key"
Obj:={aKey: variable,"Now is": A_Now,"": ["a","b","c"]}
MsgBox % Yaml_Dump(obj)
MsgBox % Yaml_Dump(obj,0)
Add can be used to append new yaml content to existing object, set second parameter to 1 to append a file
MyYaml:=Yaml("",0) ;create empty Yaml
MyYaml.Add("Test: AHK") ;add new map
MyYaml.Add("- Item2") ;add new sequence
MyYaml.Add("- - Item3") ;add new object and insert new item
MyYaml.(3).Add("- Item4") ;add an item to above created object
MyYaml.Add("Key: [b]") ;you can also overwrite values
MyYaml.Key.Add("- c") ;add new sequence item to Key
MyYaml.Key.Add("[a,h,k]") ;add new item containing object of sequences
MyYaml.Key.Add("{a:1,b:2,c:3}") ;add new item containing object of maps
MsgBox % MyYaml.Dump(1)

Example and more information[/b] - Example uses ObjTree() to present yaml object
Gosub, LoadYaml1
MyYaml:=Yaml(yamlText,0)
ObjTree(MyYaml,"Yaml loaded from variable")
MsgBox % MyYaml.(1).Foreword.() ;count sequence items in Foreword
MsgBox % MyYaml.(1).Foreword.(2) ;show content of first item in Foreword
MyYaml.Add("New Key: AutoHotkey") ;add a key/value pair.
MsgBox % MyYaml.Dump(3) ;dump yaml object to string
ObjTree(Yaml(MyYaml.Dump(1),0),"Yaml loaded from dumped object")
ExitApp

LoadYaml1:
yamlText= ;example yaml data.
(RTrim0
`%YAML 1.2 #version statement is simply ignored
- Foreword:
- Tabs - Though Yaml originally does not support tabs, I decided to implement/accept them in Yaml().

So you can use "Tab" or "2 Spaces" to indent the text.
If more than a Tab or 2 spaces are used,
map or sequence will be shifted to the latest item.

- objects can hold maps and sequences at same time

- Comments - Comments are ignored currently

Special Characters:
Escape chars:
- '"{[{]}\n' # " is allowed inside ' ', here chars can't be escaped and ' inside is not allowed
- "'"\nnew line" # key or value enclosed in " use \ to escape " and new line \n (CRLF is converted to LF)
- |+ # Scalars can be used for sequences as well as for maps
a
b
c

"all \nchars": {[:""' # keys and values can be enclosed in quotes

--- # new document
# this will insert an object in main object
# all elements will belong to it until next --- or ... is encountered

# --- is not different to sequence (- ) and indented context.
# |---
# |- Object: |Object:
# | key: value | key:=value

# these documents are inserted in our main object same as lists
# so yaml.1 will access a document or first sequence item in our document
# if key is digit number (e.g. "1: value"), you will need to use yaml[1 ""] to access it, not yaml.1 !!!

Object: # a simple Key, since there is no value it is an object
Key: Value # this key is saved in above object and is assigned a value
Sequence: # sequence elements can be accessed using digit keys
- Element 1 # yaml.1.Object.Sequence.1
- Element 2
Map: # map elements cal be accessed as usual keys in object
key: value # yaml.1.Object.Map.key
another key: value # yaml.1.Object.Map["another key"]
1: value # yaml.1.Object.Map[1 ""]

Inline Sequence: [a, b, c] # we can create inline sequence or map

Inline Map: { key1 : value # sequences and maps can be split into several lines
, key2 : value2, key3 : value3 # yaml.1.Object["Inline Map"].key2
, key4 : value4}
Sequence of sequences:
- [a, b, c]
- [1, 2, 3, " ", " ", "2"
, "1", "another value
"]
Map of maps:
2: {key:value, 1:test}
map: {key1:value1, 2:AHK}
Sequences and Maps mixed:
- [a, {key:"value"}, b, [c, d, e], f, [g, h]]
- {key:[a , b, c], test:{another:value}}
Test:
- a
- b
- - new object
- same object as above
- e
- line is appended
)
return
Change log

02.05.2014 - Version 1.0.0.14

- Moved downloads to github

- Fixed some errors

29.10.2012 - Version 1.0.0.13
- Fixed a bug with array key combined "- Key:"
- Removed NXT from vars to be cleared.
29.10.2012 - Version 1.0.0.12
- Fixed issue with continuing lines and some variables that were not cleared
28.10.2012 - Version 1.0.0.11
- Fixed an issue with {} and # in quotes
21.10.2012 - Version 1.0.0.10
- Fixed incorrect object creation as pointed out by Lexikos.
- Improved JSON numbers parsing. Now both chars before and after comma need to be digits to be kept together, see this post.
01.04.2012 - Version 1.0.0.9
- Again another fix for multiple indentation
31.03.2012 - Version 1.0.0.8
- Another fix for multiple indentation
30.03.2012 - Version 1.0.0.7
- Multiple indentation will be always reset to previous level + 1
30.03.2012 - Version 1.0.0.6
- Fixed a bug with static variables
20.03.2012 - Version 1.0.0.5
- Another fix to enclose #... in "".
19.03.2012 - Version 1.0.0.4
- Fixed JSON multiline statements
15.03.2012 - Version 1.0.0.2
- Fixed few cases where characters needs to be enclosed in '' or ""
14.03.2012 - Version 1.0.0.2
- Deleted duplicate declarations.
11.03.12 - Version 1.0.0.1
- Fix for empty continuation lines
03.03.2012
- Small fix if last line empty
01.03.2012
- Many fixes have been put in place to have closer support to Yaml syntax.
- Also many v2 fixes have been implemented
23.09.2011
- Added yaml.Save(file,JsonLevel) function.
13.09.2011
- Fixed unicode support (characters conversion)
17.06.2011
- Fixed to determine Unicode characters properly
24.04.2011
- Implemented Yaml Escape characters
21.04.2011
- fixed several issues



HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
I have changed Yaml slightly.
Now sequences are saved in a separate object (CurrentObject[""].Insert(SequenceItem)).

You can use following syntax to get/set sequences:
MyYaml[""].1 ;Sequence item
MyYaml[""].MaxIndex() ;count sequence items
;---- below does same as above ----
MyYaml.(1) ;multiple sequence parameters are supported as well, see below.
MyYaml.()
Now MyYaml.Dump() can dump any key/sequence
MyYaml:=Yaml("[a,[b,c],d]",0)
MsgBox % MyYaml.([color=red]1,2[/color]).Dump()


HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
New Add method.
MyYaml:=Yaml("- Item1`nKey: test",0) ;create new Yaml (empty yaml can be created as well)

MyYaml.Add("Test: AHK") ;add new map

MyYaml.Add("- Item2") ;add new sequence

MyYaml.Add("- - Item3") ;add new object and insert new item

MyYaml.(3).Add("- Item4") ;add an item to above created object

MyYaml.Add("Key: [b]") ;you can also overwrite values

MyYaml.Key.Add("- c") ;add new sequence item to Key

MyYaml.Key.Add("[a,h,k]")  ;add new item containing object of sequences

MyYaml.Key.Add("{a:1,b:2,c:3}")  ;add new item containing object of maps

ObjTree(MyYaml)

MsgBox % MyYaml.Dump(1)

ExitApp
To Add yaml from file, set second parameter to 1
MyYaml.Add("c:\Temp\MyYaml.yaml",[color=red]1[/color])


HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
Fixed Bug
- a: b
  c: d ;was appended to key a rather than being created as new key/value map

EDIT:
Corrected the fix, c: d created a new object which it should not.

- Another fix, "---" was not creating a new sequence item

- Fixed following
- a:
    b: c
    d: e


olfen
  • Members
  • 115 posts
  • Last active: Dec 25 2012 09:48 AM
  • Joined: 04 Jun 2005
No responses on this, so far. It's a pity.
Have not yet tested it, but looks very promising. Thanks.

safariGuest
  • Guests
  • Last active:
  • Joined: --

No responses on this, so far. It's a pity.
Have not yet tested it, but looks very promising. Thanks.

+1
Please continue to develop... Thanks!

HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
Thanks ;)

Fixed single JSON statements

MyYaml:=Yaml("
(Ltrim
	[color=grey][a,b,c]
	{key: value,another key: 123}[/color]
)",0)
MsgBox % MyYaml.(2).key "`n" MyYaml.(1).(1)
Exitapp
EDIT:
Fixed Yaml_Dump()
y:=Yaml("
(
[color=grey]1:
	key: value
	test: value[/color]
)",0)
MsgBox % Y.Dump()
ExitApp



HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008

- fixed several seqence/map errors
- fixed handling and escaping keys and values containing comment char # 'key #': 'value # not a comment' # a comment
- dumping keys and sequence at same level in JSON will result in a sequence object with key value map. :!:

yaml=
(
Object:
  - Item
  key:value
)
y:=Yaml(yaml,0)
MsgBox % y.Dump(1) ;Object: [Item,{key: value}]
y:=Yaml(y.Dump(1),0)
MsgBox % y.Dump()
results_in=
(
Object:
  - Item
  -
   key: value
)



  • Guests
  • Last active:
  • Joined: --
Hi,

Great work; just experimenting with it - the following isn't working for me, would it be possible using this real word data in an example to help my understanding?

HypeM := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HypeM.Open("GET", "http://hypem.com/playlist/popular/3day/json/1/data.js")
HypeM.Send()

Object := YAML(HypeM.ResponseText,0)
MsgBox % Object["version"]

? Currently displays a blank output.

HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
Use ObjTree(YamlObject) to debug.
ResponseText contains a key/value map in JSON syntax, here Yaml creates a sequence and the sequence contains your objects.
In Yaml() sequence is saved in Object[""].
To access it, you must use following.(Index) notation.
Object.(1)["version"] 
;above is generally same as
Object["",2,"version"]
;you can get the object itself as well
Object := (YAML(HypeM.ResponseText,0)).(1)
MsgBox % Object["version"]

EDIT:
I have also uploaded a fix, found when debugging above example.
{"item":{"key":"value"}} ;works fine now


HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008

- Implemented Yaml Escaped characters

EDIT:
- fixed to create empty item or key when - "" or - '' or key: "" or key: '' is encountered, instead of creating empty object.



  • Guests
  • Last active:
  • Joined: --
Thanks for the example; have used it in a real world situation, best stumbled finding ever.

THANK YOU!

hoppfrosch
  • Members
  • 399 posts
  • Last active: Feb 26 2016 05:31 AM
  • Joined: 25 Jan 2006
Two issues: (using your latest modification from 25.04.2011) - Using AHK_L 1.0.97.02 x86 Unicode
[*:1qncatte]Error in your example:
MsgBox % MyYaml.(1).Foreword.(2) ;show content of  first item in Foreword
[color=#ff0000]MsgBox[/color] % MyYaml.Add("New Key: AutoHotkey")
MsgBox % MyYaml.Dump(3) ;dump yaml object to string
[*:1qncatte]Demo does not work here:

Error at line 311 in #include file "X:\AutoHotkey\Lib\Yaml.ahk".

Line Text: {""
Error: The leftmost character above is illegal in an expression.

The program will exit.


--- EDIT: The sign between "" in "Line Text:{""" ...shows up as a bullett (filled circle) on my machine ...

HotKeyIt
  • Moderators
  • 7439 posts
  • Last active: Jun 22 2016 09:14 PM
  • Joined: 18 Jun 2008
Thanks hoppfrosch,

I have changed example.

Can you check if the line (311) in your file looks like this, also make sure you saved in Unicode or UTF-8 format?
static ascii:={"`a": "a","`b": "b","`t": "t","`n": "n","`v": "v","`f": "f","`r": "r",Chr(0x1B): "e","""": """",Chr(0x85): "N",Chr(0x2029): "P",Chr(0x2028): "L","": "0",Chr(0xA0): "_"}


noob1999
  • Members
  • 32 posts
  • Last active: Nov 05 2012 10:12 PM
  • Joined: 14 May 2011
Hi,

Do you have a complete tutorial/documentation like the earlier version of YAML, which is easy to use.

How do I remove keys?
How do I remove sequence values?
How do I save to file?