Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate

RFN variable storage and retrieval fast/high capacity

  • Please log in to reply
2 replies to this topic
  • Members
  • 54 posts
  • Last active: Jan 22 2014 04:35 AM
  • Joined: 26 Mar 2012
Here's a set of functions I wrote to store and retrieve large variable sets quickly and accurately.

I have been running into problems storing and dumping large sets of data from ini files, and had a need to do so on multiple machines with encrypted hard discs. This was causing IO issues with read/write times. I have tried ini, csv, local database all with their own set of issues and bugs. So I had this idea that a csv is perfect for this type of IO load, however commas and return lines need to be captured in this data.

So here's what I needed and what this satisfies:
1. write out/dump variable and their contents to a persistent file
2. auto archive these dumps
3. high capacity and low read times
4. Capture any 'normal' text characters including returns and commas etc

The what it does:
The file is written using a 'difficult to type' delimiter " ¦ "
Files are organized as such: VARIABLE_NAME¦VARIABLE_VALUE then simply strung together in the parse
When the file is read back in, variable names are assumed to be of an odd loop number, and the paired value to be of even
When variable are read back in from file, they are set global so as to 'pour' them back into the app

To generate a dump file:
First create a pipe delimited list of the variable to be captured: ie(LIST_OF_VARS=VAR1|VAR2|VAR3|VAR4|VAR5) note: omit any pipe at the end
Call the function: OUTPUT_TO_RFN(LIST_OF_VARS, "My RFN Output File.rfn", A_scriptdir . "\ARCHIVES\")
The "My RFN Output File.rfn" will be generated and if it already exists, the old moved to the specified archive folder and time stamped as a version
To overwrite the files without archive, just call the function without the archive folder parameter

To read the values back in:
Call the function: READ_FROM_RFN("My RFN Output File.rfn")
All variable found in the file will be pushed into the calling script as global variables

Here's the code:

LIST_OF_FIELDS: Pipe delimited list of variables to store
FILE_NAME: File name to store to
ARCHIVE_FOLDER: Name of folder to store archived file if it already exists(optional, archives will not be kept if omited)

RFN_DEL= % chr(5)
CONTAINER= ; init the container
	Loop, parse,LIST_OF_VARS, "|" ; build the field value relationship dynamiclly, store in LEGACY_NOTES_FILE
		CONTAINER.=A_LoopField . RFN_DEL . %A_LoopField% . RFN_DEL
	StringTrimRight, CONTAINER,CONTAINER,1 ; trimm off the last delimiter
	;check for the archive folder
		FileCreateDir, %ARCHIVE_FOLDER%
; check to see if the file exists before write and archive if requested
IfExist, %FILE_NAME%
	If(ARCHIVE_FOLDER) ; only archive if requested
	FileDelete, %FILE_NAME%
FileAppend,%CONTAINER%, %FILE_NAME%,UTF-8 ; UTF-8 seems to work best for this for some reason

FILE_NAME: File name to read back into variables

local a := 1 ; used to push the dynamic global vars below (see: http://www.autohotkey.com/forum/topic19641.html) Thanks Lexikos! 
FileRead, CONTAINER, %FILE_NAME% ;Read the file into memory

	Loop, parse,CONTAINER, % chr(5) ; parse out the variable names and values expected(var names = odd numbers, var values = even numbers)
		;~ msgbox % A_Index A_LoopField
		; odd or even logic
	if Mod(A_Index,2) ; this is an odd number, assumed to be a variable name
			LAST_VAR_NAME:=A_LoopField ; keep the name for value asignment
	else ; this is an even number, assumed to be the value of the previous entry
			%LAST_VAR_NAME%:=A_LoopField ; assign the value back into the variable set global above

And a quick example:

; list of variables to be handled

	Loop, parse,LIST_OF_FIELDS,"|" ; Just putting some stuff in the vars to test with
		%A_LoopField%:="`nPUT SOME STUFF " . A_Index



; Dump these variable to RFN file

; Open a file to read the variable back in
FileSelectFile, FILE_NAME,,%A_ScriptDir%,,(*.RFN)

Edit: changed so archive is optional

I'm using the .RFN extension just because it jives with the project this was written for, it can be anything really

I would expect these functions to run in any version of AHK and AHK_L

This was tested on WINXP 32 SP3, WIN 7 64 ULT, WIN 8 preview(winXbox lol)
AHK version: AHK_L v1.1.07.01
Tested in both VM and hardware machines

If anyone has figured out how to use a null character or something that can not be typed at all for this type of script, please let me know
Modifications are welcome, I just thought I'd share this as it has made my life easier :D

EDIT: Modified to use the chr(5) delimiter
EDIT: Fixed a bug in the file archive switch to allow paths correctly

  • Guests
  • Last active:
  • Joined: --
I regularly use Chr(5) and Chr(7) as newline and delimiter characters so I can dump data on one line and use them like so
StringSplit, Text, A_LoopField, % Chr(5)

; and/or

Loop, Parse, Text, % Chr(7)

  • Members
  • 54 posts
  • Last active: Jan 22 2014 04:35 AM
  • Joined: 26 Mar 2012
That's perfect! Thanks :D