Get a value from an associative matrix

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Get a value from an associative matrix

13 Jan 2018, 05:49

Why will this give a result ProdInfo["00053","Head1"] and this doesn't give anything ProdInfo.00053.Head1 ?
(even this work ProdInfo["00053"]["Head1"])
Is there any other way to get values from matrix fields?
(Had hoped to avoid to use quotes signs)

I have made this testcode

Code: Select all

#SingleInstance	force
#NoEnv

Array_X1 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:35, Supplier:"Supp" }
Array_X2 := { ArtNo:"001589-045-00", Head1:"Info1a", Head2:"Info2a", Count:128, Supplier:"Supp1" }
Array_X3 := { ArtNo:"GT101264-1", Head1:"Info1b", Head2:"Info2b", Count:0, Supplier:"Supp2" }
Array_X4 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:15, Supplier:"Supp" }
Array_X5 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:10, Supplier:"Supp" }
Array_X6 := { ArtNo:"939431-1", Head1:"Info1c", Head2:"Info2c", Count:17, Supplier:"Supp" }

ProdInfo := { "00053":Array_X1, "001589-045-00":Array_X2, "GT101264-1":Array_X3, "939431-1":Array_X4 }

MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "1. .: " Array_X1.ArtNo "`n2. .: " ProdInfo["00053","Head1"]  "`n3. .: "  ProdInfo.00053.Head1

; Check all fields (6x6) in the multiarray
For Every, SubArray in ProdInfo
	For Each, Item in SubArray
		MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, SubArray: %Every%`nValue [%Each%]: %Item%
The result from the first output .:
MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "1. .: " Array_X1.ArtNo "`n2. .: " ProdInfo["00053","Head1"] "`n3. .: " ProdInfo.00053.Head1
was .:
1. .: 00053
2. .: Info1
3. .:
after that, all fields are shown one by one.
SirRFI
Posts: 404
Joined: 25 Nov 2015, 16:52

Re: Get a value from an associative matrix

13 Jan 2018, 10:01

Albireo wrote:and this doesn't give anything ProdInfo.00053.Head1 ?
Presumably because it doesn't fulfil variable naming requirements. Additionally, this syntax is for objects/classes, with preset property and method names, not dynamic.

Code: Select all

111 := "222"
MsgBox % 111 ; prints "111", because it's not valid var name

MyArray :=
(Join QC
{
	"123": "qwe",
	"32b": "zxc",
	"abc": "asd"
}
)

MsgBox % MyArray.123 ",`n" MyArray.32b ",`n" MyArray.abc ; first row is empty
Use

Code: Select all

[/c] forum tag to share your code.
Click on [b]✔[/b] ([b][i]Accept this answer[/i][/b]) on top-right part of the post if it has answered your question / solved your problem.
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

13 Jan 2018, 11:31

[b]Objects -> Keys[/b] wrote:Quoted literal strings are considered purely non-numeric in v1.x, so x[1] and x["1"] are not equivalent. Additionally, if a quoted literal string is concatenated with another value (as in "0x" x), the result is treated as purely non-numeric. However, this does not apply to variables, so x[1] and x[y:="1"] are equivalent. This issue will be resolved in AutoHotkey v2, so scripts should avoid using quoted numeric literals as keys.

Code: Select all

ProdInfo := { "00053":Array_X1, "001589-045-00":Array_X2, "GT101264-1":Array_X3, "939431-1":Array_X4 } ;  "00053" is a string key
ProdInfo.00053.Head1 ; 00053 is pure numeric and treated as an integer key -> no match
ProdInfo["00053","Head1"] ; "00053" is a string -> match
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

13 Jan 2018, 13:03

Thanks!
I understand (but still not).
My wish is to be able to read a value by entering an article number + a heading, and it seems to work in this example.

Code: Select all

Array_X1 := { ArtNo:"00053", Head1:"Info1", Count:35, Supplier:"Supp" }
Array_X2 := { ArtNo:"GT101", Head1:"GTinfo", Count:0, Supplier:"Supp1" }
ProdInfo := { "00053":Array_X1, "GT101":Array_X2 }
MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "1. .: " ProdInfo["00053","Count"]
The result: 35 (Correct!)

But If I try this

Code: Select all

Array_X1 := { ArtNo:"00053", Head1:"Info1", Count:35, Supplier:"Supp" }
Array_X2 := { ArtNo:"GT101", Head1:"GTinfo", Count:0, Supplier:"Supp1" }

KeyVar = "00053","Count"
KeyVar1 = 00053
KeyVar2 = Supplier
MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "1. .: " ProdInfo[(KeyVar)] "`n2. .: " ProdInfo["(KeyVar1)",(KeyVar2)]
I got no result. (is it impossible to solve?).

Can there be other issues with using "00053" as object [KEY]?
A_AhkUser
Posts: 1147
Joined: 06 Mar 2017, 16:18
Location: France
Contact:

Re: Get a value from an associative matrix

13 Jan 2018, 13:59

Hi Albireo,

For example:

Code: Select all

p := {z:1, k:"blabla"}
o := {a:{test:7}, "9999":p}
v := ["a", "test"]
MsgBox % o[v*]
u := "9999"
MsgBox % o["" . u].z
x := "k"
MsgBox % o["" . u, (x)]
Regarding to the explanation on how works the following:

Code: Select all

MsgBox % o[v*]
see variadic function call - otherwise, just me has already quoted the relevant part of the documentation.

Hope this helps.
my scripts
SirRFI
Posts: 404
Joined: 25 Nov 2015, 16:52

Re: Get a value from an associative matrix

13 Jan 2018, 14:41

Can you elaborate your actual goal? If you don't know how to describe it - show input and desired output. I recall your name and these fields - contributed towards that at least twice, at that was a while ago.

Also, you may take a look at following:

Code: Select all

MyArray :=
(Join QC
[
	{  ArtNo: "00053",          Head1: "Info1",   Head2: "Info2",   Count: 35,   Supplier: "Supp"   },
	{  ArtNo: "001589-045-00",  Head1: "Info1a",  Head2: "Info2a",  Count: 128,  Supplier: "Supp1"  },
	{  ArtNo: "GT101264-1",     Head1: "Info1b",  Head2: "Info2b",  Count: 0,    Supplier: "Supp2"  },
	{  ArtNo: "00053",          Head1: "Info1",   Head2: "Info2",   Count: 15,   Supplier: "Supp"   },
	{  ArtNo: "00053",          Head1: "Info1",   Head2: "Info2",   Count: 10,   Supplier: "Supp"   },
	{  ArtNo: "939431-1",       Head1: "Info1c",  Head2: "Info2c",  Count: 17,   Supplier: "Supp"   }
]
)

Result := ""
for index, content in MyArray
	Result .= index ":`n`tArtNo: " content.ArtNo "`n`tHead1: " content.Head1 "`n`tHead2: " content.Head2 "`n`tCount: " content.Count "`n`tSupplier: " content.Supplier "`n`n"
MsgBox % Result
Result:

Code: Select all

1:
	ArtNo: 00053
	Head1: Info1
	Head2: Info2
	Count: 35
	Supplier: Supp

2:
	ArtNo: 001589-045-00
	Head1: Info1a
	Head2: Info2a
	Count: 128
	Supplier: Supp1

3:
	ArtNo: GT101264-1
	Head1: Info1b
	Head2: Info2b
	Count: 0
	Supplier: Supp2

4:
	ArtNo: 00053
	Head1: Info1
	Head2: Info2
	Count: 15
	Supplier: Supp

5:
	ArtNo: 00053
	Head1: Info1
	Head2: Info2
	Count: 10
	Supplier: Supp

6:
	ArtNo: 939431-1
	Head1: Info1c
	Head2: Info2c
	Count: 17
	Supplier: Supp
Use

Code: Select all

[/c] forum tag to share your code.
Click on [b]✔[/b] ([b][i]Accept this answer[/i][/b]) on top-right part of the post if it has answered your question / solved your problem.
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

14 Jan 2018, 08:44

Thank you for your time!
(It was the solution of Complicated search in arrays, that gave me inspiration to develop this desire)
But right now I've got stuck.

My first wish is to create a general function that reads a CSV file and creates a multi array from the content.
Something like this .: (if it´s possible in that way)

Code: Select all

ProdInfo := ReadFile("Test.csv","Count")
MsgBox % ProdInfo.ArtNo.ColumnHead	;  ProdInfo.00053.Count

ExitApp

ReadFile(FileName,CountField)
{ ...
...
Return %Array%
}
Suppose there is an article register with the following content .: (it's more easy to have the test values in the testfile)

Code: Select all

InputData = 
(LTrim Join
	"ArtNo";"Head1";"Head2";"Head3";"Count";"Supplier"`n
	"00053";"Info1";"Info2";"Info3";"35";"Supp"`n
	"001589-045-00";"Info1a";"Info2a";"Info3a";"128";"Supp1"`n
	`n
	"GT101264-1";"Info1b";"Info2b";"Info3b";"0";"Supp2"`n
	"00053";"Info1";"Info2";"Info3";"15";"Supp"`n`n
	"00053";"Info1";"Info2";"Info3";"10";"Supp"`n
	"939431-1";"Info1c";"Info2c";"Info3c";"17";"Supp"`n
	;;;;;`n
)
(It have some problems as .:
- Empty fields (last row) - Ignore
- Empty rows (in the middle above) - Ignore
- Many lines of same article number (00053) - Make one field and add the field "Count"
If the field "Count" does not exist - No field will be calculated
)

The next wish is to, easy get values from the matrix
The difficult right now(as I can see) - is how to handle the call.
There is no problem to add "_" (underline) before the article number, but I do not want it in any call like this.

Code: Select all

; I don't like this underscore.
Value := ProdInfo._00053.Count

; This is OK and I got it to work
Value := ProdInfo[00053][Count]

; But I want to use variables like this (doesn't work)
Value1 = 0053
Value2 = Count
Value := ProdInfo[%Value1%][%Value2%]	; or
Value := ProdInfo.%Value1%.%Value2%

; Maybe the only possible way to solve this can be something like this..
Value := ArtSearch("ProdInfo",Value1,Value2)

ArtSearch(Arr,v1,v2)
...
Return %Value%

; Or a specific handle for each Array
Value := ProdInfo(Value1,Value2)
ProdInfo(v1,v2)
---
Return %Value%
It was this structure I had planned for .:

Code: Select all

Array_X1 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:35, Supplier:"Supp" }
Array_X2 := { ArtNo:"001589-045-00", Head1:"Info1a", Head2:"Info2a", Count:128, Supplier:"Supp1" }
Array_X3 := { ArtNo:"GT101264-1", Head1:"Info1b", Head2:"Info2b", Count:0, Supplier:"Supp2" }
Array_X4 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:15, Supplier:"Supp" }
Array_X5 := { ArtNo:"00053", Head1:"Info1", Head2:"Info2", Count:10, Supplier:"Supp" }
Array_X6 := { ArtNo:"939431-1", Head1:"Info1c", Head2:"Info2c", Count:17, Supplier:"Supp" }

ProdInfo := { "00053":Array_X1, "001589-045-00":Array_X2, "GT101264-1":Array_X3, "939431-1":Array_X4 }

MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "1. .: " ProdInfo["00053"]["Count"]
But it did not work with [KEY] as a variable.
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

14 Jan 2018, 08:46

This thread seems to be related to https://autohotkey.com/boards/viewtopic.php?f=5&t=42564 and https://autohotkey.com/boards/viewtopic.php?f=5&t=42598. Why do you start new threads without providing the background information?

If you create an array from a record like

Code: Select all

	"00053";"Info1";"Info2";"Info3";"35";"Supp"`n
using something like

Code: Select all

StringSplit Fields, A_LoopField, `;, "
which removes the double-quotes enclosing the fields Fields1 (ArtNo) contains a pure numeric value 00053 which will create an integer key if Fields1 is used as a key variable for an associative array, e.g.

Code: Select all

Array[Fields1] := ...
This key can be accessed using

Code: Select all

Array.53
; or
Array.0000053
; or
Var := "00053"
Array[Var]
; or
Var := 53
Array[Var]
but not with

Code: Select all

Array["00053"]
because this would require an existing string key.

See the quote in my previous post.
  • Array[1] is not equivalent to Array["1"] - integer key vs. string key
  • Array.1 is always equivalent toArray[1] - both try to access an integer key
Edit: A bit too late!
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

14 Jan 2018, 15:49

just me wrote:......Why do you start new threads without providing the background information?....
I've previously solved the problems with Pseudo-Arrays and it has worked.
But now I had a problem, that could mean 100,000s of comparisons for each search, and it contained 1000s of searches....
It not feel right method to use Pseudo arrays, and I asked the first question (about associative arrays)
Complicated search
The proposed solution gave me inspiration to learn more about associative arrays.

Started to immerse myself in this and realized soon that it was possible to define and retrieve the values in some different ways. (like the HARSH tables I learned many years ago). Associated arrays appeared to be very powerful.

Started thinking if it was possible to make a complete solution to my problem and asked a new question. (I was sure, but not how to do)
Add the contents of a field and sort the result
It gave me no answers. (I understand - who is willing to write an complete program as answer)

Then I started writing a solution, to solve the problems, and it continued until I got stuck. I send a specific question about it.
The Array [KEY] as a variable - How?
Now I understand I was wrong to use about numeric- / string- value in an associative array key.

I want to try to solve my problem - but I do not know how.
And asked this question...
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

15 Jan 2018, 04:26

The structure and the number of needed arrays/objects depend on how you want to use the information. Are you asking for a solution for Add the contents of a field and sort the result? Or shall the same arrays/objects be used for Complicated search too?

Also, if the file contains multiple records for the same article number, are all fields except Count always identical?
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

15 Jan 2018, 11:42

Want to use the structure in many ways. I can try to explain an real example.
Every day a CSV-file is created with about 25.000 rows - In a few years it may contain 40-50000 Products.
Each product has about 70 fields of information.
The first field (art_id) is a unique field.
The structure looks like this .:
"art_id";"art_ean";"art_ean2";"Barcode3";"Barcode4";"su_art_id";"art_main_name";"art_info1";"art_info2";"art_purch_price";"art_sales_price_in";"art_grpno";"art_suno"
"000030-333-00";"7350030498777";"7326211165165";;;"567";"HISSGARDINSTÅNG";"KOLLA PRIS VID ";"HEMKOMST";28;79;1000;"FONDACO"
"00013-14";"7391487013298";"5475165165165";;;"00013.14";"JULKLAPPSPAPPER";"2 METER X 70 CM";"SORTERADE MÖNSTER";"4.5";5;7005;"HEDLUNDS"
"00014.13";"7391487014448";;;;"00014.13";"JULKLAPPSPAPPER";"4 METER X 1 METER";"SORTERADE MÖNSTER";"11.93";15;7005;"HEDLUNDS"
"00053";"7391487053386";;;;"00053";"PRESENTPAPPER";"1,5 METER X 70 CM";"METALLIC - SORTERAT";6;10;6001;"HEDLUNDS"
"GT10025";"7392248100257";;"001589-045-71";;10025;"HÄNGAMPEL";"20 X 20 X 20 CM";"VIT - ROBERT";59;149;3160;"GILBERT"
"29647";"7330010316352";;;;29647;"HATTHYLLA";"78 X 34 X 32 CM";"EK/STÅL";"85.85";129;5000;"EAST"
"BG29647";"7391691053387";;;7340073817369;29647;"DOFTLJUS I GLASLYKTA";"INICIO - HÖSTPÄRON";"BRINNTID CA 25 TIMMAR";"5.75";10;3115;"BESON"
First, I want to create an associative array with all information. (We can Call this multiarray .: ProdInfo)

Then I have another file with two fields. (no fields are unique) and many thousands of rows. For example
ArtNo;Counts
"00013-14";35
"00013-14";5
"7392248100257";14
"29647";6
Now it is possible to proceed in 2 ways
_________________________

1a) Create an unique list of the second Array, and sum the values in the second column for each unique values, ie. 00013-14 35 + 5 = 40
A table (array=Data) like this is created (maybe sorted) .:
ArtNo;Counts
"00013-14";40
"7392248100257";14
"29647";6
1b) Now it's time to make a "complicated search" and retrieve values from the Array ProdInfo above.
- If the value 00013-14 is in the column "art_id" - Great the Product is found.
- if the value 7392248100257 is not in the first column, search in columns "art_ean", "art_Ean2", "Barcode3", "BarCode4" - Great the Product is found in column "art_ean" (each of these columns is unique)
- The result may be a product previously found, and the number in the second field can be added to this product. ( I only want one of each Product)
- If the value is only in the column "su art_id", there may be many Products with the same value. (a problem that has to be handled)

1c) Now it's possible to create a result with information from both arrays.
_______________________________

2a.) Directly make an complicated search
2b.) If the result give a previosly existing Product - add the Count
2c. ) Create a result with information from both arrays.
________________________________

This is just an example I want to use associative arrays with
This code creates an associative array.

Code: Select all

#SingleInstance	force	; Endast en instans av Programmet är igång.
#NoEnv 					; För prestanda och kompatibilitet med framtida AutoHotkey, Rekommenderas.

InputData = 
(LTrim Join
	"ArtNo";"Head1";"Head2";"Head3";"Count";"Supplier"`n
	"00053";"Info1";"Info2";"Info3";"35";"Supp"`n
	"001589-045-00";"Info1a";"Info2a";"Info3a";"128";"Supp1"`n
	"GT101264-1";"Info1b";"Info2b";"Info3b";"0";"Supp2"`n
	"00053";"Info1";"Info2";"Info3";"15";"Supp"`n`n
	"00053";"Info1";"Info2";"Info3";"10";"Supp"`n
	"939431-1";"Info1c";"Info2c";"Info3c";"17";"Supp"`n
	;;;;;`n
)


ProdInfo := {}	; Create the result array

; Create an array for each product
AddHead = Count	; Specifies the field name, containing the "Counts"
; AddHead = 	; No field to count
Loop Parse, InputData, `n	; Loop Parse, InputData, `n, `"
{	ArtCount += 1
	StringSplit Row_%ArtCount%_, A_LoopField, `;, `"
	HeadCount := Row_%ArtCount%_0	; Remember the number of headers
	
	If ((StrLen(A_LoopField)+1 = Row_%ArtCount%_0) or (Row_%ArtCount%_0 = 0))	; Skip if all field is empty (;;;;;) OR empty lines ()
	{	ArtCount -= 1
		Continue
	}
	
	If (A_Index = 1)	; Handle the Header line
	{	Loop % Row_%ArtCount%_0
		{	Header_%A_Index% := Row_%ArtCount%_%A_Index%
			If (Row_%ArtCount%_%A_Index% = AddHead)
			{	CountFieldNo := A_Index	; Remember which field to add "Count"
				CountFieldExist = "Yes"
			}
		}
		; MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "AddHead .: " AddHead "`nRow_%ArtCount%_0 = " Row_%ArtCount%_0 "`nCountFieldNo = " CountFieldNo "`nCountFieldExist = " CountFieldExist
		; If the header not exist and a counted field is desired
		If !CountFieldExist and AddHead
		{	MsgBox 16, Row.: %A_LineNumber% -> %A_ScriptName%, The CountField "%AddHead%" doesn't exist and can't be handled! `nThis program must stop.
			MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, The program ends!, 1
			ExitApp
		}
		ArtCount -= 1	; Don't count the header line
		Continue
	}
	Row_%ArtCount%_1 := "_" Row_%ArtCount%_1	; Add underline before the first field ("00053" => "_00053")
	
	; Check if the first field already exist
	Loop %ArtCount%
	{	 ; Compare with previous ArtNo - Does the first field, exist earlier?
		If (A_Index < ArtCount) and (Row_%ArtCount%_1 = Row_%A_Index%_1)	; YES - Add the selected field
		{	If !AddHead	; No field to calculate
			{	MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "The Product ID .: ( " Row_%ArtCount%_1 " ) already exists! `nHow should it be handled?"
				; MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % Array_X%A_Index%.ArtNo "`n" Array_X%A_Index%.Count "`n + " Row_%ArtCount%_%CountFieldNo%
			}
			else
				Array_X%A_Index%[(AddHead)] += Row_%ArtCount%_%CountFieldNo%	; Calculate the field "AddHead"
			ArtCount -= 1	; Adjust the pointer
			Break
		}	
		
		; First field doesn't erlier exist - Create a "New array" and add that to the "Result array"
		If (A_Index = ArtCount)
		{	Loop % Row_%ArtCount%_0
			{	If A_Index = 1
				{	Array_X%ArtCount% := { (Header_%A_Index%) : Row_%ArtCount%_1 }	; Create a new array Array_X?? with first KEY and value {"ArtNo":"_00053"}
					ObjRawSet( ProdInfo, Row_%ArtCount%_1, Array_X%ArtCount% )	; Create ProdInfo := { "00053":Array_X1, "001589-045-00":Array_X2, .... }
				}
				else
					ObjRawSet( Array_X%ArtCount%, Header_%A_Index%, Row_%ArtCount%_%A_Index% )	; Add a new KEY and a Value to an existing Array
			}
		}
	}
}

; Show the result
ShowArray(ProdInfo, A_LineNumber)
MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % ProdInfo._00053.Count


ExitApp


ShowArray(ArrayName, JumpLine)
{	For i, Value in ArrayName	; For "Key", "Value" i SimpleArray
		List .= "Loop .: " A_Index "`tKey . .: " i "`nValue . . . . . . . . . . . . . . . . .: " Value "`n`n"
	Text := List
	; Text := "Show the keys .: `n`n" List
	; FileAppend %Text%, ArtRegKey.txt
	MsgBox 64, Row.: %A_LineNumber% -> %A_ScriptName%, % "Jump from Line .: " JumpLine  "`n`n" Text
}
The next step is to create a function of the arraycreate and read a csv file
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

16 Jan 2018, 03:19

Well, so actually two files have to be processed.
Every day a CSV-file is created with about 25.000 rows - In a few years it may contain 40-50000 Products. ... The first field (art_id) is a unique field.
That means there is only one record per art_id in this file?
- if the value 7392248100257 is not in the first column, search in columns "art_ean", "art_Ean2", "Barcode3", "BarCode4" - Great the Product is found in column "art_ean" (each of these columns is unique)
That means that each value in each of the columns can occur only once in the file?
2a.) Directly make an complicated search
2b.) If the result give a previosly existing Product - add the Count
2c. ) Create a result with information from both arrays.
What shall 'the result' contain? To which field of the first file shall be added?
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

16 Jan 2018, 07:45

just me wrote:Well, so actually two files have to be processed.
Yes, in this example, but there are other challenges when 3-4 files is needed to create a result file.
just me wrote:
Every day a CSV-file is created with about 25.000 rows - In a few years it may contain 40-50000 Products. ... The first field (art_id) is a unique field.
That means there is only one record per art_id in this file?
Yes! "art_id" is our product identification.
just me wrote:
- if the value 7392248100257 is not in the first column, search in columns "art_ean", "art_Ean2", "Barcode3", "BarCode4" - Great the Product is found in column "art_ean" (each of these columns is unique)
That means that each value in each of the columns can occur only once in the file?
Yes!
The five fields .: "art_id", "art_ean", "art_Ean2", "Barcode3", "BarCode4" in the example are unique fields between the records,
but a product may have the same content in several of these fields (typically, it may be "art_id" and "art_ean" that have the same content)
This is clear, when the value is found, the information for this record can be used.
But sometimes (not often) the value can not be found in these five fields. The value is only in the sixth field "su_art_id".
This field contains the supplier's article number and is not a unique field. Different supplier may have the same article number.
If there are several products with the same article number from different suppliers, the user / administrator must manually select the product referred to.
(how to handle this can always be discussed - at the moment, I think of a log file with information about all these Products, is enough
- and if the log file exist, it can be opened before this program ends. A message just when it occurs may also work.)
The important thing is that products from all suppliers with this article number are displayed.
(If only one supplier have this article number - no problem to handle)
just me wrote:
2a.) Directly make an complicated search
2b.) If the result give a previosly existing Product - add the Count
2c. ) Create a result with information from both arrays.
What shall 'the result' contain? To which field of the first file shall be added?
It depends on what the results are to be used for.
Is it an order, some fields are required and is it an inventory, other fields are interesting.
This example is most similar to an inventory, where a file contains article numbers (any of the 6 fields) and the number of products.
The second file contains the entire article registry with information about all products.
In this case, I would create a CSV result file with the following content .: "art_id";"art_ean";"Counts";"art_main_name";"art_info1";"art_info2";"art_purch_price";"art_grpno";"art_suno".
"ArtNo" and "Count" from the first file (ArtNo can be any of the six fields) but in the result, only "art_id" appears for this particular product.
Instead of only creating a CSV-file, I will create an .ods-file for LibreOffice Calc - but it is a later challenge. (CSV-file is easy to open in LO Calc - but with no formats)
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

16 Jan 2018, 10:55

OK, as far as I understand the current information, I 'd suggest the following for the 'entire article registry' file. At least, it should make your 'complicated search' less complicated. Please try the script using a real production file and see if the load time and the memory usage is reasonable for you:

Code: Select all

#NoEnv
/*
Sample file:
"art_id";"art_ean";"art_ean2";"Barcode3";"Barcode4";"su_art_id";"art_main_name";"art_info1";"art_info2";"art_purch_price";"art_sales_price_in";"art_grpno";"art_suno"
"000030-333-00";"7350030498777";"7326211165165";;;"567";"HISSGARDINSTÅNG";"KOLLA PRIS VID ";"HEMKOMST";28;79;1000;"FONDACO"
"00013-14";"7391487013298";"5475165165165";;;"00013.14";"JULKLAPPSPAPPER";"2 METER X 70 CM";"SORTERADE MÖNSTER";"4.5";5;7005;"HEDLUNDS"
"00014.13";"7391487014448";;;;"00014.13";"JULKLAPPSPAPPER";"4 METER X 1 METER";"SORTERADE MÖNSTER";"11.93";15;7005;"HEDLUNDS"

"00053";"7391487053386";;;;"00053";"PRESENTPAPPER";"1,5 METER X 70 CM";"METALLIC - SORTERAT";6;10;6001;"HEDLUNDS"
"GT10025";"7392248100257";;"001589-045-71";;10025;"HÄNGAMPEL";"20 X 20 X 20 CM";"VIT - ROBERT";59;149;3160;"GILBERT"
"29647";"7330010316352";;;;29647;"HATTHYLLA";"78 X 34 X 32 CM";"EK/STÅL";"85.85";129;5000;"EAST"
;;;;;;;;;;;;
"BG29647";"7391691053387";;;7340073817369;29647;"DOFTLJUS I GLASLYKTA";"INICIO - HÖSTPÄRON";"BRINNTID CA 25 TIMMAR";"5.75";10;3115;"BESON"
*/

InventoryPath := "Albireo1.csv"
FieldCount := 13  ; number of fields in the CSV records
; Arrays
Rec_Array := []   ; simple array to hold the file records as arrays
PID_Array := []   ; primary ID array (art_id) -> record index in Rec_Array
SID_Array := []   ; secondary ID array (art_ean | art_ean2 | Barcode3 | Barcode4) -> record index in Rec_Array
Sup_Array := []   ; array of supplier IDs (su_art_id) -> array of record indices in Rec_Array
RecCount := 0

If !(FileHandle := FileOpen(InventoryPath, "r"))
   ErrorExit("Could not open " . InventoryPath)
Progress, B zh0 fs18, Loading files ...
S := A_TickCount
For RecordIndex, Record In StrSplit(FileHandle.Read(), "`n", "`r") {
   Fields := StrSplit(Record, ";", """")
   If (A_Index = 1) { ; check the header
      If (Fields.Length() <> FieldCount) || (Fields[1] <> "art_id")
         ErrorExit("Wrong file header:`n" . Record)
      Header := Fields
      Continue
   }
   If (Fields.Length() <> FieldCount) || (Fields[1] = "") ; skip invalid records
      Continue
   RecIndex := Rec_Array.Push(Fields)
   PID_Array[Fields[1] . ""] := RecIndex
   Loop, 4 {
      If (SID := Fields[A_Index + 1]) <> ""
         SID_Array[SID . ""] := RecIndex
   }
   SuID := Fields[6]
   If !Sup_Array.HasKey(SuID . "")
      Sup_Array[SuID . ""] := []
   Sup_Array[SuID . ""].Push(RecIndex)
}
FileHandle.Close()
RecCount := Rec_Array.Length()
T := A_TickCount - S
Progress, Off

MsgBox, 0, Load Files, %InventoryPath% has been loaded in %T% ms.

MsgBox, 0, Number of valid input records, %RecCount%

ArtID := "00053"
Msg := ""
Record := Rec_Array[PID_Array[ArtID . ""]]
For FieldIndex, FieldName In Header
   Msg .= FieldName . " = " . Record[FieldIndex] . "`r`n"
MsgBox, 0, %ArtID%, %Msg%

Pause
ExitApp

ErrorExit(Msg) {
   MsgBox, 16, Error!, %Msg%`n`nThe program will be terminated!
   ExitApp
}
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

16 Jan 2018, 13:43

Thanks!
The loading time is about 1500ms (no problem)
The Task Manager show that AHK needs 172.1 MB (no problem)
This file have 71 fields and 31.081 valid records.
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

17 Jan 2018, 04:50

Thanks!

Load time can be improved by adding SetBatchLines, -1 after #NoEnv. I forgot to do it.

I added a first example for the 'complicated search' based on the samples you provided:
Products.csv:

Code: Select all

ArtNo;Counts
"00013-14";35
"00013-14";5
"7392248100257";14
"29647";6
"567";5
"NOT_VALID";10

Code: Select all

#NoEnv
SetBatchLines, -1
/*
Inventory.csv:
"art_id";"art_ean";"art_ean2";"Barcode3";"Barcode4";"su_art_id";"art_main_name";"art_info1";"art_info2";"art_purch_price";"art_sales_price_in";"art_grpno";"art_suno"
"000030-333-00";"7350030498777";"7326211165165";;;"567";"HISSGARDINSTÅNG";"KOLLA PRIS VID ";"HEMKOMST";28;79;1000;"FONDACO"
"00013-14";"7391487013298";"5475165165165";;;"00013.14";"JULKLAPPSPAPPER";"2 METER X 70 CM";"SORTERADE MÖNSTER";"4.5";5;7005;"HEDLUNDS"
"00014.13";"7391487014448";;;;"00014.13";"JULKLAPPSPAPPER";"4 METER X 1 METER";"SORTERADE MÖNSTER";"11.93";15;7005;"HEDLUNDS"

"00053";"7391487053386";;;;"00053";"PRESENTPAPPER";"1,5 METER X 70 CM";"METALLIC - SORTERAT";6;10;6001;"HEDLUNDS"
"GT10025";"7392248100257";;"001589-045-71";;10025;"HÄNGAMPEL";"20 X 20 X 20 CM";"VIT - ROBERT";59;149;3160;"GILBERT"
"29647";"7330010316352";;;;29647;"HATTHYLLA";"78 X 34 X 32 CM";"EK/STÅL";"85.85";129;5000;"EAST"
;;;;;;;;;;;;
"BG29647";"7391691053387";;;7340073817369;29647;"DOFTLJUS I GLASLYKTA";"INICIO - HÖSTPÄRON";"BRINNTID CA 25 TIMMAR";"5.75";10;3115;"BESON"
*/

InventoryPath := "Inventory.csv"
FieldCount := 13  ; number of fields in the CSV records
; Arrays
Rec_Array := []   ; simple array to hold the file records as arrays
PID_Array := []   ; primary ID array (art_id) -> record index in Rec_Array
SID_Array := []   ; secondary ID array (art_ean | art_ean2 | Barcode3 | Barcode4) -> record index in Rec_Array
Sup_Array := []   ; array of supplier IDs (su_art_id) -> array of record indices in Rec_Array
RecCount := 0

If !(FileHandle := FileOpen(InventoryPath, "r"))
   ErrorExit("Could not open " . InventoryPath)
Progress, B zh0 fs18, Loading inventory ...
S := A_TickCount
For RecordIndex, Record In StrSplit(FileHandle.Read(), "`n", "`r") {
   Fields := StrSplit(Record, ";", """")
   If (A_Index = 1) { ; check the header
      If (Fields.Length() <> FieldCount) || (Fields[1] <> "art_id")
         ErrorExit("Wrong file header:`n" . Record)
      Header := Fields
      Continue
   }
   If (Fields.Length() <> FieldCount) || (Fields[1] = "") ; skip invalid records
      Continue
   RecIndex := Rec_Array.Push(Fields)
   PID_Array[Fields[1] . ""] := RecIndex
   Loop, 4 {
      If (SID := Fields[A_Index + 1]) <> ""
         SID_Array[SID . ""] := RecIndex
   }
   SuID := Fields[6]
   If !Sup_Array.HasKey(SuID . "")
      Sup_Array[SuID . ""] := []
   Sup_Array[SuID . ""].Push(RecIndex)
}
FileHandle.Close()
RecCount := Rec_Array.Length()
T := A_TickCount - S
Progress, Off

MsgBox, 0, Load Files, %InventoryPath% has been loaded in %T% ms.`n`nNumber of valid input records: %RecCount%

/*
Products.csv:
ArtNo;Counts
"00013-14";35
"00013-14";5
"7392248100257";14
"29647";6
"567";5
"NOT_VALID";10

*/

ProductsPath := "Products.csv"
Products := []    ; array to hold records identified by PID_Array or SID_Array
Suppliers := []   ; array of records indices defined by Sup_Array (su_art_id)
Errors := []      ; array of records not found in the inventory.
ProdCount := 0    ; number of valid records in ProductsPath

Loop, Read, %ProductsPath%, `n, `r
{
   If (A_Index = 1) || (A_LoopReadLine = "")
      Continue
   ProdCount++
   Fields := StrSplit(A_LoopReadLine, ";", """")
   ID := Fields[1]
   ; Check the PID_Array (art_id)
   If (RecIndex := PID_Array[ID . ""]) {
      If !Products.HasKey(ID . "")
         Products[ID . ""] := {Count: 0, RecIndex: RecIndex}
      Products[ID . ""].Count += Fields[2]
   }
   ; Check the SID_Array (art_ean | art_ean2 | Barcode3 | Barcode4)
   Else If (RecIndex := SID_Array[ID . ""]) {
      ArtID := Rec_Array[RecIndex, 1]
      If !Products.HasKey(ArtID . "")
         Products[ArtID . ""] := {Count: 0, RecIndex: RecIndex}
      Products[ArtID . ""].Count += Fields[2]
   }
   ; Check the Sup_Array (su_art_id)
   Else If (Records := Sup_Array[ID . ""])
      ; If the ID has been found in only one record it's clearly identified
      If (Records.Length() = 1) {
         RecIndex := Records[1]
         ArtID := Rec_Array[RecIndex, 1]
         If !Products.HasKey(ArtID . "")
            Products[ArtID . ""] := {Count: 0, RecIndex: RecIndex}
         Products[ArtID . ""].Count += Fields[2]
      }
      Else
         Suppliers.Push({Line: A_LoopReadLine, Records: Records})
   ; Not found!
   Else
      Errors.Push(A_LoopReadLine)
}

MsgBox, 0, Products, Read %ProdCount% records from %ProductsPath%.

; Clearly identified records
Msg := ""
RecCount := 0
For ID, Entry In Products {
   Msg .= "ID: " . ID . " - Count: " . Entry.Count . " - RecIndex: " . Entry.RecIndex . "`n"
   RecCount++
}
MsgBox, 0, %RecCount% of %ProdCount% records found by ID, %Msg%
; Identified by su_art_id
Msg := ""
RecCount := 0
For Index, Entry In Suppliers {
   Msg .= "Line: " . Entry.Line . " - Records:"
   For Each, RecIndex In Entry.Records
      Msg .= " " . RecIndex
   Msg .= "`n"
   RecCount++
}
MsgBox, 0, %RecCount% of %ProdCount% records found by Supplier ID, %Msg%
; Not found!
Msg := ""
RecCount := 0
For Index, Line In Errors {
   Msg .= Line . "`n"
   RecCount++
}
MsgBox, 0, %RecCount% of %ProdCount% records could not be found, %Msg%

ExitApp
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

17 Jan 2018, 18:39

Great!
I think the program is very fast. Reading the article register takes now about 563-1000 ms., 176,7MB
I made an another test file like this .:

Code: Select all

; Products.csv
ArtNo;Counts
"00013-17";35
"00013-17";5
"7392248100257";14
"BG29647";6
"00013-17";10
00123ABC;35
"0105413";5
"NOT_VALID";10
the result (4 of 8 records found by ID) .:
ID: 00013-17 - Count: 50 - RecIndex: 3
ID: 0105413 - Count: 5 - RecIndex: 473
ID: BG29647 - Count: 6 - RecIndex: 24281
ID: GT10025 - Count: 14 - RecIndex: 24493
Correct!
(1 of 8 records found by Supplier ID) .:
Line: 00123ABC;35 - Records: 19 2326 25797
It's correct!
(1 of 8 records could not be found) .:
"NOT_VALID";10
Does it matter ifSetBatchLines, -1 comes before or after#NoEnv?
(the function "ErrorExit(Msg)" was missed in the second version...)

The next problem is that the field "art_ean" is field number 17 and "art_ean2" is field number 41 and so on (if I counted right).
(For example 7392248100257 and 00123ABC is not found, but when I redo the file, with the fields in the correct order, the program works.).
I have ordered the export of the CSV file with the fields in my order, I hope that it will be delivered in UTF-8 format, but not yet delivered....

Another problem is that the "Product File" may contain different number of columns.
When the file is generated by the PDA:s, the file has 6 (or 7 fields).
The first two are ArtNo and Count (after these comes the following fields.: PDA name, location, user, date and time)
Then, the input file can be manually added with products with only these two fields (ArtNo and Count)
(If I interpret the script correctly, these manually added lines will disappear - have not been tested)
I like your solution. It is very compact and fast, but not so general.
_____________________________________________________

The fact that the article register is large does not seem to cause problems for AHK (about 31,000 products and about 70 fields).
Of these products is approx. 6-8,000 active today. This register will grow by about 10% per year. One field contains whether the product is active or inactive.
But when the article register is to be read in, for example, PDAs, it's no advantage that the file is large (just taking time). PDA only needs 10 fields and these 6-8000 products.
i.e. Another need is to convert the large registry into a custom smaller registry. (Memor.csv)
or create a result with the fields .: art_id, art_name, prodinfo1, prodinfo2, count, art_purce_price and the sum (count * art_purce_price)
One result is with all products
Another result is only from art_grp_no (product group) 7000-7010
Another result is from each supplier
and so on.

I will try to interpret your program, and if it is possible to do more generally.
for example,
- A function that reads and handle a CSV file "
- A function that searches barcodes and item numbers. (as in your example)
- A function as select desired fields and rows.
- Another function that provides a desired result (in some way)
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

18 Jan 2018, 05:11

Albireo wrote:Does it matter if SetBatchLines, -1 comes before or after #NoEnv?
No. I just prefer to have the directives at the top.

All the script can do depends on available information. The structure of data (arrays/objects) depends on how they shall be processed and your personal requirements related to speed and memory usage.

Inventory file:
  • The script can find "art_ean" at any position within the record. It just has to know that the position is not always 2.
  • If the original order of the records doesn't matter and the "art_id" field is unique the records can be stored directly in the "PID_Array" (or e.g. "ProdInfo"). If you want to keep the additional arrays SID_Array and Sup_Array their keys would have to be associated with the "art_id" instead of the record index.
  • If you want to store a selection of fields, the script could do it. You just have to define the fields of interest.
  • If you want to skip "inactive" records, the script could do it. It just needs to know where to find the attribute. But it might be better to provide an inventory file for "active" records only.
  • If the time needed for a sequential search fits your requirements, additional arrays/objects are not needed.
Products file:
  • As long as only fields 1 ("ArtNo") and 2 ("Count") are used, additional fields don't matter.
Result file(s):
  • You can create every result you want from available data. You just need to define the rules.
Albireo
Posts: 1752
Joined: 16 Oct 2013, 13:53

Re: Get a value from an associative matrix

18 Jan 2018, 11:41

Thank you!
I try to understand what you do (what happen on this rows)

Code: Select all

   RecIndex := Rec_Array.Push(Fields)
   PID_Array[Fields[1] . ""] := RecIndex
   Loop, 4 {
      If (SID := Fields[A_Index + 1]) <> ""
         SID_Array[SID . ""] := RecIndex
   }
   SuID := Fields[6]
   If !Sup_Array.HasKey(SuID . "")
      Sup_Array[SuID . ""] := []
   Sup_Array[SuID . ""].Push(RecIndex)
First line RecIndex := Rec_Array.Push(Fields)
For example the Array "Fields" have (Key:Value) 1:000030-333-00 2:7350030498777 and so on.
Rec_Array.Push - "Appends values to the end of an array."
But how can I see what values it is in the array "Rec_Array"?
Whats the difference between Rec_Index += 1 and RecIndex := Rec_Array.Push(Fields)?
How can RecIndex first be 1 then 2 and then 3 and so on.
What happens?
I think it's not obvious how to handle simple-/multi- arrays.
just me
Posts: 9453
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: Get a value from an associative matrix

18 Jan 2018, 12:11

Hi,

Rec_Array.Push(Value) appends the specified value(s) to Rec_Array and returns the index of the last added value. If you append a single value to an empty array (what you actually do in this case because you pass a single reference to an array), the value will be added at position (index) 1, so 1 will be returned. The next value will be added at index 2 and 2 will be returned, and so on. Technically there is no difference between

Code: Select all

RecIndex := Rec_Array.Push(Fields)
and

Code: Select all

RecIndex := 0
...
...
Rec_Array.Push(Fields)
RecIndex += 1
or

Code: Select all

RecIndex := 0
...
...
RecIndex += 1
Rec_Array[RecIndex] := Fields
as far as you add only a single value, which is an array reference in this case.

You would access the array with

Code: Select all

Rec_Array[RecIndex]
if you want to get the whole array of one record or

Code: Select all

Rec_Array[RecIndex, FieldIndex]
if you want the contents of a single field contained in this array.

Both indices can be passed as variables. So you can do something like

Code: Select all

art_id := 1 ; index of the field art_id
...
...
RecIndex := 7
Value := Rec_Array[RecIndex, art_id]
to get the content of the field "art_id" of the seventh record.

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: Google [Bot], ishida20, jameswrightesq, Lem2001 and 401 guests