I have a series of 8 csv files (around 40k lines total) that I am working on integrating into one file and then updating the information daily. I have the code working decently (with only minor annoyances) but I am wondering if there are better ways to handle two of the processes.
#u:: ToolTip Reading All Files ;Read all files into Variables FileRead, Total, Level.txt ;Read Original File Loop, 8 ;Loop through Levels { Level :=A_Index - 1 ;Level numbers range 0-7 ToolTip Reading Level %Level% FileRead, Level%Level%, Level%Level%.csv ;Read CSV File into Level Variable StringReplace,OutputVar,Level%Level%,`n,`n,useerrorlevel ;LineCount from forums Lines%Level% :=ErrorLevel + 1 ;LineCount from forums LinesTotal :=LinesTotal + Lines%Level% } Loop, 8 { Level :=A_Index - 1 Loop, Read, Level%Level%.csv, Level.txt ;Read through Each Level file one line at a time. Append changes to Level.txt { CurrentLine ++ Lines%Level% := % Lines%Level% ToolTip Reading line %A_Index% of %LinesLevel% from Level %Level%`nLine %CurrentLine% of %LinesTotal%`n%A_LoopReadLine%`nNew Items:`n%NewItems% StringSplit, Array, A_LoopReadLine, `, ;Split Comma Separated File into individual variables If Array1 is not integer ;First Variable should be Item Number Continue ;If not then read next line IfInString, Total, %Array1% ;Item Number should appear within the Complete File Continue ;If not then StringReplace NewLine, A_LoopReadLine, `,, %A_Tab%, A ;Change to Tab Deliminated FileAppend %NewLine%`n ;Add new item to Complete File ;NewItems = %NewItems%`n%A_LoopReadLine% } } Total = ;Clear Total Variable FileRead, Total, Level.txt ;Replace with updated list StringReplace,OutputVar,Total,`n,`n,useerrorlevel;LineCount from forums Lines:=ErrorLevel + 1 ;LineCount from forums Loop, Read, Level.txt, LevelUpdated.txt ;Read file line-by-line and create updated file (Add Date column with current Level) { ToolTip Analyzing Item %A_Index% of %Lines%`n%A_LoopReadLine% StringSplit, Array, A_LoopReadLine, %A_Tab% If A_Index = 1 FileAppend %A_LoopReadLine%%A_Tab%%A_YYYY%/%A_MM%/%A_DD%`n ;On the first line add a date field If Array1 is not integer ;If not an item number skip line Continue Loop, 8 ;Loop through Levels { Level :=A_Index - 1 ;Level numbers range 0-7 IfInString, Level%Level%, %Array1% ;If Item Number found within Level { FileAppend %A_LoopReadLine%%A_Tab%%Level%`n ;Add Current Level to today's column Continue } } } ToolTip return
The process works fine and I've gotten better (somewhat) at dynamic changes to some of the variables so I don't have to type and edit 8 different versions of each loop. However I want to try to fix these two issues:
1- I have read each csv into a variable which makes it easy to locate item numbers across files and get line numbers, but I still use a reading loop to process each line. Is there a better/faster way I should be looking at doing this? Should I try to StringSplit the variable on line breaks and then process it that way?
2- I am updating each line with the current "Level" that the item is at every day. This is just appending onto the end of each line so that I end up with:
Item ID Call Number Title Author Library Year Published Total Checkouts Date Copy Created Date Last Checked Out Copy Price 10/25/2015 0000516459062 004.165 POG IPHONE THE MISSING MANUAL "Pogue David" WE-WH 2008 26 9/17/2008 9/23/2015 25 0 0000518571237 004.167 GOO 2012 ANDROID PHONES FOR DUMMIES "Gookin Dan author" WE-WH 2012 28 5/11/2012 9/16/2015 25 0
This works fine and every day I can add a new column and then eventually graph the daily level changes. It would be useful to be able to update two of the columns as I am doing this (Total Checkouts and Date Last Checked Out specifically) but the only way I can think to do this is to Loop Read to the correct line and then split that line and update the values which seems like a very slow process to add.
Any suggestions are appreciated!