Writing many small files vs adding to the end of a big file?

Get help with using AutoHotkey and its commands and hotkeys
omar
Posts: 295
Joined: 22 Oct 2015, 17:56

Writing many small files vs adding to the end of a big file?

19 Apr 2018, 18:49

I'm coding a mini application.
I need to store the date and time after each instance the code is run.

1. I can create and write to a new file every time.

2. Or... have one file and append to the end each time.

OK... so maybe the second option is the best?

Well... the thing is... I'm just worried about the code failing. It's bound to fail. It might be like 1 in 10,000 chance.
If the code failed and I got a corrupt file... then that would be a disaster.

Any thoughts?
I could maybe make backup copies after every 50 lines or something - but then I need to write more code to keep track.

Better keep simple? Doesn't really matter if I get a directory with 10,000 files.
(Or does it matter?)

Thanks.
poetbox
Posts: 28
Joined: 18 Apr 2018, 20:47

Re: Writing many small files vs adding to the end of a big file?

19 Apr 2018, 19:52

Maybe you can try to write the datetime to files at the first line.
omar
Posts: 295
Joined: 22 Oct 2015, 17:56

Re: Writing many small files vs adding to the end of a big file?

19 Apr 2018, 20:18

even first line...
just worried something might happen and the program crashes and the file becomes corrupt
User avatar
jeeswg
Posts: 4969
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Writing many small files vs adding to the end of a big file?

19 Apr 2018, 20:24

- You could have a main file and a backup file.
- The script checks the main file, if the main file is corrupted, it checks the backup file.
- When the script starts, it sets the contents of the main file, and confirms that it is not corrupted, it then does the same for the backup file, it then regularly updates the main file.
- If the main file becomes corrupted, you can use the backup file.
- The question is then that perhaps you want to update the backup file occasionally. If you do that, you should confirm that the main file is not corrupted prior to updating the backup file.
bish01
Posts: 94
Joined: 22 Feb 2016, 03:57

Re: Writing many small files vs adding to the end of a big file?

20 Apr 2018, 16:16

hi jeeswg
how do you properly check /confirm that txt file is not corrupt?
is it just fileread or something else?

thank you in advance
User avatar
jeeswg
Posts: 4969
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Writing many small files vs adding to the end of a big file?

20 Apr 2018, 20:52

- Yes, FileRead, and other checks like RegEx, and less-than/greater-than comparisons.
- There may be other ways, these are just some suggestions. I know that I've seen at least one program that stored info in files and a backup for each file.
User avatar
sooyke
Posts: 7
Joined: 21 Apr 2018, 10:23

Re: Writing many small files vs adding to the end of a big file?

21 Apr 2018, 10:34

I got a corrupt file


If you are on win10 you can use file history backup ,it stores any change made to a file for later recovery if needed.

https://www.pcworld.com/article/2974385 ... ature.html
Eureka
Posts: 4
Joined: 03 Apr 2018, 13:31
Location: Netherlands

Re: Writing many small files vs adding to the end of a big file?

21 Apr 2018, 16:55

It might be easier to create a lock-file when you start your application and remove that one when you exit.
That way you are sure that there was a problem when you start the program and the lock-file is still there.

In pseudo code (I'm not very fluent in AHK):
- check if file exists. If yes: alert and stop; if not: proceed
- create file
- (do your magic here)
- remove file
- check if file is removed. If not:alert; if yes: add entry to your logfile and exit


(Doesn't have to be a file, of course. Something like a reg-key would do to.)

Return to “Ask For Help”

Who is online

Users browsing this forum: FidoGooL, kirby, scriptors and 57 guests