Edit some text files, string length Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Edit some text files, string length

23 Jun 2017, 00:18

Greetings, everyone
There are many files that contain text, like this (contains half-width katakana and latin symbols):

B ニヌacc cpaテy ツe ハocヌe テソoノニa テaxoチトヒ yホトヒeヌモ, ノo yホeノトニト ソcィoセトp ハpoチoヌツaユヒ セoヌヒaヒモ, チaツe ノe coセトpaヨcモ paccaツトソaヒモcヨ ハo ネecヒaネ.

I need to write a script that does the following:
If the string is longer than 61 characters, and if the 61th or the 62th character is not a space, move to the left for the first space before the word and insert \n
Then move to the right after \n, check the next 61 characters in the string, and if the 61st or the 62nd character is not a space, move to the left for the first space before the word and insert \n, and so check to the end of the string, then check the next string in the file

so, string
B ニヌacc cpaテy ツe ハocヌe テソoノニa テaxoチトヒ yホトヒeヌモ, ノo yホeノトニト ソcィoセトp ハpoチoヌツaユヒ セoヌヒaヒモ, チaツe ノe coセトpaヨcモ paccaツトソaヒモcヨ ハo ネecヒaネ.
must turn into a
B ニヌacc cpaテy ツe ハocヌe テソoノニa テaxoチトヒ yホトヒeヌモ, ノo yホeノトニト \nソcィoセトp ハpoチoヌツaユヒ セoヌヒaヒモ, チaツe ノe coセトpaヨcモ paccaツトソaヒモcヨ \nハo ネecヒaネ.

And so in all the files in the folder

Please help with this
Last edited by AEBus on 26 Jun 2017, 19:14, edited 1 time in total.
Guest

Re: Edit some text files, string length

23 Jun 2017, 02:36

It will probably involve some trickery which I'm not familiar with as it no doubt has something to do with double bytes for those katakana characters, but you could have a look at https://github.com/hi5/TF#TF_Wrap

But when I do TF_Wrap("a.txt", 61) (creates a_copy.txt so you can check the result) it actually wraps around 30 or so - so you could try 120 and if the results are somewhat satisfactory do it for all files.
To write back the results of the wrap to the original file use:
TF_Wrap("!a.txt", 120) ; add a ! in front of the filename.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

26 Jun 2017, 19:12

Here's a little bit of code that might be helpful.

Code: Select all

vPath = %A_Desktop%\MyFile.txt
;vPath = C:\MyDir\MyFile.txt
FileRead, vText, % vPath
Loop, Parse, vText, `n, `r
{
	vTemp := A_LoopField
	MsgBox, % vTemp
}
You will need to read up on InStr and SubStr.

One thing that's relatively straightforward in AHK, but not well-documented AFAIK is:

'SEARCH FROM THE NTH CHARACTER BACKWARDS'
which I talk about here:
jeeswg's strings tutorial - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=7&t=32985

Best of luck.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length  Topic is solved

26 Jun 2017, 20:44

When I started out with AHK, I quickly ran into difficult problems, that weren't particularly easy to solve, and I had to try and solve them from first principles.

The problem of manually wrapping text, is an interesting problem, I hadn't really considered it much before, I would imagine that this script does more or less what you want. I would recommend using FileAppend to output the text to a new file, rather than trying to overwrite the original files. Anyhow, it's always good to keep backup copies of files. I would also use FileExist to check if the file already exists before trying to append.

Re. which files to use as input: see file loops, or use FileRead on a text file that contains a list of paths.

Btw you didn't specify what to do if there was a really long line of text with no spaces in it, currently in that case, the script just gives an error message and ends.

Code: Select all

q:: ;wrap text
vIsV1 := !!SubStr(1,0)
;vPath = %A_Desktop%\MyFile.txt
;vPath = C:\MyDir\MyFile.txt
;FileRead, vText, % vPath
vText := "B ニヌacc cpaテy ツe ハocヌe テソoノニa テaxoチトヒ yホトヒeヌモ, ノo yホeノトニト ソcィoセトp ハpoチoヌツaユヒ セoヌヒaヒモ, チaツe ノe coセトpaヨcモ paccaツトソaヒモcヨ ハo ネecヒaネ."

vOutput := ""
VarSetCapacity(vOutput, StrLen(vText)*2)
Loop, Parse, vText, `n, `r
{
	vTemp := A_LoopField
	Loop
	{
		if (StrLen(vTemp) <= 61)
		{
			vOutput .= vTemp "`n"
			break
		}
		vPos := 62
		vPosB := vPos - StrLen(vTemp) - !vIsV1
		vPos := InStr(vTemp, " ", 0, vPosB)
		if !vPos
		{
			MsgBox, % "script ended: space not found"
			return
		}
		vOutput .= SubStr(vTemp, 1, vPos-1) "`n"
		vTemp := SubStr(vTemp, vPos+1)
	}

}

vOutput := StrReplace(vOutput, "`n", "`r`n")
Clipboard := vOutput
MsgBox, % "done"
return
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Re: Edit some text files, string length

27 Jun 2017, 09:46

Code: Select all

vIsV1 := !!SubStr(1,0)
vInPath = d:\wrk\ahk\1
vOutPath = d:\wrk\ahk\2
vInFile = *.txt
vLength = 61
Loop Files, %vInPath%\%vInFile%
{
	FileDelete, %vOutPath%\%A_LoopFileName%
	Loop, Read, %A_LoopFileLongPath%
	{
		Loop, Parse, A_LoopReadLine, `r, `n
		{
			vOutput := ""
			vTemp := A_LoopField
			Loop
			{
				if (StrLen(vTemp) <= vLength)
				{
					vOutput .= vTemp
					break
				}
				vPos := vLength+1
				vPosB := vPos - StrLen(vTemp) - !vIsV1
				vPos := InStr(vTemp, " ", 0, vPosB)
				if !vPos
				{
					vOutput .= vTemp
					break
				}
				vOutput .= SubStr(vTemp, 1, vPos-1) " \n"
				vTemp := SubStr(vTemp, vPos+1)
			}
			FileAppend, %vOutput%`r`n, %vOutPath%\%A_LoopFileName%, UTF-16
		}
	}
}
And how to make the empty lines in the new file also empty?
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

27 Jun 2017, 11:56

Probably you need to add something like this just above 'if (StrLen(vTemp) <= 61)'.

Code: Select all

if (vTemp = "")
{
	vOutput .= "`r`n"
	break
}
So you do want '\n' not '`n' or '`r`n'? Does the text represent some kind of special formatting?

From your alterations, it looks like you understand AHK pretty well, did you have a play around with InStr and SubStr before trying my code? Cheers.

[EDIT:] Oh btw are your files very large? You might like to do something like:

Code: Select all

vOutput := ""
VarSetCapacity(vOutput, 1000000*2)

vOutput := ""
VarSetCapacity(vOutput, StrLen(vText)*2)

FileGetSize, vSize, % vPath
vOutput := ""
VarSetCapacity(vOutput, vSize*2)
[EDIT:] Btw it seems that by bumping my first post on this thread, it's changed the stated creation date for it!?!?
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Re: Edit some text files, string length

27 Jun 2017, 12:27

jeeswg wrote:Probably you need to add something like this just above 'if (StrLen(vTemp) <= 61)'.

Code: Select all

if (vTemp = "")
{
	vOutput .= "`r`n"
	break
}
not work, skript ignoring empty lines

Code: Select all

vIsV1 := !!SubStr(1,0)
vInPath = d:\wrk\ahk\1
vOutPath = d:\wrk\ahk\2
vInFile = *.txt
vLength = 61
Loop Files, %vInPath%\%vInFile%
{
	FileDelete, %vOutPath%\%A_LoopFileName%
	Loop, Read, %A_LoopFileLongPath%
	{
		Loop, Parse, A_LoopReadLine, `r, `n
		{
			vOutput := ""
			vTemp := A_LoopField
			Loop
			{
				if (vTemp = "")
				{
					vOutput .= "`r`n"
					break
				}
				if (StrLen(vTemp) <= vLength)
				{
					vOutput .= vTemp
					break
				}
				vPos := vLength+1
				vPosB := vPos - StrLen(vTemp) - !vIsV1
				vPos := InStr(vTemp, " ", 0, vPosB)
				if !vPos
				{
					vOutput .= vTemp
					break
				}
				vOutput .= SubStr(vTemp, 1, vPos-1) " \n"
				vTemp := SubStr(vTemp, vPos+1)
			}
			FileAppend, %vOutput%`r`n, %vOutPath%\%A_LoopFileName%, UTF-16
		}
	}
}
jeeswg wrote:So you do want '\n' not '`n' or '`r`n'? Does the text represent some kind of special formatting?
Yes, it is translated text for some japan visual novel. "\n" break current line.
Just wordwrapping not work
jeeswg wrote:From your alterations, it looks like you understand AHK pretty well, did you have a play around with InStr and SubStr before trying my code? Cheers.
Nope, i understand AHK so-so, just read documentation.
jeeswg wrote:[EDIT:] Oh btw are your files very large? You might like to do something like:

Code: Select all

vOutput := ""
VarSetCapacity(vOutput, 1000000*2)

vOutput := ""
VarSetCapacity(vOutput, StrLen(vText)*2)

FileGetSize, vSize, % vPath
vOutput := ""
VarSetCapacity(vOutput, vSize*2)
Max file size is 1,21 Mb with 20901 lines.
btw i dont understand why should write VarSetCapacity
jeeswg wrote:[EDIT:] Btw it seems that by bumping my first post on this thread, it's changed the stated creation date for it!?!?
Bumping = up the thread, as if I had written a new message
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

27 Jun 2017, 13:40

VarSetCapacity()
https://autohotkey.com/docs/commands/VarSetCapacity.htm
this function can also be used to enhance performance when building a string by means of gradual concatenation. This is because multiple automatic resizings can be avoided when you have some idea of what the string's final length will be.
If you go to ListVars (View, Variables and their contents), you see the capacity for each variable, if you want to append text to a variable, but the capacity is too small, then data has to be shuffled about.

You've changed the script to use A_LoopReadLine, I prefer to use FileRead, and parse the text. ... Maybe you need: vOutput .= "\n"
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Re: Edit some text files, string length

27 Jun 2017, 14:04

jeeswg wrote:You've changed the script to use A_LoopReadLine, I prefer to use FileRead, and parse the text.

Code: Select all

vIsV1 := !!SubStr(1,0)
vInPath = d:\wrk\ahk\1
vOutPath = d:\wrk\ahk\2
vInFile = *.txt
vLength = 61
Loop Files, %vInPath%\%vInFile%
{
	FileDelete, %vOutPath%\%A_LoopFileName%
	FileRead, vText, %A_LoopFileLongPath%
	{
		Loop, Parse, vText, `r, `n
		{
			vOutput := ""
			VarSetCapacity(vOutput, StrLen(vText)*2)
			vTemp := A_LoopField
			Loop
			{
				if (vTemp = "")
				{
					vOutput .= ""
					break
				}
				if (StrLen(vTemp) <= vLength)
				{
					vOutput .= vTemp
					break
				}
				vPos := vLength+1
				vPosB := vPos - StrLen(vTemp) - !vIsV1
				vPos := InStr(vTemp, " ", 0, vPosB)
				if !vPos
				{
					vOutput .= vTemp
					break
				}
				vOutput .= SubStr(vTemp, 1, vPos-1) " \n"
				vTemp := SubStr(vTemp, vPos+1)
			}
			FileAppend, %vOutput%`r`n, %vOutPath%\%A_LoopFileName%, UTF-16
		}
	}
}
Work fine, but adds an extra line break to the end of the file...
for example in source file 2000 strings, in result file 2001 strings
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

27 Jun 2017, 14:11

It might be the `r`n in the FileAppend line. You would have to readjust it to append `r`n%vOutput% (or %vOutput% only, if it's the first time).

Instead of appending everything one line at a time, you could collect everything in a big variable, and remove the trailing CRLF from it, and append all the text in one go.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Re: Edit some text files, string length

27 Jun 2017, 17:23

jeeswg,
Maybe you help with gui?
(btw, I solved problem with the last line, see end of the script)

Code: Select all

vIsV1 := !!SubStr(1,0)
;vInPath = d:\wrk\ahk\1
;vOutPath = d:\wrk\ahk\2
vInFile = *.txt
vLength = 61

Gui, Add, Text, x12 y9 w90 h20 +Vertical, Source Folder:
Gui, Add, Text, vInPath x112 y9 w280 h20 +Border, 
Gui, Add, Button, gSelectInPath x402 y9 w70 h20 , Browse
Gui, Add, Text, x12 y39 w90 h20 , Destination Folder:
Gui, Add, Text, vOutPath x112 y39 w280 h20 +Border, 
Gui, Add, Button, gSelectOutPath x402 y39 w70 h20 , Browse
Gui, Add, Text, x12 y69 w60 h20 , File Types:
Gui, Add, Edit, vInFile x72 y69 w80 h20 , %vInFile%
Gui, Add, Text, x162 y69 w70 h20 , String Length:
Gui, Add, Edit, vLength x232 y69 w80 h20 +Number, %vLength%
Gui, Add, Button, gStart x322 y69 w150 h20 , Start
Gui, Show, x314 y228 h103 w488, Text Wrapper
Return

SelectInPath:
FileSelectFolder, vInPath
GuiControl,,vInPath, %vInPath%
Return

SelectOutPath:
FileSelectFolder, vOutPath
GuiControl,,vOutPath, %vOutPath%
Return

Start:
If (vInPath ="")
{
	MsgBox, 16, , Please select Source Folder
	Return
}
If (vOutPath = "")
{
	MsgBox, 16, , Please select Destination Folder
	Return
}
If (vInFile = "")
{
	MsgBox, 16, , Please specify File Types
	Return
}
If (vLength = "")
{
	MsgBox, 16, , Please specify String Length
	Return
}
{
	Loop Files, %vInPath%\%vInFile%
	{
		FileDelete, %vOutPath%\%A_LoopFileName%
		FileRead, vText, %A_LoopFileLongPath%
		{
			vNewOutput := ""
			Loop, Parse, vText, `r, `n
			{
				vOutput := ""
				VarSetCapacity(vOutput, StrLen(vText)*2)
				vTemp := A_LoopField
				Loop
				{
					if (vTemp = "")
					{
						vOutput .= ""
						break
					}
					if (StrLen(vTemp) <= vLength)
					{
						vOutput .= vTemp
						break
					}
					vPos := vLength+1
					vPosB := vPos - StrLen(vTemp) - !vIsV1
					vPos := InStr(vTemp, " ", 0, vPosB)
					if !vPos
					{
						vOutput .= vTemp
						break
					}
					vOutput .= SubStr(vTemp, 1, vPos-1) " \n"
					vTemp := SubStr(vTemp, vPos+1)
					}
				vNewOutput .= vOutput "`r`n"
			}

		}
		StringGetPos,vlastLine, vNewOutput,`r,r,0
		vFileContents := SubStr(vNewOutput,1,vlastLine)
		FileAppend, %vFileContents%, %vOutPath%\%A_LoopFileName%, UTF-16
	}
	MsgBox, 64, , Done
	Return
}

GuiClose:
ExitApp
  • If i select Source Folder result dont show in control text vInPath
  • If i select Destination Folder result dont show in control text vOutPath
  • If i delete or change text in controls Edit vInFile or vLength script still uses the old variables
Last edited by AEBus on 27 Jun 2017, 18:16, edited 1 time in total.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

27 Jun 2017, 18:12

I don't do too much with AHK's GUI command. Sounds like it could be:
GUI
https://autohotkey.com/docs/commands/Gui.htm#Submit

[EDIT:] Also, since the GUI command uses v for defining a variable, if the variable starts with 'v', then you need to specify things like 'vvText' and 'vvOutput'.

==================================================

[EDIT:] A few pointers:

This does nothing: vOutput .= ""

You might like to add a VarSetCapacity line for vFileContents.

If we know for sure that vFileContents will end in `r`n, then we can do:
vFileContents := SubStr(vNewOutput,1,-2)
Not a big deal but: StringGetPos is deprecated, I would recommend using InStr instead.
Last edited by jeeswg on 27 Jun 2017, 18:31, edited 3 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
AEBus
Posts: 6
Joined: 13 Jan 2017, 14:56

Re: Edit some text files, string length

27 Jun 2017, 18:57

I solved all problems
Complete script:

Code: Select all

Gui, Add, Text, x12 y9 w90 h20 , Source Folder:
Gui, Add, Text, vvInPath x112 y9 w280 h20 +Border
Gui, Add, Button, gSelectInPath x402 y9 w70 h20 , Browse
Gui, Add, Text, x12 y39 w90 h20 , Destination Folder:
Gui, Add, Text, vvOutPath x112 y39 w280 h20 +Border
Gui, Add, Button, gSelectOutPath x402 y39 w70 h20 , Browse
Gui, Add, Text, x12 y69 w60 h20 , File Types:
Gui, Add, Edit, vvInFile x72 y69 w80 h20 , *.txt
Gui, Add, Text, x162 y69 w70 h20 , String Length:
Gui, Add, Edit, vvLength x232 y69 w80 h20 +Number, 61
Gui, Add, Button, gStart x322 y69 w150 h20 , Start
Gui, Show, x314 y228 h103 w488 , Text Wrapper
Return

SelectInPath:
FileSelectFolder, vInPath
GuiControl,,vInPath, %vInPath%
Return

SelectOutPath:
FileSelectFolder, vOutPath
GuiControl,,vOutPath, %vOutPath%
Return

Start:
Gui, Submit, NoHide
If (vInPath ="")
{
	MsgBox, 16, Error, Please select Source Folder
	Return
}
If (vOutPath = "")
{
	MsgBox, 16, Error, Please select Destination Folder
	Return
}
If (vInFile = "")
{
	MsgBox, 16, Error, Please specify File Types
	Return
}
If (vLength = "")
{
	MsgBox, 16, Error, Please specify String Length
	Return
}
{	
	vIsV1 := !!SubStr(1,0)
	Loop Files, %vInPath%\%vInFile%
	{
		FileDelete, %vOutPath%\%A_LoopFileName%
		FileRead, vText, %A_LoopFileLongPath%
		{
			VarSetCapacity(vNewOutput, StrLen(vText)*2)
			vNewOutput := ""
			Loop, Parse, vText, `r, `n
			{
				VarSetCapacity(vOutput, StrLen(vText)*2)
				vTemp := A_LoopField
				Loop
				{
					if (vTemp = "")
					{
						vOutput .= ""
						break
					}
					if (StrLen(vTemp) <= vLength)
					{
						vOutput .= vTemp
						break
					}
					vPos := vLength+1
					vPosB := vPos - StrLen(vTemp) - !vIsV1
					vPos := InStr(vTemp, " ", 0, vPosB)
					if !vPos
					{
						vOutput .= vTemp
						break
					}
					vOutput .= SubStr(vTemp, 1, vPos-1) " \n"
					vTemp := SubStr(vTemp, vPos+1)
					}
				vNewOutput .= vOutput "`r`n"
			}

		}
		VarSetCapacity(vFileContents, StrLen(vText)*2)
		vFileContents := SubStr(vNewOutput,1,-2)
		FileAppend, %vFileContents%, %vOutPath%\%A_LoopFileName%, UTF-16
	}
	MsgBox, 64, Info, Done
	Return
}

GuiClose:
GuiEscape:
ExitApp
Thank you very much for help!
[EDIT:]
The script has been modified in accordance with your recommendations =)
jeeswg wrote:[EDIT:] A few pointers:

This does nothing: vOutput .= ""

You might like to add a VarSetCapacity line for vFileContents.

If we know for sure that vFileContents will end in `r`n, then we can do:
vFileContents := SubStr(vNewOutput,1,-2)
Not a big deal but: StringGetPos is deprecated, I would recommend using InStr instead.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Edit some text files, string length

29 Nov 2018, 17:05

I've written a function here, similar to the code above.
add word wrap to a string - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=59461
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: csbellmd, Google [Bot], marypoppins_1 and 200 guests