Trie Data Structure

[Capn Odin]

I was wrong about the issue, the problem is when generating the string used to populate the ListBox, turns out that supplying a variadic function with somewhere between 10000 and 141942 strings will crash AHK.

I think the only way to tackle extra large wordlists (1M+) in Ahk is MCode.

It didn't seem to be the problem although I am sure there is a limit to what AHK can handle, but you may be right about using some compiled code, I will have to look into where this script would benefit from it.

[rommmcek]

Just tested suggestions with 1.7M words. Works perfect (absolutely no lag) if using:

Code: Select all

	if StrLen(Address) = 1
		return

Otherwise crash!
Maybe is there some kind of a bug, searching with only one letter.

[Capn Odin]

Just tested suggestions with 1.7M words. Works perfect (absolutely no lag) if using:

Code: Select all

	if StrLen(Address) = 1
		return

Otherwise crash!
Maybe is there some kind of a bug, searching with only one letter.

I had not uploaded the fix. You can try again if you want.

[Capn Odin]

I have some code that saves memory, and allow for limiting the number of results, but I feel it is cluttered and hard to follow, should I upload it ?

[rommmcek]

Crash again, but this time it was possible to terminate process immediately, so no pain,
P.s.: Post the memory code under the Spoiler.

[rommmcek]

Sorry, I used old Suggestions, the new one does not crash! But the lag is considerable (2-3 sec).

[Capn Odin]

Just tested suggestions with 1.7M words. Works perfect (absolutely no lag) if using:

Code: Select all

	if StrLen(Address) = 1
		return

Otherwise crash!
Maybe is there some kind of a bug, searching with only one letter.

Think of it this way.
If we use the English alphabet and a dictionary of 1.7M evenly distributed words, then we would get the following number of words for keys with length 1 to 4.
1 char: (1 / 26)^1 * 1.7M = 65,385 words
2 char: (1 / 26)^2 * 1.7M = 2,515 words
3 char: (1 / 26)^3 * 1.7M = 97 words
4 char: (1 / 26)^4 * 1.7M = 4 words

[rommmcek]

I didn't noticed you display all matches! So you did it! Hat off!
Now I limited the search, but the lag (for one letter) is still to big!

Consider this: Once upon the time I had a very slow machine, so I've split the file to 25 (actually there were more) smaller units and I had quick access to any word!

Is there a way to load big file into more then one Trie?
Besides, is it possible to store the Trie structure on disk, if that would be of any benefit for loading?

[Capn Odin]

Not displaying all matches is a new feature. Right now it is limited to 500 words, but it have been unlimited to this point.

[rommmcek]

Absolutely perfect, the best ever! I am speechless!

[rommmcek]

That "perfect" was for one letter entry search speed!

How do you handle capitals? In the suggestions there aren't any captals/majuscules even if present in the list! If there are two identical words except for the capital, only one is shown (with minuscules).

Big thanks for your effort!

[rommmcek]

Not exactly as I described (results are dependent on what was the search entry - minuscular/majuscular), but there is inconsistency for sure!

[Capn Odin]

Thought I would write a progress report. Deletions are done for the uploaded version. I am working on "compressing" the trie, this is working however deletion do not keep the "compression" yet.

Memory usage for the same dictionary with 2,235,194 words:

The Original Script


The Current Script


The Compressed Script

[ozzii]

I like it, thanks.
Just a question: How to 'hide' the windows after the #s ???

[rommmcek]

You can do that adding #s::Gui, Hide above Enter:: hotkey/routine.

[ozzii]

Thanks for the reply