StrSplit() returns an array of unexpected capacity

Report problems with documented functionality
obeeb
Posts: 140
Joined: 20 Feb 2014, 19:15

Re: StrSplit() returns an array of unexpected capacity

21 Aug 2017, 22:24

I am on your side but after the lengthy interaction I had with 'just me' I understand his position and can probably explain it to you better than he will so I will do that.

pneumatic wrote:
just me wrote:There's no section in the documentation telling you that Array.GetCapacity() will always return the same value as Array.Length() for simple arrays, so it does not contradict 'documented behaviour'.

The issue is with StrSplit not GetCapacity
What he means is that StrSplit returns an array and because in general an array can have different length and capacity there is no issue.

pneumatic wrote:
documentation wrote:StrSplit()
For example: "," would divide the string based on every occurrence of a comma. Similarly, [A_Tab, A_Space] would create a new array element every time a space or tab is encountered in the input string.

The part in bold isn't strictly correct because it creates more array elements than just "every time a space or tab is encountered".
Capacity is not the same as the number of elements so it is in fact correct.

pneumatic wrote:Predicted response: nothing wrong because GetCapacity doesn't return the number of "elements" but some other [undocumented] thing :D
Actually it is documented if you look at GetCapacity you won't see much but few lines above it in SetCapacity you can see the following:
The maximum number of key-value pairs the object should be able to contain before it must be automatically expanded.
So in your sentence the number of "elements" is the current number of key-value pairs and the "[undocumented] thing" is the maximum number of elements.

Also in general 'just me' thinks that if your bug report contains "For some reason..." and/or "I'm guessing..." you shouldn't post it in the bug report forum but in Ask For Help and only after confirmation that it is in fact a bug it should be posted here.



@just me
I hope that I described your position correctly if I misunderstood or misrepresented something please correct me, that was not my intention. All I wanted is to save pneumatic some frustration.
pneumatic
Posts: 112
Joined: 05 Dec 2016, 01:51

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 01:35

If the documentation says "this is when I'm going to create an element" I would expect that rule to be true of all elements, including empty ones. If you create a bunch of empty elements for me when you didn't say so and I wasn't expecting it, that could cause bugs in my program. And in fact it did; I had been using getcapacity without issue on all my arrays since I created them manually and not with StrSplit, then when I added StrSplit to my program everything broke because the documentation didn't mention that it's going to create all these empty elements.
just me
Posts: 4788
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 03:48

pneumatic wrote: If you create a bunch of empty elements for me when you didn't say so and I wasn't expecting it, ...
It doesn't create "empty elements", it creates an array capable to store additional elements without the need to expand the internal element pointer tables for each added element. AHK's Simple Arrays aren't 'simple arrays' in the common sense. They are a special kind of AHK objects containing only numeric keys (indices) starting with 1. Also, AHK 1.1 doesn't throw an error if you try to access non-existent keys/indices. It returns an empty value.
... that could cause bugs in my program.
If you have bugs, they are caused by the wrong usage of the value returned by Array.GetCapacity(). If you want to get the current number of elements in a simple array, you have to use Array.Length().
Remaining with AHK 1.1.25.02 until v2 will become beta.
pneumatic
Posts: 112
Joined: 05 Dec 2016, 01:51

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 05:00

just me wrote:It doesn't create "empty elements"


If

Code: [Select all] [Download] GeSHi © Codebox Plus

Array := ["1","2","3","4"]
Array.setCapacity(8)


is equivalent to

Code: [Select all] [Download] GeSHi © Codebox Plus

Array := ["1","2","3","4","","","",""]


then increasing the capacity could be viewed as equivalent to creating "empty" elements.
just me
Posts: 4788
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 05:42

They aren't equivalent.

Code: [Select all] [Download] GeSHi © Codebox Plus

Array := ["1","2","3","4"]
Array.setCapacity(8)
does not add elements to an array. It increases (or decreases) the size of the internal table used to store the pointers to the elements, if needed. Therefore Array.Length() will still return 4. But you can store the elements 5 thru 8 afterwards without the need for expanding the pointer table (which might cause the need to move the whole table to a new memory location).

Code: [Select all] [Download] GeSHi © Codebox Plus

Array := ["1","2","3","4","","","",""]
explicitely defines 4 empty elements, 5 - 8. So Array.Length() returns 8.
Remaining with AHK 1.1.25.02 until v2 will become beta.
User avatar
nnnik
Posts: 2271
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 05:47

Try .hasKey for a better difference between the 2.
Recommends AHK Studio
obeeb
Posts: 140
Joined: 20 Feb 2014, 19:15

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 11:14

pneumatic wrote:then increasing the capacity could be viewed as equivalent to creating "empty" elements.
Elements i.e. key-value pairs are inserted or removed by you when you write an Autohotkey script, it doesn't matter if the value is empty or even if the key is empty they are still key-value pairs in an Autohotkey array i.e [].
The capacity is the capacity of the internal implementation, if it has empty space in it, all it means is that elements can be added to the Autohotkey array without (internally)expanding it. It has nothing to do with adding "empty" elements.
GetCapacity should never be used for anything, it gives you a view into what happens internally and maybe can be used for debugging purposes but you definitely can't rely on it to accurately reflect the number of elements in the Autohotkey array(unless you do SetCapacity(0) before).

In my opinion your mistake of using it and not Length() is understandable and reasonable, Autohotkey documentation and inconsistent behavior are to blame.

Helgef wrote:
just me wrote:
Find it either unreasonable or don't care about less experienced users having problems understanding how Autohotkey behaves. They think that instead of improving the documentation or making the behavior more consistent you should come here and ask a question in Ask For Help forum or spend a year or two learning c++ and look at the source code.
pneumatic
Posts: 112
Joined: 05 Dec 2016, 01:51

Re: StrSplit() returns an array of unexpected capacity

22 Aug 2017, 20:19

just me wrote:

Code: [Select all] [Download] GeSHi © Codebox Plus

Array := ["1","2","3","4","","","",""]
explicitely defines 4 empty elements, 5 - 8. So Array.Length() returns 8.


Huh that's weird, I tested it before I posted that message and could swear it was giving me length of 4, now it's not. Must have done something wrong in my test :)

Anyway, I still think the documentation should say it's expanding the size of the array because the user can't be expected to just figure that out.

If a builder says he will build me 6 rooms in my house but end up tacking on an extension to allow 8 rooms in the future, then he should say that. You may think it's inconsequential but it isn't, details matter, the devil is in the details and programmers tend to care about all the little minutiae about which bytes are going where. afaict AHK is supposed to be some kind of noob friendly wrapper language and in this case is not being very friendly to noobs like myself who aren't checking the source code just to split a string.
just me
Posts: 4788
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 03:13

pneumatic wrote:If a builder says he will build me 6 rooms in my house but end up tacking on an extension to allow 8 rooms in the future, then he should say that.
I wouldn't mind if he does it free of charge. ;)

You didn't show how you use the value returned by Array.GetCapacity(). But if it is causing 'bugs' in your scripts, you are definitely using it wrong.

As I already tried to show, you would most probably get similar issues in many cases if you'd use VarSetCapacity(StringVariable) to get the capacity of a variable's string buffer and use it instead of StrLen(StringVariable):

Code: [Select all] [Download] GeSHi © Codebox Plus

Str := ""
Loop, 132
Str .= "A"
StrVar := Str
StrVarCapacity := VarSetCapacity(StrVar) // (A_IsUnicode ? 2 : 1) ; capacity in TCHARs (Unicode: 2 bytes, ANSI: 1 byte)
StrVarLength := StrLen(StrVar) ; length in TCHARs
MsgBox, 0, StrVar, Capacity: %StrVarCapacity% - Length: %StrVarLength%

Capacity cannot be equated with content.
Remaining with AHK 1.1.25.02 until v2 will become beta.
pneumatic
Posts: 112
Joined: 05 Dec 2016, 01:51

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 07:15

just me wrote:if it is causing 'bugs' in your scripts, you are definitely using it wrong.


Or is it StrSplit who is is doing it "wrong" by sneakily adding all this extra capacity to my array :think:
Helgef
Posts: 2396
Joined: 17 Jul 2016, 01:02
Contact:

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 08:25

pneumatic, is there any specific reason you don't want to use .length()? You should note that even if strsplit is changed to shrink the capacity, you might still get unexpected results if you use getCapacity on other arrays, for example,

Code: [Select all] [Download] GeSHi © Codebox Plus

a:=[]
loop 5
a[A_Index]:=A_Index
msgbox % a.getcapacity() "`n" a.length()

the devil is in the details

Indeed, from the docs,

Code: [Select all] [Download] GeSHi © Codebox Plus

MaxItems := Object.GetCapacity()

It doesn't say actualNumberOfElements := Object.GetCapacity(). MaxItems is (briefly) explained just above, in the setCapacity section. Extending the documentation and refering to setCapacity from getCapacity wouldn't hurt. Questions about getCapacity are often answered by setCapacity.

Cheers. :)
obeeb
Posts: 140
Joined: 20 Feb 2014, 19:15

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 12:49

Helgef wrote:...you might still get unexpected results if you use getCapacity[
A similar line in the docs could've prevented this whole thread.
Helgef wrote: on other arrays, for example,

Code: [Select all] [Download] GeSHi © Codebox Plus

a:=[]
loop 5
a[A_Index]:=A_Index
msgbox % a.getcapacity() "`n" a.length() ; 8 5

This is another inconsistency I wanted to point out and since you brought it up I will, this is inconsistent with:

Code: [Select all] [Download] GeSHi © Codebox Plus

a:=[]
loop 5
a.push(A_Index)
msgbox % a.getcapacity() "`n" a.length() ; 5 5
Helgef
Posts: 2396
Joined: 17 Jul 2016, 01:02
Contact:

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 13:12

Obeeb. That inconstency is the only reason I didn't call the change in behaviour of push you showed earlier a bug. It seems we have two options when appending an array, depending on how we want the array to expand.
Cheers.
obeeb
Posts: 140
Joined: 20 Feb 2014, 19:15

Re: StrSplit() returns an array of unexpected capacity

23 Aug 2017, 18:52

Helgef wrote:That inconstency is the only reason I didn't call the change in behaviour of push you showed earlier a bug. It seems we have two options when appending an array, depending on how we want the array to expand.

This post is going to be about Autohotkey the program not Autohotkey the language.

Before Lexikos added Push and other methods in this commit the Insert method was used to add one or more key-value pairs.
In the c++ code the _Insert method used to call the Insert method when only one key-value pair was added and the InsertAt method when more than one were added.
The Insert method is the same method that gets called when you assign a value to a key directly and if it needs to expand the internal array it calls the Expand method, here it is:

Code: [Select all] [Download] GeSHi © Codebox Plus

bool Expand() {
return SetInternalCapacity(mFieldCountMax ? mFieldCountMax * 2 : 4);
}
As you can see this is what doubles the array.

The InsertAt method which was before this change used only when adding more than one element or when the object was created doesn't call Expand
but instead calls SetInternalCapacity directly when it needs to increase the size of the internal array and expands it only by the required amount.

When Lexikos implemented the _Push method for some reason he just used InsertAt and now Expand isn't called as it was before when you add one key-value pair(he also changed the deprecated _Insert method to just call _Push).
I don't know if this was intentional or not but if I have to guess I would say that it wasn't. He added the Expand method in the first commit that implemented objects in Autohotkey(2009) and was happily using it for many years but maybe he intended to make this change, everything is possible.


Wouldn't it be great if I didn't have to guess?! Maybe there can be a place where people can report inconsistent behavior and then Lexikos can take a look, and decide if it's a bug, works as intended, needs additional documentation... maybe we can even have a forum here and call it Bug Reports, oh wait...


P.S.
StrSplit uses the Append function which also calls Expand.

Links:
Source file from commit that added the new methods
https://github.com/Lexikos/AutoHotkey_L ... object.cpp
Source before that commit
https://github.com/Lexikos/AutoHotkey_L ... object.cpp
First commit related to objects and source files from it
https://github.com/Lexikos/AutoHotkey_L ... db58d8398d
https://github.com/Lexikos/AutoHotkey_L ... object.cpp
https://github.com/Lexikos/AutoHotkey_L ... t_object.h
just me
Posts: 4788
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: StrSplit() returns an array of unexpected capacity

24 Aug 2017, 02:28

obeeb, you are the one talking about implementation details.

The AHK language related problem we are talking about is:
May I assume that Object.GetCapacity() will always return the current number of key-value pairs stored in Object?
My answer is: No!
Remaining with AHK 1.1.25.02 until v2 will become beta.

Return to “Bug Reports”

Who is online

Users browsing this forum: No registered users and 2 guests