Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Standard Library (stdlib): Simple file-system approach


  • Please log in to reply
63 replies to this topic

Poll: Should each stdlib function name begin with an underscore? (8 member(s) have cast votes)

Should each stdlib function name begin with an underscore?

  1. Yes, to reduce competition with user function names and indicate at-a-glance that the function is external. (1 votes [12.50%])

    Percentage of vote: 12.50%

  2. No, it's better not to "penalize" stdlib functions in this way. (7 votes [87.50%])

    Percentage of vote: 87.50%

  3. Other (0 votes [0.00%])

    Percentage of vote: 0.00%

Vote Guests cannot vote
Chris
  • Administrators
  • 10727 posts
  • Last active:
  • Joined: 02 Mar 2004

MyDocs/Include is overkill?

I don't like programs messing with My Documents and splitting AHK into multiple dirs around. Its enough to search A_AHKPATH\Include & A_AHKPath\Include-User. I don't see use for My Doc/Include. I don't beleive anybody outthere will have includes by user profile. We are not talking here about picture sharing.

Unfortunately, I think Vista's safeguards (described earlier) make it impractical to store custom/user functions in A_AhkPath. In addition, it would feel less professional because these custom functions are often the user's own creations and thus should be kept in a more visible, backup-friendly location. On the other hand, maybe most power users don't install AutoHotkey in the Program Files directory, opting instead to decompress the ZIP file somewhere. If that seems likely enough to you, we could have a poll to find out.

I'd considered this but it will lead to ambiguity if two or more identically-named files exist in various subdirectories.

First one found should be used, so other will not be accessible at all, the same as with user overrides...

I meant the user-includes directory, not the stdlib. Doing recursive search will lead to bugs and misbehavior even among power users due to accidentally having multiple functions of the same name. It would also reduce performance slightly. On the other hand, I see that it's a big benefit to organization; so I'm not fundamentally opposed to it. Perhaps another poll at some point...

Perhaps some log can be created so ppl know what functions are accutally included.

Good idea, but I think that it should be deferred until after the initial release, and only if there's demand.

Another drawback is that it would prevent having any subdirectories that contain files that are unrelated to functions (such as documentation, notes, etc.) So maybe this feature should be Off by default (which would allow it to be deferred until a later release).

This is good point. This can be easily solved by not putting things in this folder or reseriving some exclusion char in folder name, for instance _. So, _Doc will not be search.

Good point, but it's getting a bit too complicated for my taste. Since the function-searching will restrict files to *.ahk, maybe that will be enough of a filter/protection.

Another approach is to keep env var, like proposed AHK_LIB that will keep the paths to be searched. I beleive this approach is the best. If this env var does not exist default behavior can be recursion and if this var exists, AHK should search only paths referenced and like this it works in most other languages.

I considered this but it seems like overkill (too complicated) for the initial release. Also, recursive search overlaps with it somewhat, so maybe we should just stick to that (assuming users decide it's worthwhile).

I still think single underscore is better though, as users will have to stick forever with double score, for instance: MMenu__Create(), MMenu__Add()... or RemoteBuf__Open(). But if that makes you happy I wont argue, its trivial detail anyway.

Actually I was unhappy with double-underscore but it seemed the lesser evil than requiring the program to search again on a class name whenever a function-call containing an underscore doesn't exist as a file. Both performance and professionalism would be negatively impacted, but only if underscores will be common in non-class userlib and stdlib function names (maybe they won't be).

However, it has the drawback of adding a dependency to the registry or environment, which reduces the portability of scripts.

Not if there are some safe defaults where this env var is not present, like A_AHKPATH\Include.

It doesn't seem practical due to Vista and possibly other reasons.

Users that want their scripts portable should put their script in this folder or in Include-User folder.

I don't follow that. If it's important, please clarify.

Also, AHK can register this variable privatly, without adding it to the machine location for env vars. SO this var will always exists, either explicitely or implicitely. Something like this is done by for example Total Command which creates COMMANDER_PATH if it doesn't exist in the system. This didn't introduce portability problems with TC, contrary to that.

As far as I know, the only two types of persistent environment variables are "user" and "system". We could have a poll to decide whether this feature is necessary, and if so, whether it should be done via registry or environment (as far as I know, those are the only two choices other than some kind of INI file, which is probably a bad idea due to Vista protection).

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004
:oops: I edited my previous post without noticing that you had replied on the next page...

Chris
  • Administrators
  • 10727 posts
  • Last active:
  • Joined: 02 Mar 2004
No problem; I realized that and just re-read it.

The mechanism we discussed would only include .ahk files so other files with other extensions could be added to the directory without any issues.

Yes, I realized that too late. I just read your post.

The problem I see with allowing a registry entry to override the user path and or allowing the documents folder location to override the standard library location is in running a script from a removable device. ...if a user has a function with the same name in their user directory it will override a function that I have included on the memory drive in the /Include folder where autohotkey.exe resides. The same would be true for a registry entry. That's no good.

Then perhaps a directive could be provided that allows a script to turn off processing of user-includes, as you originally suggested. I think it's most essential to figure out what the default behavior should be, since that avoids the need for most scripts to put any lib-related directives at the top.

Forcing the standard library functions to override user functions might also be handy in cases where a system administrator might want to make a change to a function that could affect scripts for multiple users on a network without allowing the user permissions to override the changes.

That seems a bit of a stretch. I think we should focus on providing the default behavior that maximizes overall benefit to users. We just have to figure out what that is (public polls can help).

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004

Then perhaps a directive could be provided that allows a script to turn off processing of user-includes, as you originally suggested. I think it's most essential to figure out what the default behavior should be, since that avoids the need for most scripts to put any lib-related directives at the top.

That would work ok. Any suggestions for syntax for an additional directive so that I can add it to Ahkx_Include for fun?

That seems a bit of a stretch.

You are probably right and there are other methods that could be used for that purpose.

Chris
  • Administrators
  • 10727 posts
  • Last active:
  • Joined: 02 Mar 2004

Then perhaps a directive could be provided that allows a script to turn off processing of user-includes

Any suggestions for syntax for an additional directive so that I can add it to Ahkx_Include for fun?

I don't know, so you could just use something like #UserLib *None for now.

majkinetor
  • Moderators
  • 4512 posts
  • Last active: Jul 29 2016 12:40 AM
  • Joined: 24 May 2006

The problem I see with allowing a registry entry to override the user path and or allowing the documents folder location to override the standard library location is in running a script from a removable device. ...if a user has a function with the same name in their user directory it will override a function that I have included on the memory drive in the /Include folder where autohotkey.exe resides. The same would be true for a registry entry. That's no good.


Yes, this was the reason I said no MyDocs, no 10 places, just let the AHK look at its own folder. Simulating different setups is also easier this way, as different "installations" of AHK will use their own stdlib residing in their own include. Directive and MyDocs, IMO, complicates entire story. We have easy and light mechanism so far, I don't think it should be extended that way for the sake of simplicity. Having AHK looking only in its own folder for stdlib or user files is flexibile and every good thing you have with MyDocs can be also set up using what I propose. For instance, in multiuser environments each user can install/unpack its own AHK in its My Documents....
Portability is maintained with this, as if you run script from USB via AHK on the USB, it will use its own Include and will not mess with local computer installation.

This is necessary since Vista does not allow users permission to modify files in their /Program Files directory by default.

I don't know many things about Vista except it sucks :p The above sentece proves that if true. I don't know why, but I think ppl will not have problem with this, just a sense. Some forms of above limitation is present in all previous versions of Windows, just the domain was different - in Vista, its Program FIles, in XP SP2 Security utils forbids mass of things as soon as you install SP2, and there is file protections etc... People learned how to disalbe or circumvent all that sht. Maybe I'm wrong, just a thought...

Forcing the standard library functions to override user functions might also be handy in cases where a system administrator might want to make a change to a function that could affect scripts for multiple users on a network without allowing the user permissions to override the changes.

Such security measures are out of domain of stdlib. User func should override stdlib funcs always, that is normal behavior everwhere.

[Log is] good idea, but I think that it should be deferred until after the initial release, and only if there's demand.

Why not? If things are dynamic, we need to be able to see what is acctually included. Maybe some hidden tmp file in TEMP (undocumented ofc). There is no need to be more sophisticated then that as this is troubleshouting detail.
Anyway, this is not needed if you don't want to support recursion. Its easy to check manuely if problem exits with user function overriding stdlib function by accident.

As far as I know, the only two types of persistent environment variables are "user" and "system"

Well, technicaly yes.
BUT, variables AHK creates on its startup (and it MUST be executed in order to run ahk script) are also persistent as every script runns within AHK environment. So, where there is no AHK running, there is no variable, when AHK is running varialbe is there.





I agree that we should have 4 or 5 additional polls about concepts that aren't accepted by everybody so far.
Posted Image

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004

This is necessary since Vista does not allow users permission to modify files in their /Program Files directory by default.

I don't know many things about Vista except it sucks :p The above sentece proves that if true. I don't know why, but I think ppl will not have problem with this, just a sense. Some forms of above were present in all previous versions of Windows, just the domain was different - in Vista, its Program FIles, in XP SP2 Security utils forbid mass of things and there are file protections also, but ppl leanred how to disalbe or circuwent all that sht. Maybe Iam wrong, just a thought...

I understand what you're saying about wanting to use one location but Vista is not very leanient about user permissions by default and this can not be changed easily in most versions of Vista. Most versions of Vista do not allow User or Group policies to be edited either. You need to upgrade Vista to get the option. I get around this when necessary for older applications by running as administrator (since I am the only user and since an Administrator account on Vista is really just a PowerUser account - not the same as Administrator on other Windows versions - that has been renamed as TrustedInstaller - good luck hacking into it...) but it still presents limitations. If I want to use a script to organize and maintain my user defined functions then that script would have to be run as administrator to be able to modify files in the /Program Files directory. This also requires that I accept annoying "Continue" and/or "accept/deny" messages every time I launch the script. All this would be so much easier if the /MyDocuments/AutoHotkey/Include folder was used. If a user wanted to use a different location instead they could redirect the folder to somewhere else instead. The /MyDocuments/AutoHotkey/Include directory could be a virtual folder that points somewhere else so there would still be a way to use another folder instead if desired.

majkinetor
  • Moderators
  • 4512 posts
  • Last active: Jul 29 2016 12:40 AM
  • Joined: 24 May 2006
Why don't we use A_AHKPATH as default and override it with directive to point to anything that is needed for different Win flavors. So, if no directive is present it uses Include in AHK dir and directive can override this. So,

1. No #UserLib. The same as #UserLib %A_AHKPath%

2. #UserLib with no argument use MyDocuments. Or if no arguments troubles you, then explicit path as in 3, or some internal variable can be used, like #UserLib %A_UserInclude% (pointing to UserProfile\MyDocs\AutohotKey)

3. #UserLib <path> explicitely set path to any place desired. For instance #UserLib C:\Library.

Note that AHK should use 2 folders - 1 for std inc and one for user inc - in folder specified by #UserLib. So, for istance in scenario 3 it will use Include & IncludeUser from c:\Library. So, #UserLib specifes directory where library folders can be found.

I think that 1 will be most frequent scenario, users having limited Vista can just add 2., and 3. can be used as general solution.

The bottom line is that most users will never need to do anything, some smaller number will have to use 2. and 3 is there for custom uses.

Perhaps poll can solve this if you don't think the same.
Posted Image

majkinetor
  • Moderators
  • 4512 posts
  • Last active: Jul 29 2016 12:40 AM
  • Joined: 24 May 2006

Actually I was unhappy with double-underscore but it seemed the lesser evil than requiring the program to search again on a class name whenever a function-call containing an underscore doesn't exist as a file. Both performance and professionalism would be negatively impacted, but only if underscores will be common in non-class userlib and stdlib function names (maybe they won't be).

Lets examine this:

Let the user code be like:
...
MyClass_MyFunc
...

1. User declared function MyClass_MyFunc. So, Ahk doesn't search for it as it see it among list of declared functions (*).

2. User didn't declare function. AHK suppose it is library function
2.1. If it is, file MyClass_MyFunc.ahk or MyClass.ahk will exist
2.2. If not, its syntax error. ("whenever a function-call containing an underscore doesn't exist as a file." )
So, if we use _ instead __ and this makes some user functions with _ in name look like library classes (although they are not), the performance problem you are afraid for will come only in series of events finishing as 2.2, which is not very likely to happen.

The problem is (*) part as I don't know if that is true.
Posted Image

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004

Perhaps poll can solve this if you don't think the same.

I think might be a good idea. Another thing to possibly consider is that, although many users are not currently running Vista, this is likely to change within the next few years when XP support is also dropped. Also, I have been told recently the latest Win32 specification from Microsoft specifies (I haven't verified this yet but I've had a couple IT techs mentioned this to me recently) that user files and configuration related files that the application needs to modify should not be placed in the /Program Files folder. The suggested location for files created by a user is the MyDocuments folder. If this is the case, it would seem logical to use the /MyDocuments folder instead of supporting locations that were used in previous versions of Windows that are not supported in Vista.

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004

Lets examine this:

Let the user code be like:

...
MyClass_MyFunc
...

1. User declared function MyClass_MyFunc. So, Ahk doesn't search for it as it see it among list of declared functions (*).

2. User didn't declare function. AHK suppose it is library function
[list] 2.1. If it is, file MyClass_MyFunc.ahk or MyClass.ahk will exist

Unfortunately I have trouble understanding what you are trying to say at this point. Could you please try and explain a bit more? Why would either MyClass_MyFunc.ahk or MyClass.ahk matter? I had thought that we were in agreement that the filename included would be the exact function name with a .ahk extension. If that's the case then only MyClass_MyFunc.ahk would be relevant. I'm not sure I understand why MyClass_MyFunc could be confused with a Standard Library function if all Standard Library functions end up being prefixed with a _ character. If a prefix is used, the function name would have to be _MyClass_MyFunc to cause a potential conflict. This is why I had suggested using 2 _ characters to prefix internal functions (to be able to distinguish internal from external).

The poll seems to indicate so far that the _ prefix is not preferred though...

Maybe another poll could be created to see if anyone is concerned with the possibility of conflicts created from a user using the same name as a Standard Library function. Personally, after giving it a bit more thought, I'm not too concerned with the possibility of name conflicts and would prefer to use function names without having to specify a prefix for a Standard Library function. If the function is not listed in the help documentation and not declared in the script then it must be a Standard Library function...

majkinetor
  • Moderators
  • 4512 posts
  • Last active: Jul 29 2016 12:40 AM
  • Joined: 24 May 2006

Unfortunately I have trouble understanding what you are trying to say at this point.

You probably missed the start of the story, back above, to use _ as class separator as it will easy building of anything more complex then single fuction call.
It has nothing to do with _ as prefix for func names.
Posted Image

Chris
  • Administrators
  • 10727 posts
  • Last active:
  • Joined: 02 Mar 2004
Concerning userlib, I think we cannot default it to A_AhkPath because that is too often inside Program Files. It's neither practical nor professional. So we must default it to either "no userlib" or some writable folder somewhere.

1. User declared function MyClass_MyFunc. So, Ahk doesn't search for it as it see it among list of declared functions (*).

Yes this is true. If a script contains an explicit definition for a function (or one of its explicit #include files does), AutoHotkey won't ever search for that function externally. At least, that is the current plan, and it's also what fits best with the program's current design.

So if there is a call to a nonexistent function named Prefix_Suffix(), the program must first search for Prefix_Suffix.ahk in the userlib, then search for Prefix.ahk in the userlib, then do both steps again in the stdlib. So up to 4 searches would be required before the function is finally located. This will impact performance somewhat, especially if each search must recurse into subdirectories, and there are hundreds of files in those subdirectories.

corrupt
  • Members
  • 2558 posts
  • Last active: Nov 01 2014 03:23 PM
  • Joined: 29 Dec 2004
I didn't miss it but you didn't go into much detail and there are a few ways to work around the issue of functions (and maybe labels?) that might need to be added for a module to function. Would you mind giving a detailed example? I'm having a hard time picturing a scenario where there might be an issue.

One of the easiest methods might be to create an init function that a user must call somewhere in their code to enable the functionality in their script. For example, say you have a blahxx module that does some wonderful things and has a bunch of functions that heavily depend on each other but where all of them may not be called from the original script. All of these functions could be included in one file that could then be named _init_Blahxx.ahk. The file would then contain a _init_Blahxx() function but that function could just be a placeholder that does nothing except cause AutoHotkey to #Include the _init_Blahxx.ahk file containing all the necessary functions. So the user would need to call _init_Blahxx() near the beginning of the script before calling any functions from the _Blahxx module.

Edit: a need a spell checker script :(

majkinetor
  • Moderators
  • 4512 posts
  • Last active: Jul 29 2016 12:40 AM
  • Joined: 24 May 2006

Would you mind giving a detailed example?

Not at all. Remember MMenu ? Or ShellDriver ? Or RemoteBuffer. Ok, so those are pseudo classes. Lets take MMenu as good example.

MMenu has in its file according to what we addopted so far around 10 public functions which form menu API and 20 more "private" functions that are internal, and should not be used externanly. Network of dependencies is very dense - MMenu functions generaly use number of other MMenu functions. As a whole, it is a black box.

Now, with no class separator like proposed MMenu have to breaken into at least 10 files, each file having 1 MMenu API function and 1 file having additonal private functions. Plus at the end of every file I will have to put 9 includes. When I add new MMenu function in a feature, I will have to change 10 files to add this new func among includes. This makes maintance of this script very nasty thing, and without class mechanism I wouldn't add it to stdlib at all.

So, we need mechanism how to keep such "classes" as single files and class separator is one easy solution. It points this: if user wrote MMenu_Create(..), parser will first try to see if MMenu_Create.ahk exists. If not, it will try to see if part of it is the class function checking for class separator in func name - it will find MMenu to be class name and check for MMenu.ahk in library, and as it is there include it as a whole.

MMenu, ShellDriver, RemoteBuffer are just some examples. I created here at least 5 more such useful classes. Generaly, everything more complex then single function call will be hard to maintain if something like this isn't added. For instance, CoHelper is mostly made of set of functions that are used together (U must init and deinit, u must use VTable, StrtToGUID and so on..) It makes sense to add this using class so to avoid having 20 files just for COM wrappers. Even worst, those will be 20 files having 3 lines in them, as they are wrappers to native API.


This will impact performance somewhat, especially if each search must recurse into subdirectories, and there are hundreds of files in those subdirectories.

So with no recursion it is 4 look ups at worst with average 2 probably. Lets forget about recursion then.
I think that you maybe have a point with double underscore. We can make a rule that double _ is class separator and that it will be included as described above without looking for entire construct as file name. This will then be the same as for any other function, without performance issue. It doesn't look so nice, but is less evil then performance drawback. About recursion, it can be enabled in any case if some form of cashing is added . As standard library should not be changed by users (users are to use their own include folder), so this place will be mostly static (I can imagine smb change something to fix some eventual bug, or some anyoing thing, but this will be rare). You can create index.dat on installation, and check if Include dir is changed on startup (using modification date). If it is, you can rebuild index.dat if it is not (and in 99.9% of cases it will not be changed) u can use index.dat instead using File System functions. Using this file will make recursive search instant, and any other search.


Well, to be honest, whatever we choose as a solution here is better then droping this idea. We just need some mechanism to avoid brekaing more complex scritps. I also belive that "class" principle should be used in stdlib often, when developer presents number of functoins related to the same topic which are generaly used together. For instance, if you use RemoteBuffer it is obvious you will have to Open, Read Write and Close the buffer so it is good choice to add entire file imediately. Contrary if we have 50 string functions and we add them in class, this is different scenerio. User can only use String_Reverse(..) for example without any other function so in this case it would be better to use 1-1 system. But if we want, we can add all strings to the class too - perhaps we find out that ppl love string functions so much and there are not 50 but more like 15 functions. So if it is concluded that most ppl use many string function per project, we can put them in String class and load them all together (which will speed up things also as just 1 file is to be read)

One of the easiest methods might be to create an init function that a user must call somewhere in their code to enable the functionality in their script.

INIT function will defintiely be used but it can not solve above issue as if there are no classes, parser will expect each module func to be in separate file. However, INIT function must be used for modules requiering action in autorun section. This has nothing with stdlib design. Once we determine stdlib interface, I will write article how to write stdlib functions the good way (what procedures to follow, how to name things, what to do and what not to do) so INIT will be part of that story. It can also be read by users that want to write their own functions in stdlib comptabilite way, so they can propose their good scripts to be added to official stdlib (assuming they are correctly written and useful)

Edit: a need a spell checker script

I can send you some that work everywhere
Not the script though :D
Posted Image