Typed properties - experimental build available

[lexikos]

As I've mentioned, I plan for v2.1 to include typed properties:

The primary purpose is basically "struct support", but this would be more flexible than (just) having a dedicated struct type. Instances of a class with typed properties would have additional space allocated for them, coinciding with how C handles structs, so they can fill that role. The definition or redefinition of typed properties would not be permitted after creating the first instance. Aside from their use with external functions, classes utilizing typed properties could be more memory-efficient, more robust, and probably faster to initialize and delete.
Source: Potential priorities for v2.1 - AutoHotkey Community

I have uploaded a build (not even officially an alpha/pre-release yet) to experiment with.

AutoHotkey_2.1-alpha.1.1.zip (SHA256 hash)
AutoHotkey_2.1-alpha.1.2.zip (SHA256 hash) - 2023-07-21
AutoHotkey_2.1-alpha.2.1+ge0d87105.zip (SHA256 hash) - 2023-08-18

• Source code is on the ptype branch.
• Documentation not included.
• Syntax and behaviour may be substantially different in future releases.

Dynamic Properties

Properties are added to a prototype object with .DefineProp(Name, {Type: propType}).

Currently propType can be the name of a basic type, or a reference to a class (or object with a Prototype property) which has typed properties.


Typed Property Declarations

Properties can be declared in the class body with either name : propType or name : propType := initializer. I wasn't sure about this syntax, but couldn't think of anything better, and it sure is better than using static __new and DefineProp.

Currently propType can be the name of a basic type (without quotes), or the name of a previously-defined class which has typed properties.

Referring to a class which hasn't yet been defined might be possible in future.


GetOwnPropDesc

If the property is a typed property definition, the descriptor has two properties:

• Type contains the type name or class object.
• Offset contains the offset of the property, in bytes.

Basic Types

Currently only numeric types are implemented.

• Signed integer types are currently "i" followed by size in bits, e.g. i32.
• Unsigned integer types are currently "u" followed by size in bits, e.g. u32.
• Floating-point types are "f" followed by size in bits.
• There is no type name for pointer-sized integers yet.

These type names are not final. I suppose that defining actual classes for these types would make things more consistent.


Complex Types

The current implementation is focused on struct support, so properties declared as having an object type are assumed to be nested structs, with the nested struct's data being embedded directly in the outer struct.

• Each nested object is automatically constructed and __new() is called without parameters.
• Nested objects are constructed in order of definition, and destructed in reverse order, after the outer object (but before it is actually deleted).
• If a nested object defines the __value property, it is returned or invoked instead of returning the nested object itself. For instance, with a property declared as StructProp : StructClass, getting or setting this.StructProp invokes the non-static __value getter or setter defined in StructClass. This allows classes to be used to implement various types, such as different kinds of strings or smart pointers.
• A reference to any nested object is sufficient to keep the outer object alive, without the internal circular reference causing any issues.
• Currently only classes with typed properties can be used as property types; e.g. m : Map is not supported.

Own Properties

Typed properties currently aren't considered own properties, because the "property name: value" pair exists only in the prototype.


Object Internal Data

Currently structured data is allocated separately to the Object itself and held in a pointer field. Aside from being simple to implement, this was initially intended to allow a class with typed properties to wrap around an external struct pointer. Supposing that this pointer field will always exist, I thought to also allow the script to make use of it to store an arbitrary 32 or 64-bit value, avoiding the need for an additional memory allocation.

However, I will most likely change this. The memory allocation of the object itself could be expanded to include the structured data. The address of the structured data would be calculated from the object pointer and information in the prototype. This would avoid increasing the size of objects which lack typed data. For objects with typed properties, it would reduce memory fragmentation and improve memory locality. External struct pointers would instead be handled by a new pointer-to-struct metaclass.

Currently the following experimental functions exist:

• ObjAllocData(Obj, Size) allocates Size bytes and stores the address in the object's data pointer field.
• ObjFreeData(Obj) immediately frees data allocated by ObjAllocData (it would otherwise be freed when the object is deleted).
• ObjGetDataPtr(Obj) gets the object's data pointer (also valid if the object has typed properties).
• ObjGetDataSize(Obj) gets the size which was passed to ObjAllocData (if that wasn't called, it returns 0).
• ObjSetDataPtr(Obj, Ptr) sets the object's data pointer. The script may use ObjGetDataPtr to retrieve it. The value is not required to be a valid pointer, unless Obj has typed properties, in which case it had better point to a valid struct. ObjSetDataPtr does not affect nested objects, as they each have their own data pointer (which points into the outer object's original data).

Alignment

Typed properties should be aligned consistent with default MSVC packing. There is currently no way to override this.


Sizeof

There is no built-in function yet, but it can currently be implemented like this:

Code: Select all

sizeof(classobj) {
    sizer := {}
    sizer.DefineProp 'a', {type: classobj.HasProp('Prototype') ? classobj : {Prototype: classobj}}
    sizer.DefineProp 'b', {type: 'i8'}
    return sizer.GetOwnPropDesc('b').Offset
}

This wasn't supposed to work when classobj is an instance of a class with typed properties, but currently it does. (DefineProp should reject the object, but instead it takes the type information from its base.)


Restrictions

Some restrictions and safety checks are not yet implemented. (I might elaborate later.)


DllCall

An object must still implement the Ptr property to be passed to a DllCall Ptr parameter. Eventually DllCall (or a more efficient alternative which can compose or define functions at script startup) might take a class as a parameter type, in which case the class would control how the parameter is passed.

[V2User]

Congratulations!
If this feature implemented, multithread may become easier because we can know the type of a property in advance without reading its value.

[crocodile]

Thanks, this is awesome!
Would it be possible to add some examples? Like trying to replace the structures involved in GDIP.AHK.

[iseahound]

Code: Select all

#Requires AutoHotkey v2.0

class struct  {
   blue  : u32 := 0xFF
   green : u32 := 0xFF00
   red   : u32 := 0xFF0000
}

instance := struct()
MsgBox instance.blue

Just from reading your first post, I gander that this is the way to define a struct. blue in (:) type u32 is (:=) 0xFF

[lexikos]

@iseahound yes.

At runtime,

Code: Select all

struct := Class()
struct.base := Object
struct.Prototype := {}
struct.Prototype.DefineProp('blue', {type: "u32"})
struct.Prototype.DefineProp('green', {type: "u32"})
struct.Prototype.DefineProp('red', {type: "u32"})
struct.Prototype.DefineProp('__init', {call: this => (
        this.blue := 0xFF,
        this.green := 0xFF00,
        this.red := 0xFF0000
    )})

... although it would by atypical to have a 96-bit colour value.

Currently the you must either instantiate the object via Object.Call (i.e. struct() as struct inherits Call from Object), or determine the size correctly and "manually" allocate or set the pointer.

Code: Select all

instance := {base: struct.Prototype}
ObjAllocData(instance, 12)
instance.__init()
MsgBox instance.blue

[just me]

DllCall

An object must still implement the Ptr property to be passed to a DllCall Ptr parameter.

Would you please post an example?

[ntepa]

I got this to work, but how do I create a MONITORINFOEX struct?

Code: Select all

class RECT {
    Left : i32
    Top : i32
    Right : i32
    Bottom : i32
    Ptr => ObjGetDataPtr(this)
}

class MONITORINFO {
    cbSize : i32 := 40
    rcMonitor : RECT
    rcWork : RECT
    dwFlags : i32
    Ptr => ObjGetDataPtr(this)
}

DllCall("GetCursorPos", "uint64*", &point:=0)
hMonitor := DllCall("MonitorFromPoint", "int64", point, "uint", 0, "ptr")
DllCall("user32\GetMonitorInfo", "ptr", hMonitor, "ptr", MI := MONITORINFO())
MsgBox(
(
    "Monitor info from point:
    Left: " MI.rcMonitor.Left "
    Top: " MI.rcMonitor.Top "
    Right: " MI.rcMonitor.Right "
    Bottom: " MI.rcMonitor.Bottom "
    WALeft: " MI.rcWork.Left "
    WATop: " MI.rcWork.Top "
    WARight: " MI.rcWork.Right "
    WABottom: " MI.rcWork.Bottom "
    Primary: " MI.dwFlags
))

[lexikos]

@ntepa That would be difficult as I haven't implemented arrays (or fixed-size strings) yet.

ObjAllocData lets you attach a memory allocation to a live object, but I haven't implemented any way to specify the size of a struct definition other than to repeatedly define properties. I suppose that would best be done by defining an array of bytes, if there is no more appropriate type. Character arrays for strings are quite common, so there should probably be built-in support for that.

[iseahound]

From the above post:

Code: Select all

class rectangle {
   x : i32
   y : i32
   w : i32
   h : i32
   ptr => ObjGetDataPtr(this)
   size := 16
}

DllCall("GetClientRect", "ptr", WinExist("A"), "ptr", Rect := rectangle())
MsgBox rect.x ", " rect.y ", " rect.w ", " rect.h

Seems to interoperate with NumGet

Code: Select all

class rectangle {
   x : i32
   y : i32
   w : i32
   h : i32
   ptr => ObjGetDataPtr(this)
   size := 16
}

DllCall("GetClientRect", "ptr", WinExist("A"), "ptr", Rect := rectangle())
   left   := NumGet(Rect, 0, "int")
   top    := NumGet(Rect, 4, "int")
   width  := NumGet(Rect, 8, "int")
   height := NumGet(Rect, 12, "int")

MsgBox left ", " top ", " width ", " height

[thqby]

Code: Select all

class cls {
	static m := Map(),m.CaseSense :=false
}

If there is a space before :=false, a syntax error is displayed

[lexikos]

I considered some possibilities for implementing basic array types:

1. A "light-weight" approach where a property can be an array of primitives, but without a corresponding object. Accessing the property without [index] would return the raw address. More advanced behaviour would be implemented by creating a custom prototype with a single typed array property. String types would also be implemented, so for instance, the script would access MONITORINFOEX::szDevice like any string property, without an object existing for that property.

2. Create a basic object for each array, with Ptr/Size and a small subset of Array functionality. Custom subtypes can extend this instead of nesting it, but reserving space within a larger type would imply the construction of a nested (byte array) object.

3. A way just to reserve space within the struct definition (not an instance). A script can build on this to implement basically anything.

In any case, the class syntax needs some way to express arrays and/or strings with fixed-size buffers. Simply parsing a suffix like [32] at load-time would be a start, but it wouldn't allow for "constants" like in CHAR szDevice[CCHDEVICENAME]. It's also unclear how type declarations could allow for parameters like string encoding. Defining the entire structure at runtime with DefineProp allows more flexibility, but DefineProp is very verbose, and it is difficult to mix DefineProp with class declarations in alpha.1.1, especially due to the way the structure is locked when the class is extended.

The solution I've come up with turned out to be very simple: don't define the structure as each declaration is parsed. Instead, each type declaration inserts a line into static __Init(). This also gives more flexibility with ordering of classes, although you must still be careful with cyclic dependencies.


v2.1-alpha.1.2

See top post for the download link.

Type declarations previously allowed only a single identifier/name. This can now be followed by a chain of .property. Each identifier may be followed by (...) or [...], or the expression can begin with a parenthesized sub-expression. The type expression is evaluated in the context of static __Init(); i.e. this refers to the class, while the value initializer is still evaluated in the context of (instance) __Init().

A property type can now be an integer (which currently must be non-zero), to reserve this number of bytes. (I'm not sure whether I will keep this once typed arrays are natively implemented.) Invoking the base property returns its address. For example, a struct containing only ptr : 32 has a size of 32 bytes, and this.ptr returns the struct's address.

ObjGetDataSize now returns the struct size if the object's data wasn't allocated with ObjAllocData. It can be used like ObjGetDataSize(RECT()) or ObjGetDataSize(RECT.Prototype).

For example, a C-style string can be implemented like this:

Code: Select all

class CString {
    static Call(n, cp:="UTF-16") {
        p := {base: this.Prototype}
        p.UnitSize := StrPut("", cp)
        p.Codepage := cp
        p.Size := n * p.UnitSize
        p.DefineProp('Ptr', {type: p.Size})
        return {Prototype: p}
    }
    __value {
        get => StrGet(this,, this.Codepage)
        set => StrPut(value, this,, this.Codepage)
    }
}

which can be utilized like this:

Code: Select all

class XStruct {
    str : CString(32, "UTF-8")
}

x := XStruct()
x.str := "Hello!"
MsgBox x.str

If you wanted to define a type like CHAR[CCHDEVICENAME], you could do that by defining a static __Item[n] property in class CHAR.

MONITORINFOEX can be implemented like this:

Code: Select all

class Struct {
    Ptr => ObjGetDataPtr(this)
    Size => ObjGetDataSize(this)
}

class RECT extends Struct {
    Left : i32
    Top : i32
    Right : i32
    Bottom : i32
}

class MONITORINFO extends Struct {
    cbSize : i32 := this.Size  ; Note that this value is different between MONITORINFO and MONITORINFOEX.
    rcMonitor : RECT
    rcWork : RECT
    dwFlags : u32  ; DWORD is unsigned.
}

class MONITORINFOEX extends MONITORINFO {
    static CCHDEVICENAME := 32
    szDevice : CString(this.CCHDEVICENAME)
}

DllCall("GetCursorPos", "uint64*", &point:=0)
hMonitor := DllCall("MonitorFromPoint", "int64", point, "uint", 0, "ptr")
DllCall("user32\GetMonitorInfo", "ptr", hMonitor, "ptr", MI := MONITORINFOEX())
MsgBox(
(
    "Monitor info from point:
    Left: " MI.rcMonitor.Left "
    Top: " MI.rcMonitor.Top "
    Right: " MI.rcMonitor.Right "
    Bottom: " MI.rcMonitor.Bottom "
    WALeft: " MI.rcWork.Left "
    WATop: " MI.rcWork.Top "
    WARight: " MI.rcWork.Right "
    WABottom: " MI.rcWork.Bottom "
    Primary: " MI.dwFlags "
    Name: " MI.szDevice
))

@thqby That bug was fixed.

[V2User]

I considered some possibilities for implementing basic array types:

@Lexicos
There is a Lib for fastest array developed by MonoEven, which has been published in our country's comunity. You are welcome to come here.

[thqby]

Code: Select all

	if (!_tcsicmp(_T("uptr"), aName))
		return MdType::IntPtr;
	if (!_tcsicmp(_T("iptr"), aName))
		return MdType::UIntPtr;

There is a bug, uptr and iptr are reversed.

Code: Select all

struct {
  char a;
  struct {
    char b;
    int c;
  }
}

Nested anonymous structures similar to the above are not currently supported, and I find it difficult to introduce this feature in the current syntax.

Maybe so?

Code: Select all

class struct {
  a:i8
  class {
    b: i8
    c: i32
  }
}

[V2User]

Maybe so?

Code: Select all

class struct {
  a:i8
  class {
    b: i8
    c: i32
  }
}

Perhaps this could:

Code: Select all

class struct1 {
  a:i8
  cc:stuct2
}
class struct2{
 b:i8
 c:i32
}

[lexikos]

Re uptr/iptr: thanks. It doesn't actually make any difference on x64...

A class definition does not and should not imply the definition of an instance property, and therefore should not affect the structure of an instance.

Unless I'm missing something, without unions, a nested anonymous structure is just needless complication with no purpose. There is no support for unions, and even if there was, you can get the right overall structure by naming the property.

Unions are a low-level concept that I think doesn't really fit into the general "typed properties" concept. There will probably be predefined classes for structs and unions, when it becomes more clear what unique behaviour these would have that other objects would not.

Actually, a union type can be created manually, using an approach similar to my CString example.

[Helgef]

The primary purpose is basically "struct support", but this would be more flexible than (just) having a dedicated struct type

I guess I am narrow-minded, but what is the purpose of something more flexible? I would expect stuct support to be as close as possible to struct syntax, consistent with dllcall syntax, eg,

Code: Select all

struct a {
	int x
	uint y
}

I would reserve the use of a :, for bitfields.

Again, I'm probably just narrow minded, but I don't see any benefit of the added flexibility, just added complexity, distracting from the primary purpose.

I like the idea of allowing same-line initialisers. If a struct also allows method definitions, I suppose that could be useful too.

Cheers.

[iseahound]

Other than it stemming from C, some examples where the types are in "reversed" order:

SQL

Code: Select all

Create Table Persons (
  PersonID int DEFAULT 0,
  Name varchar(255),
);

Go

Code: Select all

var (
	ToBe   bool       = false
	MaxInt uint64     = 1<<64 - 1
	z      complex128 = cmplx.Sqrt(-5 + 12i)
)

Rust

Code: Select all

struct Point {
    x: i32,
    y: i32,
}

impl Default for Point {
    fn default() -> Self {
        Point {
            x: 0,
            y: 0,
        }
    }
}

Python

Code: Select all

from dataclasses import dataclass
@dataclass
class Point:
    x: float
    y: float
    z: float = 0.0

Typescript

Code: Select all

interface Person {
  name: string;
  age: number;
}

I can't say why one convention is chosen over the other

[lexikos]

what is the purpose of something more flexible?

It is more useful, almost by definition.

I was referring to:

• The ability to have a structured class which is not derived from a specific "struct" base class. For instance, WinRT has "structs", enums, delegates and runtime objects. Objects implementing any of those will need structure, but maybe we want this is Struct to be true for only the first one.
• The ability to make use of structure without having to use a nested "struct" object (in the case where an object can have properties and methods but not structure, and a structure cannot have non-structured properties or methods).

If you are only creating simple structs for the purpose of passing to DllCall, this probably won't matter to you. But having structs, which are essentially just objects with a predefined set of type-restricted properties, why would we not use them for other things? Do I need to explain the benefits that can come from type annotations, or type checking, or data structures?

One example of use outside of DllCall:

I'm particularly interested in the structs, as I have a lot of large data structures that I've designed as specialized array-type "data classes" to conserve memory.

Maybe a simple struct definition like what you show would be sufficient in sirksel's case, maybe not.

I would expect stuct support to be as close as possible to struct syntax, consistent with dllcall syntax, eg,

How does that even relate to flexibility?

That is C syntax for structs, minus the semicolons, not some ubiquitous "struct syntax". It makes sense in C, where declarations of various kinds work the same way.

I chose to leave DllCall syntax as is in v2.0 so as to not delay the release further, and because I was planning this, which will tie into DllCall's replacement (see Importing native functions). The DllCall type names are nonsense and not even self-consistent. "Char" for an 8-bit integer? "Short" but no "long"? I'm not sure that I'll keep the single-letter-number names, but whatever I settle on, DllCall will support. So ultimately structs or typed properties will be consistent with DllCall syntax...

I chose name : type because it is the only syntax that I could see potentially working for type declarations/annotations outside of classes. I am looking toward the (distant) future.

I would reserve the use of a :, for bitfields.

I can't think of a single time that I've used that feature of C/C++, and I'm not sure that I'll bother to implement them. I have not considered them at all while implementing this so far, and am unsure how they would work. However, if someone wants to implement a struct to make dealing with bitfields easier (where one struct represents a set of bitfields), they can do that now, and potentially specify the types and names of the fields within the type declaration (like my CString example). This is flexibility.

... just added complexity, distracting from the primary purpose.

"The current implementation is focused on struct support" doesn't mean that "making it easier to define simple structs to pass to DllCall" is the (or my) primary purpose, even if that is what most users will be using it for.

What complexity is it that you think was added, and why do you think it was added?

Every native AutoHotkey value that has methods or properties must support HasProp. This must include struct fields. Meeting this requirement is simpler if struct fields are properties. I did consider generating a getter/setter for each field instead of adding a new kind of property, but then there's no obvious way to identify the type of a field at runtime, or retrieve its offset or address. Fields would not seem like an actual part of the language.

Properties and methods can be added to any value (except COM objects) via the appropriate Prototype, regardless of whether the value is an Object. Some of the mechanisms I imagined specifically for use with DllCall (or its alternative) rely on the ability to define a property or method, such as the already-implemented __value. Given that we already have method and property definitions, class variable initializers and all that, it seemed like a no-brainer for structs to get all of that.

I'm not sure that limiting typed properties to only struct classes would reduce complexity.

If a struct also allows method definitions, I suppose that could be useful too.

I considered it a requirement. Some structs require specific initialization. Many structures require cleanup.

With the approach I took, I didn't have to implement same-line initializers or method definitions, because they already existed. I only had to implement the runtime behaviour of calling __init, __new and __delete for structs.

As I said, this is "not even officially an alpha/pre-release yet". I uploaded it to get some feedback (so thanks for providing it) while I do some work on other projects. It is far from complete in any respect. I wasn't sure whether the approach would even work, or exactly what behaviour might make sense for "structs" and not objects in general, or any other drawbacks. That's what I hoped to find out by making this and uploading it. Maybe typed properties will end up getting restricted to structs, but there would have to be a good reason to do so, and so far I have encountered none.

[iseahound]

Regarding the extension of : to other types—how would this interplay with struct support? If I define a : MyClass is the pointer to the class stored in a specific place? Is this just exposing how autohotkey normally stores internal references to objects?

[lexikos]

The current implementation is focused on struct support, so properties declared as having an object type are assumed to be nested structs, with the nested struct's data being embedded directly in the outer struct.

Where and how the reference to the nested object is stored is unspecified. It is not in the "struct", because that would prevent nested structs from working as expected.

I'm assuming "the pointer to the class" is supposed to be "the pointer to the instance of the class". The former is not stored in the instance at all.