Object Detection Data Tracking Script for AHK - Help

[prototype_zero]

Hello everyone,

Fairly new to AHK but have used it to make a few simple scripts. I had this idea and I was interested in seeing if it actually works.

I have a object detection script in python that writes the object bounding box coordinates (xmin, xmax, ymin, ymax) to a text file. In real time, I wanted to have an AHK script read that file while its being written, and moving the mouse to track the center of the object bounding box in real time as it reads the text file.

When I use the mousemove function, the mouse seems to be moving to the target coordinates properly as it reads them line by line. However, when I use the DLLCall mouse_event function, the mouse moves all over the screen or in erratic ways, or ways that just don't match the coordinates. Moreover, because this is designed to work in an FPS, I need to use the DLLCall mouse_event function so that the game actually registers the inputs.

I recently came across this thread which addresses the central problem in making such scripts for FPS games, which is that they mess with the mouse cursor (Keep on moving it to middle of screen etc) or other limitations (eg when the cursor reaches the edge of the screen, you cannot generate any more input in that direction).
Games that do not operate by mouse cursor (eg FPS) generally read the mouse by "Delta" information - that is, the amount that it moved in each direction since the last update. But I'm not sure exactly how to implement that into my script below.

viewtopic.php?f=19&t=10159

Any help on getting this to work would be very much appreciated as I've been wracking my brain quite a bit.

Code: Select all

#NoEnv
#SingleInstance, Force
#Persistent
#InstallKeybdHook
#UseHook
#KeyHistory, 0
#HotKeyInterval 1
#MaxHotkeysPerInterval 127

SetTitleMatchMode, 2 ; Allow for partial title matches

; Main loop
Loop {
    ; Read the content of the input file
    FileRead, ObjectData, F:\Prototype Zero\Archetype\Direct Link\object_data.txt

    ; Split the content by lines
    Loop, Parse, ObjectData, `n, `r
    {
        ; Split the line by comma to extract coordinates
        CoordArray := StrSplit(A_LoopField, ",")
        
        ; Check if the array contains at least 6 elements (xmin, xmax, ymin, ymax, confidence, id)
        if (CoordArray.Length() >= 6) {
            ; Extract coordinates
            xmin := CoordArray[1]
            xmax := CoordArray[2]
            ymin := CoordArray[3]
            ymax := CoordArray[4]
            
            ; Calculate the center of the bounding box
            center_x := (xmin + xmax) // 2
            center_y := (ymin + ymax) // 2
           
            ; Get the current mouse position
            MouseGetPos, MouseX, MouseY
                    
            ; Calculate the offset between current mouse position and target position
            OffsetX := center_x - MouseX
            OffsetY := center_y - MouseY

            ; Show tooltip with target coordinates, offset, and current mouse position
            ToolTip, Target coordinates:`nX: %center_x%`nY: %center_y%`nOffsetX: %OffsetX%`nOffsetY: %OffsetY%`nCurrent Mouse:`nX: %MouseX%`nY: %MouseY%
            
            ; Move the mouse cursor relative to its current position
            DllCall("mouse_event", uint, 1, int, OffsetX, int, OffsetY, uint, 0, int, 0)
        }
    }
}

return

; Hotkeys for Control
Pause:: Pause
^Esc:: ExitApp

Sample object detection data
xmin, xmax, ymin, ymax, confidence, ID

545, 864, 720, 926, 0.61, 1
545, 861, 721, 925, 0.56, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 723, 924, 0.52, 1
545, 861, 722, 924, 0.52, 1
546, 861, 722, 924, 0.52, 1
547, 860, 722, 923, 0.51, 1
547, 860, 722, 923, 0.51, 1
546, 860, 722, 923, 0.51, 1
546, 860, 722, 923, 0.51, 1
546, 860, 722, 923, 0.51, 1
546, 860, 722, 923, 0.51, 1
546, 860, 722, 923, 0.51, 1
546, 861, 722, 923, 0.51, 1
545, 861, 723, 923, 0.51, 1
547, 860, 722, 923, 0.50, 1
585, 847, 716, 920, 0.60, 1
611, 900, 698, 940, 0.71, 1
646, 847, 576, 771, 0.76, 1
360, 1013, 558, 1364, 0.65, 2
580, 762, 571, 770, 0.64, 1
653, 845, 877, 1062, 0.53, 2
600, 2199, 429, 1390, 0.57, 2
1232, 1331, 790, 967, 0.55, 2
1250, 1379, 732, 1039, 0.72, 2
1249, 1553, 466, 1116, 0.80, 2
1322, 1466, 460, 584, 0.77, 1
1854, 1941, 490, 616, 0.50, 2
1927, 2019, 452, 562, 0.51, 2
2095, 2216, 378, 543, 0.57, 2