Vis2 - Image to Text OCR()

Post your working scripts, libraries and tools for AHK v1.1 and older
GeneBene

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 02:00

iseahound wrote:@GeneBene Did you download the latest version from https://github.com/iseahound/Vis2/archive/master.zip ? Is there an error message? I just tested Vis2 again on Windows 10 64 bit, new install.
downloaded again and it works great, tried it on images with not that clear text and it still worked, great work :superhappy:
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 10:35

Just a quick comparison of Vis2 and Capture2Text on text from the Vis2 github page:
Original text on Vis2 github page (large font):
Vis2 - OCR()
(Capture2Text)
Q’JViSZ - OCRO
(Vis2)
Vis2 - OCROQ
Regards,
burque505
ferryiol

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 14:45

Very accurate

in the git you write

"Press Alt + Space to get the coordinates of the grey rectangle."

When I do this on windows 10 some little text popups up on upper right of screen that says x y h w but it has no data

can you tell me the steps to make it work ?
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 15:38

@ferryiol I changed

Code: Select all

               Vis2.Graphics.Subtitle.Draw("x: " this.x1() " │ y: " this.y1() " │ w: " this.width() " │ h: " this.height()
to

Code: Select all

Vis2.Graphics.Subtitle.Draw("x: " x " │ y: " y " │ w: " w " │ h: " h
on Github, does latest version fix it?
drozdman
Posts: 78
Joined: 05 Dec 2015, 01:07

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 18:29

This is really elaborate. Great work.
ferryiol

Re: Vis2 - OCR() - Update: Additional Language Support

31 Mar 2018, 21:24

iseahound wrote:@ferryiol I changed

Code: Select all

               Vis2.Graphics.Subtitle.Draw("x: " this.x1() " │ y: " this.y1() " │ w: " this.width() " │ h: " this.height()
to

Code: Select all

Vis2.Graphics.Subtitle.Draw("x: " x " │ y: " y " │ w: " w " │ h: " h
on Github, does latest version fix it?
thank you, works great
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR() - Update: Additional Language Support

04 Apr 2018, 03:39

UPDATE: Added ImageIdentify() - use it to identify objects in images. Is it useful? probably not. But it is fun to use.

USER WARNING Abuse of ImageIdentify() will get you temporarily IP banned by Google. (<12 hours) If you would like to continue using it after the ban, please sign up for a free google account and get a Cloud Vision API key and input it into the popup. Free 1000 images per month I think.

; Cloud Platform Console Help - Setting up API keys
; Step 1: https://support.google.com/cloud/answer/6158862?hl=en
; Step 2: https://cloud.google.com/vision/docs/before-you-begin
Last edited by iseahound on 04 Apr 2018, 13:02, edited 1 time in total.
bhanson
Posts: 3
Joined: 04 Apr 2018, 10:16

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 12:00

Hi,

I am also having troubles on Windows 10 Home 64-bit.

I have downloaded the latest version from https://github.com/iseahound/Vis2/archive/master.zip

It works in interactive mode, but when I try to use the other methods I get the blank box problem.

Code: Select all

#include <Vis2>  ; Equivalent to #include .\lib\Vis2.ahk


MsgBox % OCR("test.jpg")  ; << Does not work

#c:: OCR()   ; <<  Works perfect!
Esc:: ExitApp
I also tried the WinTitle (using notepad) and URL methods, which also did not work.

I ran the DetectBugs script posted earlier in this thread and I get this error:

https://i.imgur.com/0TZQk5P.png

From here, I did some further digging and this is what I found:

Code: Select all

; Line 2140 of Vis2.ahk
_cmd .= q this.tesseract q " --tessdata-dir " q fast q " " q in q " " q SubStr(out, 1, -4) q
%out% contains only the filename without extension, so the SubStr does not remove the extension (as I think intended?) and instead shortens the actual name.

  • Vis2_screenshot2f379028-4dde-47c6-b9f6-9b3713be8123.bmp
    Vis2_preprocess2f379028-4dde-47c6-b9f6-9b3713be8123.tif
    Vis2_text2f379028-4dde-47c6-b9f6-9b3713be.txt
Is an example of what gets created on my system.

Then %out% is looking for the full file name and this of course does not exist so the exception is thrown.

If I remove the SubStr and append the .txt to %out% then I can get DetectBugs to run.

Code: Select all

_cmd .= q this.tesseract q " --tessdata-dir " q fast q " " q in q " " q out q
;_cmd .= q this.tesseract q " --tessdata-dir " q fast q " " q in q " " q SubStr(out, 1, -4) q

; (...snip...)

out := out . ".txt
And my results are:
  • No text file found. #1
    Test2: No text file found. #2
Regards. :)

[edit] Also, when I run my main script again:

Code: Select all

#include <Vis2>  ; Equivalent to #include .\lib\Vis2.ahk


MsgBox % OCR("test.jpg")  ; << Does not work

#c:: OCR()   ; <<  Works perfect!
Esc:: ExitApp
The file that gets created in my temp has two extensions:

Vis2_text38ed24b5-3fc9-4feb-8a57-b4057148aee2.txt.txt
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 12:30

I'm getting the blank box again on Win7, 64-bit also. Version from 3-31-18 still working fine :D
About a missing extension, though - the intermediate file has no extension, it gets processed to text later. That's why your mod is giving you a file with .txt.txt.
Here's what it looks like on my system (in the %temp% folder, after running the 3-31-18 working version):
IntermediateFile.PNG
IntermediateFile.PNG (1.9 KiB) Viewed 6463 times
Regards,
burque505
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 12:59

@bhanson @burque505 It should be fixed now. https://github.com/iseahound/Vis2

EDIT: Also ImageIdentify() will work with input files now. Like ImageIdentify("test.jpg")
Last edited by iseahound on 04 Apr 2018, 13:24, edited 1 time in total.
bhanson
Posts: 3
Joined: 04 Apr 2018, 10:16

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 13:06

iseahound wrote:@bhanson @burque505 It should be fixed now. https://github.com/iseahound/Vis2
Working! :D
bhanson
Posts: 3
Joined: 04 Apr 2018, 10:16

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 14:15

I'm getting an error trying to call OCR() on the same bitmap with different coordinate options.

Running OCR() on a bitmap using the 3rd parameter as coordinates seems to destroy the original bitmap.

Code: Select all

#include <Vis2>
; start Gdip:

if !pToken := Gdip_Startup()
{
   MsgBox, 48, gdiplus error!, Gdiplus failed to start. Please ensure you have gdiplus on your system
   ExitApp
}

bmp := Gdip_CreateBitmapFromFile("test.jpg")


; This works
;MsgBox % OCR(bmp)
;MsgBox % OCR(bmp, , [0, 0, 1000, 120])


; Does not work
MsgBox % OCR(bmp, , [0, 0, 1000, 120])
MsgBox % OCR(bmp)


; Also does not work (what I'm trying to do)
;MsgBox % OCR(bmp, , [0, 0, 1000, 120])
;MsgBox % OCR(bmp, , [0, 138, 1000, 120]
Is this intended?

[edit] I know of course I can clone the portion of the image I want to process before sending to OCR(), but I guess I expected OCR to not modify the source input.
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 17:02

No that is an oversight of mine. It is fixed on GitHub.
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 17:21

Some ImageIdentify() examples:

Code: Select all

MsgBox % ImageIdentify("https://i.imgur.com/ffnoUVR.jpg")
modern art (90%), art (87%), painting (69%), visual arts (61%), paint (61%), acrylic paint (61%), artwork (60%), graphics (57%), graphic design (51%)

Code: Select all

MsgBox % ImageIdentify("https://i.imgur.com/ffnoUVR.jpg",,[113,159,151,91])
orange (92%), fish (92%), product (74%), fish (60%), animal figure (56%)

Fun to play around with. There's a lot of potential. General Question: How are you using Vis2? For me it's basically a toy project. I prefer the GUI, since it allows me to extract text from websites that disallow text selection.
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 18:38

Hi, iseahound, this code caused AHK_L 64-bit, 1.1.28.01, to crash, and my Win7 SP1 64-bit computer to lock up completely:

Code: Select all

MsgBox % ImageIdentify("https://i.imgur.com/ffnoUVR.jpg")
Tried it three times, same result, not really ready to do it again :)
Regards,
burque505
iseahound
Posts: 1434
Joined: 13 Aug 2016, 21:04
Contact:

Re: Vis2 - OCR(), ImageIdentify()

04 Apr 2018, 19:49

I think it works now. I commented out ObjRelease(pStream) in 3 areas, but I don't know why it's causing crashes. Does RtlMoveMemory delete the source data?
robodesign
Posts: 934
Joined: 30 Sep 2017, 03:59
Location: Romania
Contact:

Re: Vis2 - OCR(), ImageIdentify()

05 Apr 2018, 11:31

iSeaHound: I just tested the script on Win 10 x64 and it works well.

I am having eye-sight issues and I am very interested in such a tool, but I would like the process simplified / automated... I want to have it capture the text underneath the mouse cursor by detecting the line of text underneath, looking at close letters or something. Is it possible?

Best regards, Marius.
-------------------------
KeyPress OSD v4: GitHub or forum. (presentation video)
Quick Picto Viewer: GitHub or forum.
AHK GDI+ expanded / compilation library (on GitHub)
My home page.
PaulySkating

Re: Vis2 - OCR(), ImageIdentify()

05 Apr 2018, 12:50

robodesign wrote:iSeaHound: I just tested the script on Win 10 x64 and it works well.

I am having eye-sight issues and I am very interested in such a tool, but I would like the process simplified / automated... I want to have it capture the text underneath the mouse cursor by detecting the line of text underneath, looking at close letters or something. Is it possible?

Best regards, Marius.
I suspect that is possible because he allows you to do an ocr based on x y w h, so you can detect the xy of the mouse and then do an ocr
robodesign
Posts: 934
Joined: 30 Sep 2017, 03:59
Location: Romania
Contact:

Re: Vis2 - OCR(), ImageIdentify()

05 Apr 2018, 13:39

Yes, it would be only a matter of identifying the boundaries of the text line underneath the mouse cursor...
-------------------------
KeyPress OSD v4: GitHub or forum. (presentation video)
Quick Picto Viewer: GitHub or forum.
AHK GDI+ expanded / compilation library (on GitHub)
My home page.
robodesign
Posts: 934
Joined: 30 Sep 2017, 03:59
Location: Romania
Contact:

Re: Vis2 - OCR(), ImageIdentify()

23 Apr 2018, 05:43

Hello, again!

@ iseahound: would this be useful? http://www.danvk.org/2015/01/07/finding ... numpy.html

I found this bit of information trying to understand how this would be done [detect the boundaries of text].
-------------------------
KeyPress OSD v4: GitHub or forum. (presentation video)
Quick Picto Viewer: GitHub or forum.
AHK GDI+ expanded / compilation library (on GitHub)
My home page.

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: gwarble, JoeWinograd and 130 guests