FindText - Capture screen image into text and then find it

Post a reply


In an effort to prevent automatic submissions, we require that you type the text displayed into the field underneath.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: FindText - Capture screen image into text and then find it

Re: FindText - Capture screen image into text and then find it

Post by boiler » 07 Nov 2017, 21:24

jeeswg wrote:Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.

I have only done it in RDP settings to affect a remote desktop, not my local one. I did see that it was possible, but you probably have already learned more about changing that setting than I know about it.

Re: FindText - Capture screen image into text and then find it

Post by jeeswg » 07 Nov 2017, 20:49

Thanks very much boiler.

Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.

This had an interesting list of suggestions.
Disable all font smoothing in Windows 7 (ie: including - Microsoft Community
https://answers.microsoft.com/en-us/windows/forum/windows_7-desktop/disable-all-font-smoothing-in-windows-7-ie/f180e803-3317-4433-8fd2-63aadaecc2d2

I.e. I could see if the text's appearance differed if I switched off anti-aliasing. I'm actually only interested in the text in Edit controls at present. Thanks.

Re: FindText - Capture screen image into text and then find it

Post by boiler » 07 Nov 2017, 20:28

I believe it is mainly an issue with letters. Although icons and other graphics are often created using anti-aliasing techniques to make them smoother and such, they are the same no matter where they are drawn if they are pixel-based graphics. Vector graphics get drawn differently depending on various factors like size and orientation. For letters, it seems that the text rendering engine in Windows will draw letters differently when anti-aliased depending on where it shows up, which may be because they are vector-based objects. For example, if you were to look at a black letter "I" magnified, it's weight may need to be 1.5 pixels wide, and it is simulated by drawing a black vertical line next to either a grey or some other color line. Depending on where the "I" is located, the black line could be on the left and the lighter color on the right, or vice-versa. It could even be two lines sort of in between.

One way to attempt to eliminate the issue is to turn off anti-aliasing in Windows, but sometimes you don't have control over that (like if it's for some other user's machine). Also sometimes it doesn't quite work because software may follow its own rules. I've seen where turning off anti-aliasing just changed it from being multiple colors (even black letters are made up of multiple colors if you zoom in on them) to shades of grey, still subject to the same problems of being rendered differently depending on location.

I believe that could very well be the problem in what you linked, although one would hope that a good OCR algorithm would be able to handle those differences since they typically handle different fonts, sizes, colors, etc.

Re: FindText - Capture screen image into text and then find it

Post by jeeswg » 07 Nov 2017, 20:01

@boiler: Could you say a bit more about anti-aliased letters, is that for images/text or both? Is something changing how text appears to prevent aliasing, what does this, and what does it do?
Also, do you think it could have some relevance for this:
fonts: text to image (pixels don't quite match) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=37937
Thanks.

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 07 Nov 2017, 19:47

boiler wrote:Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.


Thanks for the answer :-)

Re: FindText - Capture screen image into text and then find it

Post by boiler » 06 Nov 2017, 19:30

Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 06 Nov 2017, 19:00

Why not do this:

When the script capture the screen, the script automaticaly zoom in and out to simulate various resolutions, and font sharpness too if possible... maybe 5 resolutions samples and 5 samples with different font sharpening setting...
The code generated to paste will be 10 times bigger but maybe will turn computer settings proof.

Best regards!

Re: FindText - Capture screen image into text and then find it

Post by VaVa » 06 Nov 2017, 09:52

Hi Feiyue,

Thank you for sharing this wonderful script. LOVE IT !

Here is my last question...
The script works perfectly on full desktop screen. How to fix it to work on 'Active Window, only ?

Thank you much!

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 22 Oct 2017, 06:42

I've made some minor improvements, and the generated code is easier to copy and use.

Re: FindText - Capture screen image into text and then find it

Post by songdg » 08 Oct 2017, 02:41

Thanks for sharing,this is an awesome tool!I wonder how to turn off the always on top feature,it cause a little bit inconvenient,thanks again.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 03 Oct 2017, 04:47

Frosty Fridge wrote:I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Updated to the v5.4 version, Realized your idea. :superhappy:
However, even if the returned object contains multiple locations,
You also need loop processing to find the top or leftmost one.

Re: FindText - Capture screen image into text and then find it

Post by Frosty Fridge » 02 Oct 2017, 20:06

Hello feiyue,

First off I'd like to say that this script is amazing! It's both powerful and easy to use.

I noticed that you added support for returning info in an array for multiple different images

feiyue wrote:Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5


I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Thanks for making this script!

Re: FindText - Capture screen image into text and then find it

Post by ozzii » 02 Oct 2017, 04:15

@Joe Glines
Do you need to update your custom scripts for the latest update ???

Re: FindText - Capture screen image into text and then find it

Post by ali80 » 01 Oct 2017, 17:57

Hi feiyue, thanks for the great work
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 29 Sep 2017, 11:41

Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5

Re: FindText - Capture screen image into text and then find it

Post by Helgef » 29 Sep 2017, 01:50

Joe Glines wrote:Great job feiyue!
Indeed it seems very nice.
Joe Glines wrote:I created a tutorial showing how to use the basics of the script.
Hello Joe Glines :wave:
I watched your video when you posted it, but forgot to comment. It is very nice and helpful. I hope feiyue links to it in the first post so others can find.

Cheers.

Re: FindText - Capture screen image into text and then find it

Post by vava » 28 Sep 2017, 17:19

hello All...
I like this program. but have no luck to test all the functions.
each time click on the "test" function get an error "Can't run AHK script!" .
Running windows 7 64bit with the latest Autohotkey. hope some can help.

Re: FindText - Capture screen image into text and then find it

Post by Reloaded » 28 Aug 2017, 15:17

Wow best Script i ever see :D

Re: FindText - Capture screen image into text and then find it

Post by Joe Glines » 28 Aug 2017, 12:40

Great job feiyue! This script rocks!
I created a tutorial showing how to use the basics of the script.


I also took a stab at writing some functions to click/send text, etc. to the image once it was found.

Code: [Select all] [Expand]GeSHi © Codebox Plus



You can check out a video I did using the custom scripts and more on my page here

Top