FindText - Capture screen image into text and then find it

Post a reply


In an effort to prevent automatic submissions, we require that you type the text displayed into the field underneath.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: FindText - Capture screen image into text and then find it

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 06 Dec 2017, 04:51

hal123 wrote:
Would it somehow be possible to make it work on dynamic text (Taking the text to find from a file)

My idea is somehow to generate the image to find. First I capture all the letters in the alphabet, based on the text and captured letters I then generate the image.

Code: [Select all] [Expand]GeSHi © Codebox Plus



How can I merge the letters into an "image word" that I can search for?

Updated to the v5.6 version, Add a new Function: FindText2()
This function is used for combination lookup,
for example, a 0-9 text library has been set up, and any ID number can be found.
Use Pic(Text,1) and PicN(number) when using. The simple code:

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by MikeTheNumbers » 24 Nov 2017, 05:49

Hi Feiyue.
I'm new to ATH, and have no experience with coding what so ever.
But I have been looking for a way to use AutoHotkey as an Anti-AFK for a game.

I made a post in the ask for help part of the forum https://autohotkey.com/boards/viewtopic ... 18&t=40432

I've now been doing a little digging since that post, and I stumbled upon your work, and I actually think It might just work for the purpose I need It for, with a few adjustment, adjustment I'm not able to make myself.
I was wondering if you had the time, if you could check my request out, and if possible, maybe help me out.

/Mike

Re: FindText - Capture screen image into text and then find it

Post by burque505 » 23 Nov 2017, 10:42

Thank you, feiyue, for this script and all your continued work on it.
Regards,
burque505

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 23 Nov 2017, 10:30

Updated to the v5.5 version, in order to identify a variety of color verification code, I improved the color model, now it can adapt to various colors. :D

Re: FindText - Capture screen image into text and then find it

Post by boiler » 07 Nov 2017, 21:24

jeeswg wrote:Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.

I have only done it in RDP settings to affect a remote desktop, not my local one. I did see that it was possible, but you probably have already learned more about changing that setting than I know about it.

Re: FindText - Capture screen image into text and then find it

Post by jeeswg » 07 Nov 2017, 20:49

Thanks very much boiler.

Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.

This had an interesting list of suggestions.
Disable all font smoothing in Windows 7 (ie: including - Microsoft Community
https://answers.microsoft.com/en-us/windows/forum/windows_7-desktop/disable-all-font-smoothing-in-windows-7-ie/f180e803-3317-4433-8fd2-63aadaecc2d2

I.e. I could see if the text's appearance differed if I switched off anti-aliasing. I'm actually only interested in the text in Edit controls at present. Thanks.

Re: FindText - Capture screen image into text and then find it

Post by boiler » 07 Nov 2017, 20:28

I believe it is mainly an issue with letters. Although icons and other graphics are often created using anti-aliasing techniques to make them smoother and such, they are the same no matter where they are drawn if they are pixel-based graphics. Vector graphics get drawn differently depending on various factors like size and orientation. For letters, it seems that the text rendering engine in Windows will draw letters differently when anti-aliased depending on where it shows up, which may be because they are vector-based objects. For example, if you were to look at a black letter "I" magnified, it's weight may need to be 1.5 pixels wide, and it is simulated by drawing a black vertical line next to either a grey or some other color line. Depending on where the "I" is located, the black line could be on the left and the lighter color on the right, or vice-versa. It could even be two lines sort of in between.

One way to attempt to eliminate the issue is to turn off anti-aliasing in Windows, but sometimes you don't have control over that (like if it's for some other user's machine). Also sometimes it doesn't quite work because software may follow its own rules. I've seen where turning off anti-aliasing just changed it from being multiple colors (even black letters are made up of multiple colors if you zoom in on them) to shades of grey, still subject to the same problems of being rendered differently depending on location.

I believe that could very well be the problem in what you linked, although one would hope that a good OCR algorithm would be able to handle those differences since they typically handle different fonts, sizes, colors, etc.

Re: FindText - Capture screen image into text and then find it

Post by jeeswg » 07 Nov 2017, 20:01

@boiler: Could you say a bit more about anti-aliased letters, is that for images/text or both? Is something changing how text appears to prevent aliasing, what does this, and what does it do?
Also, do you think it could have some relevance for this:
fonts: text to image (pixels don't quite match) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=37937
Thanks.

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 07 Nov 2017, 19:47

boiler wrote:Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.


Thanks for the answer :-)

Re: FindText - Capture screen image into text and then find it

Post by boiler » 06 Nov 2017, 19:30

Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 06 Nov 2017, 19:00

Why not do this:

When the script capture the screen, the script automaticaly zoom in and out to simulate various resolutions, and font sharpness too if possible... maybe 5 resolutions samples and 5 samples with different font sharpening setting...
The code generated to paste will be 10 times bigger but maybe will turn computer settings proof.

Best regards!

Re: FindText - Capture screen image into text and then find it

Post by VaVa » 06 Nov 2017, 09:52

Hi Feiyue,

Thank you for sharing this wonderful script. LOVE IT !

Here is my last question...
The script works perfectly on full desktop screen. How to fix it to work on 'Active Window, only ?

Thank you much!

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 22 Oct 2017, 06:42

I've made some minor improvements, and the generated code is easier to copy and use.

Re: FindText - Capture screen image into text and then find it

Post by songdg » 08 Oct 2017, 02:41

Thanks for sharing,this is an awesome tool!I wonder how to turn off the always on top feature,it cause a little bit inconvenient,thanks again.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 03 Oct 2017, 04:47

Frosty Fridge wrote:I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Updated to the v5.4 version, Realized your idea. :superhappy:
However, even if the returned object contains multiple locations,
You also need loop processing to find the top or leftmost one.

Re: FindText - Capture screen image into text and then find it

Post by Frosty Fridge » 02 Oct 2017, 20:06

Hello feiyue,

First off I'd like to say that this script is amazing! It's both powerful and easy to use.

I noticed that you added support for returning info in an array for multiple different images

feiyue wrote:Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5


I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Thanks for making this script!

Re: FindText - Capture screen image into text and then find it

Post by ozzii » 02 Oct 2017, 04:15

@Joe Glines
Do you need to update your custom scripts for the latest update ???

Re: FindText - Capture screen image into text and then find it

Post by ali80 » 01 Oct 2017, 17:57

Hi feiyue, thanks for the great work
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 29 Sep 2017, 11:41

Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5

Top