Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[AHK_L] Screen Reader -- a tool to get text anywhere


  • Please log in to reply
46 replies to this topic
nepter
  • Members
  • 53 posts
  • Last active: Jul 31 2016 06:47 AM
  • Joined: 29 Jun 2012

The script is written by nepter, administrator of Chinese autohotkey forum.

Function: Press a hotkey, Capslock as default, to retrieve text under cursor and show it. It could get text in gui, html, instant messager and etc. The original Capslock is replaced by Shift+Capslock.  A tray in notification area could change the hotkey.

You have to run it above WinXp sp3 and by Autohotkey_L

Version: ScreenReader 0.1.1b

 

Video on Screenr

 

You could move mouse over the script below and press Capslock. What you see would excite you, I believe.

#SingleInstance,force
#InstallKeybdHook
;#NoTrayIcon
intro:="
(
The script is written by nepter, administrator of <a href=""http://bbs.ahk8.com"">Chinese autohotkey forum</a>.

Function: Press a hotkey, Capslock as default, to retrieve text under cursor and show it. It could get text in gui, html, instant messager and etc. The original Capslock is replaced by Shift+Capslock.  A tray icon in notification area could change the hotkey.

You have to run it above WinXp sp3 and by Autohotkey_L 

Version: ScreenReader 0.1.1a
)"
menu,tray,NoStandard
menu,tray,add,Hotkey,hotkey
menu,tray,add,About...,about
menu,tray,add,Exit,exit
CoordMode,mouse,screen
KeyName:="capslock"
Hotkey,%KeyName%,main
if !uia:=ComObjCreate("{ff48dba4-60ef-4201-aa87-54103eef594e}","{30cbe57d-d9d0-452a-ab13-7ac5ac4825ee}"){
	msgbox UI Automation Failed.
	ExitApp
}
return
+CapsLock::CapsLock
exit:
	exitapp
hotkey:
	gui,1:Destroy
	gui,1:add,text,,New Hotkey
	gui,1:add,Hotkey,vChosenHotkey,%KeyName%
	gui,1:add,button,Default gbtnHK,Confirm
	gui,1:show,,%A_Space%
	return
about:
	gui,2:Destroy
	gui,2:add,link,,%intro%
	gui,2:show,,About ScreenReader 0.1.1a
	return
btnHK:
	gui,1:submit
	if (ChosenHotkey!=KeyName){
		Hotkey,%KeyName%,,off
		KeyName:=ChosenHotkey
		Hotkey,%KeyName%,main,on
	}
	gui,1:Destroy
	return
main:
MouseGetPos,x,y
;DllCall(vt)
item:=GetElementItem(x,y)
if !item.1
	return
gui,3:Destroy
;gui,3:new,ToolWindow
for k,v in item
{
	gui,3:add,edit,x5 w480 -Tabstop vedit%k%,%v%
	gui,3:add,button,X+5 yp-2 vbtn%k% gcp2cb,To Clipboard
}
	gui,3:show,,You Get
return
vas(obj,ByRef txt){
	for k,v in obj
		if (v=txt)
			return 0
	return 1
}
cp2cb:
	n:=SubStr(A_GuiControl,4)
	GuiControlGet,txt,,edit%n%
	if txt
		Clipboard:=txt
	gui,3:Destroy
return
3GuiEscape:
	gui,3:Destroy
	return
GetPatternName(id){
	global uia
	DllCall(vt(uia,50),"ptr",uia,"uint",id,"ptr*",name)
	return StrGet(name)
}
GetPropertyName(id){
	global uia
	DllCall(vt(uia,49),"ptr",uia,"uint",id,"ptr*",name)
	return StrGet(name)
}
GetElementItem(x,y){
	global uia
	item:={}
	DllCall(vt(uia,7),"ptr",uia,"int64",x|y<<32,"ptr*",element) ;IUIAutomation::ElementFromPoint
        if !element
            return
	DllCall(vt(element,23),"ptr",element,"ptr*",name) ;IUIAutomationElement::CurrentName
	DllCall(vt(element,10),"ptr",element,"uint",30045,"ptr",variant(val)) ;IUIAutomationElement::GetCurrentPropertyValue::value
	DllCall(vt(element,10),"ptr",element,"uint",30092,"ptr",variant(lname)) ;IUIAutomationElement::GetCurrentPropertyValue::lname
	DllCall(vt(element,10),"ptr",element,"uint",30093,"ptr",variant(lval)) ;IUIAutomationElement::GetCurrentPropertyValue::lvalue
	a:=StrGet(name,"utf-16"),b:=StrGet(NumGet(val,8,"ptr"),"utf-16"),c:=StrGet(NumGet(lname,8,"ptr"),"utf-16"),d:=StrGet(NumGet(lval,8,"ptr"),"utf-16")
	a?item.Insert(a):0
	b&&vas(item,b)?item.Insert(b):0
	c&&vas(item,c)?item.Insert(c):0
	d&&vas(item,d)?item.Insert(d):0
	DllCall(vt(element,21),"ptr",element,"uint*",type) ;IUIAutomationElement::CurrentControlType
	if (type=50004)
		e:=GetElementWhole(element),e&&vas(item,e)?item.Insert(e):0
	ObjRelease(element)
	return item
}
GetElementWhole(element){
	global uia
	static init:=1,trueCondition,walker
	if init
		init:=DllCall(vt(uia,21),"ptr",uia,"ptr*",trueCondition) ;IUIAutomation::CreateTrueCondition
		,init+=DllCall(vt(uia,14),"ptr",uia,"ptr*",walker) ;IUIAutomation::ControlViewWalker
	DllCall(vt(uia,5),"ptr",uia,"ptr*",root) ;IUIAutomation::GetRootElement
	DllCall(vt(uia,3),"ptr",uia,"ptr",element,"ptr",root,"int*",same) ;IUIAutomation::CompareElements
	ObjRelease(root)
	if same {
		return
	}
	hr:=DllCall(vt(walker,3),"ptr",walker,"ptr",element,"ptr*",parent) ;IUIAutomationTreeWalker::GetParentElement
	if parent {
		e:=""
		DllCall(vt(parent,6),"ptr",parent,"uint",2,"ptr",trueCondition,"ptr*",array) ;IUIAutomationElement::FindAll
		DllCall(vt(array,3),"ptr",array,"int*",length) ;IUIAutomationElementArray::Length
		loop % length {
			DllCall(vt(array,4),"ptr",array,"int",A_Index-1,"ptr*",newElement) ;IUIAutomationElementArray::GetElement
			DllCall(vt(newElement,23),"ptr",newElement,"ptr*",name) ;IUIAutomationElement::CurrentName
			e.=StrGet(name,"utf-16")
			ObjRelease(newElement)
		}
                ObjRelease(array)
		ObjRelease(parent)
		return e
	}
}
variant(ByRef var,type=0,val=0){
	return (VarSetCapacity(var,8+2*A_PtrSize)+NumPut(type,var,0,"short")+NumPut(val,var,8,"ptr"))*0+&var
}
vt(p,n){
	return NumGet(NumGet(p+0,"ptr")+n*A_PtrSize,"ptr")
}

Edited by nepter, 27 June 2013 - 03:59 PM.


Larctic
  • Members
  • 303 posts
  • Last active: May 10 2016 04:56 PM
  • Joined: 21 Jul 2012

恭喜新论坛诞生!

个人觉得阅读页面应该稍加美化。



jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009

Thank you for posting this. It is my understanding that UI Automation is the successor to MSAA (Acc LibraryAccessible Info Viewer), which is why I basically gave up working on those threads. I had been looking at UIAutomation, but was looking for some code to get me started. This script is quite informative.
 
On a side note - do you have Virtual Table Definitions for the UI Automation objects? I'd be interested in those - for instance how you'd know that ElementFromPoint is the 8th method on the IUIAutomation Interface. (EDIT: never mind - found the header files on my pc & pulled the VTables from there)
 
Also, do you mind if we add IUIAutomation to the thread title? There aren't too many IUIAutomation code examples in the forums yet, & I'd like this one to be easy to find.



nepter
  • Members
  • 53 posts
  • Last active: Jul 31 2016 06:47 AM
  • Joined: 29 Jun 2012

I made a script to analyze heade or idl file of vs 2010, and output vt definition like index,type,parameter of every function. like

Spoiler

But the script is raw and output is not readable. When I write each function by ahk, I should search msdn to make sure I write correctly. It is exhausting.

 

UI Automation is important to AHK. It could do a lot of things. The script I write is only a function of my project "One key to do every thing",  I think it is helpful for most people.

 

Thanks for attention.



ahgfaa11
  • Members
  • 1 posts
  • Last active: Aug 09 2015 10:13 AM
  • Joined: 16 May 2013

test it with win8 32bit  and Autohotkey_L 全是乱码。



nepter
  • Members
  • 53 posts
  • Last active: Jul 31 2016 06:47 AM
  • Joined: 29 Jun 2012

test it with win8 32bit  and Autohotkey_L 全是乱码。

我是Win8 x64的,获取的英文的也是乱码吗

另外你说老外看我们中文是乱码吗



hitman
  • Members
  • 5 posts
  • Last active: Jan 17 2015 04:47 PM
  • Joined: 03 Jan 2011
这里是新论坛吗?我来了。

tmplinshi
  • Members
  • 245 posts
  • Last active: Mar 12 2015 02:29 PM
  • Joined: 06 Apr 2012

Thank you, very usefull.



oldbrother
  • Members
  • 149 posts
  • Last active: Nov 09 2014 07:36 PM
  • Joined: 06 Jul 2005
How to get the word under the cursor? This script seems can only get sentences or paragraphs.
 
Thanks!


jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009

 

How to get the word under the cursor? This script seems can only get sentences or paragraphs.

 

Specifically, this script can get the text from the UI Automation Object/Element under the mouse. So, to answer your question, this script won't work for getting the single word under the cursor.

 

 

@nepter - nice video. You may want to show what browser you are using (Internet Explorer?, Firefox?). Also, I see you used another script/app called COMHelper 0.2. Could you provide this, or a link to where you got it?



nepter
  • Members
  • 53 posts
  • Last active: Jul 31 2016 06:47 AM
  • Joined: 29 Jun 2012

 

How to get the word under the cursor? This script seems can only get sentences or paragraphs.

 

I think if the control, like IE, edit control ,support hit test, it is possible to get a single word not sentences. But it is what UIAutomation can not handle with

 

@nepter - nice video. You may want to show what browser you are using (Internet Explorer?, Firefox?). Also, I see you used another script/app called COMHelper 0.2. Could you provide this, or a link to where you got it?

Its size is about 100mb and something is raw and misunderstanding I think. I will cut off some function to delieve it.



jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009

Its size is about 100mb and something is raw and misunderstanding I think. I will cut off some function to delieve it.

 

Dropbox.com offers 2gb free online storage, and Copy.com offers 20gb free online storage - both allow public links. If you are OK with COMHelper 0.2 being public open source, either of those two should work for online storage. I'd even be willing to host it if you don't want to.



Grendahl
  • Members
  • 416 posts
  • Last active: Jul 07 2014 08:01 PM
  • Joined: 10 Aug 2009

This doesn't work for me. I get a StrGet() error on line 132.

 

(Win7 64bit and the latest AHK_L)


Always have your scripts when you need them with Dropbox.
Sign up for free! http://db.tt/9Hrieqj

robert_ilbrink
  • Members
  • 561 posts
  • Last active: Mar 10 2015 08:36 PM
  • Joined: 05 May 2012

Same here, StrGet() error on line 132 using AHK  v1.1.10.01. Does AHK v1.1.10.01 not support this? ( http://l.autohotkey....s/StrPutGet.htm )



Grendahl
  • Members
  • 416 posts
  • Last active: Jul 07 2014 08:01 PM
  • Joined: 10 Aug 2009

Got it working... sort of.

 

Change line 132 from

e.:=StrGet(name)

to

e:=StrGet(name)

 

There's an extra period after the variable "e" that was messing it up.

 

 

NOW, it runs, but the text in the GUI is all Chinese. How would i go about converting that to something i can read. (english)


Always have your scripts when you need them with Dropbox.
Sign up for free! http://db.tt/9Hrieqj