100 replies to this topic

[javarike]

good,mark!

[Sean]

Change ResponseText to ResponseBody, then will get the page as SafeArray.

[Sean]

Change ResponseText to ResponseBody, then will get the page as SafeArray.

Some users may be perplexed how to save the binary data then. I used File.ahk Standard Library for that, however, it's a common practice to use ADODB.Stream object in scripting like:

sURL :=	"http://www.autohotkey.com/download/AutoHotkeyInstall.exe"
sFile:=	A_Temp "\AutoHotkeyInstall.exe"
bOverWrite := False

COM_Init()
pwhr :=	COM_CreateObject("WinHttp.WinHttpRequest.5.1")
COM_Invoke(pwhr, "Open", "GET", sURL)
COM_Invoke(pwhr, "Send")

If	psfa :=	COM_Invoke(pwhr, "ResponseBody")
{
;	pData:=	NumGet(psfa+12)
;	nSize:=	NumGet(psfa+16)
	pstm :=	COM_CreateObject("ADODB.Stream")
	COM_Invoke(pstm, "Type", 1)	; adTypeBinary:=1, adTypeText:=2
	COM_Invoke(pstm, "Open")
	COM_Invoke_(pstm, "Write", [color=red]0x2011[/color], psfa) ; VT_ARRAY | VT_UI1
	COM_Invoke(pstm, "SaveToFile", sFile, bOverWrite ? 2:1)
	COM_Invoke(pstm, "Close")
	COM_Release(pstm)
	COM_SafeArrayDestroy(psfa)
}

COM_Release(pwhr)
COM_Term()

[DeWild1]

I'm too lazy, to delve into this again.

You may want to try the COM alternative instead.

; Requires Windows Vista, Windows XP SP1, or Windows 2000 Professional SP3 and later.
; Requires WinHTTP 5.0 and Internet Explorer 5.01 or later on Windows XP, Windows 2000, and Windows NT 4.0.
; http://msdn2.microsoft.com/en-us/library/aa384106.aspx

#NoEnv
#include CoHelper.ahk ; http://www.autohotkey.com/forum/topic16631.html
CoInitialize()
pWHR := ActiveXObject("WinHttp.WinHttpRequest.5.1") ; CreateObject

SetTimeouts()
SetTimeouts(Resolve=0, Connect=60000, Send=30000, Receive=30000) {
  ; http://msdn2.microsoft.com/en-us/library/aa384061.aspx
  global pWHR
  DllCall(VTable(pWHR, 23), "Uint", pWHR, "Int", Resolve, "Int", Connect, "Int", Send, "Int", Receive)
}

; Use "HEAD" method, if it is not neccesary to retrieve ResponseText
sMethod := "GET", Ansi2Unicode(sMethod, wMethod) 
sUrl := "http://microsoft.com", Ansi2Unicode(sUrl, wUrl)
DllCall(VTable(pWHR, 9), "Uint", pWHR, "Str", wMethod, "Str", wUrl) ; Open

; Set UserAgent
sUserAgent := "A WinHttpRequest Example Program"
;sUserAgent := "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
pUserAgent := SysAllocString(sUserAgent)
DllCall(VTable(pWHR, 20), "Uint", pWHR, "UInt", WinHttpRequestOption_UserAgentString := 0 , "int64", 8, "int64", pUserAgent) ; put_Option

DllCall(VTable(pWHR, 13), "Uint", pWHR) ; Send

DllCall(VTable(pWHR, 12), "Uint", pWHR, "UIntP", pAllResponseHeaders) ; GetAllResponseHeaders
Unicode2Ansi(pAllResponseHeaders, sAllResponseHeaders)
SysFreeString(pAllResponseHeaders)
MsgBox % sAllResponseHeaders

sHeader := "content-length", Ansi2Unicode(sHeader, wHeader)
DllCall(VTable(pWHR, 11), "Uint", pWHR, "Str", wHeader, "UIntP", pValue) ; GetResponseHeader
Unicode2Ansi(pValue, sValue)
SysFreeString(pValue)
MsgBox % sValue

VarSetCapacity(StatusCode, 4, 0)
DllCall(VTable(pWHR, 14), "Uint", pWHR, "UInt", &StatusCode) ; Status
MsgBox % NumGet(StatusCode)

DllCall(VTable(pWHR, 15), "Uint", pWHR, "UIntP", pStatusText) ; StatusText
Unicode2Ansi(pStatusText, sStatusText)
SysFreeString(pStatusText)
MsgBox % sStatusText

If (sMethod != "HEAD") {
DllCall(VTable(pWHR, 16), "Uint", pWHR, "UIntP", pResponseText) ; ResponseText
Unicode2Ansi(pResponseText, sResponseText)
SysFreeString(pResponseText)
MsgBox % sResponseText
}

Release(pWHR)
CoUninitialize()


/* Settings for put_Option method: http://msdn2.microsoft.com/EN-US/library/aa383998.aspx

WinHttpRequestOption_UserAgentString := 0
WinHttpRequestOption_URL := 1
WinHttpRequestOption_URLCodePage := 2
WinHttpRequestOption_EscapePercentInURL := 3
WinHttpRequestOption_SslErrorIgnoreFlags := 4
WinHttpRequestOption_SelectCertificate := 5
WinHttpRequestOption_EnableRedirects := 6
WinHttpRequestOption_UrlEscapeDisable := 7
WinHttpRequestOption_UrlEscapeDisableQuery := 8
WinHttpRequestOption_SecureProtocols := 9
WinHttpRequestOption_EnableTracing := 10
WinHttpRequestOption_RevertImpersonationOverSsl := 11
WinHttpRequestOption_EnableHttpsToHttpRedirects := 12
WinHttpRequestOption_EnablePassportAuthentication := 13
WinHttpRequestOption_MaxAutomaticRedirects := 14
WinHttpRequestOption_MaxResponseHeaderSize := 15
WinHttpRequestOption_MaxResponseDrainSize := 16
WinHttpRequestOption_EnableHttp1_1 := 17
WinHttpRequestOption_EnableCertificateRevocationCheck := 18

Edit: Added GetAllResponseHeaders, GetResponseHeader, Get Status, Get StatusText
Edit: Added method to set the UserAgent (and other options)
Edit: Added SetTimeouts method

I am using this and it all works, but I need to use it through a proxy.

I set up the proxy like this, and it works.. (if you use urldownloadtofile or IE then it works, but not with the above code)

; END is the total amout of lines in goodproxylist.txt that I get from a loop in another place of my code...
  Random, rand, 1, %END%

FileReadLine, rawaddress, C:\temp\goodproxylist.txt, %rand%
 

  regwrite,REG_SZ,HKCU,Software\Microsoft\Windows\CurrentVersion\Internet Settings,ProxyServer,%rawaddress%
   
  regwrite,REG_DWORD,HKCU,Software\Microsoft\Windows\CurrentVersion\Internet Settings,Proxyenable,1
 
  dllcall("wininet\InternetSetOptionW","int","0","int","39","int","0","int","0")
  dllcall("wininet\InternetSetOptionW","int","0","int","37","int","0","int","0")

olfen talked about proxy stuff on the first page but the above stuff that I could understand well enough to get to work, :roll: from the above quote, he does not..

[JoeSchmoe]

This post is in response to Sean's code, posted below.

...
pwhr :=	COM_CreateObject("WinHttp.WinHttpRequest.5.1")
psink:=	ConnectIWinHttpRequestEvents(pwhr)
COM_Invoke(pwhr, "Open", "GET", sURL, "True")
COM_Invoke(pwhr, "Send")
...
IWinHttpRequestEvents(this, nStatus = "", pType = "")
{
	Critical
	If	A_EventInfo = 0
		NumPut(this,pType+0)
;	Else If	A_EventInfo = 1
;	Else If	A_EventInfo = 2
	Else If	A_EventInfo = 3
		OutputDebug, % "[START]`t" . nStatus . ": " . COM_Ansi4Unicode(pType) . "`n`n"
;	Else If	A_EventInfo = 4
	Else If	A_EventInfo = 5
	{
		Global	_ResponseText_ := COM_Invoke(NumGet(this+4), "ResponseText")
		SetTimer, ResponseText, -1
	}
	Else If	A_EventInfo = 6
		OutputDebug, % "[ERROR]`t" . nStatus . ": " . COM_Ansi4Unicode(pType) . "`n`n"
	Return	0
}

ConnectIWinHttpRequestEvents(pwhr)
{
	Static	IWinHttpRequestEvents
	If Not	VarSetCapacity(IWinHttpRequestEvents)
	{
		VarSetCapacity(IWinHttpRequestEvents,28,0), nParams=3113213
		Loop,	Parse,	nParams
		NumPut(RegisterCallback("IWinHttpRequestEvents","",A_LoopField,A_Index-1),IWinHttpRequestEvents,4*(A_Index-1))
	}
	pconn:=COM_FindConnectionPoint(pwhr,IID_IWinHttpRequestEvents:="{F97F4E15-B787-4212-80D1-D380CBBF982E}")
	psink:=COM_CoTaskMemAlloc(16), NumPut(pwhr,NumPut(&IWinHttpRequestEvents,psink+0))
	NumPut(COM_Advise(pconn,psink),NumPut(pconn,psink+8))
	Return	psink
}

Hi all... Would anyone be willing to provide some basic background about Sean's code, above? I'm not a windows programmer, and after a couple of very frustrating hours trying to figure out what the code is doing, I think that I can figure out how "IWinHttpRequestEvents" works. I think that ConnectIWinHttpRequestEvents is loading up an array of pointers to "Registered Callbacks," each of which will have a different A_EventInfo value. But what do the last four lines of the script do? (beginning with the COM_CoTaskMemAlloc line)

For other noobs trying to catch up, his code seems to be using this API:
<!-- m -->http://msdn.microsof...ry ... 85).aspx<!-- m -->
In particular, I think he is using the COM interface for it:
<!-- m -->http://msdn.microsof...ry ... 85).aspx<!-- m -->

I have to say, one of the wonderful things about AHK is its accessibility, but once you get to extensions like this, it gets pretty hellish for the nonprogrammer...

[Sean]

You don't necessarily need to understand the whole code. In practice, the essential parts you may need to know and/or modify to your need are the following two:

ResponseText: 
MsgBox, % _ResponseText_ 
Return

and

Else If   A_EventInfo = 5 
   { 
      Global   _ResponseText_ := COM_Invoke(NumGet(this+4), "ResponseText") 
      SetTimer, ResponseText, -1 
   } 

[JoeSchmoe]

Sorry for the delay in responding, but thanks a lot for your reply, Sean.

The script that I'm writing currently makes an http request about every 2.5 seconds and tends to make them more frequently when the user is interacting with it, so I'm trying to optimize performance as much as possible. Hence, I'm a bit worried about all of the callbacks that might occur with asynchronous requests and am looking to learn a bit more so that I can customize the code for my application.

After a lot of work with DR's httprequest, I'm becoming more comfortable with calling DLLs, but have much less experience with COM, even though I get the main idea, I think. I think I'd be able to figure it out if it weren't for the F97F4E15-B787-4212-80D1-D380CBBF982E string. I've figured out that it's a GUID for a COM interface. Is there a term I can search on or a suggested link where I can read about that interface? My hope is to use that documentation to learn about the 7 callbacks that are registered in IWinHttpRequestEvents, so that I can get a better sense of the performance characteristics of this version versus the synchronous version using WinInet that I currently use (based on DR's httpquery). For the record, I've read about the difference between WinHttp and WinInet, and since my application is more end-user oriented than server oriented, I haven't ruled out wininet even though it is an older library than winhttp.

[Sean]

It's documented here:
<!-- m -->http://msdn.microsof...ry ... 85).aspx<!-- m -->

[Murp-e]

This thread is all very confusing and old but is linked to in the help file ( for v1.0.47.6). Could someone help clear up how to download a web page to a variable. Wrapping it up in a simple function like UrlDownloadToVar(url) would be best.

I agree! A post that's linked from the help file should be clearer and better maintained. I think a moderator should somehow improve it. (olfen hasn't posted anything since 2008-01-19)

abandon the rediculous concept of downloading to var and use http request by derRaphael

What's wrong with URLDownloadToVar? It's so easy to use!

msgbox % URLDownloadToVar("www.google.com")
Return

URLDownloadToVar(URL){
 if !instr(url,":") 
  url = http://%url%
 httpQuery(html,URL)
 VarSetCapacity(html, -1)
Return html
}

#include httpQuery.ahk

Is this any less ridiculous?

[tank]

abandon the rediculous concept of downloading to var and use http request by derRaphael

What's wrong with URLDownloadToVar? It's so easy to use!

obviously the statement was meant as sarcazm and ridicul to the complaint

I tried the example in the first post and it seems to work but I'm trying avoid unforeseen consequences.

The alternatives posted may work as well or better but I'd rather not have a bunch of includes and more calls that I don't know.

but its true i cant spell worth a darn

Is this any less ridiculous?

:shock: now thats funny sh%^&* i dont care who you are

[tank]

I recommend using the WinHTTP COM alternative from page 2 of this thread. It doesn't suffer from some of the unresolved issues in the WinInet script.

I mean um... trying real hard not to make this sound wrong but how does this not answer

Is one function more reliable than the other?

[JoeSchmoe]

abandon the rediculous concept of downloading to var and use http request by derRaphael

What's wrong with URLDownloadToVar? It's so easy to use!

obviously the statement was meant as sarcazm and ridicul to the complaint

Hmmm... I didn't pick up on that sarcasm either and have spent quite a few hours integrating httpquery into my software because I thought that you knew something that I didn't know about how URLDownloadToVar was a hopeless piece of software. I'm incredibly busy right now, so it kindof sucks to find that out. I suppose it isn't a huge deal, though, as in the end I've realized that an approach using winsock directly is more appropriate for my needs than one using http.

It's documented here:
<!-- m -->http://msdn.microsof...ry ... 85).aspx<!-- m -->

Thanks a lot, Sean. I may not use it now (see above), but I appreciate the help.

I agree! A post that's linked from the help file should be clearer and better maintained. I think a moderator should somehow improve it. (olfen hasn't posted anything since 2008-01-19)

Even with the post being linked to in the help file, I wonder if the mods might find it too intrusive to edit olfen's post. I'm a mod on another board and try to never change people's posts unless some sort of rule is being broken. You can really piss people off that way.

I don't know if there is anyone willing to do this, but perhaps someone else who plans to be around for a whle can agree to maintain the code somewhat and start a new thread with a nice clean version of the code at the top. It'd be much more helpful to the noobs, who are, I believe, an important constituency for this language. (i.e. other languages might be more powerful, but they aren't nearly as noob friendly, especially when forum support is considered.)

Then, either Chris can update the help file (it's so simple that he might be willing to if emailed) or a Mod can put a note on the initial post in this thread identifying themselves as a mod and saying that there is a more up to date thread to read.

This may seem a little over the top, but Chris set some pretty high standards for documentation, and I think that's a big part of why when I google "autohotkey OR autoit," autohotkey.com comes up first. :-)