Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[Func] HTTPRequest: for Web APIs [AHK-B + AHK-L+Unicode+x64]


  • Please log in to reply
380 replies to this topic
traxxler
  • Members
  • 2 posts
  • Last active: Jul 13 2011 09:31 PM
  • Joined: 13 Jul 2011
I am so glad that somebody created this... thank you, really.

Would it be at all possible to see a script example that would work with this to upload videos from a PC to Youtube:

<!-- m -->http://code.google.c... ... _uploading<!-- m -->
<!-- m -->http://code.google.c... ... entication<!-- m -->

One important thing is that it needs to be able to handle quota errors as described here:

<!-- m -->http://apiblog.youtu... ... quota.html<!-- m -->

Youtube API staff told me this as I have a large number of videos to upload.

If you can do this, I would be so grateful!

brzapp
  • Members
  • 7 posts
  • Last active: Aug 27 2011 06:44 PM
  • Joined: 25 Jul 2011
Dear VeX,

First of all, thanks for your script!

I want to get values from an ASP page, but it asks for cookies.

First, I submit a request to get cookies. Then, I submit
a request with the previous headers and POSTDATA values.
However, it is not working properly. Should it be that simple or
dealing with cookies require a bit more knowledge?

;GET HEADERS
httprequest(URL:=website, POSTData, headers)

/*
HEADERS AFTER REQUEST:
HTTP/1.1 200 OK
Via: 1.0 I67V131ID
Connection: Keep-Alive
Content-Length: 7260
Date: Mon, 01 Aug 2011 09:44:52 GMT
Content-Type: text/html
Set-Cookie: ServerID=1175; path=/
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Set-Cookie: ASPSESSIONIDCSRQDTAD=LHNLMKDDKMHPFGACLGBGOJGF; path=/
Cache-control: private
*/
;POST DATA
POSTDATA=w=1&e=2&r=3

httprequest(URL:=website, POSTData, headers, "+Flag: INTERNET_FLAG_NO_COOKIES")



VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

Should it be that simple or
dealing with cookies require a bit more knowledge?

It is not that simple... not at all. Don't believe me? look at the official standard: RFC 2109.

The headers you receive from a server are called response headers. When you send a request, you should not send response headers.

More simply, you must parse the response headers, and for each 'Set-Cookie' header you find, add a 'Cookie' header to the request headers for the next request.

fincs
  • Moderators
  • 1662 posts
  • Last active:
  • Joined: 05 May 2007
Tip: I see you're directly calling WideCharToMultiByte and MultiByteToWideChar. Contrary to popular belief, StrPut() and StrGet() are available for AutoHotkey Basic.

HTH

brzapp
  • Members
  • 7 posts
  • Last active: Aug 27 2011 06:44 PM
  • Joined: 25 Jul 2011
I did not understand your remark... Was that for me? Where should I use it? Thanks

Tip: I see you're directly calling WideCharToMultiByte and MultiByteToWideChar. Contrary to popular belief, StrPut() and StrGet() are available for AutoHotkey Basic.

HTH



fincs
  • Moderators
  • 1662 posts
  • Last active:
  • Joined: 05 May 2007
That post was for the author of this library... :?:

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

...StrPut() and StrGet() are available for AutoHotkey Basic.

Good call. I'll add that tip to the OP. It makes sense that a user may want to convert text output to a different codepage.

I see you're directly calling WideCharToMultiByte and MultiByteToWideChar

StrPut() and StrGet() use MultiByteToWideChar and WideCharToMultiByte internally, so I don't understand why calling them directly is an issue. I realize HTTPRequest only supports two codepages, but I believe they cover the overwhelming majority of web responses. In unforseen cases, a separate call to StrPut() and StrGet() should be sufficient to "correct" the codepage without losing data.


On a different note, I would like someone to test HTTPRequest using an x64 version of AHK. Please see this thread for details.

fincs
  • Moderators
  • 1662 posts
  • Last active:
  • Joined: 05 May 2007
StrPut() and StrGet() are built-in functions in AutoHotkey_L, thus the function would be faster. And the code would also be less messy, too.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

StrPut() and StrGet() are built-in functions in AutoHotkey_L, thus the function would be faster.

True, but the speed advantage is on the microsecond scale. In a function that normally takes hundreds of milliseconds to complete, the practical benefit is infinitesimal.

Also, one of the key planks of this project is flavor-neutrality (compatibility between branches of AHK without specific versions for each), which would be contradicted by using StrGet() / StrPut() within the function.

And the code would also be less messy, too.

I'll let that slide....

brzapp
  • Members
  • 7 posts
  • Last active: Aug 27 2011 06:44 PM
  • Joined: 25 Jul 2011
Dear VxE,

I havenot succeeded in posting values with cookie, although I have read all RFC related to cookies etc...

Should I try to use something like InternetSetCookieEx to send cookies?

http://msdn.microsof...y/aa385108.aspx

I didn't suceed with the code below.

iscex_url:=url
iscex_cookie_name:=name
iscex_cookie_value:=value

DllCall( "WinINet\InternetSetCookieEx", iscex_url, iscex_cookie_name, iscex_cookie_value, iscex_dwFlags, iscex_dwReserved)

Thanks,
BRZapp[/code]

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
brzapp, InternetSetCookieEx uses IE's cookie database, so I would avoid using it if possible.

I don't know the particulars of the website you're trying to reach, but here's a generic illustration of cookie control using HTTPRequest:
; PSEUDOCODE
URL_Host := "http://www.somesiteimadeupasanexample.com"
Page1 := "/login.asp"

HTTPRequest( URL_Host Page1, Page_Data := "", Response_Headers := "" )

; The response headers contain several 'Set-Cookie' headers.
; This regex pulls them out and turns them into 'Cookie' headers.
Cookies := RegexReplace( Response_Headers, "(?:Set-(Cookie: \V*\v*)|\V*\v*)", "$1" )

; This regex removes the following attributes from the cookies: expires,domain,path,secure,httponly
Cookies := RegexReplace( Cookies, "i);\h+(secure|httponly|(expires|domain|path)=[^;\v]*)" )

; Look at the page data for whatever info... probably the uri of the next page?
Page2 := "/dologin.asp?submit=true&user=abcd&pass=12345"

; Get the next page and send the cookies in the request header
HTTPRequest( URL_Host Page2, Page_Data := "", Response_Headers := Cookies, "+Flag: INTERNET_FLAG_NO_COOKIES" )
; The site should have accepted the cookies and whatever other information and returned the appropriate page.
You'll probably want some way of managing the cookies (updating & storing them, etc...), but I don't have a succinct example of that.

brzapp
  • Members
  • 7 posts
  • Last active: Aug 27 2011 06:44 PM
  • Joined: 25 Jul 2011
PERFECT!!! It worked!

I was requesting the second time with the first URL...

Thanks a lot VxE!!!!

mauritius
  • Members
  • 75 posts
  • Last active: Aug 22 2015 09:53 AM
  • Joined: 04 Aug 2010
Hello,
Has anyone used this function to mimic a web browser to log into a forum (phpbb or vBulletin especially)? If yes, please give me some advice how to do it:
1. without hardcoding session id into my script
2. any way to log into vBulletin-based forum (I did succeed with phpbb but didn't with vBulletin)
3. any other remarks would be also very helpfull.
Thanks in advance for your help.

dadepp
  • Members
  • 27 posts
  • Last active: Feb 28 2012 08:33 PM
  • Joined: 12 Jun 2008
First off thank you for this great function.

It didnt fully fit my needs, so I modified it a bit:
I included resume support for downloading files, and a transfer-abort ability.

HTTPRequest mod

Hope it is usefull for some.

Plqk
  • Members
  • 17 posts
  • Last active: Nov 11 2011 02:49 PM
  • Joined: 08 Jul 2011
I am trying to fetch WoW (World of Warcraft) armoury data with HTTPrequest. It should be just as simple as:
httprequest(us.battle.net/api/wow/auction/data/medivh, inOutPost, inOutHeaders, "")

Unfortunately I am getting the following error: "There was problem with the provided URL(InternetCrackURL). ErrorLevel = 0, A_LastError = 12006"