Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[Func] HTTPRequest: for Web APIs [AHK-B + AHK-L+Unicode+x64]


  • Please log in to reply
380 replies to this topic
tmplinshi
  • Members
  • 245 posts
  • Last active: Mar 12 2015 02:29 PM
  • Joined: 06 Apr 2012

How to set max connection timeout? Like -m option in curl.exe.

 

Thanks.



maka90
  • Members
  • 1 posts
  • Last active: Jun 05 2013 04:23 PM
  • Joined: 05 Jun 2013

Hey I need to be logged in to see certain page content. Is there any way to implement some login script to HTTPRequest?

Heres what i need:

 

URL := "http://www.docskillz...hp?action=login"

wb.document.getElementsByTagName("INPUT")[7].value := Loginname
wb.document.getElementsByTagName("INPUT")[8].value := Password
wb.document.getElementsByTagName("INPUT")[10].checked := True
wb.document.getElementsByTagName("INPUT")[11].Click()



VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

How to set max connection timeout? Like -m option in curl.exe.

 

Thanks.

 

HTTPRequest does not have that option. See http://msdn.microsof...4(v=VS.85).aspx for more info.

 

 

maka90: automating web pages is more easily done with COM (your example code implies COM usage). Does your COM-based code not work? In what way? What are you trying to accomplish?



cylisme
  • Members
  • 8 posts
  • Last active: Oct 20 2013 10:51 PM
  • Joined: 05 Jul 2011

bug report

data=
URL= http://www.tingchina.com/play/yousheng/flash.asp?id=24366&inum=1&flei=恐怖惊悚&bookname=我当阴曹官的那几年&filename=001.mp3&rand=16&nexturl=play_24366_1.htm
httpQueryReferer:="http://www.tingchina.com/yousheng/24366/play_24366_0.htm"
httpAgent:="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
httpQuery(data,URL)
varSetCapacity(data,-1)
msgbox % data




url:="http://www.tingchina.com/play/yousheng/flash.asp?id=24366&inum=1&flei=恐怖惊悚&bookname=我当阴曹官的那几年&filename=001.mp3&rand=16&nexturl=play_24366_1.htm"
	Refererurl:="http://www.tingchina.com/yousheng/24366/play_24366_0.htm"
	data=
	headers =
			(
			User-Agent:Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
			Referer: %Refererurl%             
			)
			opinion=
			(LTRIM C
			;Proxy: localhost:8888
			+NO_COOKIES
			Charset:gb2312
			)
	HTTPRequest(url,data ,headers,opinion)
	msgbox % data

httpQuery

 

appears

 

6519f9aetw1e6mm263wnnj219z0qtdns.jpg

 

 

HTTPRequest

 

appears

6519f9aetw1e6mm2cfirrj213k0q0wlo.jpg

 

 

 

 

 

 

 



cylisme
  • Members
  • 8 posts
  • Last active: Oct 20 2013 10:51 PM
  • Joined: 05 Jul 2011

well  i  add  this   Binary - Normally and use the basic ahk   as you said it is not necessary until use unicode AHK



VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

It's a common source of confusion, but the charset option only effects POST data, that is, data you want to upload.

Also, it appears that tingchina.com does not specify a character set in its response headers, so HTTPRequest assumes the page text is your current windows ANSI codepage. Override this uncertainty using either the Binary or Expect:utf-8 option.



Dave_Scream
  • Members
  • 9 posts
  • Last active: Apr 13 2017 01:53 PM
  • Joined: 21 Jul 2013
I too have problems with codepages.
 
I use this request to POST data
URL:="http://vk.com/al_photos.php"
HEADER:="Content-Type: application/x-www-form-urlencoded`nX-Requested-With: XMLHttpRequest"
HEADER:=VKQueryHeader "`n" HEADER
POSTData:="act=choose_photo&al=1&mail_add=&max_files=2&scrollbar_width=16&to_id=" MyGroupID

HTTPRequest(URL, POSTData, HEAD, "Method: POST`nCharset: windows-1251`nExpect: windows-1251") 

FileAppend, %POSTData%, log.log
 

But after this request POSTData in my log file contains "?????'s" on all cyrillic symbols.
Spoiler

 
In 'HTTP analyzer' I see that WebServer returns right charset...
Spoiler


but even with "Expect: windows-1251" HTTPRequest broke cyrillic symbols.

In HTTP Analyzer response content have good cyrrilic shure
Spoiler

 
So I fighting with it half-day + half-day   and have no ideas now.

UPD:
Im using this version of AutoHotkey:
Spoiler

 
My source script file encoded as UTF-16
Spoiler


Dave_Scream
  • Members
  • 9 posts
  • Last active: Apr 13 2017 01:53 PM
  • Joined: 21 Jul 2013
Well I succeeded by not using internal HTTPRequest conversations.

I used Binary mode to force HTTPRequest skip any codepage conversations (see Binary flag at the end):
HTTPRequest(URL, POSTData, HEAD, "Method: POST`nCharset: windows-1251`nExpect: windows-1251`nBinary")
after that, I written in raw mode POSTData to file to look it in original:
file:=FileOpen("log.log", "w")
file.RawWrite(POSTData,StrLen(POSTData))
file.Close()
the results was ok! with all cyrillic symbols. And it was in ANSI (windows-1251) codepage already
Spoiler


and all I needed to do to import this string in my Unicode Autohotkey, is internal function StrGet.
str := StrGet(&POSTData, StrLen(POSTData), "CP1251")

now everythink fine.

So I dont know why HTTPRequest convert 'windows-1251' received from server (with header charset=windows-1251) to somewhat else? If I expect windows-1251 and server send to me charset=windows-1251 - then all HTTPRequest must do is to copy whole string without any routines.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

Just as a point of clarity, the "Expect:..." option only takes effect when Content-Type does not appear in the response headers, or is missing the charset

.

 

I downloaded the raw binary of vk.com's front page then opened it with a hex editor. It contains the line "<meta http-equiv="content-type" content="text/html; charset=utf-8" />". Also the string "&lang_id=0">РуÑÑкий</a>" in the page source shows that it is UTF-8, not windows-1251.

 

Most browser tools and text editors will silently decode UTF-8, so the actual page encoding is difficult to determine.



Dave_Scream
  • Members
  • 9 posts
  • Last active: Apr 13 2017 01:53 PM
  • Joined: 21 Jul 2013

Just as a point of clarity, the "Expect:..." option only takes effect when Content-Type does not appear in the response headers, or is missing the charset

Thank you.
Another question. Im trying to upload jpeg image to vk.com. It worked with httpQuery on old autohotkey. But now HTTPQuery is broken.

I tryed to upload jpeg with standart "Upload" option.
hOptions:="Method: POST`nCharset: windows-1251`nBinary`nUpload: " A_ScriptDir "\photo.jpg"
but it dont work:
Spoiler


when I used old httpQuery, the request use Multipart data and it works.
Spoiler


So i need to use the same method for upload photos but with HTTPRequest.

Im trying to manually make Headers and whole POST buffer. But I have problems with POST buffer.
Because POST body must have text (multipart header) + binary jpeg data and I cant make it work together.
I maked POST request with multipart headers at start and end, and where file binaries must be I filled with "+" character.
Spoiler


But when I copy file binaries to position where "+" starts, it dosent work!.

It is all what I get:
Spoiler


I tryed StrPut method and Memory copy method. But they work the same:
StrPut(fileContainer, &POST + pos, fileSize, "CP1251")

DllCall("RtlMoveMemory"
, "uInt", (offset:=&POST + pos)
, "uInt", &fileContainer
, "uInt", fileSize)
I have idea that I need somehow send in this order:
1. prePOST-TextData header - multipart boundary, content-type, length e.t.c
2. then send binary data from fileContainer
3. afterPOST-TextData (close multipart boundary)

But how I can make it sad.png

Maybe I can put text and binaries with File.RawWrite to some file. and try to just Upload it with standart Upload: option?

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
Are you following the example of preparing multipart data found here?

Using a file to prepare the multipart form would also work.

pantagruel
  • Members
  • 97 posts
  • Last active: Oct 01 2013 08:18 PM
  • Joined: 08 Oct 2006

Hi,

 

On Edit - I found my mistake, did not understand that the response data was buffered into the second parameter - variable. 
 
I have a script like this:
 

#v::
  URL = http://example.com/path/value
  text := HTTPRequest(URL)
  MsgBox, %text%
return

 
The Url returns text/plain, utf-8 encoded.
 
When I look at %text% it is the number of characters in the response.
 
I guess this means I should be providing a callback function to read the response, can you show me an example of that where at the end I can output the text I received back?

 

Ok I see what I did wrong, I needed to do

output_txt := ""
text := HTTPRequest(URL,output_txt)

And then get the value of output_txt



Wicked
  • Members
  • 504 posts
  • Last active: Nov 18 2018 02:17 AM
  • Joined: 07 Jun 2008
For simple stuff like that, I simply use:
start:=a_tickcount
result:=_download_to_var("http://google.com")
finish:=a_tickcount
msgbox,% "The first request took " finish-start "ms."

start:=a_tickcount
result:=_download_to_var("http://google.com")
finish:=a_tickcount
msgbox,% "The second request took " finish-start "ms."

if result.success
    fileappend,% result.responsetext,google.html

_download_to_var(url){
	static request:=comobjcreate("WinHttp.WinHttpRequest.5.1"),last,result:=object()
	if(url!=last){
		request.open("GET",url)
		last:=url
	}
	request.send()
	result.responsetext:=request.responsetext
	result.success:=request.responsetext!="failed"&&request.status==200?true:false
	return result
}
The main reason I use that, is so if the URL hasn't changed, it won't bother reopening the connection. As you can see, it speeds it up quite a bit.

3nL8f.png


cylisme
  • Members
  • 8 posts
  • Last active: Oct 20 2013 10:51 PM
  • Joined: 05 Jul 2011

i read some book about http and tcp/ip and know 

 

tcp/ip have slow start up  ;transfer as much as possible per package

 

http have  bring with the tcp/ip (got it mesage)

 

since  keep-alive not  in the   http 1.1  Standard  doc

 

but  the B/S  achieved a lot

 

 

 

conntceion:keep-alive

keep-alive:max=5,timeout=120

 

 

  Is HTTPRequest support it ?

If so could you please give some example how to use  



VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
No. HTTPRequest closes the request, connection, and wininet resources after each transaction.
 
To work with persistent HTTP connections, use a WinHttpRequest object. For more general TCP/IP stuff, check out WinSock (http://www.autohotke...ip/#entry337423)