Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

[Func] HTTPRequest: for Web APIs [AHK-B + AHK-L+Unicode+x64]


  • Please log in to reply
380 replies to this topic
VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
                                Download Version 9-27-2011
BUGFIX: typos in the header assemblage and codepage conversion sections.

OPTION: "NoHash" specified as an option turns off the default behavior of calculating the MD5 hash of downloaded data when the response headers have a "Content-MD5" header.


Can someone post a POST example for this URL:
<!-- m -->http://www.w3schools... ... tToCelsius<!-- m -->

And can we use multiple flags, when calling the functions?

Thanks,
Twhyman

As I was testing your requested example, I found some bugs. Please download the new version (link @ top).

Multiple flags (and options) must be separated by newlines. I recommend using a continuation section to set up whatever options you desire.

Here's your requested example:
Gui, Font, S11, Lucida Bright
Gui, Margin, 3, 3
Gui, Add, Text, w303 h22 center 0x200 Fahrenheit to Celcius API example.
Gui, Add, Edit, xm w150 vdegf right section
Gui, Add, Edit, ys w150 vdegc right
Gui, Add, Button, xm w303 default gdoit, Convert
Gui, Show,, Simple API Example.
Return
GuiClose:
Exitapp

doit:
	Gui, Submit, NoHide
	API_EndPoint := "http://www.w3schools.com/webservices/tempconvert.asmx/FahrenheitToCelsius"
	Headers =
	( LTRIM %
		Referer: http://www.w3schools.com/webservices/tempconvert.asmx?op=FahrenheitToCelsius
		Content-Type: application/x-www-form-urlencoded; charset=utf-8
	)
	size := HTTPRequest( API_EndPoint, Data := "Fahrenheit=" degf, Headers, "Codepage" )
	; the 'codepage' option tells HTTPRequest to convert the POST data to the charset specified in the 'Content-Type' header (UTF-8 here).
	StringLeft, Data, Data, InStr( Data, "</string" ) - 1
	StringTrimLeft, Data, Data, InStr( Data, ">", 0, 0 )
	GuiControl,, degc, % Data
Return
Tested using HTTPRequest (9-27-2011) with both AHK-Basic and AHK-L (1.1.00.00 unicode).

Chichipio
  • Guests
  • Last active:
  • Joined: --
Hey, I've been using your functions for a few day and I'm really happy with how easy you made it. Thanks.

I detected something while sniffing the packages it creates that, even though it doesn't disrupt anything, it still seems weird.

This is what's actually being sent in a basic GET request with default headers:
GET / HTTP/1.1
Accept: text/xml, text/json q=0.4, text/html q=0.3, text/* q=0.2, */* q=0.1
Accept: text/xml, text/json q=0.4, text/html q=0.3, text/* q=0.2, */* q=0.1
User-Agent: DemonoidUpdater/1.0 (Language=AutoHotkey/1.1.03.00; Platform=WIN_XP)
Host: www.whatismyip.org
Connection: Keep-Alive
Cache-Control: no-cache

Do you see the repeated "Accept:" line?

The same happens if I use a custom header containing an "Accept:" line. My line appears in the headers, but so does one with the same text as above. Do you know why this happens?

Oh, and if I use a referer in my headers, there's a similar behavior and I end up with:
Referrer: http://www.blabla.com
Referer: http://www.blabla.com

Not only does it repeat the header, but also uses the wrong name for the field "Referrer."

Like I said, it works fine regardless, but I thought you would like to know about these little details.
Keep up the good work.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

Like I said, it works fine regardless, but I thought you would like to know about these little details.
Keep up the good work.

TYVM.

I haven't done any verification on the packet-level, but you confirmed something I suspected. When setting up the connection using wininet, there are several headers that are accepted explicitly by various functions (agent, referrer, accept-types, and a few others). I thought that wininet would add them to the request headers, but had no idea whether it took duplicates into account. I will investigate the accept header more closely, possibly changing the default to simply '*/*'.

I won't claim to understand the whole 'referer'/'referrer' thing. According to this, the misspelling is correct. Having both is probably just as well.

fragman
  • Members
  • 1591 posts
  • Last active: Nov 12 2012 08:51 PM
  • Joined: 13 Oct 2009
The static variable URL_Components is declared twice and the newest AHK_L version does not like that ;)

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

The static variable URL_Components is declared twice and the newest AHK_L version does not like that ;)

:oops: *fixed*

twhyman
  • Members
  • 348 posts
  • Last active: Sep 17 2014 01:55 AM
  • Joined: 07 Dec 2005

                                Download Version 9-27-2011
BUGFIX: typos in the header assemblage and codepage conversion sections.

OPTION: "NoHash" specified as an option turns off the default behavior of calculating the MD5 hash of downloaded data when the response headers have a "Content-MD5" header.


Can someone post a POST example for this URL:
<!-- m -->http://www.w3schools... ... tToCelsius<!-- m -->

And can we use multiple flags, when calling the functions?

Thanks,
Twhyman

As I was testing your requested example, I found some bugs. Please download the new version (link @ top).

Multiple flags (and options) must be separated by newlines. I recommend using a continuation section to set up whatever options you desire.

Here's your requested example:
Gui, Font, S11, Lucida Bright
Gui, Margin, 3, 3
Gui, Add, Text, w303 h22 center 0x200 Fahrenheit to Celcius API example.
Gui, Add, Edit, xm w150 vdegf right section
Gui, Add, Edit, ys w150 vdegc right
Gui, Add, Button, xm w303 default gdoit, Convert
Gui, Show,, Simple API Example.
Return
GuiClose:
Exitapp

doit:
	Gui, Submit, NoHide
	API_EndPoint := "http://www.w3schools.com/webservices/tempconvert.asmx/FahrenheitToCelsius"
	Headers =
	( LTRIM %
		Referer: http://www.w3schools.com/webservices/tempconvert.asmx?op=FahrenheitToCelsius
		Content-Type: application/x-www-form-urlencoded; charset=utf-8
	)
	size := HTTPRequest( API_EndPoint, Data := "Fahrenheit=" degf, Headers, "Codepage" )
	; the 'codepage' option tells HTTPRequest to convert the POST data to the charset specified in the 'Content-Type' header (UTF-8 here).
	StringLeft, Data, Data, InStr( Data, "</string" ) - 1
	StringTrimLeft, Data, Data, InStr( Data, ">", 0, 0 )
	GuiControl,, degc, % Data
Return
Tested using HTTPRequest (9-27-2011) with both AHK-Basic and AHK-L (1.1.00.00 unicode).


Thanks for the example :D

Can you provide another one but this time using the SOAP example?

Thanks,
Twhyman
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
:oops: Once again, I found a bug that needed fixing...

Here's the SOAP example:
Gui, Font, S11, Lucida Bright
Gui, Margin, 3, 3
Gui, Add, Text, w303 h22 center 0x200 Fahrenheit to Celcius API example.
Gui, Add, Edit, xm w150 vdegf right section
Gui, Add, Edit, ys w150 vdegc right
Gui, Add, Button, xm w303 default gdoit, Convert
Gui, Show,, Simple API Example.
Return
GuiClose:
Exitapp

doit:
	Gui, Submit, NoHide
	API_EndPoint := "http://www.w3schools.com/webservices/tempconvert.asmx"
	Headers := "Content-Type: application/soap+xml; charset=utf-8"
	Data =
	( LTRIM JOIN
		<?xml version="1.0" encoding="utf-8"?>
		<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
			<soap12:Body>
				<FahrenheitToCelsius xmlns="http://tempuri.org/">
					<Fahrenheit>%degf%</Fahrenheit>
				</FahrenheitToCelsius>
			</soap12:Body>
		</soap12:Envelope>
	)
	size := HTTPRequest( API_EndPoint, Data, Headers, "Codepage" )
	StringLeft, Data, Data, InStr( Data, "</Fahre" ) - 1
	StringTrimLeft, Data, Data, InStr( Data, ">", 0, 0 )
	GuiControl,, degc, % Data
Return


twhyman
  • Members
  • 348 posts
  • Last active: Sep 17 2014 01:55 AM
  • Joined: 07 Dec 2005

:oops: Once again, I found a bug that needed fixing...

Here's the SOAP example:

Gui, Font, S11, Lucida Bright
Gui, Margin, 3, 3
Gui, Add, Text, w303 h22 center 0x200 Fahrenheit to Celcius API example.
Gui, Add, Edit, xm w150 vdegf right section
Gui, Add, Edit, ys w150 vdegc right
Gui, Add, Button, xm w303 default gdoit, Convert
Gui, Show,, Simple API Example.
Return
GuiClose:
Exitapp

doit:
	Gui, Submit, NoHide
	API_EndPoint := "http://www.w3schools.com/webservices/tempconvert.asmx"
	Headers := "Content-Type: application/soap+xml; charset=utf-8"
	Data =
	( LTRIM JOIN
		<?xml version="1.0" encoding="utf-8"?>
		<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
			<soap12:Body>
				<FahrenheitToCelsius xmlns="http://tempuri.org/">
					<Fahrenheit>%degf%</Fahrenheit>
				</FahrenheitToCelsius>
			</soap12:Body>
		</soap12:Envelope>
	)
	size := HTTPRequest( API_EndPoint, Data, Headers, "Codepage" )
	StringLeft, Data, Data, InStr( Data, "</Fahre" ) - 1
	StringTrimLeft, Data, Data, InStr( Data, ">", 0, 0 )
	GuiControl,, degc, % Data
Return


Thanks for the quick reply :D

What bug did you find do I need to re-download the HTTPQuery file?
Can you help on redirecting the script traffic to a proxy so I can do more debug using fiddler?

Setting Proxy: 127.0.0.0:8888 did not work.

does the function encode the POST data itself to utf-8, because I am sending Hebrew in the xml?

Thanks,
Twhyman
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006

What bug did you find do I need to re-download the HTTPQuery file?

The bug was in the POST codepage conversion and only manifested in ansi versions of AHK. If you use unicode AHK, you don't need this particular patch.

Can you help on redirecting the script traffic to a proxy so I can do more debug using fiddler?

Setting Proxy: 127.0.0.0:8888 did not work.

I don't have any experience or knowledge of using proxies, but I suspect the proxy host must be configured to act as such (I have no idea how you would configure a computer to behave like a proxy).

does the function encode the POST data itself to utf-8, because I am sending Hebrew in the xml?

HTTPRequest converts the POST text if and only if the content-type header specifies the desired charset AND the 'codepage' option is present. UTF-8 seems to be universally acceptable across the internet, so that's why I used it in the examples.

twhyman
  • Members
  • 348 posts
  • Last active: Sep 17 2014 01:55 AM
  • Joined: 07 Dec 2005
Hi,

There seems to be a problem with encoding Hebrew chars

Here is a XML i POST with PHP and the Hebrew chars are encoded correctly, Regarding the PHP encode I have found This which shows that the first letter is "Octal Escape Sequence":

Posted Image


Here is the same packet POSTed with HTTPRequest:
Posted Image

I tried to find what kind of encoding is used in AHK packet
but i could not find....
I am sending a 4 letter Hebrew word: חתול which means cat :)

Can you help with this problem, If it happens with Hebrew I believe it will happen with every non English language.

Thanks,
Twhyman :?:
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.

VxE
  • Moderators
  • 3622 posts
  • Last active: Dec 24 2015 02:21 AM
  • Joined: 07 Oct 2006
Without seeing your code, I can only guess that you are submitting the POST data in UTF-16 encoding.

If you desire, you can use StrPut() to convert the POST data prior to uploading it (just be sure to specify the Content-Length and Content-Type headers).

twhyman
  • Members
  • 348 posts
  • Last active: Sep 17 2014 01:55 AM
  • Joined: 07 Dec 2005

":2yox9oxg]Without seeing your code, I can only guess that you are submitting the POST data in UTF-16 encoding.

If you desire, you can use StrPut() to convert the POST data prior to uploading it (just be sure to specify the Content-Length and Content-Type headers).



Here is the code can you show me how to encode prior to sending and calculate the content-Length?
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory.

#Include %A_ScriptDir%\Functions\HTTPRequest.ahk
Contacts_File=%A_ScriptDir%\contacts\contacts.txt
FileEncoding , UTF-8

MsgBox % A_IsUnicode ? "Unicode" : "ANSI"
msgbox, %A_FileEncoding%

Edit_Global_Var=0

Logo_tmp=%A_Temp%\logo.png

FileInstall, logo.png, %A_Temp%\logo.png

Gui, Font, s15 cBlue, Verdana
Gui, Add, Text, x60 y5 w540 h30 Center , מערכת שליחת הודעות - 
;Gui, Tab, Main
Gui, Font
Gui, Add, Picture, x20 y220 w250 h-1, %Logo_tmp%
Gui, 1:Add, Edit, x12 y50 w260 h120 vSMS_TXT +Right, Test ENG Team TEST SMS!

Gui, 1:Add, ListView, x290 y50 w340 h390 Checked Sort, שם|טלפון
;LV_ModifyCol(1,130) ; Auto-size each column to fit its contents.
;LV_ModifyCol(2,106) ; Auto-size each column to fit its contents.
;LV_ModifyCol() ; Auto-size each column to fit its contents.

Gui, 1:Add, Button, x100 y180 w100 h30 gSend_SMS, !שלח

Gui, 1:Add, Button, x289 y445 w100 h30 gAdd_Record, הוספת רשומה
Gui, 1:Add, Button, x410 y445 w100 h30 gDelete_Record, מחיקת רשומה
Gui, 1:Add, Button, x531 y445 w100 h30 gSave_Records, שמור שינויים

Gui, Color, FFFFFF
Gui, 1:Show, h483 w646, מערכת שליחת הודעות
Read_Contatcts()

LV_ModifyCol(2, "Integer") ; For sorting purposes, indicate that
LV_ModifyCol() ; Auto-size each column to fit its contents.
Return

GuiClose:
FileDelete, %Logo_tmp%
ExitApp

Add_Record:
Gui, 2:+owner1
Gui +Disabled ; Disable main window.
Gui, 2:Add, Text, x2 y0 w440 h30 +Center, הוספת רשומה
Gui, 2:Add, Text, x382 y40 w60 h20 +Right, :שם
Gui, 2:Add, Edit, x12 y40 w360 h20 vAdd_name +Right, Name
Gui, 2:Add, Text, x382 y70 w60 h20 +Right, :טלפון
Gui, 2:Add, Edit, x12 y70 w360 h20 vAdd_Phone, Phone
Gui, 2:Add, Button, x12 y100 w100 h30 gSave_Button, שמור
Gui, 2:Add, Button, x272 y100 w100 h30 gCancel_Button, ביטול
; Generated using SmartGUI Creator 4.0
Gui, 2:Show, x559 y370 h143 w448, הוספת רשומה
Return

Cancel_Button:
2GuiEscape:
2GuiClose:
Gui,2:Destroy
Gui,1:-Disabled
Gui,1:Default
return

Save_Button:
Gui,2:Submit
Gui,1:-Disabled
Gui,2:Destroy
Gui,1:Default
LV_Add("", Add_name,Add_Phone)
LV_ModifyCol()
Edit_Global_Var=1
Return

Delete_Record:
Loop
{
RowNumber := LV_GetNext(RowNumber,"Checked") ; Resume the search at the row after that found by the previous iteration.
if not RowNumber ; The above returned zero, so there are no more selected rows.
break

LV_GetText(Name, RowNumber, 1)

MsgBox, 524340, מחיקת רשומה,?האם אתה בטוח שברצונך למחוק את הרשומה הבאה`n%NAME%
IfMsgBox, Yes
LV_Delete(RowNumber)
}
Edit_Global_Var=1
Return

Save_Records:
Gui, 1:Submit, NoHide
If(Edit_Global_Var=1)
{
Msgbox, 524356, הודעה, ?האם לשמור שינויים
IfMsgBox, Yes
{
;--- Reads All ListView's populated lines ---
FileCopy, %Contacts_File%, %Contacts_File%.bak, 1
FileDelete, %Contacts_File%
RowNumber=1
Loop,
{
LV_GetText(Name, RowNumber, 1)
LV_GetText(Phone_L_A, RowNumber, 2)

If (Name = "")
{
RowNumber=1
Break
}
FileAppend, %Name%`,%Phone_L_A%`n, %Contacts_File%
RowNumber++
}
Edit_Global_Var=0
}
}
Else
{
MsgBox, 524352, הודעה, .לא בוצעו שינויים, השמירה בוטלה
Return
}
Return

Send_SMS:
Gui, 1:Default
Gui, 1:Submit, NoHide
Send_Array =
POSTData =
Headers =
My_Options =

API_EndPoint := "http://www.w3schools.com/imsc/interfaces/largeaccount/la3.sms"
;API_EndPoint := "https://vast.bezeq.co.il/imsc/interfaces/largeaccount/la3.sms"

Headers =
( LTRIM %
Content-Type: text/xml; charset=utf-8
User-Agent: Tal_Z_BOJ/1.0 (Language=AHK/1.0.48.05; Platform=WIN7)
)

My_Options =
( LTRIM C ; options should also be separated by newlines.
CodePage
+INTERNET_FLAG_NO_COOKIES
+INTERNET_FLAG_RELOAD
)


RowNumber = 0 ; This causes the first loop iteration to start the search at the top of the list.
Loop
{
RowNumber := LV_GetNext(RowNumber,"Checked") ; Resume the search at the row after that found by the previous iteration.
if not RowNumber ; The above returned zero, so there are no more selected rows.
{
msgbox, 524352, הודעה, .לא נבחרו נמענים
;Return
Break
}


LV_GetText(Name, RowNumber, 1)
LV_GetText(Phone_L_A, RowNumber, 2)
;MsgBox The next selected row is #%RowNumber%, whose first field is "%Name%".`n%Phone_L_A%

StringSplit, Phone_A, Phone_L_A, `,
If (Phone_A0 > 1)
{
;Msgbox, It got more then 1 phone
Array_Size=%Phone_A0%
Loop, %Array_Size%
{
Current_Phone:=Phone_A%A_Index%
Fixed_Phone := RegExReplace(Current_Phone, "(^\d)", "972") ; Returns "abc123xyz"
Send_Array=<cli>%Fixed_Phone%</cli>%Send_Array%
;Msgbox, %Send_Array%
}
}
Else
{
Phone_L_A := RegExReplace(Phone_L_A, "(^\d)", "972") ; Returns "abc123xyz"
;msgbox, %Phone_L_A%
Send_Array=<cli>%Phone_L_A%</cli>%Send_Array%
}
}



POSTData =
( LTRIM JOIN
<?xml version=`"1.0`" encoding=`"utf-8`" ?>
<request>
<head>
<auth>
<account>account</account>
<user>username</user>
<pass>password</pass>
</auth>
<action>sendsms</action>
</head>
<body>
<addr>
<from>0523631700</from>
<to>%Send_Array%</to>
</addr>
<data>
<msgtype>text</msgtype>
<text>%SMS_TXT%</text>
</data>
<billing>
<port>0</port>
</billing>
</body>
</request>
)
;POSTData:="<?xml version=""1.0"" encoding=""utf-8"" ?><request><head><auth><account>account</account><user>username</user><pass>password</pass></auth><action>sendsms</action></head><body><addr><from>0523631700</from><to>" . Send_Array . "</to></addr><data><msgtype>text</msgtype><text>" . SMS_TXT . "</text></data><billing><port>0</port></billing></body></request>"
Msgbox, Sending!`n`n`n`n%POSTData%

fileappend, %POSTData%`n`n`n,%A_ScriptDir%\tal.txt
size := HTTPRequest(API_EndPoint, POSTData, Headers, My_Options)
msgbox, %POSTData%
Return

Read_Contatcts()
{
Global Contacts_File

Loop, Read, %Contacts_File%
{
Current_Read_line_TXT=%A_LoopReadLine%
Current_Read_line_Index=%A_Index%

StringSplit, Details_Array, Current_Read_line_TXT, `,
If (Details_Array0=2)
LV_Add("",Details_Array1, Details_Array2)
Else
{
RegExMatch(Current_Read_line_TXT,"^(.*?),(.*)",R_Details_Array)
;msgbox, %R_Details_Array1%`n`n%R_Details_Array2%
LV_Add("",R_Details_Array1,R_Details_Array2)
}
}
}

(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.

widow
  • Guests
  • Last active:
  • Joined: --
twhyman:
maybe you want to exklude your username and password from that sample.

twhyman
  • Members
  • 348 posts
  • Last active: Sep 17 2014 01:55 AM
  • Joined: 07 Dec 2005

twhyman:
maybe you want to exklude your username and password from that sample.


Thanks, did not noticed its was posted in two place in the code, I only changed one place :D
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.

widow
  • Guests
  • Last active:
  • Joined: --

twhyman:
maybe you want to exklude your username and password from that sample.


Thanks, did not noticed its was posted in two place in the code, I only changed one place :D

np, i recommend to change password also..