sending a tweet with HTTP COM object

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
chthonicastro
Posts: 8
Joined: 21 May 2018, 04:47

sending a tweet with HTTP COM object

21 May 2018, 05:39

Hello! First post! First, a thank you to the forum for being so wonderful, it has resolved so many needs of mine as I have tinkered away over the past year or two. Y'all have talked about and solved most everything out there.

But after quite a bit of scouring, I have not found a solution to this problem anywhere, here or across the coding internet. I am trying to interact with Twitter and the Twitter API directly via AHK, rather than through curl, or twurl, or the libraries built up around other coding languages. So please don't respond "just use x language" or "just run curl" or "just import x". I understand that these are all (better lol) options, but I'm a tinkerer, and just trying to have fun with AHK. I have a larger project that I've managed to develop all with AHK, so I want to see it through. And no, I don't want to debate whether or not a COM object is more or less acceptable than the other obviously more straightforward extra-AHK solutions. THANK YOU for putting up with the restraints. ; )

Code to follow. I think an important part of my problem is the way I am doing my HTTP header as well as the way I am finally sending the status to Twitter. I have already done EXTENSIVE work getting the OAuth to work properly, and I don't believe the problem is there. I've meticulously matched each step to the twitter requirements, as far as getting the percent encoding and hashing all in the right order. I consistently get an Error 32 "Unable to Authenticate," response. But again, I don't think the literal authorization is the problem, but rather the fact that I'm sending the request incorrectly, and as a result, the authorization doesn't match my request, and then it gets booted.

Code: Select all

WinHTTP := ComObjCreate("WinHTTP.WinHttpRequest.5.1")
WinHTTP.Open("POST", https://api.twitter.com/1.1/statuses/update.json, 0)	;per Twitter, kind of, their instructions just say, any competent set of tools will turn this into a properly formatted request FOR YOU, and it is unclear just how much of the URL should be used in the Open request
WinHTTP.SetRequestHeader("Accept", "*/*")							;per Twitter
WinHTTP.SetRequestHeader("Connection", "close")						;per Twitter
WinHTTP.SetRequestHeader("User-Agent", "OAUTH gem v.0.4.4")				;per Twitter
WinHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")     ;per Twitter
WinHTTP.SetRequestHeader("Authorization", DST)						;DST is a long string with signing keys which I have already meticulously formatted, I don't think the problem is here
WinHTTP.SetRequestHeader("Content-Length", 56)						;the length of what is included in the send request below
WinHTTP.SetRequestHeader("Host", "api.twitter.com")					;per Twitter
WinHTTP.Send(status=Just%20trying%20to%20use%20an%20app%2C%20folks%21)   ;in my code this is actually passed as a variable and there are no escaping issues with the = or %
result := WinHTTP.ResponseText										;and this is where I am used to receiving "unable to authenticate"
MsgBox %result%
Any thoughts? I can show the rest of the code on how DST is generated if everyone concurs on the above. But it's quite involved.
And if we get through to the solution, I am also happy to share the whole project : D perhaps a start to some AHK specific tools related to Twitter.
gregster
Posts: 8992
Joined: 30 Sep 2013, 06:48

Re: sending a tweet with HTTP COM object

21 May 2018, 22:33

I don't know the Twitter API, but I know a few others. With that said - and the limited code excerpt - I rather see a AHK syntax problem here.

By just looking at your code snippet (without knowing anything about the necessary headers, I would strongly assume that this line is not doing what you expect:

Code: Select all

WinHTTP.Send(status=Just%20trying%20to%20use%20an%20app%2C%20folks%21)   ;in my code this is actually passed as a variable and there are no escaping issues with the = or %
I assume, status=Just%20trying%20to%20use%20an%20app%2C%20folks%21 needs to be a literal string and therefore placed between quotation marks, like ", because function calls are expressions. In this case, it will actually somehow be evaluated by AHK as 1 (true?), while I would have expected 0 or blank (seems to have something to do with the = in this expression), but it is probably not what you want and won't be in line with the content-length header anyway...

Now, you say in the comment that you actually pass this as a variable - so that might not be the actual problem in your real code, if you assigned the string correctly to a variable ( var := "string") . But how should we know ;) ?

Btw, are you sure that this string should be in the request body? It is possible, but from the looks of it, it might as well work as part of a query string with the URL...

In my experience, error codes often hint in the right direction (although the real reason why something fails is sometimes lying elsewhere). So I would recommend showing how you created DST (how do you know it is right?), if you want help debugging the API call. I would also recommend to always link to the relevant documentation. Otherwise, potential helpers will have more work than they might willing to do or to make assumptions about what you did that might not be true. (I am personally always interested in getting APIs to work, because I already had to fiddle with different ones (in the end successfully) and try to collect more experience for future use cases, but I am not at all interested in (using) Twitter itself... so my motivation (and time) to dig deeply into the Twitter API is not very big. But we also might have people here, who have already working code).

Oh, now that I had a second quick look, I am also sure that the url string in

Code: Select all

WinHTTP.Open("POST", https://api.twitter.com/1.1/statuses/update.json, 0)
needs to be in quotation marks, too. :!: That most probably breaks the whole API call already at this point.

Now, I would also check if you made this same mistake with literal strings when you created the DST.

If that doesn't fix it, I would recommend to post your whole API call (of course, excluding your actual credentials), so that helpers can actually test the code, if they are willing to get a personal API key (not sure how that works for Twitter)... don't forget links to the used docs, then! :)
chthonicastro
Posts: 8
Joined: 21 May 2018, 04:47

Re: sending a tweet with HTTP COM object

22 May 2018, 05:53

WELL, first there is a mea culpa. I was mostly wondering if there was a glaring error in the way I was using the WinHTTP COM object, and it appears that I created a few, in "simplifying" the code so that we only had to look at what I was doing with WinHTTP! I will simply aver that essentially, I do either pass variables or use the ""s in an appropriate manner. And have otherwise tried every version possible of passing the variable or quoting the string or what have you haha in the process of trying to root out my mistake.

So, twitter, rather infuriatingly, lays it out this way in the developer docs:
Spoiler
Here is my whole script, what a beast, closely hewing to the Twitter instructions:

Code: Select all

^+a::

WinHTTP := ComObjCreate("WinHTTP.WinHttpRequest.5.1")

;used by HexTo64 function
StringCaseSense On
Chars = ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/

;used to generate oauth_nonce, a random one time identifier, used for OAuth. I generate a string of 32 numbers and convert it to base 64 before stripping non-word characters
;Twitter states "any approach which produces a relatively random alphanumeric string should be OK here."
randomNumber := Return32()
oauth_nonce := Base64(randomNumber)
oauth_nonce := StrReplace(oauth_nonce, "=", "")
oauth_nonce := StrReplace(oauth_nonce, "/", "")
oauth_nonce := StrReplace(oauth_nonce, "+", "")

;generated by twitter in App settings, used to identify my specific account. This is not my real token. Used for OAuth.
;I am just trying to post to my own account. Baby steps.
oauth_token = THISISASECRETUSEDTOIDENTIFYMYTWITTERACCOUNT

;Current date in UNIX seconds. Used for OAuth. The -8*3600 is to correct for my current time zone. Used for OAuth.
oauth_timestamp := 31536000*(A_YYYY-1970) + (A_Yday+Floor((A_YYYY-1972)/4))*86400 + A_Hour*3600 + A_Min*60 + A_Sec - 8*3600

;generated by twitter in App settings. identifies my app. Used for OAuth. It is okay for me to share this!
oauth_consumer_key = 2dR18FCdUY302FeodocTzZqgc

;default. Used for OAuth. Per Twitter.
oauth_signature_method = HMAC-SHA1

;default. Used for OAuth. Per Twitter.
oauth_version = 1.0

;used to encode the request and build oauth_signature

	;generated by Twitter in App Settings. Neither are my real secrets.
	consumer_secret = THISSECRETAUTHENTICATESMYAPP
	oauth_token_secret = THISSECRETAUTHENTICATESMYOWNTWITTERACCOUNT
	;the two secrets linked together according to Twitter's formatting. 
	signing_key = %consumer_secret%&%oauth_token_secret%

	;used in the signature base string which is ultimately hashed with the signing_key.
	status = Just trying to use an app, folks!
	encodedStatus := PercentEncode(status)
	
	;used in signature base string. Built according to Twitter's & OAuth specifications. 
	;Twitter throws a different error (bad request) if this is malformed, but I have weeded those issues out and now consistently get "unable to auhtenticate"
	parameterString = include_entities`=true&oauth_consumer_key`=%oauth_consumer_key%&oauth_nonce`=%oauth_nonce%&oauth_signature_method`=%oauth_signature_method%&oauth_timestamp`=%oauth_timestamp%&oauth_token`=%oauth_token%&oauth_version`=%oauth_version%&status`=%encodedStatus%
	parameterString := PercentEncode(parameterString)

	;takes the base URL for the request and uses the proper Percent Encoding. Used in signature base string. See comment above on malformed vs. unable to authenticate.
	url = https://api.twitter.com/1.1/statuses/update.json
	urlEncode := PercentEncode(url)

	;combines several above strings to use in hashing. Built according to Twitter's & Oauth specifications.
	signature_base_string = POST&%urlEncode%&%parameterString%

	;creates oauth_signature
	;I have tested both HMAC and HexToBase64 functions using the example strings in Twitter's documentation, and the result is a properly formed signature. I have also verified strings generated by myself using other online "we'll hash this for you" services.
	oauth_signature := HMAC(signing_key, signature_base_string, "SHA")
	oauth_signature := HexToBase64(oauth_signature)
	oauth_signature := PercentEncode(oauth_signature)

	;Authorization command. DST is built according to Twitter's & OAuth specifications. My format matches Twitter's to a T, excepting my own values of course.
	;One thing that IS infuriating is that Twitter uses a rather idiosyncratic set of excepted characters for percent encoding. And there is at least one part of their documentation where they say "now percent encode this" but then when they show a sample of the correct work it appears they only meant a specific portion of it.
	DST = OAuth oauth_consumer_key`="%oauth_consumer_key%", oauth_nonce`="%oauth_nonce%", oauth_signature`="%oauth_signature%", oauth_signature_method`="%oauth_signature_method%", oauth_timestamp`="%oauth_timestamp%", oauth_token`="%oauth_token%", oauth_version`="%oauth_version%"
	
	;To pass with the SEND command
	encodedStatus = status`=%encodedStatus%

WinHTTP.Open("POST", "https://api.twitter.com/1.1/statuses/update.json", 0)
WinHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
WinHTTP.SetRequestHeader("Authorization", DST)
WinHTTP.Send(encodedStatus)
Result := WinHTTP.ResponseText
MsgBox %result%
return

;Graciously lifted from these forums. The core work, is I believe, from Laszlo back all the way to '12
;But this is slightly different and I cannot find where it was originally lifted from. 
;See comments above, I have tested this against examples and other services, it works properly.
Base64(string, key:="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/") ;
{
    StringCaseSense On
    Loop, Parse, string
    {
        index := Mod(A_Index, 3)
        if (index = 1)
        {
            base64Index := ((Asc(A_LoopField) >> 2) & 0x3F)
            lastBin := (Asc(A_LoopField) & 0x03 ) << 4
        }
        else if (index = 2) 
        {
            base64Index := lastBin | ((Asc(A_LoopField) >> 4) & 0x0F)
            lastBin := (Asc(A_LoopField) & 0x0F) << 2       
        }
        else
        {
            base64Index := lastBin | ((Asc(A_LoopField) >> 6) & 0x03)
            StringMid, base64Char, key, base64Index + 1, 1
            code := code base64Char

            base64Index := (Asc(A_LoopField)) & 0x3F
        }
        StringMid, base64Char, key, base64Index + 1, 1
        code := code base64Char
    }
    if (index = 1)
    {
        StringMid, base64Char, key, lastBin + 1, 1
        return code base64Char "=="
    }
    else if (index = 2)
    {
        StringMid, base64Char, key, lastBin + 1, 1
        return code base64Char "="
    }
    else
    {
        return code "=="
    }
}

;used in the generation of OAuth_nonce
Return32()
{
	randomNumber =
	Loop, 32
	{
		Random, holder, 0, 9
		randomNumber = %randomNumber%%holder%
	}
	return randomNumber
}

;Basic characters to watch out for in percent encoding. The lines that have been commented out are twitter's excepted characters.
;The % is placed first in the function as there appears to be only one instance, in the final step of the signature_base_string, where the % needs to be itself encoded, and it only affects the "status".
PercentEncode(string)
{
	
	string := StrReplace(string, "%","%25")
	string := StrReplace(string, """","%22")
	;string := StrReplace(string, "-","%2D")
	;string := StrReplace(string, ".","%2E")
	string := StrReplace(string, "<","%3C")
	string := StrReplace(string, ">","%3E")
	string := StrReplace(string, "`\","%5C")
	string := StrReplace(string, "^","%5E")
	;string := StrReplace(string, "_","%5F")
	string := StrReplace(string, "``","%60")
	string := StrReplace(string, "`{","%7B")
	string := StrReplace(string, "|","%7C")
	string := StrReplace(string, "`}","%7D")
	;string := StrReplace(string, "`~","%7E")
	string := StrReplace(string, "!","%21")
	string := StrReplace(string, "#","%23")
	string := StrReplace(string, "$","%24")
	string := StrReplace(string, "&","%26")
	string := StrReplace(string, "'","%27")
	string := StrReplace(string, "`(","%28")
	string := StrReplace(string, "`)","%29")
	string := StrReplace(string, "*","%2A")
	string := StrReplace(string, "+","%2B")
	string := StrReplace(string, "`,","%2C")
	string := StrReplace(string, "`/","%2F")
	string := StrReplace(string, ":","%3A")
	string := StrReplace(string, "`;","%3B")
	string := StrReplace(string, "=","%3D")
	string := StrReplace(string, "?","%3F")
	string := StrReplace(string, "@","%40")
	string := StrReplace(string, "`[","%5B")
	string := StrReplace(string, "`]","%2D")
	string := StrReplace(string, " ","%20")
	return string
}

;Lifted from a script created by jNizM on github. https://github.com/jNizM/HashCalc
;Tested against Twitter's sample signatures and keys, as well as other online services, it works properly.

HMAC(Key, Message, Algo := "MD5")
{
    static Algorithms := {MD2:    {ID: 0x8001, Size:  64}
                        , MD4:    {ID: 0x8002, Size:  64}
                        , MD5:    {ID: 0x8003, Size:  64}
                        , SHA:    {ID: 0x8004, Size:  64}
                        , SHA256: {ID: 0x800C, Size:  64}
                        , SHA384: {ID: 0x800D, Size: 128}
                        , SHA512: {ID: 0x800E, Size: 128}}
    static iconst := 0x36
    static oconst := 0x5C
    if (!(Algorithms.HasKey(Algo)))
    {
        return ""
    }
    Hash := KeyHashLen := InnerHashLen := ""
    HashLen := 0
    AlgID := Algorithms[Algo].ID
    BlockSize := Algorithms[Algo].Size
    MsgLen := StrPut(Message, "UTF-8") - 1
    KeyLen := StrPut(Key, "UTF-8") - 1
    VarSetCapacity(K, KeyLen + 1, 0)
    StrPut(Key, &K, KeyLen, "UTF-8")
    if (KeyLen > BlockSize)
    {
        CalcAddrHash(&K, KeyLen, AlgID, KeyHash, KeyHashLen)
    }

    VarSetCapacity(ipad, BlockSize + MsgLen, iconst)
    Addr := KeyLen > BlockSize ? &KeyHash : &K
    Length := KeyLen > BlockSize ? KeyHashLen : KeyLen
    i := 0
    while (i < Length)
    {
        NumPut(NumGet(Addr + 0, i, "UChar") ^ iconst, ipad, i, "UChar")
        i++
    }
    if (MsgLen)
    {
        StrPut(Message, &ipad + BlockSize, MsgLen, "UTF-8")
    }
    CalcAddrHash(&ipad, BlockSize + MsgLen, AlgID, InnerHash, InnerHashLen)

    VarSetCapacity(opad, BlockSize + InnerHashLen, oconst)
    Addr := KeyLen > BlockSize ? &KeyHash : &K
    Length := KeyLen > BlockSize ? KeyHashLen : KeyLen
    i := 0
    while (i < Length)
    {
        NumPut(NumGet(Addr + 0, i, "UChar") ^ oconst, opad, i, "UChar")
        i++
    }
    Addr := &opad + BlockSize
    i := 0
    while (i < InnerHashLen)
    {
        NumPut(NumGet(InnerHash, i, "UChar"), Addr + i, 0, "UChar")
        i++
    }
    return CalcAddrHash(&opad, BlockSize + InnerHashLen, AlgID)
}

;Used by HMAC
CalcAddrHash(addr, length, algid, byref hash = 0, byref hashlength = 0)
{
    static h := [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "a", "b", "c", "d", "e", "f"]
    static b := h.minIndex()
    hProv := hHash := o := ""
    if (DllCall("advapi32\CryptAcquireContext", "Ptr*", hProv, "Ptr", 0, "Ptr", 0, "UInt", 24, "UInt", 0xf0000000))
    {
        if (DllCall("advapi32\CryptCreateHash", "Ptr", hProv, "UInt", algid, "UInt", 0, "UInt", 0, "Ptr*", hHash))
        {
            if (DllCall("advapi32\CryptHashData", "Ptr", hHash, "Ptr", addr, "UInt", length, "UInt", 0))
            {
                if (DllCall("advapi32\CryptGetHashParam", "Ptr", hHash, "UInt", 2, "Ptr", 0, "UInt*", hashlength, "UInt", 0))
                {
                    VarSetCapacity(hash, hashlength, 0)
                    if (DllCall("advapi32\CryptGetHashParam", "Ptr", hHash, "UInt", 2, "Ptr", &hash, "UInt*", hashlength, "UInt", 0))
                    {
                        loop % hashlength
                        {
                            v := NumGet(hash, A_Index - 1, "UChar")
                            o .= h[(v >> 4) + b] h[(v & 0xf) + b]
                        }
                    }
                }
            }
            DllCall("advapi32\CryptDestroyHash", "Ptr", hHash)
        }
        DllCall("advapi32\CryptReleaseContext", "Ptr", hProv, "UInt", 0)
    }
    return o
}

;also courtesy of these forums, Laszlo, 2012
HextoBase64(hex) 
{ 
   Loop Parse, hex
   {
      m := Mod(A_Index,3)
      x  = 0x%A_loopfield%
      IfEqual      m,1, SetEnv z, % x << 8
      Else IfEqual m,2, EnvAdd z, % x << 4
      Else {
         z += x
         o := o Code(z>>6) code(z)
      }
   }
   IfEqual m,2, Return o Code(z>>6) Code(z) "=="
   IfEqual m,1, Return o Code(z>>6) "="
   Return o
}

;used by the Base64 function
Code(i) {   ; <== Chars[i & 63], 0-base index
   Global Chars
   StringMid i, Chars, (i&63)+1, 1
   Return i
}
chthonicastro
Posts: 8
Joined: 21 May 2018, 04:47

Re: sending a tweet with HTTP COM object

22 May 2018, 08:31

The error I consistently get now is: "Error 32: Could Not Authenticate You"
The error that gets thrown when a % is in the wrong place in your encoding or you truly botch your hashing: "Error 215: Bad Authentication Data"

For many, many tries I was using a bad hashing function (dug out of the archives here), that still spit out properly formed hashes, and it would give me Error 32. But while I was working out all of the kinks of properly forming the various parts of the signature, or when I was failing to encode my hashed value into Base 64, I would get Error 215.

For all the tinkering I've done lately, I no longer get Error 215, but am still stuck with error 32. This is why I believed I had done the DST correctly, but wasn't sending the HTTP request in a recognizable fashion that matched up with the DST when it was broken down on Twitter's end.

Lastly, here is a sample DST from Twitter:

Code: Select all

OAuth oauth_consumer_key="xvz1evFS4wEEPTGEFPHBog", oauth_nonce="kYjzVBB8Y0ZFabxSWbWovY3uYSQ2pTgmZeNu2VS4cg", oauth_signature="tnnArxj06cWHq44gCs1OSKk%2FjLY%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1318622958", oauth_token="370773112-GmHxMAgYyLbNEtIKZeRNFsMKPR9EyMZeS9weJAEb", oauth_version="1.0"
and here is what mine looks like, with relevant secret values removed:

Code: Select all

OAuth oauth_consumer_key="2dR18FCdUY302FeodocTzZqgc", oauth_nonce="MzUwMjM0ODc3NDM2NTMwODQ3MTg4NzA4NTMxNjA1NzM", oauth_signature="IFUDAREDTOUNHASHTHISYOUDGETSECRETVALUESRELATEDTOMYTWITTERACCOUNTANDAPP", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1526995487", oauth_token="THISISASECRETVALUE-RELATEDTOMYAPP", oauth_version="1.0"
Here is a sample signature base string:

Code: Select all

POST&https%3A%2F%2Fapi.twitter.com%2F1.1%2Fstatuses%2Fupdate.json&include_entities%3Dtrue%26oauth_consumer_key%3Dxvz1evFS4wEEPTGEFPHBog%26oauth_nonce%3DkYjzVBB8Y0ZFabxSWbWovY3uYSQ2pTgmZeNu2VS4cg%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1318622958%26oauth_token%3D370773112-GmHxMAgYyLbNEtIKZeRNFsMKPR9EyMZeS9weJAEb%26oauth_version%3D1.0%26status%3DHello%2520Ladies%2520%252B%2520Gentlemen%252C%2520a%2520signed%2520OAuth%2520request%2521
and here is what mine looks like, with relevant secret values removed:

Code: Select all

POST&https%3A%2F%2Fapi.twitter.com%2F1.1%2Fstatuses%2Fupdate.json&include_entities%3Dtrue%26oauth_consumer_key%3D2dR18FCdUY302FeodocTzZqgc%26oauth_nonce%3DMzUwMjM0ODc3NDM2NTMwODQ3MTg4NzA4NTMxNjA1NzM%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1526995487%26oauth_token%THISISASECRETVALUE-RELATEDTOMYAPP%26oauth_version%3D1.0%26status%3DJust%2520trying%2520to%2520use%2520an%2520app%252C%2520folks%2521
chthonicastro
Posts: 8
Joined: 21 May 2018, 04:47

Re: sending a tweet with HTTP COM object

22 May 2018, 08:47

YOOOOOOO I hold twitter responsible for this. But I got it to work! Thank you all for your diligent help : > ) Feel free to pilfer for any of your Tweeting-from-AHK needs.

The open request should be WinHTTP.Open("POST", "https://api.twitter.com/1.1/statuses/up ... ities=true", 0). Et voila!

You can, conversely, remove "[?]include_entities=true" from both the HTTP open request AND the parameter list/signature base string. Twitter never explained the function of "include_entities=true" and it's not part of how you form the encoded URL for the signature base string, and it made it seem like it was not a necessary part of the open request. But if it is in your parameters list, it should be in your open request! The few examples I found online of people struggling through this did not have the include_entities=true in their open request, so I just left it out. But here he are! Phew!

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: haomingchen1998, mikeyww, mmflume, scriptor2016, ShatterCoder and 90 guests