Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Client & Server Script for TCP/IP Network Communication


  • Please log in to reply
130 replies to this topic
qtmspin
  • Members
  • 24 posts
  • Last active: Oct 18 2013 09:08 PM
  • Joined: 17 Oct 2006
I've run into a limitation and annoying 'bug' It seems if I send messages too fast, ie in a loop like the one below... only the first two get to the client, the other 3 are lost in cyberspace...

If I put a pause of 200 ms in the server code all 5 are delivered to the client. I've tried changing the "ReceivedDataSize = 4096" to both higher and smaller numbers, but nothing has worked.

server code:
Loop, 5 {
	SendText=test message,%A_Index%
	GoSub SendViaNet
}

200 ms is no good, is there another way to send these messages and be sure they will arrive?

Thanks in advance. I've done so much using this script that never would have been possible without it.

Matt

Zed Gecko
  • Members
  • 149 posts
  • Last active:
  • Joined: 23 Sep 2006
Well i´ve had no time-critical tasks for this script right now,
so i´m wildly guessing here.

First i would suppose, that you set the scripts speed at maximum with
SetBatchLines -1

If this doesn´t help, you´ll have to find out if it´s the sending part or
the receiving part that looses the data.
You could connect the sender to a Telnet-Client (like Hyperterminal)
and see if the data is send out properly.

If it does send all the data properly,
it could help to get rid of OnMessage and
call the ReceiveData function in an endless loop.

If the sender already looses the data,
there is probably no easy solution.

BTW: I don´t think changing the ReceivedDataSize will be necessary,
i can receive large html-files with the script without problem.

qtmspin
  • Members
  • 24 posts
  • Last active: Oct 18 2013 09:08 PM
  • Joined: 17 Oct 2006
It's definatetly the receiving part (my client). I did like you said and hooked up telnet, wow that was easy and cool too! The server is sending everything, however the client is only getting the first two lines.

I already used, Setbatchline -1
I've also got the priority set to realtime
I tried using a loop like this:

SetTimer, ReceiveDataSub, 0, 1

ReceiveDataSub:
	Thread, Priority, 1
	receiveData(socket, 0)
Return
I still have to slow down the sending inorder to get the data.... back to the drawing board. Thank you for your suggestions.

PhiLho
  • Moderators
  • 6850 posts
  • Last active: Jan 02 2012 10:09 PM
  • Joined: 27 Dec 2005
Well, you use a scripting language, it is not as fast as a native C program, so you must live with its limitations (speed, memory use, etc.).
Posted Image vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2")

qtmspin
  • Members
  • 24 posts
  • Last active: Oct 18 2013 09:08 PM
  • Joined: 17 Oct 2006
I've noticed that when I hook telnet up to the server I get different data then what seems to be delivered to my client.

I tested it out w/ Zed's original script and I get the same thing. Here is what happens...

If I send the text 123456 from the server, 123456 arrives at the client (telnet terminal)

However if I send a shorter string next ie (1234)

123456 arrives at the client (telnet)

I am wondering if the 'pipe' is not being emptied, or perhaps this is a telnet buffer issue?

Zed Gecko
  • Members
  • 149 posts
  • Last active:
  • Joined: 23 Sep 2006
Well,
you have probably reached the limits of this language here.
As i can see from the code you have send me,
your ReceiveData function is quite long, and even includes
some MsgBoxes.
So i suppose the buffer of the socket gets filled up while your function
is running.
It would maybe help to make the ReceiveData function just add the received data to a var
and let another (timed) subroutine work with the
data from this var.
But this could produce a lot of new problems, and i´m not shure, if it
would really reduce the "round trip time".

There might be a way to increase the buffer of the socket with the
DLLCall of setsockopt with the value SO_RCVBUF:
http://msdn2.microso...y/ms740476.aspx
but i don´t know how to handle it.

The MSDN gives some general hints on speeding up network applications:
http://msdn2.microso...y/ms738559.aspx

But the safest way would be probably multithreading, which is not really
possible in AHK (we would need something like #MaxThreadsPerHotkey for OnMessage functions).

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006

I've run into a limitation and annoying 'bug' It seems if I send messages too fast, ie in a loop like the one below... only the first two get to the client, the other 3 are lost in cyberspace...

200 ms is no good, is there another way to send these messages and be sure they will arrive?


The TCP protocol assures delivery so yes, you can. It's been years since I've done network programming so I'm not much use for providing an answer, but I'm guessing an error status isn't being checked on one of the calls, and based on what info you've provided, it's probably related to the server's send. Perhaps the network buffer is filling up and the later send's are really failing?

Unlike UDP, the TCP protocol assures delivery or a failed status when transmission is unsuccessful. There's a mistake in the code somewhere.

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006
Ok, I'm probably showing my ignorance here but I'll ask some questions that might at least serve as a learning tool for those following along. I"m basing my comments on the sample server and client. I have no idea how qtmspin may have altered the programs for his work. And as I said above, it's literally been years since I did socket programming and then it was on unix, not Windows.

Asynch sockets are complicated. Doesn't the server need to handle on FD_WRITE event so that it knows when it's ok for it to write to the socket? By using WSAAsyncSelect you've ensured that the send command won't block when a buffer is full or anything else that would otherwise make it wait, so the FD_WRITE message handler should be used so that we know it's ok to send.

Again, I'm far from an expert but I think you need a few other message handlers setup and need some error handling after the send command. Right now no checking is being done to know whether the send was successful or not.

I did a quick search and found this little primer on basic asynch socket programming, and includes a better explanation of the FD_WRITE event handling. http://www.gamedev.n...article1297.asp

qtmspin
  • Members
  • 24 posts
  • Last active: Oct 18 2013 09:08 PM
  • Joined: 17 Oct 2006
I have two sub-routines work with the data now. If the subroutines are working with the data while more data is received, will the received data be ignored? Or are the surroutines treated seperatetly (different thread?) then the Received Data function?


ReceiveData(wParam, lParam)
; By means of OnMessage(), this function has been set up to be called automatically whenever new data
; arrives on the connection. 
{
   global ShowRecieved
   global ReceivedData
   
   Gui, Submit, NoHide	
   socket := wParam
    
    ReceivedDataSize = 4096 ; Large in case a lot of data gets buffered due to delay in processing previous data.
    Loop  ; This loop solves the issue of the notification message being discarded due to thread-already-running.
    {
        VarSetCapacity(ReceivedData, ReceivedDataSize, 0)  ; 0 for last param terminates string for use with recv().
        ReceivedDataLength := DllCall("Ws2_32\recv", "UInt", socket, "Str", ReceivedData, "Int", ReceivedDataSize, "Int", 0)
        if ReceivedDataLength = 0  ; The connection was gracefully closed,
            ExitApp  ; The OnExit routine will call WSACleanup() for us.
        if ReceivedDataLength = -1
        {
            WinsockError := DllCall("Ws2_32\WSAGetLastError")
            if WinsockError = 10035  ; WSAEWOULDBLOCK, which means "no more data to be read".
                return 1
            if WinsockError <> 10054 ; WSAECONNRESET, which happens when Network closes via system shutdown/logoff.
                ; Since it's an unexpected error, report it.  Also exit to avoid infinite loop.
                MsgBox % "recv() indicated Winsock error " . WinsockError
            ExitApp  ; The OnExit routine will call WSACleanup() for us.
        }
        ; Otherwise, process the data received.
        ; Msgbox %ReceivedData%
        Loop, parse, ReceivedData, `n, `r
        {
           	ReceivedData=%A_LoopField%
           	if (ReceivedData!="") 
           	{
				GoSub ParseData
				GoSub UseData
			}
        }	
	}
    return 1  ; Tell the program that no further processing of this message is needed.
}


Chris
  • Administrators
  • 10727 posts
  • Last active:
  • Joined: 02 Mar 2004
You're probably already aware of the Critical command. It will buffer incoming messages to prevent them from being dropped when a subroutine takes too long to finish. The remarks on the OnMessage page give some details about this.

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006
I did some testing on my own. I learned a little bit, but I'm not sure what. :)

I modified the server to send the text in a loop i number of times, and to do so with a delay of j ms, implemented with sleep, j. I modified the gui so that I could specify both i and j. I tested with client and server running on the same computer.

The send code is very simple and is shown below. I did not implement an FD_WRITE event, hoping to prove send() would fail if called in a loop too quickly. I thought for sure I'd receive a WSAEWOULDBLOCK error if the loop was too quick.

Loop % Repeat
   {
      sendret := DllCall("Ws2_32\send", "UInt", socket, "Str", SendData, "Int", SendDatasize, "Int", 0)
      WinsockError := DllCall("Ws2_32\WSAGetLastError")
      if WinsockError <> 0 
         MsgBox % "send() indicated Winsock error " . WinsockError
      sleep,Delay
   }


I added Critical to the first line of the client's ReceiveData routine so it couldn't be interrupted, and modified the receive loop to loop on recv() until there was no data left, and then process it. The code is as follows:

ReceiveData(wParam, lParam)
; By means of OnMessage(), this function has been set up to be called automatically whenever new data
; arrives on the connection. 
{
	  Critical
    global ShowReceived
    global MyEdit
    global LinesReceived
    Data=
    socket := wParam
    ReceivedDataSize = 4096  ; Large in case a lot of data gets buffered due to delay in processing previous data.
    Loop  ; This loop solves the issue of the notification message being discarded due to thread-already-running.
    {
        VarSetCapacity(ReceivedData, ReceivedDataSize, 0)  ; 0 for last param terminates string for use with recv().
        ReceivedDataLength := DllCall("Ws2_32\recv", "UInt", socket, "Str", ReceivedData, "Int", ReceivedDataSize, "Int", 0)
        if ReceivedDataLength = 0  ; The connection was gracefully closed,
            ExitApp  ; The OnExit routine will call WSACleanup() for us.
        if ReceivedDataLength = -1
        {
            WinsockError := DllCall("Ws2_32\WSAGetLastError")
            if ( WinsockError = 10035 ) { ; WSAEWOULDBLOCK, which means "no more data to be read".
                break
            }
            if WinsockError <> 10054 ; WSAECONNRESET, which happens when Network closes via system shutdown/logoff.
                ; Since it's an unexpected error, report it.  Also exit to avoid infinite loop.
                MsgBox % "recv() indicated Winsock error " . WinsockError
            ExitApp  ; The OnExit routine will call WSACleanup() for us.
        }
        Data .= ReceivedData
    }
    ; process the data received.
    Loop, parse, Data, `n, `r
    {
       LinesReceived++               
    	 if (LinesReceived = 1) {
           ShowReceived = %LinesReceived%: %A_LoopField%
        } else {
           ShowReceived = %ShowReceived%`n%LinesReceived%: %A_LoopField%
       }
       ;Tooltip % ShowReceived
       GuiControl,, MyEdit, %ShowReceived%
    }
    return 1  ; Tell the program that no further processing of this message is needed.
}


With a repeat of 5 and a delay of 1ms I still received all messages on the client. With a delay of 0 ms the client would usually receive 3 out of 5, though at times the text of 2 messages would be part of one receive on the client end. If I commented out the sleep command altogether, the client would receive just 1 message.

Neither send() or recv() ever return an error status. The fact that some data is being lost but no error status returned tells me there's a coding mistake somewhere, but I don't know where it is.

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006
I found another nice site for info on Winsock programming: http://tangentsoft.net/wskfaq/

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006
So I did some more testing, adding a counter to the messages I sent so that I could see which ones were being lost. I wish I could explain this.

Posted Image

I'd love to know what is going on.

qtmspin
  • Members
  • 24 posts
  • Last active: Oct 18 2013 09:08 PM
  • Joined: 17 Oct 2006
Great work, I love it! Have you tried hooking telnet up to the server, its easy and gives great feedback - although I am not sure if it's 100% accurate...

Im sure you know this already but...
start - run - telnet
in telnet type in open 192.168.0.x and the port of the server

wtg
  • Members
  • 251 posts
  • Last active: Dec 19 2012 03:54 PM
  • Joined: 04 Oct 2006

Great work, I love it! Have you tried hooking telnet up to the server, its easy and gives great feedback - although I am not sure if it's 100% accurate...

Im sure you know this already but...
start - run - telnet
in telnet type in open 192.168.0.x and the port of the server


Ah, no I hadn't. Forgot all about doing that even though I saw your discussion above.

Doing so shows that all the messages are being sent. Telnet receives them all easily, even with a delay of 0 and a large repeat count. So, it's something on the receiving side. I'm pretty new to AHK and I've not used the Critical command before so perhaps I'm doing something wrong.

As it is now, it seems there's a bug on the receiving side or there's a problem with AHK's message handling and messages are being missed. It's probably the code. =)