Not prefixing TCP socket messages with header can be a pain.

Not including header in TCP messages for transmission over network created a huge pain at a very latter point of time in one of our software. One of my friends also has spent almost a week to resolve the painful hidden issue caused by the same i.e. not prefixing TCP socket messages with header. That’s why I thought to share and make this a golden rule during socket programming i.e.

“Golden Rule: Always consider prefixing TCP messages with a header”.

Some time ago, I wrote a socket TCP client and server program for IPC (Internal process communication) in a project having multiple processes. One of the processes has TCP Server that was responsible to only send messages continuously on a particular interval e.g. 1 second or 5 seconds or 10 seconds. And in another process a TCP client was responsible to only receive messages that was in xml format of variable length.

Initially, we had not included any header to messages for transmission over TCP socket and was working seamlessly for many months. Server and Client were sending and receiving messages properly.

TCP Messages without header:

string msg(“<Message><TimeStamp>1434140729</TimeStamp><ChannelID>8</ChannelID></Message>”);

string msg(“<Message><Event>Client Connected</Event><EventType>2</EventType></Message>”); etc.

Some months later, new messages of big length got introduced besides other messages to transfer over network and started getting corrupted messages blocking the entire system.

Issue: no header with messages.

We found a hidden issue after burning tons of time and researching i.e.  “Every second server had been filling TCP internal buffer with new messages and every in a second client was receiving the messages. It was working fine if server and client were receiving each data in a seconds. But, as soon as client was making delay in receiving message, all the new messages was getting accumulated into the TCP buffer. So for the client, it was difficult to recognize one complete message as client doesn’t know the length of the messages to read beforehand, so, was getting corrupted messages”

Solution: Include a header with all TCP messages.

Solution to the issue was to prefix all TCP messages with a header. So, client can know beforehand the length of a complete message to read from the TCP buffer. After adding a header to messages it’s never been an issue since then.

We created a header with fixed length padded numbers header to prefix message. Now, client always read a 63 byte header first and read the length of the message to know the complete length of the message to read from the buffer.

Note: Creating header, depends upon you what scheme you follow. We used fixed length padded number scheme.

 

Here is the pseudo message with header:

Header : “<PacketHeader><PacketLength>msg length </PacketLength></PacketHeader>

msg(“<PacketHeader><PacketLength>10747 </PacketLength></PacketHeader><Message><Event>Client Connected</Event><EventType>2</EventType></Message>”);

 

CONCLUTION

It is always a best practice to include header with the TCP messages whether we are getting an issues or not. This is the robust way to handle socket messages.

 

JUST TO GET AN IDEA, HERE IS THE TCP SERVER AND CLIENT CODE SNIPS TO HANDLE MESSAGES WITH HEADER

 

—— Server side Code snips——-

Message header creation and sending it: fixed length padded number scheme. So, we can read fixed header from client application. We have created header of length 63 and prefixed with messages.

 

—— Client side Code snips——-

Read socket messages with header: