One TCP Read Is Not One HSMS Message: Framing on the Length Header

Some hosts are fine in development and start losing the odd reply once several tools are connected. The log shows a T3 timeout; tcpdump shows the reply arriving. That combination is almost always the parser. It is code that assumed one recv() is one message.

TCP is a byte stream. It does not preserve boundaries. The only thing that marks a message boundary in HSMS is the 4-byte length field SEMI E37 puts in front of every message, and counting it is the application's job, not TCP's. Below are four captures taken by opening a real socket against the simulator's passive listener: two messages in one write, one message split across two writes, a body that arrives short of what its length field claimed, and a length prefix claiming 1 MiB.

The fields first. The E37 length field is 4 bytes big-endian and counts the 10-byte header plus the message text that follows it. It does not count its own 4 bytes. In the header, bytes 0-1 are the SessionID, byte 2 is the W-bit (bit 7) plus the Stream, byte 3 is the Function, byte 4 is the PType (0 = SECS-II), byte 5 is the SType (0 data, 1 Select.req, 2 Select.rsp, 5 Linktest.req, 6 Linktest.rsp, 9 Separate.req), and bytes 6-9 are the SystemBytes. A control message has no body, so its length is always 10.

One write, two messages

I opened a socket in the host role and pushed Select.req and Linktest.req back to back in a single sendall().

TX (one write, 28 bytes)
00 00 00 0A 00 0B 00 00 00 01 00 00 11 01
00 00 00 0A 00 0B 00 00 00 05 00 00 11 02

The equipment side logged two RX entries: 06:04:10.654 SType 1 (Select.req) SystemBytes 0x1101, and 06:04:10.655 SType 5 (Linktest.req) SystemBytes 0x1102. Both carry a length field of 10. The kernel delivered one segment; the receiver counted two messages. Coalescing on the send side is correct, and splitting it back on the read side is correct.

The return direction depends on when you read. Call recv() the instant the write returns and the two replies come back separately.

recv#1 (14 bytes)  00 00 00 0A 00 0B 00 00 00 02 00 00 11 01   Select.rsp,   SystemBytes 0x1101
recv#2 (14 bytes)  00 00 00 0A 00 0B 00 00 00 06 00 00 11 02   Linktest.rsp, SystemBytes 0x1102

Send the same 28 bytes again, wait 300 ms, then call recv(4096) once, and both replies are in that one read.

recv#1 (28 bytes)
00 00 00 0A 00 0B 00 00 00 02 00 00 12 01   Select.rsp,   SystemBytes 0x1201
00 00 00 0A 00 0B 00 00 00 06 00 00 12 02   Linktest.rsp, SystemBytes 0x1202

Same peer, same bytes, same order. The only variable is when I read. Code that parses the leading 14 bytes and discards the rest loses Linktest.rsp permanently in the second run — and the symptom is "we drop a reply now and then", because on a quiet link it looks like the first run and never reproduces.

This was on loopback. Over a real network the split lands somewhere else again. That it moves is the point.

One message, two writes

I checked the other direction too, sending a 14-byte Select.req as 6 bytes and then 8.

TX chunk 1   00 00 00 0A 00 0B
RX           (nothing for 800 ms)

TX chunk 2   00 00 00 01 00 00 13 01
RX           00 00 00 0A 00 0B 00 00 00 02 00 00 13 01   Select.rsp, SystemBytes 0x1301

The first chunk carries the whole length prefix, so the receiver already knew a 10-byte message was coming. Nothing happened anyway. The equipment log shows the TCP connection at 06:04:12.458 and then a single RX at 06:04:13.259, the moment the remaining 8 bytes landed. Until all 14 bytes are present it is not a message.

When the body is short of its length — and nobody runs T8

Every capture above is a control message with no body. To get a body into it I went to SELECTED, opened communications with S1F13, then sent an S1F3 deliberately 4 bytes short.

TX  00 00 00 0C 00 0B 81 0D 00 00 00 00 14 02 01 00        S1F13 W, length 12, body L[0]
RX  00 00 00 37 00 0B 01 0E 00 00 00 00 14 02 ...          S1F14, MDLN "VX-9000 Plasma Etcher", SOFTREV "SECSGEM-1.4.1"

The full S1F3 frame is 22 bytes. The length field is 18 — 10 header plus 8 body.

full   00 00 00 12 00 0B 81 03 00 00 00 00 14 03 01 01 B1 04 00 00 00 01
TX     00 00 00 12 00 0B 81 03 00 00 00 00 14 03 01 01 B1 04    (18 of 22 bytes)
RX     (nothing for 7.0 s)
TX     00 00 00 01                                              (the last 4 bytes)
RX     00 00 00 31 00 0B 01 04 00 00 00 00 14 03 ...            S1F4

Those body bytes are SEMI E5 item headers: 01 01 is a list of one element, B1 04 is a U4 using one length byte and holding 4 bytes, and 00 00 00 01 is the value. I cut the frame in the middle of that U4 value.

Between 06:04:13.763 (the S1F13 RX) and 06:04:20.768 (the S1F3 RX) the equipment log has no RX at all. It held the half-message across 7 seconds, and when the last 4 bytes arrived it logged one RX with length 18 and answered S1F4. That S1F4 comes back as an L[4] although I asked for one SVID — the simulator ignores the SVID list and returns everything. Nothing to do with framing.

What matters is that it waited 7 seconds. E37's T8, the network intercharacter timeout, is the limit on the gap between bytes of one message being received; exceed it and the receiver is supposed to treat the message as failed and close the connection. The default is 5 s. I went past it and this listener did nothing. It does not implement T8 — the same result turned up again with a 20-second frame in the article on telling the HSMS timers apart.

That is one simulator's behaviour, not a claim about equipment in general. From the host side the conclusion is the same either way. T8 is a timer you run, not one the peer runs for you. Hold a half-arrived message in the buffer with no bound and a peer that dies quietly leaves that connection parked forever in "nearly complete".

When the length prefix claims 1 MiB

Last, I made the length field lie. Ten bytes of header behind it, and a length of 0x00100000 — 1,048,576 bytes.

TX  00 10 00 00 00 0B 00 00 00 01 00 00 15 01
RX  (nothing)
next write → BrokenPipeError [Errno 32]

No RX in the equipment log. Not one. The listener read the length, saw it above its ceiling and destroyed the socket, which is why my next write died with EPIPE. This listener's ceiling is 64 KiB.

A 4-byte length field goes to 4,294,967,295. In the encoding, 1 MiB is a perfectly legal value. Whether you allocate on it or cap it and hang up is not something E37 decides for you — the implementation does. Hang up. Once the stream is misaligned, those 4 bytes are not a length any more, they are somebody else's data, and a parser that sizes a buffer from them hands over the process on one corrupt byte.

What the parser has to do

Append whatever the socket returns to an accumulator. One read is not one message.
Once the buffer holds 4 bytes, read the length. Big-endian.
Only cut a message out when 4 + length bytes are present.
After cutting, loop again. A single read can hold any number of complete messages. It is a while, not an if.
If it is short, leave it and wait for the next read — with a T8 bound on that wait. Default 5 s.
Do not allocate on the length value as given. Set the cap from the largest SECS-II message that connection can produce, and when it is exceeded drop the connection rather than carry on parsing.

If you use a commercial stack instead of writing the socket layer, this loop already exists. It is still worth knowing, both to read a capture and see why there are two messages in it, and — mainly — to decide whether a T3 timeout means the link or the parser. If the reply is on the wire and T3 still fired, the link is not the problem.

All of the above was captured by opening sockets against the passive listener in the SECS/GEM simulator: five TCP connections, SystemBytes 0x11xx through 0x15xx in that order. If you want to go further into the item header bytes, SECS-II item headers and length bytes picks up there.