Your GEM Link Is Green and MES Still Can't Rebuild the Run

The tool ran fine. The event history has holes in it.

A wet bench ran twelve carriers overnight. In the morning MES had a process-start record for every lot and a process-complete record for about two thirds of them. No alarms, no disconnects — the HSMS session stayed selected the whole shift. The equipment engineer gave the standard answer: "the link was up all night, it's an MES problem."

It wasn't. The tool rebuilt its report definitions during a recovery sequence, and after the rebuild one report carried an RPTID that the host had never linked to the process-complete CEID. The equipment still sent S6F11 on every process complete. It just sent it with an empty report list. MES parsed the message fine, found no lot ID in it, and discarded it as unusable.

That is the shape of most collection-event problems I get called into. The transport works, the parser works, and the content is empty or wrong in a way no status screen shows you.

Three messages have to succeed, not one

Dynamic event reporting in SEMI E30 is a three-step handshake carried by SEMI E5 messages, and each step answers with a one-byte acknowledgement code that people routinely ignore. The lists are short enough to paste straight into a driver, so there is no excuse for logging "setup complete" instead:

Step	Request	Reply	Codes (SEMI E5)
Define reports	S2F33	S2F34, DRACK	0 accept · 1 denied, insufficient space · 2 denied, invalid format · 3 denied, at least one RPTID already defined · 4 denied, at least one VID does not exist
Link reports to events	S2F35	S2F36, LRACK	0 accept · 1 denied, insufficient space · 2 denied, invalid format · 3 denied, at least one CEID link already defined · 4 denied, at least one CEID does not exist · 5 denied, at least one RPTID does not exist
Enable events	S2F37	S2F38, ERACK	0 accept · 1 denied, at least one CEID does not exist

ERACK really is that short — accept, or "a CEID you named does not exist", and nothing else is defined. There is no "insufficient space" case on the enable step, so an enable that fails for any other reason has no code to tell you with.

The reply at the bottom is worth a line of its own. S6F12 carries ACKC6, and E5 gives it no error taxonomy at all: 0 means accepted, any non-zero value means the host did not take the report and has no field to say why. If your host answers non-zero, log the reason on your own side, because the equipment never learns it.

DRACK 4 is the one that bites during startup. The host asks for a VID the tool doesn't publish in that software version, and per the standard the whole S2F33 is rejected — not the one bad variable, the entire define. If your host driver logs "report setup complete" without checking DRACK, you get a tool that is defined-in-your-config and undefined-on-the-equipment, and you find out weeks later.

LRACK 3 (CEID link already defined) is the mirror image: the host tries to add a link on top of an existing one instead of clearing first. Standard startup sequence I use, in this order:

1. S2F37 with CEED = FALSE and an empty CEID list — disable all events.
2. S2F35 with an empty report list per CEID, or just rely on the next step.
3. S2F33 with DATAID set and an empty report list — this deletes every report definition and every event link on the equipment.
4. Re-send S2F33 with your actual report definitions. Check DRACK.
5. S2F35 to link. Check LRACK.
6. S2F37 with CEED = TRUE and the CEID list you care about. Check ERACK.

Step 3 is the one people skip because it feels destructive. Skip it and you spend a day chasing why a report has stale VIDs in it from the last integrator.

And a defined, linked report still sends nothing if step 6 never ran. "Report is configured but the event was never enabled" is the single most common cause of a CEID that exists everywhere except in the traffic capture.

The same sequence as bytes

Steps 1, 3, 4, 5 and 6 above, on a socket. Host side is a Python client; the equipment side is the SECS/GEM simulator HSMS passive listener on 127.0.0.1:5501, SessionID 11, no fault modes set. Every frame below was logged as RX by the equipment side, so these are wire bytes and not a rendering of a message object. The raw export is committed at content/demos/secs-gem-collection-event-report-notes.json.

Select first, so the session is SELECTED and data messages are legal:

TX  00 00 00 0A  00 0B 00 00 00 01  00 00 00 14     Select.req
RX  00 00 00 0A  00 0B 00 00 00 02  00 00 00 14     Select.rsp, byte 3 = 00 = accepted

Now step 1, disable every event — S2F37 with CEED = FALSE and a zero-length CEID list:

TX  00 00 00 11  00 0B 82 25 00 00 00 00 00 15  01 02  25 01 00  01 00

82 is header byte 2: W-bit 0x80 plus Stream 2. 25 is header byte 3, Function 37. 25 01 00 is the BOOLEAN item — format code 11 octal with one length byte, one byte of payload, value 0 = FALSE. Then 01 00, a list of zero items. E5 reads that empty list as every CEID, which is why this one 17-byte frame turns the whole tool quiet.

Step 3, delete every report definition and link — S2F33 with DATAID and a zero-length report list:

TX  00 00 00 14  00 0B 82 21 00 00 00 00 00 16  01 02  B1 04 00 00 00 01  01 00

Twenty bytes total, and it is the one people skip. Note how little separates it from a real define: the difference between wiping the tool's report configuration and adding to it is 01 00 versus 01 02.

Step 4, the define itself, with two reports:

TX  00 00 00 46  00 0B 82 21 00 00 00 00 00 17
    01 02
      B1 04 00 00 00 02              U4  DATAID = 2
      01 02                          L,2 two report definitions
        01 02
          B1 04 00 00 00 65          U4  RPTID = 101
          01 03
            B1 04 00 00 00 C9        U4  VID 201
            B1 04 00 00 00 CA        U4  VID 202
            B1 04 00 00 00 CB        U4  VID 203
        01 02
          B1 04 00 00 00 66          U4  RPTID = 102
          01 02
            B1 04 00 00 00 C9        U4  VID 201
            B1 04 00 00 00 CC        U4  VID 204

B1 is format code 54 octal (U4) with one length byte; B1 04 then means four payload bytes, one U4 value. For a list, E5's length field counts elements, not bytes — 01 03 is three items, not three bytes. Get that wrong and the frame is still accepted by TCP and still logged cleanly by the receiver, which is worth remembering the next time a clean RX log is offered as proof that a message was right.

The VIDs here are the host's request. Whether 201 exists on that tool is exactly what DRACK 4 answers, and it is answered per-message: name one VID the tool does not have and all six above are rejected together.

Steps 5 and 6, link and enable:

TX  00 00 00 3A  00 0B 82 23 00 00 00 00 00 18
    01 02  B1 04 00 00 00 03
      01 02
        01 02  B1 04 00 00 04 4D  01 01  B1 04 00 00 00 65
        01 02  B1 04 00 00 04 4E  01 02  B1 04 00 00 00 65  B1 04 00 00 00 66

TX  00 00 00 1D  00 0B 82 25 00 00 00 00 00 19
    01 02  25 01 01  01 02  B1 04 00 00 04 4D  B1 04 00 00 04 4E

00 00 04 4D is CEID 1101, 04 4E is 1102 — process start and process complete on this tool. The link message gives 1101 one report and 1102 two; the enable message names both, this time with 25 01 01, BOOLEAN TRUE. Compare it byte for byte against the disable frame at the top: same Stream, same Function, one payload byte different.

None of the five drew a reply. That is a property of this listener — it implements HSMS control messages only and has no SECS-II responder — so read it as a limit of the tool, not as equipment behaviour. What it does illustrate is the failure mode the whole article is about: a Linktest.req sent straight after five unanswered setup messages came back in 0.75 ms.

TX  00 00 00 0A  00 0B 00 00 00 05  00 00 00 1A     Linktest.req
RX  00 00 00 0A  00 0B 00 00 00 06  00 00 00 1A     Linktest.rsp, 0.75 ms

Green link, zero acknowledgements. A host that reports interface health from Linktest will show that as a healthy tool with no events configured on it, all shift. The clock that should be running instead is E37's T3 reply timeout, typically 45 s; my client waited 2 s per message and moved on, so those waits are the client's, not a T-timer's.

Timing is the defect you find last

Content bugs get caught in a week. Timing bugs survive into production, because the report parses cleanly and the values are all plausible.

Three that keep recurring:

• Process complete fires before the metrology latch updates. The report carries the previous wafer's measurement. Every value is in range, so nothing alarms. You only see it by comparing a captured S6F11 against the tool's own log for the same timestamp.
• Recipe name is sampled when the report is built, not when the event occurred. The operator queues the next lot's recipe during unload, and the completed run gets attributed to a recipe it never used. This one poisons SPC data quietly for months.
• Carrier ID is read from the current load port state rather than the state at the event. With two ports cycling, roughly one in twenty records is attributed to the wrong carrier.

The fix is on the equipment side — the values in an S6F11 must be latched at event time, not read at transmit time — but you have to prove it, and the proof is always the same: a controlled run with known lot, carrier, recipe, slot and module values, captured with timestamps, checked line by line against the tool screen.

Choose event points from what the host must decide

Start from the host's decisions, not from the tool's internal state bits. A collection event earns its place if it answers something MES, SPC, or a scheduler has to act on:

• When did the carrier become available to process?
• Which recipe was actually running when material moved?
• When did each process module start and stop work?
• Did the run complete normally, abort, or stop on alarm?
• Which chamber, slot, head, lane, or station produced this measurement?
• Which operator or remote command changed state?

Exposing every PLC step as its own CEID is the classic over-delivery, and it costs you twice: noisy host integration now, and a fragile mapping that breaks on every equipment software release later. Publish the stable process milestone; keep internal step transitions in equipment diagnostics.

On GEM300 tools most of this is already answered for you. E87 defines carrier state transitions, E90 substrate tracking, E40 process jobs, E94 control jobs, E157 module process tracking — with named state models and defined transition events. If the tool claims compliance, map to those events before inventing custom CEIDs. Custom events are the ones that get renumbered.

Mapping tables rot faster than the equipment

The host stores numbers. Engineers troubleshoot with names. So keep a CEID/RPTID/VID table outside the equipment software — for every entry: numeric ID, human name, SECS item format, engineering unit, and the owner who approves changes.

Format matters more than it looks. A VID that moves from U4 to A[16] between releases will silently break a host parser that assumed a fixed-width integer, and SECS-II carries no schema for you to validate against — the format byte on each item is all you get. If your host driver doesn't log a type mismatch, it will happily coerce or drop.

The failure here is almost never missing documentation. It's documentation that doesn't match the loaded configuration. Version the table against the equipment software release, or don't bother writing it.

S6F11 volume is a capacity decision

Collection events mark meaningful moments. They are not a data-collection channel, and the difference shows up as host-side queue growth long before anyone calls it a fault.

Patterns worth cutting:

• Heartbeat-style state events published every few seconds as CEIDs.
• Full recipe or parameter lists attached to every process step instead of once per lot.
• Repeated events during mechanical retry or state bounce, with no debounce on the equipment side.
• Static equipment identity values (model, serial, software version) re-sent in every report.

For high-rate values use trace data (S2F23) or the historian path, and keep S6F11 for milestones. If you genuinely need every event and the tool floods, the honest answer is usually a per-CEID enable list at startup rather than a bigger host queue.

Offline behaviour: spooling is a contract, not a buffer

GEM spooling is defined behaviour, not "the tool remembers things". The equipment spools the configured streams while the host is gone; on reconnect the host issues S6F23 with RSDC = 0 to transmit or RSDC = 1 to purge, and the equipment answers S6F24 with RSDA — 0 accepted, non-zero refused. Messages generated during the transmission are spooled too, so order is preserved end to end. Related reading: what spooling actually saves during a long outage.

What actually needs deciding at commissioning:

• Which streams are spooled at all (spooling everything is rarely right).
• Spool depth, and whether the tool discards oldest or stops spooling when full — GEM allows either, and the difference decides whether you lose the start or the end of an outage.
• Whether the host purges after an outage longer than some threshold. My default: purge anything older than one shift rather than replay it into current-state logic.
• How MES distinguishes a spooled event from a live one. If it can't, a reconnect dump gets processed as current state, and you get twelve lots "starting" at the same second.

Non-compliant tools that just dump their queue on reconnect without waiting for S6F23 exist in the field. Test for it. Pull the network cable during a safe step and watch what comes back.

Timeouts you should have set on purpose

SEMI E37 mandates no defaults here. Its timeout parameter table lists typical values — T3 = 45 s reply, T5 = 10 s connect separation, T6 = 5 s control transaction, T7 = 10 s not-selected, T8 = 5 s network intercharacter — and every one is settable per link. Vendors ship the typicals, so they behave like defaults in the field, but you cannot wave the standard at a tool that arrived with T3 = 30 s; it is conformant. Most projects never touch them, and mostly that's fine — but T3 at 45 s means a host that is merely slow (not dead) will have the equipment time out and start retry/spool behaviour before your MES finishes a database write. If your host commits synchronously inside the S6F12 path, either make that path asynchronous or agree a longer T3 with the equipment vendor, in writing, before hookup.

Version upgrades break assumptions that CEID numbers hide

Equipment software updates change event behaviour while leaving CEID numbers intact. Before accepting a release, diff:

• Added, removed, or renamed CEIDs.
• New VIDs inserted into existing reports — a host that indexes report values by position rather than RPTID structure will shift every field.
• Format changes: integer width, ASCII length, list nesting.
• Event timing changes (reporting before a latch instead of after).
• Changed status-code meanings.
• The default report-link configuration after software load, which frequently reverts.

Keep a small host-side regression test that parses real captured S6F11 samples for your production-critical events. Checking that the TCP session connects proves nothing.

The commissioning run, in order

For each production-critical sequence:

1. Clear all report definitions and links (S2F33 with an empty report list).
2. Define reports, link to CEIDs, enable events — checking DRACK, LRACK, ERACK at each step.
3. Run a controlled lot with known lot, carrier, recipe, slot and module values.
4. Capture all S6F11/S6F12 traffic with timestamps.
5. Compare each CEID against the physical sequence you watched happen.
6. Check every report value against the equipment screen, PLC state, or tool log — not against the interface manual.
7. Restart the equipment control software and re-check whether links survived. Then restart the host and check the same thing from the other side.
8. Disconnect the host mid-run at a safe step; verify spooling, the S6F23 exchange, and duplicate handling.
9. Store the captured message set as the baseline for that equipment software version.

That baseline is the deliverable people forget. When an MES discrepancy shows up eight months later, diffing today's traffic against a known-good capture settles it in twenty minutes. Arguing from the GEM manual takes days and usually ends with both sides right about different software versions.

What to hand over

The GEM manual is not a handover package. Operations needs the working configuration: the CEID/RPTID/VID mapping with names, units and formats; the exact host-side setup sequence; sample S6F11 captures for normal run, abort, alarm stop and reconnect; the spooling and duplicate-detection rules you agreed; and a named owner per production-critical event group.

One thing to check before you sign off: run the enable sequence twice in a row without an intervening delete, and confirm you get LRACK 3 rather than a silent success. A host driver that swallows that code will look fine on day one and drift out of sync on the first tool recovery.

Practising the sequence before you are on a tool

You cannot rehearse DRACK and LRACK against real equipment on a shipping day. Point your driver at the SECS/GEM simulator instead and read back what actually left your socket, the way the capture above did — header, item formats, list counts, the DATAID / RPTID / CEED positions. That is where most driver bugs are, and it is cheap to check.

Be clear about what it is not. The listener answers HSMS control messages and nothing above them, so no DRACK, LRACK or ERACK comes back and the acknowledgement half of this article cannot be rehearsed there at all. It also will not reproduce a tool that renumbers RPTIDs during recovery — nothing but the tool does that. Byte check, not conformance test.