Who is on the other end, why trust it, what may it do — the multi-robot peer mesh

Five robots in one building, none of which knows the others exist

This is a scene we keep meeting on customer sites: a humanoid guide in the showroom, a quadruped on inspection duty, two delivery bases, and a greeting arm at the door. Every one of them is on the network. Every one of them takes natural-language instructions. And none of them has any relationship with the others.

So a lot of obvious things cannot be done:

• The inspection robot finds a leak on the third floor and has no way to send whichever unit is nearest to take a look;
• The guide robot is about to take a visitor to the second floor and does not know a delivery base is parked across the lift doors;
• You want to say "everyone to the lobby" once, and instead you say it five times.

None of this is a model-capability problem. There is simply no trustworthy link between them.

This post is about how we built that link, and why every apparently inconvenient part of the design is load-bearing. It goes into some detail, but assumes no knowledge of ROS or DDS.

The most obvious approach is also the most dangerous

The robots are already on the same LAN, and they all run ROS 2 — whose default transport (DDS) has a convenient property: put them on the same domain number (ROS_DOMAIN_ID) and they discover each other and interoperate automatically, with no code at all.

That sounds ideal. We tried it, and hit this:

A command typed on Orin5 was also executed by Orin6 — the same timestamp appears in both logs. The reason is mundane: the topic /remote_control/message carries commands, and by design every subscriber on a DDS domain receives every message. DDS has no notion of addressing a single machine and no authentication at all — any process on the domain can write anything to any topic.

In an office that is a joke. On a site with a robotic arm and a walking humanoid, it is a safety incident.

So the first decision is counter-intuitive: pin each robot's internal bus to its own machine. Every container loads the same FastDDS profile, confining traffic to 127.0.0.1; because the containers use host networking and share one loopback interface, local traffic works normally while cross-machine traffic is physically severed.

One detail is worth spelling out, because it is where the zero-coordination property comes from: every robot still uses domain 42, with nothing allocated. We tried per-robot domain numbers and it does not work — the usable range is narrow (roughly 0–101), and cloned images and multi-site deployments have no way to coordinate a unique number. Since internal traffic cannot leave the machine, everyone sharing one number is mutually invisible anyway, which means there is nothing to manage.

That decision sets the tone for everything after it: if robots are going to collaborate, the link has to be built explicitly — a link with verifiable identity and controllable permissions, not "we happen to be on the same subnet".

Identity: a robot is a key, not an IP address

The first question about any link is: how do you know who is on the other end?

IP addresses move (one of our robots changes its WiFi address several times a day). Hostnames collide (a batch of robots ships with every one called nvidia-desktop). So we use neither.

Each Agent Core generates an Ed25519 keypair on startup, and the fingerprint of the public key is that robot's identity (peer_id). That buys several things directly:

• Change subnet, change WiFi, change city — it is still the same robot;
• Discovering the same robot through different paths (LAN broadcast, a hand-entered address) collapses into one record rather than two lookalike entries, because the fingerprint matches;
• Every later request is signed with the private key, so impersonation requires stealing a key rather than guessing an address.

Discovery itself has several possible paths. Two are actually running: mDNS (automatic broadcast discovery on the same LAN, the primary path) and a manual address list (managed in the UI, for when you cross subnets or broadcast is filtered). A cross-site cloud roster and a fully offline Bluetooth bootstrap are designed but not yet coded.

Worth stating plainly: LAN broadcast (mDNS) does not cross routers — its multicast TTL is 1. We hit this between one robot and two test rigs: all three were on the office network but in different subnets, so none of them found the others. That is not a bug, it is the protocol's boundary, and it is exactly why the manual list exists.

Pairing: the same six digits on two screens

Knowing who is on the other end is not the same as trusting them. Trust has to be established by a human, once.

We use the same construction as Bluetooth pairing (SAS, short authentication string):

The important part is that the six digits are not a password one side generates and sends to the other. Each side computes them locally: sort the two public keys, sort the two nonces, then take an HMAC. Sorting the inputs guarantees both sides get the same answer; and because both public keys go into the computation, swapping either one makes the two screens disagree.

That leaves a man in the middle nowhere to go. Either he uses his own public key — in which case the two numbers differ and the human notices immediately — or he faithfully relays the real key, in which case he does not hold the matching private key and cannot sign any subsequent request. The whole job of those six digits is to bind "the public key I received over the network" to "the machine in front of me" — and only a human can do that.

Pairing is stored per direction. Which means when only one side confirms, that side looks completely healthy while the other has no record of you and rejects every request. We were bitten by this early: the UI said paired, reality was a wall of 403s. Now the default reads "waiting for the other side", until we have evidence the peer actually accepted us — our state push was acknowledged, or we received a request it signed (it only pushes to peers it has a record of) — at which point it flips to normal.

There is one more easily-confused state: the peer unpaired, versus the peer never confirmed. Both look like 403 on the wire, but the fixes are opposite — the first needs re-pairing, the second just needs someone on the other side to click approve. We track whether the pairing was ever mutual, and word the two cases differently.

Transport: why not mTLS

After pairing, every request is signed with the private key over method | path | timestamp | nonce | SHA-256(body). The receiver verifies against the public key pinned at pairing time.

There is a choice here worth explaining, because it departs from the textbook answer: the design called for mutual TLS, and we did not use it. The reason is specific — Agent Core serves one HTTPS port to both the browser dashboard and to other robots. Turning on client-certificate verification there would demand a certificate from every browser and lock the operator out.

So the shipped shape is: TLS provides encryption and carries no trust whatsoever (self-signed on both ends, verification off), and trust comes entirely from the signature and the pinned key. The properties of that combination are arguably cleaner — swapping a TLS certificate changes nothing, because it was never trusted.

Replay is bounded by two layers: a timestamp window plus a nonce cache. Why both is worth a sentence: robots running offline have no NTP and their clocks drift, so the window has to be wideable (clock_skew_s). What actually stops replay is the nonce cache — so widening the window costs a little performance and does not open a hole, because a replay inside the window still fails on the nonce.

Sharing capability: four granularities

With the link up, two robots can share four kinds of thing, coarse to fine:

Two implementation details deserve their own paragraph, because reality forced both.

One: why the tool proxy is two fixed generic tools. The first approach registered each peer's tools as entries in the local tool list (mcp__peer:<id>__<tool>). Elegant with two robots, broken with a fleet: the tool list grows linearly with the number of machines, and past a handful the tool descriptions alone cost tens of thousands of tokens. Worse, every time a peer joins or leaves, the tool list changes and the model's prompt prefix cache is invalidated wholesale. It is now two fixed tools — peer_tools(peer) to see what a peer has and peer_call(peer, tool, args) to invoke it — taking the robot id as an argument. Whether the fleet is 2 or 200, the tool list is those two.

Two: why state sharing moved from DDS to signed HTTPS. State used to be a DDS broadcast, and pinning DDS to loopback naturally cut that path. But the move also fixed a real defect on the way past: that DDS state bus had no authentication at all — any process on the domain could forge a robot's state. Over the signed link, state sharing has identity checking for the first time.

Permissions: three gates, and one sentence that has to be said plainly

This is the part that most needs to be precise.

What arrives from a peer takes one of two entirely different paths, and the guarantees differ.

The first is messages and task delegation. A peer's message enters the collector as input, not as a command; the local model sees it and decides for itself whether and how to act. Delegation runs in a subagent whose tool scope the receiver re-clips against that peer's role (the sender's list is a request, not a grant). On this path a peer only ever asks.

The second is calling a peer's tool directly (POST /api/peer/tools/call). That one goes straight to the device — no collector, no model, no conversation history. So its gates have to be mechanism:

The three gates, in order: role (viewer reads only, operator may act), tool filter (narrowing further within what the role allows), and canvas binding (the tool must already be wired to the local decision core). Any gate failing means refusal, and every acting call is announced on the activity stream — acting tools only, not read-only ones, or sensor polling would flood the audit trail.

How the role decision is made has its own scar worth recording. The first version guessed from keywords in tool names (anything containing move, arm and so on counted as an actuator). On real hardware it leaked immediately: one robot's actuators are called loco, led, speaker, switch_mode — no keyword list catches those, so a read-only viewer could drive locomotion. It now decides by declared layer and type: sensor/resource types and the whole perception layer are read-only; actuator, controller and the whole actucore execution layer act; and anything with no declared type counts as acting, because the conservative direction has to be refusal.

And then the sentence:

Granting operator means allowing that peer to drive this robot's actuators with no local model involved and no human confirmation.

That is the intended policy, not an oversight. We verified it on hardware deliberately: with the receiver's agent loop switched off entirely, the peer's call still executed — because that path never goes through the model. Which is why a newly paired peer defaults to viewer, and granting operator is a decision someone has to make explicitly.

We also found a claim that needed correcting. Config carries a require_actuator_confirm field, present in three sets of defaults, which looks like a human-confirmation gate — but no code reads it. Until it has a consumer it cannot be cited as a gate. A security mechanism that appears to exist is worse than none at all.

Delegation has one more guard: chains are capped at two hops (hop_count ≤ 2). Robot A delegates to B, B delegates to C, and it stops there — otherwise a task in a ring topology can replicate itself into a storm.

Liveness: the green dot should not lie

A small thing, but we revised it twice: "reachable" and "able to take work" are different facts.

A robot can be perfectly reachable, answering every signed request, while its agent loop is switched off — in which case nothing will process the message you send it. With a single green dot on screen, you would assume it was standing by.

So the states are reported separately: contact (a successful exchange within 30 s), advert (a broadcast seen within 90 s), and agent_running (is the peer's agent loop on). The UI shows all three rather than collapsing them into one optimistic dot.

Names are another small trap. Names change (users rename at will) and collide (a batch of robots share a default). So delegation and calls always use the fingerprint internally and names are for display only; an id suffix is appended only when names actually collide. A rename propagates on the 5-second state push — change it and other robots update within five seconds, with no re-pairing.

In closing

Looking back, there is almost no novel algorithm in the peer mesh. It is mostly three plain questions, each answer corrected by a real failure:

• Who is on the other end? — Not an IP, but the fingerprint of a key. Multi-path discovery of the same robot collapses to one record.
• Why trust it? — A human compares six digits derived from both public keys; after that every request is signed against a pinned key, and TLS encrypts without carrying trust.
• What may it do? — Read only, by default. Letting it act requires granting operator explicitly, and that is the same as allowing it to drive actuators without confirmation — a sentence that belongs in the documentation, not buried in the code.

It is all open source: in phanthymotus, agent-core/src/peer/ is the implementation and web/js/peers.js is the UI; phanthymotus-driver is the hardware driver layer.