Skip to main content

Realtime events

The Gateway broadcasts an event for every record it changes, so a client such as the web app stays current without polling. HTTP stays the only way to act. An event only tells a client that something it fetched over HTTP has changed.

Transport

The Gateway’s HTTPS broadcasts to Reverb trust only ORBIT_HOME/ca/root.pem, with the system CA directory disabled. They use the resolved Node address while retaining the Reverb hostname for TLS verification. The old Node during a role move keeps its shorter client timeouts. Reverb broadcasts share one empty private CA directory per PHP process. The Gateway creates it when that process first configures broadcasting and removes it at normal process shutdown. Reconfiguring a connection does not create another directory. The Gateway sends events to Laravel Reverb at wss://reverb.orbit. Reverb runs on the Node that holds the websocket role. Realtime events with Reverb sets it up. Without an active websocket role, the Gateway broadcasts nothing. A client connects to Reverb directly, not to the Gateway. The Gateway only publishes, as a Pusher-protocol client. The Gateway broadcasts inside the request that changed the record. It uses no queue. A broadcast waits at most 2 seconds to connect and 5 seconds in total. A failed broadcast never fails the request: the Gateway logs the warning Failed to broadcast a record event. and continues. So an API response proves that the change happened, not that its event was sent. The Gateway looks up the connection at most once per request, and only when the request broadcasts or asks for the realtime connection. It connects to the websocket Node’s WireGuard address and verifies the name reverb.orbit against the Orbit root CA. During a websocket move, clients that connected before the move still listen on the old Node’s Reverb. The Gateway then sends each event to both servers. The old server gets 0.3 seconds to connect and 0.5 seconds in total. After a failed send, the Gateway skips the old server for 30 seconds.

Channel and authorization

Every Gateway event goes out on one private channel, orbit. A client signs its subscription through POST /api/v1/broadcasting/auth. The endpoint needs Gateway access: the caller is the Gateway Node or holds an access grant to it. It identifies the caller by WireGuard address, as the rest of the API does. The same endpoint signs a viewer membership on presence-node.{id}, as Node agent channels describes.

Discovery

A client asks the Gateway where Reverb is. GET /api/v1/realtime needs Gateway access and returns:
url and key are null when no websocket role is active. channel is always orbit. The key is public. The Reverb app secret never leaves the Gateway.

Envelope

Every event has the same shape. The broadcast name equals type.
A deleted event carries a short snapshot, because the record is already gone. Task, Activity, annotation, and Process usage events are notices: they carry only identifiers, status, and counts. A client fetches the record over HTTP when it needs more.

Event catalogue

Node

Project

Instance

Process

process.status carries the Process’s stored status and desired state, and the runtime_status that the Gateway read when the action ended. Later runtime changes come from the Node agent channels. Other changes to an Instance, such as a transfer, a clone, an environment change, or a deployment, send no instance.* event. A deployment sends deployment.*.

Process usage

The agent view subscriber sends process.usage while at least one viewer.* member is on a Node agent channel. Process usage describes when. id is the sample time in Unix seconds. processes lists [id, cpu, memory_bytes] for at most 200 Processes per part, with the units of the Process list. A Process without a sample has null values.

Schedule

Database connection

Firewall rule

A stored rule never changes, so there is no firewall.updated.

Route

A Route that Orbit creates or replaces as part of another action sends no route.created. That covers the Route of a new or cloned Instance and the replacement Routes of a transfer, a target change, or a domain change.

Deploy step

id is the step’s name. data matches the deploy step endpoints, so it has no Instance id.

Deployment

data matches one row of GET /api/v1/instances/{instance}/deployments. It never carries the run’s output. The Gateway keeps the 50 newest rows per Instance and removes older rows without an event.

Annotation

A client fetches the Instance’s annotation list for the current state.

Tasks

id is the task, comment, or thread id, and 0 for tasks.updated. task_group_id is the top-level task id. A thread of the top-level task has a null task_id. A subtask’s implementer and reviewer threads carry that subtask’s id in task_id. A thread row that is still reserved, with an external id that starts with pending:, sends no event. In a request or a scheduler tick, a change only to tokens, line counts, or duration sends no event, because showing a task refreshes those values. A publish run for a new workspace report always sends task_group.updated for the tasks of that workspace. The Gateway collects the task changes of one request, one scheduler tick, or one publish run, and sends each record once when that work ends.

Activity

A change request fires activity.created when its running row is stored and activity.updated when the outcome is stored. A read is stored once, when it ends, so it fires only activity.created. A read that stores no row fires nothing. What the Gateway records lists which requests store a row. The interrupted-Activity sweep and the shutdown finalizer fire activity.updated after they save the row. A row whose command starts with activity: fires nothing, so reading the log never feeds it. A write that changes only properties fires nothing. occurred_at is the row’s created_at: the time the request started, also for a read that is written when it ends. Null columns stay null. The Gateway never deletes an Activity row, so there is no activity.deleted. Removing a Node sets its id in caller_node_id and target_node_id to null.

When no event fires

An event fires only after its action succeeds. Two writes also fire after a failure: a deployment records its outcome, and a failed request stores its Activity. Failure details that an action writes before it throws, such as failed_step, fire nothing. The HTTP response reports that failure. A create call that finds the same record already stored, such as an identical Process, Schedule, or firewall rule, changes nothing and fires nothing. No event reports a change of a Node’s or an Instance’s runtime state. A Node’s status changes only through actions that already fire node.*. A Node’s updating value is the one exception, as the node.updated row says. Hibernation keeps an Instance’s awake state outside the Instance and Process records. Creating, replacing, or deleting a task definition fires nothing. The web app loads definition lists and the definition view when those pages open.

Node agent channels

Each Node has a presence channel, presence-node.{id}. Its Node agent publishes presence and Process state there. The Gateway broadcasts nothing on these channels. Its agent view subscriber joins every channel and keeps the latest state as the Gateway view.

Membership

user_info is { "kind": "agent", "node_id": 12, "version": "0.3.0" } for the agent, { "kind": "gateway" } for the subscriber, and { "kind": "viewer", "node_id": 7 } for a viewer. A viewer’s node_id is the Node it connects from. POST /api/v1/broadcasting/auth never signs an agent.* or gateway.* member. Reverb announces a member when its first connection joins (pusher:member_added) and when its last connection leaves (pusher:member_removed). A stopped or crashed agent closes its connection, so it leaves at once.

Events

The agent publishes Pusher client events. The Reverb app accepts client events from channel members (accept_client_events_from is members). Reverb adds the sender’s user_id to each one. A subscriber ignores an event on presence-node.{id} unless its user_id is agent.{id}. A complete snapshot replaces what a subscriber knows about the Node’s units. A Process whose unit is missing reads as inactive for systemd and exited for Docker. A complete client-workspaces list replaces the Node’s workspaces in the same way. A subscriber applies a unit to a Process only when the Process runs on Node {id}, uses the same runtime, and name equals orbit-process-{process id}-{process name}.

Presence

A subscriber treats the agent as online while agent.{id} is a member and an event arrived in the last 15 seconds. It treats the agent as lost when the member leaves or 15 seconds pass without an event. Before it sees the agent, a subscriber knows nothing about the Node from this channel and uses another source, such as Prometheus.

Log channels

Live logs use two more kinds of channel. No log line goes out on orbit or presence-node.{id}. The agent publishes client-log and client-log-end on presence-node-logs.{id}. The Gateway publishes log-streams.changed, with empty data, when a stream of the Node opens or closes. The agent then fetches its stream list. The Gateway publishes log.lines and log.ended on private-log-stream.{stream}, in the envelope above, with the stream ID as id. Live logs defines their data.

Why it works this way

These reasons explain the design. Check them before you propose a change.

Reverb as a role, not inside the Gateway

A WebSocket server holds long-lived connections. The Gateway process serves every API request, so it must not hold them too. Reverb is a purpose-built server that fans events out to every client. It runs as an unmodified Laravel app, so it carries no Orbit code. The websocket role installs it with one command. Running Reverb as an Instance behind a custom proxy Route takes six manual steps, and that Instance cannot share the Gateway’s Node, because gateway conflicts with app-prod.

Broadcast inside the request, without a queue

The Gateway runs every action inside its own request and has no queue worker. A queued event could also arrive after a second change to the same record and look newer than it is.

Events do not replace HTTP

A client acts only through the API, which checks identity, access grants, and validation. An event only invalidates what a client fetched. So a lost event costs a refetch, never a wrong action.

Notices instead of full records

Reverb refuses a message over 10,000 bytes. A task with its subtasks can pass that limit. Activity properties hold operation input, which should not reach every subscriber. So these events carry identifiers, status, and counts, and a client refetches the record. A change only to tokens or duration sends nothing, because every show would then send an event and make every tab show the task again.

One channel for all Gateway events

Every subscriber of orbit has Gateway access and can already read every record over HTTP. A channel per family or per Node would add authorization rules that protect nothing.

A presence channel per Node for agents

Whoever holds the Reverb app secret can sign any membership and publish any event, so the secret stays on the Gateway. The Gateway signs agent.{id} only for Node {id}, and Reverb stamps each client event with the sender’s member ID. So a Node can report false state only about itself. A client event on a private channel carries no sender, so any member could forge another Node’s state. A Reverb app per Node would force a browser to open one connection per Node and restart Reverb on every node:add. Agent messages never pass through the Gateway, so a heartbeat costs no Gateway request.

A heartbeat timeout detects lost agents

Reverb checks for dead connections on a fixed timer, and a vanished connection stays a member for 120 to 180 seconds. Changing that timer would make Reverb Orbit-specific. A 15-second heartbeat timeout detects a lost Node without a change to Reverb.