Status worker

Internal heartbeat monitor: services check in, silence opens a hep.gg outage incident, and the next heartbeat resolves it.

Status worker

The status worker is a small internal service that keeps the hep.gg status page in sync with reality. Services push heartbeats with their current status; the worker relays changes to hep.gg and turns prolonged silence into an outage incident automatically.

How it works

  1. Services heartbeat
    Each service POSTs /heartbeat/:componentId with its current status — typically every 10 seconds via lunar-js's startHeartbeat.
  2. Changes are pushed
    The worker only calls hep.gg when the status actually changed since the last successful push, so steady heartbeats are free.
  3. Silence opens an incident
    A check loop runs every 5 seconds. If a component hasn't checked in for over 30 seconds, the worker opens a critical outage incident on hep.gg and marks the component as outage.
  4. Recovery resolves it
    The next heartbeat resolves the incident. Resolving auto-resets the component to operational on hep.gg; if the service came back as anything else (say degraded), the worker pushes that status right after.

Restart-safe by design

On boot the worker syncs from hep.gg and seeds its in-memory table: known components get their published status as a baseline (so the first heartbeat doesn't push a no-op), and any already-open incidents are re-linked so a worker restart never double-opens one. Components seeded this way are exempt from the timeout check until they heartbeat at least once.

Sending heartbeats

The easiest integration is StatusWorkerClient from lunar-js. Raw HTTP works too — see Endpoints.