Define the autonomy window
Every site should have an explicit answer to one question: how long can it operate without the control plane? The answer determines what models, policies, credentials, tools, and data must already be available locally.
The autonomy window may be minutes for a connected retail assistant or days for an industrial workflow. It should be a tested requirement, not an optimistic assumption.
Choose a safe degraded mode
An outage does not always require a binary stop. Some workloads can continue with a cached knowledge set, a smaller model, limited tools, or read-only behavior. Others must fail closed because stale policy or unavailable oversight creates unacceptable risk.
- List the capabilities that remain safe without central coordination
- Disable tools that require fresh authorization or global state
- Make the degraded state visible to operators and, when relevant, users
- Bound local queues and storage so a long outage cannot exhaust the site
Keep health local
A central monitor cannot be the only system deciding whether a local workload is healthy. The site needs its own probes, restart policy, known-good version, and enough state to recover without waiting for an upstream command.
Health checks should verify the application path, not just the process. A model server can be running while responses are too slow, a dependency is stale, or output quality has fallen below the operating threshold.
Local availability requires local evidence of health and a local path to recovery.
Reconcile after connectivity returns
Recovery is a data problem as well as a network event. Decide which local events are replayed, which central changes win, and how conflicting updates are surfaced. Synchronization must be idempotent so retrying it does not duplicate actions.
Return gradually. Re-establish identity and policy, upload permitted telemetry, compare desired and actual workload state, and only then resume normal coordination. A connection coming back is not proof that the system has converged.