Edge Agent

An Edge Agent connects a Docker host to Citadel without exposing an inbound Agent port. It opens a persistent outbound gRPC connection to Core, which sends Docker operations over that connection.

When to use Edge AgentCopy Anchor Link

Use an Edge Agent for a host behind NAT or a firewall that blocks inbound management traffic. If Core can reach the host directly, you can use the regular Agent, selected as Agent in the Platform form.

The Edge Agent host needs:

• outbound access to Core's Edge gRPC endpoint;
• Docker socket access from the Agent container;
• a persistent data volume for the Agent's key and enrollment identity.

Citadel permits only one Platform record for each Docker daemon. If the same Docker engine is already registered through Local or regular Agent access, Edge enrollment is rejected with the existing platform name. Remove the duplicate platform or keep the existing connection method; do not manage one Docker engine through two Platform records.

For production, run Citadel Core over HTTPS.

For reverse-proxy, direct Core TLS, and private-CA configuration, see TLS and secure Agent transport.

BuildsCopy Anchor Link

Edge Agent Platforms can run Citadel build projects over their existing connection to Core.

Citadel Core resolves the Git repository, packages the selected build context, and sends it through the Edge Agent stream. The Edge Agent then streams the context to Docker and pushes the configured image tags.

Core's current archive step is limited to 12 MiB, before the 16 MiB transport envelope. It does not apply .dockerignore; choose a narrow committed context with its Dockerfile inside it. See Build Context Size.

Because Edge Agents use an outbound command channel, cancelling a build closes the active build stream and the Docker Engine API build is cancelled when Docker observes the dropped request.

Build Pools can also use Edge Agent mode, but that flow is pool-scoped. Create a self-managed Build Pool, select Connection mode: Edge Agent, save the pool, then generate the enrollment command from the pool configuration. A build-pool Edge Agent does not require an Edge Agent platform resource.

Production gRPC endpointCopy Anchor Link

Edge Agents use bidirectional gRPC over HTTP/2. Set CITADEL_CORE_URL to the address supplied by the generated enrollment command. It must route gRPC traffic to Core's Edge listener. It can be the same URL used by the browser when the reverse proxy routes the two protocols to their respective listeners, as in the Caddy guide.

In production:

• use HTTPS for the public Edge Agent gRPC URL
• make sure the reverse proxy supports HTTP/2 gRPC upstreams
• do not downgrade the connection to HTTP/1.1
• allow long-lived streaming requests
• keep request and response buffering disabled for the gRPC route
• use idle timeouts longer than the agent keepalive interval

The public Edge Agent gRPC URL may differ from the browser URL because Citadel keeps separate API and Edge gRPC listeners. Configure CITADEL_EDGE_PUBLIC_GRPC_URL with the address that the Agent can reach.

If the proxy does not support gRPC over HTTP/2, the agent may connect and then fail with HTTP/2 protocol errors.

Create an Edge PlatformCopy Anchor Link

1. Open Platforms and select Add Platform.
2. Choose Docker Standalone or Docker Swarm for the host you are connecting.
3. Select Edge Agent under Connector and enter a clear name.
4. Select Save. Citadel creates the Platform and generates enrollment instructions.
5. Copy the Docker Command from the Enrollment section and run it on the target host.
6. Wait for the Platform to come online, then open it to check its resources.

The token is shown once and stored only as a hash. Copy the command before leaving the page. If you need new instructions, use Generate Enrollment Token in the saved Platform. An existing displayed token or enrolled identity changes the action label.

Run the AgentCopy Anchor Link

Run the command shown by Citadel on the Docker host you want to manage.

The command includes environment variables similar to:

CITADEL_AGENT_MODE=edge
CITADEL_EDGE_AGENT_PROFILE=edge-agent
CITADEL_CORE_URL=https://edge.citadel.example.com
CITADEL_EDGE_ENROLLMENT_TOKEN=...
CITADEL_EDGE_AGENT_KEY_PATH=/app/data/edge-agent.key
CITADEL_EDGE_IDENTITY_PATH=/app/data/edge-agent.identity.json

The container should mount:

/var/run/docker.sock:/var/run/docker.sock
/:/host:ro
citadel_edge_agent_data:/app/data

The generated Platform Edge Agent command also includes:

--label com.citadel.system=true
--label com.citadel.system-role=edge-agent

These labels make the Edge Agent a System container on the Platform it manages. Citadel keeps its diagnostics visible but blocks lifecycle and delete actions that could disconnect the Platform. Manage the container from its Docker host and include these labels in custom installation commands. Build-pool-only Edge Agents are not Platform system containers and do not receive these labels.

The Docker socket lets the agent inspect and manage local Docker resources. The read-only /host bind lets a Platform Edge Agent report the capacity of the filesystem containing Docker's configured data root. It exposes host paths to the trusted Agent process but does not allow writes through the mount. Removing it disables only disk metrics. Include this mount when configuring a Platform Edge Agent container. For ZFS, the metric represents the mounted dataset or filesystem, not total pool allocation. Build-pool-only Edge Agents do not receive this mount.

The data volume stores the agent key and enrolled identity.

Do not remove the persistent data volume unless you intend to re-enroll the agent.

Core listens for Edge gRPC on container port 8001. The optional Edge overlay maps it to 127.0.0.1:18001 on the host. This local-only binding cannot be reached by a remote Agent. Configure a reachable CITADEL_EDGE_PUBLIC_GRPC_URL and secure network access before enrolling one. Use the generated command rather than copying a hard-coded local address.

The browser's default host port is 18000; it is not the Edge gRPC endpoint.

EnrollmentCopy Anchor Link

On first start, the agent:

1. Generates an agent key pair.
2. Connects to Citadel Core.
3. Sends the enrollment token and public key.
4. Receives an agent identity from Citadel.
5. Stores the identity and private key in the configured data paths.

After enrollment, the enrollment token is no longer needed.

If the token expires before the agent connects, generate a new token from the platform page and restart the agent with the new token.

ReconnectsCopy Anchor Link

After enrollment, the agent reconnects with its saved identity.

Citadel verifies reconnects with a cryptographic challenge. The agent signs the challenge with its private key, and Citadel verifies it with the public key saved during enrollment.

Because of this, the data volume is important. If the agent loses its key or identity file, Citadel will not recognize it as the same enrolled agent.

Reconnects use exponential backoff with jitter. Short network interruptions should recover automatically without re-enrollment.

Platform statusCopy Anchor Link

An Edge Agent platform is online when the agent has an active connection to Citadel Core.

The agent sends regular heartbeats with:

• Docker reachability
• Docker version
• hostname
• agent version
• supported capabilities

If the agent stops, loses network access, or cannot reach Citadel Core, the platform becomes unavailable for operations that need the agent.

Citadel also validates the agent protocol version and advertised capabilities during enrollment and reconnect. If the protocol or required capabilities are incompatible with Core, the connection is rejected with a session rejection message.

Docker Swarm node coverageCopy Anchor Link

An ordinary Edge Agent configured for a Docker Swarm Platform represents the connected manager. It manages cluster-scoped Swarm resources and supplies the manager's node-local Docker data.

Do not copy that Platform enrollment token into a global Swarm Service or start one enrolled ordinary Edge Agent on every Node. Use Install node agents on the Platform's Cluster node coverage card. Citadel then creates a separate cluster-scoped bootstrap flow and a Citadel System global Docker Service for eligible Nodes not already covered by the manager connection. Its tasks run the existing Agent in a restricted outbound swarm-node profile and expose no inbound management port. The ordinary Edge Agent remains the manager/control-plane connector; satellite tasks are separate Node data sources under the same Swarm Platform.

Installing the System Service is an explicit privileged action because its tasks mount each Node's Docker socket read-write. With coverage installed, Citadel aggregates current Containers and routes worker Task inspect, logs, statistics, and Terminal to the owning Node. Covered Nodes also contribute Images, Volumes, and local Networks with their owning Node identified.

Change Task lifecycle through the owning Service or Stack; direct start, stop, and restart actions on individual Tasks are not available.

Docker operationsCopy Anchor Link

When you use an Edge Agent platform, Citadel routes supported Docker operations over the agent's active connection.

For example:

1. You open containers for the platform.
2. Citadel sends a containers command over the Edge Agent stream.
3. The agent runs the Docker operation locally.
4. The agent streams the result back to Citadel.

The Edge Agent does not give Citadel shell access to the host. It only handles the supported agent commands.

Supported operations include platform status, container list/logs/stats/inspect/create/start/stop/restart/delete/exec, image list/inspect/pull/build/push/delete, build-host checks, volume list/inspect/create/delete, network list/inspect/create/delete, stack apply, and deployment apply.

Rotate enrollmentCopy Anchor Link

Generate a new enrollment token when:

• the first token expired before use
• the token was copied to the wrong place
• the platform was created but the agent was never enrolled

Creating a new token does not rotate an already enrolled agent identity. For an enrolled agent, keep the persistent data volume.

Revoke accessCopy Anchor Link

Use revoke when the agent identity should no longer connect.

Revoking:

• marks the Edge Agent binding as revoked
• disconnects the active session
• prevents reconnect with the stored identity

After revocation, start over with a new enrollment if the host should be connected again.

TroubleshootingCopy Anchor Link

Platform stays offline

Check that the agent container is running and can reach CITADEL_CORE_URL.

Enrollment token is rejected

The token may be expired, already used, copied incorrectly, or revoked. Generate a new token and restart the agent with the new value.

Agent reconnects as a new install

The data volume may not be persisted. Keep the /app/data mount so the key and identity survive container restarts.

Docker operations fail

Check that the container has access to /var/run/docker.sock and that Docker is running on the host.

Core URL works in a browser but the agent cannot connect

The agent container must be able to reach the URL from inside Docker. Use the externally reachable Citadel URL, not a browser-only or machine-local address.

HTTP/2 server closed the connection

The Edge Agent gRPC endpoint is probably being served through an HTTP/1.1-only route or proxy. Point CITADEL_CORE_URL at the HTTP/2 gRPC endpoint and configure the proxy for gRPC pass-through or gRPC upstream forwarding.

Dashboard works but the Edge Platform stays disconnectedCopy Anchor Link

A working dashboard or /health response only verifies the browser/API route. An Agent can remain running and repeatedly log Edge connection ended; reconnecting or Core rejected the Edge connection when the separate gRPC route is blocked or points to the browser listener.

Check the full connection path:

1. Agent URL: use the HTTPS CITADEL_CORE_URL from Core's enrollment instructions. The Agent connects outbound; it does not need an inbound port or its own public DNS record. Core's browser hostname can also serve Edge traffic when the proxy routes the gRPC service separately.
2. CDN, load balancer or gateway, if present: verify that every intermediary supports bidirectional streaming gRPC and enable it for this hostname or route when required. Check the service's supported ports, TLS certificate validation, HTTP/2 negotiation and upstream protocol settings. Browser login redirects, interactive challenges, buffering and short stream timeouts can prevent an Agent connection even when ordinary web requests work. Use an endpoint and access policy compatible with authenticated Agent traffic.
3. Reverse proxy: enable HTTP/2 and route /citadel.edge.v1.EdgeAgentService/ to Core's Edge listener using gRPC upstream forwarding. With host Nginx and the standard Edge overlay, this is grpc_pass grpc://127.0.0.1:18001, forwarding to container port 8001. Use the actual Edge host port for your installation. The browser route's proxy_pass to port 18000 cannot replace it. Follow the complete host Nginx configuration or Caddy configuration.
4. Apply and verify: back up the affected configuration, validate it with your proxy's configuration checker, then apply or reload it. For host Nginx, run sudo nginx -t before sudo systemctl reload nginx. Verify the dashboard and gRPC route separately, and inspect the intermediary's logs to identify which hop rejects the request.

For example, Cloudflare requires Network → gRPC to be enabled for a proxied domain, with port 443, TLS and HTTP/2 at the origin. Use Full (strict) TLS with a valid origin certificate. Its disabled gRPC setting produces 403. Other providers expose different controls and error codes; consult your provider's documentation. See Cloudflare's gRPC requirements.

These responses help locate the failure, but do not by themselves establish its cause:

With an HTTP/2-capable curl, send an empty request to the public Edge route, replacing the example hostname with the hostname in CITADEL_CORE_URL:

curl --http2 --max-time 15 --silent --show-error \
  --request POST \
  --header 'Content-Type: application/grpc' \
  --header 'TE: trailers' \
  --data-binary '' --dump-header - --output /dev/null \
  https://citadel.example.com/citadel.edge.v1.EdgeAgentService/Connect

The expected response is HTTP/2 200, content-type: application/grpc, and grpc-status: 16 (Unauthenticated). This is an intentional rejection: the empty request contains no enrollment token or Agent identity. It verifies that traffic reaches the Edge authentication handler, not that an Agent has enrolled. An HTTP 200 with HTML is not a successful gRPC check.

Keep the existing Agent container and /app/data volume while repairing ingress. An Agent still retrying a connection should reconnect automatically after the route is fixed. Confirm that the Platform becomes Online and receives fresh observations. Do not delete its identity or generate a new token solely to fix a proxy failure; investigate enrollment separately if authentication still fails.

Security notesCopy Anchor Link

• Use HTTPS for Citadel Core in production.
• Treat enrollment tokens as short-lived secrets.
• Keep the agent data volume private.
• Revoke an agent if its host or data volume is compromised.
• Do not expose the Docker socket to containers you do not trust.