Skip to main content
Version: next

Enable CertOps

What the flag does​

CERTOPS_ENABLED is a platform-wide, fail-closed rollout flag for all CertOps routes: the workspace-scoped routes and the machine-token-authenticated executor routes. When the flag is unset or false, every CertOps route returns 404 NOT_FOUND regardless of role, and the CertOps UI surfaces stay hidden.

Self-hosted TokenTimer has no plan gating for CertOps. Once the flag is on, every workspace on the deployment can use CertOps (subject to normal RBAC). Cloud SaaS adds a separate plan check; that does not apply here.

This lets CertOps code ship dark and lets you enable the feature deliberately, per deployment, when you are ready.

Default behavior when unset​

The resolution order is env > DB > default, fail-closed:

  1. If the CERTOPS_ENABLED env var is set to a recognized boolean string, it wins.
  2. If it is absent, empty, or unrecognized, the code falls back to a system_settings.certops_settings.enabled database value.
  3. If that is also absent, the default is false.

Accepted truthy strings: 1, true, yes, on, enabled. Accepted falsy strings: 0, false, no, off, disabled. Anything else is treated as not set and falls through to the database value.

note

Unset means disabled. There is no way to accidentally default to enabled.

Where to set it​

Docker Compose​

The Compose file already passes the flag through to both the api service and the worker-endpoint-check service. Set it in your .env:

CERTOPS_ENABLED=true

Then recreate the affected services:

docker compose up -d

Kubernetes​

Set the env var on both the backend and worker workloads (for example via your ConfigMap or the Helm passthroughs api.env / worker.env):

CERTOPS_ENABLED: "true"

Kubernetes ConfigMap changes are not automatically re-read into a running container's environment, so after updating the config, restart the deployments:

kubectl rollout restart deployment/<backend-deployment>
# and the worker workloads, if they set the flag
warning

The worker needs the flag too, not just the API. The endpoint-check worker bridges endpoint SSL observations into the CertOps inventory and calls into the same gated service layer. If the worker's flag is out of sync with the API's, observations silently stop flowing into CertOps.

Rollout checklist​

  1. Confirm you are on a TokenTimer version that ships CertOps.
  2. Set CERTOPS_ENABLED=true on the API/backend process.
  3. Set CERTOPS_ENABLED=true on the endpoint-check worker process (same value, same time).
  4. Restart or redeploy both, since env vars are read into the process environment at container start.
  5. Run the verification steps below.
  6. Record the change in your ops log; the flag is your kill switch for the entire CertOps surface.

Verification steps​

  1. Route check (enabled): call any CertOps route with valid credentials. You should see a real response (success payload or a normal auth/validation error), not a blanket 404 NOT_FOUND.
  2. Dashboard check: log in and open a workspace. You should see CertOps surfaces appear: the CertOps enrichment on certificate tokens, the public PEM card in Import tokens, and the Certificate operations page reachable from the Control Center certificate-operations panel footer link and from Workspace Preferences (last section).
  3. Worker bridge check: with an HTTPS endpoint monitor configured, wait for the next endpoint check pass. You should see the observed certificate recorded in the managed-certificate inventory (visible in the token detail CertOps panel).
  4. Fail-closed check (optional but recommended): flip the flag back to false in a staging environment and restart. You should see every CertOps route return 404 NOT_FOUND again, with the UI surfaces hidden.

Rollback​

There is no migration or data cleanup involved in rolling back. Set CERTOPS_ENABLED to false (or remove the key, which falls back to the database value, then to false), restart the backend and worker, and all CertOps routes immediately return 404 again.

note

No jobs, machine tokens, or evidence records are deleted on rollback. They simply become inaccessible until the flag is re-enabled.

The system_settings.certops_settings database override exists for cases where you want a runtime kill switch without a redeploy, but the env var always takes priority when set to a recognized value. Do not rely on the database value while the env var is also set; the env var wins.

Fail-closed boot requirements​

Enabling CERTOPS_ENABLED turns on the routes, but two Ed25519/AES wrap keys must also be present in the API process environment before certain operations will succeed. Both are fail-closed: the operations that need them throw a specific ..._ENCRYPTION_KEY_MISSING error rather than silently degrading, and neither key is ever logged or returned in a response.

Env varFormatRequired for
CERTOPS_SIGNING_ENCRYPTION_KEY64 hex chars (32 bytes)Generating the control-plane Ed25519 signing key and signing dispatched jobs. Reading an existing public key does not need it.
CERTOPS_REGISTRATION_ENCRYPTION_KEY64 hex chars (32 bytes)Encrypting agent registration replay credentials at rest during agent bootstrap.

Generate either with:

openssl rand -hex 32

Set both alongside CERTOPS_ENABLED before you plan to onboard your first agent; a workspace that only uses external executors (no native agent, no signed dispatch) can run without them, but job signing and agent registration will fail with a clear _MISSING error code the moment they are needed. There is no separate rollout step beyond setting the two env vars once, since both are process-level configuration, not per-workspace settings.

Kill switch vs the platform flag​

CERTOPS_ENABLED and the workspace kill switch are two independent gates, not layers of the same control:

  • CERTOPS_ENABLED (this flag) is deployment-wide, env/DB-backed, and requires a process restart to change. It is the switch for whether CertOps exists on your instance at all.
  • Workspace kill switch (certops.kill_switch.manage permission, toggled from Certificate operations) is per-workspace, takes effect immediately with no restart, and pauses new job creation and dispatch for one workspace while leaving inventory, status, audit, and evidence reporting readable. Approving or rejecting an already-pending-approval job is deliberately still allowed while paused, since that is exactly the kind of decision an operator needs during an incident.

Use the platform flag to decide whether CertOps ships on your deployment at all; use the workspace kill switch as your day-to-day pause button when something looks wrong in one workspace.

Troubleshooting​

SymptomLikely causeFix
All CertOps routes return 404 NOT_FOUND for every workspaceCERTOPS_ENABLED unset or false platform-wideSet CERTOPS_ENABLED=true on the backend (and worker), restart both
Flag flipped in config but behavior has not changedProcesses have not restarted, so they still hold the old environmentRestart/redeploy the backend and worker; env is read at container start
Executor (machine token) requests return 404 but the dashboard UI worksMachine-token routes and workspace routes share the same flag, so if only executor calls fail the flag is not the problemCheck the executor route path (/api/v1/certops/...) and the token's scopes
Endpoint observations are not appearing in CertOps inventoryWorker's CERTOPS_ENABLED is out of sync with the backend'sSet the flag identically on both, restart the worker
Flag set but value seems ignoredValue not in the recognized truthy/falsy sets, so it falls through to DB/defaultUse true or false exactly (or another recognized string)

Next steps​