Install and upgrade an agent

Goal

Install the native TokenTimer agent on a host that holds certificates, so renewals, deployments, reloads, and verification run automatically under your control.

Which integration do you want?

This page covers the native agent, which does the certificate work for you. If you only want to report on renewals your own scripts already perform, use Connect an external executor instead.

What gets installed

The agent is a zero-dependency Node.js daemon. It is outbound-only: it opens HTTPS connections to your TokenTimer instance and never listens on a port, so it needs no inbound firewall rule.

Path	Purpose
/opt/tokentimer-agent/app	The agent package, read-only at runtime
/opt/tokentimer-agent/state	Config dir, mode 0700, owned by the agent user
.../state/config.json	Agent configuration, mode 0600
.../state/credential	Per-agent credential, written by the agent at registration
.../state/bootstrap.env	Bootstrap token, deleted automatically after first successful registration
.../state/registration-id.json	Client-generated registration id used to recover an interrupted registration; cleared once the credential is stored
.../state/outbox/	Durable queue of completed job results and evidence not yet acknowledged by the control plane; drained on restart before new work is claimed
.../state/job-journal/	Side-effect journal written before the first external mutation. An unresolved entry deliberately blocks automatic re-execution of that job, so a crash mid-issuance cannot silently repeat it
.../state/acme/	certbot and acme.sh config, work, and log directories
/etc/systemd/system/tokentimer-agent.service.d/override.conf	Generated ReadWritePaths allowlist
/etc/polkit-1/rules.d/50-tokentimer-agent.rules	Optional, only when you allow service reloads

The systemd unit is hardened (ProtectSystem=strict, NoNewPrivileges=true), and the agent runs as a dedicated tokentimer-agent system user rather than root.

Requirements

Requirement	Detail
OS	Linux with systemd. The installer generates a systemd unit and a polkit rule, so it does not support non-systemd Linux, macOS, or Windows.
Runtime	Node.js 22 or later (>=22.0.0 <25.0.0)
Privileges	sudo/root to run the installer (creates a system user, unit file, and directories)
Network	Outbound HTTPS from the host to your TokenTimer API URL. No inbound access needed.
Control plane	CertOps enabled, plus CERTOPS_SIGNING_ENCRYPTION_KEY and CERTOPS_REGISTRATION_ENCRYPTION_KEY set (see Configuration)

Non-systemd hosts

There is no packaged installer for macOS, Windows, or systemd-less Linux. The agent itself is portable Node.js and can be run manually (node /path/to/app/bin/tokentimer-agent.js with TOKENTIMER_AGENT_CONFIG_DIR set), but you are then responsible for the service supervision, sandboxing, and file permissions that the systemd unit normally provides.

Step 1 - Run the installer

Copy the agent package to the host, then run the installer from the package directory. It does not need a token yet: it pauses at a hidden interactive prompt asking for one, so the secret never reaches your shell history or the process list. Create the token in step 2 below and paste it there.

sudo ./scripts/install-agent.sh \
  --api-url https://tokentimer.example.com \
  --workspace-id <your workspace id>

The installer prompts for the bootstrap token when neither TOKENTIMER_AGENT_BOOTSTRAP_TOKEN nor --bootstrap-token is set. For unattended installs, prefer the environment variable instead of waiting on the prompt:

sudo TOKENTIMER_AGENT_BOOTSTRAP_TOKEN=ttboot_... ./scripts/install-agent.sh \
  --api-url https://tokentimer.example.com \
  --workspace-id <your workspace id>

Avoid --bootstrap-token on shared hosts

It works, but arguments are visible in process listings while the installer runs. The prompt and the environment variable both avoid that exposure.

Run it with --dry-run first to print every action without touching the system:

sudo ./scripts/install-agent.sh --api-url https://tokentimer.example.com \
  --workspace-id <id> --dry-run

Useful options

Option	Use it for
--write-path PATH	An absolute directory the agent may write certificates into. Repeatable. Never grants all of /etc.
--write-paths-file FILE	One absolute path per line, merged with --write-path
--reload-service NAME	Allow systemctl reload for nginx, apache/apache2, httpd, or haproxy via a generated polkit rule
--ca-bundle PATH	PEM bundle when your control plane uses a private CA
--allow-insecure-local-http	Development only, and only for loopback hosts
--uninstall	Stop and remove the service, app dir, unit, drop-in, and polkit rule

A typical nginx host:

sudo ./scripts/install-agent.sh \
  --api-url https://tokentimer.example.com \
  --workspace-id <id> \
  --write-path /etc/letsencrypt \
  --write-path /etc/nginx/certs \
  --reload-service nginx

ReadWritePaths is not the same as filesystem permissions

--write-path only lets the systemd sandbox reach a directory. The tokentimer-agent user must still be able to write there. Grant that before enabling deploy jobs, for example:

sudo setfacl -m u:tokentimer-agent:rwx /etc/nginx/certs

Without this, deploys fail with a permission error even though the path is allowlisted.

--write-path is not remembered across reinstalls

Every run of install-agent.sh regenerates the systemd sandbox drop-in (tokentimer-agent.service.d/override.conf) from scratch, from only the --write-path flags given on that invocation. This is deliberate (a stale broad grant should not silently outlive an operator narrowing it), but it means re-running the installer for any reason — troubleshooting, upgrading, changing --api-url — without repeating every --write-path from the original install will silently shrink the sandbox back down to the state directory only.

The failure mode this produces is misleading: the next real deploy job fails with EROFS: read-only file system opening a temp file under your certificate directory, which looks like a filesystem or permissions problem. It is not — it is the sandbox. Confirm the current grant with:

systemctl cat tokentimer-agent | grep ReadWritePaths

If your certificate directory is missing from that line, re-run the installer with the full original set of --write-path flags (and any --reload-service), then systemctl daemon-reload.

Polkit is used rather than sudoers deliberately: the unit keeps NoNewPrivileges=true, under which sudo cannot escalate at all.

Step 2 - Create a bootstrap token

1. Open Certificate operations at /certops/agents (the Agents tab).
2. Click Deploy an agent to open the install wizard and create a bootstrap token.
3. Copy it and paste it at the installer's prompt from step 1. It is a single-use ttboot_ token, shown exactly once, and it is consumed at the agent's first registration.

Step 3 - Verify registration

systemctl status tokentimer-agent
journalctl -u tokentimer-agent -f

You should see the agent register and begin heartbeating, and the Agent fleet panel on the Certificate operations page flip it to registered. The fleet panel shows status, last heartbeat, clock drift, and version compatibility.

Confirm the bootstrap token file is gone once registration succeeded:

sudo ls -l /opt/tokentimer-agent/state/

bootstrap.env should no longer be present; the agent removes it after the first successful registration and keeps only the per-agent credential. Removal is best-effort, so if the file is still there (for example because of a read-only or oddly-permissioned state dir), delete it yourself once the agent is registered.

Step 4 - Start in dry-run, then enable execution

The agent ships with execution.dryRun set to true and execution.enabled set to false, so a fresh install exercises the whole trust chain (claim, signature verification, policy evaluation, planning) and reports what it would do, with no filesystem or process side effects.

Leave it that way until you have seen a job plan correctly, then enable real execution in config.json:

{
  "execution": {
    "enabled": true,
    "dryRun": false
  }
}

Restart the agent after editing: sudo systemctl restart tokentimer-agent.

Upgrading

The agent does not self-update, which is deliberate: an execution-plane process that can update itself is an execution-plane process an attacker can update.

1. Copy the new package to the host.
2. Re-run install-agent.sh with the same flags. The installer stages the new tree, swaps it in atomically, and restarts the systemd unit, so the new code is what actually runs. The state directory, credential, and keys are preserved, so the agent keeps its identity and does not need a new bootstrap token.
3. Confirm the new version in the Agent fleet panel.

To force a fleet upgrade, raise CERTOPS_AGENT_MIN_AGENT_VERSION on the control plane; older agents are then blocked at registration and heartbeat with CERTOPS_AGENT_COMPATIBILITY_BLOCKED rather than silently continuing to claim work.

Retiring and uninstalling

Retire the agent from the Agent fleet panel first, then uninstall on the host. Retiring is not just bookkeeping: it invalidates the agent's credential (its protocol calls then return 410) and lets the agent exit cleanly on its next heartbeat instead of being killed mid-job.

Moving an agent to a different workspace is not supported

There is no in-place transfer of an agent between workspaces. To put an agent under a different workspace, retire it here first, then install a fresh agent and register it in the target workspace with a new bootstrap token.

Retiring an agent that is still working

Retirement deliberately refuses to yank work out from under a running agent. If the agent currently holds an actively leased job, a plain retire is blocked:

{
  "error": "CertOps agent has actively leased jobs",
  "code": "CERTOPS_AGENT_RETIRE_BLOCKED",
  "dependencies": { "leasedJobs": 2 }
}

The right first move is usually to wait: let the job finish, then retire. If you cannot wait (the host is already gone, or the agent is misbehaving), force it. A force retire requires an attributable reason and is rejected with 400 CERTOPS_AGENT_RETIRE_REASON_INVALID without one. This is enforced server-side, not just in the dashboard form, because a forced retire can abandon real work and someone will need to know why later.

A forced retire immediately fences the agent's in-flight work rather than leaving it to time out, and it splits that work by how far it got:

Job status when forced	Becomes	Why
claimed (dispatched, no execution started)	cancelled with error_code: CERTOPS_AGENT_FORCE_RETIRED	Nothing was done yet, so cancelling is safe and clean
running (execution had started)	orphaned_unknown_effect, flagged needs_operator_reconciliation	The agent may already have ordered a certificate, written a key, deployed a file, or reloaded a service. TokenTimer will not guess

The response reports both lists (fenced.cancelledJobIds, fenced.orphanedJobIds), and the CERTOPS_AGENT_RETIRED audit event records force, reason, leasedJobs, and both id lists.

Anything in orphanedJobIds needs a human. Work through Reconciling interrupted jobs for each one before assuming the certificate is in a known state. If an orphaned job was a renewal, it also raises a cert_renewal_failed alert, so it reaches you through normal alerting rather than sitting unnoticed.

Retiring is idempotent: retiring an already-retired agent returns 200 with the original retiredAt unchanged, and does not fence anything a second time.

On the host, a retired agent notices at its next heartbeat (which returns 410), logs control plane retired this agent; exiting cleanly, and exits with status 86. The systemd unit sets RestartPreventExitStatus=86, so systemd deliberately does not respawn it into a 410 loop. systemctl status tokentimer-agent then shows the service inactive with exit code 86: that is a successful retirement, not a crash. Compare with a genuine failure, which shows a non-86 status and Restart= kicking in with repeated activating (auto-restart).

Uninstalling

Once retired, on the host:

sudo ./scripts/install-agent.sh --uninstall

This removes the service, app directory, unit, drop-in, and polkit rule. The state directory and system user are preserved so you cannot destroy keys by accident. Remove them deliberately once you are sure:

sudo rm -rf /opt/tokentimer-agent
sudo userdel tokentimer-agent

warning

Uninstalling the agent does not retire it in TokenTimer, and retiring it does not uninstall it. Do both. An uninstalled but un-retired agent keeps a valid credential and simply looks offline; an un-uninstalled but retired agent keeps running locally and gets 410 on every call.

Also remember that certificates that agent discovered stay pinned to it: jobs for them will sit at pending until you assign a new agent or let a new agent rediscover them. See How a job is bound to one agent.

Troubleshooting

Symptom	Likely cause	Fix
Service in activating (auto-restart) with status=1/FAILURE	Startup failure, most often registration or config	Read the real reason with journalctl -u tokentimer-agent -n 50 --no-pager; the unit status alone never says why
Service inactive with exit code 86	Not a failure: the agent was retired in TokenTimer and exited cleanly	Expected. Uninstall the host files if you are done, or register a new agent
Installer refuses the API URL	URL is not https:	Use HTTPS, or --allow-insecure-local-http for a loopback dev host only
Registration fails as unauthorized	Bootstrap token already used, expired, or revoked	Tokens are single-use; create a new one in the dashboard
Registration fails and CertOps routes 404	CertOps disabled on the instance	Enable it on API and worker (see Enable CertOps)
Registration fails referencing an encryption key	CERTOPS_REGISTRATION_ENCRYPTION_KEY unset or malformed	Set 64 hex characters and restart the API
Agent blocked at registration or heartbeat	Version outside the accepted window	Upgrade the agent, or widen the version range on the control plane
TLS handshake failure to the control plane	Private CA not trusted by the agent	Reinstall with --ca-bundle, or set caBundlePath in config.json
Deploy fails with a permission error	Path allowlisted but not writable by the agent user	Grant write access (setfacl) to tokentimer-agent
Reload fails	Service not allowlisted at install time	Reinstall with --reload-service NAME
Clock drift warnings in the fleet panel	Host clock skew	Fix NTP; signed jobs enforce a validity window
Jobs plan but never change anything	Still in dry-run	Set execution.enabled: true and execution.dryRun: false, then restart
A job stays pending and this agent never claims it	The job is pinned to a different agent (the one that discovered the certificate), or requires a target selector this agent does not declare	Check the job's assigned agent; see How a job is bound to one agent
Renew job creation is refused for a certificate you can see in inventory	The certificate was only observed, so no agent has key custody	Install the agent on the host that actually serves it; see which certificates are renewable
Agent looks healthy but no automatic renewals ever happen	The CertOps maintenance worker is not running on the control plane, so no renewal jobs are created	This is a server-side problem, not an agent one; see Worker scheduling
NTP badge in the fleet panel stays unknown	timedatectl is missing or not usable on the host (containers, some WSL setups, non-systemd Linux)	Confirm timedatectl show -p NTPSynchronized --value runs as the tokentimer-agent user; unknown is reported as null, never guessed

Related

• Worked example: Cloudflare DNS-01 - a full tested walkthrough from install to a verified certificate on disk.
• DNS-01 providers - credentials and zone routing for DNS-01 challenges.
• Automation and executors - how jobs are planned, routed, and dispatched.
• Reconciling interrupted jobs - when an agent stops reporting midway.
• Configuration reference - control-plane variables that affect the fleet.