Audit logging

The Eviden KMS server can write a cryptographically-chained, tamper-evident audit trail of every KMIP operation to a local JSONL file. Each line is a JSON object; the file is human-readable and can be parsed by any standard tooling.

Audit logging is disabled by default. No file is created and no background writer thread is spawned until the feature is explicitly enabled.

Enable audit logging

[audit]
enabled = true

[audit.file]
path = "/var/log/cosmian-kms/audit.jsonl"

cosmian_kms --audit-enable --audit-file-path /var/log/cosmian-kms/audit.jsonl

export KMS_AUDIT_ENABLE=true
export KMS_AUDIT_FILE_PATH=/var/log/cosmian-kms/audit.jsonl
cosmian_kms

When audit.file.path is omitted the file defaults to <root-data-path>/audit.jsonl.

Configuration reference

Tip: if you see AuditFileStore: channel full in the server log under sustained high load, raise --audit-channel-capacity. When the channel is full the event is dropped (non-blocking) and an error! line is emitted — the request itself is never blocked.

Client IP and reverse proxies

By default (--audit-trusted-proxy-cidrs empty), client_ip in every audit event is always the direct TCP peer address — the X-Forwarded-For header is ignored entirely.

If the KMS runs directly reachable by clients (no reverse proxy/load balancer in front), leave this unset: the TCP peer address is always the real client.

If the KMS runs behind a reverse proxy or load balancer, the direct TCP peer is always the proxy, not the real client. Set --audit-trusted-proxy-cidrs to the proxy's IP/CIDR so the audit middleware knows it can trust X-Forwarded-For coming from that address — otherwise every audit event will record the proxy's IP instead of the real client's.

Never trust X-Forwarded-For unconditionally: any direct caller can set that header to an arbitrary value, corrupting the forensic trail (e.g. framing another IP, or hiding its own). Restricting trust to known proxy CIDRs prevents this while still letting a legitimate reverse proxy forward the real client IP.

Audit failure mode

By default (continue) the KMS keeps serving even when an audit event cannot be queued — the event is dropped, an error! is logged, and the request succeeds normally.

Set --audit-failure-mode reject to enforce strict auditability: if an event cannot be placed in the writer channel (channel full or writer task dead), the KMS returns HTTP 503 to the client instead of the normal KMIP response. The KMIP operation has already executed at this point; the 503 signals that its outcome was not recorded.

Audit file size cap

--audit-file-max-size-bytes (or [audit.file] max_size_bytes in TOML) stops the writer from appending to the audit file once it reaches the configured size. This is a write-stop cap, not rotation or retention — the writer never deletes, truncates, or rolls the file on its own.

[audit.file]
max_size_bytes = 1073741824 # 1 GiB

Behavior:

• Omitted (the default): unlimited, today's behavior.
• The event that pushes the file to or past the cap is still persisted — only events after that one are dropped (subject to --audit-failure-mode, exactly like a full channel or a dead writer).
• Once capped, the condition does not clear itself: an external process truncating or rotating the file does not resume writing. The KMS must be restarted after the log is safely remediated. KMS-aware rotation/reopen is a possible future improvement.
• A 0 value is rejected at startup as a configuration error.

Event schema

Each line in the JSONL file is a complete JSON object with the following fields:

Example event:

{
  "id": 4,
  "timestamp": "2026-05-06T20:31:42.321328507Z",
  "operation": "Encrypt",
  "user": "admin",
  "object_uid": "417fe2de-827d-48d0-8d51-851bec315b76",
  "algorithm": "AES",
  "client_ip": "127.0.0.1",
  "result": "Success",
  "duration_ms": 1,
  "request_id": "c1f728c0-85f2-498c-8f47-9759d57a2745",
  "prev_hash": "e492c0f02860bc6c428259d44414651eda3aaaee2f48eb857144c940ac0fe909",
  "row_hash": "699a2837830af4a26fe79aeb48509fc707507e514da5850d953366a14e730c38"
}

Hash chain

Every persisted event includes a SHA-256 hash chain that makes tampering detectable offline.

The hash is computed over a canonical byte sequence of the event's fields: id || timestamp || operation || user || object_uid || algorithm || client_ip || result || duration_ms || request_id || prev_hash

prev_hash of the first event (id = 0) is the 32-byte all-zeros sentinel.

Any modification to a field in any row — including reordering rows, deleting rows, or appending forged rows — breaks at least one prev_hash → row_hash link and is detected by ckms audit verify.

Durability

Each write is followed by fsync() to ensure data is physically written to disk. Events survive an OS crash or power failure as long as the storage medium has confirmed the write.

On restart, the audit writer task always verifies the entire chain — every row's hash and its link to the previous row — before it appends any queued event, then reads the last 64 KiB of the file to decide how to resume. This runs in the background: the KMS starts and serves traffic immediately, without waiting for verification to finish, and any events submitted in the meantime are queued and written once recovery completes — see Startup recovery below.

Startup recovery

No condition found in the audit log — whether at the tail or anywhere in the middle of the file — ever prevents the KMS from starting. Recovery is routed by cause:

A torn write is the common case after an ungraceful restart (OOM kill, pod eviction, power loss) and is expected to happen periodically at fleet scale — it does not indicate tampering.

Concurrent instances (rolling updates)

The KMS takes a best-effort, non-blocking exclusive lock (<audit-file-path>.lock) before recovering or writing to the audit file, preventing two live instances — e.g. old and new pods overlapping during a rolling update on a shared volume — from corrupting the same log. If the lock is held by another instance, the KMS still starts and serves immediately; audit events are buffered (up to --audit-channel-capacity) and flushed in order once the lock becomes available.

Unwritable path (permissions, read-only mount, disk fault)

A path that cannot be opened for a reason unrelated to log content is treated as a deployment fault, not corruption. The KMS starts and serves traffic; audit events are buffered in the writer's channel (up to --audit-channel-capacity) while it retries opening the path — only once that buffer fills does an event get dropped with an error! log line. Audit logging resumes automatically once the fault is fixed, with no restart required.

Verify the chain offline

You can run the following command:

ckms audit verify --path /var/log/cosmian-kms/audit.jsonl

--path also accepts a directory, verifying every non-sealed *.jsonl file in it as its own independent chain. Sealed *.corrupt.jsonl evidence files from past recoveries are not independent chains; they are checked through the SHA-256 recorded in their live log's reanchor:

ckms audit verify --path /var/log/cosmian-kms/

Sample output: intact chain:

/var/log/cosmian-kms/audit.jsonl: chain OK: 42 events verified

Sample output: tampered file:

TAMPERED: /var/log/cosmian-kms/audit.jsonl event id=17 (line 18) has an invalid row_hash

For every audit:reanchor event encountered, verify also confirms the sealed evidence file it references still exists next to the log and its SHA-256 still matches the digest recorded in the event — this is what makes deleting or altering sealed evidence after the fact detectable:

MISSING EVIDENCE: /var/log/cosmian-kms/audit.jsonl: reanchor event id=0 references sealed file
audit.20260814T140233Z.9f3ac1b2.corrupt.jsonl which no longer exists

Exit codes: 0 = intact (and all sealed evidence present and unaltered), 1 = broken, tampered, or missing/altered sealed evidence.

With --verbose, a summary line is printed for every event:

id=0  2026-05-06T20:31:15Z  Create   chain=ok
id=1  2026-05-06T20:31:15Z  Encrypt  chain=ok
...

Best practices

• Use an append-only filesystem or object store (e.g. S3 with Object Lock) for the audit file.
• Restrict read access to the audit file to the KMS process user and auditors only; the file contains usernames and operation details.
• Retain audit files for the compliance window required by your framework (PCI-DSS Req. 10.7: 12 months; HIPAA §164.312(b): 6 years). This includes sealed *.corrupt.jsonl files left behind by a seal-and-roll recovery — they are forensic evidence and are never deleted automatically; clean them up as part of your retention/rotation process.
• Monitor the recovery audit events and server logs when a torn-write or seal-and-roll recovery happens — the KMS no longer refuses to start on audit-log corruption, so these are the primary operator signals for noticing and triaging it.
• For SIEM ingestion and CEF export, see SIEMs.