How-To: Integrate Provisioning Tools¶
Connect a manufacturing or provisioning station to the Keyserver so that it issues one device certificate per device, under one approved operation per batch. The single-device flow is in Tutorial Part 3: Issue a Device Certificate; this page turns it into a production loop.
How it works¶
sequenceDiagram
participant D as Device
participant S as Provisioning station
participant K as Keyserver
participant A as Approver
S->>K: POST /api/operations<br/>(maxusagecount = batch size)
K-->>S: 201, operation ID
A->>K: Approve (if the CA key<br/>requires approvers)
S->>K: GET /api/operations/{id}<br/>until approved
loop Each device in the batch
D->>D: Generate key pair and CSR
D-->>S: CSR (the private key<br/>stays on the device)
S->>K: POST /api/certificates<br/>(CSR, operation ID)
K-->>S: 200, certificate chain
S->>S: Verify the chain, store it,<br/>log the issuance
S-->>D: Device certificate chain
endDiagram: Device; Provisioning station; Keyserver; Approver; POST /api/operations (maxusagecount = batch size); 201, operation ID; Approve (if the CA key requires approvers); GET /api/operations/{id} until approved; Each device in the batch; Generate key pair and CSR; CSR (the private key stays on the device); POST /api/certificates (CSR, operation ID); 200, certificate chain; Verify the chain, store it, log the issuance; Device certificate chain
The station opens one operation for the batch and spends one of its uses per certificate. When the uses run out or the operation expires, the station opens a new one for the remaining devices. The Keyserver receives only CSRs, never a device private key.
Prerequisites¶
- A CA key in status
INITIALIZED, with Required Approvers set for production and the Issued Device Certificate Validity (days) set to the lifetime your devices need. See Create and Manage Keys in the Keyserver. - A role with the devcertsigner nature scoped to the CA key, attached to the station's subject. See the Nature reference.
- The device root certificate, exported once by an administrator as in Step 4: View or Export a Key's Certificate of the key management guide, and stored on the station as
root.pem. The station verifies every chain against this copy, not against the root returned with the chain. -
A POSIX shell (bash, zsh or dash), with
curl,jqandopenssl.mktemp,cmpandbasenameare preinstalled on Linux and macOS. Install the rest: -
The Keyserver URL:
Step 1: Choose How the Station Authenticates¶
| Station | Subject type | Flow | Setup |
|---|---|---|---|
| An operator is present and logs in at the start of the shift | User | Authorization Code with PKCE, through your identity provider | PKCE Client Integration Guide |
| Unattended, and can keep a client secret in a secret store | Client | Client Credentials | Integrate a Headless Build Server, Steps 1 to 5 |
Either way, the station ends up with an access token, exported as TOKEN. Never use the internal Authserver or a local password login in production. How a token maps to a subject is in Subjects.
Tokens expire during a long batch. When a request returns 401, get a new token for the same subject and continue under the same operation. A different subject, such as the next operator after a shift change, cannot use the operation while creatorowns is set; it opens its own. See Creator ownership.
Step 2: Open One Operation per Batch¶
Size the operation to the batch: maxusagecount is the number of devices, and validms covers the approval wait plus the run. Do not round either up. An approval is not bound to any CSR, so these two values are the limit on what a compromised station can issue. See the "What a batch approval authorizes" warning in Production differences from the tutorial. The description (16 to 4096 characters) is what approvers see, so name the line and the lot.
export CAKEY="device-ca"
export BATCH_SIZE=500
export VALIDMS=28800000 # 8 hours
OPERATION_ID=$(
jq -n --arg key "$CAKEY" --argjson n "$BATCH_SIZE" --argjson v "$VALIDMS" \
--arg d "Line 2, lot 4711: 500 sensor boards" \
'{validms: $v, description: $d,
"new-keyusagerestrictions": [{keyid: $key, maxusagecount: $n}]}' |
curl -sS --tlsv1.3 -X POST "$KEYSERVER/api/operations" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- | jq -r '.id // error("no operation created: \(tostring)")'
)
export OPERATION_ID
[ -n "$OPERATION_ID" ] && echo "Operation $OPERATION_ID"
Expected output:
If the CA key requires approvers, notify them and poll until approved is true, with the loop in Sign Code Using the REST API. Give the approver role only to people other than the station's subject; the Keyserver does not stop a subject from approving its own operation. See Approval Workflow.
Before you start the batch, check the operation:
curl -sS --tlsv1.3 -H "Authorization: Bearer $TOKEN" \
"$KEYSERVER/api/operations/$OPERATION_ID" |
jq '{approved, usage: [.keyusagerestrictions[] | {keyid, usagecount, maxusagecount}]}'
| Check | Must be | If it is not |
|---|---|---|
approved | true | Wait for the approvers. Requests return 409 "Need more approvers" until then |
keyid | The CA key | Create a new operation naming the CA key |
maxusagecount | The batch size | Cancel the operation and create a new one |
usagecount | 0 | Another run has used this operation. Create a new one |
Step 3: Collect and Check Each Device's CSR¶
Each device generates its key pair and CSR, ideally inside its secure element, and hands only the CSR to the station. Exceptions where a key must be injected are covered in Issue Device Certificates. For a dry run without devices, generate a test CSR as in Step 7: Generate a Device Key Pair and CSR of the device certificate tutorial.
Save each CSR as csr/<device-id>.csr. Check it before submitting:
Expected output:
OpenSSL before 3.2 puts spaces around =, and OpenSSL 1.1 prints verify OK.
The Keyserver checks neither the CSR's subject nor its self-signature, and issues a certificate for whatever the CSR carries. Reject a CSR whose subject is not the device you expect. Also reject one longer than 4096 characters, which the Keyserver refuses with 400.
Step 4: Issue the Certificates¶
The script below submits every CSR in csr/ under the operation, using the request from the tutorial's Step 10. It stops at the first refusal; a rerun skips devices that already have a chain.
#!/bin/sh
# Issues one device certificate per CSR in $CSR_DIR under $OPERATION_ID.
set -eu
: "${KEYSERVER:?Keyserver base URL}"
: "${TOKEN:?access token for the station subject (Step 1)}"
: "${OPERATION_ID:?operation for this batch (Step 2)}"
: "${CAKEY:?CA key ID}"
: "${ROOT_PEM:?stored device root certificate}"
CSR_DIR=${CSR_DIR:-csr}
OUT_DIR=${OUT_DIR:-chains}
mkdir -p "$OUT_DIR"
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
reason() { # the error message from a problem-detail body
jq -r '([.properties[]?.message] | join("; ")) as $m
| if $m == "" then (.detail // "no detail") else $m end' "$1" 2>/dev/null ||
head -c 300 "$1"
}
issued=0
for CSR in "$CSR_DIR"/*.csr; do
[ -e "$CSR" ] || { echo "No CSRs in $CSR_DIR"; exit 1; }
DEVICE=$(basename "$CSR" .csr)
[ -s "$OUT_DIR/$DEVICE.pem" ] && continue # issued in an earlier run
CODE=$(
jq -n --arg op "$OPERATION_ID" --arg key "$CAKEY" --arg csr "$(cat "$CSR")" \
'{operationid: $op, keyid: $key, fullchain: true, csrpem: $csr}' |
curl -sS --tlsv1.3 -o "$TMP/resp.json" -w '%{http_code}' \
-X POST "$KEYSERVER/api/certificates" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @-
) || CODE=000
case "$CODE" in
200) ;;
000|5??)
echo "$DEVICE: HTTP $CODE, outcome unknown; the certificate may have been issued"
exit 1 ;;
*)
echo "$DEVICE: refused with HTTP $CODE: $(reason "$TMP/resp.json")"
exit 1 ;;
esac
jq -r '.certificates' "$TMP/resp.json" > "$TMP/chain.pem"
openssl req -in "$CSR" -pubkey -noout > "$TMP/csr.pub"
openssl x509 -in "$TMP/chain.pem" -pubkey -noout > "$TMP/crt.pub"
if ! openssl verify -CAfile "$ROOT_PEM" -untrusted "$TMP/chain.pem" \
"$TMP/chain.pem" > /dev/null || ! cmp -s "$TMP/csr.pub" "$TMP/crt.pub"; then
mv "$TMP/chain.pem" "$OUT_DIR/$DEVICE.pem.rejected"
echo "$DEVICE: chain does not verify against $ROOT_PEM or does not match the CSR"
exit 1
fi
mv "$TMP/chain.pem" "$OUT_DIR/$DEVICE.pem"
printf '%s %s %s operation=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$DEVICE" \
"$(openssl x509 -in "$OUT_DIR/$DEVICE.pem" -noout -serial)" "$OPERATION_ID" |
tee -a "$OUT_DIR/issued.log"
issued=$((issued + 1))
done
echo "Issued $issued certificates"
Run it:
Expected output, one line per device:
<timestamp> Device12345 serial=<hex> operation=<operation-id>
<timestamp> Device12346 serial=<hex> operation=<operation-id>
Issued 2 certificates
The script:
- Stops at once if a variable it needs is unset.
- Reads only CSRs. No device private key passes through it.
- Builds each request body with
jq, which escapes the newlines in the PEM. - Writes the HTTP status to
CODEand the response body to a temporary file, so it can branch on the status and print the Keyserver's reason on a refusal. - Verifies the device certificate against the stored
root.pemand checks that its public key is the CSR's. A chain that fails either check is kept as<device-id>.pem.rejected, so that you can investigate it, and the run stops. A rerun submits that device again. - Appends each issuance to
chains/issued.logwith the serial number and the operation ID.
Step 5: Continue When the Operation Runs Out¶
A refused request does not issue a certificate. Fix the cause, then rerun the script; devices with a chain are skipped.
| Refusal | Meaning | Next |
|---|---|---|
409 "The key has already been used the allowed number of times in this operation" | The batch's uses are spent | Open a new operation for the remaining devices (Step 2) and rerun with its ID |
410 "Operation has expired" or "Operation is cancelled" | validms has elapsed, or the operation was cancelled (by DELETE or an approver's Revoke) | Open a new operation. It needs approval again if the CA key requires approvers |
401 | The token expired | Get a new token for the same subject and rerun under the same operation |
When the batch ends with uses left, cancel the operation so that nobody can spend them:
curl -sS --tlsv1.3 -o /dev/null -w '%{http_code}\n' -X DELETE \
-H "Authorization: Bearer $TOKEN" "$KEYSERVER/api/operations/$OPERATION_ID"
Expected output:
Retries and lost responses¶
POST /api/certificateshas no idempotency key, and an issued certificate cannot be fetched again. If the response is lost (status000or a 5xx in the script), the certificate may have been issued and counted. Resubmitting the CSR issues a second certificate for the same device.- Issued certificates cannot be recalled by cancelling the operation. See Cancelling. Record every certificate you know of for a device.
- From release 1.1.2, a failed issuance (a
400"Unable to create certificate for CSR" or a 5xx) still counts against the operation's uses; earlier releases do not count it. The CSR checks in Step 3 avoid most of them. - Leave
serialnumberout of the request unless your PKI assigns serial numbers; the Keyserver then picks a random one. If you set it, keeping serial numbers unique is up to the station.
What to log on the station¶
For each issuance the audit log records a createcertificate entry with the CA key, the operation ID, the subject, the time, and the request body including the CSR. See Audit Coverage Reference. It does not record the issued certificate, refused requests, or which physical device received the certificate.
The station's log fills those gaps:
- device ID, certificate serial number and operation ID for every issued certificate (
chains/issued.login the script) - the chain itself; the API cannot return an issued certificate again
- every refusal or unknown outcome, with the HTTP status and the Keyserver's reason (the script prints these to the terminal only; keep that output)
- never a token, a client secret, or device key material
To reconcile a batch, filter the audit log by Operation ID as in Step 3: Filter Audit Logs of the audit log guide and compare the createcertificate entries with chains/issued.log, matching each entry's CSR to csr/<device-id>.csr; the CSR is the only value both logs share. An audit entry with no matching log line is a certificate whose response was lost, or whose chain the station rejected (<device-id>.pem.rejected). From release 1.1.2, usagecount from GET /api/operations/{id} can be higher than both after a failed issuance.
If something fails¶
| Symptom | Cause |
|---|---|
401 "The user/client is not valid" | The token is valid but no subject matches it. Check the Subject ID (Troubleshoot API Errors) |
401 partway through a batch | The token expired. See Step 5 |
403 "Access forbidden" | The station's role lacks createcertificate on the CA key |
403 "Authentication by CI/CD authorization server is required for method 'createcertificate'" | The token comes from the administrative identity provider, not the CI/CD authorization server (Troubleshoot API Errors) |
403 "License violation event" | Device certificates are not enabled in this Keyserver's license. The Dashboard shows the licensed features |
409 "Need more approvers" | The operation is not approved yet (Step 2) |
409 "With creatorowns flag set in operation, only the operation creator can access the operation" | A different subject created the operation, for example the previous shift's operator (Step 1) |
409 "Only CA keys can be used to sign CSRs" | CAKEY names a FW or ROOT key |
410 "Signing key has been revoked" | The CA key was revoked. Stop the line; a new operation on that key is refused too |
400 | The CSR is malformed or longer than 4096 characters (Step 3) |
The chain does not verify against root.pem | The CA key is not under the root you stored, or root.pem is from another hierarchy. Do not ship the certificate |
Further statuses and response bodies are in Error Codes: Certificate issuance.