Skip to content

How-To: Integrate the Keyserver with GitLab CI

Sign firmware, binaries or release artifacts from a GitLab CI pipeline. Each job authenticates with a GitLab-issued token, and the pipeline creates a Keyserver operation, waits for approval, signs through the PKCS#11 library and verifies the signature.

How it works

sequenceDiagram
    participant G as GitLab CI job
    participant K as Keyserver

    Note over G: GitLab issues an ephemeral ID token<br/>(id_tokens, aud=keyserver)
    G->>K: POST /api/operations (Bearer KSC_ID_TOKEN)
    K->>K: Validate against GitLab JWKS,<br/>map identity claim to subject
    K-->>G: 201 operation id
    G->>K: Sign orders (PKCS11 module or REST)
    K-->>G: Signature

Diagram: GitLab CI job; Keyserver; GitLab issues an ephemeral ID token (id_tokens, aud=keyserver); POST /api/operations (Bearer KSC_ID_TOKEN); Validate against GitLab JWKS, map identity claim to subject; 201 operation id; Sign orders (PKCS11 module or REST); Signature

With federation (Option A), GitLab gives every job a fresh, short-lived ID token, so no secret is stored. The Keyserver validates the token against your GitLab instance's published signing keys (JWKS), treats every GitLab-issued token as a service account, and maps a configurable identity claim to a registered client subject. The subject's roles decide which keys the pipeline may use, and the audit log records every order under that subject.

Choosing an authentication option

Option A: GitLab JWT (recommended) Option B: Client Credentials (trial Authserver)
Secrets stored None; a new token per job Client ID and secret as CI variables
Identity granularity Per project (or per your instance's claim) Per registered client
Fit Production pipelines Testing against a trial environment

The pipeline in this guide uses Option A.

Prerequisites

  • A GitLab cloud or on-premise installation with CI runners configured
  • A Keyserver instance configured by Cryptera for GitLab JWT (Option A) or OAuth2 Client Credentials (Option B)
  • Your assigned Keyserver URL: *.<UNIQUE-ID>.cryptera-security.io
  • A signing key configured with the approval requirements you need
  • Runners that can pull golang:1.25 and ubuntu:24.04. The jobs install git, curl, jq, openssl and the OpenSSL PKCS#11 engine (libengine-pkcs11-openssl) themselves
  • The KSC_* environment variables and the OpenSSL configuration are documented in the PKCS#11 Library Reference

Step 1: Register the Project as a Keyserver Subject

How the identity mapping works

The Keyserver maps the GitLab ID token to a registered subject through an identity claim that Cryptera sets per identity provider when provisioning your instance. The default for GitLab is the project_id claim, a numeric ID that survives project renames and moves. With this default:

  • Register one Keyserver subject per GitLab project, with the numeric project ID as the Subject ID.
  • Wildcard subjects (gitlab/group/*) do not work with numeric IDs; register each project explicitly.

If your instance maps a different claim (for example project_path, which allows group wildcards), the Subject ID must match that claim's value. Cryptera can confirm which claim your instance uses.

First thing to check when a new pipeline fails

A token whose project_id matches no subject gets 401 "The user/client is not valid" on every call. See Error Codes Reference.

Configure the subject

In the Keyserver Admin UI:

  • Create a client subject whose Subject ID is the GitLab project ID, shown on the project page under Settings → General → Project ID.
  • Give it a role built from the fwsigner nature, scoped to the signing key (for example fw-quickstart). See Manage Roles in the Keyserver.

If using GitLab On-Prem

For a self-managed GitLab installation, send Cryptera the following during provisioning:

  • The issuer URL, for example https://gitlab.customer-a.com.
  • If the Keyserver backend cannot reach your instance, its JWK from https://gitlab.customer-a.com/-/jwks. After rotating your GitLab instance's signing key, send Cryptera the new JWK.

Step 2: Configure Pipeline Authentication

Every job that calls the Keyserver requests its own ID token with the audience keyserver:

id_tokens:
    KSC_ID_TOKEN:
        aud: keyserver

An operation created in one job stays usable in a later job, because both jobs' tokens map to the same project subject.

Option B: Client Credentials (trial Authserver)

Trial use only

The trial Authserver is for evaluation. Never use it for production firmware signing or certificate issuance.

Create the CI variables KS_CLIENT_ID and KS_CLIENT_SECRET (masked and protected), then request a token in the job. Here KSC_ID_TOKEN holds an access token; the variable name is historical:

export AUTH_CREDS=$(printf '%s' "$KS_CLIENT_ID:$KS_CLIENT_SECRET" | base64 | tr -d '\n')
export KSC_ID_TOKEN=$(curl -sS --tlsv1.3 \
  -X POST \
  -H "Authorization: Basic $AUTH_CREDS" \
  -d "grant_type=client_credentials" \
  "https://authserver.<UNIQUE-ID>.cryptera-security.io/oauth2/token" \
  | jq -r .access_token)

Step 3: Verify the Token Claims

Before adding the full pipeline, decode the ID token in a test job and check that GitLab issued what the Keyserver expects. The job prints the claims, never the token:

check_token:
  image: ubuntu:24.04
  id_tokens:
    KSC_ID_TOKEN:
      aud: keyserver
  before_script:
    - apt-get update && apt-get install -y jq
  script: |
    PAYLOAD=$(printf '%s' "$KSC_ID_TOKEN" | cut -d '.' -f 2 | tr '_-' '/+')
    while [ $((${#PAYLOAD} % 4)) -ne 0 ]; do PAYLOAD="$PAYLOAD="; done
    printf '%s' "$PAYLOAD" | base64 -d | jq '{iss, aud, project_id, project_path}'
Claim Must be If it is not
iss Your GitLab instance URL, as provisioned with Cryptera The Keyserver rejects tokens from unconfigured issuers (401)
aud keyserver Wrong or missing aud in the id_tokens block
project_id The value registered as the Subject ID in Step 1 Every call returns 401 "The user/client is not valid"
project_path Any Only relevant if your instance maps on project_path instead

Step 4: Add the Pipeline

The pipeline is one .gitlab-ci.yml file that calls two scripts. Add all three to your repository:

.gitlab-ci.yml
scripts/request_operation_await_approval.sh
scripts/sign_binary.sh

The pipeline runs three jobs:

  1. build_libpkcs11ks builds the PKCS#11 library from pinned sources. To download the prebuilt library instead, see Use the prebuilt library.
  2. request_operation_await_approval creates an operation and waits for approval, as described in Step 5: Create the Operation and Wait for Approval.
  3. sign_binary signs a test file through the library and verifies the signature, as described in Step 6: Sign and Verify. It writes the OpenSSL configuration and a copy of the library to build/.

Adapt the variables to your instance and key. The sign_binary job loads the library through the OpenSSL pkcs11 engine, which is removed in OpenSSL 4.x. To use a provider instead, read the warning in Provider interface before switching.

.gitlab-ci.yml
stages:
  - build
  - await_approval
  - sign

variables:
  # Keyserver URL
  KSC_API_SERVER: "https://keyserver.<UNIQUE-ID>.cryptera-security.io"
  # Must match the key's Hash Algorithm
  HASH_ALGORITHM: "-sha256"
  # Signing key ID in the Keyserver
  KEYID: "fw-quickstart"
  # Shown to approvers and recorded in the audit log
  KSC_OPERATION_DESCRIPTION: "GitLab PKCS11 signing"
  # Pin the tags; do not track default branches in a signing pipeline
  LIBPKCS11KS_REPO: "https://github.com/cryptera-device-security/libpkcs11ks.git"
  LIBPKCS11KS_TAG: "v1.0.4"
  PKCS11MOD_REPO: "https://github.com/cryptera-device-security/pkcs11mod.git"
  PKCS11MOD_TAG: "v1.0.1"
  # Approval wait in seconds
  WAIT_FOR_APPROVAL: 120
  # libpkcs11ks HTTP timeout for calls to the Keyserver (default 10s)
  KSC_HTTP_CLIENT_TIMEOUT: "15s"

build_libpkcs11ks:
  stage: build
  # libpkcs11ks v1.0.4 requires Go 1.25 or later
  image: golang:1.25
  variables:
    CGO_ENABLED: 1
    GOOS: linux
    GOARCH: amd64
  before_script:
    - apt-get update && apt-get install -y git
    - cd $CI_PROJECT_DIR
    # Clone as siblings: libpkcs11ks's go.mod replaces pkcs11mod with ../pkcs11mod
    - git clone --depth 1 --branch "$PKCS11MOD_TAG" "$PKCS11MOD_REPO"
    - git clone --depth 1 --branch "$LIBPKCS11KS_TAG" "$LIBPKCS11KS_REPO"
    # pkcs11mod generates part of its cgo shim
    - cd pkcs11mod && go generate ./...
  script:
    - cd $CI_PROJECT_DIR/libpkcs11ks
    - chmod +x build.sh
    - ./build.sh
  artifacts:
    paths:
      - libpkcs11ks/libpkcs11ks.so
    expire_in: 1 week

request_operation_await_approval:
  stage: await_approval
  image: ubuntu:24.04
  id_tokens:
    KSC_ID_TOKEN:
      aud: "keyserver"
  before_script:
    - export DEBIAN_FRONTEND=noninteractive
    - apt-get update
    - apt-get install -y jq curl
  script:
    - bash scripts/request_operation_await_approval.sh
  artifacts:
    reports:
      dotenv: approval.env
    expire_in: 1 hour

sign_binary:
  stage: sign
  image: ubuntu:24.04
  needs:
    - job: build_libpkcs11ks
    - job: request_operation_await_approval
  id_tokens:
    KSC_ID_TOKEN:
      aud: "keyserver"
  before_script:
    - apt-get update
    - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends
      ca-certificates curl jq openssl libengine-pkcs11-openssl
  script:
    - bash scripts/sign_binary.sh
  artifacts:
    name: "signed-output-$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA"
    when: always
    expire_in: 14 days
    paths:
      - build/openssl-ks.cnf
      - build/libpkcs11ks.so
      - test.txt
      - test.txt.sig
      - certificate.pem
      - pubkey.pem

Step 5: Create the Operation and Wait for Approval

scripts/request_operation_await_approval.sh
#!/usr/bin/env bash
# Creates a Keyserver operation for $KEYID, waits for approval, and hands the
# operation ID to later jobs through the approval.env dotenv report.
set -euo pipefail

: "${KSC_API_SERVER:?set in .gitlab-ci.yml variables}"
: "${KSC_ID_TOKEN:?requested by the id_tokens block of this job}"
: "${KEYID:?set in .gitlab-ci.yml variables}"
: "${KSC_OPERATION_DESCRIPTION:?set in .gitlab-ci.yml variables}"
: "${WAIT_FOR_APPROVAL:?set in .gitlab-ci.yml variables}"

# The operation must outlive the approval wait, the queue time before the sign
# job starts, and the signing itself. Keep a margin of several minutes.
VALIDMS=$(( (WAIT_FOR_APPROVAL + 480) * 1000 ))

echo "== Create operation for $KEYID (valid ${VALIDMS} ms) =="
BODY=$(jq -n --arg d "$KSC_OPERATION_DESCRIPTION" --arg k "$KEYID" --argjson v "$VALIDMS" \
  '{validms: $v, description: $d, "new-keyusagerestrictions": [{keyid: $k, maxusagecount: 1}]}')

RESPONSE=$(curl -sS --tlsv1.3 -X POST "$KSC_API_SERVER/api/operations" \
  -H "Authorization: Bearer $KSC_ID_TOKEN" \
  -H 'Content-Type: application/json' \
  -d "$BODY") || { echo "Create-operation request failed"; exit 1; }

KSC_OPERATION_ID=$(jq -r '.id // empty' <<<"$RESPONSE" 2>/dev/null || true)
if [ -z "$KSC_OPERATION_ID" ]; then
  echo "Failed to create the operation. Keyserver response:"
  printf '%s\n' "$RESPONSE"
  exit 1
fi
echo "Operation ID $KSC_OPERATION_ID"

# Notify approvers here (Slack, email): the Keyserver sends no notifications.
# Keys with no required approvers are approved immediately.
echo "== Wait for approval (max ${WAIT_FOR_APPROVAL}s) =="
STATUS="false"
DEADLINE=$(( SECONDS + WAIT_FOR_APPROVAL ))
while [ "$SECONDS" -lt "$DEADLINE" ]; do
  # A network error or a 5xx counts as "not yet". Any other refusal, such as
  # 410 "Operation is cancelled", ends the wait at once.
  RESP=$(curl -sS --tlsv1.3 -w '\n%{http_code}' "$KSC_API_SERVER/api/operations/$KSC_OPERATION_ID" \
           -H "Authorization: Bearer $KSC_ID_TOKEN") || RESP=$'\n000'
  CODE=${RESP##*$'\n'}
  case "$CODE" in
    200) STATUS=$(jq -r '.approved' <<<"${RESP%$'\n'*}") ;;
    000|5??) STATUS="error" ;;
    *) echo "Keyserver refused the poll (HTTP $CODE):"; printf '%s\n' "${RESP%$'\n'*}"; exit 1 ;;
  esac
  [ "$STATUS" = "true" ] && break
  echo "Not approved yet ($STATUS)"
  sleep 5
done
[ "$STATUS" = "true" ] || { echo "Approval timed out"; exit 1; }
echo "Operation $KSC_OPERATION_ID is approved"

echo "KSC_OPERATION_ID=$KSC_OPERATION_ID" > "$CI_PROJECT_DIR/approval.env"

The script:

  • Stops at once if a variable it needs is unset, and names where the value comes from.
  • Sets validms to the approval wait plus 8 minutes, so the operation outlives the wait, the queue time before sign_binary starts, and the signing. At the default WAIT_FOR_APPROVAL of 120 seconds this is 600000 ms.
  • Builds the request body with jq, so a quote in the description cannot break the JSON.
  • Prints the Keyserver's full response if the operation cannot be created.
  • Polls every 5 seconds until the operation is approved or WAIT_FOR_APPROVAL runs out. Approvers approve it on the Keyserver's Approvals page. A network error or a 5xx counts as "not approved yet"; any other error stops the job and prints the Keyserver response.
  • Writes KSC_OPERATION_ID to approval.env. GitLab passes it to sign_binary as an environment variable (a dotenv report).

Expected job log (the number of Not approved yet lines depends on how long approval takes):

== Create operation for fw-quickstart (valid 600000 ms) ==
Operation ID <operation-id>
== Wait for approval (max 120s) ==
Not approved yet (false)
Operation <operation-id> is approved

Step 6: Sign and Verify

scripts/sign_binary.sh
#!/usr/bin/env bash
# Signs test.txt through libpkcs11ks under the approved operation, then verifies it.
set -euo pipefail

: "${KSC_API_SERVER:?set in .gitlab-ci.yml variables}"
: "${KSC_ID_TOKEN:?requested by the id_tokens block of this job}"
: "${KSC_OPERATION_ID:?missing - passed from request_operation_await_approval via approval.env}"
: "${KEYID:?set in .gitlab-ci.yml variables}"
: "${HASH_ALGORITHM:?set in .gitlab-ci.yml variables}"

# Generated files go to build/.
OUT="$CI_PROJECT_DIR/build"
mkdir -p "$OUT"
cp -f "$CI_PROJECT_DIR/libpkcs11ks/libpkcs11ks.so" "$OUT/libpkcs11ks.so"

echo "== Generate OpenSSL PKCS#11 engine config =="
cat >"$OUT/openssl-ks.cnf" <<CNF
openssl_conf = openssl_init

[openssl_init]
engines = engine_section
providers = provider_sect

[provider_sect]
default = default_sect
base = base_sect

[default_sect]
activate = 1

[base_sect]
activate = 1

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib/x86_64-linux-gnu/engines-3/libpkcs11.so
MODULE_PATH = $OUT/libpkcs11ks.so
init = 0
CNF
export OPENSSL_CONF="$OUT/openssl-ks.cnf"

echo "== Sign test file under operation $KSC_OPERATION_ID =="
echo "Test" > test.txt
openssl dgst -engine pkcs11 -keyform engine \
  -sign "pkcs11:object=${KEYID};type=private" \
  "$HASH_ALGORITHM" -out test.txt.sig test.txt

echo "== Fetch the key's certificate and verify =="
curl -sS --fail-with-body --tlsv1.3 \
  -H "Authorization: Bearer $KSC_ID_TOKEN" \
  "$KSC_API_SERVER/api/keys/$KEYID" | jq -er '.certificate' > certificate.pem
openssl x509 -in certificate.pem -pubkey -noout > pubkey.pem

# A zero exit status from the signing command is not proof of a signature; this is.
openssl dgst -verify pubkey.pem "$HASH_ALGORITHM" -signature test.txt.sig test.txt

The script:

  • Copies the library into build/ and writes build/openssl-ks.cnf, which loads it through the pkcs11 engine. The dynamic_path shown is the engine's location on amd64 Ubuntu 24.04; adjust it for other images.
  • Signs test.txt with the key pkcs11:object=$KEYID;type=private. The library reads KSC_API_SERVER, KSC_ID_TOKEN, KSC_OPERATION_ID and the other KSC_* variables from the job environment.
  • Fetches the key's certificate and verifies the signature against its public key, failing the job if it does not verify.

Expected output at the end of the job log:

Verified OK

Use the prebuilt library

To skip the build, delete the build_libpkcs11ks job and its entry under needs: in sign_binary. Then replace the cp -f line in sign_binary.sh with a download to the path the OpenSSL configuration uses:

curl -sSL --fail -o "$OUT/libpkcs11ks.so" \
  https://github.com/cryptera-device-security/libpkcs11ks/releases/download/v1.0.4/libpkcs11ks.so
# Check the SHA-256 you recorded on first download:
# echo "<RECORDED_SHA256>  $OUT/libpkcs11ks.so" | sha256sum -c -

Keep the version pinned and record the module's SHA-256, as Getting the library describes. Cryptera does not publish checksums for it.

Operational practices

Practices shared by every CI integration (secrets and tokens, operation budgets, least privilege, verification) are in CI practices in the PKCS#11 Library Reference. For GitLab:

  • Gate signing in GitLab as well. Run the signing stages only after merge-request approval, or as a manual job.
  • maxusagecount is set in request_operation_await_approval.sh. If sign_binary signs more than one file, raise it to the number of signatures.
  • Remove the check_token job from Step 3 once the integration works.

If something fails

Symptom Cause and fix
"Failed to create the operation" The script prints the Keyserver response below it. Read properties[].message
KSC_OPERATION_ID: missing in sign_binary approval.env did not reach the job: sign_binary must list request_operation_await_approval under needs:
401 on every call The token's iss is not a configured identity provider. For on-premise GitLab, check the issuer URL provisioned with Cryptera (Step 1)
401 "The user/client is not valid" on every call No Keyserver subject matches the token's project_id (Step 1), or your instance maps a different claim (Step 3)
403 "Access forbidden" The subject's role lacks createoperation or order for $KEYID. Check the fwsigner nature scoping
"Approval timed out" after Not approved yet (false) No approver acted within WAIT_FOR_APPROVAL. Notify approvers, or raise the wait (validms follows it)
"Approval timed out" after Not approved yet (error) Every poll failed with a network error or a 5xx. Check KSC_API_SERVER and that the runner can reach the Keyserver
"Keyserver refused the poll (HTTP 410)" The operation was cancelled (an approver clicked Revoke) or its key revoked; the printed body says which. Run the pipeline again
409 "Need more approvers" on sign Signing started before approval completed. sign_binary must depend on the wait job through needs:
409 "…only the operation creator can access the operation" The two jobs' tokens map to different subjects, because your instance maps a per-job claim. Ask Cryptera to map a per-project claim
410 "Operation has expired" validms elapsed before signing, for example after a long queue before sign_binary. Raise the margin in the script
410 "Operation is cancelled" An approver clicked Revoke in the Approvals view, or the operation was deleted. Run the pipeline again
410 "Signing key has been revoked" KEYID names a revoked key. Use another key
OpenSSL: pkcs11:object=... not found KSC_ID_TOKEN or KSC_API_SERVER is missing from the signing job's environment, or the subject cannot see the key

Full status-code reference: Error Codes. PKCS#11-specific issues: PKCS#11 Library Reference.