Administration

Krkn Operator documentation v1.1.0; captured October 13, 2026 from website commit 3b059252814d83dc2ff56caa6031603229ed9cdf.

• 1: Cluster Management
• 2: User Management
• 3: Registry Management
• 4: Mirror Signed Images
• 5: Provider Configuration
• 6: Elasticsearch Management
• 7: Cloud Credentials Management
• 8: Backup and Restore

Admin creates Group
      │
      ├── Assigns Clusters (target-1, target-2, ...)
      ├── Assigns Permissions (View, Run, Cancel)
      ├── Assigns Registry Visibility
      └── Assigns Cloud Credential Visibility
            │
            └── Users in this group inherit everything

1 - Cluster Management

Cluster Management Admin

Register target Kubernetes clusters for chaos scenario execution. Krkn Operator runs on a management cluster and executes scenarios against registered targets; it does not run scenarios against itself.

Operations

Once registered, clusters become available for assignment to groups through User Management.

2 - User Management

User Management Admin

Organize users through groups that define cluster access and permissions. The group is the central unit of the permission model — users inherit all capabilities from their assigned group.

Groups

A group defines what its members can access and do on the platform.

Groups also receive permission flags that control what operations their members can perform:

Users

Users are created and assigned to a group during creation.

Permission Cascade

Group "SRE Team"
  ├── Clusters: production-us, production-eu
  ├── Permissions: View, Run
  └── Members: alice, bob
        │
        ├── alice → can run scenarios on production-us, production-eu
        └── bob   → can run scenarios on production-us, production-eu

3 - Registry Management

Registry Management Admin

Configure private container registries for chaos scenario images and control which groups can access them. By default, scenarios are pulled from the public Quay.io registry. Private registries enable air-gapped deployments and custom scenario images.

Creating a Private Registry

Configure a private registry by providing its connection settings (URL, credentials, TLS configuration).

Visibility

Each registry has a visibility setting that controls who can use it:

Impact on Scenario Selection

When a user selects a private registry during scenario execution, only the scenarios that have been mirrored to that registry will be available. Scenarios not present in the private registry will not appear in the selection list.

The Operator requires a valid signature for each selected scenario image by default. For the commands and workload-image override, see Mirror Signed Images.

4 - Mirror Signed Images

Mirror Signed Images Admin

For a private or disconnected environment, mirror both the scenario image and any helper workload image it launches. Copy their Cosign signatures with the images. A normal docker pull/docker push, podman pull/podman push, or image-only mirror can leave the signatures behind.

The source repository is quay.io/krkn-chaos/krkn-hub-multiarch. Its tags include scenarios such as node-cpu-hog and pod-scenarios, and helper images such as workload-krkn-hog and workload-krkn-tools. Choose the tags required by the scenarios you plan to run.

Mirror directly between registries

On a machine that can reach Quay.io and your private registry, install Cosign and authenticate to the destination registry. The example mirrors node-cpu-hog and its workload-krkn-hog helper into the same repository. Replace the destination with your own registry host and repository.

SOURCE=quay.io/krkn-chaos/krkn-hub-multiarch
MIRROR=registry.example.com/chaos/krkn-hub-multiarch

cosign login registry.example.com
cosign copy "$SOURCE:node-cpu-hog" "$MIRROR:node-cpu-hog"
cosign copy "$SOURCE:workload-krkn-hog" "$MIRROR:workload-krkn-hog"

cosign copy carries the image and its associated signature artifacts. Repeat it for every scenario and workload tag you need. Preserve the tag names so the private registry’s scenario list and the IMAGE override use the same names as the source. Do not use an image-only copy as a substitute.

For a fully disconnected registry, save the signed images on a connected machine, transfer the resulting directories through your approved transfer path, and load them into the destination registry:

# Connected machine
SOURCE=quay.io/krkn-chaos/krkn-hub-multiarch
cosign save --dir ./node-cpu-hog "$SOURCE:node-cpu-hog"
cosign save --dir ./workload-krkn-hog "$SOURCE:workload-krkn-hog"

# After transferring both directories to a machine with access to the private registry
MIRROR=registry.example.com/chaos/krkn-hub-multiarch
cosign login registry.example.com
cosign load --dir ./node-cpu-hog "$MIRROR:node-cpu-hog"
cosign load --dir ./workload-krkn-hog "$MIRROR:workload-krkn-hog"

cosign save and cosign load include the associated signatures; a container image tarball alone does not.

Verify the mirror

On a connected staging machine, obtain the Krkn Cosign public key from the Krknctl v0.14.2-beta revision used by Operator 1.1.0 through a trusted channel, then verify both destination images:

MIRROR=registry.example.com/chaos/krkn-hub-multiarch
curl -fsSLo cosign.pub https://raw.githubusercontent.com/krkn-chaos/krknctl/991cbce5383836e40f2134ef456dd71c0b40ceec/pkg/verify/cosign.pub
cosign verify --key cosign.pub --insecure-ignore-tlog=true "$MIRROR:node-cpu-hog"
cosign verify --key cosign.pub --insecure-ignore-tlog=true "$MIRROR:workload-krkn-hog"

Krkn uses key-based signatures, so the --insecure-ignore-tlog=true option skips the transparency-log check while Cosign still checks the public key and signed image digest. If verification fails, inspect the destination registry’s support for Cosign signature artifacts and copy the signed image again. Do not re-sign a changed image with an unrelated key: the Operator trusts the Krkn key.

Configure the Operator and workload override

1. In Registry Management, create a private registry with Registry URL registry.example.com and Scenario Repository chaos/krkn-hub-multiarch. Configure credentials, TLS, and group visibility as needed.
2. In Run Scenarios, select that registry and choose node-cpu-hog. The Operator resolves and verifies the mirrored scenario image before running it.
3. In the scenario’s parameter form, override IMAGE with registry.example.com/chaos/krkn-hub-multiarch:workload-krkn-hog. This is the helper pod image used by node-cpu-hog; selecting the private scenario registry does not rewrite this parameter automatically.

Other scenarios may launch different helper images. For example, network-chaos uses workload-krkn-tools, and pod-network-filter uses workload-krkn-network-chaos. Mirror the matching workload-* tag with Cosign and set that scenario’s IMAGE parameter to its private URL. Consult the selected scenario’s parameters and the source tag list for the exact image.

If a run does not start

• Scenario missing from the list: confirm its tag exists in the configured scenario repository and the user can access that registry.
• Unsigned or untrusted scenario: confirm the image and signature were copied together, then verify the destination tag with the Krkn public key. A changed image digest invalidates the original signature.
• Workload pod cannot pull its image: check the IMAGE override, the target cluster’s access to the private registry, and its pull credentials and TLS configuration.

5 - Provider Configuration

Provider Configuration Admin

Configure target providers that integrate with external cluster management platforms. The provider configuration interface adapts dynamically based on the selected provider.

Supported Providers

Cloud Provider Configuration

Krkn Operator supports cloud scenarios that use saved provider credentials. Administrators create and manage these credentials through Cloud Credentials Management. Users can select credentials their group is allowed to access.

The following scenarios interact with provider infrastructure and need the corresponding cloud or bare-metal credentials:

6 - Elasticsearch Management

Elasticsearch Management Admin

Administrators can configure and save Elasticsearch endpoints that users can reference when executing chaos scenarios. This enables centralized observability configuration without requiring users to manually enter connection details.

Overview

Elasticsearch integration allows krkn scenarios to index execution metrics, logs, and telemetry data for analysis and historical tracking. Instead of requiring each user to configure Elasticsearch details manually, admins can:

• Pre-configure Elasticsearch endpoints with connection details
• Save named configurations for reuse across all scenarios
• Control which Elasticsearch endpoints are available platform-wide

Users can then select from the saved Elasticsearch configurations when setting global parameters during scenario execution.

Adding an Elasticsearch Endpoint

1. Navigate to Administration > Elasticsearch Management
2. Click Add Elasticsearch
3. Configure the following details:

4. Click Save to add the configuration to the saved list

Managing Saved Configurations

View Saved Elasticsearch Endpoints

The Elasticsearch Management page displays all saved configurations:

• Name: Friendly identifier shown to users
• URL: Elasticsearch endpoint
• Index: Target index pattern
• Actions: Edit or Delete options

Edit an Endpoint

1. Click the Edit icon next to the saved configuration
2. Modify any field (Name, URL, Index, credentials, SSL verification)
3. Click Save to update

Delete an Endpoint

1. Click the Delete icon next to the saved configuration
2. Confirm the deletion

How Users Select Elasticsearch

When users configure global parameters for a scenario or workflow, they can:

1. Expand the Global parameters section
2. Locate the Elasticsearch parameter
3. Click the dropdown to view all saved Elasticsearch configurations
4. Select an Elasticsearch endpoint by name (e.g., “Production ES”)

The selected configuration automatically applies the admin-defined connection details (URL, index, credentials) to the scenario execution.

Users cannot:

• Add new Elasticsearch endpoints from the scenario configuration interface
• Edit existing saved configurations
• View sensitive details like passwords (only admins can see/edit credentials)

Users can:

• Choose from the dropdown list of saved Elasticsearch endpoints
• Opt not to use Elasticsearch (by leaving the parameter unset)

Best Practices

Naming Conventions

Use descriptive names that indicate the purpose or environment:

• ✅ Production Observability
• ✅ Staging Metrics
• ✅ Dev Team Shared ES
• ❌ ES1, Elasticsearch, Test

Security

• Use dedicated credentials for krkn with limited write permissions to specific indices
• Enable SSL verification for production endpoints
• Restrict index patterns to prevent accidental data overwrites (e.g., krkn-* not *)

Index Patterns

Use time-based or scenario-based index naming:

• krkn-chaos-{YYYY-MM-DD} — daily indices for easy retention management
• krkn-scenarios-* — scenario-specific index pattern
• krkn-{environment}-* — environment-specific indices

Verification

After adding an Elasticsearch endpoint, verify it works:

1. Run a simple chaos scenario with the Run Scenarios feature
2. Select the saved Elasticsearch configuration in Global parameters
3. Execute the scenario
4. Check the Elasticsearch index for new documents:

   curl -X GET "https://elasticsearch.example.com:9200/krkn-chaos-*/_search?pretty" \
     -u username:password
   

Successful indexing confirms the configuration is correct and accessible from the krkn-operator platform.

7 - Cloud Credentials Management

Cloud Credentials Management Admin

Administrators can configure and save cloud provider credentials that users select when running cloud-dependent chaos scenarios. Credentials are stored as Kubernetes Secrets and injected into scenario pods via SecretKeyRef — plaintext cloud secrets never appear in the Custom Resource or pod environment values.

Overview

Many krkn scenarios need cloud API access (for example node stop/start, zone outages, or power outages). Instead of asking every user to paste access keys into scenario parameters, admins can:

• Pre-configure named credentials for each supported cloud provider
• Control visibility so credentials are available to everyone or only to selected groups
• Reuse the same credential across single runs and Chaos Studio workflows

Users then select a saved credential when configuring a scenario. Matching cloud parameter fields are filled from the Secret and locked in the UI.

Supported Providers

When a credential is applied, the platform also sets CLOUD_TYPE to the value expected by krkn-hub for that provider:

Adding a Cloud Credential

1. Navigate to Settings > Cloud Credentials
2. Click Create Cloud Credential (or the add action)
3. Configure the common fields, then the provider-specific fields below
4. Click Save to create the credential

Common fields

Provider-specific fields

AWS

GCP

GCP credentials are mounted into the scenario pod as a file. The operator also sets GOOGLE_APPLICATION_CREDENTIALS to that mount path.

Azure

OpenStack

Baremetal (IPMI/BMC)

VMware

IBM Cloud

Visibility

Each credential has a visibility setting that controls who can use it:

Managing Saved Credentials

View Credentials

The Cloud Credentials page lists saved configurations with:

• Name: Identifier shown in scenario selection
• Provider: Cloud provider type
• Access: Everyone or assigned groups
• Actions: Edit or Delete

Admins can filter the list by name, provider, and access type.

Edit a Credential

1. Click Edit next to the credential
2. Update description, visibility, groups, or provider secret fields
3. Click Save

The provider cannot be changed after creation — create a new credential if you need a different provider. Secret values are never shown after creation. Leave a secret field empty to keep the existing value; provide a new value only when rotating credentials.

Delete a Credential

1. Click Delete next to the credential
2. Confirm the deletion

How Credentials Are Injected

When a run references a cloud credential:

1. The console (and API) store only the credential name on the run (cloudCredentialRef)
2. Cloud-related environment variables from the request are stripped so plaintext keys cannot land in the Custom Resource
3. The operator loads the credential Secret and injects values into the scenario pod using SecretKeyRef (and file mounts where required, such as GCP service-account JSON)

Users cannot:

• Create or edit credentials from the scenario wizard
• View secret values for saved credentials
• Select credentials outside their visibility (group membership)

Users can:

• Choose from the credentials available to them
• Clear the selection and enter parameters manually when no credential is needed

How Users Select Credentials

When configuring parameters for a single scenario or a Chaos Studio node:

1. Open the Load Cloud Credential section (shown when the scenario has cloud-related fields)
2. Select a saved credential from the dropdown (only credentials your group can access)
3. Matching cloud fields become read-only and show masked placeholders (••••••••)
4. Fields for other cloud providers are hidden so the form only shows the applied provider
5. Continue configuring non-cloud parameters and run the scenario

In Chaos Studio, set the credential per node when configuring that node’s scenario. Each cloud-dependent node can use a different credential.

Best Practices

Naming

Use names that encode provider and environment:

• ✅ aws-prod-ci
• ✅ azure-staging-sre
• ✅ bmc-lab-rack-a
• ❌ creds, test, key1

Access control

• Prefer group-based visibility for production credentials
• Use Everyone only for shared non-production accounts
• Rotate keys by editing the credential rather than creating duplicates with the same name

Security

• Never paste long-lived cloud secrets into scenario parameter fields when a saved credential exists
• Treat credential Secrets like any other cluster secret: limit RBAC outside the operator namespace
• Review group membership before granting access to production cloud accounts

Verification

After adding a credential, verify injection works:

1. Run a cloud-dependent scenario with Run Scenarios
2. Select the saved credential under Load Cloud Credential
3. Confirm cloud fields are disabled and show masked placeholders
4. Execute the run and confirm the job starts successfully
5. Optionally inspect the scenario pod: cloud env vars should reference the credential Secret by name, not plaintext values

8 - Backup and Restore

Backup and Restore Admin

The Krkn Operator includes a built-in backup and restore feature accessible from the web console. Backups contain portable operator configuration for:

• Admin settings
• Cluster targets
• Users and groups
• Private registries
• Elasticsearch configuration

The backup also includes the operator-managed secrets required for these settings.

This feature is not a complete Kubernetes or cluster backup. It does not include runtime registration objects, ACM integration data, uploaded files, file metadata, or the JWT signing key.

Creating a Backup

1. Navigate to the Backup & Restore card in the console.
2. Click Download Backup.
3. The operator creates a compressed .tar.gz archive and saves it to your local machine.

Restoring from a Backup

1. Open the Backup & Restore card in the console.
2. Click Upload & Restore.
3. Select a previously downloaded .tar.gz backup archive and confirm the restore.
4. Monitor the restore progress from the same page.

Only one restore can run at a time. The maximum upload size is 100 MB.

The restore runs asynchronously. Existing resources are updated with the corresponding configuration from the backup, while resources that do not exist are created. Resources absent from the backup are not deleted.

After the restore completes, the operator refreshes the connectivity status of restored target clusters.

Restore Status

Restore status is available from the same page and can be:

• In progress
• Completed
• Failed

Restore job status is held in memory and is not persisted across operator restarts.