IO Hog Scenario
Overview
The IO Hog scenario is designed to create disk I/O pressure on one or more nodes in your Kubernetes/OpenShift cluster for a specified duration. This scenario helps you test how your cluster and applications respond to high disk I/O utilization and storage-related bottlenecks.
How It Works
The scenario deploys a stress workload pod on targeted nodes. These pods use stress-ng to perform intensive write operations to disk, consuming I/O resources according to your configuration. The scenario supports attaching node paths to the pod as a hostPath volume or using custom pod volume definitions, allowing you to test I/O pressure on specific storage targets.
When to Use
Use the IO Hog scenario to:
- Test your cluster’s behavior under disk I/O pressure
- Validate that I/O resource limits are properly configured
- Evaluate the impact of disk I/O contention on application performance
- Test whether your monitoring systems properly detect disk saturation
- Verify that storage performance meets requirements under stress
- Simulate scenarios where pods perform excessive disk writes
- Test the resilience of persistent volume configurations
- Validate disk I/O quotas and rate limiting
Key Configuration Options
In addition to the common hog scenario options, IO Hog scenarios support:
| Option | Type | Description |
|---|
io-block-size | string | The size of each individual write operation performed by the stressor |
io-write-bytes | string | The total amount of data that will be written by the stressor. Can be specified as a percentage (%) of free space on the filesystem or in absolute units (b, k, m, g for Bytes, KBytes, MBytes, GBytes) |
io-target-pod-folder | string | The path within the pod where the volume will be mounted |
io-target-pod-volume | dictionary | The pod volume definition that will be stressed by the scenario (typically a hostPath volume) |
Modifying the structure of io-target-pod-volume might alter how the hog operates, potentially rendering it ineffective.
Example Values
io-block-size: "1m" - Write in 1 megabyte blocksio-block-size: "4k" - Write in 4 kilobyte blocksio-write-bytes: "50%" - Write data equal to 50% of available free spaceio-write-bytes: "10g" - Write 10 gigabytes of data
Usage
Select your deployment method to get started:
1 - IO Hog Scenarios using Krkn
To enable this plugin add the pointer to the scenario input file scenarios/kube/io-hog.yaml as described in the
Usage section.
io-hog options
In addition to the common hog scenario options, you can specify the below options in your scenario configuration to target specific pod IO
| Option | Type | Description |
|---|
io-block-size | string | the block size written by the stressor |
io-write-bytes | string | the total amount of data that will be written by the stressor. The size can be specified as % of free space on the file system or in units of Bytes, KBytes, MBytes and GBytes using the suffix b, k, m or g |
io-target-pod-folder | string | the folder where the volume will be mounted in the pod |
io-target-pod-volume | dictionary | the pod volume definition that will be stressed by the scenario. |
Modifying the structure of io-target-pod-volume might alter how the hog operates, potentially rendering it ineffective.
Usage
To enable hog scenarios edit the kraken config file, go to the section kraken -> chaos_scenarios of the yaml structure
and add a new element to the list named hog_scenarios then add the desired scenario
pointing to the hog.yaml file.
kraken:
...
chaos_scenarios:
- hog_scenarios:
- scenarios/kube/io-hog.yml
2 - IO Hog Scenario using Krkn-Hub
This scenario hogs the IO on the specified node on a Kubernetes/OpenShift cluster for a specified duration. For more information refer the following documentation.
Run
If enabling Cerberus to monitor the cluster and pass/fail the scenario post chaos, refer docs. Make sure to start it before injecting the chaos and set CERBERUS_ENABLED environment variable for the chaos injection container to autoconnect.
$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-io-hog
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
Note
–env-host: This option is not available with the remote Podman client, including Mac and Windows (excluding WSL2) machines.
Without the –env-host option you’ll have to set each enviornment variable on the podman command line like -e <VARIABLE>=<value>$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v <path-to-kube-config>:/root/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-io-hog
OR
$ docker run -e <VARIABLE>=<value> --net=host --pull=always -v <path-to-kube-config>:/root/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-io-hog
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
Tip
Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v ~kubeconfig:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:<scenario>Supported parameters
The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:
Example if –env-host is used:
export <parameter_name>=<value>
OR on the command line like example:
-e <VARIABLE>=<value>
See list of variables that apply to all scenarios here that can be used/set in addition to these scenario specific variables
| Parameter | Description | Default |
|---|
| TOTAL_CHAOS_DURATION | Set chaos duration (in sec) as desired | 180 |
| IO_BLOCK_SIZE | string size of each write in bytes. Size can be from 1 byte to 4m | 1m |
| IO_WORKERS | Number of stressorts | 5 |
| IO_WRITE_BYTES | string writes N bytes for each hdd process. The size can be expressed as % of free space on the file system or in units of Bytes, KBytes, MBytes and GBytes using the suffix b, k, m or g | 10m |
| NAMESPACE | Namespace where the scenario container will be deployed | default |
| NODE_SELECTOR | defines the node selector for choosing target nodes. If not specified, one schedulable node in the cluster will be chosen at random. If multiple nodes match the selector, all of them will be subjected to stress. If number-of-nodes is specified, that many nodes will be randomly selected from those identified by the selector. | "" |
| TAINTS | List of taints for which tolerations need to created. Example: [“node-role.kubernetes.io/master:NoSchedule”] | [] |
| NODE_MOUNT_PATH | the local path in the node that will be mounted in the pod and that will be filled by the scenario | "" |
| NUMBER_OF_NODES | restricts the number of selected nodes by the selector | "" |
| IMAGE | the container image of the stress workload | quay.io/krkn-chaos/krkn-hog |
Note
In case of using custom metrics profile or alerts profile when CAPTURE_METRICS or ENABLE_ALERTS is enabled, mount the metrics profile from the host on which the container is run using podman/docker under /home/krkn/kraken/config/metrics-aggregated.yaml and /home/krkn/kraken/config/alerts.For example:
$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-io-hog
3 - IO Hog using Krknctl
krknctl run node-io-hog (optional: --<parameter>:<value> )
Can also set any global variable listed here
| Parameter | Description | Type | Default |
|---|
--chaos-duration | Set chaos duration (in sec) as desired | number | 60 |
--oo-block-size | sSze of each write in bytes. Size can be from 1 byte to 4 Megabytes (allowed suffix are b,k,m) | string | 1m |
--io-workers | Number of stressor instances | number | 5 |
--io-write-bytes | string writes N bytes for each hdd process. The size can be expressed as % of free space on the file system or in units of Bytes, KBytes, MBytes and GBytes using the suffix b, k, m or g | string | 10m |
--node-mount-path | the path in the node that will be mounted in the pod and where the io hog will be executed. NOTE: be sure that kubelet has the rights to write in that node path | string | /root |
--namespace | Namespace where the scenario container will be deployed | string | default |
--node-selector | Node selector where the scenario containers will be scheduled in the format “=”. NOTE: Will be instantiated a container per each node selected with the same scenario options. If left empty a random node will be selected | string | |
--taints | List of taints for which tolerations need to created. For example [“node-role.kubernetes.io/master:NoSchedule”]" | string | [] |
--number-of-nodes | restricts the number of selected nodes by the selector | number | |
--image | The hog container image. Can be changed if the hog image is mirrored on a private repository | string | quay.io/krkn-chaos/krkn-hog |
To see all available scenario options
krknctl run node-io-hog --help
