Node Interface Down

Brings one or more network interfaces down on a target node for a configurable duration, then restores them. Can be used to simulate network partitions, NIC failures, or loss of connectivity at the node level.

How to Run Node Interface Down Scenarios

Choose your preferred method to run node interface down scenarios:

Example scenario file: node_interface_down.yaml

Configuration

- id: node_interface_down
  image: quay.io/krkn-chaos/krkn-network-chaos:latest
  wait_duration: 0
  test_duration: 60
  label_selector: "node-role.kubernetes.io/worker="
  instance_count: 1
  execution: parallel
  namespace: default
  # scenario specific settings
  target: ""
  interfaces: []
  recovery_time: 30
  taints: []

For the common module settings please refer to the documentation.

target: the node name to target (used when label_selector is not set)
interfaces: a list of network interface names to bring down (e.g. ["eth0", "bond0"]). Leave empty to auto-detect the node’s default interface
recovery_time: seconds to wait after bringing the interface(s) back up before continuing. Set to 0 to skip the recovery wait

Usage

To enable node interface down scenarios edit the kraken config file, go to the section kraken -> chaos_scenarios of the yaml structure and add a new element to the list named network_chaos_ng_scenarios then add the desired scenario pointing to the scenario yaml file.

kraken:
    ...
    chaos_scenarios:
        - network_chaos_ng_scenarios:
            - scenarios/openshift/node_interface_down.yaml

Note

You can specify multiple scenario files of the same type by adding additional paths to the list:

kraken:
    chaos_scenarios:
        - network_chaos_ng_scenarios:
            - scenarios/openshift/node_interface_down-1.yaml
            - scenarios/openshift/node_interface_down-2.yaml

You can also combine multiple different scenario types in the same config.yaml file. Scenario types can be specified in any order, and you can include the same scenario type multiple times:

kraken:
    chaos_scenarios:
        - network_chaos_ng_scenarios:
            - scenarios/openshift/node_interface_down.yaml
        - pod_disruption_scenarios:
            - scenarios/pod-kill.yaml
        - node_scenarios:
            - scenarios/node-reboot.yaml

Run

python run_kraken.py --config config/config.yaml

Run

$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario

$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
OR
$ docker run -e <VARIABLE>=<value> --net=host --pull=always -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario

TIP: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:

kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v ~/kubeconfig:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:<scenario>

Supported parameters

The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:

ex.) export <parameter_name>=<value>

See list of variables that apply to all scenarios here that can be used/set in addition to these scenario specific variables

Parameter	Description	Type	Default
TOTAL_CHAOS_DURATION	Duration in seconds to keep the interface(s) down	number	60
NODE_SELECTOR	Label selector to choose target nodes. If not specified, a schedulable node will be chosen at random	string	`""`
NAMESPACE	Namespace where the chaos workload pod will be deployed	string	default
INSTANCE_COUNT	Restricts the number of nodes selected by the label selector	number	1
EXECUTION	Execution mode for multiple nodes: `serial` or `parallel`	enum	parallel
INTERFACES	Comma-separated list of interface names to bring down (e.g. `eth0` or `eth0,bond0`). Leave empty to auto-detect the default interface	string	`""`
RECOVERY_TIME	Seconds to wait after bringing the interface(s) back up	number	0
WAIT_DURATION	Time in seconds to wait between scenarios when multiple scenario configs are defined in the same file	number	0
SCENARIO_TYPE	Plugin key krkn dispatches the scenario on. Fixed by the container image, not a knob to change.	-	network_chaos_ng_scenarios
SCENARIO_FILE	Path to the scenario file baked into the container image. Fixed by the image, not a knob to change.	-	scenarios/kube/node-interface-down.yml
IMAGE	The container image used to run the interface-down workload on the node	string	quay.io/krkn-chaos/krkn-network-chaos:latest
NODE_NAME	The node name to target (used when label selector is not set)	string	`""`
TAINTS	List of taints for which tolerations need to be created. Example: `["node-role.kubernetes.io/master:NoSchedule"]`	string	`""`
SERVICE_ACCOUNT	Optional service account for the chaos workload pod	string	`""`

NOTE In case of using custom metrics profile or alerts profile when CAPTURE_METRICS or ENABLE_ALERTS is enabled, mount the metrics profile from the host on which the container is run using podman/docker under /home/krkn/kraken/config/metrics-aggregated.yaml and /home/krkn/kraken/config/alerts. For example:

$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-custom-metrics-profile>:/home/krkn/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/home/krkn/kraken/config/alerts -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down

krknctl run node-interface-down [--<parameter> <value>]

Can also set any global variable listed here

Node Interface Down Parameters

Parameter	Description	Type	Default	Possible Values
--test-duration	Duration in seconds the node interface(s) will be brought down	number	60
--node-selector	Node selector to target nodes in the format “<selector>=<value>”. A workload pod will be scheduled on each selected node. If left empty a random node will be selected	string	`""`
--node-name	The specific node name to target. If set, takes precedence over node-selector	string	`""`
--namespace	Namespace where the scenario workload pod will be deployed	string	default
--instance-count	Number of nodes to target when multiple nodes match the node-selector	number	1
--execution	When multiple nodes are targeted, execute the scenario on all of them in parallel or serial	enum	parallel	parallel/serial
--interfaces	Comma-separated list of network interfaces to bring down (e.g. eth0,eth1). If empty, the default interface is auto-detected	string	`""`
--recovery-time	Additional time in seconds to wait after the node becomes Ready again before proceeding (allows the node to stabilize)	number	0
--wait-duration	Time in seconds to wait between scenarios when multiple scenario configs are defined in the same file	number	0
--image	The container image used to run the interface-down workload on the node	string	quay.io/krkn-chaos/krkn-network-chaos:latest
--service-account	The service account associated with the scenario workload pod	string	`""`
--taints	Comma-separated list of tolerations to assign to the workload pod so it can be scheduled on tainted nodes	string	`""`