Node Interface Down

Brings one or more network interfaces down on a target node for a configurable duration, then restores them. Can be used to simulate network partitions, NIC failures, or loss of connectivity at the node level.

How to Run Node Interface Down Scenarios

Choose your preferred method to run node interface down scenarios:

Example scenario file: node_interface_down.yaml

Configuration 

- id: node_interface_down
  image: quay.io/krkn-chaos/krkn-network-chaos:latest
  wait_duration: 0
  test_duration: 60
  label_selector: "node-role.kubernetes.io/worker="
  instance_count: 1
  execution: parallel
  namespace: default
  # scenario specific settings
  target: ""
  interfaces: []
  recovery_time: 30
  taints: []

For the common module settings please refer to the documentation.

  • target: the node name to target (used when label_selector is not set)
  • interfaces: a list of network interface names to bring down (e.g. ["eth0", "bond0"]). Leave empty to auto-detect the node’s default interface
  • recovery_time: seconds to wait after bringing the interface(s) back up before continuing. Set to 0 to skip the recovery wait

Usage

To enable node interface down scenarios edit the kraken config file, go to the section kraken -> chaos_scenarios of the yaml structure and add a new element to the list named network_chaos_ng_scenarios then add the desired scenario pointing to the scenario yaml file.

kraken:
    ...
    chaos_scenarios:
        - network_chaos_ng_scenarios:
            - scenarios/openshift/node_interface_down.yaml

Run 

python run_kraken.py --config config/config.yaml

Run 

$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario

$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
OR
$ docker run -e <VARIABLE>=<value> --net=host --pull=always -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario

TIP: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:

kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host --pull=always -v ~/kubeconfig:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:<scenario>

Supported parameters

The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:

ex.) export <parameter_name>=<value>

See list of variables that apply to all scenarios here that can be used/set in addition to these scenario specific variables

Parameter Description Default
TOTAL_CHAOS_DURATION Duration in seconds to keep the interface(s) down 60
RECOVERY_TIME Seconds to wait after bringing the interface(s) back up 0
NODE_SELECTOR Label selector to choose target nodes. If not specified, a schedulable node will be chosen at random ""
NODE_NAME The node name to target (used when label selector is not set)
INSTANCE_COUNT Restricts the number of nodes selected by the label selector 1
EXECUTION Execution mode for multiple nodes: serial or parallel parallel
INTERFACES Comma-separated list of interface names to bring down (e.g. eth0 or eth0,bond0). Leave empty to auto-detect the default interface ""
NAMESPACE Namespace where the chaos workload pod will be deployed default
TAINTS List of taints for which tolerations need to be created. Example: ["node-role.kubernetes.io/master:NoSchedule"] ""
SERVICE_ACCOUNT Optional service account for the chaos workload pod ""

NOTE In case of using custom metrics profile or alerts profile when CAPTURE_METRICS or ENABLE_ALERTS is enabled, mount the metrics profile from the host on which the container is run using podman/docker under /home/krkn/kraken/config/metrics-aggregated.yaml and /home/krkn/kraken/config/alerts. For example:

$ podman run --name=<container_name> --net=host --pull=always --env-host=true -v <path-to-custom-metrics-profile>:/home/krkn/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/home/krkn/kraken/config/alerts -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d containers.krkn-chaos.dev/krkn-chaos/krkn-hub:node-interface-down

krknctl run node-interface-down [--<parameter> <value>]

Can also set any global variable listed here

Node Interface Down Parameters

Argument Type Description Required Default Value
--chaos-duration number Duration in seconds to keep the interface(s) down false 60
--recovery-time number Seconds to wait after bringing the interface(s) back up before continuing false 0
--node-selector string Label selector to choose target nodes false node-role.kubernetes.io/worker=
--node-name string Node name to target (used when node-selector is not set) false
--namespace string Namespace where the chaos workload pod will be deployed false default
--instance-count number Number of nodes to target from those matching the selector false 1
--execution enum Execution mode when targeting multiple nodes: serial or parallel false parallel
--interfaces string Comma-separated list of interface names to bring down. Leave empty to auto-detect the default interface false
--image string The chaos workload container image false quay.io/krkn-chaos/krkn-network-chaos:latest
--taints string List of taints for which tolerations need to be created false