Skip to main content

Kubernetes probe

With the proliferation of custom resources & operators, especially in the case of stateful applications, the steady-state is manifested as status parameters/flags within Kubernetes resources. K8s Probe addresses verification of the desired resource state by allowing users to define the Kubernetes GVR (group-version-resource) with appropriate filters (field selectors/label selectors). The experiment makes use of the Kubernetes Dynamic Client to achieve this. The probe supports following CRUD operations:

  • create: It creates Kubernetes resource based on the data provided inside the probe.k8sProbe/inputs.data field.
  • delete: It deletes matching kubernetes resource via GVR and filters (field selectors/label selectors).
  • present: It checks for the presence of kubernetes resource based on GVR and filters (field selectors/label selectors).
  • absent: It checks for the absence of kubernetes resource based on GVR and filters (field selectors/label selectors).

Defining the probe

You can define the probes at .spec.experiments[].spec.probe path inside the chaos engine.

kind: Workflow
apiVersion: argoproj.io/v1alpha1
spec:
templates:
- inputs:
artifacts:
- raw:
data: |
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
spec:
experiments:
- spec:
probe:
####################################
Probes are defined here
####################################

Schema

Listed below is the probe schema for the Kubernetes probe, with properties shared across all the probes and properties unique to the Kubernetes probe.

FieldDescriptionTypeRangeNotes
groupFlag to hold the group of the kubernetes resource for the k8sProbeMandatoryN/A type: stringThe group contains group of the kubernetes resource on which k8sProbe performs the specified operation.
versionFlag to hold the apiVersion of the kubernetes resource for the k8sProbeMandatoryN/A type: stringThe version contains apiVersion of the kubernetes resource on which k8sProbe performs the specified operation
resourceFlag to hold the kubernetes resource name for the k8sProbeMandatoryN/A type: stringThe resource contains the kubernetes resource name on which k8sProbe performs the specified operation.
namespaceFlag to hold the namespace of the kubernetes resource for the k8sProbeMandatoryN/A type: stringThe namespace contains namespace of the kubernetes resource on which k8sProbe performs the specified operation.
fieldSelectorFlag to hold the fieldSelectors of the kubernetes resource for the k8sProbeOptionalN/A type: stringThe fieldSelector contains fieldSelector to derived the kubernetes resource on which k8sProbe performs the specified operation.
labelSelectorFlag to hold the labelSelectors of the kubernetes resource for the k8sProbeOptionalN/A type: stringThe labelSelector contains labelSelector to derived the kubernetes resource on which k8sProbe performs the specified operation.
operationFlag to hold the operation type for the k8sProbeMandatoryN/A type: stringThe operation contains operation which should be applied on the kubernetes resource as part of k8sProbe. It supports four type of operation. It can be one of create, delete, present, absent.
dataFlag to hold the Kubernetes resource manifestOptionalN/A type: stringThe data contains Kubernetes resource manifest that is relevant solely for the purpose of the creation operation.

Run properties

FieldDescriptionTypeRangeNotes
probeTimeoutFlag to hold the timeout of the probeMandatoryN/A type: stringThe probeTimeout represents the time limit for the probe to execute the specified check and return the expected data
attemptFlag to hold the attempt of the probeMandatoryN/A type: integerThe attempt contains the number of times a check is run upon failure in the previous attempts before declaring the probe status as failed.
intervalFlag to hold the interval of the probeMandatoryN/A type: stringThe interval contains the interval for which probes waits between subsequent retries
probePollingIntervalFlag to hold the polling interval for the probes (applicable for all modes)OptionalN/A type: stringThe probePollingInterval contains the time interval for which continuous and onchaos probe should be sleep after each iteration
initialDelaySecondsFlag to hold the initial delay interval for the probesOptionalN/A type: integerThe initialDelaySeconds represents the initial waiting time interval for the probes.
stopOnFailureFlags to hold the stop or continue the experiment on probe failureOptionalN/A type: booleanThe stopOnFailure can be set to true/false to stop or continue the experiment execution after probe fails

Definition

probe:
- name: "check-app-status"
type: "k8sProbe"
k8sProbe/inputs:
group: ""
version: "v1"
resource: "pods"
namespace: "default"
fieldSelector: "status.phase=Running"
labelSelector: "app=nginx"
operation: "present" # it can be present, absent, create, delete
mode: "EOT"
runProperties:
probeTimeout: 5s
interval: 2s
attempt: 1

Create Operation

It creates the kubernetes resource based on the data specified inside in the probe.k8sProbe/inputs.data field.

Use the following example to tune this:

# create the given resource provided inside data field
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
appinfo:
appns: "default"
applabel: "app=nginx"
appkind: "deployment"
chaosServiceAccount: litmus-admin
experiments:
- name: pod-delete
spec:
probe:
- name: "create-percona-pvc"
type: "k8sProbe"
k8sProbe/inputs:
# group of the resource
group: ""
# version of the resource
version: "v1"
# name of the resource
resource: "persistentvolumeclaims"
# namespace where the instance of resource should be created
namespace: "default"
# type of operation
# supports: create, delete, present, absent
operation: "create"
# contains manifest, which can be used to create the resource
data: |
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: percona-mysql-claim
labels:
target: percona
spec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi
mode: "SOT"
runProperties:
probeTimeout: 5s
interval: 2s
attempt: 1

Delete Operation

It deletes matching kubernetes resources via GVR and filters (field selectors/label selectors) provided at probe.k8sProbe/inputs.

Use the following example to tune this:

# delete the resource matched with the given inputs
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
appinfo:
appns: "default"
applabel: "app=nginx"
appkind: "deployment"
chaosServiceAccount: litmus-admin
experiments:
- name: pod-delete
spec:
probe:
- name: "delete-percona-pvc"
type: "k8sProbe"
k8sProbe/inputs:
# group of the resource
group: ""
# version of the resource
version: "v1"
# name of the resource
resource: "persistentvolumeclaims"
# namespace of the instance, which needs to be deleted
namespace: "default"
# labels selectors for the k8s resource, which needs to be deleted
labelSelector: "openebs.io/target-affinity=percona"
# fieldselector for the k8s resource, which needs to be deleted
fieldSelector: ""
# type of operation
# supports: create, delete, present, absent
operation: "delete"
mode: "EOT"
runProperties:
probeTimeout: 5s
interval: 2s
attempt: 1

Present Operation

It checks for the presence of kubernetes resource based on GVR and filters (field selectors/labelselectors) provided at probe.k8sProbe/inputs.

Use the following example to tune this:

# verify the existance of the resource matched with the given inputs inside cluster
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
appinfo:
appns: "default"
applabel: "app=nginx"
appkind: "deployment"
chaosServiceAccount: litmus-admin
experiments:
- name: pod-delete
spec:
probe:
- name: "check-percona-pvc-presence"
type: "k8sProbe"
k8sProbe/inputs:
# group of the resource
group: ""
# version of the resource
version: "v1"
# name of the resource
resource: "persistentvolumeclaims"
# namespace where the instance of resource
namespace: "default"
# labels selectors for the k8s resource
labelSelector: "openebs.io/target-affinity=percona"
# fieldselector for the k8s resource
fieldSelector: ""
# type of operation
# supports: create, delete, present, absent
operation: "present"
mode: "SOT"
runProperties:
probeTimeout: 5s
interval: 2s
attempt: 1

Absent Operation

It checks for the absence of kubernetes resource based on GVR and filters (field selectors/labelselectors) provided at probe.k8sProbe/inputs.

Use the following example to tune this:

# verify that the no resource should be present in cluster with the given inputs
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
appinfo:
appns: "default"
applabel: "app=nginx"
appkind: "deployment"
chaosServiceAccount: litmus-admin
experiments:
- name: pod-delete
spec:
probe:
- name: "check-percona-pvc-absence"
type: "k8sProbe"
k8sProbe/inputs:
# group of the resource
group: ""
# version of the resource
version: "v1"
# name of the resource
resource: "persistentvolumeclaims"
# namespace where the instance of resource
namespace: "default"
# labels selectors for the k8s resource
labelSelector: "openebs.io/target-affinity=percona"
# fieldselector for the k8s resource
fieldSelector: ""
# type of operation
# supports: create, delete, present, absent
operation: "absent"
mode: "EOT"
runProperties:
probeTimeout: 5s
interval: 2s
attempt: 1