New to KubeDB? Please start here.

Restart ClickHouse

KubeDB supports restarting the ClickHouse database via a ClickHouseOpsRequest. Restarting is useful if some pods are got stuck in some phase, or they are not working correctly. This tutorial will show you how to use that.

Before You Begin

  • At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using kind.

  • Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps here.

  • To keep things isolated, this tutorial uses a separate namespace called demo throughout this tutorial.

  $ kubectl create ns demo
  namespace/demo created

Note: YAML files used in this tutorial are stored in docs/examples/clickhouse folder in GitHub repository kubedb/docs.

Deploy ClickHouse

In this section, we are going to deploy a ClickHouse database using KubeDB.

apiVersion: kubedb.com/v1alpha2
kind: ClickHouse
metadata:
  name: clickhouse-prod
  namespace: demo
spec:
  version: 24.4.1
  clusterTopology:
    clickHouseKeeper:
      externallyManaged: false
      spec:
        replicas: 3
        storage:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi
    cluster:
        name: appscode-cluster
        shards: 2
        replicas: 2
        podTemplate:
          spec:
            containers:
              - name: clickhouse
                resources:
                  limits:
                    memory: 4Gi
                  requests:
                    cpu: 500m
                    memory: 2Gi
            initContainers:
              - name: clickhouse-init
                resources:
                  limits:
                    memory: 1Gi
                  requests:
                    cpu: 500m
                    memory: 1Gi
        storage:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi
  deletionPolicy: WipeOut

Let’s create the ClickHouse CR we have shown above,

$ kubectl create -f https://github.com/kubedb/docs/raw/v2025.7.31/docs/examples/clickhouse/restart/clickhouse-cluster.yaml
clickhouse.kubedb.com/clickhouse-prod created

Apply Restart opsRequest

apiVersion: ops.kubedb.com/v1alpha1
kind: ClickHouseOpsRequest
metadata:
  name: restart
  namespace: demo
spec:
  type: Restart
  databaseRef:
    name: clickhouse-prod
  timeout: 3m
  apply: Always
  • spec.type specifies the Type of the ops Request
  • spec.databaseRef holds the name of the ClickHouse CR. It should be available in the same namespace as the opsRequest
  • The meaning of spec.timeout & spec.apply fields will be found here

Note: The method of restarting the combined node is exactly same as above. All you need, is to specify the corresponding ClickHouse name in spec.databaseRef.name section.

Let’s create the ClickHouseOpsRequest CR we have shown above,

$ kubectl create -f https://github.com/kubedb/docs/raw/v2025.7.31/docs/examples/clickhouse/restart/ops.yaml
clickhouseopsrequest.ops.kubedb.com/restart created

Now the Ops-manager operator will first restart the controller pods, then broker of the referenced clickhouse.

➤ kubectl get clickhouseopsrequest -n demo clickhouse-restart
NAME                 TYPE      STATUS       AGE
clickhouse-restart   Restart   Successful   8m33s

➤ kubectl describe clickhouseopsrequest -n demo clickhouse-restart 
Name:         clickhouse-restart
Namespace:    demo
Labels:       <none>
Annotations:  <none>
API Version:  ops.kubedb.com/v1alpha1
Kind:         ClickHouseOpsRequest
Metadata:
  Creation Timestamp:  2025-08-25T08:51:27Z
  Generation:          1
  Resource Version:    789914
  UID:                 f7986361-2463-4a98-bf51-aa6808ab2aae
Spec:
  Apply:  IfReady
  Database Ref:
    Name:   clickhouse-prod
  Timeout:  10m
  Type:     Restart
Status:
  Conditions:
    Last Transition Time:  2025-08-25T08:51:27Z
    Message:               ClickHouse ops-request has started to restart ClickHouse nodes
    Observed Generation:   1
    Reason:                Restart
    Status:                True
    Type:                  Restart
    Last Transition Time:  2025-08-25T08:53:40Z
    Message:               Successfully Restarted ClickHouse pods
    Observed Generation:   1
    Reason:                RestartPods
    Status:                True
    Type:                  RestartPods
    Last Transition Time:  2025-08-25T08:51:35Z
    Message:               get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0
    Observed Generation:   1
    Status:                True
    Type:                  GetPod--clickhouse-prod-appscode-cluster-shard-0-0
    Last Transition Time:  2025-08-25T08:51:35Z
    Message:               evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0
    Observed Generation:   1
    Status:                True
    Type:                  EvictPod--clickhouse-prod-appscode-cluster-shard-0-0
    Last Transition Time:  2025-08-25T08:51:40Z
    Message:               running pod; ConditionStatus:False
    Observed Generation:   1
    Status:                False
    Type:                  RunningPod
    Last Transition Time:  2025-08-25T08:52:00Z
    Message:               get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1
    Observed Generation:   1
    Status:                True
    Type:                  GetPod--clickhouse-prod-appscode-cluster-shard-0-1
    Last Transition Time:  2025-08-25T08:52:00Z
    Message:               evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1
    Observed Generation:   1
    Status:                True
    Type:                  EvictPod--clickhouse-prod-appscode-cluster-shard-0-1
    Last Transition Time:  2025-08-25T08:52:40Z
    Message:               get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0
    Observed Generation:   1
    Status:                True
    Type:                  GetPod--clickhouse-prod-appscode-cluster-shard-1-0
    Last Transition Time:  2025-08-25T08:52:40Z
    Message:               evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0
    Observed Generation:   1
    Status:                True
    Type:                  EvictPod--clickhouse-prod-appscode-cluster-shard-1-0
    Last Transition Time:  2025-08-25T08:53:20Z
    Message:               get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1
    Observed Generation:   1
    Status:                True
    Type:                  GetPod--clickhouse-prod-appscode-cluster-shard-1-1
    Last Transition Time:  2025-08-25T08:53:20Z
    Message:               evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1
    Observed Generation:   1
    Status:                True
    Type:                  EvictPod--clickhouse-prod-appscode-cluster-shard-1-1
    Last Transition Time:  2025-08-25T08:53:40Z
    Message:               Controller has successfully restart the ClickHouse replicas
    Observed Generation:   1
    Reason:                Successful
    Status:                True
    Type:                  Successful
  Observed Generation:     1
  Phase:                   Successful
Events:
  Type     Reason                                                                               Age    From                         Message
  ----     ------                                                                               ----   ----                         -------
  Normal   Starting                                                                             9m15s  KubeDB Ops-manager Operator  Start processing for ClickHouseOpsRequest: demo/clickhouse-restart
  Normal   Starting                                                                             9m15s  KubeDB Ops-manager Operator  Pausing ClickHouse databse: demo/clickhouse-prod
  Normal   Successful                                                                           9m15s  KubeDB Ops-manager Operator  Successfully paused ClickHouse database: demo/clickhouse-prod for ClickHouseOpsRequest: clickhouse-restart
  Warning  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0    9m7s   KubeDB Ops-manager Operator  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0
  Warning  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0  9m7s   KubeDB Ops-manager Operator  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-0
  Warning  running pod; ConditionStatus:False                                                   9m2s   KubeDB Ops-manager Operator  running pod; ConditionStatus:False
  Warning  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1    8m42s  KubeDB Ops-manager Operator  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1
  Warning  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1  8m42s  KubeDB Ops-manager Operator  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-0-1
  Warning  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0    8m2s   KubeDB Ops-manager Operator  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0
  Warning  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0  8m2s   KubeDB Ops-manager Operator  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-0
  Warning  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1    7m22s  KubeDB Ops-manager Operator  get pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1
  Warning  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1  7m22s  KubeDB Ops-manager Operator  evict pod; ConditionStatus:True; PodName:clickhouse-prod-appscode-cluster-shard-1-1
  Normal   RestartPods                                                                          7m2s   KubeDB Ops-manager Operator  Successfully Restarted ClickHouse pods
  Normal   Starting                                                                             7m2s   KubeDB Ops-manager Operator  Resuming ClickHouse database: demo/clickhouse-prod
  Normal   Successful                                                                           7m2s   KubeDB Ops-manager Operator  Successfully resumed ClickHouse database: demo/clickhouse-prod for ClickHouseOpsRequest: clickhouse-restart

Cleaning up

To cleanup the Kubernetes resources created by this tutorial, run:

kubectl delete clickhouseopsrequest -n demo clickhouse-restart
kubectl delete clickhouse -n demo clickhouse-prod
kubectl delete ns demo

Next Steps