Kubernetes控制平面组件：Kubelet详解（五）：切换docker运行时为containerd

云原生学习路线导航页（持续更新中）

• kubernetes学习系列快捷链接
  • Kubernetes架构原则和对象设计（一）
  • Kubernetes架构原则和对象设计（二）
  • Kubernetes架构原则和对象设计（三）
  • Kubernetes控制平面组件：etcd（一）
  • Kubernetes控制平面组件：etcd（二）
  • Kubernetes控制平面组件：API Server详解（一）
  • Kubernetes控制平面组件：API Server详解（二）
  • Kubernetes控制平面组件：调度器Scheduler（一）
  • Kubernetes控制平面组件：调度器Scheduler（二）
  • Kubernetes控制平面组件：Kubelet详解（一）：API接口层介绍
  • Kubernetes控制平面组件：Kubelet详解（二）：核心功能层
  • Kubernetes控制平面组件：Kubelet详解（三）：CRI 容器运行时接口层
  • Kubernetes控制平面组件：Kubelet详解（四）：gRPC 与 CRI gRPC实现
  • Kubernetes控制平面组件：Kubelet详解（五）：切换docker运行时为containerd
  • Kubernetes控制平面组件：Kubelet 之 Static 静态 Pod

本文是kubernetes的控制面组件Kubelet系列第五篇文章，本篇详细讲解了如何将集群的容器运行时从docker切换到containerd的全过程

• 希望大家多多 点赞 关注 评论 收藏，作者会更有动力继续编写技术文章

1.Docker 与 containerd 差异

• 在 Kubernetes控制平面组件：Kubelet详解（三）：CRI 容器运行时接口层 中，我们学了不同运行时的差异。
  • 使用 dockershim 作为运行时，kubelet调用dockershim后，需要依次经过dockershim、docker daemon、containerd之后，才能调用到oci runtime，非常厚重，而且docker本身包含很多kubernetes用不到的功能组件（如storage、networking等），由于代码内置在kubernetes中而不得不携带
  • 后来谷歌带头为cr定义了cri规范之后，docker不得不将container和image的部分抽取出来成为独立的containerd组件，直接使用containerd作为运行时，去除了kubelet调用 dockershim和docker daemon 的两层，效率大大提升
• 使用kubeadm安装测试集群时，我们通常会使用docker作为运行时，本文将讲述如何将集群的运行时，从docker切到containerd

2.切换docker为containerd

• 官方文档：https://kubernetes.io/zh-cn/docs/setup/production-environment/container-runtimes/#containerd

2.1.Stop service

• 因为要改他们的配置，所以先把kubelet、docker、containerd都停掉

systemctl stop kubelet
systemctl stop docker
systemctl stop containerd

2.2.Create containerd config folder

2.2.1.生成containerd配置文件

• 创建containerd的配置文件目录，生成 containerd 的默认配置文件并写入系统配置路径
• 提示：如果已有自定义配置，直接运行会丢失原有配置，建议先备份

  sudo mkdir -p /etc/containerd
  containerd config default | sudo tee /etc/containerd/config.toml
  

  • containerd config default
    • 调用 containerd 工具生成默认配置的 TOML 格式内容
    • 输出到标准输出（stdout）
    • |（管道）：将前一个命令的输出传递给后续命令
  • sudo tee /etc/containerd/config.toml
    • tee：同时输出到屏幕 并写入文件
    • /etc/containerd/config.toml：containerd 的主配置文件路径

2.2.2.container配置文件示例

• 生成 containerd 配置文件内容 示例如下：

  disabled_plugins = []
  imports = []
  oom_score = 0
  plugin_dir = ""
  required_plugins = []
  root = "/var/lib/containerd"
  state = "/run/containerd"
  temp = ""
  version = 2[cgroup]path = ""[debug]address = ""format = ""gid = 0level = ""uid = 0[grpc]address = "/run/containerd/containerd.sock"gid = 0max_recv_message_size = 16777216max_send_message_size = 16777216tcp_address = ""tcp_tls_ca = ""tcp_tls_cert = ""tcp_tls_key = ""uid = 0[metrics]address = ""grpc_histogram = false[plugins][plugins."io.containerd.gc.v1.scheduler"]deletion_threshold = 0mutation_threshold = 100pause_threshold = 0.02schedule_delay = "0s"startup_delay = "100ms"[plugins."io.containerd.grpc.v1.cri"]device_ownership_from_security_context = falsedisable_apparmor = falsedisable_cgroup = falsedisable_hugetlb_controller = truedisable_proc_mount = falsedisable_tcp_service = trueenable_selinux = falseenable_tls_streaming = falseenable_unprivileged_icmp = falseenable_unprivileged_ports = falseignore_image_defined_volumes = falsemax_concurrent_downloads = 3max_container_log_line_size = 16384netns_mounts_under_state_dir = falserestrict_oom_score_adj = falsesandbox_image = "k8s.gcr.io/pause:3.5"selinux_category_range = 1024stats_collect_period = 10stream_idle_timeout = "4h0m0s"stream_server_address = "127.0.0.1"stream_server_port = "0"systemd_cgroup = falsetolerate_missing_hugetlb_controller = trueunset_seccomp_profile = ""[plugins."io.containerd.grpc.v1.cri".cni]bin_dir = "/opt/cni/bin"conf_dir = "/etc/cni/net.d"conf_template = ""ip_pref = ""max_conf_num = 1[plugins."io.containerd.grpc.v1.cri".containerd]default_runtime_name = "runc"disable_snapshot_annotations = truediscard_unpacked_layers = falseignore_rdt_not_enabled_errors = falseno_pivot = falsesnapshotter = "overlayfs"[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = ""[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options][plugins."io.containerd.grpc.v1.cri".containerd.runtimes][plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = "io.containerd.runc.v2"[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]BinaryName = ""CriuImagePath = ""CriuPath = ""CriuWorkPath = ""IoGid = 0IoUid = 0NoNewKeyring = falseNoPivotRoot = falseRoot = ""ShimCgroup = ""SystemdCgroup = false[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = ""[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options][plugins."io.containerd.grpc.v1.cri".image_decryption]key_model = "node"[plugins."io.containerd.grpc.v1.cri".registry]config_path = ""[plugins."io.containerd.grpc.v1.cri".registry.auths][plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.headers][plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]tls_cert_file = ""tls_key_file = ""[plugins."io.containerd.internal.v1.opt"]path = "/opt/containerd"[plugins."io.containerd.internal.v1.restart"]interval = "10s"[plugins."io.containerd.internal.v1.tracing"]sampling_ratio = 1.0service_name = "containerd"[plugins."io.containerd.metadata.v1.bolt"]content_sharing_policy = "shared"[plugins."io.containerd.monitor.v1.cgroups"]no_prometheus = false[plugins."io.containerd.runtime.v1.linux"]no_shim = falseruntime = "runc"runtime_root = ""shim = "containerd-shim"shim_debug = false[plugins."io.containerd.runtime.v2.task"]platforms = ["linux/amd64"]sched_core = false[plugins."io.containerd.service.v1.diff-service"]default = ["walking"][plugins."io.containerd.service.v1.tasks-service"]rdt_config_file = ""[plugins."io.containerd.snapshotter.v1.aufs"]root_path = ""[plugins."io.containerd.snapshotter.v1.btrfs"]root_path = ""[plugins."io.containerd.snapshotter.v1.devmapper"]async_remove = falsebase_image_size = ""discard_blocks = falsefs_options = ""fs_type = ""pool_name = ""root_path = ""[plugins."io.containerd.snapshotter.v1.native"]root_path = ""[plugins."io.containerd.snapshotter.v1.overlayfs"]root_path = ""upperdir_label = false[plugins."io.containerd.snapshotter.v1.zfs"]root_path = ""[plugins."io.containerd.tracing.processor.v1.otlp"]endpoint = ""insecure = falseprotocol = ""[proxy_plugins][stream_processors][stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]path = "ctd-decoder"returns = "application/vnd.oci.image.layer.v1.tar"[stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]path = "ctd-decoder"returns = "application/vnd.oci.image.layer.v1.tar+gzip"[timeouts]"io.containerd.timeout.bolt.open" = "0s""io.containerd.timeout.shim.cleanup" = "5s""io.containerd.timeout.shim.load" = "5s""io.containerd.timeout.shim.shutdown" = "3s""io.containerd.timeout.task.state" = "2s"[ttrpc]address = ""gid = 0uid = 0
  

2.3.Update default config

• 上面生成了默认配置/etc/containerd/config.toml，其中有几个值得关注的值

  vi /etc/containerd/config.toml
  sed -i s#k8s.gcr.io/pause:3.5#registry.aliyuncs.com/google_containers/pause:3.5#g /etc/containerd/config.toml
  sed -i s#'SystemdCgroup = false'#'SystemdCgroup = true'#g /etc/containerd/config.toml
  

1. 文件中有个配置sandbox_image，值为 k8s.gcr.io/pause:3.5，即容器sandbox的默认使用镜像，k8s.gcr.io/pause:3.5 国内拉取不到，可以根据需要改成 阿里云镜像 registry.aliyuncs.com/google_containers/pause:3.5
2. 还有个配置 SystemdCgroup，cgroup的driver，默认是false，需要开启

2.4.Edit kubelet config and add extra args

• containerd配置文件设置好了之后，还需要修改kubelet的配置，让kubelet知道接下来运行时将会使用containerd

# 先通过查看kubelet服务，找到kubelet的服务配置文件路径
[root@VM-226-235-tencentos ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node AgentLoaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)Drop-In: /usr/lib/systemd/system/kubelet.service.d└─10-kubeadm.confActive: inactive (dead) since Sat 2025-05-17 23:22:46 CST; 42min agoDocs: https://kubernetes.io/docs/# 编辑配置文件，添加或修改一条 包含 KUBELET_EXTRA_ARGS 的 Environment，表示额外的配置：--container-runtime=remote, --container-runtime-endpoint 为 containerd 的 sock
# 这个socket就是grpc发送的位置
vi /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.5"

2.5.Restart

• 都编辑好了之后，进行重启。发现没有再启动docker，因为运行时里用不到了

systemctl daemon-reload
systemctl restart containerd
systemctl restart kubelet

2.6.Config crictl to set correct endpoint

• 创建crictl的文件，指定 containerd 的 sock

cat <<EOF | sudo tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
EOF

2.7.使用crictl操作容器和镜像

# 获取 pod sandbox 容器
[root@VM-226-235-tencentos ~]# crictl pods
POD ID              CREATED             STATE               NAME                                           NAMESPACE           ATTEMPT             RUNTIME
8f5a4d8d532af       20 minutes ago      Ready               nginx-sts-2                                    default             0                   (default)
f45055e2ebb27       21 minutes ago      Ready               nginx-deployment-585449566-rz58v               default             0                   (default)
606b6cec9e655       21 minutes ago      Ready               my-release-1-etcd-0                            default             0                   (default)