rancher 问题排查

🧙 Questions

☄️ Ideas

查询rancher的容器，进入容器看日志

docker ps | grep rancher/rancher
# f693ac71a645        rancher/rancher:v2.4.4                "entrypoint.sh"          15 months ago       Up 40 hours         0.0.0.0:8080->80/tcp, 0.0.0.0:4443->443/tcp                                                                                                                       rancher
docker logs -f --tail=100 f693ac71a645

通过kubectl查询k8s的进程

kubectl get pods --all-namespaces

查询pod日志，状态为Pending没有日志

kubectl logs -f ispong-container-6d55c7c54c-ls6ng -n ispong-dev

直接重启docker

systemctl restart docker

问题

Failed to pull image "busybox": failed to pull and unpack image "docker.io/library/busybox:latest": failed to resolve reference "docker.io/library/busybox:latest": failed to do request: Head "https://registry-1.docker.io/v2/library/busybox/manifests/latest": dial tcp: lookup registry-1.docker.io on 10.13.172.139:53: server misbehaving

子节点镜像无法拉取

# 将本地镜像推到harbor
nerdctl tag busybox:latest 10.13.196.76:30003/library/busybox:latest
nerdctl push 10.13.196.76:30003/library/busybox:latest

nerdctl tag rancher/local-path-provisioner:v0.0.32 10.13.196.76:30003/library/local-path-provisioner:v0.0.32
nerdctl push 10.13.196.76:30003/library/local-path-provisioner:v0.0.32


# 修改local-path-provisioner环境变量的镜像
kubectl -n local-path-storage set image deployment/local-path-provisioner \
  local-path-provisioner=10.13.196.76:30003/library/local-path-provisioner:v0.0.32
kubectl -n local-path-storage get deployment local-path-provisioner -o jsonpath='{.spec.template.spec.containers[0].image}'

# 修改local-path-provisioner的configMap
kubectl edit configmap local-path-config -n local-path-storage
  containers:
  - name: helper-pod
    image: 10.13.196.76:30003/library/busybox:latest

# 重启 local-path-provisioner的deployment
kubectl -n local-path-storage rollout restart deployment local-path-provisioner
kubectl -n local-path-storage rollout status deployment local-path-provisioner

国产超融合，集群实效， 默认的8472无法访问

kubectl apply -f - <<'EOF'
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-canal
  namespace: kube-system
spec:
  valuesContent: |-
    flannel:
      backendPort: 18472
EOF

<!-- 重启 -->
kubectl -n kube-system rollout restart daemonset/rke2-canal
kubectl -n kube-system rollout status daemonset/rke2-canal \
  --timeout=300s

<!-- 每个节点检测 -->
ip -d link show flannel.1 | grep dstport

<!-- 日志检测 -->
kubectl -n kube-system logs daemonset/rke2-canal \
  -c kube-flannel --tail=100 |
  grep 'VXLAN config'

🔗 Links

• rancher docs