KubeVirt 04：容器化数据导入 – 小菜园

COntainerd Data Importer(CDI) 是能够将多种来源的数据导入到 Kubernetes 中的 PVC，随后用做 VM 的磁盘。

所以 CDI 会创建相关的 PVC，并且使用 PV，建议在 Kubernetes 中部署 Rook-Ceph，提供持久的块存储，并且后续还需通过 CSI 实现对 VM 的快照和克隆。

Rook-Ceph 的部署和使用可参考：

使用以下命令安装最新版本的 CDI 通过 Operator：

```bash
vagrant@master01:~$ export VERSION=$(basename $(curl -s -w %{redirect_url} https://github.com/kubevirt/containerized-data-importer/releases/latest))
vagrant@master01:~$ echo $VERSION
v1.59.0

vagrant@master01:~$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml

namespace/cdi created
customresourcedefinition.apiextensions.k8s.io/cdis.cdi.kubevirt.io created
clusterrole.rbac.authorization.k8s.io/cdi-operator-cluster created
clusterrolebinding.rbac.authorization.k8s.io/cdi-operator created
serviceaccount/cdi-operator created
role.rbac.authorization.k8s.io/cdi-operator created
rolebinding.rbac.authorization.k8s.io/cdi-operator created
deployment.apps/cdi-operator created

vagrant@master01:~$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml
cdi.cdi.kubevirt.io/cdi created
```

等待 Operator 部署完成，然后检查状态：

```bash
vagrant@master01:~$ kubectl get cdi cdi -n cdi
NAME   AGE   PHASE
cdi    52s   Deployed
vagrant@master01:~$ kubectl get pod -n cdi
NAME                               READY   STATUS    RESTARTS   AGE
cdi-apiserver-75bb86c8c4-84m5j     1/1     Running   0          51s
cdi-deployment-645dc8bc99-vj4fc    1/1     Running   0          51s
cdi-operator-67d655447d-zhpb7      1/1     Running   0          67s
cdi-uploadproxy-67f586465b-fjntg   1/1     Running   0          51s
```

部署完成后，会引进 `DataVolumes` ，通过创建该资源，可以抽象的创建 PVC。

通过声明 DadaVolume 资源定义，来创建 PVC，并将一个 Fedora 的映像导入到该 PVC 中：

```bash
cat <<EOF > dv_fedora.yml
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: "fedora"
spec:
  storage:
    resources:
      requests:
        storage: 5Gi
  source:
    http:
      url: "https://download.fedoraproject.org/pub/fedora/linux/releases/37/Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.raw.xz"
EOF

kubectl -n cdi-demo create -f dv_fedora.yml
```

这里的 `source` 用的是 `http` ,除此之外还可以使用 `pvc` 、`registry` 、`s3` 等，具体可以通过 `kubectl explain DataVolume.spec.source` 来进行查看。

然后，它就报错了。

```bash
vagrant@master01:~$ kubectl get pod -n cdi-demo
NAME                                                  READY   STATUS             RESTARTS      AGE
importer-prime-4d87fc83-0832-42df-bd2c-f970098b75ac   0/1     CrashLoopBackOff   4 (45s ago)   2m35s

vagrant@master01:~$ kubectl logs importer-prime-4d87fc83-0832-42df-bd2c-f970098b75ac -n cdi-demo
I0428 04:53:21.137080       1 importer.go:103] Starting importer
E0428 04:53:21.138511       1 importer.go:133] exit status 1, blockdev: cannot open /dev/cdi-block-volume: Permission denied

kubevirt.io/containerized-data-importer/pkg/util.GetAvailableSpaceBlock
    pkg/util/util.go:130
kubevirt.io/containerized-data-importer/pkg/util.GetAvailableSpaceByVolumeMode
    pkg/util/util.go:100
main.main
    cmd/cdi-importer/importer.go:131
runtime.main
    GOROOT/src/runtime/proc.go:267
```

查看 PVC 和 DataVolume 的状态：

```bash
vagrant@master01:~$ kubectl get pvc -n cdi-test
NAME                                         STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      VOLUMEATTRIBUTESCLASS   AGE
fedora                                       Pending                                                                        rook-ceph-block   <unset>                 3m27s
prime-ac1aca3c-15cc-4c8b-ab77-3ef2cb044846   Bound     pvc-e995b1ec-c7eb-41ed-9fae-7f6ce7a5ebdd   5Gi        RWX            rook-ceph-block   <unset>

vagrant@master01:~$ kubectl get datavolume -n cdi-test
NAME     PHASE              PROGRESS   RESTARTS   AGE
fedora   ImportInProgress   N/A        7          16m
```

经过查找资料，解决方法参考 [Non-root Containers And Devices](https://kubernetes.io/blog/2021/11/09/non-root-containers-and-devices/)

如果使用的 Containerd ，修改配置中的：

```toml
[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    device_ownership_from_security_context = true
```

如果使用的是 CRI-O，修改配置中的：

```toml
[crio.runtime]
device_ownership_from_security_context = true
```

根据使用的 Container Runtime 修改不同的配置文件，然后重启，就正常了。

查看状态：

```bash
vagrant@master01:~$ kubectl get datavolume -n cdi-test
NAME     PHASE              PROGRESS   RESTARTS   AGE
fedora   ImportInProgress   86.27%     11         35m
```

等待导入成功后，查看资源状态：

```bash
vagrant@master01:~$ kubectl get DataVolume,pvc,pod -n cdi-test
NAME                                PHASE       PROGRESS   RESTARTS   AGE
datavolume.cdi.kubevirt.io/fedora   Succeeded   100.0%     11         46m

NAME                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/fedora   Bound    pvc-e995b1ec-c7eb-41ed-9fae-7f6ce7a5ebdd   5Gi        RWX            rook-ceph-block   <unset>                 46m
```

执行导入的 Pod 已经被移除了。

使用以下的 VM 定义创建 fedora-vm，注意将 ssh 公钥换成自己的：

```bash
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/os: linux
  name: fedora-vm
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/domain: fedora-vm1
    spec:
      domain:
        cpu:
          cores: 1
        devices:
          disks:
          - disk:
              bus: virtio
            name: disk0
          - cdrom:
              bus: sata
              readonly: true
            name: cloudinitdisk
        machine:
          type: q35
        resources:
          requests:
            memory: 512M
      volumes:
      - name: disk0
        persistentVolumeClaim:
          claimName: fedora
      - cloudInitNoCloud:
          userData: |
            #cloud-config
            hostname: fedora-vm1
            ssh_pwauth: True
            disable_root: false
            ssh_authorized_keys:
            - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxl86q6k6zEKHLmnNtMKZcgCTweRAm7ChsXTi0s7+40L4rCrRqB6Sq/D9e3tXc7vV3Mcqf0G6oNhh2Y0ZlYHu6ADCfj8Dw8fQscsflypDj0B9EXIb3j2Eetp0jI73H/tOe8X7dQGgmTG6bjJ2NmnQL/uAJpiGnGzAe06E4CrK5Q9plOD00E3gKbMhJbxZBOKY6rHfEPhVEhFC8jXjCwTshtRSpF8EA947oPzeAqun9WO5ffwewvBo1/FfyU9OOvDYXj0qvAYM0pUVlHtSts3vsvO3HaxkFSpyQxxT9StnIsD9dtY9UanaRQu0b3W7j+7xv+TqHHVTvAi8D25a/or3w3bpXwor488Qok+IhxE36LJiARoJ++4v7G5VYZSCkuPsmOPNUoMXJgksy9TAtQkU1PL7dF0v3s/4+BJ6BQ9s2BQtLhRXatIBiXU/Hxfhp7OewWt3vmlsy0qKKN6oEo6VUxMYjPTmlCaPX9YXdKwgAQv9/+Fmgadvpxha+R4qB4fk= vagrant@master01
        name: cloudinitdisk
```

创建 vm：

```bash
vagrant@master01:~$ kubectl -n cdi-test apply -f fedora-vm1_pvc.yml 
virtualmachine.kubevirt.io/fedora-vm created
```

查看状态：

```bash
vagrant@master01:~$ kubectl get vmi -n cdi-test
NAME        AGE   PHASE     IP               NODENAME   READY
fedora-vm   28s   Running   10.244.241.115   master01   True
```

连接到虚拟机：

```bash
vagrant@master01:~$ virtctl ssh -i .ssh/id_rsa fedora@fedora-vm.cdi-test
The authenticity of host 'vmi/fedora-vm.cdi-test:22 (192.168.121.1:7890)' can't be established.
ECDSA key fingerprint is SHA256:qmiiDH3244ZW1cfwYav30En+1CoLOR/dSY3ixx5Ul0Y.
Are you sure you want to continue connecting (yes/no)? yes
[fedora@fedora-vm ~]$ lsblk 
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sr0     11:0    1    1M  0 rom  
zram0  251:0    0  435M  0 disk [SWAP]
vda    252:0    0    5G  0 disk 
├─vda1 252:1    0    1M  0 part 
├─vda2 252:2    0 1000M  0 part /boot
├─vda3 252:3    0  100M  0 part /boot/efi
├─vda4 252:4    0    4M  0 part 
└─vda5 252:5    0  3.9G  0 part /home
                                /
```

VM 可以正常使用 CDI 导入的 PVC 用作根。