PersistentVolume(PV)是指由集群管理员配置提供的某存储系统上的段存储空间,它是对底层共享存储的抽象,将共享存储做为种可由用户申请使的资源,实现了“存储消费”机制。经过存储插件机制,PV支持使用多种网络存储系统或云端存储等多种后端存储系统,例如,NFS、RBD和Cinder等。PV是集群级别的资源,不属于任何名称空间,用户对PV资源的使须要经过PersistentVolumeClaim(PVC)提出的使申请(或称为声明)来完成绑定,是PV资源的消费者,它向PV申请特定大小的空间及访问模式(如rw或ro),从建立出PVC存储卷,后再由Pod资源经过PersistentVolumeClaim存储卷关联使,以下图:
html
尽管PVC使得用户能够以抽象的方式访问存储资源,但不少时候仍是会涉及PV的很多属性,例如,因为不一样场景时设置的性能参数等。为此,集群管理员不得不经过多种方式提供多种不一样的PV以满不一样用户不一样的使用需求,二者衔接上的误差必然会致使用户的需求没法所有及时有效地获得知足。Kubernetes从1.4版起引入了一个新的资源对象StorageClass,可用于将存储资源定义为具备显著特性的类(Class)而不是具体的PV,例如“fast”“slow”或“glod”“silver”“bronze”等。用户经过PVC直接向意向的类别发出申请,匹配由管理员事先建立的PV,或者由其按需为用户动态建立PV,这样作甚至免去了须要先建立PV的过程。
PV对存储系统的支持可经过其插件来实现,目前,Kubernetes支持以下类型的插件。
官方地址:https://kubernetes.io/docs/concepts/storage/storage-classes/
nginx
由上图咱们能够看到官方插件是不支持NFS动态供给的,可是咱们能够用第三方的插件来实现,下面就是本文要讲的。git
GitHub地址:https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploygithub
一、下载所需文件web
for file in class.yaml deployment.yaml rbac.yaml ; do wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/$file ; done
二、建立RBAC受权后端
# cat rbac.yaml kind: ServiceAccount apiVersion: v1 metadata: name: nfs-client-provisioner --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["create", "update", "patch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner namespace: default roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner rules: - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io
三、建立Storageclass类api
# cat class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: managed-nfs-storage provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME' parameters: archiveOnDelete: "false"
四、建立NFS的deployment,修改相应的NFS服务器IP及挂载路径服务器
# cat deployment.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner --- kind: Deployment apiVersion: apps/v1 metadata: name: nfs-client-provisioner spec: replicas: 1 selector: matchLabels: app: nfs-client-provisioner strategy: type: Recreate template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: quay.io/external_storage/nfs-client-provisioner:v2.0.0 volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.1.100 - name: NFS_PATH value: /huoban/k8s volumes: - name: nfs-client-root nfs: server: 192.168.1.100 path: /huoban/k8s
下面是一个StatefulSet应用动态申请PV的示意图:网络
例如:建立一个nginx动态获取PVapp
# cat nginx.yaml --- apiVersion: v1 kind: Service metadata: name: nginx labels: app: nginx spec: ports: - port: 80 name: web clusterIP: None selector: app: nginx --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: selector: matchLabels: app: nginx serviceName: "nginx" replicas: 3 template: metadata: labels: app: nginx spec: imagePullSecrets: - name: huoban-harbor terminationGracePeriodSeconds: 10 containers: - name: nginx image: harbor.huoban.com/open/huoban-nginx:v1.1 ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-nfs-storage" resources: requests: storage: 1Gi
启动以后咱们能够看到一下信息
# kubectl get pod,pv,pvc NAME READY STATUS RESTARTS AGE pod/nfs-client-provisioner-fcb58977d-l5cs4 1/1 Running 0 20h pod/web-0 1/1 Running 0 175m pod/web-1 1/1 Running 0 175m pod/web-2 1/1 Running 0 175m NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/default-test-claim-pvc-e5a66781-b46e-4191-8f51-5d1a571ca530 1Mi RWX Delete Bound default/test-claim managed-nfs-storage 20h persistentvolume/default-www-web-0-pvc-0a578ef2-63e3-49bb-87c0-88166d3e0e65 1Gi RWO Delete Bound default/www-web-0 managed-nfs-storage 20h persistentvolume/default-www-web-1-pvc-78061eb6-c36b-44db-9472-f2684f85a4b9 1Gi RWO Delete Bound default/www-web-1 managed-nfs-storage 20h persistentvolume/default-www-web-2-pvc-ec760344-a35a-4048-b8aa-6452d6a62337 1Gi RWO Delete Bound default/www-web-2 managed-nfs-storage 20h NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/test-claim Bound default-test-claim-pvc-e5a66781-b46e-4191-8f51-5d1a571ca530 1Mi RWX managed-nfs-storage 20h persistentvolumeclaim/www-web-0 Bound default-www-web-0-pvc-0a578ef2-63e3-49bb-87c0-88166d3e0e65 1Gi RWO managed-nfs-storage 20h persistentvolumeclaim/www-web-1 Bound default-www-web-1-pvc-78061eb6-c36b-44db-9472-f2684f85a4b9 1Gi RWO managed-nfs-storage 20h persistentvolumeclaim/www-web-2 Bound default-www-web-2-pvc-ec760344-a35a-4048-b8aa-6452d6a62337 1Gi RWO managed-nfs-storage 20h
如今,咱们在NFS服务器上也能够看到自动生成了3个挂载目录,单pod删除以后数据还会存在
# ll drwxrwxrwx 2 root root 4096 Oct 23 17:31 default-www-web-0-pvc-0a578ef2-63e3-49bb-87c0-88166d3e0e65 drwxrwxrwx 2 root root 4096 Oct 23 17:31 default-www-web-1-pvc-78061eb6-c36b-44db-9472-f2684f85a4b9 drwxrwxrwx 2 root root 4096 Oct 23 17:40 default-www-web-2-pvc-ec760344-a35a-4048-b8aa-6452d6a62337
StatefulSet应用有如下特色:
1.惟一的网络标识
2.域名访问(<statefulsetName-index>.<service-name>.svc.cluster.local) 如:web-0.nginx.default.svc.cluster.local
3.独立的持久存储
4.有序的部署和删除