对于Kubernetes的service,不管是cluster-ip和nodeport均是四层的负载,集群内的服务如何实现七层的负载均衡,这就须要借助于ingress,ingress控制器实现的方式有不少,好比nginx,contour,haproxy,trafik,lstio。几种经常使用的ingress功能对比和选型能够参考这里www.kubernetes.org.cn/5948.htmlhtml
ingress-nginx是七层的负载均衡器,负责统一管理外部对k8s cluster中的service的请求。主要包含前端
总结用ingress好处:
一、同台服务器不一样业务不须要再给每一个业务映射端口(Nodeport),只须要每台机安装一个ingress,利用ingress反代CluserIP,前端机访问Ingress固定端口
二、添加新业务只须要再建立一个ingress反代新业务的service,再去前端Nginx反代配置servername里面添加一个域名便可以访问新业务,经过不一样的域名访问不一样的业务,不须要再配反代node
hdss7-200机主机上操做nginx
docker pull traefik:v1.7-alpine docker tag c36f69007d98 harbor.od.com/k8s/traefik:v1.7 docker push harbor.od.com/k8s/traefik:v1.7
清单下载地址:https://github.com/traefik/traefik/tree/v1.7/examples/k8sgit
mkdir -p /data/k8s-yaml/traefik && cd /data/k8s-yaml/traefik
cat > /data/k8s-yaml/traefik/rbac.yaml <<'eof' apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system eof
cat > /data/k8s-yaml/traefik/daemonset.yaml <<'eof' apiVersion: apps/v1 kind: DaemonSet metadata: name: traefik-ingress-controller namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 containers: - image: harbor.od.com/k8s/traefik:v1.7 name: traefik-ingress-lb ports: - name: http containerPort: 80 hostPort: 81 securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE args: - --api - --kubernetes - --logLevel=INFO - --insecureskipverify=true - --kubernetes.endpoint=https://10.4.7.10:7443 - --accesslog - --accesslog.filepath=/var/log/traefik_access.log - --traefiklog - --traefiklog.filepath=/var/log/traefik.log - --metrics.prometheus imagePullSecrets: - name: harbor eof
hostPort: 81 为ingress的程序80端口映射到宿主机供提供访问的端口github
kubectl apply -f http://k8s-yaml.od.com/traefik/rbac.yaml kubectl apply -f http://k8s-yaml.od.com/traefik/daemonset.yaml
mkdir /data/k8s-yaml/nginxtest
cat > /data/k8s-yaml/nginxtest/ingress.yml <<'eof' apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-web namespace: default annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: nginxtest.od.com http: paths: - path: / backend: serviceName: nginx-test servicePort: 80 eof
主机名为nginxtest.od.com,反代到svc的name为nginx-test,路径为/,端口80web
cat > /data/k8s-yaml/nginxtest/svc.yml <<'eof' apiVersion: v1 kind: Service metadata: labels: k8s-app: nginx-test name: nginx-test namespace: default spec: ports: - port: 80 protocol: TCP selector: app: nginx-test sessionAffinity: None eof
svc标签选择器app: nginx-test,反代pod为app:nginx-testdocker
cat > /data/k8s-yaml/nginxtest/deploy.yml <<'eof' apiVersion: apps/v1 kind: Deployment metadata: name: nginx-test labels: app: nginx-test spec: replicas: 2 selector: matchLabels: app: nginx-test template: metadata: labels: app: nginx-test spec: containers: - name: nginx-test image: harbor.od.com/public/nginx:v1.7.9 ports: - name: web containerPort: 80
hdss7-11.host.com上操做shell
cat >> /var/named/od.com.zone <<'eof' nginxtest A 10.4.7.10 eof vi /var/named/od.com.zone 2020100504 ; serial # 日期加1 systemctl restart named
在hdss7-11.host.com和hdss7-12.host.com上操做api
cat >/etc/nginx/conf.d/nginxtest.com.conf <<'eof' upstream default_backend_traefik { server 10.4.7.21:81 max_fails=3 fail_timeout=10s; server 10.4.7.22:81 max_fails=3 fail_timeout=10s; } server { server_name nginxtest.od.com; location / { proxy_pass http://default_backend_traefik; proxy_set_header Host $http_host; proxy_set_header x-forwarded-for $proxy_add_x_forwarded_for; } } eof nginx -s reload
如下都在hdss7-21.host.com或hdss7-22上操做
[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/deploy.yml [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/svc.yml [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/ingress.yml [root@hdss7-22 ~]# kubectl get ing Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress NAME CLASS HOSTS ADDRESS PORTS AGE nginx-web <none> nginxtest.od.com 80 18h [root@hdss7-22 ~]# kubectl get pods -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES coredns-57c78bdbcd-lsf5z 1/1 Running 4 30h 172.7.21.3 hdss7-21.host.com <none> <none> traefik-ingress-controller-9n8zb 1/1 Running 0 11h 172.7.21.5 hdss7-21.host.com <none> <none> traefik-ingress-controller-wxnqw 1/1 Running 0 11h 172.7.22.4 hdss7-22.host.com <none> <none>
[root@hdss7-22 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-test-558df79dc9-d95rp 1/1 Running 0 9h 172.7.21.2 hdss7-21.host.com <none> <none> nginx-test-558df79dc9-qw2fj 1/1 Running 0 9h 172.7.22.2 hdss7-22.host.com <none> <none> [root@hdss7-22 ~]# kubectl exec -it nginx-test-558df79dc9-d95rp -- /bin/bash root@nginx-test-558df79dc9-d95rp:/# echo WEB1 > /usr/share/nginx/html/index.html root@nginx-test-558df79dc9-d95rp:/# exit exit [root@hdss7-22 ~]# kubectl exec -it nginx-test-558df79dc9-qw2fj -- /bin/bash root@nginx-test-558df79dc9-qw2fj:/# echo WEB2 > /usr/share/nginx/html/index.html