问题一：三级等保,Spring Boot Actuator的漏洞文章 (opens new window)

1.1 通过nginx返回页面解决

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-prod
  namespace: prod
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~ /actuator {
          return 403;
      }          

1.2 生产环境用阿里云ack,需要手动开启snippet注解能力 (opens new window)

---

问题二：反省代理呼叫中心 (opens new window)的api,K8S VERSION:v1.17.9

2.1 通过snippet的注解来实现proxy_pass

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-base
  annotations:
    kubernetes.io/ingress.class: "nginx"
    prometheus.io/http_probe: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"    
    nginx.ingress.kubernetes.io/app-root: /login
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~ /actuator {
          return 403;
      }          
      location ~ /comm {
        proxy_pass http://www.sobot.com;
        proxy_set_header Host 'www.sobot.com';
      }        
    nginx.ingress.kubernetes.io/use-regex: "true" # 启用了正则表达式支持
spec:
  backend:
    serviceName: gateway
    servicePort: 11001
  rules:
  - host: xxx.com  
    http:
      paths:
      - path: /base/                                                                                        
        backend:
          serviceName: web-base                                                                                       
          servicePort: 80

2.2 查看nginx-ingress-controller配置文件是否生效

[root@k8s-master ~]# kubectl exec -it -n ingress-nginx nginx-ingress-controller-996cdf458-2sx99 -- /bin/bash
www-data@k8s-node01:/etc/nginx$ cat nginx.conf|grep -A 10  "comm"
		location ~ /comm {
			proxy_pass http://www.sobot.com;
			proxy_set_header Host 'www.sobot.com';
		}

		if ($uri = /) {
			return 302 /wonder/login;
		}

		location /web-util-mxgraph-dashboard/ {

www-data@k8s-node01:/etc/nginx$

2.3 访问发现,页面都是404,查看nginx-ingress-controller日志发现请求都发到gateway了

[root@k8s-master ~]#  kubectl logs -f -n ingress-nginx nginx-ingress-controller-996cdf458-gl2k9

2.4 查看gateway的日志

[root@k8s-master ~]#  kubectl logs -f  gateway-77f7d486c8-8gbf4

2.5 对比发现是启用了use-regex的annotations,关闭即可正常使用

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-base
  annotations:
    kubernetes.io/ingress.class: "nginx"
    prometheus.io/http_probe: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"    
    nginx.ingress.kubernetes.io/app-root: /login
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~ /actuator {
          return 403;
      }          
      location ~ /comm {
        proxy_pass http://www.sobot.com;
        proxy_set_header Host 'www.sobot.com';
      }        
spec:
  backend:
    serviceName: gateway
    servicePort: 11001
  rules:
  - host: xxx.com  
    http:
      paths:
      - path: /base/                                                                                        
        backend:
          serviceName: web-base                                                                                       
          servicePort: 80

---

问题三：接口模糊匹配导致404

本以为本次需求到此结束,没想到...周五下班电话响个不停,https://xxx.com/configure2/v2/mgnt/mxgraph-data/command接口之前正常,现在访问是404,其次会跳转到智齿官网。

3.1 回忆了下最近变动,只加了server-snippet。仔细观察发现该接口是commxxx开头的会模糊匹配从而走到proxy_pass,修改配置如下

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-base
  annotations:
    kubernetes.io/ingress.class: "nginx"
    prometheus.io/http_probe: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"    
    nginx.ingress.kubernetes.io/app-root: /login
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~ /actuator {
          return 403;
      }          
      location ~ /comm/ {          # 匹配以 /comm/ 开头的请求
        proxy_pass http://www.sobot.com;
        proxy_set_header Host 'www.sobot.com';
      }        
spec:
  backend:
    serviceName: gateway
    servicePort: 11001
  rules:
  - host: xxx.com  
    http:
      paths:
      - path: /base/                                                                                        
        backend:
          serviceName: web-base                                                                                       
          servicePort: 80

---

问题四：nginx-ingress-controller循环重启

再续前缘,周一早餐正常上班,发现nginx-ingress-controller的pod重启了,那首当其冲的当然是先倒杯水去。

4.1 nginx-ingress-controller一直循环重启,心里简直🦙,大早上刚来就...,真是天生🐮🐴,紧接着开发反馈开发环境的登陆页404,项目测试环境反馈页面异常返回502(以为是后端的问题,看下请求没到前端,内心os:🦙),着手排查nginx-ingress-controller

[root@k8s-master ~]#  kubectl logs -f -n ingress-nginx nginx-ingress-controller-996cdf458-nsnsq

4.2 如图日志,错误信息指出，在正则表达式定义的位置proxy_pass 指令不能包含 URI 部分,改改改...

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-base
  annotations:
    kubernetes.io/ingress.class: "nginx"
    prometheus.io/http_probe: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"    
    nginx.ingress.kubernetes.io/app-root: /login
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~ /actuator {
          return 403;
      }          
      location  /comm/ {          # 匹配以 /comm/ 开头的请求
        proxy_pass http://www.sobot.com;
        proxy_set_header Host 'www.sobot.com';
      }        
spec:
  backend:
    serviceName: gateway
    servicePort: 11001
  rules:
  - host: xxx.com  
    http:
      paths:
      - path: /base/                                                                                        
        backend:
          serviceName: web-base                                                                                       
          servicePort: 80 

---

4.3 回顾: 发现本次出现问题的,都是🌹问题三🌹 修改完配置后,前端重新发布的后出现的故障.因为我们发布前端的时候都会重新生成ingress的规则.当有新建删除 ingress、更新证书等操作时，nginx-ingress 会触发 nginx reload。