weixin_39838231
weixin_39838231
2020-11-27 10:25

Our Labels & selectors replaced by vault-operator is breaking our Vault services

Describe the bug: If the vault-operator has restarted/redeployed, the controller itself will replace our Vault service's labels & selectors from app: name to app.kubernetes.io/name: vault. This breaks the Vault service (and we have three that are production)

Expected behaviour: I expect the service to stay up, but instead get a 503 every time the vault-operator overwrites the labels and selectors. We can disable the vault-operator to prevent these labels and selectors from being updated.

Steps to reproduce the bug: Unable to reproduce, but I can provide our vault instance yaml, (or anything else for that matter on request)

Additional context: We have been aware of this issue for about a month now. I believe the issue is that the code is not backwards compatible with the Vault setup we have running.

There could be two possibilities where to fix this in the code, but I wouldn't know where is best.

We could either add the following line withVaultLabels(v, selectorLs), to the selectors value here: https://github.com/banzaicloud/bank-vaults/blob/master/operator/pkg/controller/vault/vault_controller.go#L712, which would match line 708 so that we can control any additional labels/selectors that we want in the vault instance

Or for entire backwards compatibility, we could add app: vault into here: https://github.com/banzaicloud/bank-vaults/blob/master/operator/pkg/apis/vault/v1alpha1/vault_types.go#L663

Environment details: - Kubernetes version (e.g. v1.10.2): v1.16.2 - Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): AWS - bank-vaults version (e.g. 0.4.17): 0.5.1 - Install method (e.g. helm or static manifests): static manifests - Logs from the misbehaving component (and any other relevant logs): n/a - Resource definition (possibly in YAML format) that caused the issue, without sensitive data:

vault.yaml:


apiVersion: vault.banzaicloud.com/v1alpha1
kind: Vault
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"vault.banzaicloud.com/v1alpha1","kind":"Vault","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"vault-build"},"name":"vault","namespace":"default"},"spec":{"annotations":{"iam.amazonaws.com/role":"k8s-vault"},"bankVaultsImage":"banzaicloud/bank-vaults:0.5.1","config":{"api_addr":"https://vault.build.ifdsfs.com","default_lease_ttl":"12h","listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":true}},"max_lease_ttl":"720h","storage":{"dynamodb":{"ha_enabled":"true","read_capacity":5,"region":"eu-west-1","table":"vault","write_capacity":5}},"telemetry":{"statsd_address":"localhost:9125"},"ui":true},"externalConfig":{"auth":[{"type":"userpass"}],"policies":[{"name":"admin","rules":"path \"*\" {\n  capabilities = [\"create\", \"read\", \"update\", \"delete\", \"list\", \"sudo\"]\n}\n"}],"secrets":[{"description":"General secrets.","options":{"version":1},"path":"secret","type":"kv"}]},"image":"vault:1.0.1","serviceAccount":"vault","serviceType":"ClusterIP","size":1,"unsealConfig":{"kubernetes":{"secretNamespace":"default"}}}}
  creationTimestamp: "2019-10-22T11:04:08Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: vault-build
  name: vault
  namespace: default
  resourceVersion: "126215781"
  selfLink: /apis/vault.banzaicloud.com/v1alpha1/namespaces/default/vaults/vault
  uid: ab9412d3-f4bb-11e9-8e19-0ab559d736b4
spec:
  annotations:
    iam.amazonaws.com/role: k8s-vault
  bankVaultsImage: banzaicloud/bank-vaults:0.5.1
  config:
    api_addr: https://vault.example.com
    default_lease_ttl: 12h
    listener:
      tcp:
        address: 0.0.0.0:8200
        tls_disable: true
    max_lease_ttl: 720h
    storage:
      dynamodb:
        ha_enabled: "true"
        read_capacity: 5
        region: eu-west-1
        table: vault
        write_capacity: 5
    telemetry:
      statsd_address: localhost:9125
    ui: true
  credentialsConfig:
    env: ""
    path: ""
    secretName: ""
  envsConfig: null
  etcdSize: 0
  etcdVersion: ""
  externalConfig:
    auth:
    - type: userpass
    policies:
    - name: admin
      rules: |
        path "*" {
          capabilities = ["create", "read", "update", "delete", "list", "sudo"]
        }
    secrets:
    - description: General secrets.
      options:
        version: 1
      path: secret
      type: kv
  fluentdConfig: ""
  fluentdEnabled: false
  fluentdImage: ""
  image: vault:1.0.1
  nodeAffinity: {}
  nodeSelector: null
  podAntiAffinity: ""
  securityContext: {}
  serviceAccount: vault
  servicePorts: null
  serviceType: ClusterIP
  size: 1
  statsdDisabled: false
  statsdImage: ""
  tolerations: null
  unsealConfig:
    kubernetes:
      secretName: ""
      secretNamespace: default
    options: {}
  vaultAnnotations: null
  vaultConfigurerAnnotations: null
  vaultConfigurerLabels: null
  vaultConfigurerPodSpec:
    containers: null
  vaultEnvsConfig: null
  vaultPodSpec:
    containers: null
  watchedSecretsAnnotations: null
  watchedSecretsLabels: null
status:
  leader: vault-0
  nodes:
  - vault-0

/kind bug

该提问来源于开源项目:banzaicloud/bank-vaults

  • 点赞
  • 回答
  • 收藏
  • 复制链接分享

5条回答