Unable to use Vault Webhook with External Vault

Describe the bug: It doesn't work. Logs aren't quite useful.

Expected behaviour: I should see the secret mutate

Steps to reproduce the bug: Deploy the helm chart.

Deploy this rbac change too to avoid the error here: https://github.com/coreos/vault-operator/issues/359

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: vault-tokenreview-binding
  namespace: vswh
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: vswh-vault-secrets-webhook
  namespace: vswh

Additional context: Log Output:

PS C:\> gcloud logging read "labels.k8s-pod/app_kubernetes_io/name=vault-secrets-webhook" --project REDACTED --format json --freshness 15h | jq 'reverse[] | .textPayload ' --raw-output -j
2019/09/20 22:42:55 Failed to request new Vault token Error making API request.

URL: PUT https://vault.borg.dev/v1/auth/kubernetes/traffic/login
Code: 500. Errors:

* {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User \"system:serviceaccount:vswh:vswh-vault-secrets-webhook\" cannot create resource \"tokenreviews\" in API group \"authentication.k8s.io\" at the cluster scope","reason":"Forbidden","details":{"group":"authentication.k8s.io","kind":"tokenreviews"},"code":403}
2019/09/20 22:43:00 Failed to request new Vault token Error making API request.

URL: PUT https://vault.borg.dev/v1/auth/kubernetes/traffic/login
Code: 500. Errors:

* {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User \"system:serviceaccount:vswh:vswh-vault-secrets-webhook\" cannot create resource \"tokenreviews\" in API group \"authentication.k8s.io\" at the cluster scope","reason":"Forbidden","details":{"group":"authentication.k8s.io","kind":"tokenreviews"},"code":403}
2019/09/20 22:43:06 Failed to request new Vault token Error making API request.

URL: PUT https://vault.borg.dev/v1/auth/kubernetes/traffic/login
Code: 500. Errors:

* {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User \"system:serviceaccount:vswh:vswh-vault-secrets-webhook\" cannot create resource \"tokenreviews\" in API group \"authentication.k8s.io\" at the cluster scope","reason":"Forbidden","details":{"group":"authentication.k8s.io","kind":"tokenreviews"},"code":403}
2019/09/20 22:43:12 Failed to request new Vault token Error making API request.

URL: PUT https://vault.borg.dev/v1/auth/kubernetes/traffic/login
Code: 500. Errors:

* {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User \"system:serviceaccount:vswh:vswh-vault-secrets-webhook\" cannot create resource \"tokenreviews\" in ACode: 500. Errors:

* {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User \"system:serviceaccount:vswh:vswh-vault-secrets-webhook\" cannot create resource \"tokenreviews\" in API group \"authentication.k8s.io\" at the cluster scope","reason":"Forbidden","details":{"group":"authentication.k8s.io","kind":"tokenreviews"},"code":403}
2019/09/20 22:43:19 Received new Vault token
2019/09/20 22:43:20 Renewed Vault Token
2019/09/20 22:44:55 Received new Vault token
2019/09/20 22:44:55 Initial Vault token arrived
2019/09/20 22:44:56 Renewed Vault Token
2019/09/20 22:44:56 Vault token renewal closed
time="2019-09-20T22:52:34Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:34Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:34Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:34Z" level=info msg="Listening on https://:8443"
time="2019-09-20T22:52:57Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:57Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:57Z" level=warning msg="no tracer active"
time="2019-09-20T22:52:57Z" level=info msg="Listening on https://:8443"
time="2019-09-21T10:35:54Z" level=warning msg="no tracer active"
time="2019-09-21T10:35:54Z" level=warning msg="no tracer active"
time="2019-09-21T10:35:54Z" level=warning msg="no tracer active"
time="2019-09-21T10:35:54Z" level=info msg="Listening on https://:8443"
time="2019-09-21T10:36:05Z" level=warning msg="no tracer active"
time="2019-09-21T10:36:05Z" level=warning msg="no tracer active"
time="2019-09-21T10:36:05Z" level=warning msg="no tracer active"
time="2019-09-21T10:36:05Z" level=info msg="Listening on https://:8443"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=warning msg="no tracer active"
time="2019-09-21T10:41:07Z" level=info msg="Listening on https://:8443"
time="2019-09-21T10:41:07Z" level=info msg="Listening on https://:8443"
time="2019-09-21T10:41:07Z" level=info msg="Listening on https://:8443"

Environment details: - Kubernetes version: 1.13.7-gke.8 - Platform: GKE Private Cluster - bank-vaults version: 0.5.1 - Install method (e.g. helm or static manifests): Helm - Logs from the misbehaving component (and any other relevant logs): - Resource definition (possibly in YAML format) that caused the issue, without sensitive data:

I want to mutate this secret:

apiVersion: v1
data:
  estafette-key: dmF1bHQ6a3YvcmF4L2djcC1vcHMvI2VzdGFmZXR0ZS1zYV9qc29u
kind: Secret
metadata:
  annotations:
    vault.security.banzaicloud.io/vault-addr: https://vault.borg.dev
    vault.security.banzaicloud.io/vault-path: kubernetes-traffic
    vault.security.banzaicloud.io/vault-role: traffic
  name: sample-secret2
  namespace: vswh
type: generic

Cluster Firewall Rules:

cy:~ (REDACTED)$ gcloud --project REDACTED-vpc compute firewall-rules list     --filter 'name~^gke-traffic'     --format 'table(
        name,
        network,
        direction,
        sourceRanges.list():label=SRC_RANGES,
        allowed[].map().firewall_rule().list():label=ALLOW,W
        targetTags.list():label=TARGET_TAGS
    )'
NAME                         NETWORK   DIRECTION  SRC_RANGES       ALLOW                         TARGET_TAGS
gke-traffic-345c95b5-all     main-net  INGRESS    10.216.128.0/18  tcp,udp,icmp,esp,ah,sctp      gke-traffic-345c95b5-node
gke-traffic-345c95b5-master  main-net  INGRESS    10.250.1.16/28   tcp:10250,tcp:443,tcp:8443    gke-traffic-345c95b5-node
gke-traffic-345c95b5-vms     main-net  INGRESS    10.110.0.0/16    icmp,tcp:1-65535,udp:1-65535  gke-traffic-345c95b5-node

/kind bug

Vault Configuration(deployed via Terraform)

resource "vault_auth_backend" "kubernetes-traffic" {
  type = "kubernetes"
  path = "kubernetes-traffic"
}

resource "vault_kubernetes_auth_backend_config" "example" {
  backend            = vault_auth_backend.kubernetes-traffic.path
  kubernetes_host    = "https://35.228.0.0"
  kubernetes_ca_cert = <<eot certificate----- redacted eot resource backend="vault_auth_backend.kubernetes-traffic.path" role_name="traffic" bound_service_account_names='["vswh-vault-secrets-webhook"]' bound_service_account_namespaces='["*"]' token_ttl="3600" token_policies='["default",'></eot>

该提问来源于开源项目：banzaicloud/bank-vaults