Kubernetes Authentication, Authorization & Admission Control

by Anish

Posted on Tuesday February 12, 2019

---

Introduction

This sample chapter extracted from the book, Kubernetes for DevOps .

Get this book on Just $9 or Ask Author for Discount

---

Multiple steps involved to by the Kubernetes API server, before granting/revoking access for the managed kubernetes resources. it's start with

TLS Security

Kubernetes cluster, the API serves on port 443. The API server presents a certificate. This certificate is often self-signed, so $USER/.kube/config on the user's machine typically contains the root certificate for the API server's certificate, which when specified is used in place of the system default root certificate

As shown in the below example the API server is running on the port 6443

[email protected]:# kubectl cluster-info
Kubernetes master is running at https://172.16.2.13:6443
KubeDNS is running at https://172.16.2.13:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

by default the kubernetes admin X.509 CA certificate, Private Key and Certificate revocation list is present in the in the KUBE_HOME=/etc/kubernetes/ directory

[email protected]:# ls -l /etc/kubernetes/pki/ca.*
-rw-r--r-- 1 root root 1025 Jan 23 11:14 /etc/kubernetes/pki/ca.crt
-rw------- 1 root root 1679 Jan 23 11:14 /etc/kubernetes/pki/ca.key
-rw-r--r-- 1 root root   17 Feb  7 13:15 /etc/kubernetes/pki/ca.srl

Authentication

Once you have located the API server, next step is to perform the Authentication, there are two kind of resources which access the kubernetes API server

• Service Account
• Normal User

User and serviceAccount can belongs to one or more groups, groups are designed to grant permission to several users at once, there are reserved built-in group in the kube-system namespace

• The system:unauthenticated group is used for requests where none of the authentication plugins could authenticate the client.

• The system:authenticated group is automatically assigned to a user who was authenticated successfully.

• The system:serviceaccounts group encompasses all ServiceAccounts in the system.

• The system:serviceaccounts:<namespace> includes all ServiceAccounts in a specific namespace.

ServiceAccounts

ServiceAccounts are kubernetes managed resources and are scope to individual namespaces. for example let's GET all the sa available in the kube-system.

[email protected]:# kubectl get sa -n kube-system
NAME                                 SECRETS   AGE
attachdetach-controller              1         20d
bootstrap-signer                     1         20d
certificate-controller               1         20d
clusterrole-aggregation-controller   1         20d
coredns                              1         20d
cronjob-controller                   1         20d
daemon-set-controller                1         20d
default                              1         20d
..............
..............

1. Creating ServiceAccounts

Let's create a sa under the namespace dev

[email protected]:# kubectl create sa test -n dev
serviceaccount/test created

List out the sa under the namespace dev

[email protected]:# kubectl get sa -n dev 
NAME      SECRETS   AGE
default   1         4d
test      1         17m

you might notice a default ServiceAccounts is already exists. Each namespaces have a default servicesccount

2. Inspect ServiceAccount

kubectl describe the sa test under the namespace dev

[email protected]:# kubectl describe sa test -n dev 
Name:                test
Namespace:           dev
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   test-token-hvbtq
Tokens:              test-token-hvbtq
Events:              <none>

• Mountable secrets: Pods using this ServiceAccount can only mount these Secrets i.e test-token-hvbtq
• Tokens: : JWT Authentication token i.e test-token-hvbtq
• Image pull secrets : Credentials for pulling container images from a private image repository not set.

3 . Debugging Secrets

View the secrets of the test-token-hvbtq

[email protected]:# kubectl describe secrets test-token-hvbtq -n dev 
Name:         test-token-hvbtq
Namespace:    dev
Labels:       <none>
Annotations:  kubernetes.io/service-account.name=test
              kubernetes.io/service-account.uid=8d0fcb37-2e8a-11e9-8c68-fa163e589bc0

Type:  kubernetes.io/service-account-token

Data
====
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZXYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoidGVzdC10b2tlbi1odmJ0cSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJ0ZXN0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiOGQwZmNiMzctMmU4YS0xMWU5LThjNjgtZmExNjNlNTg5YmMwIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRldjp0ZXN0In0.caDSrXwkrIGXboIQ5Ag9G-2lp0Ltli5bb8V9O8a0rbVvDJChukQbAXko1pVqKA7eNlT-qOBRl1K6CKAZHiYvDEfhhCv68GF6YncDlY-1eaUdNNO-CT6d2DHXEWb6gSGK-3P1dxRqqJjVE4FBsMfLDfnky203DMuToz6BsnxgUJY8aQOl-3z8AFJJOV2-c4i5da2wKQfb2meGVQkTI_bDSdp-aq0PP9si5UXKKsCzeOBbpuEO1Xk66ggf9cIoREbJujy9zJN_QMcdYS3o1lOuIThAIRpkDid4y1VH_ulKMnt7JBrTyOxeFK1UobC2kZbP2LFCzoNIBSXvypEVlrR-hw
ca.crt:     1025 bytes
namespace:  3 bytes

The token value is JSON Web Token, the decoded format.

//Header
{
  "alg": "RS256",
  "kid": ""
}
//Payload
{
  "iss": "kubernetes/serviceaccount",
  "kubernetes.io/serviceaccount/namespace": "dev",
  "kubernetes.io/serviceaccount/secret.name": "test-token-hvbtq",
  "kubernetes.io/serviceaccount/service-account.name": "test",
  "kubernetes.io/serviceaccount/service-account.uid": "8d0fcb37-2e8a-11e9-8c68-fa163e589bc0",
  "sub": "system:serviceaccount:dev:test"
}

4. Assigning ServiceAccount to Pod

The YAML file for assigning serviceAccountName to test and launching a alpine pod with curl installed

[email protected]:# cat alpine-curl.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: alpine
spec:
  serviceAccountName: test
  containers:
  - image: byrnedo/alpine-curl
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

• Creating POD from the given YAML definition

[email protected]:# kubectl create -f alpine-curl.yaml -n dev 
pod/alpine created

• Verifying pods are in the RUNNING state

[email protected]:# kubectl get pods -n dev
NAME      READY     STATUS    RESTARTS   AGE
alpine    1/1       Running   0          48s

• Exec to the alpine pod

[email protected]:/home/ansible# kubectl exec alpine  -it sh -n dev 
/ # 

• Locate the Mountable secretes which is present in the /var/run/secrets/kubernetes.io/serviceaccount/token

# cat /var/run/secrets/kubernetes.io/serviceaccount/token 
eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZXYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoidGVzdC10b2tlbi1odmJ0cSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJ0ZXN0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiOGQwZmNiMzctMmU4YS0xMWU5LThjNjgtZmExNjNlNTg5YmMwIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRldjp0ZXN0In0.caDSrXwkrIGXboIQ5Ag9G-2lp0Ltli5bb8V9O8a0rbVvDJChukQbAXko1pVqKA7eNlT-qOBRl1K6CKAZHiYvDEfhhCv68GF6YncDlY-1eaUdNNO-CT6d2DHXEWb6gSGK-3P1dxRqqJjVE4FBsMfLDfnky203DMuToz6BsnxgUJY8aQOl-3z8AFJJOV2-c4i5da2wKQfb2meGVQkTI_bDSdp-aq0PP9si5UXKKsCzeOBbpuEO1Xk66ggf9cIoREbJujy9zJN_QMcdYS3o1lOuIThAIRpkDid4y1VH_ulKMnt7JBrTyOxeFK1UobC2kZbP2LFCzoNIBSXvypEVlrR-hw/

• Query API server with the --header "Authorization: Bearer $TOKEN"

# export TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
# curl https://172.16.2.13:6443/api --header "Authorization: Bearer $TOKEN" --insecure
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "172.16.2.13:6443"
    }
  ]
}/ 

• Alternatively to locate Mountable secrete in the pods by describing the pods itself

kubectl describe pods alpine -n dev 
.....

Containers:
  busybox:
    Container ID:  docker://f330b20da3b95211774f53167e1f2e9149cbe144627f6f24608948cce3641767
    Image:         byrnedo/alpine-curl
    Image ID:      docker-pullable://byrnedo/[email protected]:e8cf497b3005c2f66c8411f814f3818ecd683dfea45267ebfb4918088a26a18c
    Port:          <none>
    Host Port:     <none>
    Command:
      sleep
      3600
    State:          Running
      Started:      Tue, 12 Feb 2019 12:00:30 +0530
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from test-token-hvbtq (ro)

5. Deleting ServiceAccount

Deleting the sa test under namespace dev

[email protected]:# kubectl delete sa test -n dev 
serviceaccount "test" deleted

Any Authentication performed using the old token will be Unauthorized

# export TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
# curl https://172.16.2.13:6443/api --header "Authorization: Bearer $TOKEN" --insecure
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401
}

6. ServiceAccount Practical Use-Case

Some of the practical use case of setting sa

• Jenkins: for Managing CI/CD pipeline in kubernetes cluster

kubectl -n kube-system create sa jenkins
kubectl create clusterrolebinding jenkins --clusterrole cluster-admin --serviceaccount=<namespace>:jenkins

• Helm: Kubernetes Repository Manager

kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

User Accounts

The Next form of Authentication in kubernetes API is with normal user, this is usually done with TLS authentication and typical steps involved

• User generate a strong RSA or EC keys
• User create CSR and bind their Common Name attribute for example [email protected]
• User submit this CSR to kubernetes admin.
• Kuberntetes admin verify the content of CSR before Issuing the certificate.
• Kubernetes admin will sign the CSR with rootCA and rootCA private key and generate the x.509 Certificate. the kubernetes PKI information is usually located in the /etc/kubernetes/pki/

We will bet into this Authentication using normal account in detail in the upcoming topic of Setting up Role-Based Access Control

---

Authorization

After the request is authenticated as coming from a specific user or sa , the request must be authorized.

For example exec to the alpine kubectl exec alpine -it sh -n dev and list all pods in the dev name space the request will be forbidden

# export TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
# curl https://172.16.2.13:6443/api/v1/namespaces/dev/pods --header "Authorization: Bearer $TOKEN" --insecure
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "pods is forbidden: User \"system:serviceaccount:dev:test\" cannot list pods in the namespace \"dev\"",
  "reason": "Forbidden",
  "details": {
    "kind": "pods"
  },
  "code": 403
}

In order to grant Authorization proper policy or RBAC control needs to be in placed, for an instance in order to give pod permission to query all pods in the given namespace, a role and corresponding rolebinding needs to be defined.

The below YAML file has setup necessary authorization for role and rolebinding under the namespace dev

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: test-manager
  namespace: dev
rules:
- apiGroups: ["", "batch", "extensions", "apps"]
  resources: ["pods"]
  verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: test-binding
  namespace: dev
subjects:
- kind: ServiceAccount
  name: test
  namespace: dev
roleRef:
  kind: Role
  name: test-manager
  apiGroup: rbac.authorization.k8s.io

[email protected]:# kubectl delete  -f rbac.yaml
role.rbac.authorization.k8s.io "test-manager" deleted
rolebinding.rbac.authorization.k8s.io "test-binding" deleted

The role is set to query the resource pods with all known HTTP verbs

# curl https://172.16.2.13:6443/api/v1/namespaces/dev/pods --header "Authorization: Bearer $TOKEN" --insecure
"kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/namespaces/dev/pods",
    "resourceVersion": "2302485"
  },
  "items": [
    {
      "metadata": {
        "name": "alpine",
        "namespace": "dev",
        "selfLink": "/api/v1/namespaces/dev/pods/alpine",
        "uid": "0abb388b-2e97-11e9-8c68-fa163e589bc0",
        "resourceVersion": "2300673",
        "creationTimestamp": "2019-02-12T07:23:06Z"

....

We will cover RBAC in much detail in upcoming topic let's stick with next important concept Admission Control

---

Admission Control

In Kubernetes, Admission Controllers enforce semantic validation of objects during create, update, and delete operations.

First check if the admission registration API is enabled in your cluster by running:

[email protected]:# kubectl api-versions | grep admission
admissionregistration.k8s.io/v1beta1

Some example of semantic validation can be done through the Admission Controllers

• Validate that image tags are are having certain pre-configured prefix or postfix.
• Reject an image if it is being pulled from dockerhub directly.
• Reject an image that has high or critical CVEs that have a fix available, but allow high-severity if no fix is available yet
• Never reject images from a specific registry/repository
• Others

There are two types of Webhook Admission controllers in Kubernetes 1.9.

• ValidatingAdmissionWebhook
• MutatingAdmissionWebhook

These webhooks is that they can be dynamically configured after the start of the api-server, this enables anyone with the correct Role-based access control (RBAC) to extend the concept of Admission Control.

Next Reading RBAC

---

Thanku for reading !!! Give a Share for Support

Asking for donation sound bad to me, so i'm raising fund from by offering all my Nine book for just $9

---

---