CKAD, woah!

kubernetes quotas

kubectl create namespace pippo
kubectl create quota myhq --hard=cpu=1,memory=1G,pods=2 --namespace=pippo
kubectl run --restart=Never busybox --image=busybox --namespace=pippo




Error from server (Forbidden): pods "busybox" is forbidden: failed quota: myhq: must specify cpu,memory


You can create your pod with requests and limits:


kubectl run --restart=Never busybox --image=busybox --namespace=pippo --limits=cpu=100m,memory=512Mi --requests=cpu=50m,memory=256Mi --dry-run -o yaml > mypod.yaml

spec:
  containers:
  - image: busybox
    imagePullPolicy: IfNotPresent
    name: busybox
    resources:
      limits:
        cpu: 100m
        memory: 512Mi
      requests:
        cpu: 50m
        memory: 256Mi

obviously all values in "requests" must be <= values in limits

CKAD CNCF Kubernetes Certification

Today I have attempted CKAD certification test.

It was TOUGH, but not impossible, actually I was expecting worse in term of complexity. And pass threshold is 66%, which is mild.

Good thing is that each of the 19 questions is self-contained, so you can easily skip it and try later.

This fellow says already everything on the topic:
https://medium.com/@nassim.kebbani/how-to-beat-kubernetes-ckad-certification-c84bff8d61b1

I would say:

"Kubernetes in Action" book is surely very complete (I have ready it 3 times) but not really preparing you for the test. Too verbose IMHO.

MUST 1: Take the Udemy course "https://www.udemy.com/course/certified-kubernetes-application-developer/", definitely excellent and very hands on.

MUST 2: and do at least 3 times all the exercises here https://github.com/dgkanatsios/CKAD-exercises


then go for the test. You might fail at the first attempt (time pressure is very high!) but it will make you stronger. You have a free retake, so no worries, lick your wounds and try again a couple of weeks later.

CAVEAT CANEM: many exercises entail usage of namespaces - remember to practice usage of namespaces.

IMPORTANT: if you google around for CKAD VOUCHER or CKAD DISCOUNT you should find some magic words to get a 20% rebate on the exam (normally 300 USD.... I got it for 250 or so).

kubernetes "change-cause" to describe a deployment

$ kubectl run nginx --image=nginx --replicas=4

$ kubectl annotate deployment/nginx kubernetes.io/change-cause='initial deployment'
deployment.extensions/nginx annotated

$ kubectl set image deploy nginx nginx=nginx:1.7.9

$ kubectl annotate deployment/nginx kubernetes.io/change-cause='nginx:1.7.9'
deployment.extensions/nginx annotated

$ kubectl set image deploy nginx nginx=nginx:1.9.1

$ kubectl annotate deployment/nginx kubernetes.io/change-cause='nginx:1.9.1'
deployment.extensions/nginx annotated


$ kubectl rollout history deploy nginx
deployment.extensions/nginx
REVISION CHANGE-CAUSE

5 initial deployment
6 nginx:1.7.9
7 nginx:1.9.1



This seems to me a very good practice, to be able to trace all changes in PROD.

You can always trace what changed:

kubectl rollout history deploy nginx --revision=6

deployment.extensions/nginx with revision #6
Pod Template:
  Labels:       pod-template-hash=7b74859c78
        run=nginx
  Containers:
   nginx:
    Image:      nginx:1.7.9
    Port:       
    Host Port:  
    Environment:        
    Mounts:     
  Volumes:      

kubernetes mount file on an existing folder

With ConfigMap and Secret you can "populate" a volume with files and "mount" that volume to a container, so that the application can access those files.


echo "one=1" > file1.properties
echo "two=2" > file2.properties
kubectl create configmap myconfig --from-file file1.properties --from-file file2.properties
kubectl describe configmaps myconfig

Name:         myconfig
Namespace:    default
Labels:       
Annotations:  

Data
====
file1.properties:
----
one=1

file2.properties:
----
two=2

Events:  

Now I can mount the ConfigMap into a Pod, as described here


cat mypod.yml

apiVersion: v1
kind: Pod
metadata:
  name: configmap-pod
spec:
  containers:
    - name: test
      image: nginx
      volumeMounts:
        - name: config-vol
          mountPath: /etc/config
  volumes:
    - name: config-vol
      configMap:
        name: myconfig
        items:
          - key: file1.properties
            path: myfile1.properties

kubectl create -f mypod.yml

kubectl exec -ti configmap-pod bash

cat /etc/config/myfile1.properties
one=1




Now I change the image to vernetto/mynginx, which contains already a /etc/config/file0.properties
The existing folder /etc/config/ is completely replaced by the volumeMount, so file0.properties disappears!
Only /etc/config/file1.properties is there.

They claim that one can selectively mount only one file from the volume, and leave the original files in the base image:
https://stackoverflow.com/questions/33415913/whats-the-best-way-to-share-mount-one-file-into-a-pod/43404857#43404857 using subPath, but it is definitely not working for me.

ports and pods

#start a pod with default parameters
kubectl run nginx --image=nginx --restart=Never

kubectl describe pod nginx

Node: node01/172.17.0.36
IP: 10.32.0.2

#we can reach nginx with the "IP" address
curl 10.32.0.2:80

but "curl 172.17.0.36:80" doesn't work!

kubectl describe nodes node01
InternalIP: 172.17.0.36


10.32.0.2 is the Pod's IP (=same IP for all containers running in that Pod):
kubectl exec -ti nginx bash
hostname -i
10.32.0.2

The Node IP cannot be used as such to reach the Pod/Container.

Setting the spec.container.ports.containerPort will not change neither the IP nor the POrt at which nginx is running: this parameter is purely "declarative" and is only useful when exposing the Pod/Deployment with a Service.

If you want to "expose" to an IP other than the Pod's IP:

kubectl expose pod nginx --port=8089 --target-port=80

kubectl describe service nginx
Type: ClusterIP
IP: 10.99.136.123
Port: 8089/TCP
TargetPort: 80/TCP
Endpoints: 10.32.0.2:80

NB this IP 10.99.136.123 is NOT the Node's IP nor the Pod's IP. It's a service-specific IP.

curl 10.99.136.123:8089

kubectl get service --all-namespaces

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default nginx ClusterIP 10.99.136.123 8089/TCP 16m



Ref:
https://kubernetes.io/docs/concepts/cluster-administration/networking/

Kubernetes academy

https://kubernetes.academy/lessons/introduction-to-kubectl awesome productivity tips from John Harris

source < (kubectl completion bash)

kubectx

kubens

kube-ps1 + kubeon

#doc on a k8s object
kubectl explain pod.spec.containers.ports

#grep json
kubectl get pod -n kube-system kube-scheduler-master -ojson | jq .metadata.labels

#show custom columns
kubectl get pod -n kube-system kube-scheduler-master -o custom-columns=NAME:.metadata.name,NS:.metadata.namespace


#show labels

kubectl get pod -n kube-system --show-labels

#show column with value of given label

kubectl get pod -n kube-system -L k8s-app

#filter by label value

kubectl get pod -n kube-system -l k8s-app=kube-dns -L k8s-app

#sort by

get pod -n kube-system -l k8s-app=kube-dns --sort-by='{.status.containerStatuses[*].restartCount}'

#trace execution (very verbose)

get pod -n kube-system -l k8s-app=kube-dns --sort-by='{.status.containerStatuses[*].restartCount}' -v10

https://kubernetes.academy/lessons/introduction-to-ingress

awesome Kubernetes Best Practices videos

there is a whole series, all same good

book: kubernetes in action

https://github.com/luksa/kubernetes-in-action the code

CRI-O

https://cri-o.io/

CRI-O = "Container Runtime Interface" "Open Container Initiative"
"a lightweight alternative to using Docker as the runtime for kubernetes"

https://www.quora.com/How-is-CRI-O-different-from-Docker-technology

"The CRI-O Container Engine is a implementation of a CRI (Kubernetes Container Runtime interface) that dedicated to Kubernetes. It implements only the features necessary to implement the CRI. Basically whatever Kubernetes needs. The goal to be as simple as possible and to never ever break Kubernetes. CRI-O is only for running containers in production. It runs OCI containers based on OCI images, which basically says it can run any container image sitting at Docker.io, Quay.IO, or any other container registry. It also launches OCI containers with runc.

Docker has a whole bunch of different technology, but I am guessing you are asking about the Docker daemon. Docker daemon is a general purpose container engine that implements API for launching OCI Container using the same runc that CRI-O uses. Docker daemon supports multiple different orchestrators including the Docker Client, Docker Swarm, Kubernetes, Mesosphere. It also supports everything from playing with containers to building containers.

The team behind CRI-O believes that building containers and developing and playing with containers should be done by different tools than the container engine that is used by Kubernetes. The CRI-O team has developed the Podman and Buildah container engines for developing/playing with containers and building container images.

Since these three tasks are done separately CRI-O can run with much tighter security than is required for building and developing containers."




CRI-O and kubeadm

https://katacoda.com/courses/kubernetes/getting-started-with-kubeadm-crio





What is a "pause" container and a "PID namespace sharing" ? https://www.ianlewis.org/en/almighty-pause-container


What is Weave ? https://www.weave.works/docs/cloud/latest/overview/

What is a Nodeport ? https://kubernetes.io/docs/concepts/services-networking/service/#nodeport

yipee.io, the online kubernetes yaml generator

I agree with every word written here:

https://yipee.io/wp-content/uploads/2018/10/yipee-whitepaper-oct-2018.pdf


beyond a very basic deployment, working directly with yaml files is suicidal.

I will explore this yipee and see if I can use it regularly, as replacement for a really primitive "vi mypod.yml"

Kubernetes Java client to generate yaml files for you

If you - like me - hate having to type YAML by hand, you can take advantage of a pre-built K8S Model and YAML serialization tool:

https://github.com/fabric8io/kubernetes-client

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>kubernetes-client</artifactId>
    <version>4.1.3</version>
</dependency>

import io.fabric8.kubernetes.api.model.Pod;
import io.fabric8.kubernetes.api.model.apps.DeploymentBuilder;
import io.fabric8.kubernetes.client.DefaultKubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.internal.SerializationUtils;

public class KubernetesClientTest {
 public static void main(String[] args) throws JsonProcessingException {
  String yaml = SerializationUtils.dumpAsYaml(new DeploymentBuilder().withNewSpec().endSpec().build());
  System.out.println(yaml);
  
  KubernetesClient client = new DefaultKubernetesClient();
  Pod pod = new Pod();
  System.out.println(pod);
  String podyaml = SerializationUtils.dumpAsYaml(pod);
  System.out.println(podyaml);
 }

}

The model is incredibly rich.... one has only to learn how to use it...

good presentation of Azure Kubernetes

https://youtu.be/gmN732qN1Gg


The session is available also here https://myignite.techcommunity.microsoft.com/sessions/65005
and here https://mediusproduction.blob.core.windows.net/presentations/Ignite2018/BRK2396.pptx you can download the slides (on the movie the resolution is quite lame)

What is https://en.wikipedia.org/wiki/Microsoft_Ignite ?


Azure DevSpaces https://docs.microsoft.com/en-us/azure/dev-spaces/
it's a direct tie between Visual Studio and AKS, with debugging of containers.


Azure Container Registry https://docs.microsoft.com/en-us/azure/container-registry/

Namespace-level RBAC security access, using AD groups.

Key Vaults contain Kubernetes and Application secrets (NO certificates in container!)

ACI https://docs.microsoft.com/en-us/azure/container-instances/ Azure Container Instances

OSB https://github.com/azure/open-service-broker-azure

Training modules:

https://docs.microsoft.com/en-us/learn/azure/ Introduction to Azure

https://docs.microsoft.com/en-us/learn/modules/welcome-to-azure/3-tour-of-azure-services an introduction to Azure Services

Quarkus Kubernetes and Katacoda

https://quarkus.io/guides/kubernetes-guide

I open a Katacoda lab with minikube in it https://www.katacoda.com/courses/kubernetes/launch-single-node-cluster

First step: https://quarkus.io/guides/getting-started-guide.html

git clone https://github.com/quarkusio/quarkus-quickstarts.git
cd getting-started
mvn compile quarkus:dev

now open a new terminal and "curl http://localhost:8080/hello"



cd quarkus-quickstarts/
cd getting-started-kubernetes/

#install graalvm
mkdir /root/graalvm
cd /root/graalvm
curl -L -o graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz https://github.com/oracle/graal/releases/download/vm-1.0.0-rc13/graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz
tar xvfz graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz
export GRAALVM_HOME=/root/graal/graalvm-ce-1.0.0-rc13

mvn package -Pnative

here I get plenty of compilation errors, so I am giving up...


GraalVM is available here http://www.graalvm.org/downloads/
and more instructions here https://www.graalvm.org/docs/getting-started/

Kubernetes cheat sheet 3

Network Policies

kubectl get networkpolicy

kubectl describe networkpolicy

Name:         payroll-policy
Namespace:    default
Created on:   2019-03-08 08:47:51 +0000 UTC
Labels:       <none>
Annotations:  <none>
Spec:
  PodSelector:     name=payroll
  Allowing ingress traffic:
    To Port: 8080/TCP
    From:
      PodSelector: name=internal
  Allowing egress traffic:
    <none> (Selected pods are isolated for egress connectivity)
  Policy Types: Ingress

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: internal-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      name: internal
  policyTypes:
  - Egress
  - Ingress
  ingress:
    - {}
  egress:
  - to:
    - podSelector:
        matchLabels:
          name: mysql
    ports:
    - protocol: TCP
      port: 3306

  - to:
    - podSelector:
        matchLabels:
          name: payroll
    ports:
    - protocol: TCP
      port: 8080

VOLUMES

https://portworx.com/basic-guide-kubernetes-storage/ good article

https://kubernetes.io/docs/concepts/storage/volumes/


https://kubernetes.io/docs/concepts/storage/persistent-volumes/

kind: PersistentVolume
apiVersion: v1
metadata:
  name: task-pv
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteOnce
  awsElasticBlockStore:
    volumeID: vol-867g5kii
    fsType: ext4

https://github.com/kodekloudhub/kubernetes-challenge-1-wordpress

Exploring VS Code plugin for Kubernetes

Crafting yaml by hand is not ideal.

Install VS Code:

https://linuxize.com/post/how-to-install-visual-studio-code-on-centos-7/


the simply type "code" ( I could not find the shortcut in Applications/Programming)

ctrl-shift-X , type "Kubernetes", and install the first one (by Microsoft)

and also "Kubernetes Tools" https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools

https://github.com/Azure/vscode-kubernetes-tools source code here


https://code.visualstudio.com/docs/azure/kubernetes

Of course you must have AKS CLI installed https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-yum?view=azure-cli-latest

When you start "code" and click on the Kubernetes icon on the left, you get all the clusters listed in /home/centos/.kube/config file

Kubernetes cheat sheet 2

Namespaces

kubectl get pods --namespace=dev
kubectl get pods --namespace=default

kubectl config set-context $(kubectl config current-context) --namespace=dev

ConfigMap

kubectl create configmap myconfigmap --from-literal=APP_COLOR=blue
kubectl create -f myconfigmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: myconfigmap
data:
  APP_COLOR: blue
  APP_MODE: prod

then you inject into a container definition using

envFrom: 
- configMapRef
    name: myconfigmap

kubectl get configmaps
kubectl describe configmaps db-config

Secrets

kubectl create secret generic mysecret --from-literal=mykey=myvalue

apiVersion: v1
kind: Secret
metadata:
  name: app-secret
data:
  DBHost: mysql
  DBUser: root
  DBPassword: password

kubectl create -f secret_data.yaml

SECURITY

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

you can declare at Pod or container level:

spec:
  securityContext:
    runAsUser: 1000
    capabilities:
      add: ["MAC_ADMIN"]

#check which user runs the container
kubectl exec ubuntu-sleeper whoami



kubectl create serviceaccount dashboard-sa
kubectl get serviceaccount
kubectl describe serviceaccount dashboard-sa
kubectl describe secret dashboard-sa-account-token

curl https://myip/api -insecure --header "Authorization: Bearer PASTE_THE_TOKEN_HERE"

#change serviceaccount for a deployment
kubectl --record deployment.apps/web-dashboard set serviceaccount dashboard-sa

RESOURCES

resources:
  requests:
    memory: "1Gi"
    cpu: 1

Taints and Tolerations

kubectl taint nodes node-name key=value:taint-effect

taint-effect can be: NoSchedule, PreferNoSchedule, NoExecute

key=value can be app=blue


tolerations:
- key: "app"
operator: "Equal"
value: "blue"
effect: "NoSchedule"



to remove taint:

kubectl taint nodes master node-role.kubernetes.io/master:NoSchedule-

NODE SELECTOR

nodeSelector:
  size: Large

where size is a key and Large a value

to label a node:
kubectl label node mynode key=value

affinity:
 nodeAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:
  nodeSelectorTerms:
  - matchExpressions:
    - key: color
   operator: In
   values:
   - blue
  

Readiness Probe

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/

in the spec/containers/ section for each container:

readinessProbe:
  httpGet:
    path: /api/ready
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

beside httpGet you can have: "tcpSocket: port:", "exec: command:"

Liveness Probe

livenessProbe:
  httpGet:
    path: /api/ready
    port: 8080

Kubernetes cheat sheet 1

https://kubernetes.io/docs/reference/kubectl/cheatsheet/

alias k=kubectl

PODS

https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/

kubectl run nginx --image=nginx


kubectl create -f nginx.yml

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
    - name: nginx-container
      image: nginx

kubectl describe pods nginx-pod


kubectl get pods -o wide

kubectl edit pod nginx-pod

kubectl delete pod nginx-pod


kubectl get pod pod-name -o yaml > pod-definition.yaml

Replication Controller

kubectl create -f rc-definition.yml

apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx
  spec:
    replicas: 3
    selector:
      app: nginx
    template:
      metadata:
        name: nginx
        labels:
          app: nginx
      spec:
        containers:
          - name: nginx
            image: nginx
            ports:
              - containerPort: 80

kubectl get rc

Replica Set

kubectl create -f rs-definition.yml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: frontend
  labels:
    app: guestbook
    tier: frontend
spec:
# modify replicas according to your case
  replicas: 3
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
      tier: frontend
    spec:
      containers:
        - name: php-redis
          image: gcr.io/google_samples/gb-frontend:v3

kubectl get replicaset

kubectl describe replicaset

kubectl replace

kubectl scale --replicas=3 rs/myrs

kubectl get rs myrs -o yaml

Deployments

https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

kubectl create -f mydeployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
  spec:
    replicas: 3
    selector:
      matchLabels:
        app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
            - containerPort: 80

kubectl get all




An overall good explanation of Kubernetes is here https://dzone.com/storage/assets/11459286-dzone-refcard292-advancedkubernetes314.pdf

Excellent Kubernetes Developer Certification training on Udemy

https://www.udemy.com/certified-kubernetes-application-developer for only 11 USD !

Mumshad Mannambeth is a fantastic trainer, focusing on concepts but analyzing every detail in a very visual manner.


https://www.cncf.io/certification/ckad/ Exam Homepage

https://github.com/cncf/curriculum/blob/master/CKAD_Curriculum_V1.14.1.pdf exam details



Candidate Handbook: https://www.cncf.io/certification/candidate-handbook

Exam Tips: https://www2.thelinuxfoundation.org/ckad-tips

CKA Certification (Kubernetes Administrator)

https://github.com/cncf/curriculum/blob/master/CKA_Curriculum_V1.12.0.pdf here the topics to be covered


I would start by reading the official doc https://kubernetes.io/docs/concepts/

Kubernetes Master : kube-apiserver, kube-controller-manager and kube-scheduler

Non-master node : kubelet, kube-proxy

Control Plane , kubectl,

etcd, kube-scheduler, kube-controller-manager

PodSpecs , Cluster DNS ,

(to be continued)