Sizes of busybox-1.21.1 and busybox-1.22.0 (with equivalent config, static uclibc build): text data bss dec hex filename 891522 497 7584 899603 dba13 busybox. You can poke around in the busybox container for a while, but you’ll quickly find that running small container with a few Linux utilities in it provides limited value, so exit out: exit There’s an old saying that “nobody runs an operating system just to run an operating system” and the same is true with containers. Busybox based on Buildroot 2014.11 container and rootfs builder for minimal Docker base images equipped with curl-7.39.0 including SSL/TLS support. For Buildroot 2015.02 have a look at the 2015.02 branch. The rootfs build can be found in the tarmaker directory. It is heavily inspired by radial/core-busyboxplus.
This is part 7 in a series of blog posts covering my learning experience for the CKAD exam. You can find the other parts in the series here:
After a quick look at Ballerina earlier this week, let’s start our sprint towards the end of the CKAD series! In this part we’ll touch on Services and Networking, specifically covering these two exam objectives:
- Understand Services
- Demonstrate basic understanding of NetworkPolicies
A special note on NetworkPolicies: There is an inconsistency between Azure and Calico NetworkPolicies on AKS. I have a seperate post explaining the troubleshooting I did to figure that out. I believe Calico to be implemented correctly, and that’s what I’ll describe in this post.
But, let’s start with Services. We’ve used Services a couple of times before, but let’s cover the topic a little more in depth now.
Understand Services
Top crossover cars 2020. A service within kubernetes is a way to expose applications. This can either be exposing them within the cluster or to the outside world. A service name will also act as a DNS record.
So why would we need services? Why not just send traffic straight to our pods? There’s a couple of reasons you don’t want to do that: first and foremost, pods are ephemeral, meaning pods can dynamically be rescheduled on new hosts, or could even be deleted and recreated when you update a deployment. A second reason is that – depending on your networking model – your pods might only be reachable within the cluster. The default networking in Kubernetes is kubenet, which is an overlay on top of a physical network. This means, outside actors cannot reach your pods.
Enter services with a solution. A service acts as a network abstraction on top of your pods. A service is both a DNS record and an IP address; and has a port assigned to it. When you communicate to a service, you communicate to the service IP on the service port, which will be translated to a pod (load balanced) and the ports actual port.
There are 4 types of services:
- ClusterIP: exposes your service on an IP that is available within the cluster.
- NodePort: this makes a port on each nodes IP address translate those connections to your service.
- LoadBalancer: this will manage an external load balancer (typically a cloud provider’s load balancer). This will (in the backend) create a clusterIP and a NodePort as well.
- ExternalName: this maps the service name to an CNAME.
Note that there are 3 ports in play end the end:
- Port: The port the service will be listening on;
- NodePort: A port on each node that translates connections to your service;
- TargetPort: The port your pods are listening on.
Now that we’ve done this, let’s play around with services a bit.
Let’s start off with a simple nginx deployment:
As usual, we can create this via
kubectl create -f nginx-deploy.yaml
Let’s now create a ClusterIP service to connect to our nginx.
Once created, we can see our service:
And from within the cluster, we should be able to connect to our service:
But we won’t be able to connect to this from the outside. For this, we’ll create a new service with the type Load Balancer:
Busybox With Curl
We can also create this service, which will take a bit longer as an external load balancer will need to be created:
Now we should be able to connect to our service on that IP, right from our web browser:
Demonstrate basic understanding of NetworkPolicies
By default, all traffic in a kubernetes cluster can flow unrestricted. Even pods in different namespaces can communicate with each other. NetworkPolicies allow you restrict traffic to only the traffic flows you actually want to allow. NetworkPolicies work on a allow-list, meaning that once a NetworkPolicy is applied to a pod all traffic is denied by default and only allowed traffic will flow.
Within a NetworkPolicy you define which traffic can flow from a certain source to a certain destination. Sources and destinations can either be IP addresses or ranges or can be pods within your cluster. You define which pods with the right selectors.
If you are planning – like me – to run this example on an Azure Kubernetes Cluster, make sure your cluster is enabled for Network Policies. They aren’t by default. You’ll want to add the
--networkpolicy calico
flag to your az aks create
. It cannot be applied after a cluster has already been created.Let’s build a simple example to demonstrate how NetworkPolicies work. We’ll create 4 pods within the same namespace. 2 busybox-curl pods, 1 with a trusted label, 1 without a trusted label and 2 nginx web servers, both with label app=web, and 1 with label env=dev and the other with label env=prod.
Let’s create a new namespace for our networking work, and set it as the default for our kubectl:
By now I expect you’d be able to create the YAML for these pods, so try to write this yourself before checking mine:
We can create this with
kubectl create -f pods.yaml
. Let’s get the IPs of our Nginx servers – and try if we can connect to both from both our busyboxes:If your results are the same as mine, all traffic flows, from both busyboxes to both nginx servers.
Let’s now define our first NetworkPolicy, which will allow traffic only from busyboxes with the label trusted=true. This will look like this:
We can deploy this policy with
kubectl create -f policy1.yaml
. Creating this policy has no impact on the running pods (they remain up and running) but will impact network network almost instantly. Let’s test it out:Let’s now also deploy our second policy, which will limit egress traffic from our trusted busybox, only to the dev environment. This policy looks like this:
Let’s create this policy as well, and see what the effect is.
kubectl create -f policy2.yaml
That was cool. Let’s make our experiment a little more complex by adding two new nginx pods: one with the label
env=dev
(but no longer the app label, so policy1 doesn’t apply) – and one with the label trusted=yes
. And then it’ll be our job to figure out which traffic flows are allowed in our experiment, those new arrows are the purple arrows in our drawing below:The following YAML will create our new pods:
Let’s start with the two arrows going from busybox-curl-1:
Let’s do the same for busybox-curl-2:
This will make our picture look like this:
Conclusion of the working of NetworkPolicy (using Calico)
We’ve deployed a couple of NetworkPolicies now, and I hope you have the same understanding as I do:
- If there are no policies, all traffic is allowed.
- If there is only a ingress policy, traffic from the sources mentioned in the ingress policy is allowed.
- If there is only a egress policy, traffic from to the destinations mentioned in the egress policy is allowed.
- If there is a combination of ingress and egress policies, only traffic allowed by both will be allowed.
Conclusion
In this part we repeated some of our work with Services on which we touched earlier. We also took a look into NetworkPolicies. For those of you interested in the ramblings of a mad man looking for answers, I have a seperate post explaining some of the troubleshooting I went through to figure out something was broken was Azure NetworkPolicies.
- PDF Link: cheatsheet-kubernetes-A4.pdf, Category: Cloud
- Blog URL: https://cheatsheet.dennyzhang.com/cheatsheet-kubernetes-A4
- Related posts: Kubectl CheatSheet, Kubernetes Yaml, #denny-cheatsheets
File me Issues or star this repo.
1.1 Common Commands
Name | Command |
---|---|
Run curl test temporarily | kubectl run --generator=run-pod/v1 --rm mytest --image=yauritux/busybox-curl -it |
Run wget test temporarily | kubectl run --generator=run-pod/v1 --rm mytest --image=busybox -it wget |
Run nginx deployment with 2 replicas | kubectl run my-nginx --image=nginx --replicas=2 --port=80 |
Run nginx pod and expose it | kubectl run my-nginx --restart=Never --image=nginx --port=80 --expose |
Run nginx deployment and expose it | kubectl run my-nginx --image=nginx --port=80 --expose |
List authenticated contexts | kubectl config get-contexts , ~/.kube/config |
Set namespace preference | kubectl config set-context <context_name> --namespace=<ns_name> |
List pods with nodes info | kubectl get pod -o wide |
List everything | kubectl get all --all-namespaces |
Get all services | kubectl get service --all-namespaces |
Get all deployments | kubectl get deployments --all-namespaces |
Show nodes with labels | kubectl get nodes --show-labels |
Get resources with json output | kubectl get pods --all-namespaces -o json |
Validate yaml file with dry run | kubectl create --dry-run --validate -f pod-dummy.yaml |
Start a temporary pod for testing | kubectl run --rm -i -t --image=alpine test-$RANDOM -- sh |
kubectl run shell command | kubectl exec -it mytest -- ls -l /etc/hosts |
Get system conf via configmap | kubectl -n kube-system get cm kubeadm-config -o yaml |
Get deployment yaml | kubectl -n denny-websites get deployment mysql -o yaml |
Explain resource | kubectl explain pods , kubectl explain svc |
Watch pods | kubectl get pods -n wordpress --watch |
Query healthcheck endpoint | curl -L http://127.0.0.1:10250/healthz |
Open a bash terminal in a pod | kubectl exec -it storage sh |
Check pod environment variables | kubectl exec redis-master-ft9ex env |
Enable kubectl shell autocompletion | echo 'source <(kubectl completion bash)' >>~/.bashrc , and reload |
Use minikube dockerd in your laptop | eval $(minikube docker-env) , No need to push docker hub any more |
Kubectl apply a folder of yaml files | kubectl apply -R -f . |
Get services sorted by name | kubectl get services –sort-by=.metadata.name |
Get pods sorted by restart count | kubectl get pods –sort-by=’.status.containerStatuses[0].restartCount’ |
List pods and images | kubectl get pods -o=’custom-columns=PODS:.metadata.name,Images:.spec.containers[*].image’ |
List all container images | list-all-images.sh |
kubeconfig skip tls verification | skip-tls-verify.md |
Ubuntu install kubectl | 'deb https://apt.kubernetes.io/ kubernetes-xenial main' |
Reference | GitHub: kubernetes releases |
Reference | minikube cheatsheet, docker cheatsheet, OpenShift CheatSheet |
1.2 Check Performance
Name | Command |
---|---|
Get node resource usage | kubectl top node |
Get pod resource usage | kubectl top pod |
Get resource usage for a given pod | kubectl top <podname> --containers |
List resource utilization for all containers | kubectl top pod --all-namespaces --containers=true |
1.3 Resources Deletion
Name | Command |
---|---|
Delete pod | kubectl delete pod/<pod-name> -n <my-namespace> |
Delete pod by force | kubectl delete pod/<pod-name> --grace-period=0 --force |
Delete pods by labels | kubectl delete pod -l env=test |
Delete deployments by labels | kubectl delete deployment -l app=wordpress |
Delete all resources filtered by labels | kubectl delete pods,services -l name=myLabel |
Delete resources under a namespace | kubectl -n my-ns delete po,svc --all |
Delete persist volumes by labels | kubectl delete pvc -l app=wordpress |
Delete state fulset only (not pods) | kubectl delete sts/<stateful_set_name> --cascade=false |
1.4 Log & Conf Files
Name | Comment |
---|---|
Config folder | /etc/kubernetes/ |
Certificate files | /etc/kubernetes/pki/ |
Credentials to API server | /etc/kubernetes/kubelet.conf |
Superuser credentials | /etc/kubernetes/admin.conf |
kubectl config file | ~/.kube/config |
Kubernets working dir | /var/lib/kubelet/ |
Docker working dir | /var/lib/docker/ , /var/log/containers/ |
Etcd working dir | /var/lib/etcd/ |
Network cni | /etc/cni/net.d/ |
Log files | /var/log/pods/ |
log in worker node | /var/log/kubelet.log , /var/log/kube-proxy.log |
log in master node | kube-apiserver.log , kube-scheduler.log , kube-controller-manager.log |
Env | /etc/systemd/system/kubelet.service.d/10-kubeadm.conf |
Env | export KUBECONFIG=/etc/kubernetes/admin.conf |
1.5 Pod
Name | Command |
---|---|
List all pods | kubectl get pods |
List pods for all namespace | kubectl get pods -all-namespaces |
List all critical pods | kubectl get -n kube-system pods -a |
List pods with more info | kubectl get pod -o wide , kubectl get pod/<pod-name> -o yaml |
Get pod info | kubectl describe pod/srv-mysql-server |
List all pods with labels | kubectl get pods --show-labels |
List all unhealthy pods | kubectl get pods –field-selector=status.phase!=Running –all-namespaces |
List running pods | kubectl get pods –field-selector=status.phase=Running |
Get Pod initContainer status | kubectl get pod --template '{{.status.initContainerStatuses}}' <pod-name> |
kubectl run command | kubectl exec -it -n “$ns” “$podname” – sh -c “echo $msg >>/dev/err.log” |
Watch pods | kubectl get pods -n wordpress --watch |
Get pod by selector | kubectl get pods –selector=”app=syslog” -o jsonpath='{.items[*].metadata.name}’ |
List pods and images | kubectl get pods -o=’custom-columns=PODS:.metadata.name,Images:.spec.containers[*].image’ |
List pods and containers | -o=’custom-columns=PODS:.metadata.name,CONTAINERS:.spec.containers[*].name’ |
Reference | Link: kubernetes yaml templates |
1.6 Label & Annontation
Name | Command |
---|---|
Filter pods by label | kubectl get pods -l owner=denny |
Manually add label to a pod | kubectl label pods dummy-input owner=denny |
Remove label | kubectl label pods dummy-input owner- |
Manually add annonation to a pod | kubectl annotate pods dummy-input my-url=https://dennyzhang.com |
1.7 Deployment & Scale
Name | Command |
---|---|
Scale out | kubectl scale --replicas=3 deployment/nginx-app |
online rolling upgrade | kubectl rollout app-v1 app-v2 --image=img:v2 |
Roll backup | kubectl rollout app-v1 app-v2 --rollback |
List rollout | kubectl get rs |
Check update status | kubectl rollout status deployment/nginx-app |
Check update history | kubectl rollout history deployment/nginx-app |
Pause/Resume | kubectl rollout pause deployment/nginx-deployment , resume |
Rollback to previous version | kubectl rollout undo deployment/nginx-deployment |
Reference | Link: kubernetes yaml templates, Link: Pausing and Resuming a Deployment |
1.8 Quota & Limits & Resource
Name | Command |
---|---|
List Resource Quota | kubectl get resourcequota |
List Limit Range | kubectl get limitrange |
Customize resource definition | kubectl set resources deployment nginx -c=nginx --limits=cpu=200m |
Customize resource definition | kubectl set resources deployment nginx -c=nginx --limits=memory=512Mi |
Reference | Link: kubernetes yaml templates |
1.9 Service
Name | Command |
---|---|
List all services | kubectl get services |
List service endpoints | kubectl get endpoints |
Get service detail | kubectl get service nginx-service -o yaml |
Get service cluster ip | kubectl get service nginx-service -o go-template='{{.spec.clusterIP}}’ |
Get service cluster port | kubectl get service nginx-service -o go-template='{{(index .spec.ports 0).port}}’ |
Expose deployment as lb service | kubectl expose deployment/my-app --type=LoadBalancer --name=my-service |
Expose service as lb service | kubectl expose service/wordpress-1-svc --type=LoadBalancer --name=ns1 |
Reference | Link: kubernetes yaml templates |
1.10 Secrets
Name | Command |
---|---|
List secrets | kubectl get secrets --all-namespaces |
Generate secret | echo -n 'mypasswd' , then redirect to base64 --decode |
Get secret | kubectl get secret denny-cluster-kubeconfig |
Get a specific field of a secret | kubectl get secret denny-cluster-kubeconfig -o jsonpath=”{.data.value}” |
Create secret from cfg file | kubectl create secret generic db-user-pass –from-file=./username.txt |
Reference | Link: kubernetes yaml templates, Link: Secrets |
1.11 StatefulSet
Name | Command |
---|---|
List statefulset | kubectl get sts |
Delete statefulset only (not pods) | kubectl delete sts/<stateful_set_name> --cascade=false |
Scale statefulset | kubectl scale sts/<stateful_set_name> --replicas=5 |
Reference | Link: kubernetes yaml templates |
1.12 Volumes & Volume Claims
Name | Command |
---|---|
List storage class | kubectl get storageclass |
Check the mounted volumes | kubectl exec storage ls /data |
Check persist volume | kubectl describe pv/pv0001 |
Copy local file to pod | kubectl cp /tmp/my <some-namespace>/<some-pod>:/tmp/server |
Copy pod file to local | kubectl cp <some-namespace>/<some-pod>:/tmp/server /tmp/my |
Reference | Link: kubernetes yaml templates |
1.13 Events & Metrics
Name | Command |
---|---|
View all events | kubectl get events --all-namespaces |
List Events sorted by timestamp | kubectl get events –sort-by=.metadata.creationTimestamp |
1.14 Node Maintenance
Busybox Docker
Name | Command |
---|---|
Mark node as unschedulable | kubectl cordon $NDOE_NAME |
Mark node as schedulable | kubectl uncordon $NDOE_NAME |
Drain node in preparation for maintenance | kubectl drain $NODE_NAME |
1.15 Namespace & Security
Name | Command |
---|---|
List authenticated contexts | kubectl config get-contexts , ~/.kube/config |
Set namespace preference | kubectl config set-context <context_name> --namespace=<ns_name> |
Switch context | kubectl config use-context <cluster-name> |
Load context from config file | kubectl get cs --kubeconfig kube_config.yml |
Delete the specified context | kubectl config delete-context <cluster-name> |
List all namespaces defined | kubectl get namespaces |
List certificates | kubectl get csr |
Check user privilege | kubectl –as=system:serviceaccount:ns-denny:test-privileged-sa -n ns-denny auth can-i use pods/list |
Check user privilege | kubectl auth can-i use pods/list |
Reference | Link: kubernetes yaml templates |
1.16 Network
Name | Command |
---|---|
Temporarily add a port-forwarding | kubectl port-forward redis-134 6379:6379 |
Add port-forwaring for deployment | kubectl port-forward deployment/redis-master 6379:6379 |
Add port-forwaring for replicaset | kubectl port-forward rs/redis-master 6379:6379 |
Add port-forwaring for service | kubectl port-forward svc/redis-master 6379:6379 |
Get network policy | kubectl get NetworkPolicy |
1.17 Patch
Name | Summary |
---|---|
Patch service to loadbalancer | kubectl patch svc $svc_name -p '{'spec': {'type': 'LoadBalancer'}}' |
1.18 Extenstions
Name | Summary |
---|---|
Enumerates the resource types available | kubectl api-resources |
List api group | kubectl api-versions |
List all CRD | kubectl get crd |
List storageclass | kubectl get storageclass |
1.19 Components & Services
1.19.1 Services on Master Nodes
Name | Summary |
---|---|
kube-apiserver | exposes the Kubernetes API from master nodes |
etcd | reliable data store for all k8s cluster data |
kube-scheduler | schedule pods to run on selected nodes |
kube-controller-manager | node controller, replication controller, endpoints controller, and service account & token controllers |
1.19.2 Services on Worker Nodes
Name | Summary |
---|---|
kubelet | makes sure that containers are running in a pod |
kube-proxy | perform connection forwarding |
Container Runtime | Kubernetes supported runtimes: Docker, rkt, runc and any OCI runtime-spec implementation. |
1.19.3 Addons: pods and services that implement cluster features
Name | Summary |
---|---|
DNS | serves DNS records for Kubernetes services |
Web UI | a general purpose, web-based UI for Kubernetes clusters |
Container Resource Monitoring | collect, store and serve container metrics |
Cluster-level Logging | save container logs to a central log store with search/browsing interface |
1.19.4 Tools
Busybox Curl Pod
Name | Summary |
---|---|
kubectl | the command line util to talk to k8s cluster |
kubeadm | the command to bootstrap the cluster |
kubefed | the command line to control a Kubernetes Cluster Federation |
Kubernetes Components | Link: Kubernetes Components |
1.20 More Resources
Busybox Curl Wget
License: Code is licensed under MIT License.