Category: Kubernetes

Kubernetes Realtime Interview Questions To Crack Interview

Posted on by mahesh

What have you done with Kubernetes?

Answer:

A prepared answer can be more effective than an impromptu one. Here’s a structured list of possible experiences:

  • Created clusters
  • Upgraded master and node pool versions
  • Deployed applications via Helm charts
  • Developed health check scripts for various workloads
  • Deployed Spinnaker for continuous delivery
  • Configured Horizontal Pod Autoscaler (HPA)
  • Managed day-to-day tasks like ConfigMaps, Secrets, Persistent Volumes (PVs), and Persistent Volume Claims (PVCs)
  • Troubleshot operational issues
  • Worked with StatefulSets
  • Created and signed Certificate Signing Requests (CSRs)

You have 2 different contexts (A and B). Context A has a secret named foo. How can you create the same secret in Context B?

Answer:

  1. Switch to Context A
  2. Run kubectl get secret foo -o yaml > foo.yaml
  3. Switch to Context B
  4. Run kubectl apply -f foo.yaml

What are the common methods to implement Ingress?

Answer:

Ingress is implemented by Ingress Controllers. Common options include:

  • Load Balancers (e.g., GCP, AWS)
  • Nginx Ingress Controller

Why do we need Kubernetes? What problems does it solve?

Answer:

Kubernetes addresses several issues that arise when using containers, such as:

  • Orchestration
  • Inter-container communication
  • Autoscaling
  • Observability
  • Security
  • Persistent and shared volumes

What is the difference between Ingress and Ingress Controller?

Answer:

  • Ingress: Defines how external traffic should be routed to services within the cluster. It does not handle traffic directly.
  • Ingress Controller: Implements the rules defined by Ingress and manages the traffic routing, e.g., through a Load Balancer or Nginx.

What is the most common type of Ingress Controller?

Answer:

Load Balancers are the most common type of Ingress Controller.

Which Ingress Controllers are officially supported by Kubernetes?

Answer:

Kubernetes officially supports:

  • AWS Ingress Controller
  • GCE Ingress Controller
  • Nginx Ingress Controller

What are some other Ingress Controllers besides the three supported by Kubernetes?

Answer:

Other Ingress Controllers include:

  • HAProxy
  • Istio Ingress
  • Traefik
  • Skipper
  • Voyager
  • Tyk Operator
  • Gloo
  • AKS Application Gateway Ingress Controller (Azure)
  • Ambassador
  • Enroute

How to troubleshoot a crashing container?

Answer:
You can use the --previous option with the logs command to view the logs of a crashed container:
kubectl logs --previous

What happens if a container uses too much CPU or memory?

Answer:

  • Memory: The container will be evicted.
  • CPU: The container will be throttled.

How do you manage scaling in Kubernetes?

Answer:

  • Horizontal Pod Autoscaler (HPA): Scales the number of pods based on CPU or custom metrics.
  • Vertical Pod Autoscaler (VPA): Adjusts the resource requests and limits for pods.
  • Cluster Autoscaler: Adjusts the size of the cluster by adding or removing nodes based on pending pods and node utilization.

For more details, see: Replex Blog on Kubernetes Scaling

How have you used RBAC with Kubernetes?

Answer:

RBAC (Role-Based Access Control) in Kubernetes is used to define permissions for different users or service accounts within the cluster. It works similarly to AWS IAM policies. Key components include:

  • Roles: Define what actions are permitted.
  • RoleBindings: Link users or service accounts to roles.
  • Subjects: Entities (users, service accounts) that get assigned roles.

How do you manage security for 200 microservices in your cluster?

Answer:

Use RBAC to define roles and assign them to users or service accounts. Each role specifies access permissions, which helps in managing security efficiently.

Tell me about the hardest production Kubernetes issue you solved.

Answer:

A common scenario might be:

Issue: A microservice update led to incorrect HPA settings, causing containers to crash and affecting other services.

Solution: Fixed the HPA settings and implemented circuit-breakers to prevent cascading failures.

How do you create YAML files for Pods without internet access?

Answer:

Use the kubectl explain command to view the fields for resource definitions:

 
  • kubectl explain pod –recursive
  • kubectl explain pv --recursive

How can you manage SSL certificates in Kubernetes?

Answer:

Store SSL certificates as Secrets. Then, mount these Secrets in your Pods to use the certificates.

What open-source tool can help you switch Kubernetes contexts easily?

Answer:

kubectx is a tool that simplifies switching between Kubernetes contexts.

How do you get the names of Kubernetes resources from the command line?

Answer:

Use the command:

kubectl api-resources

It lists all available resource types and their short names.

What are some Kubernetes verbs other than run, create, or apply?

Answer:

Additional verbs include:

set,explain,get,edit,delete,rollout,scale,autoscale,certificate,cluster-info,top,cordon,uncordon,drain,taint,describe,logs,attach,exec,port-forward,proxy,cp,auth,debug,diff,patch,replace,wait,kustomize,label,annotate,completion,api-resources,api-versions,config,plugin,version

 

What do you get when you run kubectl api-resources?

Answer:

You get a list of Kubernetes resources like Pods, Secrets, ConfigMaps, etc.

 

How do you deploy an additional scheduler on a Kubernetes cluster (not GKE)?

Answer:

  1. Package the scheduler as a Docker image.
  2. Push the image to a container registry.
  3. Create a deployment YAML file with kind: Deployment and component: scheduler in the kube-systemnamespace.
  4. Apply the deployment with:
kubectl apply -f scheduler.yaml

List out 2 use cases for DaemonSets and explain why it is more appropriate to use DaemonSets than Deployments for those use cases:

  • Answer:
    1. Log Collection Pod: DaemonSets are ideal for log collection because you want to ensure that logs are collected from every node in the cluster. Deploying a logging pod using a DaemonSet ensures that each node runs a copy of the pod, collecting logs efficiently across the entire cluster.
    2. Monitoring Pod (e.g., Dynatrace or Datadog): DaemonSets are better suited for monitoring tasks because each node needs to be monitored. By deploying the monitoring agent as a DaemonSet, you ensure that every node is covered without having to modify every individual deployment.

How to move workloads to a new node pool?

  • Answer: Use the cordon and drain commands.
    1. Cordon: This command prevents new pods from being scheduled on the node. (Command: kubectl cordon <node-name>)
    2. Drain: This command safely evicts existing pods from the node. (Command: kubectl drain <node-name>)

Is ClusterIP private or public?

  • Answer: Private. ClusterIP is accessible only within the Kubernetes cluster.

Which service type allows access to your services from the internet: ClusterIP or NodePort?

  • Answer: NodePort. NodePort exposes the service on a port on each node’s IP, which can be accessed from outside the cluster.

For a service, when we use NodePort, what does every node do?

  • Answer: Every node in the cluster assigns the service an external port and proxies the traffic to the appropriate pods within the service.

What does it mean when we say that a node proxies a service?

  • Answer: The node forwards incoming traffic to one of the pods that are part of the service

What are two ways to let a container have access to a secret?

  • Answer:
    1. Volume Mount: Mount the secret as a volume inside the container.
    2. Environment Variable: Pass the secret as an environment variable to the container.

How can a container have access to a secret via an environment variable?

  • Answer: You define the environment variable in the YAML file, and the container can access it using a command like echo $MY_SECRET.

One-liner command to run a pod with the nginx:alpine image?

  • Answer: kubectl run nginx-pod --image=nginx:alpine (Here, nginx-pod is an arbitrary pod name.)

How do you make a kubectl command run in a different namespace instead of the default one?

  • Answer: Use the -n <namespace_name> option with your kubectl command.

Command to create a namespace?

  • Answer: kubectl create ns foobar

When using the kubectl command, how do you get output in JSON format?

  • Answer: kubectl get nodes -o json

In the kubectl expose command, what is the difference between port and targetPort?

  • Answer:
    • port: This is the port on the cluster (service).
    • targetPort: This is the port on the container (like how ALB works).

Command to expose a pod as a service?

  • Answer: kubectl expose pod foobarpod --name foobarservice --port 6379 --target-port 6379

Command to scale a deployment named foobardeployment to 2 replicas?

  • Answer: kubectl scale deployment foobardeployment --replicas=2

Can you scale a Kubernetes service?

  • Answer: No, you can scale deployments and replicasets, but not services.

By default, can a pod in one namespace talk to another pod in another namespace?

  • Answer: Yes, by default, pods can communicate across namespaces.

By default, where do YAML files for static pod files go?

  • Answer: The YAML files for static pods go to /etc/kubernetes/manifests/ on the node.

What is a static pod?

  • Answer: Static Pods are managed directly by the kubelet on a specific node, without the API server observing them. They are defined by YAML files placed in a specific directory on the node.

There is a pod named foo that is in a CrashLoopBackOff state. How do you find the cause using a kubectlcommand?

  • Answer: Use kubectl describe pod foo to see why the pod is in the CrashLoopBackOff state. For example, you might find that the container’s command has a misspelling.

Scenario: A container keeps crashing because its “command” section has a misspelling. How do you fix this?

  • Answer:
    1. Generate the YAML file.
    2. Fix the issue in the YAML file.
    3. Delete the crashing pod.
    4. Re-run the corrected YAML file using kubectl apply -f.

What is the difference between PV and PVC?

  • Answer: PV (Persistent Volume) is essentially a disk volume of some sort, while PVC (Persistent Volume Claim) is a link between that volume and a pod.

How does the master server authenticate itself to etcd?

  • Answer: The master server authenticates itself to etcd using 2 certificates and 1 key, sending commands to etcd. On the master node, these configurations are in /etc/kubernetes/manifests/etcd.yaml, which points to the cert files and key file.

Examples of commands the master server can send to etcd (once authenticated with certs and key):

  • Answer:
    • member list
    • snapshot save /tmp/etcd-backup.db
    • snapshot status /tmp/etcd-backup.db -w table

Steps to create a pod named foo with the image redis and CPU request set to 2 CPUs and memory request set to 400MiB:

  • Answer:
    1. First, create a YAML file:
      kubectl run --generator=run-pod/v1 foo --image=redis --dry-run=client -o yaml > foo.yaml
    2. Edit the YAML file to include the following in the resources section of spec:
      yaml
      resources:
      requests:
      cpu: "2"
       memory: "400MiB"
    3. Apply the YAML file:
      kubectl apply -f foo.yaml

Can you think of some general areas of Kubernetes where you would want to think about security:

Answer:
a. Your container images
b. Your container registry
c. Your Kubernetes runtime infrastructure (e.g., etcd)
d. Hosts (where Kubernetes nodes are running)
e. Kubernetes Secrets
f. Kubernetes Certificates
g. RBAC entities

Processes within a container: How do they (security-wise) talk to the API server running on the master node?

Answer: Using a Kubernetes Service Account

How do you generate a CSR within the Kubernetes system?

Answer:
a. Create a .csr file using the openssl command (and a private key, but it does not matter to Kubernetes)
b. Encode it
c. Create a YAML file (Kind: CertificateSigningRequest) using the encoded CSR
d. Apply the YAML file using kubectl apply -f CertificateSigningRequest.yaml

If you have created a CertificateSigningRequest, but you have not approved it yet, what status do you get if you run the kubectl get csr command?

Answer: You will see that the request is in a “pending” state.

Command to approve a CSR?

Answer: kubectl certificate approve foo-csr

Example output: certificatesigningrequest.certificate.k8s.io/foo-csr approved

Kubectl command to create a role:

Answer: kubectl create role

Detailed example:

kubectl create role foo --resource=pods --verb=create,list,get,update,delete --namespace=development role.rbac.authorization.k8s.io/foo created
 

Command to describe a role:

Answer: kubectl describe role foo -n foo_namespace

Why is it important to keep etcd secure and encrypted?

Answer: etcd stores all your Kubernetes data, including Kubernetes Secrets.

3 Steps for creating a CA (Certificate Authority) on the master node?

Answer: (On a managed Kubernetes like GKE and EKS, you don’t need to do this):
a. Create a private key
b. Create a CSR
c. Self-sign the CSR

When you tell Kubernetes to run a pod, who decides which node gets that pod?

Answer: Scheduler

What if you don’t like the default scheduler that comes with Kubernetes?

Answer: You can always run your own scheduler.

If a node has a taint, what do you have to do to your pod for it to be able to run on that node?

Answer: You have to give the pod the same toleration.

If you want a pod to run on a specific node, which feature do you have to use?

Answer: Node Affinity

If we already have a liveness probe, why do we need a readiness probe?

Answer: There are times when a container fails a liveness probe, and yet we do not want the container to be killed. For example, if a container takes time to be ready (e.g., loads a large data set). In this case, the liveness probe would fail, and without a readiness probe, Kubernetes would kill the container. A readiness probe tells Kubernetes to wait for the container to finish its preparation work.

What is a “logging driver”?

Answer: Docker has the ability to send logs to various places (e.g., awslogs, fluent, and many more). Each one of these is a logging driver.

Which component collects and aggregates the metrics?

Answer: cAdvisor (which is part of kubelet on worker nodes). Those metrics are then sent to Metric Server (running on master nodes). Metric Server exposes them via kube-api (also running on the master node).

When you run kubectl top, which component are you talking to?

Answer: kube-api (which gets its data from Metric Server).

By default, a container runs with what UID?

Answer: 0 (i.e., root). This can be potentially bad if the container somehow gains access to the host OS.

What is the idea behind “Security Context”?

Answer: Security Context defines the level of permissions given to the container as it runs. By default, it runs with UID 0, which is potentially dangerous. Using runAsUser, runAsGroup, and fsGroup, we can limit what the container can do on the host. This is “Security Context.”

What is the “Ambassador Pattern”?

Answer: When the sidecar proxies the connections from the main container to the outside world.

What is the “Adapter Pattern”?

Answer: When the sidecar re-formats the output of the application running on the main container to another desired format. This way, you don’t have to re-write the application code when there is a need to re-format the output for another system to consume.

Can you describe a use case where the ambassador pattern can be of use?

Answer: If you have a legacy application that cannot be modified, but you need to change the port on which this app listens, the ambassador container can listen on the new port and pass on the traffic to the old port that was not modified.

What is the difference between a label and a selector?

Answer: Labels are basically tags. Selectors use key-value pairs to pick out objects (e.g., pods) to work on.

What is a network policy in Kubernetes?

Answer: A network policy is equivalent to a Security Group in AWS. You define who can talk to whom via network policy.

Network Policies often rely on what?

Answer: Labels and selectors.

When do maxSurge and maxUnavailable come into play?
Answer: During Rolling Updates of Deployments.

  • maxSurge: Defines the maximum number of additional pods that can be created during the update.
  • maxUnavailable: Specifies the maximum number of pods that can be unavailable during the update.

Why do we need HPA (Horizontal Pod Autoscaler) when we already have maxSurge and maxUnavailable?
Answer: HPA handles autoscaling based on resource usage, while maxSurge and maxUnavailable are specifically for managing pods during rolling updates.

What is the difference between Service Port and Target Port?
Answer:

  • Service Port: The port on which the service is exposed to users.
  • Target Port: The port on the container where the application is running and listening.

If you make a mistake with labels and/or selectors while configuring a service, how does it manifest?
Answer: The service will be created, but it won’t have any endpoints associated with it.

How can you check if you have permission to update pods using kubectl on your Mac?
Answer: Run kubectl auth can-i update pods. If you have permission, you’ll get a “yes” response

How do you ensure that a kubectl command is executed on the correct cluster when managing 100 of  Clusters?
Answer: Switch to the correct context using kubectl config use-context <context-name>. Tools like kubectx can help manage multiple contexts.

What is a quick way to create a service in Kubernetes?
Answer: Use the kubectl expose command.
Example: kubectl expose pod mypod --name=myservice --port=80 --target-port=8080 --type=ClusterIP

How does Kubernetes handle DNS internally?
Answer: Kubernetes uses DNS pods running in the kube-system namespace. You can view these by running: kubectl get po -A | grep dns.

If you’re on a node, how can you check for running containers?
Answer: Use the command docker ps.

How can you quickly generate a YAML file for a pod you usually create via the command line?
Answer: Add -o yaml --dry-run=client to your command to output the YAML without actually creating the pod.

What is a ClusterRoleBinding in Kubernetes?
Answer: A ClusterRoleBinding binds a user or a group to a cluster-wide role, granting them the permissions defined in that role.

How do you associate a pod with a Service Account in a YAML file?
Answer: Add serviceAccountName: <service-account-name> under the spec section.

How can you list all network policies in the default namespace?
Answer: Run kubectl get netpol.

Can you use pod selectors in an ingress network policy?
Answer: Yes, pod selectors are commonly used in ingress network policies to define which pods are allowed to receive traffic.

What is the purpose of api-versions in Kubernetes?
Answer: Kubernetes is composed of multiple APIs, each at different stages of maturity (alpha, beta, stable), which is why there are various API versions.

How can you find the correct API version to use for network policies?
Answer: Run kubectl api-versions | grep -i network.

Where is the default kubectl configuration file located on a Mac or PC?
Answer: The default configuration file is located at ~/.kube/config.

How can you scale a deployment named foo to 10 replicas using the command line?
Answer: Run kubectl scale deployment foo --replicas=10.

What should you do if you suspect issues with the control plane pods?
Answer: Run kubectl -n kube-system get pods to check the status of the control plane pods.

What does the ImagePullBackOff state indicate?
Answer: The ImagePullBackOff state means Kubernetes cannot pull the specified container image, possibly due to the image being unavailable or permission issues.

If scaling a deployment is stuck due to a control plane issue, what do you do after fixing the control plane?
Answer: No further action is needed. The controller-manager will automatically resume its job.

What is a namespace in Kubernetes, and why is it used?
Answer: A namespace provides a way to divide cluster resources between multiple users or teams, allowing for isolation of resources within the same cluster.

How would you troubleshoot pods in a CrashLoopBack state?
Answer:

  1. Run kubectl describe pod <pod-name> to check the events.
  2. Run kubectl logs -p <pod-name> to view the pod’s logs before it crashed.

What are the functions of the Kubernetes control plane, and where do they reside?
Answer:

  • Functions: API server, etcd, scheduler, controller-manager, cloud-controller-manager.
  • Location: These components run on the master node.

What are the components of a Kubernetes worker node?
Answer: Docker (container runtime), kubelet, kube-proxy.

Which Kubernetes component is responsible for tainting and placement of pods on nodes?
Answer: The scheduler.

What are annotations used for in Kubernetes, and how are they different from labels and selectors?

Answer:

  • Annotations: Used to attach non-identifying metadata to objects (e.g., contact info, version details, or URLs). Think of them like comments or sticky notes attached to a Kubernetes object.
  • Difference from Labels: Labels are key-value pairs that identify and organize Kubernetes objects. They can be used by selectors to filter and manage resources.
  • Selectors: Only work with labels, not annotations. You can query or group resources based on labels, but not annotations.

Is deployment and service the same? Explain the difference or the sameness between the two concepts.

Answer:

  • Deployment: Similar to running a terraform apply for pods, managing and ensuring they match the desired state. It handles scaling, updates, and rollbacks of pods.
  • Service: Acts as an entry point to access the deployed pods. Users interact with the service, which routes traffic to the appropriate pods, abstracting the underlying infrastructure.

What are the three main characteristics you should focus on to troubleshoot what can go wrong between pods and services?

Answer:

  1. Target Port: The port on the container that the service routes traffic to.
  2. Labels: Ensure that the pods have the correct labels that match the service selectors.
  3. Selectors: Ensure the service is correctly selecting the intended pods.

What are the mechanisms to expose an application running in Kubernetes to the outside world?

Answer:

  • Mechanism Flow: Pods → Service → Public IP → DNS → External Users
  • Explanation: The service provides an IP that external users can access. DNS can then map a human-readable name to this IP.

List some useful commands to troubleshoot Pods issues:

Answer:

  • kubectl describe pod [podname]: Detailed information about a pod, including events.
  • kubectl port-forward [podname] 3000:80: Forward a local port to a pod’s port.
  • kubectl get pods -o wide: Extended output, including node names and IPs.
  • kubectl logs [podname]: View the logs of a pod.
  • kubectl exec -ti [podname] -- bash: Execute a bash shell inside the pod.

What is port-forwarding?

Answer:

  • Concept: Creates a link between a port on your local machine (e.g., laptop) and a port on a pod.
  • Example: If a pod’s service is running on port 443, you can use port-forwarding to access it on your local machine, e.g., https://localhost:4430.

Pods can have startup and runtime errors. Explain what some of these errors mean and 2-3 common culprits.

Answer:

  1. ImagePullBackOff:

    • Meaning: The Docker image could not be retrieved.
    • Culprits: Incorrect registry name, bad image name, or the image no longer exists.

  2. CrashLoopBackOff:

    • Meaning: The container repeatedly crashes after starting.
    • Culprits: Container has no tasks to run or an improperly configured readiness probe.

  3. RunContainerError:

    • Meaning: The container could not start.
    • Culprits: Network issues or authorization problems.

  4. Pods in Pending State:

    • Meaning: The pod is waiting to be scheduled.
    • Culprits: Insufficient resources or connectivity issues between nodes.

  5. Pods in a Not Ready State:

    • Meaning: The pod is scheduled but not fully up.
    • Culprits: A failing readiness probe.

Can you schedule regular pods on the master node (general Kubernetes, not Managed kubernets like GKE, EKS)?

Answer:

  • Yes, but you need to remove the default NoSchedule taint from the master node first.

You have a node A with taint=blue. You have a Pod X with toleration for taint=blue. Would pod X always be placed on Node A? Explain your answer in detail.

Answer:

  • No. Taint acts as a barrier. Pod X can be scheduled on Node A because it tolerates the taint, but it might also be scheduled on other nodes without that taint, depending on other scheduling criteria.

What is the use case for node affinity vs nodeSelector?

Answer:

  • nodeSelector:
    • Use Case: Simple scheduling based on labels. For example, you can ensure that all pods belonging to a certain team or environment (e.g., dev) are placed on specific nodes with matching labels.
  • nodeAffinity:
    • Use Case: More complex scheduling rules, including preferences and soft requirements. For example, you can ensure that pods are scheduled based on specific hardware characteristics, geographic location, or to ensure pods are colocated (or not colocated) with others.
  • Difference: nodeSelector is a basic, exact-match filter, while nodeAffinity allows for more nuanced, conditional scheduling rules.

How do you find out what image of the running container (in a pod)?

Answer:

  • Command: kubectl describe pod [podname] | grep -i image
  • Explanation: This command filters the pod description to show the image used by the container(s) within the pod.

Command used to find out what node the pods are running on:

Answer:

  • Command: kubectl get pods -o wide
  • Explanation: This command provides additional details about each pod, including the node it is running on.

What does the READY column in the output of the “kubectl get pods” command indicate?

Answer:

  • Explanation: The READY column shows how many containers in the pod are supposed to be running vs. how many are actually running. For example, 1/1 means all containers are running, while 0/1 indicates a problem with the pod’s containers.

What happens if all master nodes are unavailable on GKE? Would that impact workloads running on the worker nodes?

Answer:

  • Explanation: The workloads running on the worker nodes will continue to run. However, new deployments, scaling, or any operations that require interaction with the master nodes will not be possible. This is similar to Hadoop, where the worker nodes continue processing even if the master is down, but no new jobs can be submitted.

What is the difference between a daemonset and a deployment?

Answer:

  • DaemonSet: Ensures that a copy of a pod runs on every node in the cluster. It’s used for tasks that should run on all nodes, like logging or monitoring agents.
  • Deployment: Manages a set of identical pods, allowing for updates, scaling, and rollbacks. It’s used for stateless applications.
  • Difference: A DaemonSet is for services that must run on every node, while a Deployment is for scalable, stateless services that can run on one or more nodes.

What is the default deployment strategy of a Kubernetes deployment?

Answer:

  • Default: Rolling Update
  • Other Strategies:
    • Blue-green deployment: Deploys a new version alongside the old one, then switches traffic once the new version is ready.
    • Canary deployment: Gradually shifts traffic from the old version to the new one.
    • A/B Testing: Deploys different versions to different segments of users for testing.

In a replica set definition, how do we tell the replica set that a set of pods is part of the replica set?

Answer:

  • Using Selectors:
    yaml
     
    spec:
      replicas: 3          selector:     
          matchLabels:
        app: myapp
  • Explanation: The selector matches the labels on the pods that the replica set should manage.

What are the benefits of resource limits in Kubernetes?

Answer:

  • Benefits:
    • Prevents resource contention: Ensures that no single container consumes more resources than allocated, preventing it from affecting other containers.
    • Resource management: Helps in maintaining a balanced resource allocation, preventing any runaway container from monopolizing resources.
    • Alerting: You get alerts if resource usage approaches the set limits, allowing proactive management.

Explain what is meant by resource request and resource limits setting.

Answer:

  • Resource Request:
    • Definition: The minimum amount of resources (CPU/memory) a container needs to start and run. The scheduler only assigns the pod to a node that can meet this request.
  • Resource Limit:
    • Definition: The maximum amount of resources a container is allowed to use. If the container tries to use more than this, it can be throttled or terminated.
  • Analogy: The request is the “entry ticket,” and the limit is the “bad boy level” that the container should not exceed.

How do you see which pods or nodes are using the most resources?

Answer:

  • Command:
    • kubectl top pod
    • kubectl top nodes
  • Explanation: These commands provide real-time metrics on resource usage (CPU/memory) for pods and nodes, helping you identify the highest consumers.

Can a POD span more than 1 “node”?

Answer: No.

Explanation: A Pod is the smallest deployable unit in Kubernetes, and it is always bound to a single node. Even though a Pod can contain multiple containers, all containers within the Pod share the same network namespace and storage, so they must run on the same node.

Does a Pod always get an IP?

Answer: Yes.

Explanation: Each Pod in Kubernetes is assigned a unique IP address within the cluster. This allows containers within the Pod to communicate with other Pods in the cluster without using network address translation (NAT).

Let’s say that you want to add a “sleep” command to your container. Where does that go in the YAML file?

Answer: In the spec section: command: ['sleep'].

Explanation: The command field in the Pod’s YAML file defines the entry point for the container. If you want the container to run a specific command, such as sleep, you would place it in the spec section under containers.

Can you edit any live object using the “kubectl edit” command?

Answer: No.

Explanation: While you can edit many Kubernetes resources with kubectl edit, some resources may not allow editing or may have restrictions on what can be changed. For example, certain fields of resources like Pods cannot be edited once they are created.

Command to edit the configuration of a live pod:

Answer: kubectl edit pod foo.

Explanation: This command opens the Pod’s configuration in your default text editor, allowing you to make changes. Once saved, the changes are applied immediately.

What dictates how much resources a container gets?

Answer: request and limit parameters.

Explanation: The request parameter specifies the minimum amount of CPU and memory a container needs, while the limit parameter sets the maximum amount the container can use. These parameters help Kubernetes allocate resources efficiently.

Pods come and go. So, how in the world does Kubernetes provide any real service?

Answer: A Service’s IP NEVER changes. You can point DNS to it. Behind the “service” are the ephemeral Pods.

Explanation: Kubernetes uses Services to provide a stable IP address and DNS name for accessing Pods. Even if Pods are replaced, the Service maintains a consistent endpoint for clients.

You run kubectl get pods and you see a Pod that is in “completed” state. What does that mean?

Answer: This means that the Pod came up, did its job, and finished. It did not crash. It is not running. You can still get to its logs.

Explanation: A Pod in the “completed” state indicates that it successfully executed its task and exited without errors. It’s common for Pods running batch jobs or scripts.

What kind of troubleshooting have you done in Kubernetes?

Answer: This depends on your experience, but some ideas include ingress issues, capacity problems, Pods crashing, slow service, certificate expiration, etc.

Explanation: Troubleshooting in Kubernetes can involve various issues, from networking and resource management to application-specific problems.

How is Anthos Service Mesh compared to Istio?

Answer: Anthos Service Mesh is a managed service. It is cheap ($50 a month for 100 endpoints per cluster as of Jan 2022). It comes with dashboards automatically. So, definitely a good choice. Also, no more hassles of upgrading Istio.

Explanation: Anthos Service Mesh simplifies the management of service meshes by offering a managed version of Istio, reducing the operational overhead for users.

Who manages virtual IPs of services?

Answer: kube-proxy.

Explanation: kube-proxy is responsible for managing virtual IPs and routing traffic to the correct Pod based on the Service definition.

Ingress works at which OSI layer?

Answer: Layer 7 (HTTP or HTTPS).

Explanation: Ingress operates at the application layer (Layer 7), handling HTTP and HTTPS traffic and providing features like host-based routing and SSL termination.

Where does kube-proxy run?

Answer: On each node. You can think of this as any other network proxy (e.g., HAProxy or Nginx or Squid) running on each node managing traffic in and out of nodes.

Explanation: kube-proxy runs on every node in the cluster and handles network traffic routing to ensure that requests reach the correct Pods.

When implementing Prometheus, why is it best to use the Adapter pattern?

Answer: Because otherwise, you will have to rewrite each application’s “data” to the format that Prometheus expects. The Prometheus sidecar will do that and send the data along without you having to modify the application container.

Explanation: The Adapter pattern simplifies integration with Prometheus by handling the conversion of application metrics to the format that Prometheus expects.

What is Kubelet and where does it run?

Answer: Main agent on the worker nodes.

Explanation: Kubelet is the primary agent that runs on each worker node in a Kubernetes cluster, ensuring that containers are running as expected.

What is the difference between Docker Compose and Kubernetes?

Answer:

  • Docker Compose: Simple way to run multi-container Docker Applications (defined in YAML file).
  • Kubernetes: It is a full-fledged Orchestration Tool.

Explanation: Docker Compose is designed for local development and testing of multi-container applications, while Kubernetes is a robust orchestration system for managing containerized applications at scale.

What is kubeadm used for?

Answer: To deploy Kubernetes on existing VMs kind of by hand (running commands for the master node and worker nodes).

Explanation: kubeadm is a tool that simplifies the deployment of a Kubernetes cluster by automating the installation and configuration of the necessary components.

When we run kubectl run pods, that gets to the API server on the master node. What does the API server do with that request?

Answer: It gives it to the kubelet on one worker node.

Explanation: The API server processes the request and passes the instructions to the Kubelet on a worker node, which then starts the Pod.

How do you deploy a stateless application on Kubernetes?

Answer: Simply use “deployments” (Not StatefulSet or ReplicaSet).

Explanation: Deployments are used for stateless applications because they manage the desired state of Pods and ensure that the specified number of replicas are running at any given time.

What is an endpoint in Kubernetes?

Answer: Nothing but an IP and a port. That’s it.

Explanation: An endpoint in Kubernetes represents the IP and port of a service that can receive traffic. It’s the

What is the relationship between a Service and an Endpoint?

Answer: When a client hits a Service, the Service acts like a load balancer and needs to know where to send the request. It forwards the request to an Endpoint, which is automatically created by Kubernetes based on a match with a pod. The Endpoint contains the pod’s IP and port, essentially routing traffic from the Service to the correct pod.

Conceptual Flow:

  • Service → Endpoint (automatically created) → Pod’s IP and Port

How can you access the kubelet API?

Answer: There are two ways to access the kubelet API:

  1. Using a curl command pointing to port 10250 of a worker node.
  2. Using an open-source tool called kubeletctl.

When creating a pod using the kubectl run command, how can you supply a command to run on the container (like sleep 3600)?

Answer: Use the --command -- option in the kubectl run command.

  • Example: kubectl run foo --image=nginx --command -- sh -c "sleep 3600"

How can you log in to a pod (assuming it only has 1 container)?

Answer: Use the following command: kubectl exec foo -it /bin/bash.

When creating a pod, what are the three restart policy options?

Answer:

  1. Always
  2. Never
  3. OnFailure

What are the use cases for each restart policy option when creating a pod?

Answer:

  1. Always: Used for Deployments or ReplicaSets.
  2. Never: Used for one-time pod runners, e.g., via command line.
  3. OnFailure: Used for jobs that need to retry on failure.

If there is a pod already running and you want to restart using a different image, how do you do that using the command line?

Answer: Use the kubectl set image command.

How do you get logs from a container (not a pod) via the command line?

Answer: Use the command kubectl logs <pod-name> -c <container-name>.

What does an “operator” pod do?

Answer: An operator pod manages the lifecycle of complex, stateful applications, like a MySQL cluster. It can automatically handle tasks like restarting failed pods or promoting a read-only pod to a leader in case the leader pod crashes. This automation ensures that the setup remains stable without manual intervention.

What is a Custom Resource Definition (CRD)?

Answer: A CRD allows you to define your own custom resource types in Kubernetes, beyond the standard types like pods or deployments. Operators often use CRDs to extend Kubernetes with new capabilities.

Command to get a list of contexts you have defined:

Answer: Use the command kubectl config get-contexts.

Which file holds your context definitions?

Answer: The file is ~/.kube/config.

By default, can pod A in namespace A talk to pod B in namespace B?

Answer: Yes, by default, pod A can communicate with pod B across namespaces.

What is a Headless service in Kubernetes?

Answer: A Headless service has a service IP but, instead of load-balancing, it returns the IPs of the associated pods. This is useful for direct pod communication without a single service IP acting as a load balancer.

When does Kubernetes pull a new version of an image upon pod creation?

Answer: Kubernetes pulls a new version of an image if:

  1. The image is tagged as :latest.
  2. The imagePullPolicy: Always is specified in the pod’s YAML file.

How do you manage costs on Kubernetes?

Answer: Managing costs involves three areas:

  1. Control Plane: Limited options for cost management.
  2. Worker Nodes: Ensure that autoscaling is configured properly.
  3. Pod Resources: Optimize CPU and memory usage by using tools like Metrics Server or open-source tools like Kubecost.

What is rehydrating in Kubernetes?

Answer: Rehydrating involves running the same cluster using new nodes with a newer version of Kubernetes, then moving the pods to the new nodes. It is the opposite of draining.

Command to drain a node?

Answer: Use the command kubectl drain <node-name>.

How do you monitor your Kubernetes clusters?

Answer: Common monitoring tools include:

  • Prometheus and Kibana (open-source solutions).
  • Dynatrace and Datadog (paid solutions).

How do containers on the same pod communicate?

Answer: Containers in the same pod communicate over localhost.

If you create an ingress, how will the traffic be impacted?

Answer: Nothing! Until you have an Ingress Controller, an ingress rule does nothing.

Explanation:
When you create an ingress resource in Kubernetes, it defines how external traffic should be routed to services within your cluster. However, the ingress resource by itself doesn’t affect the traffic unless there is an Ingress Controller deployed in the cluster. The Ingress Controller is responsible for processing these ingress rules and managing the routing of traffic accordingly. Without an Ingress Controller, the ingress rules are simply inert.

Does Ingress Controller need to read packets?

Answer: Yes, it needs to read the headers.

Explanation:
Ingress Controllers need to inspect the headers of incoming HTTP(S) requests to determine how to route the traffic based on the ingress rules. These rules may direct traffic to different services based on factors like the request’s URL path, host, or other headers. Reading the headers allows the Ingress Controller to apply the appropriate routing logic.

How do you create an Ingress Controller? Provide an example.

Answer: You can create a deployment using an Nginx image. That would be one way of doing it.

Explanation:
Creating an Ingress Controller often involves deploying a containerized application that understands and implements the ingress rules. Nginx is a popular choice for an Ingress Controller in Kubernetes. You can create a Kubernetes Deployment using the official Nginx Ingress Controller image, which will handle traffic routing based on the ingress resources defined in your cluster.

Example YAML for creating an Nginx Ingress Controller: 

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
spec:
 replicas: 1
 selector:
 matchLabels:
  app: nginx-ingress
 template:
  metadata:
   labels:
    app: nginx-ingress
 spec:
  containers:
   – name: nginx-ingress-controller
    image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:latest
  args:
   – /nginx-ingress-controller

How do you tell an Ingress to use an Ingress Controller?

Answer: In the spec section, there is a configuration item called “backend”. There you can point to a service (e.g., based on Nginx deployment).

Explanation:
In Kubernetes, the ingress resource includes a specification section where you define the rules for routing traffic. The backend configuration within this spec points to the service that should handle the traffic. The Ingress Controller, which is typically associated with a specific service, will use these rules to determine how to route traffic to the appropriate backend service.

What is the difference between Ingress and Egress in Kubernetes?
Answer:

  • Ingress: Controls the incoming traffic to the Kubernetes cluster, typically for services.
  • Egress: Controls the outgoing traffic from the Kubernetes cluster to external services or destinations.

How do you ensure high availability for a Kubernetes cluster?
Answer:

  • Multiple Master Nodes: Distribute control plane components across multiple nodes.
  • Etcd Clustering: Run etcd in a cluster with odd numbers for quorum.
  • Multiple Worker Nodes: Distribute workloads across multiple worker nodes.
  • Cloud Load Balancer: Use a load balancer to distribute traffic across master nodes.

What are some common Kubernetes security best practices?
Answer:

  1. RBAC (Role-Based Access Control): Implement RBAC for fine-grained access control.
  2. Network Policies: Use network policies to control communication between pods.
  3. Pod Security Policies: Enforce security context constraints on pods.
  4. Secrets Management: Store sensitive data securely using Kubernetes Secrets.
  5. Image Scanning: Regularly scan container images for vulnerabilities.

What is the role of etcd in Kubernetes?
Answer:

  • etcd is the key-value store used by Kubernetes to store all cluster data, including configuration, state, and metadata. It is the backbone of Kubernetes, providing strong consistency and high availability.

How do you manage storage in Kubernetes?
Answer:

  • Persistent Volumes (PVs): Abstractions for storage resources in the cluster.
  • Persistent Volume Claims (PVCs): Requests for storage by applications.
  • Storage Classes: Define different types of storage (e.g., SSD, HDD) and dynamic provisioning.
  • CSI (Container Storage Interface): A standard for exposing storage systems to Kubernetes.

How do you secure communication between pods in Kubernetes?
Answer:

  • Network Policies: Define rules that specify how pods are allowed to communicate with each other.
  • mTLS (Mutual TLS): Use service meshes like Istio to enforce mTLS for pod-to-pod communication.
  • Calico: A network and network security solution that can enforce network policies and secure pod communication.

How does Kubernetes handle service discovery?
Answer:

  • DNS: Kubernetes provides an internal DNS service that automatically creates DNS entries for services.
  • Environment Variables: Kubernetes injects environment variables for each service into the pods at runtime.

If a client sends a request to the API server, which component intercepts that request to determine if it should be processed?
Answer:

  • Admission Controllers: These are components that intercept requests to the Kubernetes API server after authentication and authorization but before the request is processed. They enforce various policies and can be used to validate or mutate incoming requests.

How can you follow the DRY (Don’t Repeat Yourself) principle in Kubernetes across different environments (dev, staging, prod)?
Answer:

  • Solution: Use ConfigMaps to store environment-specific configurations. The application code remains the same, while only the values in the ConfigMaps differ across environments.

Why do we need Persistent Volumes (PV) and Persistent Volume Claims (PVC) in Kubernetes? Why not just use one?
Answer:

  • Reason: The separation of PVs and PVCs allows for decoupling between storage providers and users. PVs define the storage resource, while PVCs request and use the storage. This decoupling allows for greater flexibility and reuse of storage resources.

Why do we need StorageClasses in Kubernetes?
Answer:

  • Reason: StorageClasses enable decoupling between storage definitions and the actual storage backend. They allow you to define different types of storage (e.g., fast, slow, EBS, GCS, NFS) and use them as needed without being tied to a specific storage type.

Explain how a container mounts a volume in Kubernetes.
Answer:

  • Process: Containers rely on the pod to define the volume and give it a name. The container then uses that name to mount the volume on a specific directory within the container’s file system. Multiple containers within the same pod can mount the same volume on different mount points.

Explain the flow from a Certificate Signing Request (CSR) to a user having permission to describe a pod in Kubernetes.
Answer:

  1. CSR Creation: A CSR is created.
  2. Submission: The CSR is submitted to the Kubernetes cluster.
  3. Approval: An administrator approves the CSR within Kubernetes.
  4. Certificate Issuance: A valid certificate is issued, which includes an embedded user.
  5. Role Creation: A role is created with specific permissions (e.g., describe a pod).
  6. Role Binding: The user/certificate is associated with the role via a RoleBinding. This grants the user the necessary permissions.

Why do we need Node Pools in Kubernetes?
Answer:

  • Purpose: Node Pools allow you to group nodes with similar configurations (e.g., CPU, memory). This enables better resource allocation by directing specific workloads to appropriate nodes, such as directing high-resource pods to a high-capacity node pool.

What is the connection between values.yaml (Helm) and ConfigMaps in Kubernetes?
Answer:

  • Connection: The values.yaml file in Helm charts stores environment-specific values, which can then be used to populate ConfigMaps in Kubernetes. This ensures that the same code can be reused across multiple environments, with only the configurations changing.

Why do you need a “Job” in Kubernetes?
Answer:

  • Purpose: A Job is used to create a pod for the sole purpose of performing a task only once. It ensures that the task runs to completion, even if the pod fails and needs to be restarted.

Why do you need a “CronJob” in Kubernetes?
Answer:

  • Purpose: A CronJob is used to schedule the creation of pods at regular intervals to perform a recurring task. It’s ideal for tasks that need to run on a schedule, such as backups or periodic data processing.

Walk me through the steps of how you store a password in Kubernetes (e.g., a database password).
Answer:

  1. Base64 Encoding: Encode the password using Base64.
  2. Create a Secret: Create a Kubernetes Secret using the encoded password in the data section as a key-value pair.
  3. Mount the Secret: Mount the Secret on a pod/container, either as an environment variable or as a file.

You have a Docker image (that runs a web server) but no Kubernetes cluster. Walk me through how you take this service live. Assume no scaling or HA is needed.
Answer:

  1. Create a Cluster: Set up a Kubernetes cluster using a cloud provider or local tools like Minikube.
  2. Upload the Image: Push the Docker image to a container registry like GCP Container Registry.
  3. Create a YAML File: Write a YAML file that defines a Deployment using the Docker image.
  4. Create a Service: Define a Service with type LoadBalancer to expose the Deployment.
  5. DNS Configuration: Create a DNS record pointing to the Service’s external endpoint.

What is the difference between an Ingress and a Network Policy in Kubernetes?
Answer:

  • Ingress: Functions like an API Gateway, directing external traffic to different services within the cluster based on URL paths or hostnames. It manages how requests are routed to various services.
  • Network Policy: Acts as a firewall rule, controlling the network access between pods within the cluster. It defines what traffic is allowed to and from specific pods, effectively managing internal communication and enhancing security.