| path | title |
|---|---|
/learnings/kubernetes |
Learnings: Kubernetes |
- Learning Kubertetes >
- Book Recommendations
- Master <-- contains etcd, api server, scheduler
- note: scheduler may move "running" containers to reorg the cluster
- workers <-- kublet, kube-proxy, container runtime
Pod <-- set of co-located containers. A worker instance runs many pods. List pods, not containers.
A pod template can define how many replica pods to create for a container(s).
Seems like people conflate the word "pod" to mean, "pod template" and "pod instance" ????? The former is what you'd feed to ReplicationController / ReplicaSet, and the latter is what you get when you call kubectl get pods.
- (if asked?) all containers running same service can "share" same IP or DNS name <-- controlled via environmental variables
Performed via DNS or environmental variables.
If running on supported cloud platforms, can actually request additional instances in cluster if load scaling needs > current cluster size
(hmm)
download
minikube start <-- might take a while
Then install K8 client and run kubectl
kubectl cluster-info <-- should return useful info like DNS name to dashboard
minikube dashboard <-- opens up dashboard in your default browser
"Just" sends REST commands to master.
kubectl has bash and zsh tab completions!!
kubectrl get -o <-- -o will output the YAML description of that print Object
kubectrl edit RESOURCE NAME <-- will pop open your editor, and then when save/quit it will post the file back to the API server.
kubectrl exec <--- how to get into the container(s) of a pod.
$ kubectl cluster-info # <-- information about the current default cluster / is a good configuration check
$ kubectl get services # <-- gets the services aka the routing rules
$ kubectl get pods # <-- gets the deployed pods and information about them (for the services)
$ kubectl logs $PODNAME # <-- logs from the thing
Q: What cluster is my default?
A: kubectl config view | grep current-context will get you close / to the next part of the YAML to read
kubectl get pods,deployments # list TWO resources at a time
kubectl run -it—image=$SOMEDOCKERIMAGE—restart=never /bin/sh
(But you will need to delete the pod when you exit the container, k8s will not do thst cleanup for you )
Configuration often done by YAML (and behind the scenes kubectl is POSTing it into)... but can also mbe JSON.
Think of YAML as a representation of the request body <-- RESTful, and not just on outputted document formats!!!
Each pod gets own internal IP addresss.
Pods also = instances of container or group of containers.
Does NOT span worker instances.
Can tell pod to do port forwarding without using a load balancer.
Can use labels on (pods, workers) to act on things as a bunch, or schedule based on capabilities of workers instances (big GPUs, etc).
Can also namespace objects.
Can also have init containers that run before the other containers ie to lay some required files and stuff down.
Can be configured with a volume that is an auto-checked-out git repo. (just not a private one! Use a git sidecar or something better like putting the files in the container...)
Remember if you set a volume to be a folder on the node, if the node goes away or gets rescheduled you won't have that data. Use your cloud providers elastic block storage long term file systems for that.
it also knows about Amazon EBS and Azure's disk/file stuff too.
can use a PersistentVolumeClaim to use a PersistentVolume set up by your k8 cluster admin.
NOTE: the location etc of this volume is the same across all of the instances of that pod template. This may or may not be a good idea (databases).
on init k8 sets environmental variables pointing to each service that exists in the moment. (ie if service is called odyssey-database ODYSSEY_DATABASE_SERVICE_HOST and ODYSSEY_DATABASE_SERVICE_PORT).
OR
provides a DNS server: odyssey-database.default.svc.cluster.local
^^^ default above is the k8 namespace the service is in!!!
(svc.cluster.local is the default domain in the pods, so you could refer to it by domain name of only odyssey-database)!!
- ping won't work <-- because of the virtual IP magic.
- can also use this to abstract where resources live: maybe even outside the k8 cluster!!!
$ kubectl expose replicationcontroller MYREPLICATIONCONTROLLER --type=
Runs a pod but don't restart the container when the container quits success. (failure causes re-schedule).
Can have parallelism and number of completions
Also can configure CronJobs.
Remember that each pod (instance) has its own IP address. Need to use load balancer so other services can address the pods as if they are one.
AND/OR you want a consistent IP address for a pod - ie a single IP for your database pod, regardless of pods moving around as they die or are scheduled away.
(can also tell load balancer to just return IP addresses of a pod, instead of using the one external one. It will return multiple A records, so if you're good with DIG you can get all the IP addresses of all the pod instances you have). (set clusterIP to None).
Creating load balancer will not create an external IP for you, but you can use kubectl describe MY-LOAD-BALANCER-SERVICE to get the port it's running on, then just use that port.
MEANING: because Minikube doesn't support creating external IP addresses it WILL act like it's super class, NodePort.
Keep-Alive will just send to the same pod as it used the first time...
Instead of needing to set up a public load balancer that sits in front of your k8 loadbalancer service, can create an Ingess service. This will look at PATH or domain name to figure out where to send the request. Also supports cookies as it works on Layer 7.
Ingress load balancers also terminate SSL traffic.
There’s also ClusterIp and NodePort kinds. see (NodePort Vs ClusterIP Vs LoadBalancer Vs Ingress)[https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0]
Want to create ReplicationController objects to manage pods - just creating pods directly will assign them to a node directly, and if the node goes down your pod will not be recreated.
Pods created by an application controller aren't tied to their creating replication controller.
Give ReplicationController a set of labels to manage pods that have that label.
You can relabel a pod to make the replication controller spin up a new one, then you could (say) examine it for what went wrong.
Technically deprecated, prefer ReplicaSets and/or Deployments instead
Runs one replica of a pod on every worker instance.
Creates ReplicaSets, but takes care of (rolling, or however you configure them) deployments.
Also provides rollout undo option!
Can do pause to halt rollback ie for canary deploys.
kubectl get deployments -a # get all deploys across cluster
kubectl describe deployment $deploymentname
kubectl get svc
kubectl describe service $servicename
Like ReplicaSets, but can:
- preserve network identity (well... not the IP...)
- have seperate storages for each pod (instance)
- boot up one at a time
To use you need a Service (at least). Need to create headless.
StatefulSet instances are found by SRV DNS entries.
When stateful pods fail need to have an admin go in and delete the pod - then another (identical!) pod will take its place.
Have to set for each container in the pod (no such thing as pod wide settings).
Can also pass info about the container downward.
<<Kubernetes_Environmental_Configuration>>
Can use env block and create variables that looks at metadata.namespace, for example. K8 will introspect and pass that info downward.
Can even pass cluster name, etc etc downward.
NOTE: can NOT use environmental variables for labels or annotations - these must be written to injected volumes. (because these may change!)
Can create ConfigMap resources to hold config. Can reference these variables in env setting (special keys for this) one by one, have k8 put all of them in there (prefixed by something), use it to override args entry for container. Can also have mounted volume created to hold file version of these so your app can read it (including just mounting specific files so you don't blow away real directory mounting a Docker volume into /etc!)
Yup, k8 has them (essentially as a subclass of ConfigMap).
Integration with Vault via K8 Service Account Token
See also:
- Learning_Ops_Vault
- A sample Rails app with secrets, K8, and Vault
Use a secret to do this! Builtin!
kubectl get configmap $artifactName -o yaml
Without the -o yaml part you'll just see the name of the configmap, which is not super helpful.
Run before the main pod is launched. Needs to execute its task then quit before the main container is launched (not for long running tasks.)
Different file system from main container.
Can have multiple.
- Pull down a bash image to lay down some infrastructure / adjust something. But you don't want bash in your main container because of size limits... Then just do your work in the
commandpart of theinitContainerin your PodSpec.
container can define commands to run as part of lifecycle:
preStart<-- may be called before, or after, ENTRYPOINTpreStop<-- K8's management of container will block, also must execute before the grace period happens (race!)
Not much difference from regular containers, beyond the fact that these are not long running. So you have to be quick or your debug information will just go away.
Steps:
-
use
kubectl get podsto wait for a pod named YOUR-THING-init-ADJECTIVE-NOUN-NUMBERS-AND-LETTERS -
When that shows up use that name to do
kubectl describe pod -
kubectl describe podwill tell you what stage the container is in (ie downloading your docker image, booting the thing up, etc etc) -
When it looks up, do
kubectl logson that name. Logs should spew forth from the container. -
Eventually the init container will go away and be replaced with the non-init version of the container
Kubernetes shuts down your app with the following process:
- Will send
SIGTERM - Will wait (grace period)
- Will send
SIGKILL - container removed from load balancers
See also
- Graceful shutdown in k8
- Learning about k8 and Unix Signals
- Learning_Ops_Docker_PID1_Signals
Create YAML describing your object (pod, ReplicationController, etc) and POST it to the API server.
Called "livenessProbe" in k8 yaml.
Can set path, port, and initial delay seconds (remember to give your app time to boot!)
Q: What about environments ie Kakfa where your broker string needs/wants to include all the addresses of all the brokers (to communicate with each directly-ish?)
A: Create a Kafka pod and run multiple instances of it. Tell the broker strings about those IP addresses.
"Hey, I need a PostgreSQL install, does one exist Out There? If so go grab me it and instantiate it on the cluster"
So like Heroku addons(??)
Helm: (from the deis people): create a "chart" which gets turned into a pod (eerrr.... tiller) which gets deployed.
- [REVIEW]: worth looking into when thinking about making this stuff developer repeatable????
Are you on GCP? You might want to read Google_Cloud_Platform_Kubernetes_Engine