Day 6 - Data and Volume with Kubernetes

Volume management with Kubernetes is the process of managing the storage resources that are used by containers in a Kubernetes cluster. Kubernetes supports various types of volumes, such as persistent volumes, ephemeral volumes, and plugins for storing volumes. Each type of volume has different characteristics and use cases.

Persistent volumes (PVs) are storage resources that exist beyond the lifetime of a pod. They can be manually provisioned by administrators or dynamically provisioned by Kubernetes using storage classes. PVs are consumed by pods via persistent volume claims (PVCs), which are requests for storage by users. PVs and PVCs are independent of any specific pod or node, and can be mounted by multiple pods simultaneously.

Ephemeral volumes are storage resources that are bound to the lifetime of a pod. They are deleted when a pod ceases to exist. Ephemeral volumes include emptyDir volumes, which are created when a pod is assigned to a node and can store data that survives container restarts, and hostPath volumes, which allow a pod to access the filesystem of the node that it is running on.

Plugins for storing volumes are extensions that enable Kubernetes to use external or vendor-specific storage systems. Some of the plugins for storing volumes are NFS, which allows a pod to mount an NFS share, CSI, which is a standard interface for container storage, and OpenEBS, which is a cloud-native storage solution that provides local and distributed persistent volumes.

Volume management in Kubernetes involves the following steps.

•  Provisioning: A PV is created in advance by the administrator in static mode, or via a StorageClass provided by the administrator in dynamic mode.

•  Binding: The PV is bound and assigned to a PVC.

•  Using: The container consumes a PV, via the PVC.

•  Releasing: The container releases the PV, removing the PVC.

•  Reclaiming: Kubernetes reclaims the storage resources previously used by the PV.

Type Of Volumes

There are different types of volumes in Kubernetes, which are ways to store and manage data that persists beyond the lifecycle of a pod or a node. Some of the common types of volumes are:

•  local: This type of volume stores data on devices mounted locally to your cluster's nodes. This is useful for applications that need high performance and low latency access to local storage. However, this type of volume is not portable across nodes and may not be suitable for highly available or scalable applications.

•  hostPath: This type of volume stores data within a named directory on a node. This is mainly designed for testing purposes and does not work with multi-node clusters. This type of volume is also not portable across nodes and may cause data loss or corruption if the node fails or is replaced.

•  gcePersistentDisk: This type of volume mounts a Google Compute Engine (GCE) Persistent Disk in your pod. This type of volume is similar to awsElasticBlockStore, but it works with GCE instances and disks. This type of volume also supports ReadWriteMany access mode, which allows multiple pods to read and write to the same disk.

•  emptyDir: An emptyDir volume is a type of volume in Kubernetes that is created when a pod is assigned to a node, and is initially empty. It can be used to store temporary data that is shared among the containers in the same pod, or to provide scratch space for applications that need fast and low-latency access to local storage. However, an emptyDir volume is not persistent, which means that the data is deleted when the pod is removed from the node, or when the node fails or is replaced. Therefore, an emptyDir volume is not suitable for storing data that needs to survive across pod or node failures, or for scaling or migrating applications across nodes.

If you want to use an emptyDir volume in your pod, you need to specify it in the pod's YAML file under the volumes section and give it a name. Then, you need to mount the volume in the containers that need to access it, by referencing the name of the volume in the volume Mounts section and specifying the mount path. 

Further Reading about Volume can be found at Kubernetes Volume.

EmptyDir Volumes Example

Before proceeding here we have below Docker project for which image is already created on DockerHub.

This project simply store data in volume in survive Container restart.

=> User enters information of Items using Curl.
=> Data stored in Volumes
=> Data Pulled using Curl and Container restart operation performed to check data.

{
"name": "app",
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "test",
"license": "ISC",
"dependencies": {
"express": "^4.17.1",
"body-parser": "^1.19.0"
}
}

const path = require('path');
const fs = require('fs');

const express = require('express');
const bodyParser = require('body-parser');

const app = express();

const filePath = path.join(__dirname, 'items', 'text.txt');

app.use(bodyParser.json());

app.get('/items', (req, res) => {
fs.readFile(filePath, (err, data) => {
if (err) {
return res.status(500).json({ message: 'Failed to open file.' });
}
res.status(200).json({ items: data.toString() });
});
});

app.post('/items', (req, res) => {
const newText = req.body.text;
if (newText.trim().length === 0) {
return res.status(422).json({ message: 'Text must not be empty!' });
}
fs.appendFile(filePath, newText + '\n', (err) => {
if (err) {
return res.status(500).json({ message: 'Storing the text failed.' });
}
res.status(201).json({ message: 'Item List was stored!' });
});
});

app.listen(3000); 

version: "3"
services:
items_v:
build: .
volumes:
- items_v:/app/items
ports:
- 80:3000
volumes:
items_v:

FROM node:14-alpine

WORKDIR /app

COPY package.json .

RUN npm install

COPY . .

EXPOSE 3000

CMD [ "node", "app.js" ]

root@aim2022:/home/shreeganesh/Desktop/Docker# docker compose up -d --build                                               
[+] Building 2.0s (11/11) FINISHED                                                                                        
 => [items_v internal] load build definition from Dockerfile     0.0s 
 => => transferring dockerfile: 157B                             0.0s 
 => [items_v internal] load .dockerignore                        0.0s 
...
...
 
root@aim2022:/home/shreeganesh/Desktop/Docker# docker ps -a                                                               
CONTAINER ID   IMAGE      COMMAND      CREATED         STATUS         PORTS                             
      NAMES                                                                                                               
a30d589364bb   docker-items_v   "docker-ent" 3 seconds ago   Up 3 seconds   
0.0.0.0:80->3000/tcp, :::80->3000/ tcp   docker-items_v-1                                                                                                    
root@aim2022:/home/shreeganesh/Desktop/Docker#

Post compiling above source code we can store and fetch records from volume using curl as below.

To post data

root@aim2022:~# curl --location 'localhost/items'
{"items":""}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# curl --location 'localhost/items' \
--header 'Content-Type: application/json' \
--data '{"text" : "Sugar"}'
{"message":"Item List was stored!"}root@aim2022:~# 
root@aim2022:~#

To Fetch data

root@aim2022:~# curl --location 'localhost/items'
{"items":"Teabag\nCup\nSugar\n"}root@aim2022:~# 
root@aim2022:~#

Now let we restart container and check if data persist.

root@aim2022:/home/shreeganesh/Desktop/Docker# docker compose down                                                        
[+] Running 2/2                                                                                                           
 ✔ Container docker-items_v-1  Removed                                       10.3s 
 ✔ Network docker_default      Removed                                       0.5s 
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            

Let we restart container as below.
 
root@aim2022:/home/shreeganesh/Desktop/Docker# docker compose up -d --build                                               
[+] Building 1.9s (11/11) FINISHED                                                                                        
 => [items_v internal] load build definition from Dockerfile                0.0s 

Now check data if still persists as below

root@aim2022:~# curl --location 'localhost/items'
{"items":"Teabag\nCup\nSugar\n"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~#

So as per practice data persist post container restart.

Now we will inegrate the same Docker project in kubenetes and see volumes working for the same as below.

=> Let we build image and push to repository -Docker Hub as below.

root@aim2022:/home/shreeganesh/Desktop/Docker# docker build -t abhishek2023/kube-testing:2.0 .                            
[+] Building 7.3s (11/11) FINISHED                                               docker:default 
 => [internal] load build definition from Dockerfile                             0.0s 
 => => transferring dockerfile: 157B                                             0.0s 
 => [internal] load .dockerignore                                                0.0s 
 => => transferring context: 2B                                                  0.0s 
...
...
 => => writing image sha256:d92dda82e9d99ac04d868eb9354281df87a78a34afdf5a7d4bea0d036488c9c5    0.0s 
 => => naming to docker.io/abhishek2023/kube-testing:2.0                                        0.0s 
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            
root@aim2022:/home/shreeganesh/Desktop/Docker# docker images                                                              
REPOSITORY                  TAG       IMAGE ID       CREATED          SIZE                                                
abhishek2023/kube-testing   2.0       d92dda82e9d9   6 seconds ago    124MB                                               
docker-items_v              latest    b3b04cc9e328   31 minutes ago   124MB                                               
abhishek2023/kube-testing   1.0       24919abfb315   9 days ago       124MB                                               
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            

root@aim2022:/home/shreeganesh/Desktop/Docker# docker images                                                              
REPOSITORY                  TAG       IMAGE ID       CREATED          SIZE                                                
abhishek2023/kube-testing   2.0       d92dda82e9d9   54 seconds ago   124MB                                               
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            
root@aim2022:/home/shreeganesh/Desktop/Docker#                                                                            
root@aim2022:/home/shreeganesh/Desktop/Docker# docker push abhishek2023/kube-testing:2.0                                  
The push refers to repository [docker.io/abhishek2023/kube-testing]                                                       
030a3222236a: Pushed ...
 
root@aim2022:/home/shreeganesh/Desktop/Docker#

Now we need to prepare Deployment.yaml and Service.yaml file that will be used for Kubernetes as below.

service.yaml

apiVersion: v1
kind: Service
metadata:
name: items-service
spec:
selector:
app: items
type: LoadBalancer
ports:
- protocol: "TCP"
port: 80
targetPort: 3000

 deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: items-deployment
spec:
replicas: 1
selector:
matchLabels:
app: items
template:
metadata:
labels:
app: items
spec:
containers:
- name: items
image: abhishek2023/kube-testing:2.0 

shreeganesh@aim2022:~$ minikube status                                                                                    
minikube                                                                                                                  
type: Control Plane                                                                                                       
host: Running                                                                                                             
kubelet: Running                                                                                                          
apiserver: Running                                                                                                        
kubeconfig: Configured                                                                                                    
                                                                                                                          
shreeganesh@aim2022:~$                                                                                                                                                                                                     
shreeganesh@aim2022:~$ kubectl get deployment                                                                             
No resources found in default namespace.                                                                                  
shreeganesh@aim2022:~$                                                                                                    
shreeganesh@aim2022:~$ kubectl get pods                                                                                   
No resources found in default namespace.                                                                                  
shreeganesh@aim2022:~$                                                                                                    
shreeganesh@aim2022:~$                                                                                                    
shreeganesh@aim2022:~$ kubectl apply -f=service.yaml -f=deployment.yaml                                                   
the path "service.yaml" does not exist                                                                                    
the path "deployment.yaml" does not exist                                                                                 
shreeganesh@aim2022:~$ cd /home/shreeganesh/Desktop/Docker                                                                
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$ kubectl apply -f=service.yaml -f=deployment.yaml                                    
service/items-service created                                                                                             
deployment.apps/items-deployment created                                                                                  
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$ kubectl get pods                                                                    
NAME                                READY   STATUS              RESTARTS   AGE                                            
items-deployment-66f9bbc898-rwzm2   0/1     ContainerCreating   0          5s                                             
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$ kubectl get deployments                                                             
NAME               READY   UP-TO-DATE   AVAILABLE   AGE                                                                   
items-deployment   1/1     1            1           12s                                                                   
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$ kubectl get services                                                                
NAME            TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE                                             
items-service   LoadBalancer   10.99.52.35        80:32727/TCP   17s                                             
kubernetes      ClusterIP      10.96.0.1             443/TCP        113d                                            
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$                                                                                     
shreeganesh@aim2022:~/Desktop/Docker$ minikube service items-service                                                      
|-----------|---------------|-------------|-----------------------------|                                                 
| NAMESPACE |     NAME      | TARGET PORT |             URL             |                                                 
|-----------|---------------|-------------|-----------------------------|                                                 
| default   | items-service |          80 | http://192.168.59.100:32727 |                                                 
|-----------|---------------|-------------|-----------------------------|                                                 
🎉  Opening service default/items-service in default browser...                                                           


root@aim2022:~# curl --location 'http://192.168.59.100:32727/items'
{"items":"New Cup\n"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# curl --location 'http://192.168.59.100:32727/items' \
--header 'Content-Type: application/json' \
--data '{ "text" : "New Mug"}'
{"message":"Item List was stored!"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# curl --location 'http://192.168.59.100:32727/items'
{"items":"New Cup\nNew Mug\n"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~#

Till now we haven't used any Kubernetes spsecific volume configuration. Our project is working in Kubernetes cluster as of now.

Let we proceed to implement EmptyDir Volume as below.

To embed emptyDir we will need to update deployment.yaml file as below. And post that we need to rebuild the deployment.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: items-deployment
spec:
replicas: 1
selector:
matchLabels:
app: items
template:
metadata:
labels:
app: items
spec:
containers:
- name: items
image: abhishek2023/kube-testing:2.0
volumeMounts:
- mountPath: /app/items
name: items-volume
volumes:
- name: items-volume
emptyDir: {}

shreeganesh@aim2022:~/Desktop/Docker$ kubectl apply -f=deployment.yaml                                                    
deployment.apps/items-deployment configured 
 

Post applying file let we try to put and get data as below.

root@aim2022:~# curl --location 'http://192.168.59.100:32727/items' --header 
'Content-Type: application/json' --data '{ "text" : "New Mug"}'
{"message":"Item List was stored!"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~# curl --location 'http://192.168.59.100:32727/items'
{"items":"New Mug\n"}root@aim2022:~# 
root@aim2022:~# 
root@aim2022:~#   
 So we can see we are able to put and get data using emptyDir.  

 

No comments:

Post a Comment

Subscribe to: Posts (Atom)

Total Pageviews