Make large static data files available to kubernetes pods - file

I have a few quite large UTF-8 data files that pods need to load into memory on start up - from a couple of hundred KBs to around 50 MB.
The project (including helm chart) is open source but some of these files are not - otherwise I would probably just include them in the images. My initial thinking was to create configmaps but my understanding is that 50 MB is more than configmaps were intended for, so that might end up being a problem in some circumstances. I think secrets would also be overkill - they aren't secret, they just shouldn't be put on the open internet.
For performance reasons I'd rather have a copy in memory in each pod rather than going for a shared cache but I might be wrong on that. At the very least that will likely add more complexity than it's worth.
Are configmaps the way to go?

From my point of view, the best solution would be using init container to download the files from a secured storage (as it was mentioned by Andrew Savinykh in the comments), to the pod's volume and then use it in the pod's main container.
Please see the example:
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
# These containers are run during pod initialization
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://kubernetes.io
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}

Related

What are the correct /etc/exports settings for Kubernetes NFS Storage?

I have a simple NFS server (followed instructions here) connected to a Kubernetes (v1.24.2) cluster as a storage class. When a new PVC is created, it creates a PV as expected with a new directory on the NFS server.
The NFS provider was deployed as instructed here.
My issue is that containers don't seem to be able to perform all the functions they expect to when interacting with the NFS server. For example:
A PVC and PV are created with the following yml:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mssql-data
spec:
storageClassName: nfs-client
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
This creates a directory on the NFS server as expected.
Then this deployment is created to use the PVC:
apiVersion: apps/v1
kind: Deployment
metadata:
name: mssql-deployment
spec:
replicas: 1
selector:
matchLabels:
app: mssql
template:
metadata:
labels:
app: mssql
spec:
terminationGracePeriodSeconds: 30
hostname: mssqlinst
securityContext:
fsGroup: 10001
containers:
- name: mssql
image: mcr.microsoft.com/mssql/server:2019-latest
ports:
- containerPort: 1433
env:
- name: MSSQL_PID
value: "Developer"
- name: ACCEPT_EULA
value: "Y"
- name: SA_PASSWORD
value: "Password123"
volumeMounts:
- name: mssqldb
mountPath: /var/opt/mssql
volumes:
- name: mssqldb
persistentVolumeClaim:
claimName: mssql-data
The server comes up and responds to requests but does so with the error:
[S0002][823] com.microsoft.sqlserver.jdbc.SQLServerException: The operating system returned error 1117(The request could not be performed because of an I/O device error.) to SQL Server during a read at offset 0x0000000009a000 in file '/var/opt/mssql/data/master.mdf'. Additional messages in the SQL Server error log and operating system error log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.
My /etc/exports file has the following contents:
/srv *(rw,no_subtree_check,no_root_squash)
When the SQL container starts, it doesn't undergo any container restarts but the SQL service within the container appears to get into some sort of restart loop until a connection is attempted and then it throws the error and appears to stop.
Is there something I'm missing in the /etc/exports file? I tried variations with sync, async, and insecure but can't seem to get past the SQL error.
I gather from the error that this has something to do with the container's ability to read/write from/to the disk. Am I in the right ballpark?
The config that ended up working was:
/srv *(rw,no_root_squash,insecure,sync,no_subtree_check)
This was after a reinstall of the cluster. No significant changes elsewhere but still seems like there may have been more to the issue than this one config.

How to override env variables in React JS application with Kuberenetes?

I've a simle React JS application and it's using a environment variable(REACT_APP_BACKEND_SERVER_URL) defined in .env file. Now I'm trying to deploy this application to minikube using Kubernetes.
This is my deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-ui
spec:
replicas: 1
selector:
matchLabels:
app: test-ui
template:
metadata:
name: test-ui-pod
labels:
app: test-ui
spec:
containers:
- name: test-ui
image: test-ui:1.0.2
ports:
- containerPort: 80
env:
- name: "REACT_APP_BACKEND_SERVER_URL"
value: "http://127.0.0.1:59058"
When I run the application, it's working but REACT_APP_BACKEND_SERVER_URL is giving the value which I defined in .env file. Not the one I'm overriding. Can someone help me with this please? How to override the env variable using Kubernetes deployment?
After starting the app with your deployment YAML and checking for the environment variables I see the environment variables for that environment variable.
REACT_APP_BACKEND_SERVER_URL=http://127.0.0.1:59058
you can check that by doing an kubectl exec -it <pod-name> -- sh and running env command.
So you can see that REACT_APP_BACKEND_SERVER_URL is there in the environment variables. It's available for your application to use. I suspect that you may need to understand better from the React app side on how to use the .env file.

Erratic and slow behavior with create-react-app and Skaffold kubernetes

I have Skaffold working well with local development server and database deployments. I'm trying to get working on the create-react-app front-end, but the behavior is incredibly slow and erratic.
Issues
The main problems are the following:
It takes upwards of five minutes from running skaffold dev --port-forward --tail for it finally start spinning up. Running just a docker build takes less than 30 seconds.
When it finally starts spinning up, it just sits on Starting the development server... for another two minutes.
Then, nine times out of ten, I get the following errors after several minutes (there are three because that is how many replicas there are):
One out of ten times, it will actually go into the Compiled Successfully! You can now view in the browser. It never does launch in Chrome though.
Changes to JS in create-react-app are never reflected in new browser. You have to stop and run Skaffold again. Skaffold does say Syncing 1 files for <image>... Watching for changes..., but nothing changes even after a refresh.
What I've tried
I've really simplified what I'm trying to do to make it easier to sort this out, so I'm using just an OOTB create-react-app application. The behavior is the same regardless.
minikube delete and minikube start several times (did this because even the server deployment started acting erratically after trying create-react-app)
Code and Steps to Reproduce
I'm on macOS Mojave (10.14.6) using Docker for Mac, Kubernetes (v1.16.0), minikube (v1.4.0), Skaffold (v0.39.0), and create-react-app. I'll have to skip the installation process for all of these since it is fairly lengthy, so the following steps assume you have this already setup.
Make a project directory:
mkdir project
Make a Kubernetes manifest directory and move into it:
mkdir k8s && cd k8s
Make a client-deployment.yaml and add the following:
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-deployment
spec:
replicas: 3
selector:
matchLabels:
component: web
template:
metadata:
labels:
component: web
spec:
containers:
- name: client
image: testapp/client
ports:
- containerPort: 3000
Make a client-cluster-ip-service.yaml and add the following:
apiVersion: v1
kind: Service
metadata:
name: client-cluster-ip-service
spec:
type: ClusterIP
selector:
component: web
ports:
- port: 3000
targetPort: 3000
Move back into the parent:
cd ..
Create a skaffold.yaml and add the following:
apiVersion: skaffold/v1beta15
kind: Config
build:
local:
push: false
artifacts:
- image: testapp/client
context: web
docker:
dockerfile: Dockerfile.dev
sync:
manual:
- src: "**/*.js"
dest: .
- src: "**/*.html"
dest: .
- src: "**/*.css"
dest: .
deploy:
kubectl:
manifests:
- k8s/client-deployment.yaml
- k8s/client-cluster-ip-service.yaml
portForward:
- resourceType: service
resourceName: client-cluster-ip-service
port: 3000
localPort: 3000
Start a new create-react-app project:
npx create-react-app test-app
Change into the directory:
cd test-app
Create a Dockerfile.dev and add the following:
FROM node:alpine
WORKDIR '/app'
EXPOSE 3000
CMD ["npm", "run", "start"]
COPY package* ./
RUN npm install
COPY . .
Create a .dockerignore file and add the following:
node_modules
*.swp
Go back into the parent directory:
cd ..
Make sure minikube is running:
minikube start
Run the skaffold.yaml:
skaffold dev --port-forward --tail
This is what produces the issues for me.
Ok. Disregard. Started with one replica and it worked fine. Two worked fine. Three worked if skaffold was already running, but not from a fresh skaffold dev --port-forward --tail.
skaffold ssh and then did a top. Was running out of RAM... well was at 86% utilization. Increased it from the default 2GB to 8GB and now it works fine.
First deleted the VM with minikube delete and then created a new one with minikube start --memory='8g'. All good now.

Where is the cassandra.yaml location on the MAC?

I am working on the docker environment, and executed docker exec -it mycassandra cqlsh. Then, I am inserting the data, and it is occurring the following error:
WriteTimeout - Error from server: code=1100
By this, it tells me that I need to find out the cassandra.yaml document and amend the write-time, but I can not find that on my MAC.
Could you tell me how can I find it and how to amend the document?
Thanks.
for those who have installed it as brew install cassandra, yaml file will located in usr/local/etc/cassandra
If you are running the official cassandra image then cassandra.yaml may be found at /etc/cassandra/cassandra.yaml in the container. If you want to create a custom cassandra.yaml file then you may try to overwrite it in your Dockerfile or docker-compose.yml file. For example, in my docker-compose.yml file I have something like:
services:
cassandra:
image: cassandra:3.11.4
volumes:
- ./cassandra.yaml:/etc/cassandra/cassandra.yaml
which causes the cassandra.yaml file in the container to be overwritten by my local cassandra.yaml.
I hope this helps.
From the provided example, it seems that the database is executed from inside a container. So the cassandra.yaml that you're looking for will be created on the fly when the container is started up, based on the configuration that you provided.
We have set Cassandra Containers with Kubernetes, and execute them in docker, based on the instructions found here, and been able to modify the settings of the cassandra.yaml file in the configuration of the statefulset, updating the variables in env for the spec of the container.
For example, to modify the seeds list, the cluster name, and the rack of the C* cluster named c-test-qa:
apiVersion: apps/v1
kind: StatefulSet
...
spec:
serviceName: c-test-qa
replicas: 1
selector:
matchLabels:
app: c-test-qa
template:
metadata:
labels:
app: c-test-qa
spec:
containers:
- name: c-test-qa
image: cassandra:3.11
imagePullPolicy: IfNotPresent
...
env:
- name: CASSANDRA_SEEDS
value: c-test-qa-0.c-test-qa.qa.svc.cluster.local
- name: CASSANDRA_CLUSTER_NAME
value: "testqa"
- name: CASSANDRA_RACK
value: "DC1"
- name: CASSANDRA_RACK
value: "CustomRack1"
...
On MacOS :
It will be found in either of the below locations:
Cassandra package installations: /etc/cassandra
Cassandra tarball installations: install_location/conf
DataStax Enterprise package installations: /etc/dse/cassandra
DataStax Enterprise tarball installations: install_location/resources/cassandra/conf

How to provision persistent volume claim for software install in kubernetes

I am trying to provision PVC for Solr deployment in k8s and mount it as /opt/solr, which is default Solr installation directory. This way I plan to target both Solr installation and data under it on PVC. However, while storage gets provisioned just fine and statefulset gets created, my deployment doesn't work because /opt/solr ends up empty. What is a proper way to do it? Here my deployment.yaml:
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: solr
labels:
app: solr
spec:
volumeClaimTemplates:
- metadata:
name: datadir
annotations:
volume.alpha.kubernetes.io/storage-class: slow
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 2Gi
serviceName: solr-svc
replicas: 1
template:
metadata:
labels:
app: solr
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- solr-pool
topologyKey: "kubernetes.io/hostname"
terminationGracePeriodSeconds: 300
containers:
- name: solr
image: solr:6.5.1
imagePullPolicy: IfNotPresent
resources:
requests:
memory: 512M
cpu: 500m
ports:
- containerPort: 8983
name: solr-port
protocol: TCP
env:
- name: VERBOSE
value: "yes"
command:
- bash
- -c
- "exec /opt/solr/bin/solr start"
volumeMounts:
- name: solr-script
mountPath: /docker-entrypoint-initdb.d/
- name: datadir
mountPath: /opt/solr/
volumes:
- name: solr-script
configMap:
name: solr-configs
nodeSelector:
pool: solr-pool
Provisioned storage is empty by default and there might be a Deleting retain policy on provisioned storage be sure to check those configurations. You can also exec to your pod and examine mounted volume and see if it's working properly or not (permission issues, read only file system)
In may case there was a conflict with docker container configuration which used /opt/solr as a location for solr install and my attempt to mount separate PV under same location. Once this PV is mounted obviously I loose solr install. The fixes for this are:
create another docker image which uses separate location
change solr config to use different location for data
change PV volume location

Resources