I'm very new to docker and docker-compose things. I have tried to use Kiwi TCMS open source project which supposed to use with Docker.
My question is can I run the projects on docker on the same server, I'm supposed to configure my development and production sites on the same server (CentOS)?
I'm following the below link to install docker and configure the kiwitcms application for the first time, I read the basics about the docker and how its working and all.
https://kiwitcms.readthedocs.io/en/latest/installing_docker.html#
I want to use PostgreSQL as my Database, but the existing latest docker image has MariaDB. So after I pulled latest version of kiwitcms from docker hub using the following command,
docker pull kiwitcms/kiwi
Should I change to "docker-compose.yml" file db image value, and save it to a local new directory,
https://raw.githubusercontent.com/kiwitcms/Kiwi/master/docker-compose.yml
[![enter image description here][1]][1]
Also there can we edit use our own DB name, user name passwords right?
then execute the following command.
docker-compose up -d
I'm very new to this model and read most of the articles related to docker, but this leads me few of these doubts, hence asking for these clarifications.
Thanks,
Karthik.
The web app and the DB server are 2 different images (aka 2 different servers). For example how we use Postgres in testing see:
https://github.com/kiwitcms/Kiwi/blob/master/docker-compose.postgres
Related
this is my first deployment, I have gone so far as to create and configure Ubuntu instance on lightsails following official guide. However, I run into problems when instead of an empty project I use my own Django hosting React frontend.
My project works on my computer and if I start up django's production server thorugh manage.py runserver but does not run at all if I use gunicorn to run it with:
gunicorn my_app.wsgi --bind 0.0.0.0:8000 --workers 4
It seems that it cannot find proper filepaths of any static files but I do not undertstand how to configure it to use it since Django already has all the right paths.
One of the errors in the browser console:
Refused to execute http://3.127.76.103/static/rest_framework/js/bootstrap.min.js as script because "X-Content-Type-Options: nosniff" was given and its Content-Type is not a script MIME type.
How can I start fixing it, I do not understand why it does not see the any files.
I have been running Kiwi TCMS as a Docker container, the version running is 9.0.1. Now I have seen the latest version 10.0, so like to update it. Not sure this will not break the existing data.
So, I have taken the database and uploads files backup by following it,
https://kiwitcms.org/blog/atodorov/2018/07/30/how-to-backup-docker-volumes-for-kiwi-tcms/
But, how can I downgrade to an older version image, if the latest version does not work as expected or any problem comes? I found the docker tag concept and not sure how to apply it?
Can you confirm following will restore to an older version?
Restore the DB files and uploaded files that already taken backup.
Change the image in docker-compose file to an older one? Like my case, I could want to pull the version 9.0.1 image, so changing the image: kiwitcms/kiwi:9.0.1 works?
web:
container_name: kiwi_web
depends_on:
- db
restart: always
image: kiwitcms/kiwi:9.0.1
Or how should I tag version 9.0.1, and then restore the tagged version, in case of reverting the latest upgrade?
But, how can I downgrade to an older version image, if the latest version does not work as expected or any problem comes? I found the docker tag concept and not sure how to apply it?
The easiest way would be to destroy the instance you were not happy with, start a new one (with v9.0.1) from scratch and restore the files/DB from your backup. I am assuming you made the backup before the upgrade.
image: kiwitcms/kiwi:9.0.1 works?
In theory that would work but in practice it won't because Kiwi TCMS upstream doesn't ship version tagged container images. See https://kiwitcms.org/#subscriptions, lines Only latest releases vs. Versioned releases from kiwitcms/version.
Or how should I tag version 9.0.1
Depending on what you've already done or not done this could either be possible or not possible:
Execute docker images and if you see something like
kiwitcms/kiwi latest 9f23186803d5 8 weeks ago 736MB
kiwitcms/kiwi <no tag> <another image hash> ...
this means that latest points to v10 while the untagged image is your previous image, 9.0.1 in your case. You can then use docker tag command to tag it and make use of it inside docker-compose.yml file.
If you only see a kiwitcms/kiwi:latest image this means you have removed the previous one from your system. You don't have access to it. Your options are:
Rebuild that from source using the same git tag or
Purchase a subscription to be able to access the private kiwitcms/version repository if it is critical to restore back to the old version.
As a good practice you can follow these 2 advices:
Use the "restore-from-backup" technique or maybe "clone the docker image" so that you are upgrading and experimenting with a different Kiwi TCMS instance, not the main one that you use. If you are happy with the latest version then you can upgrade the main instance. Note here that by "clone the docker image" I mean make the running image/container files available on a different machine or possibly the same host machine but launch a secondary Kiwi TCMS instance + a secondary DB. ATM I'm not sure how exactly to do this but either docker itself or other Linux tools should be able to help here.
docker tag your local kiwitcms/kiwi images when you download/deploy/upgrade them so that you can have a reference to the version at this point in time. This will avoid losing the older image.
Extra tip: with docker tag you can also provide your own customized image names and possibly upload these images to your own docker repository for later access/backup or distributing them between multiple host systems.
I've been studying "Kubernetes Up and Running" by Hightower et al (first edition) Chapter 13 where they discussed creating a Reliable MySQL Singleton (Since I just discovered that there is a second edition, I guess I'll be buying it soon).
Using their MySQL reliable singleton example as a model, I've been looking for some sample YAML files to make a similar deployment with Microsoft SQL Server (Express) on Docker Desktop for Kubernetes.
Apparently I need YAML files to deploy
Persistent Volume
Volume claim (should this be NFS?)
SQL Server (Express edition) replica set (in spite of the fact that this is just a singleton).
I've tried this example but I'm confused because it does not contain a persistent volume & claim and it does not work. I get the error
Error: unable to recognize "sqlserver.yml": no matches for kind "Deployment" in version "apps/v1beta1"
Can someone please point me to some sample YAML files that are not Azure specific that will work on Docker Desktop Kubernetes for Windows 10? After debugging my application, I'll want to deploy this to Azure (AKS).
Wed Jul 15 2020 Update
I left out the "-n namespace" for the helm install command (possibly because I'm using Helm and you are using helm v2?).
That install command still did not work. Then I did a
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
Now this command works:
helm install todo-app-database stable/mssql-linux
Progress!
When I do a "k get pods" I see that my todo-app-mssql-linux database is in the pending state. So I did a
kubectl get events
and I see
Warning FailedScheduling pod/todo-app-database-mssql-linux-8668d9b88c-lsh5l 0/1 nodes are available: 1 Insufficient memory.
I've been google searching for "Kubernetes insufficient memory" and can find no match.
I suspect this is a problem specific to "Docker Desktop Kubernetes".
When I look at the output for
helm -n ns-todolistdemo template todo-app-database stable/mssql-linux
I see the deployment is asking for 2Gi. (Interesting: when I use the template command, the "-n ns-todolistdemo" does not cause an error like it does with the install command).
So I do
kubectl describe deployment todo-app-database-mssql-linux >todo-app-database-mssql-linux.yaml
I edit the yaml file to change 2Gi to 1Gi.
kubectl apply -f todo-app-database-mssql-linux.yaml
I get this error:
error: error parsing todo-app-database-mssql-linux.yaml: error converting YAML to JSON: yaml: line 9: mapping values are not allowed in this context
Hmm... that did not work. I try delete:
kubectl delete deployment todo-app-database-mssql-linux
kubectl create -f todo-app-database-mssql-linux.yaml
I get this error:
error: error validating "todo-app-database-mssql-linux.yaml": error validating data: invalid object to validate; if you choose to ignore these errors, turn validation off with --validate=false
So I try apply:
kubectl apply -f todo-app-database-mssql-linux.yaml
Same error!
Shucks.... Is there a way to adjust the memory allocation for Docker Desktop?
Thank you
Siegfried
Short answer
https://github.com/helm/charts/blob/master/stable/mssql-linux/templates/pvc-master.yaml
Detailed Answer
Docker For Desktop comes already with a default StorageClass :
This storage class is responsible for auto-provisioning of PV whenever you create a PVC.
If you have a YAML definition of PVC (persistent volume claim), you just need to keep storageClass empty, so it will use the default.
k get storageclass
NAME PROVISIONER AGE
hostpath (default) docker.io/hostpath 11d
This is fair enough as Docker-For-Desktop Cluster is a one node cluster. So if your DB crashes and the cluster opens it again , it will not move to another node, because simply, you have a single node :)
Now should write the YAML of PVC from scratch ?
No , you don't need. Because Helm should be your best friend.
( I explained below Why you have to use Helm even without deep learning curve)
Fortunately, the community provides a chart called stable/mssql-linux..
Let's run it together :
helm -n <your-namespace> install todo-app-database stable/mssql-linux
# helm -n <namespace> install <release-name> <chart-name-from-community>
If you want to check the YAML (namely PVC) that Helm computed, you can run template instead of install
helm -n <your-namespace> template todo-app-database stable/mssql-linux
Why I give you the answer with Helm ?
Writing YAML from scratch lets reinventing the wheel while others do it.
The most efficient way is to reuse what community prepare for you.
However, you may ask: How can i reuse what others doing ?
That's why Helm comes.
Helm comes to be your installer of any application on top of kubernetes regardless how much YAML does your app require.
Install it now and hit the ground choco install kubernetes-helm
I'm trying to run the project in Jhipster based on this tutorial by the creator himself :https://www.youtube.com/watch?v=d1MEM8PdAzQ but it can't connect to Postgres
See errors below:
Caused by: org.postgresql.util.PSQLException: The server requested password-based authentication, but no password was provided.
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:473)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:203)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:65)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:146)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:35)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:47)
at org.postgresql.jdbc42.AbstractJdbc42Connection.<init>(AbstractJdbc42Connection.java:21)
at org.postgresql.jdbc42.Jdbc42Connection.<init>(Jdbc42Connection.java:28)
at org.postgresql.Driver.makeConnection(Driver.java:415)
at org.postgresql.Driver.connect(Driver.java:282)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:316)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:518)
How do I connect Jhipster with postgreSQL? I am a newbie on jhipster
JHipster creates 3 configuration files: -
application.yml - main Spring Boot configuration file.
application-dev.yml
application-prod.yml
The application.yml file contains common properties, the other 2 hold specific properties to development and production environments.
If you look at application-dev.yml you'll see something like the following: -
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:postgresql://localhost:5432/myapp
username: myapp
password:
However, you still have to create your PostgreSQL database - the easiest way is via the pgAdmin tool but you can also create it via command line tools - a quick google will help you there!
If you don't want to work with docker compose file Version 3 in docker swarm mode, where it is best practice to use docker secrets, then you can create an .application.env-file and link it in your docker-compose.yml (Version 2) with 'env_file:'
$ cat .application.env
SPRING_PROFILES_ACTIVE=prod,swagger
SPRING_DATASOURCE_URL=jdbc:postgresql://postgresql:5432/database_name
SPRING_DATASOURCE_USER=database_user
SPRING_DATASOURCE_PASSWORD=database_password
JHIPSTER_SLEEP=10
[...]
At least I use it this way to keep the credentials away from my jhipster-projects, which are on Github, where I also want to put the *.yml-files.
Using Rails 3.2.2, finishing up my migration from sqlite to postgres 9.2.
Used answer in this tutorial as a guide to install postgres and got stuck on Step 11 where it asks run heroku db:pull where I get:
Failed to connect to database: Sequel::AdapterNotFound -> LoadError: cannot load such file --pg
I dug deeper and found db:pull (taps gem) is deprecated and came across a few recommendations for pg:transfer. Installed pg:transfer, but I get the impression it may be *nix only(?) as if I run: heroku pg:transfer it returns:
Heroku client internal error. No such file or directory - .env (Errno:ENOENT)
If I do pg:transfer with -f and -t it gives me:
'env' is not recognized as an internal or external command, operable program or batch file which means it isn't bound to path or doesn't exist as a command in windows.
Any thoughts on above errors?
Resolved by using pg:backups gem, which was recommended as the replacement for taps in the Heroku docs. I used this guide and uploaded my dump to dropbox for Heroku to pick it up.
Here's my exact list of steps and cmds:
Added pgbackups from heroku.com add-ons to my instance.
heroku pgbackups:capture DATABASE (this just backs up your heroku db)
pg_dump -h localhost -U <pg username> -Fc dbname > dbname.dump
Moved dbname.dump into a folder on my dropbox
In Dropbox, right-click on dbname.dump => "Share link"
Cancel the sharing dialogue pop-up, right-click on "Download button", Copy Link Address (Chrome)
heroku pgbackups:restore DATABASE <paste dropbox download link here>
Dropbox trickiness: don't use the file link provided by Dropbox since it's an html redirect and will cause pg:restore to fail, even though the extension ends in .dump
Instead, navigate to your dropbox page and "right-click copy link address" on the Download button. That's the address you use in your pgbackups:restore (should be something like db.dump?token=<long random string>)
A bit clunky, but got the job done. If you know a better way please let me know!
You need to make a .env file containing something like:
DATABASE_URL=postgres://localhost/myapp_development
References:
https://github.com/ddollar/heroku-pg-transfer
https://devcenter.heroku.com/articles/config-vars#local-setup