Reset all tables data but and restore initial settings - kiwi-tcms

I have using Kiwi TCMS System in development, so I have created more test data. Now I want to delete all the test data, so I tried to reset the Kiwi DB tables using the following command as suggested in the article,
https://kiwitcms.org/blog/atodorov/2018/07/30/how-to-backup-docker-volumes-for-kiwi-tcms/
First taken the backup,
docker exec -it kiwi_web /Kiwi/manage.py dumpdata --all --indent 2 > database.json
Then deleted all table records,
docker exec -it kiwi_web /bin/bash -c '/Kiwi/manage.py sqlflush | /Kiwi/manage.py dbshell'
I know this command has deleted everything including initial setup data completely from DB, so I have tried to apply the initial setup using the following commands,
docker exec -it kiwi_web /Kiwi/manage.py migrate
docker exec -it kiwi_web /Kiwi/manage.py createsuperuser
docker exec -it kiwi_web /Kiwi/manage.py refresh_permissions
The First 2 worked fine, but the last refresh_permissions was throwing an error, Refer to the error screenshot
This leads, some of the permission-related settings like groups are not restored.
Seems I'm doing some wrong way to delete all data and restore with the required initial setup. pLease correct me what would be the best way here.

You are getting errors because you don't understand what is happening. The commands and blog post referenced above are for backup & restore of the entire database, not selectively removing information which you don't want to be there anymore:
sqlflush will remove all information from all tables
migrate will create DB schema (which already exists) and initial records in some tables. Because the DB schema already exists and the internal Django reference to which migration has been applied last hasn't changed you get a "No migrations to apply" message
Because no migrations were applied some initial records aren't created. In this case a group with hard-coded name "Administrator".
refresh_permissions fails because it is looking for a group with the name "Administrator"
You can either:
Re-create all missing records by hand or
Create a clean installation of Kiwi TCMS and use that as your main instance or
Keep the POC instance and delete only the records you don't want - either one-by-one from the admin panel or using an ORM query via manage.py shell.

Related

Do not drop and create database when scripting drop/create database objects and data with mssql-scripter

I'm trying to script the database objects and data of my database to later move it to a server where I don't have backup/restore rights. Instead I'm using the Generate Scripts method and I use mssql-scripter to generate the scripts.
I have a .bat file with the following script code to generate my SQL script file.
set
timevar=%date:~4,2%%date:~7,2%%date:~10,4%-%time:~0,2%%time:~3,2%%time:~6,2%
mssql-scripter --server 10.100.8.8 -d Dev_db -f .\%timevar%.sql
--schema-and-data --script-drop-create --target-server-version 2016 --target-server-edition Standard --check-for-existence --include-dependencies --constraint-names --collation -U ScriptingUser -P 1234 --exclude-use-database
The problem is that it's also scripting DROP DATABASE and CREATE DATABASE, which I don't want. I would only like to DROP and CREATE database objects and later populate tables with the scripted data.
Has anyone faced this problem and have you found a solution?
After fiddling around with the options for longer, I managed to find the right parameter and work-around to solve my problem.
The exact code that I ran is:
set
timevar=%date:~4,2%%date:~7,2%%date:~10,4%-%time:~0,2%%time:~3,2%%time:~6,2%
mssql-scripter --server 10.100.8.8 -d Dev_db -f .\%timevar%.sql --schema-and-data
--script-drop-create --target-server-version 2016 --target-server-edition Standard --check-for-existence --constraint-names --collation -U ScriptingUser -P 1234 --exclude-use-database --include-objects "dbo." --display-progress
The key change I added the --include-objects parameter, with a twist. The way I changed by scripts is by adding code snippet:
--include-objects "dbo."
This tells mssql-scripter to only script out objects that contain the "dbo." keyword(substring) in the fully qualified name.
Also I remove this parameter from my initial command:
--include-dependencies
since I script out everything in my database under the dbo schema.
This scripts out:
all of the objects in my database
it includes a IF EXISTS check
it issues a DROP query to drop the existing
it issues a CREATE query to create the new one
it issues multiple INSERT statements to also populate the database with data

move neo4j database from one server to another

I have a server that uses neo4j 3.5.x with docker. Now I want to move that database to another server.
This time I see that neo4j released 4.0. I just copied data folder which contains only graph.db
I run the script I used last time
sudo docker run --name vis --restart unless-stopped --log-opt max-size=50m --log-opt max-file=10 -p3001:7474 -p3002:7473 -p3003:3003 -d -v /data:/data -v /conf:/conf -v /logs:/logs --env NEO4J_AUTH=none neo4j
When I run this I see that I can reach it from 7474 which is fine. BUT it asks password to see the data. Though I didn't set a password WHY IT ASKS?
I tried everything possible like neo4j, 123, 1234, test or live it empty. none worked.
it gives error
neo4j-driver.chunkhash.bundle.js:1 WebSocket connection to 'ws://0.0.0.0:7687/' failed: Error in connection establishment: net::ERR_ADDRESS_INVALID
Is there a proper/robust way to import data between neo4j database servers? Can I use this https://neo4j.com/developer/kb/export-sub-graph-to-cypher-and-import/
If you go to the Neo4j desktop, and then select the graph. Open up Manage options. Then choose Open Terminal.
Once there you can use the database backup command. (Here is an example)
bin\neo4j-admin backup --backup-dir=c:/backup --name=graph.db-20200107
This will backup the database to the specified backup directory.
Then you can zip that backup directory, and then unzip the backup directory, and restore it on the new server.
(Here is an example)
bin\neo4j-admin restore --from=c:/backup/graph.db-20200107 --database=graph.db --force=true
Note: The 'graph.db-20200107' is an example of the name that you give the database backup. You can name it whatever you want.
-yyyguy

Restore corrupt mongo db from WiredTiger files

So here is my scenario:
Today my server was restarted by our hoster (acpi shutdown).
My mongo database is a simple docker container (mongo:3.2.18)
Because of an unknown reason the container wasn't restarted on reboot (restart: always was set in docker-compose).
I started it and noticed the volume mapping were gone.
I restored them to the old paths, restarted the mongo container and it started without errors.
I connected to the database and it was completely empty.
> show dbs
local 0.000GB
> use wekan
switched to db wekan
> show collections
> db.users.find();
>
Also I already tried db.repairDatabase();, no effect.
Now my _data directory contains a lot of *.wt files and more. (File list)
I found collection-0-2713973085537274806.wt which has a file size about 390MiB.
This could be the data I need to restore, assuming its size.
Any way of restoring this data?
I already tried my luck using wt salvage according to this article, but I can't get it running - still trying.
I know backups,backups,backups! Sadly this database wasn't backuped.
Related GitHub issue, contains details to software.
Update:
I was able to create a .dump file with the WiredTiger Data Engine tool. However I can't get it imported into a mongoDB.
Try running a repair on the mongo db container. It should repair your database and the data should be completely restored.
Start mongo container in bash mode.
sudo docker-compose -f docker-compose.yml run mongo bash
or
docker run -it mongo bash
Once you are inside the docker container, run mongo db repair.
mongod --dbpath /data/db --repair
The DB should repaired successfully and all your data should be restored.

How to populate a heroku postgresql database with a sql file

First of, I want to say, that I am not a DB expert and I have no experience with the heroku service.
I want to deploy a play framework application to the heroku service. And I need a database to do so. So I created a postgresql database with this command, since it's supported by heroku:
Users-MacBook-Air:~ user$ heroku addons:create heroku-postgresql -a name_of_app
And I got this as response
Creating heroku-postgresql on ⬢ benchmarkingsoccerclubs... free
Database has been created and is available
! This database is empty. If upgrading, you can transfer
! data from another database with pg:copy
So the DB is now existing but empty of course. For development I worked with a local H2 Database.
Now I would want to populate the DB on heroku using a sql file, since it's quite a lot of data. But I couldn't find how to do that. Is there a command for the heroku CLI, where I can hand over the sql file as an argument and it populates the database? The File basically consists of a few tables which get created and around 10000 Insert commands.
EDIT: I also have CSV files from all the tables. So if there is a way how I can populate the Postgres DB with those would be also great
First, run the following to get your database's name
heroku pg:info --app <name_of_app>
In the output, note the value of "Add-on", which should look something like this:
Add-on: postgresql-angular-12345
Then, issue the following command:
heroku pg:psql <Add-on> --app <name_of_app> < my_sql_file.sql
For example (assuming your sql commands are in file test.sql):
heroku pg:psql postgresql-angular-12345 --app my_cool_app < test.sql

Dropping a postgres database in cmdline, still seeing the database when \list

I'm trying to drop my database and create a new one through the command line.
I login using psql postgres and then do a \list, see a list of the two databases i created which i now want to delete. so i tried using a DROP DATABASE databasename;
I don't see any error while executing that statement but when i try to \list again to see if that DB are deleted, i still see that that the DB exists. Can someone please tell me why this could happen? and how to surely delete those DB.
There are a couple caveats to DROP DATABASE:
It can only be executed by the database owner.
It cannot be executed while you or anyone else are connected to the target database.
I generally use the dropdb command-line tool to do this, since it's a wrapper around DROP DATABASE which doesn't require you to explicitly connect first. It still has the caveat that there can't be any users currently connected to the database, but it's generally quicker/easier to use.
I would recommend you try issuing a command like this:
dropdb -h <host> -U <user> -p <port> <name of db to drop>
Similarly, you can use the createdb command-line tool to create a database.
More info on DROP DATABASE: http://www.postgresql.org/docs/current/static/sql-dropdatabase.html
Edit:
Also, it is worth looking in the Postgres log (likely in /var/log/postgresql by default) to see if perhaps there is anything in there that wasn't surfaced in the results.

Resources