ArangoDB upgrade loses data - database

I feel I'm doing something wrong with the ArangoDB upgrade process. The end result from the upgrade is that my databases exist, my users exist, my collections exist, but there are no documents in my collections. Obviously this is an issue. I've had this problem occur twice, upgrading from 2.3.1 -> 2.3.4, and 2.3.4 -> 2.4 in Windows. I used the same procedure in both cases:
Stopped the ArangoDB service
Made a backup copy of my ArangoDB directory from Program Files
Installed the new version of ArangoDB
Copied the contents of the database folder from the old ArangoDB directory to the new one, excluding the system database (I feel like this is where I go wrong...)
Then I open a command prompt to the bin directory and run arangod --upgrade
The upgrade output seems right to me, it finds the old databases and upgrades them, which is evident by the fact that they exist, along with the collections. But as stated before the collections are all empty. Thankfully this has been in a dev environment, but I worry about upgrading my production environment. Am I doing something wrong or is this a bug?

I've tried to reproduce this with the step 2.3.5 to 2.4.1 using the x64 Arango packages
What I did:
First, ran arangod from the shell with its own database directory outside of the program directory:
bin\arangod.exe c:\ee --console
Created a collection, inserted data (like the js/server/tests/aql-optimizer-rule-use-index-for-sort.js setUp()-function does)
then installed the new version, ran
bin\arangod.exe c:\ee --upgrade
then
bin\arangod.exe c:\ee --console
AQL_EXECUTE("for u in UnitTestsAqlOptimizeruse_index_for_sort_XX return u")
Which gave me all 100 documents which I put into the collection.
Next I tried with running the arangod service, with the var\lib folder inside of the Porgram Files folder. I connected using arangosh, inserted the documents into the collection again, verified with
db._query("for u in UnitTestsAqlOptimizeruse_index_for_sort_XX return u").toArray();
that all data was there.
Then stopped the service, installed 2.4.1, stopped the service, and used explorer to copy over the ArangoDB 2.4.1\var\lib directory, run the arangod --upgrade with success restarted the service, and used arangosh to successfully revalidate the collection and its documents again.
So, as this seems similar to what you did, can you try to reproduce this with a minimal set of data and send us your var\lib directory?

As it turns out the problem was related to replication. I would replicate data from the production db to use during development. Then when I would upgrade or stop the Arango service on the dev db all the documents would vanish. BUT when I used arango backup and restore to copy the production DB data, everything worked as expected. The newest version of Arango is supposed to have fixed the issue, but I haven't had any time to test it.

Related

Kiwi-tcms v8.0 database migration problem with test runs

I just migrated Kiwi tcms from v7.1 on a server A to v8.0 on a server B (I virtualized our application server).
I use a pgsql container for the db.
For the migration I used the backup-restore method.
The db volume on server B is brand new and I created the schema using /Kiwi/manage.py migrate
Then I restored the .json. But I ran into schema problem because of the changes in v8.0 of the primary key. I just replaced in the json file the old names by the new one and the restoration passed.
The application runs fine except that I have now a problem with test runs when I go to the "search for test runs" page:
DataTables warning: table id=resultsTable - Requested unknown parameter 'run_id' for row 0, column 0. For more information about this error, please see http://datatables.net/tn/4
I am not sure if I made any mistake during the migration or if there is actually a bug in the migration process.
For info: I tried on my test container in v7.3 to migrate the database in v8.0. Everything went fine but I had the same problem at the end.
Thank you by advance for you support !
EDIT 1
I solved my problem following the advice of #Alexander Todorov and restarting from scratch the migration steps:
I uploaded the image of kiwi 7.1 on my docker registry so I migrated to my server B with kiwi 7.1. Now I can focus on upgrade.
I use pgsql container, so I didn't need to update mariadb container before migrating.
I upgraded to kiwi 8.0 using the kiwitcms/kiwi:latest image.
I am not sure why the upgrade from the 7.3 version didn't work the first time but from the 7.1 to 8.0 everything went fine.
It is possible that I backed up from 7.1 and restored on a 7.3. Everything was fine using 7.3 at this moment but I had trouble with the migration to 8.0...
Anyway, thanks for your support !
EDIT 2
I don't know how it is possible but, I can't reproduce the exact same migration on another instance. I get the issue each time I try.
What I have now:
A preprod instance of kiwi working fine in v8.0
What I want:
A production instance in v8.0
What I tried:
Create a new instance in v8.0 and backup the preprod db to restore it on the prod => fails with the error I had before...
DataTables warning: table id=resultsTable - Requested unknown parameter 'run_id' for row 0, column 0. For more information about this error, please see http://datatables.net/tn/4
Create a new instance in v7.1 and migrate in the exact same way as I did on my preprod => fails with the same error...
I am really clueless on this :/
I just migrated Kiwi tcms from v7.1 on a server A to v8.0 on a server B (I virtualized our application server).
That is what is causing your problems. You are trying to restore one version of DB schema + data onto another version (on the second server). In between the 2 versions there are quite a lot of DB migrations and what you are trying to do will always lead to failure.
You can use server A and upgrade in place to the latest version, then dump the data, move to server B (with the same version) and restore the data into the clean DB and decommission server A.
You may also setup server B to have the older version of Kiwi TCMS, migrate the data there and then upgrade server B to v8.0.
No idea why it worked but I did the following, and everything seems to be fixed. My symptoms were exactly like those described here (7.3->8.0, mariaDB, pruned after 8.0, DataTables warning)
$ cd Kiwi
$ sudo docker exec -it kiwi_db bash
$ mysql -ukiwi -p -h 127.0.0.1 (pw kiwi from yml)
Then within mariaDB
use kiwi;
show tables;
describe testruns_testrun;
select * from testruns_testrun;
It was here I refreshed the page to pull out the ID value to look against when all of a sudden my test runs loaded. I did a
sudo docker-compose down && sudo docker-compose up -d
To verify the issue doesn't come back and it didn't. I'm no DBA, just poking around. I did make a new run from a plan, which worked until I navigated away, and tried searching for the test run. I did the above afterwards.

SQL Server Database Project

I want to use database project for script deployment in Azure SQL Server, I don't want to import full database. I just want to use database project for delta script. I added a project and included one script file with none as build action that contains create table statement , I am publishing the project, It's completing successfully but create statement is not executing. What is wrong here? Is there any other way to do this?
TLDR: Set your build action to "Post Deployment Script".
Longer:
What happens in SSDT is that all the files that have a build action of "Build" are built into a model of what the database should look like. When the deploy happens that model is compared to the target database and if there are any changes, a change script it generated and then optionally deployed.
If you have any file marked pre or post deployment script then they are either prepended or appended to the change script and will be run as part of the deployment.
If you have any files with a build action of "None" then SSDT ignores them, you could put anything in there, even an ascii picture of a donkey and the project will still build and deploy (obviously your ascii donkey won't get deployed anywhere).
If you just want to use SSDT to do your deployments you can just set the build action to pre or post deploy and it will be included. This is pretty odd though, either don't use SSDT or use SSDT and put the model of your entire database in there.
Personally, I would use SSDT properly and live the dream.
Ed

Running different versions of postgresql side by side

I have postgresql 9.3 installed.
I would like to have also postgres 9.6.1 installed.
Each application is using a different DB. Most of the times I don't run both applications, so I don't need them to run concurrently.
I downloaded the installer recommended by postgres, and installed 9.6.1, but then it seems that 9.3 is not able to start anymore. I'm getting an error trying to run sudo service postgres start:
Starting PostgreSQL 9.3 database server
The PostgreSQL server failed to start. Please check the log output.
The log file is empty (not sure that's the interesting one) - /var/log/postgresql/postgresql-9.3-main.log
Any idea how to be able to run both instances?
You need to check the postgresql.conf config file.
If you want to run both instances at the same time then they will need to be run on different ports otherwise they will conflict. The default is 5432, change this for one of the DB's.
Then make sure that the data directory, log file are unique for each instance.

Oracle Database Mobile Server first steps

I download jdeveloper studio edition and install it in C:\Oracle\Middleware
then I install weblogic instance using C:\Oracle\Middleware\wlserver_10.3\common\bin\config.exe
to C:\myproject\domains\myweblogic
Then download DMS_11.1.0.0.0_WINDOWS.zip and install it to C:\Oracle\Middleware\Mobile.
thereafter cant start
C:\Oracle\Middleware\Mobile\Mobile\Server\bin\runmobileserver.bat
unless i change the variable to this batch file
set DOMAIN_DIR=C:\myproject\domains\myweblogic
what do I do next to see that it works?
is there a simple example that creates a sigle table and synchronized between server and mobile clients?
very difficult to find information on this technology. interfere very popular tags (Oracle, Database).
If you install Database Mobile Server directly in the WebLogic directory, you will not need to edit the batch file. For instance, I believe the default install directory is:
C:\Oracle\Middleware\wlserver_10.3
So if you used the default install directory, you would install Database Mobile Server there, with no added 'Mobile' or other directory. If you chose to install WebLogic elsewhere, then replace the above with that directory.
There is another small issue to be aware of.
The installer will give you warning about installing in a non-empty
directory. You should click through the warning. This is a known issue
documented in the Release Notes.
Regarding your other question about a demo, there is a Java demo found in
<MOBILE_HOME>\Mobile\Server\demos\consolidator_api
You will need to download a client and perform a sync, which is documented here:
http://download.oracle.com/docs/cd/E22663_01/doc.11100/e22681/instsqlite.htm#BABGAACI
I hope that helps, good luck with your project. Let me know if you have further questions.
Eric Jensen, Oracle PM

Reinstall xampp

Im using a mac. Xampp stopped working after the latest update 10.6.8 and I think it may be a direct result of the update. Anyway, Id like to reinstall xampp however i didnt backup my database and i cant get into localhost/phpmyadmin to import it. Is there anyway I can backup my database another way and reinstall xampp? or is there another problem as to why the pages dont load.
I get this error when i go to localhost/phpmyadmin: The requested URL /phpmyadmin was not found on this server.
When you use XAMPP, it automatically freezes all the other database. If you are using mysql, you can backup them either using
1.mysqladmin tool (http://dev.mysql.com/downloads/gui-tools/5.0.html)
directly login as the admin and select the databases that you need and backup.
or
2.using command line (manual backup):
goto mysql/bin and then follow this tutorial: http://linux.byexamples.com/archivesc/90/backup-and-restore-mysql-database/
When you uninstall xampp, the databases (in var/mysql) and the htdocs folders are not normally deleted by the uninstall wizard. When you reinstall xampp, those existing folders will be used. It's also a good idea to back up those folders directly to reduce the risk of loss.

Resources