We are trying to build out a set of new Oracle database environments using multitenant & ASM for the first time. We have the basic build down using Ansible, however our pdb's are being created using Oracle standard tablespaces, rather than the tablespaces we are specifying in our CDB (our CDB tablespace names are changing as expected). Which config file do we need to change to touch the PDB tablespaces as well?
For example; we create our CDB with USER_AUTO_01 rather than USERS. We would expect our pluggable to be the same, however it is retaining USERS.
We are able to rename them along with the datafiles after they are built, however with ASM, this is time consuming in the environments we are building.
Are you creating these PDBs from seed using CREATE PLUGGABLE DATABASE? If so, my understanding is that the tablespaces are cloned from PDB$SEED, not the CDB.
You could modify PDB$SEED to rename the USERS tablespace.
But if it's possible, I think it would be easier to try using the extra CREATE PLUGGABLE DATABASE clauses to specify DEFAULT TABLESPACE USER_AUTO_01 and USER_TABLESPACES=ALL EXCEPT('USERS')
Edit: Oh, I see you said you want to use dbca silent mode. Well, dbca has less options, but you could:
call dbca with your parameters
and -createUserTableSpace false
and -customScripts to call a short SQL script to create your USER_AUTO_01 tablespace and run ALTER DATABASE DEFAULT TABLESPACE USER_AUTO_01;
I'm using a postgresql (9.6) database in my project which is currently in development stage.
For production I want to use an exact copy/mirror of the database-cluster with a slightly different name.
I am aware of the fact that I can make a backup and restore it under a different cluster-name, but is there something like a mirror function via the psql client or pgAdmin (v.4) that mirrors all my schemas and tables and puts it in a new clustername?
In PostgreSQL you can use any existing database (which needs to be idle in order for this to work) on the server as a template when you want to create a new database with that content. You can use the following SQL statement:
CREATE DATABASE newdb WITH TEMPLATE someDbName OWNER dbuser;
But you need to make sure no user is currently connected or using that database - otherwise you will get following error.
ERROR: source database "someDbName" is being accessed by other users
Hope that helped ;)
I use the doctrine migrations bundle to track changes in my database structure. I would like to ensure that when I'm deploying / adding a new server for my application that:
(A) the database schema is up to date (doctrine:migrations:migrate)
(B) the database always contains a pre-defined set of data
For (B) a good example is roles. I want a certain set of roles to always be present. I realize it is possible with database migrations, but I don't like the idea of mixing schema changes with data changes. Also if I use MySql migrations I would have to create a equivalent Sqlite migration for my test database.
Another option I'm aware of is data fixtures. However from reading the documentation I get the feeling that fixtures are more for loading test data. Also if I changed a role name I don't know how that would be updated using fixtures (since they either delete all data in the database before loading or append to it). If I use append then unique keys would also be a problem.
I'm considering creating some sort of command that takes a set of configuration files and ensures that certain tables are always in a consistent state matching the config files - but if another option exists I'd like to use it of course.
What is the best way to handle loading and managing required data into a database?
If you're using Doctrine Migrations, you can generate initial migration with whole database schema, then you should generate migrations (doctrine:migrations:generate or doctrine:migrations:diff) for all changes that are made in database structure AND also add there queries that will migrate existing data.
Fixtures are designed to pre-populate data (with doctrine:fixtures:load) and, in my opinion, they should be kept up-to-date with latest database schema and executed after doctrine:migrations:migrate / doctrine:schema:create.
So finally:
Create base migration with initial database schema (instead of executing doctrine:schema:create just generate migration file and migrate it)
Create new migrations for each database schema change AND for migrating existing data (such as role name changing)
Keep fixtures up-to-date with latest schema (you can use --append option and only update fixtures instead of deleting all database data first)
Then, when deploying new instance you can run doctrine:schema:create, then doctrine:migrations:version --add --all --no-interaction (mark all migrations as migrated, because you have already created latest schema) and doctrine:fixtures:load which will populate data to the database (also latest version, so data migrations from Doctrine migrations files are not required).
Note: Existing instances should NOT use doctrine:schema:update, but only doctrine:migrations:migrate. In our app we even block usage of this command, in app/console:
use Symfony\Component\Console\Output\ConsoleOutput;
use Symfony\Component\Console\Helper\FormatterHelper;
// Deny using doctrine:schema:update command
if(in_array(trim($input->getFirstArgument()), ['doctrine:schema:update'])) {
$formatter = new FormatterHelper();
$output = new ConsoleOutput(ConsoleOutput::VERBOSITY_NORMAL, true);
$formattedBlock = $formatter->formatBlock(['[[ WARNING! ]]', 'You should not use this command! Use doctrine:migrations:migrate instead!'], 'error', true);
$output->writeln($formattedBlock);
die();
}
This is what I figured out from my experience. Hope you will find it useful :-)
We can delete all nodes and relationships by following query.
MATCH (n) OPTIONAL MATCH (n)-[r]-() DELETE n,r
But newly created node get internal id as ({last node internal id} + 1) . It doesn't reset to zero.
How can we reset neo4j database such as newly created node will get id as 0?
From 2.3, we can delete all nodes with relationships,
MATCH (n)
DETACH DELETE n
Shut down your Neo4j server, do a rm -rf data/graph.db and start up the server again. This procedure completely wipes your data, so handle with care.
run both commands.
match (a) -[r] -> () delete a, r
above command will delete all nodes with relationships.
then run ,
match (a) delete a
and it will delete nodes that have no relationships.
Dealing with multiple databases.
According to Neo4j manage multiple databases documentation:
One final administrative difference is how to completely clean out one database without impacting the entire instance with multiple databases. When dealing with a single instance and single database approach, users can delete the entire instance and start fresh. However, with multiple databases, we cannot do that unless we are comfortable losing everything from our other databases in that instance.
The approach is similar to other DBMSs where we can drop and recreate the database, but retain everything else. Cypher’s command for this is CREATE OR REPLACE DATABASE <name>. This will create the database (if it does not already exist) or replace an existing database with a clean one.
When neo4j is initiated, it is possible to access two databases, a system database and a default (neo4j) database. To clear/reset neo4j database:
1 - Switch to system database:
:use system
2 - Show all databases created with the instance:
SHOW DATABASES
3 - Run the command to clear the database.
CREATE OR REPLACE DATABASE <name>
In my experience, there are two ways to reset a Neo4j database, depending on what you need.
Method 1: Simply delete all nodes/relationships/indexes/constraints
In Neo4j Browser, or in Py2neo with graph.run() (link).
# All nodes and relationships.
MATCH (n) DETACH DELETE n
# All indexes and constraints.
CALL apoc.schema.assert({},{},true) YIELD label, key RETURN *
However, despite being convenient, this approach is not suitable in case of using command neo4j-admin.bat import for BULK import, i.e. ideal for importing millions of nodes at once quickly.
Method 2: Reset database for BULK Import Tool
It's not possible to BULK import when the database is not empty. I tried the above method, but still received the error:
Import error: C:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\data\databases\neo4j already contains data, cannot do import here
Caused by:C:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\data\databases\neo4j already contains data, cannot do import here
java.lang.IllegalStateException: C:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\data\databases\neo4j already contains data, cannot do import here
To tackle this issue, I deleted the following folders:
c:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\data\databases\neo4j
and
c:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\data\transactions\neo4j
Then carried out the Import command:
"C:\Users\[username]\AppData\Local\Neo4j\Relate\Data\dbmss\dbms-dd16c384-78c5-4c21-94f3-b0e63e6c4e06\bin\neo4j-admin.bat" import --database=neo4j --multiline-fields=true --nodes=node_ABC.csv --nodes=node_XYZ.csv relationships=relationship_LMN.csv --relationships=relationship_UIO.csv
Start the Neo4j database. In Neo4j Desktop, the labels and relationships should now be recognized.
Notice that the database I deleted (neo4j) and the database I imported to are the same.
This worked for me with ver. 4.3.2 of the community editition:
Stop the server
cd <neo home>
rm -Rf data/databases/* data/transactions/*
Restart the server
Now you've again the system and the neo4j DBs. The command above deletes the system DB too, and that seems necessary, since deleting a regular DB only (which, in the community edition can only be 'neo4j') makes the metadata in the system DB inconsistent and you start seeing errors.
data/dbms seems to contain the user credentials and you can keep it if you want to keep existing users (else, you'll go back to the default neo4j/test user).
The recommended method is to use the DROP or CREATE Cypher commands, however, these are available in the enterprise edition only (I think it's a shame that a basic feature like this is part of their premium offer, but that's it).
This command deletes everything but requires apoc to be installed :
CALL apoc.periodic.iterate('MATCH (n) RETURN n', 'DETACH DELETE n', {batchSize:1000})
If you are using it on a docker container, you can do
docker-compose rm -f -s -v myNeo4jService
Since neo4j only runs current database specified in the conf file, an easy way to start a new and clean db is to change the current database in the neo4j.conf file and then restart neo4j server.
dbms.active_database=graph.db --> dbms.active_database=graph2.db
Some might argue that the database name is changed. But as of this writing [2018-12], neo4j doesn't support multiple database instances. There is no need for us to differentiate between databases, thus the name of the database is not used in our code.
You can clear/truncate the database with the command below:
MATCH (n) DETACH DELETE n
What this command does is, it matches all the nodes in the database, then detaches all the relationships the matched nodes have and finally deleting the nodes themselves.
Scenario:
I have 3 environments that I am using, Dev, UAT and Live. Each of which having it's own database, MyDb_Dev, MyDb_UAT, MyDb_Live.
Then I have a VS2012 Database project in my solution that contains all my scripts. This works nicely when I make changes to my model database (MyDb_Model) that is located locally.
What I want to do:
I want to use the FileTables in SQL 2012 (which I understand how to set up), however I don't know how to script them to be able to configure the options to handle my environments. When I generate the scripts, it will hard code the name to be MyDb_Model as the FileGroup. Also, that said, when I do try and publish to my Dev database, it's complaining about the database options not being able to take the new scripts. When I script include the options of the Model database, it'll complain when I try to publish to my Dev database because of duplicating names.
Question:
Can you script FileTables (with the database options) using the database project in V2012 to be configurable or do I need to manually make my own scripts?
Prefered:
Compare MyDb_Model to Database project.
Publish to MyDb_Dev as a newly created database.
Sounds like you'll want project variables to handle this where the variable contains the environment-specific text for each. You'd then use that variable in your objects instead of the hard-coded paths. The following would create a FileTable called "DocumentStore" and use the value for a variable called "FileTableDirectoryVariable" that you set up in your Project Properties - SQLCMD Variables. Set each of those in your Publish Profiles to use the correct directory, and you should be good. If you're using different filegroups for these tables, you should be able to tweak the FileGroup setting in a similar manner using a SQLCMD Variable.
CREATE TABLE DocumentStore AS FileTable
WITH (
FileTable_Directory = '$(FileTableDirectoryVariable)',
FileTable_Collate_Filename = database_default
);
GO