ALL,
I have a text file which contains multiple SQL statements, like:
CREATE TABLE a();
CREATE TABLE b();
INSERT INTO a() VALUES();
INSERT INTO b() VALUES();
This file is generated from the SQLite database.
What I'd like to do is to load this file into PostgreSQL database. I already created the database on the server and now I want to populate the database structure and the data.
The whole DB structure contain in 1 file.
Is it possible to just load this file into the PostgreSQL? Or I will have to split the file and then manually create all tables and issue "LOAD" command?
Thank you.
PG_RESTORE will not work unless the source file is from a PG_DUMP.
Your best bet is to fire this off as an .SQL file on connection to the database.
eg.
psql -d [database] -U [user] -f [file from SQLite].sql
As long as the commands in the file are executable and the syntax will work with Postgres this will create your objects and populate them.
Related
My customers runs an very old (seems to me) Sybase 12.5.2 database. I want/need to export all tables from a database to multiple (for each table) flat (text) files. I have access to ISQL command line prompt with the admin user. I havent worked ever with an Sybase database before.
Sybase Adaptive Server Enterprise (ASE) allows multiple databases to be hosted. You don't specify whether only one of the databases in the database server needs to be exported or if all of them do.
For each database, the following query will list the names of the tables
select name from sysobjects where type = 'U'
Sybase ASE also comes with a tool called "bcp" which stands for "Bulk Copy". It is an easy way of creating a flat file of a table's contents.
bcp database.schema.table out file_name -c -U username -S server_name
It has more options that may be of interest, especially around field and row terminators. Documentation for the most relevant version (12.5.1) can be found here:
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.dc30191_1251/html/utility/BABGCCIC.htm
i have been using BCP commands to export data from sybase environments.bcp is a command line utility which you can use it to export data from multiple types of databases
below is a very example and you can try it for
bcp Table Name out OUTPUT FILE PATH\FILENAME.dat -S SERVER NAME -U USERNAME -P PASSWORD -F Format -r row_terminator -e error output file path and name
You can create a batch file with such commands and do multiple exports on one hit.
If you have access to any ETL tool you can exporting the data using the same as well.
I am trying to get a thorough understanding of sqlite3 so that I can run some basic queries through DB Browser for SQLite (http://sqlitebrowser.org/).
To do so, I've imported NYC Taxi data for 1 month, and tried (for many hours) to import this data on sqlite3.
.mode csv <Table_Name>
.import <path/to/file/data.csv> <Table_Name>
Once that finishes, I issue the following SQL statement:
.out <path/to/file/data.db>
select * from <table_name>;
Then, when I try to use DB Browser for SQLite to verify that the database has been populated with data, I get a prompt:
SQLCipher Encryption
Please enter the key used to encrypt the database
Why is it getting auto-encrypted? Is there another way to get my csv file into a database?
The message means that the file is not recognized as a database file. This can happen if the file is encrypted.
But in this case, the output generated by .output is the same as what would be printed on the screen. This is not a database file at all.
To get a copy of the entire database file, use .backup.
To get a copy of a single table, use .dump tablename, then execute those SQL statements in a new database:
sqlite3 data.db < file_generated_by_dump
Here are the details:
The database has to be archived such that records older than 6 months can be copied to a new database and deleted from the main(production) database. The complexity here will be to copy all rows in all tables which have reference each other. After that, these copied rows from some of the tables (which are really huge and whose data is no more needed) will be deleted.
The postgres database is an Amazon RDS instance.
What is the best way to achieve this?
I was thinking either a Springboot application
OR
Have postgresql.conf invoke a shell script which invokes a sql batch.
For the second approach, I am not sure how to edit a amazon RDS postgresql.conf file and where to specify the shell script. Where would be the sql batch written? This is a little new to me, appreciate any pointers.
It will be much faster if you do everything server side instead of using a Springboot application. The problem is not dump/restore which you could easily do with pg_dump utility or psql -d dbname -t -A -F";" -c "SELECT * FROM yourdata WHERE cutdate<=current_timestamp-interval '6 months'" > output.csv
But you have to guarantee that everything that is exported is loaded into the second database and that you do not delete anything that has not been exported.
I would first SELECT a subset of primary keys into a temporary table. Then use server side COPY command to export the preselected keys (and all its dependencies)
COPY (SELECT d.* FROM yourdata d INNER JOIN temporal t WHERE d.pk=t.pk) To '/tmp/yourdata.csv' WITH CSV DELIMITER ',';
After all the export files have been generated
DELETE FROM yourdata WHERE pk IN (SELECT pk FROM temporal)
Then on the backup database do
COPY yourdata(column1,column2,column3) FROM '/tmp/yourdata.csv' DELIMITER ',' CSV
You can write a script that invokes all that commands on server side using psql command line tool and last move the imported files into a permanent location just in case something went wrong and you need to process them again.
See Save PL/pgSQL output from PostgreSQL to a CSV file and How to import CSV file data into a PostgreSQL table?
First forgive me for my English. It is a little bad. Second forgive my ignorance, i'm newiest in postgres
I'm having trouble when I try to up a backup database on another database. I need to dump the database just to get one table, but I only have the files that was in /var/lib/pgsql/data/base/
Here what I try:
I create a database named "test" with OID 227763 so I put the files of the old database to this new database with another OID. I fix the folder and files permissions, but when I log into "test" and run select * from pg_tables; the tables does not appears to me. And when I try to create the table on PhpPgAdmin, I got
ERROR: relation already exists
I'm trying to do this because I need to know which of this files is the table that i want. I will log into database and run SELECT oid,* from pg_class; to get the OID.
I found the old OID database in /var/lib/pgsql/data/global/pg_database
If anyone can help me, I thank you.
There are many ways to backup and restore an entire database or a single table. It sounds like you need to be using pgDump instead of working on individual files. A file level copy is likely to corrupt your database if not in backup mode and if not copying the entire thing + archive logs.
If you MUST copy it by files, make sure the database is shut down for maximum safety.
For me, if I had one table to backup, I'd use pg_dump
pg_dump -U {user-name} {source_db} -f {dumpfilename.sql}
you can use the -t flag to list a single table if you like.
Someone has sent me a database dump as a .sql file dumped using phymyadmin interface. I am trying to restore the dump using the mysql command prompt, however I keep getting empty tables. The .sql file creates a database before creating tables and populating them. When the empty tables message first showed up I thought it was because the database had to be created before running the script, so I created the db and ran the script again, however the tables still show up as empty set.
I tried these steps,
logged in as root.
create database x (this is the name of the db in the create db command in the .sql file)
mysql x -u root -p < my_x_db.sql
logged in as root
show databases
use x
show tables -- empty set
What should I do different and how can I troubleshoot this?
Thanks
There was a create database statement in the .sql file. I solved this by simply commenting the statement. I had already created the database with the same name externally.