made a backup of a remote database by the following
pg_dump dbname | gzip > filename.gz
made a new database that i want to import it to
CREATE DATABASE new_clone;
tried importing into new database by the following command:
gunzip -c /path/to/backup/filename.sql.gz | psql new_clone
working, imported parts of the database, but ran into error
out of memory
no other information... most tables and schemas have appeared, but I cannot get past this problem.
ive tried doing a simple
pg_dump dbname > filename
and a
psql dbname < infile
tried without compression, and it fails as well.
my guess is that a blob inside the database is too large? is there a way to skip the blobs that cause errors?
database size is 10GB uncompressed, 2.2GB compressed. i have 135GB free disk space, and 32GB ram. running ubuntu 14.04, postgres v 9.6.2
appreciate any help at all. thanks.
I have same issue. After little digging I found that is not psql issue, but sdin: after change COPY ... FROM stdin to COPY ... from '/path/to/file/with.dump' all became fine. So when you try feed psql through stdin, IMHO, OS, not psql, trying load all dump into memory, and, as dump bigger, than memory process can have, you got error.
PS I'm sorry for my bad English, it is not my native language.
To improve k.o. answer, psql --no-readline could be used to avoid stdin(readline) OOM issue.
I had the same issue recently. I am working on ubuntu 20.04 and postgres version is 12. You should try reading the first line of the error. The main error is written on the first line. In my case, postgis-scripts were missing. As a result, I was getting the OOM error. To fix it, run:
sudo apt-get install postgis postgresql-12-postgis-scripts
I had the same problem with a 30GB database dump, and splitting the file up works well for me.
To back it up do a dump of the file and, pipe it through split
pg_dump dbname | split -b 1000m - backup.dump
Then restore it using a pipe from cat
createdb dbname
cat backup.dump* | psql dbname
https://www.postgresql.org/docs/8.1/backup.html#BACKUP-DUMP-LARGE
Related
I am trying to move a schema from one database to another using pg_dump.
I am running this command to take a backup of my schema
pg_dump.exe -h myhost -U postgres -d mydb-n schema> C:\Temp\schema.dump
To restore it I copy the SQL into pgadmin, but I get this error. I don't know what the issue is ?
ERROR: syntax error at or near "2"
LINE 284: 2 \N Unregistered. Item \N 2016-07-13 00:00:00 \N
Is there a best recommended way to backup/restore a postgres database? I often run into issues similar issues when backing up/restoring databases and I start to worry that I will loose valuable data one day.
A plain text pg_dump cannot be restored with pgAdmin since pgAdmin cannot perform COPY FROM STDIN, which is used by pg_dump for the data by default.
You'll have to use psql to restore the dump.
An alternative would be to use the --inserts option of pg_dump so that the data are dumped as INSERT statrments. This comes with a performance penalty though.
I have a question about importing a sql file to postgres CLI. I may have been improperly importing my file or either I may have some User or Database privilege?!? issue. Anyways, these are just my hunches. I am trying to pinpoint the cause of this message after importing a sql file.
The message that I get is:
No relations found.
The steps I did to get into Postgres are:
I typed in:
sudo -i -u postgres
psql
then i created a new role, altered the role permission
and then created a new database as well
i got all my commands from this site http://blog.jasonmeridth.com/posts/postgresql-command-line-cheat-sheet/
last step was I imported a sql file by typing:
psql -d db_name_dev -U username_dev -f /www/dbexport.sql
Now when I go inside the database I created "db_name_dev" by typing
psql db_name_dev and check to see any content imported by typing \dt
I get
No relations found.
here is also a table and role list from my command line..
http://screencast.com/t/8ZMqBLNRb
I'm thinking my database might also have some access privilege issue..
also here is an additional issue i ran into.. hope this helps..
http://screencast.com/t/BJy0ZjrALm6h
thanks,
any feedback would be appreciated
ok so after a few research and readings, i found out my .sql file was empty.. here are some links ive read and learnt more about pg_dump command dbforums.com/showthread.php?1646161-Postgresql-Restores and pg_dump vs pg_dumpall? which one to use to database backups?
I have a ton of postgresql dump files I need to peruse through for data. Do I have to install Postgresql and "recover" each one of them into new databases one by one? Or I'm hoping there's a postgresql client that can simply open them up and I can peek at the data, maybe even run a simple SQL query?
The dump files are all from a Postgresql v9.1.9 server.
Or maybe there's a tool that can easily make a database "connection" to the dump files?
UPDATE: These are not text files. They are binary. They come from Heroku's backup mechanism, this is what Heroku says about how they create their backups:
PG Backups uses the native pg_dump PostgreSQL tool to create its
backup files, making it trivial to export to other PostgreSQL
installations.
This was what I was looking for:
pg_restore db.bin > db.sql
Thanks #andrewtweber
Try opening the files with text editor - the default dump format is plain text.
If the dump is not plain text - try using pg_restore -l your_db_dump.file command. It will list all objects in the database dump (like tables, indexes ...).
Another possible way (may not work, haven't tried it) is to grep through the output of pg_restore your_db_dump.file command. If I understood correctly the manual - the output of pg_restore is just a sequence of SQL queries, that will rebuild the db.
In newer versions you need to specify the -f flag with a filename or '-' for stdout
pg_restore -f - dump_file.bin
I had this same problem and I ended up doing this:
Install Postgresql and PGAdmin3.
Open PGAdmin3 and create a database.
Right click the db and click restore.
Ignore file type.
Select the database dump file from Heroku.
Click Restore.
pg_restore -f - db.bin > db.sql
Dump files are usually text file, if Not compressed, and you can open them with a text editor. Inside you will find all the queries that allow the reconstruction of the database ...
If you use pgAdmin on Windows, can just backup the file as plain text, there is one option when you do backup instead of pg_dump in command line prompt.
i create backup from localhost MYSQL database(drupal) using PHPMyAdmin (file format *.sql). size of this backup = 20MG. i create new database in PHPMyAdmin my live(online) server. Now, when i import backup sql Files i see this error :
#2006 - MySQL server has gone away
i know this error fix with this:
edit ../sql/bin/my.ini
set max_allowed_packet to e.g. 16M
but my server support said: better way is restore mysql using:
mysql -u username -p dbname < file.sql
now, i don't know how to work with this command line for remote server?!
You need SSH access to your server to execute that command using a terminal. If support told you you should use that command, I would think you have SSH access. The SQL file would have to be on your server, so you'd need to transfer it there first (using for example scp).
But if you're not used to the command line, I would recommend first spending some time learning the basics before jumping right into it ;)
I am in a new project and I have to work with an existing PostgreSQL database now. The application is built with CakePHP. The problem I am facing while importing is that the file size of the database is around 4.8 GBs and I am totally new to PostgreSQL.
I have worked on MySql and NoSqls but PostgreSQL is appearing to be a tricky part for me.
I even tried to import the file via Console, but again failed. I am using PostgreSQL on Ubuntu (Core i3). Please help me out of this.
Thanks in advance.
Nishant Shrivastava
there are more possibilities
psql yourdb -f yourfile
psql yourdb < yourfile
cat yourfile | psql yourdb
psql yourdb
\i yourdb