How can I run pg_dumpall with Heroku? - database

How can I use pg_dumpall with Heroku? The default "database backup" feature from Heroku is pg_dump with the click of a button, which doesn't include roles, so I want to do pg_dumpall ... I'm trying pg_dumpall -h myherokuurl.compute-1.amazonaws.com -l mypassword -U myUser > dump.sql
I'm getting this error:
pg_dumpall: error: query failed: ERROR: permission denied for table pg_authid
pg_dumpall: error: query was: SELECT oid, rolname, rolsuper, rolinherit, rolcreaterole, rolcreatedb, rolcanlogin, rolconnlimit, rolpassword, rolvaliduntil, rolreplication, rolbypassrls, pg_catalog.shobj_description(oid, 'pg_authid') as rolcomment, rolname = current_user AS is_current_user FROM pg_authid WHERE rolname !~ '^pg_' ORDER BY 2
My first thought was to create a new user with the correct privileges. So, I logged using heroku pg:psql DATABASE -a my-app-name then tried create user myUser with password 'mypassword' but got the error ERROR: permission denied to create role
I'm honestly not sure what's going on I'm kind of just guessing. Any troubleshooting ideas would be appreciated! (in the meantime I'm just trying to learn more about Postgres)

If your problem is just about the pg_authid catalog, you should be able to use recent versions of pg_dumpall with the --no-role-passwords option.
This commonly works in hosted environments where pg_authid is inaccesible, e.g. on AWS. The only downside is that the passwords of Postgres users will be missing from the dump.
However, you appear to have a more limited, perhaps shared environment, where you can't even create new Postgres users. I am not certain if there is any chance to get pg_dumpall working there.

Related

Cannot boot Chainlink: opening db: failed to open db:

When I "chainlink node start",
I get the error:
"Cannot boot Chainlink: opening db: failed to open db: failed to connect to host=/private/tmp user=myname database=: server error (FATAL: unrecognized configuration parameter "?application_name" (SQLSTATE 42704))"
I follow this youtube video: https://youtu.be/ZB3GLtQvgME?t=2017
I have a .env
but it is not reading from there.
No matter what I change on DATABASE_URL,
I get the same error.
ETH_URL=wss://kovan.infura.io/ws/v3/
FEATURE_EXTERNAL_INITIATORS=true
LOG_LEVEL=debug
ETH_CHAIN_ID=42
MIN_OUTGOING_CONFIRMATIONS=2
LINK_CONTRACT_ADDRESS=
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
ALLOW_ORIGINS=*
DATABASE_URL=postgresql://localhost:5432/kovan_demo?sslmode=disable
DATABASE_TIMEOUT=0
FEATURE_FLUX_MONITOR=true
MINIMUM_CONTRACT_PAYMENT=0
CHAINLINK_DEV=true
The database parameters are set via environment variable
As Richard said, you need to set DATABASE_URL as an environment variable. For some reason, the chainlink client cannot read file .env and set the environment variable as expected.
The way how I solve the problem is to set DATABASE_URL as an environment variable explicitly by command:
export DATABASE_URL=postgresql://<usrname>:<passswd>#localhost:5432/chainlink_node?sslmode=disable
You can find all configurations of Chainlink node here.
Hope it helps.
The format to connect to the database is the following:
DATABASE_URL=postgresql://$USERNAME:$PASSWORD#$SERVER:$PORT/$DATABASE
I always follow this steps to create the database:
sudo -u postgres psql
create database $DATABASE;
create user $USERNAME with encrypted password '$PASSWORD';
grant all privileges on database $DATABASE to $USERNAME;
Just change the variables
sudo -u postgres psql
create database chainlink;
create user username with encrypted password 'pass';
grant all privileges on database chainlink to username;
And then:
DATABASE_URL=postgresql://username:pass#localhost:5432/chainlink?sslmode=disable
I don't recommend at all this credentials for production.

psql: error: FATAL: password authentication failed for user "Davinci"

Im trying to setup postgres database for my django project, so i donwnloaded the installer together with the pgadmin. i set a default password when installing and i used it to login to the pgadmin and it worked, i now wanted to run so cli commmand, so i added postgres to path in my system enviroment variable in other to be able to run it on the cli and it works, i mean the system recognise the commands but im getting errors which i think its comming from the database itself. i have no idea about this error, this is my first time using it.
PS: it asked for my system user password and i entered it, then i got the error bellow.
C:\Users\Davinci>psql
Password for user Davinci:
psql: error: FATAL: password authentication failed for user "Davinci"
Try running psql -U postgres and then inserting the password you chose at pgadmin.
User postgres is the default, but it seems like Windows is trying to run psql -U Davinci when you type only psql, so you have to specify which user (-U for user / postgres for the username) when trying to connect.

Not allowed do anything in PostgreSQL

I've seen plenty of answers across the internet saying that to combat the
psql:FATAL: role <username> does not exist
one must use the createuser command, but about a dozen different attempts with that command have only yielded a similar error message.
I've installed PostgreSQL using Ubuntu 15.04's sudo apt-get install postgresql, but PostgreSQL permits to do absolutely nothing.
Do I need to install some dependencies or something?
Try doing these actions as the postgres user:
$ sudo -u postgres -i
The PostgreSQL installation makes a postgres user which runs the PostgreSQL service. It is essentially the superuser of your database. In most cases, you don't know the password for this user but any superuser of a system can log in to other accounts.

How can I attach a database to an app in Heroku?

I'm using Heroku's Postgres addon, and I created a new production database from the Heroku Postgres addon page.
I Didn't add it directly to my App using the Resources page of my App.
Now I want to attach this database to my App so it'll be recognized by the heroku pg command.
I'm able to use the database btw after setting the DATABASE_URL config var of my app to point to it, but heroku pg command doesn't recognize it yet.
Additional info: The previous database was Shared, and the new one is a Production.
Heroku add-ons may now be attached across applications and multiple times on a single app.
heroku addons:attach ADDON_NAME -a APP_NAME
Source: https://devcenter.heroku.com/changelog-items/646
To know the name of your addon, do:
heroku addons
Source: https://devcenter.heroku.com/articles/managing-add-ons
Did you add the database using the app-independent https://postgres.heroku.com/ site? Or did you just create a postgresql database in your Heroku control panel?
If you created your database on https://postgres.heroku.com/, you will not see the database via your heroku pg:info command. What you can do to add your database to your application, however, would be to:
Log into https://postgres.heroku.com/.
Click on the database you want to attach to your application.
Under 'Connection Settings', click the configuration button at the top right.
Then click the 'URL' option.
Copy your database URL, this should be something like "postgres://blah:blah#ec2-23-23-122-88.compute-1.amazonaws.com:5432/omg".
In your application, on the command line, run heroku config:set DATABASE_URL=postgres://blah:blah#ec2-23-23-122-88.compute-1.amazonaws.com:5432/omg
What we did there was assign your database to the DATABASE_URL environment variable in your application. This is the variable that's used by default when you provision databases locally to your application, so theoretically, assigning this value should work just fine for you.
To get your database that you created at https://postgres.heroku.com/ attached to your actual heroku app that you are working on you can't use any of the pg backup commands and as far as I can tell there is no supported Heroku way of attaching a database to a heroku app.
You can however create a backup of your database using pg_dump and then use pg_restore to populate your new database that is attached to your app:
pg_dump -i -h hostname -p 5432 -U username -F c -b -v -f "backup-filename" database_name
Once that is complete you can populate your new database with:
pg_restore -i -h new_hostname -p 5432 -U new_username -d new_database_name -v "same_backup_filename"
Even if you are upgrading from the "basic plan" to a the "crane plan" you still have to do a backup and restore, but since the db's are already attached to your app you have the advantage of using the heroku backup commands.

Import SQL dump into PostgreSQL database

We are switching hosts and the old one provided a SQL dump of the PostgreSQL database of our site.
Now, I'm trying to set this up on a local WAMP server to test this.
The only problem is that I don't have an idea how to import this database in the PostgreSQL 9 that I have set up.
I tried pgAdmin III but I can't seem to find an 'import' function. So I just opened the SQL editor and pasted the contents of the dump there and executed it, it creates the tables but it keeps giving me errors when it tries to put the data in it.
ERROR: syntax error at or near "t"
LINE 474: t 2011-05-24 16:45:01.768633 2011-05-24 16:45:01.768633 view...
The lines:
COPY tb_abilities (active, creation, modtime, id, lang, title, description) FROM stdin;
t 2011-05-24 16:45:01.768633 2011-05-24 16:45:01.768633 view nl ...
I've also tried to do this with the command prompt but I can't find the command that I need.
If I do
psql mydatabase < C:/database/db-backup.sql;
I get the error
ERROR: syntax error at or near "psql"
LINE 1: psql mydatabase < C:/database/db-backu...
^
What's the best way to import the database?
psql databasename < data_base_dump
That's the command you are looking for.
Beware: databasename must be created before importing.
Have a look at the PostgreSQL Docs Chapter 23. Backup and Restore.
Here is the command you are looking for.
psql -h hostname -d databasename -U username -f file.sql
I believe that you want to run in psql:
\i C:/database/db-backup.sql
That worked for me:
sudo -u postgres psql db_name < 'file_path'
I'm not sure if this works for the OP's situation, but I found that running the following command in the interactive console was the most flexible solution for me:
\i 'path/to/file.sql'
Just make sure you're already connected to the correct database. This command executes all of the SQL commands in the specified file.
Works pretty well, in command line, all arguments are required, -W is for password
psql -h localhost -U user -W -d database_name -f path/to/file.sql
Just for funsies, if your dump is compressed you can do something like
gunzip -c filename.gz | psql dbname
As Jacob mentioned, the PostgreSQL docs describe all this quite well.
make sure the database you want to import to is created, then you can import the dump with
sudo -u postgres -i psql testdatabase < db-structure.sql
If you want to overwrite the whole database, first drop the database
# be sure you drop the right database !!!
#sudo -u postgres -i psql -c "drop database testdatabase;"
and then recreate it with
sudo -u postgres -i psql -c "create database testdatabase;"
Follow the steps:
Go to the psql shell
\c db_name
\i path_of_dump [eg:-C:/db_name.pgsql]
I tried many different solutions for restoring my postgres backup. I ran into permission denied problems on MacOS, no solutions seemed to work.
Here's how I got it to work:
Postgres comes with Pgadmin4. If you use macOS you can press CMD+SPACE and type pgadmin4 to run it. This will open up a browser tab in chrome.
If you run into errors getting pgadmin4 to work, try killall pgAdmin4 in your terminal, then try again.
Steps to getting pgadmin4 + backup/restore
1. Create the backup
Do this by rightclicking the database -> "backup"
2. Give the file a name.
Like test12345. Click backup. This creates a binary file dump, it's not in a .sql format
3. See where it downloaded
There should be a popup at the bottomright of your screen. Click the "more details" page to see where your backup downloaded to
4. Find the location of downloaded file
In this case, it's /users/vincenttang
5. Restore the backup from pgadmin
Assuming you did steps 1 to 4 correctly, you'll have a restore binary file. There might come a time your coworker wants to use your restore file on their local machine. Have said person go to pgadmin and restore
Do this by rightclicking the database -> "restore"
6. Select file finder
Make sure to select the file location manually, DO NOT drag and drop a file onto the uploader fields in pgadmin. Because you will run into error permissions. Instead, find the file you just created:
7. Find said file
You might have to change the filter at bottomright to "All files". Find the file thereafter, from step 4. Now hit the bottomright "Select" button to confirm
8. Restore said file
You'll see this page again, with the location of the file selected. Go ahead and restore it
9. Success
If all is good, the bottom right should popup an indicator showing a successful restore. You can navigate over to your tables to see if the data has been restored propery on each table.
10. If it wasn't successful:
Should step 9 fail, try deleting your old public schema on your database. Go to "Query Tool"
Execute this code block:
DROP SCHEMA public CASCADE; CREATE SCHEMA public;
Now try steps 5 to 9 again, it should work out
Summary
This is how I had to backup/restore my backup on Postgres, when I had error permission issues and could not log in as a superuser. Or set credentials for read/write using chmod for folders. This workflow works for a binary file dump default of "Custom" from pgadmin. I assume .sql is the same way, but I have not yet tested that
I use:
cat /home/path/to/dump/file | psql -h localhost -U <user_name> -d <db_name>
Hope this will help someone.
If you are using a file with .dump extension use:
pg_restore -h hostname -d dbname -U username filename.dump
I noticed that many examples are overcomplicated for localhost where just postgres user without password exist in many cases:
psql -d db_name -f dump.sql
You can do it in pgadmin3. Drop the schema(s) that your dump contains. Then right-click on the database and choose Restore. Then you can browse for the dump file.
I used this
psql -d dbName -U username -f /home/sample.sql
Postgresql12
from sql file:
pg_restore -d database < file.sql
from custom format file:
pg_restore -Fc database < file.dump
I had more than 100MB data, therefore I could not restore database using Pgadmin4.
I used simply postgres client, and write below command.
postgres#khan:/$ pg_restore -d database_name /home/khan/Downloads/dump.sql
It worked fine and took few seconds.You can see below link for more information.
https://www.postgresql.org/docs/8.1/app-pgrestore.html

Resources