Copy a postgres database without LOCK permissions - database

I need to copy a postgres DB from one server to another, but the credentials I have do not have permission to lock the database so a pg_dump fails. I have full read/update/insert rights to the DB in question.
How can I make a copy of this database? I'm not worried about inconsistencies (it is a small database on a dev server, so minimal risks of inconsistencies during the extract)
[edit] Full error:
$ pg_dump --username=bob mydatabase > /tmp/dump.sql
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation sl_node
pg_dump: The command was: LOCK TABLE _replication.sl_node IN ACCESS SHARE MODE

ERROR: permission denied for relation sl_node
This is your real problem.
Make sure the user bob has SELECT privilege for _replication.sl_node. Is that by any chance a Slony system table or something?

This worked for me
sudo -u postgres pg_dump -Fc -c db_name > file_name.pgdump
Then create a DB and run pg_restore it:
sudo -u postgres /usr/local/pgsql/bin/pg_restore -U postgres -d db_name -v file_name.pgdump

pg_dump doesn't lock the entire database, it does get an explicit lock on all the tables it is going to dump, though. This lock is taken in "access share mode", which is the same lock level required by a SELECT statement: it's intended just to guard against one of the tables being dropped between it deciding which tables to dump and then getting the data.
So it sounds like your problem might actually be that it is trying to dump a table you don't have permission for? PostgreSQL doesn't have database-level read/update/insert rights, so maybe you're just missing the select privilege from a single table somewhere...
As Frank H. suggested, post the full error message and we'll try to help decode it.

You need SELECT permissions (read) on all database objects to make a dump, not LOCK permissions (whatever that may be). What's the complete error message when you start pg_dump to make a dump?

https://forums.aws.amazon.com/thread.jspa?threadID=151526
this link helped me a lot. It refers to another one,
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.PostGIS
I first change the ownship to rds_superuser, then paste this piece of code,
CREATE FUNCTION exec(text) returns text language plpgsql volatile AS $f$
BEGIN EXECUTE $1; RETURN $1; END; $f$;
SELECT exec('ALTER TABLE ' || quote_ident(s.nspname) || '.' || quote_ident(s.relname) || ' OWNER TO rds_superuser')
FROM (
SELECT nspname, relname
FROM pg_class c JOIN pg_namespace n ON (c.relnamespace = n.oid)
WHERE nspname in ('tiger','topology') AND
relkind IN ('r','S','v') ORDER BY relkind = 'S')
s;
thereafter, I am able to dump my whole database.

Did you run 'pg_dump' with the correct -U (user who owns that db) ? If yes, then just like other poster said, check the permissions.
HTH

This worked for me -d dbname -n schemaname
pg_dump -v -Fc -h <host> -U <username> -p -d <db_name> -n <schema_name> > file_name.pgdump
default schema is public

Related

MSSQL sp_dropextendedproperty are not permitted

I am new to MSSQL, and I am trying to move db from one server to another.
I am using SQLCMD to restore the DB
However, I am stuck on the following line, (script has many like these I just took one for sake of example)
on running SQLCMD -S servername\SQLEXPRESS -U myuser -P somepassword#1234 -i C:\1.sql -o C:\1.txt which has following code
USE [MoodleNew]
GO
EXEC sys.sp_dropextendedproperty
#name=N'MS_SSMA_SOURCE' ,
#level0type=N'SCHEMA',
#level0name=N'my_schema',
#level1type=N'TABLE',
#level1name=N'my_table_name'
GO
I am getting following error
Object is invalid. Extended properties are not permitted on 'my_schema.my_table_name', or the object does not exist.
No idea what the issue is, any help will be appreciated.

Backup MSSQL DB Schema

I'm trying to back up my production database to my local dev machine with the following constraints:
It will be a regular backup, so using the UI should (ideally?) be avoided.
Some tables are marked to be deleted, so these should not be included in the backup.
I would like to be able to pass the solution (file/package/etc) to other members of the team and they should only have to change a couple of variables in one file and then they can execute and get their own backup.
The DB is over 100GB and contains data that I won't need. I have identified the top largest tables and would only like to take say 5k rows from each - this should provide me with enough data for my purposes and limit space used on my local drives.
I have tried beginning with backing up the schema only using the following methods:
Using the UI to backup Schema only (Tasks -> Generate Scripts)
Get the following error:
Microsoft.SqlServer.Management.Smo.FailedOperationException: Discover dependencies failed. ---> System.ArgumentException: Item has already been added. Key in dictionary: 'Server[#Name='PRODSQL']/Database[#Name='Database1']/UnresolvedEntity[#Name='SomeObjectName' and #Schema='Some.Schema.SomeObjectName']' Key being added: 'Server[#Name='PRODSQL']/Database[#Name='Database1']/UnresolvedEntity[#Name='SomeObjectName' and #Schema='Some.Schema.SomeObjectName']' at System.Collections.SortedList.Add(Object key, Object value) at Microsoft.SqlServer.Management.Smo.DependencyTree..ctor(Urn[] urns, DependencyChainCollection dependencies, Boolean fParents, Server server) at Microsoft.SqlServer.Management.Smo.DependencyWalker.DiscoverDependencies(Urn[] urns, Boolean parents) --- End of inner exception stack trace --- at Microsoft.SqlServer.Management.SqlScriptPublish.GeneratePublishPage.worker_DoWork(Object sender, DoWorkEventArgs e) at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e) at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)
So I moved on.
Tasks -> Copy Database
I get a message saying there is not enough space on disk
Extract Data-tier Application
Get same error as in 1. above.
Powershell script, and a batch file calling sqlcmd on the generated .sql files after the PS script was run.
I was sure this method would work and it took me 2 days to get this far, but still working through multiple errors.
Basically I am doing the following:
Create db objects from the source DB (Schemas, SPs, Tables, Views, UDFs, Triggers, Indexes) and output them to .sql files - Roughly followed http://cfmumbojumbo.com/index.cfm/coding/using-powershell-to-backup-your-stored-procedures-and-triggers/ with some more work added.
If the database already exists on my server, kill, drop, then recreate it (DropCreate.sql):
IF(db_id(#DatabaseName) IS NOT NULL)
BEGIN
DECLARE #SQL VARCHAR(max)
SELECT #SQL = COALESCE(#SQL,'') + 'Kill ' + Convert(VARCHAR, SPId) + ';'
FROM MASTER..SysProcesses
WHERE DBId = DB_ID(#DatabaseName) AND SPId <> ##SPId
EXEC(#SQL);
END
DROP DATABASE MYDATABASE
CREATE DATABASE MYDATABASE ON PRIMARY (...)
The .bat file is essentially doing this
sqlcmd -S %Server% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\DropCreateDB.sql
#loop through and execute multiple .sql files in the directory
for /f %f in (`dir /b C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\StoredProcedures\`) do sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i %f
#Just one sql file in this directory, execute it
sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\Schemas\AllSchemas.sql
sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\Tables\AllTables.sql
.............
The latest error I'm experiencing is:
Changed database context to 'master'.
Msg 6107, Level 14, State 1, Server MYSERVER, Line 1
Only user processes can be killed.
do was unexpected at this time.
Everywhere I turn I am experiencing new errors and have spent over 2 days on it, and I haven't even got to getting the data yet..
TLDR: Is there any easier way backup MSSQL Db schema and top n rows of data from certain tables?

Postgres pg_dump issue

Good Day,
I've been trying to restore a dump file using the psql client and I'm getting this error:
psql.bin:/home/user/Desktop/dump/dumpfile.sql:907:
ERROR: more than one function named "pg_catalog.avg"
CONTEXT: COPY pg_aggregate, line 1, column aggfnoid: "pg_catalog.avg"
I created the dump file from a different Postgres DB (version: 9.4.5) using the command:
pg_dump --username=pgroot ${tables} --no-owner --no-acl --no-security
--no-tablespaces --no-unlogged-table-data --data-only dbname > dumpfile.sql
Where ${tables} is a variable in the for:
-T table1 -T table2 -T table3 ...
This is because I'm dumping specific tables listed in a new-line delimited file. Hence its not the entire database but specific tables I want to dump.
I tried loading the the dump file int another Postgres DB (9.6) using the following command:
psql -d dbname -U superuser -v "ON_ERROR_STOP=1" -f
${DUMP_DIR}dumpfile.sql -1 -a > ${LOG_ERR_DIR}dumpfile.log
2>${LOG_ERR_DIR}dumpfile.err
This gave the error mentioned above. It seems this error is occurring because the dump file tries to add the function "pg_catalog.avg" to the database and it gives an error because it already exists.
The sql file generated by the pg_dump does not have anywhere in it where it creates the pg_catalog.avg function, so i don't know why this is occurring.
So I tried dropping the database and creating it from template0, and still I got the error. It seems to me that its a bug based on the follwoing post:
Re: BUG #6176: pg_dump dumps pg_catalog tables
I'm stuck trying to reslove this issue. If anyone can help me resolve this issue please respond?
Thank you in advance,
j3rg
I found out what was causing this issue. It seems that there was an extra newline in the file containing the table listing. This was causing an extra table argument with no table specified and in turn pg_dump exported the sys tables into the file. I file I was searching for the avg function was the wrong file too.

How to migrate a MySQL4 to MySQL5 database

I have a database (db4) that was created by MySQL4, and a database (db5) that was created by MySQL5. db4 contains several tables with the charset latin1 and several indices, but no data that was encrypted using the MySQL "PASSWORD" function. db5 is empty.
I want to migrate all tables and indices from db4 to db5 (which are actually on the same server). Ideally this should be done without any loss of information and within a short period of time.
Which terminal commands do I need to download the complete database from MySQL4 and insert the data afterwards to db5? Do I have to re-create the indices?
You can make a dump of database in mysql4 using mysqldump. And than upload it to MySQL5 using mysql command.
mysqldump dbname > file
mysql dbname < file
All the indexes will be recreated automatically.
In case anyone else needs to move a database from Mysql4 to Mysql5, here's what I did.
dump the database from the mysql4 server
mysqldump -uuser -ppass db4 > db4.sql
fix some syntax problems (source)
# change comment style from -- to #
sed -r -i -e 's/^--(.*)$/#\1/' db4.sql
# change type declaration keyword from "TYPE" to "ENGINE"
sed -i -e 's/) TYPE=/) ENGINE=/' db4.sql
# adapt timestamp field definition
sed -i -e 's/timestamp(14) NOT NULL,$/timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,/' db4.sql
on the mysql5 server you can now import the modified SQL dump
mysql -uuser -ppass db5 < db4.sql
Sven, I think doing a backup of your data in db4 and restoring it in db5 will work for you.
Backup
mysqldump database_name > file_name.sql
Restore
mysql < file_name.sql
It can be also done within a single step, using the following command:
mysqldump -u dbo4 --password="..." --default-character-set="latin1" db4 | mysql -S /tmp/mysql5.sock -u dbo5 --password="..." --default-character-set="latin1" db5
Unfortunately the standard-values with special characters are not correctly imported, and there seems to be no way to avoid that: How to maintain character-set of standard-values when uploading MySQL-dump.

How to use sqlcmd to create a database

I have a .sql script and I want to build a database from it. How to do it in sqlcmd? I know it's like:
CREATE DATABASE dbName
GO
But how to specify the .sql script and location to build the database?
Use #Jeremiah Peschka's answer to supply the sqlcmd utility with the script to execute.
As for the location for the newly created database, it can be specified as part of the CREATE DATABASE command:
CREATE DATABASE dbName
ON (
NAME = dbName_dat,
FILENAME = 'D:\path\to\dbName.mdf'
)
LOG ON (
NAME = dbName_log,
FILENAME = 'D:\path\to\dbName.ldf'
)
As you can see from the linked article, you can specify other properties as well, like initial size, maximum size etc.
This is very simple. Just enter following command to run sqlcmd:
sqlcmd -U sa -P password
Then enter:
CREATE DATABASE MYDB
GO
This will create the db. Then enter "quit" to exit.
without a file:
sqlcmd -Q "CREATE DATABASE HelloWorld"
sqlcmd -i C:\path\to\file.sql
More options can be found in SQL Server books online.
use this:
sqlcmd -i "c:\my scripts\my script.sql"
see sqlcmd description at MS for other options

Resources