I have a tar.gz which is supposed to contain a complete Postgres 9.4 database backup.
I attemped to restore the database using pg_restore, but I realised (after I unpacked the file) that it has the same structure as the /var/lib/postgresql/9.4/main directory running on my localhost.
So I stopped the postgresql service running on localhost, I replaced my main directory with the one from the archive as described here: Restore postgresql from files.
Moreover I've set owner and permissions in a recursive manner like this:
chown -R postgres:postgres /var/lib/postgresql/9.4/main/
chmod -R 700 /var/lib/postgresql/9.4/main/.
I've restarted the service:
root#EED-015-LIN:~# service postgresql status
9.3/main (port 5433): online
9.4/main (port 3007): online,recovery
...But when I try to connect using psql
postgres#EED-015-LIN:~$ psql -p 3007
psql: FATAL: the database system is starting up
What am I missing?
Related
I'm following this guide to restore a database backup
https://learn.microsoft.com/en-us/sql/linux/tutorial-restore-backup-in-sql-server-container?view=sql-server-ver15
I used docker cp command to copy the DB backup files to the container
docker exec -it SQLContainer mkdir /var/opt/mssql/backup
docker cp MyDb.bak SQLContainer:/var/opt/mssql/backup/
However when trying to restore the DB by running the following query in SSMS, an error message is shown
RESTORE DATABASE MyDB FROM DISK='/var/opt/mssql/backup/MyDB.bak'
Operating system error 5(Access is denied.).
I tried copying using docker cp -a, which sets file ownership to same as destination, but I got this error.
docker cp -a MyDb.bak SQLContainer:/var/opt/mssql/backup/
Error response from daemon: getent unable to find entry "mssql" in passwd database
I'm using Microsoft's image and I don't know the password for root user, the container runs using mssql user, so chown doesn't work either. How can I change the file permissions so DB restore works?
Turns out when I copied the database backup files from the Windows host to the Ubuntu machine, the files were owned by root user and all other users didn't have read permission.
Adding read permission to the file before copying to the docker container works and the server was able to read the files.
sudo chmod a+r MyDb.bak
sudo docker cp MyDb.bak SQLContainer:/var/opt/mssql/backup/
I had a similar issue just today, about the same time as this incident, just attaching MDF and LDF files.
By doing the chmod go+w before copying files without the -a I was able to get SQL Server to treat them as writeable. Prior to that I was getting error messages whenever any update was performed.
EDIT-2
I found out that the database doesn't even start after making the file location change.
This is with the default file location:
$pg_isready
/var/run/postgresql:5432 - accepting connections
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
pg_lsclusters output is green.
After the file location has changed on postgresql.conf:
$pg_isready
/var/run/postgresql:5432 - no response
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down root /mnt/Data/postgresdb/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Here the output is red.
Following this post here, I tried to start the cluster manually:
$pg_ctlcluster 9.5 main start
Warning: the cluster will not be running as a systemd service. Consider using systemctl:
sudo systemctl start postgresql#9.5-main
Error: You must run this program as the cluster owner (root) or root
I tried the same command with sudo:
Error: Config owner (postgres:124) and data owner (root:0) do not match, and config owner is not root
Which again makes me think the problem might lie with permissions of the directory. The directory is owned by root whose ownership I am unable to change.
EDIT-1
I've been working on this and I'd like to distill this post further to give more specifics. This is my current situation:
I installed postgres: sudo apt-get install postgresql and postgresql-contrib
I used sudo -U postgres psql to get into the postgres shell (I'm not sure if this is what I need to do)
show data_directory returns: /var/lib/postgresql/9.5/main
The data directory is located in Ubuntu ext4 formatted hard drive. I also have a 1 TB NTFS formatted hard disk mounted on /mnt/Data (which is mounted automatically on boot). What I tried:
Stop the postgres service: sudo systemctl stop postgresql
Create a new directory /mnt/Data/postgresdb and copy contents of the previous main to this which gives me a full path of /mnt/Data/postgresdb/postgresql/9.5/main using: sudo rsync -av /var/lib/postgresql/ /mnt/Data/postgresdb/postgresql/
Edit /etc/postgresql/9.5/main/postgresql.conf to change data_directory from the path mentioned above to /mnt/Data/postgresdb/postgresql/9.5/main
Start the postgres service: sudo systemctl start postgresl
Run sudo -U postgres psql but get the error that was mentioned in the original post.
These are the permissions on the respective main directories:
ls -l /var/lib/postgresql/9.5/
total 4.0K drwx------ 19 postgres postgres 4.0K Jan 16 12:40 main
ls -l /mnt/Data/postgresdb/postgresql/9.5/
total 4.0K drwxrwxrwx 1 root root 4.0K Jan 16 12:13 main
From the looks of it, the default directory is owned by "postgres" and the new directory is owned by root. However, when I try to change ownership to postgres: chown -R postgres main, it doesn't output any error, but the ownership doesn't change. I'm curious whether this is because this drive is NTFS formatted and is mounted.
Here is my /etc/fstab:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda5 during installation
UUID=3f5a9875-89a3-4ce5-b778-9d7aaf148ed6 / ext4 errors=remount-ro 0 1
# swap was on /dev/sda6 during installation
UUID=85c3f4d4-e450-435b-8dd6-cf1b2cbd8fc2 none swap sw 0 0
/dev/disk/by-label/Data /mnt/Data auto nosuid,nodev,nofail,x-gvfs-show 0 0
Any ideas on how I can go about fixing this?
ORIGINAL POST
Recently, I installed Postgresql for storing some data for my research. The dataset came with instructions on how to setup the data on a Postgresql database (if interested, more info on that here and here). I installed Postgresql and set up a "role" and used the script that was provided for loading the database. It worked but I underestimated the size of the dataset and the script quit saying there was no more space.
I have two drives on my computer a 250G SSD drive with Windows and Ubuntu installed (125G each). And a 1TB HDD NTFS formatted where I store my data. So I thought moving the database to a folder on the other drive would be helpful. I purged all the data and the database to start afresh and followed the instructions here to move the database directory. However, after moving the directory, when I try to connect using psql I get the following error:
~ psql -U username -d postgres 14:48:33
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
How can I fix this? I am running 64-bit Ubuntu 16.04 with Postgresql-9.5. As mentioned earlier, I moved the DB directory a NTFS formatted filesystem (not sure if that cause any problems).
Thanks.
As mentioned in the comments the NTFS was the problem. I ended up resizing my bigger hard drive with 100GB formatted as ext4 and was able to launch postgres with the new data directory without any problems.
I'm trying to change the default data directory for MSSQL Server RC1 2017 after installation and setup in linux (Ubuntu-16.10).
I used the following command to set the default data directory. Then restarted the mssql server.
sudo /opt/mssql/bin/mssql-conf set filelocation.defaultdatadir /mnt/var/opt/mssql/data/
systemctl restart mssql-server.service
After this I tried to create a simple database "test"
sqlcmd -s localhost -U sa -P "someStrongPassword" -Q "CREATE DATABASE test"
The error returned is as follows:
MODIFY FILE encountered operating system error 31(A device attached to
the system is not functioning.) while attempting to expand the
physical file '/mnt/var/opt/mssql/data/test.mdf'.
CREATE DATABASE
failed. Some file names listed could not be created. Check related
errors.
The error log indicates an OS error:
/mnt/var/opt/mssql/data/test.mdf: Operating system error 31(A device
attached to the system is not functioning.) encountered.
I cannot mount the data directory by any means. The permissions to "/mnt" directory are set to 777 too. Changing the default data directory to any other folder, works perfectly fine. Is this a known or recent bug with mssql server?
Yes, there is an issue with using remote storage through NFS and SMB that came up in CTP 2.1 and was not yet fixed in RC1. See release notes: https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-release-notes#a-idrc1-rc1-july-2017-a
The only workarounds are to use local storage or CTP 2.0. We are working on a fix. Release ETA is TBD ATM.
I need to restore a large SQL Server database on a Linux Docker instance (https://hub.docker.com/r/microsoft/mssql-server-linux/)
I'm moving my .bak file to the docker and executing this command in mssql shell:
RESTORE DATABASE gIMM_Brag FROM DISK = '/var/opt/mssql/backup/BackupFull8H_gIMM.bak' WITH MOVE '[gIMM].Data' T'/var/opt/mssql/data/gIMM.mdf', MOVE '[gIMM].Log' TO '/var/opt/mssql/data/gIMM.ldf', MOVE 'TraceabilityData' TO '/var/opt/mssql/data/gIMM.TraceData.mdf', MOVE 'TraceabilityIndexes' TO '/var/opt/mssql/data/gIMM.TraceIndex.mdf', MOVE 'KpiData' TO '/var/opt/mssql/data/gIMM.KpiData.mdf', MOVE 'KpiIndexes' TO '/var/opt/mssql/data/gIMM.KpiIndex.mdf'
I'm mapping correctly every file that need to and I definitely have enough space on the docker instance but I'm getting this error:
Error: The backup or restore was aborted.
The same error occurs with a windows version of this docker actually... And as it's not supposed to be a Express version, the database size shouldn't be the issue here.
If anyone has more information about what is causing this error !
Thanks,
#TOUDIdel
You have to use the actual file system paths on linux rather than the virtual paths that are shown in the error.
RESTORE DATABASE Northwind FROM DISK='/var/opt/mssql/Northwind.bak' WITH MOVE 'Northwind' TO '/var/opt/mssql/data/NORTHWND.MDF', MOVE 'Northwind_log' TO '/var/opt/mssql/data/NORTHWND_log.ldf'
http://www.raditha.com/blog/archives/restoring-a-database-on-ms-sql-server-for-linux-docker/
You didn't mention it, but the thing that tricked me up was that I wasn't copying the BAK file to my Docker instance. In Terminal with docker and your mssql container running...
1) get container ID:
$docker inspect -f '{{.Id}}' <container_name>
2) copy BAK file to docker instance:
docker exec -i <container_id> bash -c 'cat > /var/opt/mssql/backup.bak' < '/source/path/backup.bak'
3) log into mssql:
mssql -u sa -p 'myPassword'
3) restore db: (you can replace this with your restore script, though this was sufficient for me)
RESTORE DATABASE [MyDatabase] FROM DISK = N'/var/opt/mssql/backup.bak' WITH FILE = 1, NOUNLOAD, REPLACE, STATS = 5
When I had this problem, it's because the restore command was taking long enough for mssql to time out (with a totally unhelpful error message). Specifying a long timeout when connecting allowed the restore to complete. eg
mssql -s localhost -p "<sa_password>" -t 36000000 -T 36000000
I am not sure it is worth mentioning, but neither of the answers alone worked when moving a .bak made in Windows server to the docker running in Linux version.
(Note that I am using the code from the two previous answers and thus any credit should go to the below-mentioned authors)
TabsNotSpaces' solution was good until step 3 where the restore crashed with path mismatch (C:/path_to_mssql_server could not be found).
Vinicius Krauspenhar's answer was then necessary to remap the MDF and LOG files to fully complete the backup.
Thus the solution that worked for me when importing a windows-server-made .bak file into the Linux docker instance was:
In Terminal with docker and your SQL Server container running...
1) get container ID:
$docker inspect -f '{{.Id}}' <container_name>
2) copy BAK file to docker instance:
docker exec -i <container_id> bash -c 'cat > /var/opt/mssql/backup.bak' < '/source/path/backup.bak'
3) log into mssql or in any DB software and
RESTORE DATABASE Northwind FROM DISK='/var/opt/mssql/Northwind.bak' WITH MOVE 'Northwind' TO '/var/opt/mssql/data/NORTHWND.MDF', MOVE 'Northwind_log' TO '/var/opt/mssql/data/NORTHWND_log.ldf'
I want to install postgresql for use as the backend to a Windows application.
This seems to be no problem if postgresql is not already installed on the system.
If postgresql is already installed then unless the command line parameters contain the superpassword etc of the existing installation then the install fails.
As I will likely never know the superpassword or other account details of any pre-existing postgresql instances and the machine owners may not either it seems that this will frustrate any attempt to install postgresql in such a situation.
I believe it is possible to install completely independent instances of sql server but is this possible for postgresql?
BTW:
If the command line does contain the correct superpassword then the install just seems to overwrite the existing install and ignores parameters like --prefix etc .
I used init db to create a new database cluster before doing a second install but this new cluster was ignored?
In general you can have multiple independent instances of PostgreSQL. Strictly speaking it's database cluster with separate:
data directory
configuration (e.g. postgresql.conf, pg_hba.conf)
listening TCP/UDP port (default 5432+)
owner user and superuser role
locale and default encoding
log file
postmaster server process (on Windows postgres.exe)
Perfect well-done example is Debian with easy to use postgresql-common infrastructure (pg_ctlcluster, pg_lsclusters, pg_createcluster, pg_dropcluster, included SSL, log rotation and so on).
EDIT:
I found it's rather easy to install second, third, etc. instance of same versioned PostgreSQL under Windows with EnterpriseDB's installer, no need to use initdb and pg_ctl (assuming 64-bit installation, probably you need to use Program Files (x86) for 32-bit installation):
Open cmd with admin privileges (Run as Administrator)
Execute: cd "C:\Program Files\PostgreSQL\9.0\installer\server"
Create new database cluster (press Enter on every step): initcluster.vbs postgres postgres 12345 "C:\Program Files\PostgreSQL\9.0" "C:\Program Files\PostgreSQL\9.0\data2" 5433 DEFAULT
Register as Windows Service: startupcfg.vbs 9.0 postgres 12345 "C:\Program Files\PostgreSQL\9.0" "C:\Program Files\PostgreSQL\9.0\data2" postgresql-x64-9.0-2
Run newly created service postgresql-x64-9.0-2 using services.msc and you have second server
Change 12345 to your password specified during PostgreSQL installation. You don't have to use data2 directory, use whatever you like (but of course not existing data directory).
On Windows 7 I had success following these steps. You'll need the PsExec.exe utility available in the Sysinternals Suite. I assume here that the path to the Sysinternals Suite and the path to the bin folder of your existing PostgreSQL installation are in your PATH environment variable.
Open a cmd.exe window and enter the following command to open a prompt as the Network Service account.
psexec -i -u "nt authority\network service" cmd.exe
The Network Service account won't have access to your PATH, so cd 'C:\PostgreSQL\9.3\bin' and then enter the following command to initialize a data directory for your new instance. I've called mine "data2". It doesn't have to be in the postgres directory, but that's where the default data directory goes, so it's a reasonable choice.
initdb "C:\PostgreSQL\9.3\data2"
Edit C:\PostgreSQL\9.3\data2\postgresql.conf so that port = 5433 (the default instance uses 5432, and you shouldn't have two instances on the same port)
Leave the Network Service cmd prompt and in your standard prompt enter the following command to register the new service. Here I've named my new instance "pg_test"
pg_ctl register -N pg_test -U "nt authority\network service" -D "C:\PostgreSQL\9.3\data2"
Run the following command to start the service.
net start pg_test
The database owner role will be 'YOURMACHINENAME$'. If you want to change this to the standard 'postgres', you have to first create a new super user role that can rename the owner. From the command prompt, enter the following to create this super user.
createuser -s -r -l -i -P -h localhost -p 5433 -U YOURMACHINENAME$ mysuperuser
Finally, connect to the server with psql (psql -U mysuperuser -h localhost -p 5433 postgres) and enter the following commands to rename your database owner and add a password.
ALTER USER "YOURMACHINENAME$" RENAME TO postgres;
ALTER USER postgres WITH PASSWORD 'yourpassword';
Something like this should work (if not it's probably bug):
postgresql-9.0.4-1-windows_x64.exe ^
--mode unattended ^
--prefix c:\postgres\9.0-second ^
--servicename postgresql-x64-9.0-second ^
--serviceaccount postgres2 ^
--servicepassword <password> ^
--serverport 5433 ^
--superaccount postgres ^
--superpassword <password>
EDIT: after a couple of tests I believe it's not possible to create different Postgres instances of the same version using the One-click installer. Sorry.
OTOH you could always play with initdb and pg_ctl and use the existing installation to create a new instance. It would not be as easy as just starting the installer but it's doable.