Permission denied when creating tablespace in postres - database

I'm trying to set up a postgres tablespace on a secondary volume on a fresh installation of Ubuntu 16.04. My primary volume has only 60GB on it and I need a restore a ~55GB database. I'm using a fresh install of postgresql-9.5.
I made the user postgres a super admin so that it would be able to chmod whatever it wants (I know this is not recommended, but I'm getting a little desperate).
sudo usermod -aG sudo postgres
As user postgres, I did the following.
I've created a folder on my secondary drive (named postgres_data) and set owner to postgres.
postgres#Eli:/media/rp3/ExtraDrive1$ ls -lisa
total 28
2 4 drwxrwxrwx+ 4 root root 4096 Nov 9 07:46 .
262146 4 drwxr-x---+ 3 root root 4096 Nov 9 05:39 ..
11 16 drwx------ 2 root root 16384 Nov 2 08:14 lost+found
10485761 4 drwxrwxr-x 3 postgres postgres 4096 Nov 9 07:46 postgres_data
I then created a nested folder (named data), also owned by postgres. I did this because I read that the user postgres must own not just the folder I want the tablespace in, but the folder containing that folder.
postgres#Eli:/media/rp3/ExtraDrive1/postgres_data$ ls -lisa
total 12
10485761 4 drwxrwxr-x 3 postgres postgres 4096 Nov 9 07:46 .
2 4 drwxrwxrwx+ 4 root root 4096 Nov 9 07:46 ..
10485762 4 drwxrwxr-x 2 postgres postgres 4096 Nov 9 07:46 data
I connected to postgres as user postgres and attempted to create a tablespace:
create tablespace mappify_data location '/media/rp3/ExtraDrive1/postgres_data/data';
But I got a permissions error:
create tablespace mappify_data location '/media/rp3/ExtraDrive1/postgres_data/data';
I've tried changing permissions with chmod 700, changing ownership to postgres:postgres with chown, and creating the folders as the user postgres, but all yield the same result.
I'd appreciate any advice I could get. I'm at my wits' end :(

Does your linux run with SELinux ? I have read threads where the problem was SELinux.
I just had a similar problem and eventually found that another possible cause of error is of the user postgres does not have the rights to enter the directories above the one for the tablespace.
So in your case, make sure that the user postgres can browse in the hierarchy of directories /media/rp3/ExtraDrive1/postgres_data/.

I was trying to set up a tablespace on a USB drive but somehow postgres was not getting the permission. I always used to get "permission denied" error message in psql. The problem was with directory permissions and traversing thru parent folders for user=postgres. Finally this answer here helped me..... permission denied in a folder for a user after chown and chmod
root#G41:~# chmod a+x /media/revoltman
root#G41:~# chmod a+x /media/revoltman/PRASHANTH2
root#G41:~# chmod a+x /media/revoltman/PRASHANTH2/dir1
testdb2=# CREATE TABLESPACE tspace2 OWNER postgres LOCATION
'/media/revoltman/PRASHANTH2/dir1';
CREATE TABLESPACE

Related

BootstrapSystemDataDirectories() failure (hresult 0x8007010b)

Trying to mount a db into a mssql docker container
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=Str0ngP#ssw0rd!
ENV MSSQL_TCP_PORT=1433
EXPOSE 1433
COPY mydb.mdf /var/opt/mssql/data/mydb.mdf
COPY mydb_log.ldf /var/opt/mssql/data/mydb_log.ldf
ENTRYPOINT /opt/mssql/bin/sqlservr
EDIT
It seems that the only thing that prevents the image from running as a container is when I add those two COPY instructions within the Dockerfile. Everything works fine when I remove the two COPY.
In fact, it says that it can't copy c:\tempdata\master.mdf to /var/opt/mssql/data/master.mdf. But why is that?
Lately, when
Structure
All files are in the same folder on my local machine.
myfolder
/Dockerfile
/mydb.mdf
/mydb_log.ldf
Environment
Windows 10 for Workstation
Docker Desktop 4.5.1 (74721) (
Engine 20.10.12,
Compose 1.29.2,
Kubernetes 1.22.5,
Snyk 1.827.0,
Credential Helper 0.6.4)
Visual Studio Code 1.67.2
Error obtained
The image is built in a flawless fashion, letting believe everything's fine. But when I run it, I get an error:
ERROR: BootstrapSystemDataDirectories() failure (HRESULT 0x8007010b)
To run the image, I type the following command:
docker run -p 1433:1433 myimage
or even
docker run myimage
and both fashions creates the same error.
When I type in:
docker images
I can see:
REPOSITORY TAG IMAGE ID CREATED SIZE
myimage latest ffc13a86b57b 28 seconds ago 2.83GB
Which confirms that the image is correctly created.
FINAL EDIT
I thought I would share the resulting Dockerfile and final solution.
The Goal
The goal was to take a client's database MDF and LDF files in SQL Server and mount them in a Docker Container to avoid the process of installing a local SQL Server instance which I don't really need.
Lesson LEARNED
As #AlwaysLearning states, the COPY instructions are processed through the root user of the container, hence taking ownership over the /var/opt/mssql. Doing exactly as she/he said solved the problem. So folder's ownership needs to be given back to mssql user as described in #AlwaysLearning's answer. BIG THX!
Final Solution
The final solution is to be able to mount/attach the client's database files to the containerized instance of SQL Server. For that to work, I needed to write a shell script which does just that.
attach-db.sh
sleep 15s
/opt/mssql-tools/bin/sqlcmd -S . -U sa -P $tr0ngP#ssw0rd! -Q "CREATE DATABASE [mydb] ON (FILENAME = '/var/opt/mssql/data/mydb.mdf'),(FILENAME = '/var/opt/mssql/data/mydb_log.ldf') FOR ATTACH"
This comes from here: Attaching databases via a dockerfile
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=$tr0ngP#ssw0rd!
COPY mydb.mdf /var/opt/mssql/data/mydb.mdf
COPY mydb_log.ldf /var/opt/mssql/data/mydb_log.ldf
COPY attach-db.sh /var/opt/mssql/data/attach-db.sh
ENTRYPOINT /var/opt/mssql/data/attach-db.sh & /opt/mssql/bin/sqlservr
Running the built image
docker run -p 1433:1433 --hostname mydb myimage
Connecting to database
Download and install Azure Data Studio is required to connect to a containerized SQL Server instance.
If you check the logs for the Docker container you'll see that the complete error message is:
2022-06-09 00:12:57.28 Server Setup step is copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf'.
2022-06-09 00:12:57.33 Server ERROR: Setup FAILED copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf': 5(Access is denied.)
ERROR: BootstrapSystemDataDirectories() failure (HRESULT 0x80070005)
This happens because the Dockerfile COPY actions are performed as the root user which leave the file system objects owned by the root user as seen with:
$ ls -la /var/opt/mssql/data
total 12
drwxr-xr-x 1 root root 4096 Jun 9 00:12 .
drwxrwx--- 1 root root 4096 Jun 9 00:12 ..
-rw-r--r-- 1 root root 0 Jun 9 00:06 mydb.mdf
-rw-r--r-- 1 root root 0 Jun 9 00:06 mydb_log.ldf
The SQL Server service itself is executed using the mssql user so now it doesn't have access to the /var/opt/mssql/data directory to add its own files. You can correct that situation by changing the ownership of the files and directories to the mssql user, i.e.:
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=Str0ngP#ssw0rd!
COPY mydb.mdf /var/opt/mssql/data/mydb.mdf
COPY mydb_log.ldf /var/opt/mssql/data/mydb_log.ldf
USER root
RUN chown -R mssql:root /var/opt/mssql
USER mssql
Now the container will start successfully and you can see that the SQL Server service was able to copy its bootstrap files into the /var/opt/mssql/data directory:
$ ls -la /var/opt/mssql/data
total 81168
drwxr-xr-x 1 mssql root 4096 Jun 9 00:23 .
drwxrwx--- 1 mssql root 4096 Jun 9 00:23 ..
-rw-r----- 1 mssql root 256 Jun 9 00:23 Entropy.bin
-rw-r----- 1 mssql root 4653056 Jun 9 00:23 master.mdf
-rw-r----- 1 mssql root 2097152 Jun 9 00:23 mastlog.ldf
-rw-r----- 1 mssql root 8388608 Jun 9 00:23 model.mdf
-rw-r----- 1 mssql root 14090240 Jun 9 00:23 model_msdbdata.mdf
-rw-r----- 1 mssql root 524288 Jun 9 00:23 model_msdblog.ldf
-rw-r----- 1 mssql root 524288 Jun 9 00:23 model_replicatedmaster.ldf
-rw-r----- 1 mssql root 4653056 Jun 9 00:23 model_replicatedmaster.mdf
-rw-r----- 1 mssql root 8388608 Jun 9 00:23 modellog.ldf
-rw-r----- 1 mssql root 14090240 Jun 9 00:23 msdbdata.mdf
-rw-r----- 1 mssql root 524288 Jun 9 00:23 msdblog.ldf
-rw-r--r-- 1 mssql root 0 Jun 9 00:06 mydb.mdf
-rw-r--r-- 1 mssql root 0 Jun 9 00:06 mydb_log.ldf
-rw-r----- 1 mssql root 8388608 Jun 9 00:23 tempdb.mdf
-rw-r----- 1 mssql root 8388608 Jun 9 00:23 tempdb2.ndf
-rw-r----- 1 mssql root 8388608 Jun 9 00:23 templog.ldf
Edit:
It's worth pointing out that the Dockerfile COPY command can also set owner+group attributes on-the-fly whilst copying files into the image. This then alleviates the need to switch to USER root and back to USER mssql so as to apply chown manually, i.e.:
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=Str0ngP#ssw0rd!
COPY --chown=mssql:root mydb.mdf /var/opt/mssql/data/mydb.mdf
COPY --chown=mssql:root mydb_log.ldf /var/opt/mssql/data/mydb_log.ldf

postgresql error connecting after moving data directory

EDIT-2
I found out that the database doesn't even start after making the file location change.
This is with the default file location:
$pg_isready
/var/run/postgresql:5432 - accepting connections
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
pg_lsclusters output is green.
After the file location has changed on postgresql.conf:
$pg_isready
/var/run/postgresql:5432 - no response
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down root /mnt/Data/postgresdb/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Here the output is red.
Following this post here, I tried to start the cluster manually:
$pg_ctlcluster 9.5 main start
Warning: the cluster will not be running as a systemd service. Consider using systemctl:
sudo systemctl start postgresql#9.5-main
Error: You must run this program as the cluster owner (root) or root
I tried the same command with sudo:
Error: Config owner (postgres:124) and data owner (root:0) do not match, and config owner is not root
Which again makes me think the problem might lie with permissions of the directory. The directory is owned by root whose ownership I am unable to change.
EDIT-1
I've been working on this and I'd like to distill this post further to give more specifics. This is my current situation:
I installed postgres: sudo apt-get install postgresql and postgresql-contrib
I used sudo -U postgres psql to get into the postgres shell (I'm not sure if this is what I need to do)
show data_directory returns: /var/lib/postgresql/9.5/main
The data directory is located in Ubuntu ext4 formatted hard drive. I also have a 1 TB NTFS formatted hard disk mounted on /mnt/Data (which is mounted automatically on boot). What I tried:
Stop the postgres service: sudo systemctl stop postgresql
Create a new directory /mnt/Data/postgresdb and copy contents of the previous main to this which gives me a full path of /mnt/Data/postgresdb/postgresql/9.5/main using: sudo rsync -av /var/lib/postgresql/ /mnt/Data/postgresdb/postgresql/
Edit /etc/postgresql/9.5/main/postgresql.conf to change data_directory from the path mentioned above to /mnt/Data/postgresdb/postgresql/9.5/main
Start the postgres service: sudo systemctl start postgresl
Run sudo -U postgres psql but get the error that was mentioned in the original post.
These are the permissions on the respective main directories:
ls -l /var/lib/postgresql/9.5/
total 4.0K drwx------ 19 postgres postgres 4.0K Jan 16 12:40 main
ls -l /mnt/Data/postgresdb/postgresql/9.5/
total 4.0K drwxrwxrwx 1 root root 4.0K Jan 16 12:13 main
From the looks of it, the default directory is owned by "postgres" and the new directory is owned by root. However, when I try to change ownership to postgres: chown -R postgres main, it doesn't output any error, but the ownership doesn't change. I'm curious whether this is because this drive is NTFS formatted and is mounted.
Here is my /etc/fstab:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda5 during installation
UUID=3f5a9875-89a3-4ce5-b778-9d7aaf148ed6 / ext4 errors=remount-ro 0 1
# swap was on /dev/sda6 during installation
UUID=85c3f4d4-e450-435b-8dd6-cf1b2cbd8fc2 none swap sw 0 0
/dev/disk/by-label/Data /mnt/Data auto nosuid,nodev,nofail,x-gvfs-show 0 0
Any ideas on how I can go about fixing this?
ORIGINAL POST
Recently, I installed Postgresql for storing some data for my research. The dataset came with instructions on how to setup the data on a Postgresql database (if interested, more info on that here and here). I installed Postgresql and set up a "role" and used the script that was provided for loading the database. It worked but I underestimated the size of the dataset and the script quit saying there was no more space.
I have two drives on my computer a 250G SSD drive with Windows and Ubuntu installed (125G each). And a 1TB HDD NTFS formatted where I store my data. So I thought moving the database to a folder on the other drive would be helpful. I purged all the data and the database to start afresh and followed the instructions here to move the database directory. However, after moving the directory, when I try to connect using psql I get the following error:
~ psql -U username -d postgres 14:48:33
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
How can I fix this? I am running 64-bit Ubuntu 16.04 with Postgresql-9.5. As mentioned earlier, I moved the DB directory a NTFS formatted filesystem (not sure if that cause any problems).
Thanks.
As mentioned in the comments the NTFS was the problem. I ended up resizing my bigger hard drive with 100GB formatted as ext4 and was able to launch postgres with the new data directory without any problems.

Cakephp File permission (Mac os)

The Filelog.php has been chmod 777, but i still get the errors below. How can i fix it? Thanks.... (Mac os)
failed to open stream: Permission denied in /Applications/XAMPP/xamppfiles/htdocs/oven-master/app/vendor/cakephp/cakephp/src/Log/Engine/FileLog.php
-rwxrwxrwx 1 daemon admin 3068 Jul 27 09:49 BaseLog.php
-rwxrwxrwx 1 daemon admin 3088 Jul 27 09:49 ConsoleLog.php
-rwxrwxrwx 1 daemon admin 6370 Jul 27 09:49 FileLog.php
-rwxrwxrwx 1 daemon admin 4570 Jul 27 09:49 SyslogLog.php
You should give permission in logs and tmp those 2 directory recursively.
May be those directory may not exist. Then you have to create in cakephp root and then give permission recursively
I had the same problem:
how i solved it:
open terminal in your cakephp root project folder and type:
sudo bin/cake server

"[Errno 2] No such file or directory" when trying to backup a database in pgAdmin4

I have finally managed to get around managing to manually set "PostgreSQL Binary Path" in preferences (why should the user have to do this when it wasn't necessary in pgAdmin3?).
Though to be honest, I have no idea what is meant with the second path field called "EDB Advanced Server Binary Path". But I get no more error message concerning that so I assume it is o.k. to leave that empty.
Now I am stuck on the next step.
When trying to backup the database I have to enter a filename. No matter what I do here, I keep getting this error:
[Errno 2] No such file or directory
Well, it can not be that I have to create a file before it is created by pgAdmin, can it?
What could I be doing wrong?
Alex
update: answering to #n33rma here are the print screens:
I solved the problem through the command line.
If we use a previous format, it has the same response as you had:
$ pg_restore -d <DBNAME> --username=postgres g=<FILENAME>
pg_restore: [archiver] could not open input file "g=<FILENAME>": No such file or directory
If we change the last param, it works properly:
$ pg_restore -d <DBNAME> --username=postgres <FILENAME>
I suppose the GUI-command changed in v4, but the console pg_restore is not.
I just updated to pgAdmin v4.1 and this solved the issue :-)
Thanks all for helping!
You need to give absolute path with the file name to sql file like below.
Windows
C:/test/test.sql
Linux
/u01/test.sql
Enter only a filename, no path, because pgadmin puts the backup in its own directory, specifically, in ~/.pgadmin/storage/(user).
I suspect that path for "PostgreSQL Binary Path" is not set properly in pgAdmin4.
pgAdmin4 is Web application and pgAdmin3 is Desktop application, So pgAdmin4 needs to know path of utilities like pg_dump & pg_restore binaries so that it can execute them.
In pgAdmin4 Goto: File > Preferences > Paths > Binary paths > PostgreSQL Binary Path
If you are using Windows than provide path like,
C:\Program Files\PostgreSQL\9.6\bin
If you are using Linux than provide path like,
/opt/PostgreSQL/9.5/bin
user#mint:/opt/PostgreSQL/9.5/bin$ lsh pg_du* pg_res*
-rwxr-xr-x 1 xxx xxx 150K Sep 2 2015 pg_restore
-rwxr-xr-x 1 xxx xxx 49K Sep 2 2015 pg_resetxlog
-rwxr-xr-x 1 xxx xxx 83K Sep 2 2015 pg_dumpall
-rwxr-xr-x 1 xxx xxx 364K Sep 2 2015 pg_dump
On MacOS 10.11 and using postgres.app I resolved it this way:

PostgreSQL create tablespace no permission

I've got a problem when I create tablespace for PostgreSQL. The following are the steps:
mkdir /postgres
chown postgres.postgres /postgres
su - postgres
psql
create tablespace p1 location '/postgres'
In this step I got a error:
could not set permissions on directory "/postgres": Permission denied
The directory ownership is correct:
[root#dev ~]# ls -la /postgres
总用量 8
drwxr-xr-x. 2 postgres postgres 4096 12月 2 13:17 .
dr-xr-xr-x. 28 root root 4096 12月 3 06:57 ..
the user is postgres
[root#dev contrib]# ps -ef|grep postgres
postgres 1971 1 0 08:21 ? 00:00:01 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
I'm running on CentOS.
fix:
setenforce 0
At a wild guess I'd say you're on Mac OS X and your PostgreSQL is running as the user postgres_ (note the underscore), as is used by some PostgreSQL packages.
ps -ef | grep postgres or ps aux|grep postgres should show you what user the server is running as. Make sure the directory is owned by that user.
Update based on extra info in comments:
You're on CentOS, not Mac OS X. Your PostgreSQL is running as user postgres, which is the same owner as the directory. It thus seems likely that you are having issues with SELinux. If, for testing purposes only, you run:
setenforce 0
are you then able to run the CREATE TABLESPACE command? (DROP the tablespace after creating it with SELinux temporarily off; if you don't, and restart, PostgreSQL will fail to start up).
If creation fails with SELinux temporarily disabled, you must either exempt PostgreSQL from your SELinux policy, create the tablespace at a location that the SELinux policy permits, or set appropriate SELinux attributes on the tablespace directory so that PostgreSQL can manipulate it. Or you can turn SELinux off entirely, but that's not really preferable.
There might be hints in dmesg, or in CentOS's SELinux helper tool, to tell you specific SELinux booleans you can turn on or off to control this. See the help for the setsebool command, the Fedora Security Guide, the CentOS SELinux howto, etc.
Perhaps the best option is to just change the SELinux context of the file. See the documentation. You can use chcon, but then the change will be lost after a file system relabel. It's better to use semanage as discussed in the next page of the linked manual.

Resources