postgresql error connecting after moving data directory - database

EDIT-2
I found out that the database doesn't even start after making the file location change.
This is with the default file location:
$pg_isready
/var/run/postgresql:5432 - accepting connections
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
pg_lsclusters output is green.
After the file location has changed on postgresql.conf:
$pg_isready
/var/run/postgresql:5432 - no response
$pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down root /mnt/Data/postgresdb/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Here the output is red.
Following this post here, I tried to start the cluster manually:
$pg_ctlcluster 9.5 main start
Warning: the cluster will not be running as a systemd service. Consider using systemctl:
sudo systemctl start postgresql#9.5-main
Error: You must run this program as the cluster owner (root) or root
I tried the same command with sudo:
Error: Config owner (postgres:124) and data owner (root:0) do not match, and config owner is not root
Which again makes me think the problem might lie with permissions of the directory. The directory is owned by root whose ownership I am unable to change.
EDIT-1
I've been working on this and I'd like to distill this post further to give more specifics. This is my current situation:
I installed postgres: sudo apt-get install postgresql and postgresql-contrib
I used sudo -U postgres psql to get into the postgres shell (I'm not sure if this is what I need to do)
show data_directory returns: /var/lib/postgresql/9.5/main
The data directory is located in Ubuntu ext4 formatted hard drive. I also have a 1 TB NTFS formatted hard disk mounted on /mnt/Data (which is mounted automatically on boot). What I tried:
Stop the postgres service: sudo systemctl stop postgresql
Create a new directory /mnt/Data/postgresdb and copy contents of the previous main to this which gives me a full path of /mnt/Data/postgresdb/postgresql/9.5/main using: sudo rsync -av /var/lib/postgresql/ /mnt/Data/postgresdb/postgresql/
Edit /etc/postgresql/9.5/main/postgresql.conf to change data_directory from the path mentioned above to /mnt/Data/postgresdb/postgresql/9.5/main
Start the postgres service: sudo systemctl start postgresl
Run sudo -U postgres psql but get the error that was mentioned in the original post.
These are the permissions on the respective main directories:
ls -l /var/lib/postgresql/9.5/
total 4.0K drwx------ 19 postgres postgres 4.0K Jan 16 12:40 main
ls -l /mnt/Data/postgresdb/postgresql/9.5/
total 4.0K drwxrwxrwx 1 root root 4.0K Jan 16 12:13 main
From the looks of it, the default directory is owned by "postgres" and the new directory is owned by root. However, when I try to change ownership to postgres: chown -R postgres main, it doesn't output any error, but the ownership doesn't change. I'm curious whether this is because this drive is NTFS formatted and is mounted.
Here is my /etc/fstab:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda5 during installation
UUID=3f5a9875-89a3-4ce5-b778-9d7aaf148ed6 / ext4 errors=remount-ro 0 1
# swap was on /dev/sda6 during installation
UUID=85c3f4d4-e450-435b-8dd6-cf1b2cbd8fc2 none swap sw 0 0
/dev/disk/by-label/Data /mnt/Data auto nosuid,nodev,nofail,x-gvfs-show 0 0
Any ideas on how I can go about fixing this?
ORIGINAL POST
Recently, I installed Postgresql for storing some data for my research. The dataset came with instructions on how to setup the data on a Postgresql database (if interested, more info on that here and here). I installed Postgresql and set up a "role" and used the script that was provided for loading the database. It worked but I underestimated the size of the dataset and the script quit saying there was no more space.
I have two drives on my computer a 250G SSD drive with Windows and Ubuntu installed (125G each). And a 1TB HDD NTFS formatted where I store my data. So I thought moving the database to a folder on the other drive would be helpful. I purged all the data and the database to start afresh and followed the instructions here to move the database directory. However, after moving the directory, when I try to connect using psql I get the following error:
~ psql -U username -d postgres 14:48:33
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
How can I fix this? I am running 64-bit Ubuntu 16.04 with Postgresql-9.5. As mentioned earlier, I moved the DB directory a NTFS formatted filesystem (not sure if that cause any problems).
Thanks.

As mentioned in the comments the NTFS was the problem. I ended up resizing my bigger hard drive with 100GB formatted as ext4 and was able to launch postgres with the new data directory without any problems.

Related

Unable to mount data directory in MSSQL RC1 server setup in linux

I'm trying to change the default data directory for MSSQL Server RC1 2017 after installation and setup in linux (Ubuntu-16.10).
I used the following command to set the default data directory. Then restarted the mssql server.
sudo /opt/mssql/bin/mssql-conf set filelocation.defaultdatadir /mnt/var/opt/mssql/data/
systemctl restart mssql-server.service
After this I tried to create a simple database "test"
sqlcmd -s localhost -U sa -P "someStrongPassword" -Q "CREATE DATABASE test"
The error returned is as follows:
MODIFY FILE encountered operating system error 31(A device attached to
the system is not functioning.) while attempting to expand the
physical file '/mnt/var/opt/mssql/data/test.mdf'.
CREATE DATABASE
failed. Some file names listed could not be created. Check related
errors.
The error log indicates an OS error:
/mnt/var/opt/mssql/data/test.mdf: Operating system error 31(A device
attached to the system is not functioning.) encountered.
I cannot mount the data directory by any means. The permissions to "/mnt" directory are set to 777 too. Changing the default data directory to any other folder, works perfectly fine. Is this a known or recent bug with mssql server?
Yes, there is an issue with using remote storage through NFS and SMB that came up in CTP 2.1 and was not yet fixed in RC1. See release notes: https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-release-notes#a-idrc1-rc1-july-2017-a
The only workarounds are to use local storage or CTP 2.0. We are working on a fix. Release ETA is TBD ATM.

Restore postgresql 9.4 database from files

I have a tar.gz which is supposed to contain a complete Postgres 9.4 database backup.
I attemped to restore the database using pg_restore, but I realised (after I unpacked the file) that it has the same structure as the /var/lib/postgresql/9.4/main directory running on my localhost.
So I stopped the postgresql service running on localhost, I replaced my main directory with the one from the archive as described here: Restore postgresql from files.
Moreover I've set owner and permissions in a recursive manner like this:
chown -R postgres:postgres /var/lib/postgresql/9.4/main/
chmod -R 700 /var/lib/postgresql/9.4/main/.
I've restarted the service:
root#EED-015-LIN:~# service postgresql status
9.3/main (port 5433): online
9.4/main (port 3007): online,recovery
...But when I try to connect using psql
postgres#EED-015-LIN:~$ psql -p 3007
psql: FATAL: the database system is starting up
What am I missing?

PostgreSQL create tablespace no permission

I've got a problem when I create tablespace for PostgreSQL. The following are the steps:
mkdir /postgres
chown postgres.postgres /postgres
su - postgres
psql
create tablespace p1 location '/postgres'
In this step I got a error:
could not set permissions on directory "/postgres": Permission denied
The directory ownership is correct:
[root#dev ~]# ls -la /postgres
总用量 8
drwxr-xr-x. 2 postgres postgres 4096 12月 2 13:17 .
dr-xr-xr-x. 28 root root 4096 12月 3 06:57 ..
the user is postgres
[root#dev contrib]# ps -ef|grep postgres
postgres 1971 1 0 08:21 ? 00:00:01 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
I'm running on CentOS.
fix:
setenforce 0
At a wild guess I'd say you're on Mac OS X and your PostgreSQL is running as the user postgres_ (note the underscore), as is used by some PostgreSQL packages.
ps -ef | grep postgres or ps aux|grep postgres should show you what user the server is running as. Make sure the directory is owned by that user.
Update based on extra info in comments:
You're on CentOS, not Mac OS X. Your PostgreSQL is running as user postgres, which is the same owner as the directory. It thus seems likely that you are having issues with SELinux. If, for testing purposes only, you run:
setenforce 0
are you then able to run the CREATE TABLESPACE command? (DROP the tablespace after creating it with SELinux temporarily off; if you don't, and restart, PostgreSQL will fail to start up).
If creation fails with SELinux temporarily disabled, you must either exempt PostgreSQL from your SELinux policy, create the tablespace at a location that the SELinux policy permits, or set appropriate SELinux attributes on the tablespace directory so that PostgreSQL can manipulate it. Or you can turn SELinux off entirely, but that's not really preferable.
There might be hints in dmesg, or in CentOS's SELinux helper tool, to tell you specific SELinux booleans you can turn on or off to control this. See the help for the setsebool command, the Fedora Security Guide, the CentOS SELinux howto, etc.
Perhaps the best option is to just change the SELinux context of the file. See the documentation. You can use chcon, but then the change will be lost after a file system relabel. It's better to use semanage as discussed in the next page of the linked manual.

Oracle XE Database Configuration failed

I am trying to create an oracle xe database in my vps.
VPS OS : Cent OS.
When try to run
/etc/init.d/oracle-xe configure
it throws an error Database confiration failed and to check the logs but logs just shows
ORA-01034: ORACLE not available
Below is the history...
[root#vmcx-43 Disk1]# rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm
Preparing... ########################################### [100%]
/var/tmp/rpm-tmp.51363: line 186: bc: command not found
1:oracle-xe /var/tmp/rpm-tmp.51363: line 186: bc: command not fo und########################################### [100%]
Executing post-install steps...
/var/tmp/rpm-tmp.97984: line 76: bc: command not found
/var/tmp/rpm-tmp.97984: line 77: bc: command not found
/var/tmp/rpm-tmp.97984: line 78: [: -gt: unary operator expected
/var/tmp/rpm-tmp.97984: line 82: bc: command not found
You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database.
[root#vmcx-43 Disk1]# /etc/init.d/oracle-xe configure
Oracle Database 11g Express Edition Configuration
-------------------------------------------------
This will configure on-boot properties of Oracle Database 11g Express
Edition. The following questions will determine whether the database should
be starting upon system boot, the ports it will use, and the passwords that
will be used for database accounts. Press <Enter> to accept the defaults.
Ctrl-C will abort.
Specify the HTTP port that will be used for Oracle Application Express [8080]:
Specify a port that will be used for the database listener [1521]:
Specify a password to be used for database accounts. Note that the same
password will be used for SYS and SYSTEM. Oracle recommends the use of
different passwords for each database account. This can be done after
initial configuration:
Password can't be null. Enter password:
Password can't be null. Enter password:
Confirm the password:
Do you want Oracle Database 11g Express Edition to be started on boot (y/n) [y]: n
Starting Oracle Net Listener...Done
Configuring database...
Database Configuration failed. Look into /u01/app/oracle/product/11.2.0/xe/config/log for details
[root#vmcx-43 Disk1]# cd /u01/app/oracle/product/11.2.0/xe/config/log
[root#vmcx-43 log]# ls
CloneRmanRestore.log cloneDBCreation.log postDBCreation.log postScripts.log
[root#vmcx-43 log]# tail postScripts.log
commit
*
ERROR at line 1:
ORA-01034: ORACLE not available
Process ID: 0
Session ID: 0 Serial number: 0
[root#vmcx-43 log]# tail CloneRmanRestore.log
select TO_CHAR(systimestamp,'YYYYMMDD HH:MI:SS') from dual
*
ERROR at line 1:
ORA-01034: ORACLE not available
Process ID: 0
Session ID: 0 Serial number: 0
Add your servers name and IP to the /etc/hosts file
I had same issues.
I uninstalled oracle-xe. See How to reconfigure Oracle 10g xe on Linux
Then followed
yum install bc
rpm -i oracle-xe.rpm
/etc/init.d/oracle-xe configure
Everything went fine.
yum install bc
Then try again.
Ok the solution may sound weird but today i got the exactly same error while installing Oracle Xe on centos. I struggled a lot to find the answer but in the end the problem was the way i was installing the rpm.
Initailly i used the command
$rpm -ivh oracle-xe.rpm
and somehow it was giving the same error which you are getting.
After that i tried
$rpm -i oracle-xe.rpm
and it worked for me. Not very sure why will the "h" flag, which is the hash flag cause an issue but it worked for me.
for debian ... how to install oracle-XE from rpm
Configuring database...
Database Configuration failed. Look into /u01/app/oracle/product/11.2.0/xe/config/log for details
nano /u01/app/oracle/product/11.2.0/xe/config/scripts/init.ora
comment # memory_target=100663296
/etc/init.d/oracle-xe configure // will work
I too faced the similar issue on Linux Mint 17.3. Fortunately, I found the solution sooner. The issue is simply that your shared memory file is not where Oracle expects it to be i.e. /dev/shm but you'd be having it at /run/shm with /dev/shm linking to it.
So, to resolve this issue, before configuring the database, you must perform the below steps in order
$ sudo rm -rf /dev/shm
$ sudo mkdir /dev/shm
$ sudo mount -t tmpfs shmfs -o size=2048m /dev/shm
I have tested it, works perfect.
After googling 'oracle sucks' in frustration over the lack of logging from the installation I managed to resolve the issue that caused the configuration to fail on a docker container running the Hortonworks HDP 2.6 Sandbox:
Oracle XE requires 1 Gb of shared memory and fails otherwise (I didn't try 512 mb) according to https://blogs.oracle.com/oraclewebcentersuite/implement-oracle-database-xe-as-docker-containers.
vi /etc/fstab
change/add the line to:
tmpfs /dev/shm tmpfs defaults,size=1024m 0 0
Then reload the configuration by:
mount -a
Keep in mind that if you later restart the docker container you might have to do 'mount -a' once more as it starts with the default set on the container ~ 65 mb.
Normally the failed configuration will have succeeded in creating a listener and you will have to kill this before rerunning configuration.
ps -aux | grep tnslsnr
kill {process identified in the step above}
Lost a full day to this one as none of the other answers on this page worked for me (Ubuntu).
Proper instructions where here
The main trick missing from other tutorials was to execute
sed -i 's,/var/lock/subsys,/var/lock,' /etc/init.d/oracle-xe
before
/etc/init.d/oracle-xe configure
check the permissions for: /u01/
In my case these were set to root:root I changed this to oracle:dba and it worked for me.
But before that I tried the following:
Setting up the IP/hostname in the /etc/hosts
installing bc and reinstalling oracle
both the steps did not work for me but I uninstalled and reinstalled oracle-xe, changed permissions and then ran the command for configure.

PostgreSQL 9: could not fsync file "base/16386": Invalid argument

I'm trying to test a small PostgreSQL setup, so I cobbled together a quick local install. However, when I'm trying to create my personal db with createdb, it chokes on errors like this (notably, it starts with base/16384 the first time, and increments each time I run it). Anyone know what's going on here, or if there's some trivial config I missed that would cause this? Thanks, and this is somewhat time-critical, so please respond if you do know anything. Thanks!
UPDATES:
I'm running this on a CentOS 5 server, apologies that I don't have too many further details (it's a shared account on that server). uname -a has the following output:
Linux {OMITTED} 2.6.18-194.11.4.el5 #1 SMP Tue Sep 21 05:04:09 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
I installed PostgreSQL from source from:
http://wwwmaster.postgresql.org/download/mirrors-ftp/source/v9.0.1/postgresql-9.0.1.tar.bz2
built in my home directory and installed to prefix=$HOME/local/pgsql.
Here's a terminal readout for me attempting to create my user's db on a fresh data setup:
[htung#{OMITTED}:~]$ killall postgres
LOG: autovacuum launcher shutting down
LOG: received smart shutdown request
LOG: shutting down
LOG: database system is shut down
[htung#{OMITTED}:~]$ rm -r tmp
mk[1]+ Done ../local/pgsql/bin/postgres -D $HOME/tmp (wd: ~/tmp)
(wd now: ~)
[htung#{OMITTED}:~]$ mkdir tmp
[htung#{OMITTED}:~]$ local/pgsql/bin/initdb -D $HOME/tmp
The files belonging to this database system will be owned by user "htung".
This user must also own the server process.
The database cluster will be initialized with locale en_US.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".
fixing permissions on existing directory /afs/{OMITTED}/htung/tmp ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /afs/{OMITTED}/htung/tmp/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.
Success. You can now start the database server using:
local/pgsql/bin/postgres -D /afs/{OMITTED}/htung/tmp
or
local/pgsql/bin/pg_ctl -D /afs/{OMITTED}/htung/tmp -l logfile start
[htung#{OMITTED}:~]$ local/pgsql/bin/postgres -D $HOME/tmp
LOG: database system was shut down at 2010-11-15 13:47:25 PST
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
[1]+ Stopped local/pgsql/bin/postgres -D $HOME/tmp
[htung#{OMITTED}:~]$ bg
[1]+ local/pgsql/bin/postgres -D $HOME/tmp &
[htung#{OMITTED}:~]$ local/pgsql/bin/createdb
ERROR: could not fsync file "base/16384": Invalid argument
STATEMENT: CREATE DATABASE htung;
createdb: database creation failed: ERROR: could not fsync file "base/16384": Invalid argument
[htung#{OMITTED}:~]$
I would guess that you're possibly running into the SE linux system here. I'd recommend to either turn off SELinux and see if that works, or to install from RPMs available from the postgresql website.

Resources