I'm going to upgrade postgreSQL from the version 9 to 11.
I've exported a database using this command C:\Program Files\PostgreSQL\11\bin\pg_dump.exe --file "C:\\Users\\bnechmi\\DOCUME~1\\db_kd" --host "myhost.com" --port "5450" --username "sce" --no-password --verbose --format=t --blobs --no-tablespaces "kd5"
On the linux server "myhost.com", postgreSQL 9 is installed.
On my new server I have installed postgreSQL 11. I created new database called kd5 then I imported the database using this command pg_restore -h localhost -d kd5 -U postgres ./mypath/db_kd --no-tablespaces --verbose 2>log1126.log
this is the contend of the file log1126.log
pg_restore: creating ACL "kd5.TABLE version"
pg_restore: creating ACL "kd5.TABLE version_bdgeo"
pg_restore: creating ACL "kd5.TABLE version_ihm"
pg_restore: creating ACL "kd5.TABLE version_suroit"
pg_restore: creating ACL "kd5.TABLE zone_tarifaire"
WARNING: errors ignored on restore: 476
Then I have connected to the database kd5, when I run the command \dt I get the message Did not find any relations
How can I import the exported database ?
Related
I am in the process of doing export and import with postgres database.
I had used the following command to take the backup of postgres db
C:\dirs> pg_dump -U postgres -p 15432 -W -F t cgate-next-demo > .\dbexport_10th_February_2022.tar
Password:*****
I have unzipped dbexport_10th_February_2022.tar file and proceeded with database import. As a initial step, I had dropped the database.
#drop database if exists "cgate-next-demo";
And I had recreated the empty database.
#create database "cgate-next-demo";
In order to do this, I have logged in to psql once,
C:\dirs> psql -U postgres -p 15432
Password for user postgres:*****
postgres=#
For database import I have used the following command.
C:\dirs> psql -U postgres -p 15432 -d cgate-next-demo <restore.sql
While I do that I have got the following error. I took this excerpt from console logs.
ERROR: could not open file "$$PATH$$/6052.dat" for reading: No such
file or directory HINT: COPY FROM instructs the PostgreSQL server
process to read a file. You may want a client-side facility such as
psql's \copy.
Can someone guide on what would've caused this issue.
You are doing this in the wrong fashion. Rather than unpacking the archive, pass it as argument to pg_restore. That will do everything for you.
I'm trying to recover my db from server in local. I have db.bak file that contains all database.
I used this command in psql:
CREATE DATABASE my_db;
then in root I used:
sudo -u postgres psql my_db < db.bak;
all tables are recovered except 4 of them, while the tables exist in the .bak file. But on another computer all tables have been recovered correctly.
the problem was in installation of postgit. it should be installed by this command:
sudo apt install postgresql-10
sudo apt install postgresql-10-postgis-2.4
sudo apt install postgresql-10-postgis-scripts
it solved my problem.
I want to set up the psql terminal tool in Centos 6.6
I have been given access to as database and i just want to use the terminal for writing queries to the database for information. I have no prior experience with psql before but I want to move on from the pgadmin3 gui.
I started off by installing psql:
yum install postgresql
but when I try to access it, ie. typing [root#localhost]# psql I get the following error:
psql: FATAL: database "root" does not exist
I've tried using:
psql --host=<DB instance endpoint> --port=<port> --username=<master user name> --password --dbname=<database name>
but that fails to work too, maybe this is really basic but im completely lost for setting this up
Use:
psql -U my_pgadmin_username postgres
or
psql -U my_pgadmin_username -h localhost postgres
Alternately, more typical usage:
sudo -u postgres psql
I have a tar.gz which is supposed to contain a complete Postgres 9.4 database backup.
I attemped to restore the database using pg_restore, but I realised (after I unpacked the file) that it has the same structure as the /var/lib/postgresql/9.4/main directory running on my localhost.
So I stopped the postgresql service running on localhost, I replaced my main directory with the one from the archive as described here: Restore postgresql from files.
Moreover I've set owner and permissions in a recursive manner like this:
chown -R postgres:postgres /var/lib/postgresql/9.4/main/
chmod -R 700 /var/lib/postgresql/9.4/main/.
I've restarted the service:
root#EED-015-LIN:~# service postgresql status
9.3/main (port 5433): online
9.4/main (port 3007): online,recovery
...But when I try to connect using psql
postgres#EED-015-LIN:~$ psql -p 3007
psql: FATAL: the database system is starting up
What am I missing?
I'm running through the Cloudera Manager (free edition) and I reached the point where the wizard is creating the Hive Metastore Database.
This error is shown and halts the configuration process.
using /var/run/cloudera-scm-agent/process/40-hive-metastore-create-tables/hadoop-conf as HADOOP_CONF_DIR
I cant seem to find any information that might cause this?
Every thing has been configured correctly up to this point, everything installed and user names and passwords are correct.
Has anybody seen this error before? Thoughts?
Error Log:
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:688)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1094)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2337)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at com.cloudera.enterprise.dbutil.SqlRunner.open(SqlRunner.java:109)
at com.cloudera.enterprise.dbutil.SqlRunner.runSingleQuery(SqlRunner.java:80)
at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.countTables(HiveMetastoreDbUtil.java:191)
... 2 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2540)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:612)
... 20 more
ok cloudera is using version HIVE 0.10, that doesn't support remote login
but you need to go around that bug, logging to the server that is getting the error the cloudera manager will tell you the ip
1) create login in to the Server that fails to install HIVE
2) Create a $HADDOP_HOME
$HADOOP_HOME="/usr/lib/hadoop/"
3) INSTALL postgres in the server that fails
`$ sudo apt-get install postgresql`
$ cat /etc/postgresql/9.1/main/postgresql.conf | grep -e listen -e standard_conforming_strings
modify this to lines in the file
listen_addresses = '*'
standard_conforming_strings = off
You also need to configure authentication for your network in pg_hba.conf. You need to make sure that the PostgreSQL user that you will create in the next step will have access to the server from a remote host. To do this, add a new line into pg_hba.con that has the following information:
host <database> <user> <network address> <mask> password
Start PostgreSQL Server
$ sudo service postgresql start
Use chkconfig utility to ensure that your PostgreSQL server will start at a boot time:
chkconfig postgresql on
You can use the chkconfig utility to verify that PostgreSQL server will be started at boot time, for example:
chkconfig --list postgresql
Step 2: Install the Postgres JDBC Driver
Before you can run the Hive metastore with a remote PostgreSQL database, you must configure a JDBC driver to the remote PostgreSQL database, set up the initial database schema, and configure the PostgreSQL user account for the Hive user.
To install the PostgreSQL JDBC Driver on a Debian/Ubuntu system:
Install libpostgresql-jdbc-java and symbolically link the file into the /usr/lib/hive/lib/ directory.
$ sudo apt-get install libpostgresql-jdbc-java
$ ln -s /usr/share/java/postgresql-jdbc4.jar /usr/lib/hive/lib/postgresql-jdbc4.jar
Step 3: Create the metastore database and user account
bash# sudo –u postgres psql
bash$ psql
postgres=# CREATE USER hiveuser WITH PASSWORD 'mypassword';
postgres=# CREATE DATABASE metastore;
postgres=# \c metastore;
You are now connected to database 'metastore'.
postgres=# \i /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.10.0.postgres.sql
SET
SET
...
Now you need to grant permission for all metastore tables to user hiveuser. PostgreSQL does not have statements to grant the permissions for all tables at once; you'll need to grant the permissions one table at a time. You could automate the task with the following SQL script:
bash# sudo –u postgres psql
metastore=# \o /tmp/grant-privs
metastore=# SELECT 'GRANT SELECT,INSERT,UPDATE,DELETE ON "' || schemaname || '"."' || tablename || '" TO hiveuser ;'
metastore-# FROM pg_tables
metastore-# WHERE tableowner = CURRENT_USER and schemaname = 'public';
metastore=# \o
metastore=# \i /tmp/grant-privs
You can verify the connection from the machine where you'll be running the metastore service as follows:
psql –h myhost –U hiveuser –d metastore
metastore=#
Step 4: Configure the Metastore Service to Communicate with the PostgreSQL Database
change the IP of the AWS amazon master Server, or your master server, don't use DNS name
$find / -name hive-site.xml
$nano /run/cloudera-scm-agent/process/27-hive-metastore-create-tables/hive-site.xml
in the File search for:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:postgresql://myhost/metastore</value>
</property>
and change to the correct IP that is your Master Hadoop Server where u are running Cloudera Manager
also every link in that file that is not correctly write to the hadoop master Cloudera manager connector , you will have to change to the correct IP
after all this just get back to the autoinstall of cloudera manager and run again and it will be all good :)
that it all the installation that you have to work around our contract cloudera support (that's their business) :)
all this it works fine for me when i have this problem in de cloudera CDH 4.X with sorl
Regards
Go to this link :
http://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
Go to this topic: Installing the MySQL JDBC Driver
Follow the instruction. Finally Restart your hive service
Thx Kumar