I'm running through the Cloudera Manager (free edition) and I reached the point where the wizard is creating the Hive Metastore Database.
This error is shown and halts the configuration process.
using /var/run/cloudera-scm-agent/process/40-hive-metastore-create-tables/hadoop-conf as HADOOP_CONF_DIR
I cant seem to find any information that might cause this?
Every thing has been configured correctly up to this point, everything installed and user names and passwords are correct.
Has anybody seen this error before? Thoughts?
Error Log:
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:688)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1094)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2337)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at com.cloudera.enterprise.dbutil.SqlRunner.open(SqlRunner.java:109)
at com.cloudera.enterprise.dbutil.SqlRunner.runSingleQuery(SqlRunner.java:80)
at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.countTables(HiveMetastoreDbUtil.java:191)
... 2 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2540)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:612)
... 20 more
ok cloudera is using version HIVE 0.10, that doesn't support remote login
but you need to go around that bug, logging to the server that is getting the error the cloudera manager will tell you the ip
1) create login in to the Server that fails to install HIVE
2) Create a $HADDOP_HOME
$HADOOP_HOME="/usr/lib/hadoop/"
3) INSTALL postgres in the server that fails
`$ sudo apt-get install postgresql`
$ cat /etc/postgresql/9.1/main/postgresql.conf | grep -e listen -e standard_conforming_strings
modify this to lines in the file
listen_addresses = '*'
standard_conforming_strings = off
You also need to configure authentication for your network in pg_hba.conf. You need to make sure that the PostgreSQL user that you will create in the next step will have access to the server from a remote host. To do this, add a new line into pg_hba.con that has the following information:
host <database> <user> <network address> <mask> password
Start PostgreSQL Server
$ sudo service postgresql start
Use chkconfig utility to ensure that your PostgreSQL server will start at a boot time:
chkconfig postgresql on
You can use the chkconfig utility to verify that PostgreSQL server will be started at boot time, for example:
chkconfig --list postgresql
Step 2: Install the Postgres JDBC Driver
Before you can run the Hive metastore with a remote PostgreSQL database, you must configure a JDBC driver to the remote PostgreSQL database, set up the initial database schema, and configure the PostgreSQL user account for the Hive user.
To install the PostgreSQL JDBC Driver on a Debian/Ubuntu system:
Install libpostgresql-jdbc-java and symbolically link the file into the /usr/lib/hive/lib/ directory.
$ sudo apt-get install libpostgresql-jdbc-java
$ ln -s /usr/share/java/postgresql-jdbc4.jar /usr/lib/hive/lib/postgresql-jdbc4.jar
Step 3: Create the metastore database and user account
bash# sudo –u postgres psql
bash$ psql
postgres=# CREATE USER hiveuser WITH PASSWORD 'mypassword';
postgres=# CREATE DATABASE metastore;
postgres=# \c metastore;
You are now connected to database 'metastore'.
postgres=# \i /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.10.0.postgres.sql
SET
SET
...
Now you need to grant permission for all metastore tables to user hiveuser. PostgreSQL does not have statements to grant the permissions for all tables at once; you'll need to grant the permissions one table at a time. You could automate the task with the following SQL script:
bash# sudo –u postgres psql
metastore=# \o /tmp/grant-privs
metastore=# SELECT 'GRANT SELECT,INSERT,UPDATE,DELETE ON "' || schemaname || '"."' || tablename || '" TO hiveuser ;'
metastore-# FROM pg_tables
metastore-# WHERE tableowner = CURRENT_USER and schemaname = 'public';
metastore=# \o
metastore=# \i /tmp/grant-privs
You can verify the connection from the machine where you'll be running the metastore service as follows:
psql –h myhost –U hiveuser –d metastore
metastore=#
Step 4: Configure the Metastore Service to Communicate with the PostgreSQL Database
change the IP of the AWS amazon master Server, or your master server, don't use DNS name
$find / -name hive-site.xml
$nano /run/cloudera-scm-agent/process/27-hive-metastore-create-tables/hive-site.xml
in the File search for:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:postgresql://myhost/metastore</value>
</property>
and change to the correct IP that is your Master Hadoop Server where u are running Cloudera Manager
also every link in that file that is not correctly write to the hadoop master Cloudera manager connector , you will have to change to the correct IP
after all this just get back to the autoinstall of cloudera manager and run again and it will be all good :)
that it all the installation that you have to work around our contract cloudera support (that's their business) :)
all this it works fine for me when i have this problem in de cloudera CDH 4.X with sorl
Regards
Go to this link :
http://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
Go to this topic: Installing the MySQL JDBC Driver
Follow the instruction. Finally Restart your hive service
Thx Kumar
Related
I've upgraded MariaDB from 5.5 to 10.2 (on CentOS 7.7) following this tutorial: https://support.plesk.com/hc/en-us/articles/213403429
MariaDB 10.2 is running. User/Websites can connect to their databases. But I can't connect with admin anymore.
MYSQL_PWD=`cat /etc/psa/.psa.shadow` mysqldump -u admin --all-databases --routines --triggers > /tmp/all-databases-new.sql
Version check failed. Got the following error when calling the 'mysql' command line client
ERROR 1045 (28000): Access denied for user 'admin'#'localhost' (using password: YES)
FATAL ERROR: Upgrade failed
I tried to set a new password for admin but...
systemctl stop mariadb
[root#host ~]# mysqld_safe --skip-grant-tables --skip-networking &
[1] 76597
[root#host ~]# 200317 08:23:11 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
200317 08:23:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
[1]+ Done mysqld_safe --skip-grant-tables --skip-networking
[root#host ~]# mysql -u root
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111)
It's no longer possible to access the Plesk GUI because Plesk can't access the psa database anymore. What to do now?
Many thanks in advance!
The reason why "mysqld_safe --skip-grant-tables --skip-networking &" failed was, that Mariadb could not write the pid-file. Mariadb tried to write it it in /var/run/mariadb but that directory was non existent. You can create the directory or change the configuration, so that Mariadb uses another directory for the pi-file. After that it was possible to execute "mysqld_safe --skip-grant-tables --skip-networking &" and to set a new password for the database user "admin".
For all internal purposes, Plesk gets the database password from /etc/psa/.psa.shadow. So I put it in plain text, which works but of course plain text is bad. Does anybody know how to store the password as hash again? I think it was stored as AES128 hash, which needs the right secret?
I created an oracle database on docker container using the following command:
docker run -d --name oracle -p 8080:8080 -p 1521:1521 quay.io/maksymbilenko/oracle-12c
The db params are:
hostname: localhost
port: 1521
sid: xe
service name: xe
username: system
password: oracle
I would like to enable unified audit, perform a query such as SELECT and see it on the audit log.
I have access to this DB with sqlplus using the following command:
sqlplus system/oracle#//localhost:1521/xe
or via the app dbeaver.
First issue:
According to what I have found, in order to enable unified audit I need to Shutdown the database.
This can be done by using sqlplus and running the next command:
shutdown immediate
When I did that, the action has failed because: ORA-01031: insufficient privileges
Is there any way that I could just run a regular query to enable the unified audit?
Do I have to shut it off before?
Is there any way that I could do all that I want just with running queries? ( enable, create audit and view actions)
I successfully installed the offical MSSQL ODBC drivers in Linux and I can connect to every database and execute commands with isql:
sudo isql -v dsn user pw
However, if I execute a SQL statement (select ...) in my own application it is not possible to fetch data ("Driver does not support this function") -> I think it is because isql needs sudo rights.
Is there any way to execute isql without sudo?
I tried to change (chmod, chown) some rights for isql, but I still need sudo.
I found the solution. The problem was not isql, but the location of odbc.ini and odbcinst.ini!
As "sudo" the connection is looking at /etc/odbc.ini and /etc/odbcinst.ini
As "normal" user the connection is looking at a path like that: /usr/local/odbc.ini and /usr/local/odbcinst.ini
I edited both files and the connection worked!
this should not be the case. Can you try a SELECT ##Version? Alternatively, can you try running the same query via sqlcmd?
Steps to install and run sqlcmd are here: https://blogs.msdn.microsoft.com/sqlnativeclient/
I have a tar.gz which is supposed to contain a complete Postgres 9.4 database backup.
I attemped to restore the database using pg_restore, but I realised (after I unpacked the file) that it has the same structure as the /var/lib/postgresql/9.4/main directory running on my localhost.
So I stopped the postgresql service running on localhost, I replaced my main directory with the one from the archive as described here: Restore postgresql from files.
Moreover I've set owner and permissions in a recursive manner like this:
chown -R postgres:postgres /var/lib/postgresql/9.4/main/
chmod -R 700 /var/lib/postgresql/9.4/main/.
I've restarted the service:
root#EED-015-LIN:~# service postgresql status
9.3/main (port 5433): online
9.4/main (port 3007): online,recovery
...But when I try to connect using psql
postgres#EED-015-LIN:~$ psql -p 3007
psql: FATAL: the database system is starting up
What am I missing?
i can only find psql command, but can't find any other postgres commands or tools, can
anyone tell me how to create database and connect to it using the default postgres
shipped with mac lion?
only if it doesn't work ,i dont' want to install another postgres instance.
PostgreSQL will be started by Launchd, if configured so.
Look for /System/Library/LaunchDaemons/org.postgresql.postgres.plist, this is an XML file and you will need to modify some settings there, like listen_address.
Default location for the PGDATA files is /var/pgsql, also found in the plist file above.
Cluster is owned by the _postgres user, so to access the “tools”, you should execute them via the sudo -u _postgres … prefix, like: sudo -u _postgres pg_ctl -D /var/pgsql -w start
I highly recommend creating another role with superuser permissions, as _postgres is sharing password with the Administrator account of your Lion.
You can also use the Server Admin tools for Lion to control PostgreSQL and other services:
sudo serveradmin list
# look for postgres in the output
sudo serveradmin fullstatus postgres
sudo serveradmin stop postgres
sudo serveradmin start postgres
Check the version of your PostgreSQL cluster, using: sudo -u _postgres psql -tc "select version()" and refer to the official manuals on how to create the database and setup initial access. Manual for the latest PostgreSQL version can be found here.
If you do not have the software installed, I strongly recommend the Enterprise DB package. It makes installing pretty darn easy.