Unable to Import table from RDMS from Hive - database

I am Trying to Import table from RDMS to Hive but It is giving error. Sqoop eval command working fine, able fetch records.
sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=false" \
> --connect "jdbc:tibero:thin:#hostname:8629:DBI" \
> --driver com.tmax.tibero.jdbc.TbDriver \
> --username XXX --password XXXX \
> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 15:16:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 15:16:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 15:16:38 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
21/09/27 15:16:38 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
21/09/27 15:16:38 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
21/09/27 15:16:38 INFO manager.SqlManager: Using default fetchSize of 1000
21/09/27 15:16:38 INFO tool.CodeGenTool: Beginning code generation
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.0.1.0-187/hadoop-mapreduce
21/09/27 15:16:41 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/8fd46edea8696bb5156e637113626b10/DMSDBA.cmm_cadorg_Tb.jar
21/09/27 15:16:41 ERROR tool.ImportTool: Import failed: No primary key could be found for table DMSDBA.cmm_cadorg_Tb. Please specify one with --split-by or perform a sequential import with '-m 1'.
I also tried to pass --split-by
> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1\
> --split-by DORG_UPDT_EMP_NO
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 13:41:10 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 13:41:10 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 13:41:10 ERROR tool.BaseSqoopTool: Error parsing arguments for import:
21/09/27 13:41:10 ERROR tool.BaseSqoopTool: Unrecognized argument: DORG_UPDT_EMP_NO
Also tried to pass MapReduce argument but again getting error
> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1\
> -m 4
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 15:16:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 15:16:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 15:16:38 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
21/09/27 15:16:38 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
21/09/27 15:16:38 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
21/09/27 15:16:38 INFO manager.SqlManager: Using default fetchSize of 1000
21/09/27 15:16:38 INFO tool.CodeGenTool: Beginning code generation
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.0.1.0-187/hadoop-mapreduce
21/09/27 15:16:41 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/8fd46edea8696bb5156e637113626b10/DMSDBA.cmm_cadorg_Tb.jar
21/09/27 15:16:41 ERROR tool.ImportTool: Import failed: No primary key could be found for table DMSDBA.cmm_cadorg_Tb. Please specify one with --split-by or perform a sequential import with '-m 1'.
Please help me on this. Thank You So much in Advance.
please Ignore formatting if there is any issue in format.

Try passing target directory.
sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" \
--connect "jdbc:tibero:thin:#hostname:8629:DBI" \
--driver com.tmax.tibero.jdbc.TbDriver \
--target-dir /project/whatever \
--username XXX --password XXXX \
--table DMSDBA.cmm_cadorg_Tb \
--hive-import \
--create-hive-table \
--hive-table DMSDBA.cmm_cadorg_Tb1

Related

openGauss installation error reporting app_0bd0ce80, no need to create symbolic link?

openGauss installation error app_0bd0ce80, no need to create symbolic link
The error message when executing the installation script is as follows, how to solve this?
[FAILURE] og01:
Using omm:dbgrp to install database.
Using installation program path : /opt/huawei/install/app_0bd0ce80
$GAUSSHOME points to /opt/huawei/install/app_0bd0ce80, no need to create symbolic link.
Traceback (most recent call last):
File "/opt/huawei/install/om/script/local/Install.py", line 645, in
functionDict[g_opts.action]()
File "/opt/huawei/install/om/script/local/Install.py", line 588, in startCluster
dn.start(self.time_out)
File "/opt/huawei/install/om/script/local/../gspylib/component/Kernel/Kernel.py", line 87, in start
+ " Error: Please check the gs_ctl log for "
Exception: [GAUSS-51607] : Failed to start instance. Error: Please check the gs_ctl log for failure details.
View the log, because of memory overflow, turn up the opengauss,
s_install \
-X /opt/software/openGauss/clusterconfig.xml \
-l /home/omm/gs_install.log \
--gsinit-parameter="--encoding=UTF8" \
--dn-guc="max_connections=10" \
--dn-guc="max_process_memory=2GB" \
--dn-guc="shared_buffers=128MB" \
--dn-guc="bulk_write_ring_size=128MB" \
--dn-guc="cstore_buffers=16MB" \
--autostart=yes

Karaf feature:install does nothing

I've downloaded ServiceMix 7.0.1 and followed the documentation. I need to install Camel features using:
karaf#root>feature:install camel-sql
But when I do that nothing happens. There's no response until I hit ctrl-c. The log file doesn't help either.
2020-03-25 14:13:25,709 | INFO | Thread-16 | FeaturesServiceImpl
| 8 - org.apache.karaf.features.core - 4.0.9 | Adding features: camel-sql/[2.24.2,2.24.2]
And the online documentation is sparse to say the least.
Update
Using -v -t gives the below output, but still nothing thereafter:
karaf#root>feature:install -v -t camel-sql
Adding features: camel-sql/[2.24.2,2.24.2]
You have to update Maven configuration in etc/org.ops4j.pax.url.mvn.cfg in your ServiceMIX 7.0.1 installation. Change:
org.ops4j.pax.url.mvn.repositories= \
http://repo1.maven.org/maven2#id=central, \
http://repository.springsource.com/maven/bundles/release#id=spring.ebr.release, \
http://repository.springsource.com/maven/bundles/external#id=spring.ebr.external, \
http://zodiac.springsource.com/maven/bundles/release#id=gemini, \
http://repository.apache.org/content/groups/snapshots-group#id=apache#snapshots#noreleases, \
https://oss.sonatype.org/content/repositories/snapshots#id=sonatype.snapshots.deploy#snapshots#noreleases, \
https://oss.sonatype.org/content/repositories/ops4j-snapshots#id=ops4j.sonatype.snapshots.deploy#snapshots#noreleases
to:
org.ops4j.pax.url.mvn.repositories= \
https://repo1.maven.org/maven2#id=central, \
http://repository.springsource.com/maven/bundles/release#id=spring.ebr.release, \
http://repository.springsource.com/maven/bundles/external#id=spring.ebr.external, \
http://zodiac.springsource.com/maven/bundles/release#id=gemini, \
http://repository.apache.org/content/groups/snapshots-group#id=apache#snapshots#noreleases, \
https://oss.sonatype.org/content/repositories/snapshots#id=sonatype.snapshots.deploy#snapshots#noreleases, \
https://oss.sonatype.org/content/repositories/ops4j-snapshots#id=ops4j.sonatype.snapshots.deploy#snapshots#noreleases
repo1.maven.org no longer allows access using http. https is required.
Can you please try bundle:install mvn:org.apache.camel/camel-sql/2.24.2

Custom build of openSSL for OS X - limit ciphers suites

Main goal
I need openSSL. Apple deprecated this library since 10.7 It is even impossible to create context when ignoring warnings about deprecation.
I decided to build openSSL and tweak it to my needs:
disable SSL protocol (only TLS should be supported)
limit cipher suites to secure one
What I have done
How I'm trying to build it.
I've downloaded sources version 1.0.2d and configured it this way:
./Configure -openssldir=$(pwd)/output-build \
enable-ec_nistp_64_gcc_128 \
no-ssl2 no-ssl3 \
no-shared no-comp \
enable-TLS_DHE_RSA_WITH_AES_256_CBC_SHA \
enable-TLS_DH_RSA_WITH_AES_256_CBC_SHA \
enable-TLS_DHE_DSS_WITH_AES_256_CBC_SHA \
enable-TLS_DH_DSS_WITH_AES_256_CBC_SHA \
enable-TLS_RSA_WITH_AES_256_CBC_SHA \
enable-TLS_DHE_RSA_WITH_AES_128_CBC_SHA \
enable-TLS_DH_RSA_WITH_AES_128_CBC_SHA \
enable-TLS_DHE_DSS_WITH_AES_128_CBC_SHA \
enable-TLS_DH_DSS_WITH_AES_128_CBC_SHA \
enable-TLS_RSA_WITH_AES_128_CBC_SHA \
enable-TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA \
enable-TLS_RSA_WITH_3DES_EDE_CBC_SHA \
darwin64-x86_64-cc
Problem
Switches for disabling SSL are clear and I'm happy with it.
Switches for cipher suites are very inconvenient, I'm not sure how they work. They are not documented, ./Configure --help prints this:
Usage: Configure [no-<cipher> ...] [enable-<cipher> ...]
[experimental-<cipher> ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx]
[no-hw-xxx|no-hw] [[no-]threads] [[no-]shared]
[[no-]zlib|zlib-dynamic] [no-asm] [no-dso] [no-krb5] [sctp] [386]
[--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]]
[--test-sanity] os/compiler[:flags]
I just want listed cipher suites and nothing else. Format of this switches give me impression that I have to disable quite long list of other cipher suites. Is there better way so I can disabled all but those listed above?

Unknown parameter "includeObjects" throws exception in command line liquiBase

I'm using liquibase 3.3.2 and i'm trying use the "includeObjects" parameter. This is the command line:
java -jar liquibase.jar \
--driver=com.ingres.jdbc.IngresDriver \
--url="jdbc:ingres://localhost:21071/local" \
--username=ingres \
--password=ingres \
--logLevel=debug \
--classpath="iijdbc.jar" \
"--currentDateTimeFunction=now()" \
--changeLogFile=db.changelog.xml \
--defaultSchemaName=ingres \
--includeObjects=personal \
generateChangeLog
This is the exception thrown.
Unexpected error running Liquibase: Unknown parameter: 'includeObjects'
SEVERE 24/03/15 10:05 AM: liquibase: Unknown parameter: 'includeObjects'
liquibase.exception.CommandLineParsingException: Unknown parameter: 'includeObjects'
at liquibase.integration.commandline.Main.parseOptions(Main.java:702)
at liquibase.integration.commandline.Main.run(Main.java:124)
at liquibase.integration.commandline.Main.main(Main.java:94)
Can anyone point what I am doing wrong?
Supposedly in version 3.3.2 the "includeObject" option is now available:
Liquibase 3.3.2 Released
The option --includeObjects is a parameter that belongs to the commands diff or diffChangeLog.
I guess it can not be used on the command generateChangeLog.
As Jens mentioned in the comment, the following should work as the include/excludeObjects arguments come AFTER the generate/diffChangeLog command apparently:
java -jar liquibase.jar \
--driver=com.ingres.jdbc.IngresDriver \
--url="jdbc:ingres://localhost:21071/local" \
--username=ingres \
--password=ingres \
--logLevel=debug \
--classpath="iijdbc.jar" \
"--currentDateTimeFunction=now()" \
--changeLogFile=db.changelog.xml \
--defaultSchemaName=ingres \
generateChangeLog --includeObjects=personal
I did the same approach (with mulitple tables comma-seperated) and it worked fine.

How to find methods exposed in a D-Bus interface

Question: Is there any way in which we can find out the methods (and their signatures) which are exposed in a D-Bus interface?
Issue Description: In my phone, I am calling BlueZ methods using D-Bus to adapter interface, when checked on phone 2 of these methods are not available.
Intention is to check if the method name/signatures are modified in other device, I don't have access to code so looking to find the methods in an interface
Using dbus-send, you can list the available services on your system:
Session:
dbus-send --session \
--dest=org.freedesktop.DBus \
--type=method_call \
--print-reply \
/org/freedesktop/DBus \
org.freedesktop.DBus.ListNames
System:
dbus-send --system \
--dest=org.freedesktop.DBus \
--type=method_call \
--print-reply \
/org/freedesktop/DBus \
org.freedesktop.DBus.ListNames
You'll get an answer like that:
array [
string "org.freedesktop.DBus"
string ":1.1"
string ":1.26"
string "org.asamk.Signal"
]
And if you want to list all methods available behind a dbus service, you can still use dbus-send to introspect the dbus service.
For example with org.asamk.Signal:
dbus-send --system --type=method_call --print-reply \
--dest=org.asamk.Signal \
/org/asamk/Signal \
org.freedesktop.DBus.Introspectable.Introspect
You'll get this kind of result (truncated)
<node name="/org/asamk/Signal">
<interface name="org.asamk.Signal">
<method name="sendMessage" >
...parameters
</method>
<method name="sendGroupMessage" >
...parameters
</method>
</interface>
</node>
Here there are 2 methods, sendMessage and sendGroupMessage
You can also take a look at D-Feet.
With an extra google search and dbus understanding, using D-Bus Introspection helps to get the methods (with signatures) exposed on that particular interface. More information available at link.

Resources