I am using a SqlServer database and need to connect to a Hive database. The end goal is to be able to push data from SqlServer to a Hive table. Connecting to SqlServer from Hive via Sqoop is not an option. How would I accomplish this?
How big is the data ? If your database contains few tables than you can export the sqlserver db to excel or csv and load them into hive else you can write some code to accompolish this.
Related
I want to copy tables in various schemas from SQL Server to snowflake. I understand that snowflake COPY works well to load huge amount of data into snowflake, provided I have CSV data as input.
However, I am unable to figure out an efficient way to export SQL Server data in CSV format. I went through some of the threads in this forum on this topic and found that PowerShell tool export-csv is a good option. But does it work well with thousands of tables in SQL Server?
If not, what other option should I try to move the data from SQL Server to snowflake? Please note that this is not a one time data load. I am looking for a process that can run daily to load data from SQL Server to snowflake.
Thanks in advance!
P.S: I tried the SQL Server bcp tool. But it doesn't generate a standardized CSV file.
Is there a way to perform a SQL Dump in a SQL Server? Because on mysql we can easily do that with mysqldump command but on a SQL Server when I try to export the data it seems that it's only allowing to export per table and not per database. Now when I try to do the copy database it tries to copy the source db to a destination db. Now I came from MYSQL and I thought that it will somehow behave the same way.
Any idea on how to achieve this?
Thanks for those who will reply
In SQL Server you would either use Backup and Restore, or extract all the schema and data into a .bacpac file.
I have a database called transportdb it contains certain Datatables with some data so now i want to export this complete MySql database with all data using sql query in Mysql
I am wondering if there is a way to replicate an oracle table over to a sql server table. Both servers are on the same network. I would like to have this automated perhaps using a stored procedure on the sql instance. Is there any way to do this automatically? Or can anyone point me to a good sql script to connect to an oracle db server and export a table file?
You could use an Oracle linked server on SQL-Server. In this way you would use the Oracle data directly from SQL-server.
Take a look at this tutorial: http://blogs.msdn.com/b/dbrowne/archive/2013/10/02/creating-a-linked-server-for-oracle-in-64bit-sql-server.aspx
You can use the Import Data function located under Tasks when you right click on a database in SQL Server Management Studio. It will walk you through picking the Source, the Destination, and how the columns map to each other.
Look at this:
http://www.microsoft.com/en-us/download/details.aspx?id=28766
Maybe it should help you
It is possible to export data from HDFS to RDBMS table using Sqoop.
But it seems like we need to have existing table.
Is there some parameter to tell Sqoop do the 'CREATE TABLE' thing and export data to this newly crated table?
If yes, is it going to work with Oracle?
I'm afraid that Sqoop do not support creating tables in the RDBMS at the moment. Sqoop uses the table in RDBMS to get metadata (number of columns and their data types), so I'm not sure where Sqoop could get the metadata to create the table for you.
You can actually execute arbitrary SQL queries and DDL via sqoop eval, at least with MySQL and MSSQL. I'd expect it to work with Oracle as well. MSSQL example:
sqoop eval --connect 'jdbc:sqlserver://<DB SERVER>:<DB PORT>;
database=<DB NAME>' --query "CREATE TABLE..."
--username <USERNAME> -P
I noticed you use Oracle too. Certain sqoop vendor-specific sqoop connectors support that, including Oracle. Sqoop's Oracle direct connect mode has option to do that
https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_create_oracle_tables
24.8.5.4. Create Oracle Tables
-Doraoop.template.table=TemplateTableName
Creates OracleTableName by replicating the structure and data types of
TemplateTableName. TemplateTableName is a table that exists in Oracle
prior to executing the Sqoop command.
ps. You'll have to use --direct sqoop export option to activate sqoop direct mode = 'Data Connector for Oracle and Hadoop' (aka OraOOP - older name).