I am new to Snowflake, I need to export some tables from our Snowflake DW to SQL Server which is hosted on AWS Cloud. and can we automate it like a daily Job somewhere to fetch realtime data from Snowflake? can somebody guide me with the steps needed for this?
The easiest way - without buying a third party tool - might be a SQL Server linked server to Snowflake. The steps are documented: https://snowflakecommunity.force.com/s/article/how-to-set-up-a-linked-server-from-sql-server-to-snowflake
Once you've setup the linked server a recurring SQL Server agent job can run the pull.
Related
The client has on-premise SQL Server on his side. We want to replicate his database to our PostgreSQL Azure. The replication we want can have maximum 1min delay. ADF is not the option - copying is too frequent.
What tools can we use? We thought about Azure Sync or SymetricDS. Eventually Would SQL Server -> SQL Server be easier to execute? Any suggestions even beyond Azure are much appreciated.
As #Martin Cairney said on-prem SQL Server -> Azure PostgreSQL doesn't exist in Azure DataSync, so you would need to make use of offical or a 3rd party replication tool. Azure DataSync only support Azure SQL as the Hub Database.
Offical tool, such as Azure data factory. You can set a trigger condition to reduce the frequency of copy activities.
3rd party tool, sunch as skyvia. You said SymmetricDS maybe not support Azure PostgreSQL, according to its official doc.
I am somewhat surprised (still after all these SQL Server Installed Instance (Windows VM Azure) that pushing data, on a nightly schedule, to an SQL Azure database is not straight forward. I see some articles and direction to 'migrate' schemas and data, but what about a nightly job to push from my SQL Server instance to individual client SQL Azure data stores?
Should I start with SSIS? Azure data factory? Python libraries? Why isn't a connection between the two 'native'?
Again, all links and references so far have been for one time migration. I want the two in a data Eco-system with reliable flow.
John
We do this using SSIS running from the on prem side, because we already have a bunch of SSIS projects hosted on prem, and have yet to migrate anything into azure data factory. We are using SQL authentication to make the connection to the SQL Azure database.
We have 2 SQL Server databases. One of them is to be moved to AWS cloud.
Is Amazon RDS the ONLY approach to move the SQL Server to AWS?
What is the best way to automate data transfer of table data from the SQL Server that IS in the cloud to the SQL Server that is NOT in the cloud?
The volume and complexity of data is not high in our case. We need to migrate only ONE table and we'd like to automate it to run every night.
Would AWS Glue be useful for this? Would it help in loading data from INSIDE the cloud to OUTSIDE?
I also just found DMS - https://aws.amazon.com/dms/
But the homepage says "Migrate your databases **to** AWS with minimal downtime"
Can I do the opposite with DMS? Can I migrate data **FROM** AWS to outside AWS?
Amazon RDS is not the only approach to host SQLServer in the cloud. You can use an EC2 instance and install SQLServer manually. This could make sense in case you want to have sys_admin privileges. But be careful, as you would have to do patching all by yourself. I would not recommend doing that.
DMS is a good way to migrate your data every night. The service you are looking for is continuous data replication from DMS. Glue can also be used for this task. Glue's purpose was never to do database replication. Hence, it could be a pain monitoring the replication & debug in case of errors. That is why I would recommend DMS.
We want to pull data from Redshift database into SQL Server. Currently, we are using SQL Workbench to analyze Redshift database. We referred the following link -
Connect Your Cluster By using SQL Workbench
Here we used - RedshiftJDBC41-1.1.17.1017 driver to connect.
Now we want to push this data to DWH in SQL Server 2016.
So what is the best way to accomplish this? There is not much information on net on how to pull data from Redshift.
Can we have SSIS component or something in SSDT/SSIS2015 to load this data?
Or can we have a middle layer of any tool (any ETL tools or Excel) which will dump this Redshift data into CSV or something which will be readable in SSIS?
Data from Amazon Redshift can be extracted and consequently loaded into almost any platform.
You have several options to move data from Redshift to SQL Server.
ETL Tool - You can use a commercial ETL tool. SSIS is a
perfectly legitimate way to extract from Redshift and to land the
data in SQL Server.
S3 Files - You can unload the data from Redshift into S3 buckets and
then use SSIS or bcp to copy data from buckets to your SQL Server
Local file system -- You can run the unload command to extract data
to local file system. You can have psql running locally in a server
inside of your network.
For information on how to connect SSIS and any other ETL or BI tool to Redshift please see this link.
Amazon Redshift provides ODBC Connectors which is not supported by any Sql Server BI Tools. MS BI stack only support OLE DB Providers and it is being provided by a third party vendor
I am trying to build SSAS cube sourcing data from Amazon RedShift.
I need to create a new db in SQL Server, and move some data from an Oracle db to this SQLServer db;
I read that this function can be implemented by using SSIS or the SQL Server Migration Assistant for Oracle;
I wonder if someone can point me to a tutorial or a document where I can see how this task is performed step by step
The data moved from the Oracle db is updated on a day basis, so the data moved has to be updated regularly.
Have you looked at the SQL Server Migration Assistant for Oracle?
Step by step document on how to use it.
How about this:
http://aspalliance.com/947_building_a_sql_server_2005_integration_services_package_using_visual_studio_2005
and this:
http://msdn.microsoft.com/en-us/library/ms169917.aspx