I can't really find anything online about how to do this.
There are a few separate, offline, Microsoft Databases existing...
Everyone has begun staging different .accdb files in an Amazon S3 bucket - I'm hoping Snowflake now provides an easy (ish) solution to reading them into the SQL database I'm building.
The short answer is that you can't. Snowflake can import text files in various formats (csv, XML, JSON, etc) but is has no extract capabilities so it can't connect to applications and read data from them: asking it to read a MS Access file is no different from asking it to read an Oracle or SQL Server file.
You probably have 2 options:
Export the data from MS Access to a file format that Snowflake
can ingest
Use an ETL tool that can read from MS Access and
write to S3 as text files (or directly to Snowflake, which is
probably simpler
You should be able to connect to Snowflake in Microsoft Access through an ODBC connection. You first need to install the Snowflake ODBC Driver and configure a DSN.
Related
I've been trying to sort through Microsoft's extensive documentation, but, cannot find the answer I'm looking for, hence, posting it here for the experts!
I have a table in a database in MS SQL Server 2016, that I read/write to using MS SSMMS. I would like to export this single table into my Azure storage account for further analysis in the MS Data Science Virtual Machine, but cannot find a way to do this. Any suggestions?
Thanks.
You can also use tools built into the MS Data Science Virtual Machine (DSVM) to first export from SQL Server to CSV. BCP (command line) is one tool.
If you want a graphic tool, use SSMS and use the "import and export" option to save result of your query to CSV file. Then you can copy the CSV file to a Azure storage account using Azcopy (command line) or Azure Storage explorer (graphical) also available on the DSVM. Hope this helps.
Hi all I have an Alteryx workflow that is designed to read data from Amazon Redshift with a Connect In-DB tool. I've been provided CSV files but I don't have access to Redshift, I want to run the workflow but I can't figure out how to import a CSV into the Connect In-DB tool. Any ideas how to do this?
Alteryx Connect In-DB tool flow image
As #johnjps111 mentioned in the comment, you need to replace all the in-database tools with an Input Tool that reads the .csv. This assumes that the .csv was filtered and summarized to form the same output that would be coming from the in database tools. Otherwise you would need to replicate them with their normal counterparts.
I use several Microsoft Access databases on a regular basis to create reports. To get the source data, I currently have to log in to SAP BW (via SAP NetWeaver), run the source data report, export the results as a .csv file (but actually saving it as a .txt file), and then import that file into Microsoft Access. Is there a way that I can have Access pull the data from SAP BW directly?
Any help is appreciated!
All of the databases used by SAP are industry standard databases and the data is thus going to be stored in a system that supports ODBC.
As far as I know, SAP in general uses Sybase which is also what SQL server was originally based on.
So SAP is running on an industry standard SQL server (Sybase or SQL server). If running on IBM, then the data is in DB2 (often the as400 system).
You thus simply need to contact your IT department and obtain the required ODBC connection strings to the database. You “might” also need to install the latest Sybase drivers if you not running SAP on SQL server but again such information would be available from your SAP support folks.
So you simply setup linked tables in access to the SAP database, and thus no export or download or importing of data is required – you be reporting on live data at all times. The “challenge” is thus of course to grasp the table structures in SAP - a LARGE challenge since in most cases a report you been using for exports is the result of MANY related tables joined together into a "easy" view for exporting. So be prepared for some complex quires to get the data the way you want.
I have a .GDB database (old one) and the data in it is very important
I need to convert that .gdb database to a SQL Server database - can anyone help me...
Create connections to both source GDB and Destination SQL Server in ArcCatalog. Copy everything from source and paste it into the destination. You won't be able to do it with SQL tools alone.
Lacking ESRI software, for simple cases, my workflow is to use the GDAL C++ API to read the GDB. This requires the GDAL File GDB driver. Then I will use Microsoft.SqlServer.Types to transfer to SQL Server. This involves low-level APIs and you need to understand the spatial types in the respective libraries. It gets complex if you have polygons with rings, for example.
I'm not aware of a tool that will automatically convert between these database types. You'll need to use an application that can read the old database type (Firebase), learn the table design, create a similar table design in SQL Server, and use the application to load the data from Firebase to SQL Server.
Typically, this kind of work is called ETL (Extract/Transform/Load) and is done with migration tools like SQL Server Integration Service (SSIS). SSIS is free with SQL Server, and there are a lot of books available on how to use it - but like learning to develop software, this isn't a small task.
The easiest way to export Esri File Geodatabase FGDB (.gdb) data to MS SQL Server is with ArcGIS for Desktop at the Standard or Advanced level.
You may also want to try exporting to shapefile (SHP) format (an open transitional format) then import to your MS SQL Server. I've seen a tool online that has worked for me called Shape2SQL.
Esri also has an open File Geodatabase API that you can use to write your own too.
I highly recommend FME Workbench for GIS data conversion. It's like SQL Server Integration Services (ETL) but for GIS. Graphical interface, connect data readers with data writes, insert transforms, run them, etc.
I have a client that uses a point-of-sale solution involving an Access database for its back-end storage. I am trying to provide this client with a service that involves, for SLA reasons, the need to copy parts of this Access database into tables in my own database server which runs SQL Server 2008. I need to do this on a periodic basis, probably about 5 times a day.
Is there an easy programmatic way to do this, or an available tool? I don't want to handcraft what I assume is a relatively common task.
I am running this on SQL Azure, so there's no way for me to run prepackaged software on the server. It would either have to be open source and portable to Azure or executable on the client's computer.
I'm unfortunately thinking I'm going to have to roll my own tool to do this. Any suggestions or more tools that are out there that can do this themselves before I go ahead?
David, I looked at multiple solution for a similar problem: converting from dbf to mysql, here are 3 solutions (all commercial - but relatively inexpensive) that can work for you:
Full Convert
SQL Manager
ESF
Other than that I couldn't find a good robust data conversion tool that would be open source or free. At least not for DBF to MySQL conversion. There might be something out there for SQL/Access. You could roll out your own solution, but is it worth your time?
DISCLOSURE: I ended up using Full Convert.
Also all of these products generate some sort of batch file, that can be scheduled using Task Manager.
There are two things to consider:
connectivity
ETL tool
For connectivity, you will need to establish VPN tunnel of some sort between the client server and your server.
Then use SSIS to connect to MS Access, to create packages to pull data from MS Access to SQL Server database. On SQL Server, you will need to create new schema, to mirror or be close MS Access
On connectivity side, another option - since MS Access db is in the file, you may be able to FTP the file to your server and point SSIS to the file