I am in an odd situation where I cannot connect to the server using python. I can however connect to the server in other ways using SQL Server Management, so from that end I can execute any query. The problem however is parsing in pandas, data retrieved from SQL Manager. As far as I am aware, data from SQL Manager can be retrieved as csv, txt or rpt. Parsing any of these formats is a pain in the neck and it's not always the same for all tables. My question is then, what is the fastest way to parse any of the file formats that SQL Manager can output in pandas? Is there a standard format that SQL Manager can output and which is parsed the same way in pandas for all tables? Has anyone faced this problem, or is there another workaround?
Related
I want to copy tables in various schemas from SQL Server to snowflake. I understand that snowflake COPY works well to load huge amount of data into snowflake, provided I have CSV data as input.
However, I am unable to figure out an efficient way to export SQL Server data in CSV format. I went through some of the threads in this forum on this topic and found that PowerShell tool export-csv is a good option. But does it work well with thousands of tables in SQL Server?
If not, what other option should I try to move the data from SQL Server to snowflake? Please note that this is not a one time data load. I am looking for a process that can run daily to load data from SQL Server to snowflake.
Thanks in advance!
P.S: I tried the SQL Server bcp tool. But it doesn't generate a standardized CSV file.
I need to join SQL Server data with some data from Essbase cube. Result will be in SQL Server. Is it possible to create linked essbase cubu in SQL Server 2014 to query data? Or any other way how to get data physically from essbase to SQL Server not using export files? Something like direct connect.
Thanks,
There aren't any native ways to implement what your want. Setting this up will typically require some sort of automation to extract the data in the cube to a CSV file, then loading that data using SSIS or your preferred ETL tool. The automation to extract data will frequently be a combination of batch file, MaxL and report script or a calc script with a data export command.
Working on a data data accuracy project. I have to find a way to compare data from a query from a SQL Server db with the data from a query from a MS Access db. The data on both db's should be identical, but sometimes there are errors. I have looked at data comparison tools but these seem to only be able to compare data from identical db vendors.
Is there a process that someone has used in the past to do this or an idea on how I might best approach this?
You can look at both data sets in Access, SQL, or Excel:
If the data set is small enough, I recommend Excel.
If you know SQL, you can export your Access data to text files, then do a Bulk Insert and get everything into SQL Server.
If you want to look at both data sets in Access, try this:
Go to your ODBC Data Source Administrator (searching for 'ODBC' from your Start menu should be sufficient)
Create a new System DSN connecting to your SQL Server db
Open your Access db (I'm using 2010, your version may be different)
Go to External Data->ODBC Database->Machine Data Source
Link to your tables of choice from your SQL Server
Query away!
I am quite new to SQL Server but I'm looking for a tool that integrates Excel with SQL Server and provide a two way connection Read/write.
I want to be able to pull data from SQL server and perform some evaluation/data manipulation and then write the data back to the server.
Basically my client receives Excel raw data from vendors which they perform some validation on the spreadsheet then send the spreadsheet back but a copy of the data needs to be in some sort of data management system. I have test MDS and I'm not full satisfied. The functionality I'm looking for is
Data validation
Data match - match and merge /consolidate two or more worksheets into one
read/write to sql
I Do not want the import/export wizard and don't want to use SSIS and they are both not suitable.
There's Google - like it has never been before. And there are quite a big number of subject experts posting on their blogs for the love of helping people like you and me.
So check out here multiple ways you can import data into SQL Server, without using SSIS such as,
bcp Utility
e.g.
bcp dbo.ImportTest in 'C:\ImportData.txt' -T -SserverName\instanceName
Bult Insert using T-SQL
e.g.
`BULK INSERT dbo.ImportTest`
`FROM 'C:\ImportData.txt'`
`WITH ( FIELDTERMINATOR =',', FIRSTROW = 2 )`
Note the article was published and last updated in 2012. So you may further check the compatibility for older versions if you are using any.
PS: I still believe you could be using SSIS as not to Re-invent the wheel...
I am using Enterprise Miner 6.2 and want to create a data source but my option is a SAS Table. How do I go about exporting SQL Server or Excel data into a SAS table?
SAS has many ways of connecting to and/or reading data from disparate sources. I haven't used Enterprise Miner, so I'm not sure which of SAS' methods are available to you directly from within EM, but it's likely there will be someone at your site who has some interface to Base SAS and who can help you/advise what data access products are installed and how you can use them.
For SQL Server data, SAS/Access to SQL Server or SAS/Access to OLE DB will allow you to read directly from SQL Server tables in place. Alternatively, someone could provide you with a dump of the data you need from the SQL Server database.
For Excel data, there are also SAS/Access products, but SAS also has native capabilities to read in the data if saved as, for example, a .csv or .txt file.
To help answer you further, perhaps can you come back with some details about what SAS products/interfaces are available to you?