I have a set of large CSV files with many columns each that I need to import into a SQL Azure database. Ordinarily I would use the import wizard in SQL Server Management Studio. However, the wizard does not appear to be an option when connecting to SQL Azure in SSMS. Is that correct? And if so, what is the recommended tool for accomplishing this task? I'm looking for a tool that will infer from the data what the columns should be allowing me to override the data type as needed. Since I have a lot of columns in each of the files I'd like to avoid the tedious work of manually writing the SQL code to generate the tables.
This worked for me:
Open SQL Server Management Studio
Connect to Azure
Right-click the database
Go to Tasks > Import Data
Select your flat file(s)
Upload to Azure SQL and create an SSIS package based on this workflow
I sometimes get errors with CSV files this way, but either using an Excel file or inspecting the options of the CSV data columns in the Import Wizard should suffice.
Make sure you have appropriate permissions assigned to your user account.
They could've / should've made this easier, like a SFTP + insert or a GUI import directly to Azure SQL like in Hue.
When you are transferring any data to SQL Database, the data should be structured. The proces will be to convert your CSV to a table structure and then migrate it directly to SQL Azure. Actually you can write a stored procedure in SSMS to do it all in one.
Because CSV file could be tab, comma, or any other character delimited, you can do bulk insert in local DB first as described here and then sync the table to SQL Azure.
Related
I want to connect an excel file to several tables in SQL Server. I learned how to make the connection by going to
Data - > Get External Data -> From Other Sources - > From SQL Server
and inputting the server name and selecting a table.
I want to select about 10 more tables. Is there a way to select another table since I have already set up the connection? Or do I have to go through this whole process each time?
In my experience the best way to do this is to use an Excel Add-In called Powerpivot: https://msdn.microsoft.com/en-us/library/gg413497(v=sql.110).aspx
It has a data import feature where you can import multiple tables at once from MS SQL Server (and actually supports import for many other database and file types).
I'm trying to copy a database from Azure to my local SQL Express 2014 instance.
My first try was to use the import wizard to create tables and import data but this creates tables without any constraint, identity specification, etc.
My next try was to create schema using a separated script and then use the wizard only for the data: now I have the problem that I have to manually enable identity insert and disable foreign keys. Even doing this I still get some errors so eventually abandoned this option.
Then I moved to SQLAzureMW and this generates some .dat files inside a folder (BCP I think) that I'm not certain how to easily import them in one shoot.
Could you please provide some guidence to get this resolved?
You use Data Tier Application. Using your Azure management portal you create the export (a backpac file) which can then be imported on-premise or into another Azure database.
Archive an Azure SQL database to a BACPAC file using the Azure Portal.
Import a BACPAC File to Create a New User Database
I've created the structure of my database first in PhpMyAdmin and exported it to a .sql file.
Now I'm looking everywhere in SQL Server Management Studio where I can import/add the data in a new database.
Does anybody where to look or what to click?
I'm using the 2014 version (CTP2)
If you have a .sql file which contains SQL statements, you can just copy and paste the contents (or open the file in a query window) and run it. This assumes it has all of the create table etc. statements to create the schema/structure and not just insert statements for the data.
Check the top of the file to make sure that it is first selecting the correct database, if not add a USE statement to select the correct database.
You didn't say how big the file was, but if it is quite large and has the insert statements (data as well as schema), then you'll probably want to run by CLI using sqlcmd command. Much faster and SSMS won't freak out.
Another alternative option to running the .sql file/code is to set up a data source for mysql and just use odbc to access the database itself.
Bear in mind that there are real and very annoying differences between mysql and t-sql that can make migration a pain. If you're just creating a few tables, it may not be an issue, but if there are a ton of tables with lots of fields of different data types, you may run into issues.
If you are looking to import table structure, you can copy-paste the content and run inside SSMS in a query window. Beware of syntax differences with MySQL and SQL Server. You will most likely get errors. You need to convert your SQL script from MySQL dialect to SQL Server dialect (or just add them manually if they are not too many). If you set the databases to a SQL standard-compatibility mode at the very beginning, you will have much less trouble.
If you are ONLY looking just to import the data into existing tables inside the SQL Server only, you can do the same (i.e. copy-paste and run in query window). You will have less trouble with that.
Open the server, open "Databases" and right click the database, go to "Tasks" and then Import Data...
I have had the most 'trouble free' success importing to SQL via a flat file method (comma delimited .txt file), the only stipulation when creating a flat file (i.e from Access) make sure the text identifier is set to {none} and not "".
To import the file: in the SQL Server Management Studio right click on Databases and create a new database. Then right click on the new database -> Tasks -> Import Data... The import window opens: in the DATA SOURCE option select Flat File Source and select the .txt file...click NEXT. In the DESTINATION field select SQL Server Native Client 11.0 and go through the import process. This worked very well for me.
Is there a way to do a batch update on SQL Server from a row of data in Excel? We have excel documents that contain 2000+ plus rows and need to be imported in SQL Server. Is there a way to do a batch insert of these guys without calling the database over and over to insert one row at a time?
SQL Server Integrations Services offers a wizard based import which can help you easily set up a package to import an excel file. You can even save the package and schedule it to repeat the import in the future.
You have other options, as well. If you save the excel file to a comma or tab delimited text file, you can use the BULK INSERT t-sql command. See an example on the sqlteam.com forums.
Another T-SQL option is SELECT INTO. Excel is a valid OLEDB or ODBC data source from T-SQL. Here's an example.
There's also a command line import tool included with Microsoft SQL Server called BCP. Good documentation on BCP and the other options can be found on MSDN at: http://msdn.microsoft.com/en-us/library/ms187042.aspx
You can create an SSIS package to read your Excel file. When you create your task, you can select a connection type of "Excel", and then it helps you create an "Excel Connection Manager". Then you can easily send the data to your SQL Server table. Here's a tutorial on how to import an Excel file into SQL Server (2005). Give it a look.
Yes! Use the import/export wizard of the SSMS! Use an Excel-source and a SQL Server destination. You can also create a SSIS-Package in the BIDS or use the BULK INSERT-statement from T-SQL, if you convert your Excel-sheets in to CSV-files.
Can SQL Server or db2 do entire database exports like oracle (using exp command)?
I've searched the internets and found bcp for SQL Server. But it seems I would have to iterate over all the tables to get what I want.
For db2 it looks to be roughly the same. Is there something I'm missing? Anyone have any suggestions and/or any opinions? Thanks ahead of time.
This is for SQL SERVER
Backup & Restore
To take an entire database with SQL Server, you can do a BACKUP and RESTORE
BACKUP: http://msdn.microsoft.com/en-us/library/ms186865.aspx
RESTORE: http://msdn.microsoft.com/en-us/library/ms186858.aspx
Export and Import
You can right click a database in SQL Server Management Studio, and under TASKS, click on EXPORT DATA. Follow the Wizard to choose the objects you want to export and put them into the appropriate location.
Custom SSIS for Raw format
Build a SSIS package that will read data from source table and put it into a RAW file on disk for later use. Raw files holds the structure of the table and the data.
DB2 for Linux, UNIX, and Windows has a utility called db2move, which generates the DDL to rebuild the database from scratch, and iterates through all the tables to dump their contents to flatfiles via the EXPORT command.