I'd like to save data from an SQL Server database table to a file, then load it into another database that has the same table already created in it
How can I do this?
I know there should be some simple way of doing it, but Stack Overflow search and Google aren't yielding good answers (or I'm not asking a good question).
You could use the Import/Export Wizard. Right click on your database in Management Studio and look for import data and export data under the tasks item.
Check bcp utility and BULK INSERT statement.
the bcp utility and BULK INSERT as mentioned by #Lukasz Lysik are good for moving lots of data. However, sp_generate_inserts by Narayana Vyas Kondreddi allows for many filtering options of the data, and good when there isn't that much data.
Related
I have a production database of 20 TB data. We migrated our database from Oracle to SQL Server. Our old application was based on a Cobol based platform. After migrating to SQL Server indexes are giving good results.
I am creating a schema with new set of indexes without any data. Now I want to migrate only the data.
Import/Export utility will take load log time and will fill up the log files also. Is there any other alternative of this ?
My advice would be:
Set the recovery model to simple. See here.
Remove the indexes.
Batch insert the rows or use select into (this minimizes logging).
Re-create the indexes.
I admit that I haven't had to do this sort of thing in a long time in SQL Server. There may be other methods that are faster -- such as backing up a table space/partition and restoring it in another location.
You may use bcp utility to import/export data. For full details see here https://learn.microsoft.com/en-us/sql/tools/bcp-utility?view=sql-server-2017
I want to export data from a query to excel file. I know that there is a lot of questions like these one here, but no one is acceptable in my situation.
For example, like this topic using OPENROWSET: T-SQL: Export to new Excel file
insert into OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=D:\testing.xls;',
'SELECT * FROM [SheetName$]') select * from SQLServerTable
It only execute successfully when I create testing.xls myself, and also define exactly number of columns that will be export from my query in this excel file. Otherwise, an error occur: Column name or number of supplied values does not match table definition.
I also try another solution here: http://weblogs.sqlteam.com/mladenp/archive/2006/07/25/10771.aspx using Exec master..xp_cmdshell
It is really work, but, I heard that xp_cmdshell is a big security threat for SQL Server. So, may be I shouldn't use it.
Is there any other way I can try?
If you're looking for a quick and dirty solution, you can use Management Studio itself.
Here are the steps:
write your query and run it
right click the Results pane and select "Save Results As..."
select your folder/filename and ensure CSV type is selected below
now open your CSV file using Excel; if appropriate save it in Excel as a native Excel format
That won't help you if you need programmatic solution, in which case you have to use Microsoft (Microsoft.Office.Interop.Excel) or 3rd party solutions; you can even build one using Office Open XML.
If you need TSQL solution, OPENROWSET which you are mentioning in the question should be fine.
You can use Excel-only solution and import data using Data pane in the Excel itself (Import from SQL server).
Another possibility taht doesn't have to be coupled to SQL Server or Excel itself is Powershell. However, I'm not into Powershell so if you prefer this method you will have to investigate a bit more.
Also, this is a bit outdated but interesting read.
Try using a newer version of Excel for this. From your query it looks like you’re using Excel 97 – there are probably a ton of bugs and incompatibilities that exist in this version.
If this doesn’t help other options are:
- Do this manually from SSSM like Ozren suggested
- Try creating SSIS package for this
Here are couple other threads to get you started
http://www.connectionstrings.com/excel/
How do you transfer or export SQL Server 2005 data to Excel
Export SQL query data to Excel
What is the best method to import data from an Excel worksheet? As of now I am use SSMS Express so I don't have access to SQL Import Wizard. I also don't have permissions to execute the BULK INSERT command.
My current workflow is as follow: Clean up the excel file, save as CSV, and import it into a SQLite database. Use an IDE like RazorSQL to generate SQL INSERT statements.
This worked nicely until I hit an Excel file about 75000 rows. SSMS just gives an error saying "query finished with errors" or something like that. No error message is shown. I tried adding GO at the end of each line but I got out of memory error.
What are my options?
To answer your question, the best method to import data from excel, in my past experience has been to read excel data into c#, do any clean up and formatting as necessary since excel likes to mess with the data, then use SqlBulkCopy (you only need select/insert permissions) to insert into SQL Server. See this SO answer if you need help reading excel from C#
Update: Given you're not a dev, try using the bcp utility (you should only need select/insert permission)you may need to save the excel file as CSV first, then import it directly into sql server, see this SO answer
You can use following:
bcp utility (between file system data dump and database),
OPENQUERY (can be used from SSMS, works between external datasource like Excel/csv and database),
BULK INSERT (can be used from SSMS, works between external file with user-defined structure and database),
SSIS (usually as dtsx package, has its own GUI, works with various souces and destinations)
Set of INSERT statements (all of them one after another, eventually sliced with GO or packed with UNION ALL)
Set of records serialized in XML variable (can be used from SSMS only; you have to serialize/deserialize it by your self using FOR XML and XML functions)
There are surely other possibilities, but these are maybe most used ones.
EDIT: It seems to me that you could try with GO after every 5-10K lines in your script.
If that doesn't work, XML serialization/deserialization could be the way to go.
Could you use a linked server to connect to the Excel Document? How to use Excel with SQL Server linked servers and distributed queries
A quick and dirty workaround: pull the rows in batches of 50k.
select * from employee limit 50000
select * from employee limit 50000, 100000
From
http://www.razorsql.com/articles/mysql_limit_query.html
Want to create a script to export Data and tables and views to a sql script.
I have SQL Server 2008 r2.
So far I've only been able to automatically generate an sqlscript for all tables and views. But the data wasn't included.
OR is there any easier way to export data, tables, views, from one SQL Server to my ISP's SQL server?
Regards
Tea
If for some reason a backup/restore won't work for you, SSMS' Generate Scripts tool includes an advanced scripting option to include data:
Here are some options to think over (prioritised in terms of what I would recommend):-
A simple backup and restore will be the easiest and quickest solution;
Using a data scripting tool (like Red-Gate's Data Compare) could solve your needs;
Use the database comparison as part of Visual Studio.
A SSIS package could be developed to pump data back and forth between the two instances; or
Write your own script using the SET IDENTITY INSERT ON / OFF command for the identity seeded tables
The easiest way to do this is to create a backup, copy the .bak file to the other server, and restore the backup there.
Like #jhewlett said that will be the best way to do it. to answer the question in the comment section. no it shouldn't be a problem. Just make sure that the SQL Server Versions are the same. Had a bit of an issue not to long ago where there were two pc's with different releases of the R2 installed and couldn't restore the backup. Other thing you can also do is to script the entire database with data, but this will not be recommended as it could take a long time to generate the script and for it to finish running on the other computer.
Or you can simply just stop the SQL server instance and copy the database away onto an external hard drive and re-attach it to the other server. just remember to start the instances after doing this step.
I use Navicat Premium for these kind of things in mysql. It generates sql from data, tables, views and anything else. It provides tools to copy or synchronize table from one database on different server or platforms as well. For example I use it so much to transfer my tables from MySQL to a SQLite database, So easy and fast. Otherwise I had to transfer it manually with so much trouble.
very good tool and required for any DB admin or programmer. It support MySQL, Oracle, MS SQL Server, PostgreSQL and SQLite.
To Generate a schema with data follow these steps.
Select database to generate a schema '>' right click '>' Tasks '>' Generate schema '>' click NEXT in popup window '>'
select DB object to generate schema and click NEXT '>'
Go to advance option and scroll down '>'
Find Type of data to script and select one option as you need. '>'
and then Next Next and finish it.
Enjoy it.
If you don't want to port all tables data (for example you need to port just some base data in special tables) scripting options is not useful for you. In this case you'll have two options. First is using some third parties tools such as Red-Gate and Second way is writing the script by yourself. I prefer Second option because except the expensive price of most of them i want to run just little script for little delete, update and inserting purpose. But the important problem is here: maybe the record count is too long to write scripts record by record. I Think the linked Server is good point to solve that. It's enough for describing Just Declare Linked Server as you see in Images and get new script in your source DB and write scripts with access to both source and destination DB. Attached image must be clear.
Create New Linked Server:
Write Destination SQL Server Address:
Fill Login Info:
Now you have Linked Server:
Write script and enjoy:
Hope this help.
I'm not a good SQL programmer, I've got only the basics, but I've heard of some BCP thing for fast data loading. I've searched the internet and it seems to be a command-line only utility, and not something you can use in code.
The thing is, I want to be able to make very fast inserts and updates in a SQL Server 2008 database. I would like to have a function in the database that would accept:
The name of the table I want to execute an insert/update operation against
The names of the columns I'll be feeding data to
The data in a CSV format or something that SQL can read stupid-fast
A flag indicating weather the function should perform an insert or update operation
This function would then read this CSV string and genarate the necessary code for inserting/updating the table.
I would then write code in C# to call that function passing it the table name, column names, a list of objects serialized as a CSV string and the insert/update flag.
As you can see, this is intended to be both fast and generic, suitable for any project dealing with large amounts of data, and thus a candidate to my company's framework.
Am I thinking right? Is this a good idea? Can I use that BCP thing, and is it suitable to every case?
As you can see, I need some directions on this... thanks in advance for any help!
In C#, look at SQLBulkCopy. It's what SSIS uses in the background.
For true bcp/BULK INSERT, you'd need bulkadmin rights which may not be allowed
Have you considered using SQL Server Integrated Services (SSIS). It's designed to do exactly what you describe. It is very fast. You can insert data on a transactional basis. And you can set it up to run on a schedule. And much more.