This question already has answers here:
Bulk Insert of Generic List C# into SQL Server
(4 answers)
Closed 9 years ago.
Is there a fast, efficient way in VB.NET/ADO.NET to insert large amount of data from Geneneric.List(Of Integer) into SQL Server table besides looping thru the list and issuing individual INSERT commands? I am limited to .NET 3.5 and SQL Server 2005.
Thanks!
Ship XML with all the changes to a stored procedure.
Here is an old example here:
http://granadacoder.wordpress.com/2009/01/27/bulk-insert-example-using-an-idatareader-to-strong-dataset-to-sql-server-xml/
Here is a smaller example, but shows the basics.
http://www.mindfiresolutions.com/Sending-Multiple-Records-As-XML-To-SQL-Server-Stored-Procedure-1861.php
Send xml to stored procedure. Shred the xml to a #variable or #temp table. Do your UPDATES / INSERTS (or MERGE/UPSERT) using the #variable or #temp table.
http://weblogs.asp.net/dwahlin/archive/2009/09/30/passing-multiple-records-to-a-stored-procedure-in-sql-server.aspx
Another example.
What I like to do is create a strong dataset. Put your data into the strong dataset. Then send the ds.GetXml() to the stored procedure.
That way, you get strong typing (using the strong dataset), and you don't have to write your own xml-maker, you piggy back off of .GetXml(). Hint: After creating the strong dataset, remove the namespace (tempuri or something like that)
Related
This question already has answers here:
Search for a string in all tables, rows and columns of a DB
(15 answers)
Closed 9 years ago.
I've been scouring online for an example of how to do this but haven't found anything at all. All the queries I've found assume you know what table you want to search.
I'm looking for a SQL query to simply search the ENTIRE database for a specific word.
There has to be such a thing right?
This is for MS SQL 2005/2008
Thanks
What do you mean by "entire database"? You need to find your values in tables only, or in object definitions, too?
I assume the former. In this case, you don't have to really know the structure of your DB. Try those views below. With them, you can construct your select queries on all the tables / colums. Just filter out non-*char columns, views and system tables, and you're ready to go - you can "automatically" generate multiple select statements.
select top 100 * from information_schema.tables
select top 100 * from information_schema.columns
The other option, is to use some addon to SSMS, like this one:
http://www.ssmstoolspack.com/
It has an option to search entire database.
But be advised, that both solutions will have a great impact on performance of your server.
I'm trying insert a large xml document (about 10MB) into a Sql Server 2008 table, the XML document is built at run-time.
Is there a better way to perform this insert. I'm using a simple insert command with one parameter of type string, but it doesn't work. In the table, the field is showing a NULL value.
Is there a way to do this with a single insert?
Try to use BULK INSERT expression to increase performance.
http://msdn.microsoft.com/ru-ru/library/ms188365.aspx
I have a table with a VARBINARY(MAX) field (SQL Server 2008 with FILESTREAM)
My requirement is that when I go to deploy to production, I can only supply my IT team with a group of SQL scripts to be executed in a certain order. A new table I am making in production has this VARBINARY(MAX) field. Usually with new tables, I will script out the CREATE TABLE script. And, if I have data I need to go with it, I will then script out the INSERT scripts. Not too complicated.
But with VARBINARY(MAX), the Stored Procedure I was using to generate the INSERT statements fails on that table. I tried selecting that field, printing it, copying it, converting to hex, etc. The main issue I have with that is that it doesn't select all the data in the field. I do a check DATALENGTH([FileColumn]) and if the source row contains 1,004,382 bytes, the max I can get the copied or selected data when inserting again is 8000. So basically it is truncated (i.e. invalid) data.....
How can I do this better? I tried Googling this like crazy but I must be missing something. Remember, I can't access the filesystem. This has to be all scripted.
If this is a one time (or seldom) thing to do, you can try scripting the data out from the SSMS Wizard as described here:
http://sqlblog.com/blogs/eric_johnson/archive/2010/03/08/script-data-in-sql-server-2008.aspx
Or, if you need to do this frequently and want to automate it, you can try the SQL# SQLCLR library (which I wrote and while most of it is free, the function you need here is not). The function to do this is DB_DumpData and it also generates INSERT statements.
But again, if this is a one time or infrequent task, then try the data export wizard that is built into Management Studio. That should allow you to then create the SQL script that you can run in Production. I just tested this on a table with a VARBINARY(MAX) field containing 3,365,964 bytes of data and the Generate Scripts wizard generated an INSERT statement with the entire hex string of 6.73 million characters for that one value.
UPDATE:
Another quick and easy way to do this in a manner that would allow you to copy / paste the entire INSERT statement into a SQL script and not have to bother with BCP or SSMS Export Wizard is to just convert the value to XML. First you would CONVERT the VARBINARY to VARCHAR(MAX) using the optional style of "1" which gives you a hex string starting with "0x". Once you have the hex string of the binary data you can concatenate that into an INSERT statement and that entire thing, when converted to XML, can contain the entire VARBINARY field. See the following example:
DECLARE #Binary VARBINARY(MAX) = CONVERT(VARBINARY(MAX),
REPLICATE(
CONVERT(NVARCHAR(MAX), 'test string'),
100000)
)
SELECT 'INSERT INTO dbo.TableName (ColumnName) VALUES ('+
CONVERT(VARCHAR(MAX), #Binary, 1) + ')' AS [Insert]
FOR XML RAW;
Don't script from SSMS
bcp the data out/in, or use something like SSMS tools to generate INSERT statements
It more than a bit messed up, but in the past and on the web I've seen this done using a base64-encoded string. You use an xml value to wrap the string and from there you can convert it to a varbinary. Here's an example:
http://blogs.msdn.com/b/sqltips/archive/2008/06/30/converting-from-base64-to-varbinary-and-vice-versa.aspx
I can't speak personally to how effective or performant this is, though, especially for large values. Because it is at best an ugly hack, I'd tuck it away inside a UDF somewhere, so that if a better method is found you can update it easily.
I have never tried anything like this before, but from the documentation for SQL Server 2008 R2, it sounds like using SUBSTRING will work to get the entire varbinary value, although you may have to work with it in chunks, using UPDATEs with the .WRITE clause to append the data.
Updating Large Value Data Types
Use the .WRITE (expression, #Offset, #Length) clause to perform a partial or full update of varchar(max), nvarchar(max), and varbinary(max) data types. For example, a partial update of a varchar(max) column might delete or modify only the first 200 characters of the column, whereas a full update would delete or modify all the data in the column.
For best performance, we recommend that data be inserted or updated in chunk sizes that are multiples of 8040 bytes.
Hope this helps.
I have 20 record group that i need to batch insert them all in one connection so there is two solution (XML or stored procedure). this operation frequently executed so i need fast performance and least overhead
1) I think XML is performs slower but we can freely specify how many record we need to insert as a batch by producing the appropriate XML, I don't know the values of each field in a record, there maybe characters that malformed our XML like using " or filed tags in values so how should i prevent this behavior ?
2) using stored procedure is faster but i need to define all input parameters which is boring task and if i need to increase or decrease the number of records inserted in a batch then i need to change the SP
so which solution is better in my environment with respect to my constrains
XML is likely the better choice, however there are other options
If you're using SQL Server 2008 you can use Table Valued parameters instead.
Starting with .NET 2.0 you had the option to use the SQLBulkCopy
If you're using oracle you can pass a user defined type but I'm not sure what versions of ODP and Oracle that works with.
Note these are all .NET samples. I don't know that this will work for you. It would probably help if you include the database and version and client technology that you're using.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
T-SQL stored procedure that accepts multiple Id values
Does T-SQL accomodate for array values as parameters for stored procedures? If so how can this be achieved.
T-SQL dialect in SQL Server 2005 does not support neither arrays, nor anything similar, so they have to be emulated. SQL Server 2008, however, supports table-valued parameters, which can be used as arrays.
What I prefer to use instead is a comma (or other special character) separated list which I will split/explode first thing in my sproc. This will then give me a table of values to work with and that I can then join on or perform other actions on later on in my stored procedures.
You can also look into passing in table parameters, but I kind of like my way more just as a personal preference.
It was often achieved passing in a CSV, which is obviously limited. With SQL 2005 an Xml parameter may be much better suited, with a serializer suited to your needs perhaps.
There may be more and I'll come back if I think of any.
There are no arrays in sqlserver. However, What are you trying to do? What would the values as parameters be used for? You can send sql an "array" but the sp would have to be dynamic.