I been going through this tutorial
http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx
and them make a SP like this
CREATE PROCEDURE [dbo].[spTEST_InsertXMLTEST_TEST](#UpdatedProdData nText)
AS
DECLARE #hDoc int
exec sp_xml_preparedocument #hDoc OUTPUT,#UpdatedProdData
INSERT INTO TBL_TEST_TEST(NAME)
SELECT XMLProdTable.NAME
FROM OPENXML(#hDoc, 'ArrayOfTBL_TEST_TEST/TBL_TEST_TEST', 2)
WITH (
ID Int,
NAME varchar(100)
) XMLProdTable
EXEC sp_xml_removedocument #hDoc
Now my requirements require me to mass insert and mass update one after another. So first I am wondering can I merge those into one SP? I am not sure how it works with this OPENXML but I would think it would just be making sure that the XPath is right.
Next what happens while it would be running this combined SP and something goes wrong. Would it roll back all the records or just stop and the records that happened before this event that crashed it would be inserted?
A transaction is atomic. Either all inserted records are commited, either all are rolled back. A statement will always do the updates as part of a transaction. So this INSERT is either all going to commit, or is going to rollback and no row att all is going to be inserted.
In SQL 2005 you should avoid using NTEXT types and OPENXML. They inneficient and NTEXT is actually deprecated, and there are much better alternatives:
use XML datatype instead of NTEXT
use the XML data type methods instead of OPENXML:
.
create procedure usp_insertxml (#data xml)
as
begin
insert into table (id, name)
select x.value('ID', 'INT'),
x.value('NAME', 'varchar(100)')
from #data.nodes('ArrayOfTBL_TEST_TEST/TBL_TEST_TEST') t(x);
end
Related
I'm new to SQL Server and stored procedures and could do with a couple of pointers regarding transaction handling on a bug I've inherited.
I have two stored procedures, one inserts a record passed into it, then it calls another one where the first thing it does is read what was inserted.
But sometimes it completes successfully without processing the data. My suspicion is that the selects are happening before the insert has 'hit' the table and retrieve no records, and the stored procedure doesn't handle that.
I don't have time to re-engineer just yet, but the transaction handling looks suspect. Below is a rough outline of what the stored procedures do.
procedure sp1
(#id, #pbody)
as
begin
begin try
set nocount on;
begin
insert into tbl1 (id, tbody)
values (#id, #pbody)
exec sp2 #id
end
end try
begin catch
execute sperror
end catch
end
go
procedure sp2 (#id)
as
begin
begin try
set nocount on;
declare #vbody varchar(max)
select #vbody = tbody -- I don't believe this step always retrieves the row inserted by sp1
from tbl1 with (nolock)
where id = #id
create table #tmp1 (id, msg)
insert into #tmp1
select id, msg
from openjson........
while exists(select top 1 * from #tmp1) -- this looks similar to above, not sure the insert has finished before the read
begin
** do some stuff **
end
end try
begin catch
execute sperror
end catch
end
go
sp2 is using the WITH (NOLOCK) query hint, which can have unintended side-effects. Missing rows is just one of them.
Using NOLOCK? Here's How You'll Get the Wrong Query Results. - Brent Ozar Unlimited®
I'd strongly recommend removing that hint unless you really understand what it does and have a very good reason for using it.
I have a stored procedure that suddenly started returning NULL on only one of two supposedly equivalent SQL Server 2014 machines (primary and failover).
The specific operation that is suddenly misbehaving is:
declare #skus TABLE (intSkuId int, attributes xml);
insert into #skus
VALUES
(11443, '<attributes><style>basic_mug</style><color>white</color><size>15oz</size></attributes>'),
(11444, '<attributes><style>basic_mug</style><color>black</color><size>15oz</size></attributes>');
select
s.intSkuId, s.xmlAttributes,
att.query('.') as element
from
tb_Skus s
outer apply
xmlAttributes.nodes(N'//attributes/*') as atts(att)
where
s.intSkuId = 11443;
Note that this would normally be run against a segment of the tb_Skus table, not a single value, but I'm simplifying for debugging purposes.
This returns the following result:
intSkuId: 11443
xmlAttributes: <attributes><style>basic_mug</style><color>white</color><size>15oz</size></attributes>
element: NULL
Note also that the following script works as intended:
DECLARE #x xml;
SELECT #x = xmlAttributes
FROM tb_Skus s
WHERE s.intSkuId = 11443
SELECT #x AS xmlAttributes, att.query('.') AS element
FROM #x.nodes(N'//attributes/*') AS atts(att)
yielding the same value for xmlAttribute, but I get the expected values for element:
<style>basic_mug</style>
<color>white</color>
<size>15oz</size>
So, is there something wrong with the first query? And if not, are there session or database settings that could alter the behavior of this function?
There is no need to use .nodes() method in this case. It is needed just when there is a need to convert/shred XML data type into a rectangular/relational data.
Please try the following.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (intSkuId INT, xmlAttributes XML);
INSERT INTO #tbl (intSkuId, xmlAttributes) VALUES
(11443, N'<attributes>
<style>basic_mug</style>
<color>white</color>
<size>15oz</size>
</attributes>');
-- DDL and sample data population, end
SELECT *
, xmlAttributes.query('/attributes/*') AS [element]
FROM #tbl
where intSkuId=11443;
Assume that I have a table on my local which is Local_Table and I have another server and another db and table, which is Remote_Table (table structures are the same).
Local_Table has data, Remote_Table doesn't. I want to transfer data from Local_Table to Remote_Table with this query:
Insert into RemoteServer.RemoteDb..Remote_Table
select * from Local_Table (nolock)
But the performance is quite slow.
However, when I use SQL Server import-export wizard, transfer is really fast.
What am I doing wrong? Why is it fast with Import-Export wizard and slow with insert-select statement? Any ideas?
The fastest way is to pull the data rather than push it. When the tables are pushed, every row requires a connection, an insert, and a disconnect.
If you can't pull the data, because you have a one way trust relationship between the servers, the work around is to construct the entire table as a giant T-SQL statement and run it all at once.
DECLARE #xml XML
SET #xml = (
SELECT 'insert Remote_Table values (' + '''' + isnull(first_col, 'NULL') + ''',' +
-- repeat for each col
'''' + isnull(last_col, 'NULL') + '''' + ');'
FROM Local_Table
FOR XML path('')
) --This concatenates all the rows into a single xml object, the empty path keeps it from having <colname> </colname> wrapped arround each value
DECLARE #sql AS VARCHAR(max)
SET #sql = 'set nocount on;' + cast(#xml AS VARCHAR(max)) + 'set nocount off;' --Converts XML back to a long string
EXEC ('use RemoteDb;' + #sql) AT RemoteServer
It seems like it's much faster to pull data from a linked server than to push data to a linked server: Which one is more efficient: select from linked server or insert into linked server?
Update: My own, recent experience confirms this. Pull if possible -- it will be much, much faster.
Try this on the other server:
INSERT INTO Local_Table
SELECT * FROM RemoteServer.RemoteDb.Remote_Table
The Import/Export wizard will be essentially doing this as a bulk insert, where as your code is not.
Assuming that you have a Clustered Index on the remote table, make sure that you have the same Clustered index on the local table, set Trace flag 610 globally on your remote server and make sure remote is in Simple or bulk logged recovery mode.
If you're remote table is a Heap (which will speed things up anyway), make sure your remote database is in simple or bulk logged mode change your code to read as follows:
INSERT INTO RemoteServer.RemoteDb..Remote_Table WITH(TABLOCK)
SELECT * FROM Local_Table WITH (nolock)
The reason why it's so slow to insert into the remote table from the local table is because it inserts a row, checks that it inserted, and then inserts the next row, checks that it inserted, etc.
Don't know if you figured this out or not, but here's how I solved this problem using linked servers.
First, I have a LocalDB.dbo.Table with several columns:
IDColumn (int, PK, Auto Increment)
TextColumn (varchar(30))
IntColumn (int)
And I have a RemoteDB.dbo.Table that is almost the same:
IDColumn (int)
TextColumn (varchar(30))
IntColumn (int)
The main difference is that remote IDColumn isn't set up as as an ID column, so that I can do inserts into it.
Then I set up a trigger on remote table that happens on Delete
Create Trigger Table_Del
On Table
After Delete
AS
Begin
Set NOCOUNT ON;
Insert Into Table (IDColumn, TextColumn, IntColumn)
Select IDColumn, TextColumn, IntColumn from MainServer.LocalDB.dbo.table L
Where not exists (Select * from Table R WHere L.IDColumn = R.IDColumn)
END
Then when I want to do an insert, I do it like this from the local server:
Insert Into LocalDB.dbo.Table (TextColumn, IntColumn) Values ('textvalue', 123);
Delete From RemoteServer.RemoteDB.dbo.Table Where IDColumn = 0;
--And if I want to clean the table out and make sure it has all the most up to date data:
Delete From RemoteServer.RemoteDB.dbo.Table
By triggering the remote server to pull the data from the local server and then do the insert, I was able to turn a job that took 30 minutes to insert 1258 lines into a job that took 8 seconds to do the same insert.
This does require a linked server connection on both sides, but after that's set up it works pretty good.
Update:
So in the last few years I've made some changes, and have moved away from the delete trigger as a way to sync the remote table.
Instead I have a stored procedure on the remote server that has all the steps to pull the data from the local server:
CREATE PROCEDURE [dbo].[UpdateTable]
-- Add the parameters for the stored procedure here
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
--Fill Temp table
Insert Into WebFileNamesTemp Select * From MAINSERVER.LocalDB.dbo.WebFileNames
--Fill normal table from temp table
Delete From WebFileNames
Insert Into WebFileNames Select * From WebFileNamesTemp
--empty temp table
Delete From WebFileNamesTemp
END
And on the local server I have a scheduled job that does some processing on the local tables, and then triggers the update through the stored procedure:
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc', #optvalue='true'
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc out', #optvalue='true'
EXEC REMOTESERVER.RemoteDB.dbo.UpdateTable
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc', #optvalue='false'
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc out', #optvalue='false'
If you must push data from the source to the target (e.g., for firewall or other permissions reasons), you can do the following:
In the source database, convert the recordset to a single XML string (i.e., multiple rows and columns combined into a single XML string).
Then push that XML over as a single row (as a varchar(max), since XML isn't allowed over linked databases in SQL Server).
DECLARE #xml XML
SET #xml = (select * from SourceTable FOR XML path('row'))
Insert into TempTargetTable values (cast(#xml AS VARCHAR(max)))
In the target database, cast the varchar(max) as XML and then use XML parsing to turn that single row and column back into a normal recordset.
DECLARE #X XML = (select '<toplevel>' + ImportString + '</toplevel>' from TempTargetTable)
DECLARE #iX INT
EXEC sp_xml_preparedocument #ix output, #x
insert into TargetTable
SELECT [col1],
[col2]
FROM OPENXML(#iX, '//row', 2)
WITH ([col1] [int],
[col2] [varchar](128)
)
EXEC sp_xml_removedocument #iX
I've found a workaround. Since I'm not a big fun of GUI tools like SSIS, I've reused a bcp script to load table into csv and vice versa. Yeah, it's an odd case to have the bulk operation support for files, but tables. Feel free to edit the following script to fit your needs:
exec xp_cmdshell 'bcp "select * from YourLocalTable" queryout C:\CSVFolder\Load.csv -w -T -S .'
exec xp_cmdshell 'bcp YourAzureDBName.dbo.YourAzureTable in C:\CSVFolder\Load.csv -S yourdb.database.windows.net -U youruser#yourdb.database.windows.net -P yourpass -q -w'
Pros:
No need to define table structures every time.
I've tested and it worked way faster than inserting directly through
the LinkedServer.
It's easier to manage than XML (which is limited to
varchar(max) length anyway).
No need of an extra layout of abstraction (tools like SSIS).
Cons:
Using the external tool bcp through the xp_cmdshell interface.
Table properties will be lost after ex/im-poring csv (i.e. datatype, nulls,length, separator within value, etc).
In SQL Server, there's no way to create a temp table on the fly from the results of a stored procedure, ala:
CREATE TABLE #temptable AS
EXEC spMyStoredProc
or
EXEC spMyStoredProc INTO #temptable
or something like that. Instead, you have to know the SP layout beforehand, and have to do something like this:
CREATE TABLE #temptable (col1 INT, col2 VARCHAR(255))
INSERT INTO #temptable
EXEC spMyStoredProc
Is there a functional reason why this is the case? Maybe a limitation of SQL Server? Or is it just something that hasn't been added to the SQL spec yet, and I can hold out hope that one day they'll support it?
A stored procedure can return many result sets, or none, and it can vary entirely depending upon the execution of the stored procedure.
When it is compiled it's meta-information does not describe it as having any specific expectable result set output.
I expect given those constraints, they elected not to implement this because of the lack of strong typing of what a stored procedure may return.
Not from a sproc, but you can use tabled values functions to do something similar.
Select * From fnMyFunction
You'd be able to insert into a # table if you desired.
Try this out
DECLARE #temptable TABLE (ID INT, NAME VARCHAR(255))
declare #query varchar(max)
set #query='Select whatever from whereever'
INSERT INTO #temptable
EXEC (#Query)
select *from #temptable
If I needed such functionality I would use an inline UDF, like this:
CREATE PROCEDURE MySample
AS
SELECT a,b,c FROM dbo.MyInlineUDF(1,2,3)
GO
SELECT * INTO #t
FROM dbo.MyInlineUDF(1,2,3) WHERE 1=0
INSERT INTO #t EXEC MySample
I'm executing stored procedures using SET FMTONLY ON, in order to emulate what our code generator does. However, it seems that the results are cached when executed like this, as I'm still getting a Conversion failed error from a proc that I have just dropped! This happens even when I execute the proc without SET FMTONLY ON.
Can anyone please tell me what's going on here?
Some statements will still be executed, even with SET FMTONLY ON. You "Conversion failed" error could be from something as simple as a set variable statement in the stored proc. For example, this returns the metadata for the first query, but throws an exception when it runs the last statement:
SET FMTONLY on
select 1 as a
declare #a int
set #a = 'a'
As for running a dropped procedure, that's a new one to me. SQL Server uses the system tables to determine the object to execute, so it doesn't matter if the execution plan is cached for that object. If you drop it, it is deleted from the system tables, and should never be executable. Could you please query sysobjects (or sys.objects) just before you execute the procedure? I expect you'll find that you haven't dropped it.
This sounds like a client-side error. Do you get the same message when running through SQL Management Studio?
Have you confirmed that there isn't another procedure with the same name that's owned by a different schema/user?
DDL statements are parsed, but ignored when run if SET FMTONLY ON has been executed on the connection. So if you drop a proc, table, etc when FMTONLY is ON, the statement is parsed, but the action is not executed.
Try this to verify
SET FMTONLY OFF
--Create table to test on
CREATE TABLE TestTable (Column1 INT, Column2 INT)
--insert 1 record
INSERT INTO TestTable (Column1, Column2)
VALUES (1,2)
--validate the record was inserted
SELECT * FROM TestTable
--now set format only to ON
SET FMTONLY ON
--columns are returned, but no data
SELECT * FROM TestTable
--perform DDL statement with FMTONLY ON
DROP TABLE TestTable
--Turn FMTONLY OFF again
SET FMTONLY OFF
--The table was dropped above, so this should not work
SELECT * FROM TestTable
DROP TABLE TestTable
SELECT * FROM TestTable