I have a linked server that I have to fetch data from. I'm joining on a table that I expect very few rows from. The query is below, and seems to be returning all of the rows to the original server to do the sort there.
I'm looking for a way to tell the query to filter on the target machine, with a query hint or something else.
Query
INSERT INTO #DealerHierarchy(DealerId, Level)
SELECT cd.ParentId, cd.Level
FROM [dbo].[AssignedDealer] ad
JOIN [nlsdb].[nls].[dbo].[vw_parentDealers] cd ON cd.RootId = ad.DealerId
WHERE ad.UserId = #userId
AND ad.IsActive = 1
AND (#DealerId IS NULL OR ad.DealerId = #DealerId)
When I add the following line, it seems to change and only send back the needed rows
and cd.RootId = 72311
I have tried moving out the local query into a separate temp table, and then select from the view WHERE DealerId IN (select from temp table) but it still runs slowly. Adding the REMOTE hint in the JOIN also does nothing.
Query plan:
https://www.brentozar.com/pastetheplan/?id=r1iazaaFZ
Slow code executed on linked server
declare #p1 int
set #p1=7
exec sp_prepexec #p1 output,N'#P1 numeric(10)',N'SELECT "Tbl1007"."ParentId" "Col1010","Tbl1007"."Level" "Col1011" FROM "nls"."dbo"."vw_parentDealers" "Tbl1007" WHERE #P1="Tbl1007"."RootId"',72311
select #p1
Fast code executed on linked server
declare #p1 int
set #p1=10
exec sp_prepexec #p1 output,NULL,N'SELECT "Tbl1007"."ParentId" "Col1010","Tbl1007"."Level" "Col1011" FROM "nls"."dbo"."vw_parentDealers" "Tbl1007" WHERE "Tbl1007"."RootId"=(72311.)'
select #p1
You can force a specific query to be run on the remote database by using OPENQUERY. OPENQUERY doesn't accept a parameter, so you can make it dynamic by further wrapping it in EXEC.
Example
DECLARE #SearchString NVARCHAR = ...
DECLARE #OpenQueryString NVARCHAR = 'SELECT * FROM OPENQUERY(remotedb, ''' + #SearchString + ''')'
EXEC (#OpenQueryString)
Related
On SQL Server, I am testing to create a Plan guide to force a query from Java to use a specific optimal execution-plan always. Some how the SQL query not using this Plan Guide
I followed, https://technet.microsoft.com/en-us/library/ms190772%28v=sql.105%29.aspx?f=255&MSPPError=-2147217396
SQL comes from Java,
declare #p1 int
set #p1=1986
exec sp_prepexec #p1 output,N'#P0 nvarchar(4000)',N'SELECT Top 1 Amount,col2 from dbo.table1
where ExId=#P0
order by id
',N'0a8e8e31-2a05-0000-8ece-0003fd69e692'
select #p1
Plan Guide I created,
DECLARE #xml_showplan nvarchar(max);
SET #xml_showplan = (select cast(query_plan as nvarchar(max)) from sys.dm_exec_cached_plans cp
cross apply sys.dm_exec_sql_text (cp.plan_handle) st
cross apply sys.dm_exec_query_plan (cp.plan_handle) qp
where st.text like '%Top 1 Amount%'
and objtype='Prepared')
--select (#xml_showplan)
EXEC sp_create_plan_guide
#name = N'ForceuserLoanAmountRequests',
#stmt = N'SELECT Top 1 Amount,col2 from dbo.table1 where ExId=#P0 order by id',
#type = N'SQL',
#module_or_batch = NULL,
#params = N'#P0 nvarchar(4000)',
#hints = #xml_showplan;
Appreciate if you can help me to get thru the plan guide used by SQL query from java.
Created a Plan Guide with the SQL collected from SQL Profiler,
EXEC sp_create_plan_guide
#name = N'ForceuserLoanAmountRequests',
#stmt = N'SELECT Top 1 Amount,col2 from table1
where ExId=#P0
order by ID
',
#type = N'SQL',
#module_or_batch = NULL,
#params = N'#P0 nvarchar(4000)',
#hints = #xml_showplan;
GO
Please read the article sp_create_plan_guide (Transact-SQL) pay attention to Remarks:
Plan Guide Matching Requirements
For plan guides that specify #type = 'SQL' or #type = 'TEMPLATE' to
successfully match a query, the values for batch_text and
#parameter_name data_type [,...n ] must be provided in exactly the
same format as their counterparts submitted by the application. This
means you must provide the batch text exactly as the SQL Server
compiler receives it. To capture the actual batch and parameter text,
you can use SQL Server Profiler. For more information, see Using SQL
Server Profiler to Create and Test Plan Guides.
This means that the statement should exactly match to what is written in your plan guide, but in your code plan guide is created for a statement that differs from those sending by java: it has only one row while java code has 4 rows with CR and additional spaces.
If you evaluate any hash from java statement and your statement provided in sp_create_plan_guide you'll see they are different.
Another thing I suspect that may be wrong is how you get a "good plan". Since you just assign #xml_showplan the result of a query that can produce many rows, I'm not sure you get the plan you want. How do you ensure that you capture the "good£ plan and not the same plan that java code produced?
Observe the following simple SQL code:
CREATE TABLE #tmp (...) -- Here comes the schema
INSERT INTO #tmp
EXEC(#Sql) -- The #Sql is a dynamic query generating result with a known schema
All is good, because we know the schema of the result produced by #Sql.
But what if the schema is unknown? In this case I use Powershell to generate a Sql query like that:
SET #Sql = '
SELECT *
INTO ##MySpecialAndUniquelyNamedGlobalTempTable
FROM ($Query) x
'
EXEC(#Sql)
(I omit some details, but the "spirit" of the code is preserved)
And it works fine, except that there is a severe limitation to what $Query can be - it must be a single SELECT statement.
This is not very good for me, I would like to be able to run any Sql script like that. The problem, is that no longer can I concatenate it to FROM (, it must be executed by EXEC or sp_executesql. But then I have no idea how to collect the results into a table, because I have no idea of the schema of that table.
Is it possible in Sql Server 2012?
Motivation: We have many QA databases across different Sql servers and more often than not I find myself running queries on all of them in order to locate the database most likely to yield best results for my tests. Alas, I am only able to run single SELECT statements, which is inconvenient.
We use SP and OPENROWSET for this purpose.
At first create SP based on a query you need, than use OPENROWSET to get data into temp table:
USE Test
DECLARE #sql nvarchar(max),
#query nvarchar(max)
SET #sql = N'Some query'
IF OBJECT_ID(N'SomeSPname') IS NOT NULL DROP PROCEDURE SomeSPname
SET #query =N'
CREATE PROCEDURE SomeSPname
AS
BEGIN
'+#sql+'
END'
EXEC sp_executesql #query
USE tempdb
IF OBJECT_ID(N'#temp') IS NOT NULL DROP TABLE #temp
SELECT *
INTO #temp
FROM OPENROWSET(
'SQLNCLI',
'Server=SERVER\INSTANCE;Database=Test;Trusted_Connection=yes;',
'EXEC dbo.SomeSPname')
SELECT *
FROM #temp
I have two servers: SQLSERVER01 and SQLSERVER02 and I am trying to updated data on SERVER01 from SERVER02 (SERVER01 is a linked server).
My update query is currently dynamic which looks something like this
DECLARE #SQL NVARCHAR(MAX)
DECLARE #ID INT
SET #ID = 1
Set #SQL = 'Update SERVER01.MyDatbase.dbo.MyTable
set ModifiedDate = GetDate(), SomeOtherValue = ''xyz''
Where Id = ' Convert(varchar(10), #ID)
If I now call
EXEC(#SQL)
it will sometimes work, but other times it will just hang there for ages and when I run sp_active I see PREEMPTIVE_OLEDBOPS.
So, I then tried using open query as follows
Select *
From OpenQuery(SERVER01,
'Update SERVER01.MyDatbase.dbo.MyTable
set ModifiedDate = GetDate(), SomeOtherValue = ''xyz''
Where Id = 1')
but I get this error:
The OLE DB provider "SQLNCLI11" for linked server "SERVER01" indicates that either the object has no columns or the current user does not have permissions on that object.
When I run the following select query I successfully return the row I'm trying to update:
Select *
From OpenQuery(SERVER01,
'Select *
From SERVER01.MyDatbase.dbo.MyTable
Where Id = 1')
I know that the user that is running the query has permissions, but I really don't know where to go from here. I read somewhere this error has something to do with the update query not returning a row. Is there any truth to this?
How can I resolve this?
Thanks
Okay, I found the answer. This works successfully:
execute (#SQL) at SERVER01
Due to the constraints within the workplace I have to use a local stored procedure to call another remote stored proc on a linked sql server, however the problem lies in passing a necessary parameter to the remote stored proc.
This is the query I constructed:
select *
from OPENQUERY([REMOTE_SRVR],'exec db.dbo.dwStoredProc_sp ''#id''')
In order to pass #id to the remote stored proc I understand I could concatenate the above as a string and then use exec
Something along the lines of:
set #query = 'select * from OPENQUERY([REMOTE_SRVR], ''EXEC db.dbo.dwStoredProc_sp '' #id '''''
exec(#query)
I cannot get the local stored proc to successfully call the other. The single quote mess doesn't help!
I get the error: Could not find stored procedure 's'
To help with the quote mess I like to do this in steps. It is more code but easier to understand. I am not sure from your example if #id is an integer. In that case you can lose the double quotes around __ID__.
set #query = 'EXEC db.dbo.dwStoredProc_sp ''__ID__'''
set #query = REPLACE(#query,'__ID__',#id)
set #query = REPLACE(#query,'''','''''')
set #query = REPLACE('SELECT * FROM OPENQUERY([REMOTE_SRVR], ''__REMOTEQUERY__'')','__REMOTEQUERY__',#query)
You could avoid dynamic queries by simply by using EXEC (..., ParamValue) AT LinkedServer (see product's documentation, example [L. Using a parameter with EXECUTE and AT linked_server_name]):
1) On target server:
CREATE PROCEDURE dbo.Proc1( #id NVARCHAR(50) )
AS
SELECT #id AS [id];
GO
2) On the source server you create the linked server and then you can call the stored procedure using EXEC ... AT ... syntax:
DECLARE #p1 NVARCHAR(50);
SET #p1 = N'DROP TABLE dbo.CocoJambo'
EXECUTE (N'dbo.Proc1 ? ' , #p1 ) AT LOCALINKEDSEREV
Output:
id
------------------------
DROP TABLE dbo.CocoJambo
Assume that I have a table on my local which is Local_Table and I have another server and another db and table, which is Remote_Table (table structures are the same).
Local_Table has data, Remote_Table doesn't. I want to transfer data from Local_Table to Remote_Table with this query:
Insert into RemoteServer.RemoteDb..Remote_Table
select * from Local_Table (nolock)
But the performance is quite slow.
However, when I use SQL Server import-export wizard, transfer is really fast.
What am I doing wrong? Why is it fast with Import-Export wizard and slow with insert-select statement? Any ideas?
The fastest way is to pull the data rather than push it. When the tables are pushed, every row requires a connection, an insert, and a disconnect.
If you can't pull the data, because you have a one way trust relationship between the servers, the work around is to construct the entire table as a giant T-SQL statement and run it all at once.
DECLARE #xml XML
SET #xml = (
SELECT 'insert Remote_Table values (' + '''' + isnull(first_col, 'NULL') + ''',' +
-- repeat for each col
'''' + isnull(last_col, 'NULL') + '''' + ');'
FROM Local_Table
FOR XML path('')
) --This concatenates all the rows into a single xml object, the empty path keeps it from having <colname> </colname> wrapped arround each value
DECLARE #sql AS VARCHAR(max)
SET #sql = 'set nocount on;' + cast(#xml AS VARCHAR(max)) + 'set nocount off;' --Converts XML back to a long string
EXEC ('use RemoteDb;' + #sql) AT RemoteServer
It seems like it's much faster to pull data from a linked server than to push data to a linked server: Which one is more efficient: select from linked server or insert into linked server?
Update: My own, recent experience confirms this. Pull if possible -- it will be much, much faster.
Try this on the other server:
INSERT INTO Local_Table
SELECT * FROM RemoteServer.RemoteDb.Remote_Table
The Import/Export wizard will be essentially doing this as a bulk insert, where as your code is not.
Assuming that you have a Clustered Index on the remote table, make sure that you have the same Clustered index on the local table, set Trace flag 610 globally on your remote server and make sure remote is in Simple or bulk logged recovery mode.
If you're remote table is a Heap (which will speed things up anyway), make sure your remote database is in simple or bulk logged mode change your code to read as follows:
INSERT INTO RemoteServer.RemoteDb..Remote_Table WITH(TABLOCK)
SELECT * FROM Local_Table WITH (nolock)
The reason why it's so slow to insert into the remote table from the local table is because it inserts a row, checks that it inserted, and then inserts the next row, checks that it inserted, etc.
Don't know if you figured this out or not, but here's how I solved this problem using linked servers.
First, I have a LocalDB.dbo.Table with several columns:
IDColumn (int, PK, Auto Increment)
TextColumn (varchar(30))
IntColumn (int)
And I have a RemoteDB.dbo.Table that is almost the same:
IDColumn (int)
TextColumn (varchar(30))
IntColumn (int)
The main difference is that remote IDColumn isn't set up as as an ID column, so that I can do inserts into it.
Then I set up a trigger on remote table that happens on Delete
Create Trigger Table_Del
On Table
After Delete
AS
Begin
Set NOCOUNT ON;
Insert Into Table (IDColumn, TextColumn, IntColumn)
Select IDColumn, TextColumn, IntColumn from MainServer.LocalDB.dbo.table L
Where not exists (Select * from Table R WHere L.IDColumn = R.IDColumn)
END
Then when I want to do an insert, I do it like this from the local server:
Insert Into LocalDB.dbo.Table (TextColumn, IntColumn) Values ('textvalue', 123);
Delete From RemoteServer.RemoteDB.dbo.Table Where IDColumn = 0;
--And if I want to clean the table out and make sure it has all the most up to date data:
Delete From RemoteServer.RemoteDB.dbo.Table
By triggering the remote server to pull the data from the local server and then do the insert, I was able to turn a job that took 30 minutes to insert 1258 lines into a job that took 8 seconds to do the same insert.
This does require a linked server connection on both sides, but after that's set up it works pretty good.
Update:
So in the last few years I've made some changes, and have moved away from the delete trigger as a way to sync the remote table.
Instead I have a stored procedure on the remote server that has all the steps to pull the data from the local server:
CREATE PROCEDURE [dbo].[UpdateTable]
-- Add the parameters for the stored procedure here
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
--Fill Temp table
Insert Into WebFileNamesTemp Select * From MAINSERVER.LocalDB.dbo.WebFileNames
--Fill normal table from temp table
Delete From WebFileNames
Insert Into WebFileNames Select * From WebFileNamesTemp
--empty temp table
Delete From WebFileNamesTemp
END
And on the local server I have a scheduled job that does some processing on the local tables, and then triggers the update through the stored procedure:
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc', #optvalue='true'
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc out', #optvalue='true'
EXEC REMOTESERVER.RemoteDB.dbo.UpdateTable
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc', #optvalue='false'
EXEC sp_serveroption #server='REMOTESERVER', #optname='rpc out', #optvalue='false'
If you must push data from the source to the target (e.g., for firewall or other permissions reasons), you can do the following:
In the source database, convert the recordset to a single XML string (i.e., multiple rows and columns combined into a single XML string).
Then push that XML over as a single row (as a varchar(max), since XML isn't allowed over linked databases in SQL Server).
DECLARE #xml XML
SET #xml = (select * from SourceTable FOR XML path('row'))
Insert into TempTargetTable values (cast(#xml AS VARCHAR(max)))
In the target database, cast the varchar(max) as XML and then use XML parsing to turn that single row and column back into a normal recordset.
DECLARE #X XML = (select '<toplevel>' + ImportString + '</toplevel>' from TempTargetTable)
DECLARE #iX INT
EXEC sp_xml_preparedocument #ix output, #x
insert into TargetTable
SELECT [col1],
[col2]
FROM OPENXML(#iX, '//row', 2)
WITH ([col1] [int],
[col2] [varchar](128)
)
EXEC sp_xml_removedocument #iX
I've found a workaround. Since I'm not a big fun of GUI tools like SSIS, I've reused a bcp script to load table into csv and vice versa. Yeah, it's an odd case to have the bulk operation support for files, but tables. Feel free to edit the following script to fit your needs:
exec xp_cmdshell 'bcp "select * from YourLocalTable" queryout C:\CSVFolder\Load.csv -w -T -S .'
exec xp_cmdshell 'bcp YourAzureDBName.dbo.YourAzureTable in C:\CSVFolder\Load.csv -S yourdb.database.windows.net -U youruser#yourdb.database.windows.net -P yourpass -q -w'
Pros:
No need to define table structures every time.
I've tested and it worked way faster than inserting directly through
the LinkedServer.
It's easier to manage than XML (which is limited to
varchar(max) length anyway).
No need of an extra layout of abstraction (tools like SSIS).
Cons:
Using the external tool bcp through the xp_cmdshell interface.
Table properties will be lost after ex/im-poring csv (i.e. datatype, nulls,length, separator within value, etc).