SQL Server: OPENQUERY in easy words (with example) - sql-server

I am pretty new to SQL Server, and I have stumbled upon this right here:
BEGIN TRANSACTION loadTTAllocations
INSERT INTO #ttaAllocs
SELECT
company_code AS CompanyId,
job_number AS JobNo,
...
is_blocked AS IsBlocked
FROM
OPENQUERY([SRV_TimeTac],
'SELECT DISTINCT
job.individual_value_1 AS job_number,
job.individual_value_3 AS company_code,
...
now() AS queryTimeStamp
FROM
pm_altran.pm_tasks_subprojects AS taskCode
INNER JOIN
pm_altran.pm_tasks_subprojects AS job
ON job.id = taskCode.mother_id
AND job.is_done = 0
AND NOT job.is_blocked
INNER JOIN
pm_node_to_user AS n2u
ON n2u.node_id = taskCode.id
AND n2u.access = 1
AND n2u.is_todo = 1
LEFT JOIN
altran_pm_user_user_settings AS u
ON u.administrators_id = n2u.user_id
WHERE
taskCode.object_type = ''task''
AND taskCode.is_paid_non_working = 0
AND taskCode.id > 50');
SET #rowCount = ##ROWCOUNT
SET #eventDetails = 'End loadTTAllocations: ' + CAST(#rowCount as VARCHAR(10)) + ' rows affected';
COMMIT TRANSACTION loadTTAllocations
The issue is the OPENQUERY.
What I understand from the docs is basically, that it is a query just on another server. In this case the SRV_TimeTac?
So in this example, we are querying on another server called "SRV_TimeTac" and returning results to be loaded finally into the temp table called #ttaAllocs.
Is this correct?
Thank you for your help.

OPENQUERY (Transact-SQL):
Executes the specified pass-through query on the specified linked
server. This server is an OLE DB data source. OPENQUERY can be
referenced in the FROM clause of a query as if it were a table name.
OPENQUERY can also be referenced as the target table of an INSERT,
UPDATE, or DELETE statement. This is subject to the capabilities of
the OLE DB provider. Although the query may return multiple result
sets, OPENQUERY returns only the first one.
Arguments
linked_server
Is an identifier representing the name of the linked server.
' query '
Is the query string executed in the linked server. The maximum length of the string is 8 KB.
So, to answer your question, yes, that is what your query is doing. The value of linked_server in your case is SRV_TimeTac and the value of ' query ' is your long literal string. The value of ' query ' is run on the linked server and the result set returned to the server you ran OPENQUERY on. That result set is then inserted into the variable #ttaAllocs.
You interestingly then COMMIT the transaction you started before the INSERT, which seems odd considering you only inserted the value into a variable and didn't effect any persisted objects. The Table Variable may well have ended up being written to disc, rather than stored in Memory, if it got large enough but then you'd very likely be committing the transaction to tempdb.

Related

SQL Server linked-server supplied inconsistent metadata for a column

I've encountered a problem executing a T-SQL statement (inserting and updating) on linked Oracle servers when executing the query twice.
The first execution works like a charm, but when trying a second time, I get this error:
The OLE DB provider "OraOLEDB.Oracle" for linked server "Linked_Server" supplied inconsistent metadata. An extra column was supplied during execution that was not found at compile time.
I've already looked it up and tried several approaches:
Use of openquery instead of original statement as suggested here: MSSQL Linked Server error: The OLE DB provider "OraOLEDB.Oracle" for linked server supplied inconsistent metadata for a column
Use OPTION (RECOMPILE) at the end of original statement as well as on the openquery statement as suggested here: https://dba.stackexchange.com/questions/275605/linked-server-throws-metadata-error or here https://learn.microsoft.com/en-us/answers/questions/98208/linked-server-throws-metadata-error.html
One thing I've encountered is when using OPTION (RECOMPILE) the statement is infinite and I had to cancel it manually when executing the query the 2nd time.
Original statement:
declare #insert_cmd varchar(3000) = dbo.substringProc('
insert into LINKED_SERVER.TABLE (E_KNZ_NR, E_KNZZTRH_PERIODE_DAT, E_KNZZTRH_IST_WT, E_KNZZTRH_ERSTERF_TS)
select
id_dwh_knz,
monitor_dat,
monitor_wert,
SYSDATETIME()
from temp_nes_kennzahl zr
left outer join (
select
E_KNZ_NR,
periode_dat=cast(E_KNZZTRH_PERIODE_DAT as date),
E_KNZZTRH_IST_WT
from LINKED_SERVER2.TABLE
) dwh_zr
on zr.id_dwh_knz = dwh_zr.E_KNZ_NR
and zr.monitor_dat = dwh_zr.periode_dat
where dwh_zr.periode_dat is null
order by monitor_dat asc, id_dwh_knz asc ') -- OPTION (RECOMPILE)
exec (#insert_cmd);
Statement with openquery:
declare #insert_cmd varchar(3000) = dbo.substringProc('
insert into openquery (LINKED_SERVER, ''SELECT E_KNZ_NR, E_KNZZTRH_PERIODE_DAT, E_KNZZTRH_IST_WT, E_KNZZTRH_ERSTERF_TS FROM LINKED_SERVER.TABLE'')
select
id_dwh_knz,
monitor_dat,
monitor_wert,
SYSDATETIME()
from temp_nes_kennzahl zr
left outer join (
select * from openquery (LINKED_SERVER2,''SELECT E_KNZ_NR,CAST(E_KNZZTRH_PERIODE_DAT as DATE) AS periode_dat,E_KNZZTRH_IST_WT FROM LINKED_SERVER2.TABLE'')
) dwh_zr
on zr.id_dwh_knz = dwh_zr.E_KNZ_NR
and zr.monitor_dat = dwh_zr.periode_dat
where dwh_zr.periode_dat is null
order by monitor_dat asc, id_dwh_knz asc') -- OPTION (RECOMPILE)
exec (#insert_cmd);
(The 'substringProc' is just a helper method to retrieve the related user/database for production or test environment and 'dwh' stands for the datawarehouse at the linked oracle server)
Would be nice if someone got a solution for that, since I'm struggling with this error for quite a long time and both of the queries work - but only once. I've also read that there might be a problem with the execution plan stored in the cache but I have no clue how to workaround such an issue.
Thanks in advance

SSIS SQL TASK MAX(DATE) to Variable in DATA FLOW

OK this seems like it should be insanely easy, but I cannot figure it out. Every where I look online says to create temp tables and VB scripts and I cannot believe I have to do that. My goal is to insert all the records in a table with a date later than the max date in that destination table.
UPDATE The 2 tables are in two different non linked SQL databases
So:
Select #[User::Dated] = MAX(Dateof) from Table2
Insert into Table2
Select *
From Table1
Where DateOf > #[User::Dated]
I am trying to do this in SSIS. I declared a variable, the SQL execution step looks like it is assigning the single row output to it. But when I got go into the data flow it give me no parameters to choose, when I force the known parameter which is in the project scope it says no parameter exists
Create two OLE DB data sources each pointing at you two databases.
Create a variable called max_date and make its data type String.
Place an Execute SQL Task on the Control Flow, change its connection type to OLE DB and for the connection select the name of the data source that contains Table2. Set the ResultSet to Single Row. Add the following for the SQLStatement:
SELECT CAST(MAX(Dateof) AS VARCHAR) AS max_date FROM Table2
Go to the Result Set pane, click Add and enter the following:
Result Name: max_date
Variable Name: User::max_date
You can now use the max_date variable in an expression to create a SQL statement, for example you could use it in another Execute SQL Task which would use the second Data Connection like so:
"INSERT INTO Table2
SELECT *
FROM Table1
WHERE DateOf > '" + #[User::max_date] + "'"
Or in an OLE DB Source in a data flow like so:
"SELECT *
FROM Table1
WHERE DateOf > '" + #[User::max_date] + "'"
You can do this in a single SQL Task if you want:
Insert into Table2
Select *
From Table1
Where DateOf > (Select MAX(Dateof) from Table2)
If you want to use multiple Execute SQL Task items in the control flow, or want to make use of the parameter in a data flow instead, you have to change the General > Result Set option for your MAX() query to Single Row, then move from General to Result Set and Add a new variable for your result set to occupy.
To use that variable in your INSERT INTO.... query via Execute SQL Task, you'll construct your query with a ? for each parameter and map them in the parameter mapping section. If a variable is used multiple times in a query it's easiest to use a stored procedure, so you can simply pass the relevant parameters in SSIS.

Logic and Checking Tables within SQL Server Stored Procedures

I'm getting my feet wet with Stored Procedures finally and need a little assistance with my first one.
I get the basic layout and have set up a SP that does nothing but set up two variables and select them, returning no information.
Right the history, I have two main record tables tblRecordsCurrent (RecordID) and tblRecordsHistorical (RecordID) as well as a parent table to both tblDataSet (DatasetID) that has a flag dsCurrent.
Each day I upload 5000 records to tblRecordsCurrent, using a new DataSetID. When I upload a new dataset, the previous day's dataset (5000 records) is moved to tblRecordsHistorical and the DSCurrent flag in the tblDataSet table is updated to False.
I will feed the StoredProcedure a DataSetID to move, and the destination ('C' or 'H' for Current/Historical). First I need to check that the DataSetID exists, then if there are any records in the relevant tblRecordsCurrent/Historical with the DataSetID.
1 - How do I find out if the DataSetID exists, which I presume would be a SELECT COUNT(*) from tblDataSet where DataSetID=#DataSet, but how is that implemented, do I just do:
#DataSetExistCount=SELECT COUNT(*) from tblDataSet where DataSetID=#DataSet
if #DataSetExistCount== then
#errCount = #errCount + 1
else
>>> MORE LOGIC <<<
END
EDIT:
This was a great kick-start to my Stored Procedure, which I have now posted under a separate Error Trapping based question if anyone is interested in complex nested logic in Stored Procedures:
Error Handling in SQL Server Stored Procedures
You just check to see if it exists:
If Exists ( select 1 from tblDataSet where DataSetID = #dataSet )
begin
-- Do your move logic
end
else
begin
-- Doesn't exist so do something else
end

SQL Update across Linked Server

I have a Linked Server set up on my host Server: "MyLinkedServer"
I have an SP on my server "HostServer".
I am calling a stored proc on HostServer that updates a table in DatabaseA on MyLinkedServer with values from a table in DatabaseB on MyLinkedServer.
I have other SPs that run fine in the same scenario, but they are doing inserts and deletes, however this SP fails to update the table in DatabaseA (no error returned, just no changed data), and if I change connections to actually run the SP on "MyLinkedServer" it works without a problem.
UPDATE MyLinkedServer.DataBaseA.dbo.MyTable
SET Column1 = db2.Column1
FROM MyLinkedServer.DataBaseA.dbo.MyTable db1
INNER JOIN
(
SELECT TOP 1 Column1
FROM MyLinkedServer.DataBaseB.dbo.MyTable db2
WHERE db2.Id = 2
) AS db2 ON db2.Id = 2
WHERE db1.Id = 1
I believe you'll need to reference the alias you reference in the from statement. Does changing
UPDATE MyLinkedServer.DataBaseA.dbo.MyTable
into
UPDATE db2
fix your issue?

Hidden Features of SQL Server

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What are some hidden features of SQL Server?
For example, undocumented system stored procedures, tricks to do things which are very useful but not documented enough?
Answers
Thanks to everybody for all the great answers!
Stored Procedures
sp_msforeachtable: Runs a command with '?' replaced with each table name (v6.5 and up)
sp_msforeachdb: Runs a command with '?' replaced with each database name (v7 and up)
sp_who2: just like sp_who, but with a lot more info for troubleshooting blocks (v7 and up)
sp_helptext: If you want the code of a stored procedure, view & UDF
sp_tables: return a list of all tables and views of database in scope.
sp_stored_procedures: return a list of all stored procedures
xp_sscanf: Reads data from the string into the argument locations specified by each format argument.
xp_fixeddrives:: Find the fixed drive with largest free space
sp_help: If you want to know the table structure, indexes and constraints of a table. Also views and UDFs. Shortcut is Alt+F1
Snippets
Returning rows in random order
All database User Objects by Last Modified Date
Return Date Only
Find records which date falls somewhere inside the current week.
Find records which date occurred last week.
Returns the date for the beginning of the current week.
Returns the date for the beginning of last week.
See the text of a procedure that has been deployed to a server
Drop all connections to the database
Table Checksum
Row Checksum
Drop all the procedures in a database
Re-map the login Ids correctly after restore
Call Stored Procedures from an INSERT statement
Find Procedures By Keyword
Drop all the procedures in a database
Query the transaction log for a database programmatically.
Functions
HashBytes()
EncryptByKey
PIVOT command
Misc
Connection String extras
TableDiff.exe
Triggers for Logon Events (New in Service Pack 2)
Boosting performance with persisted-computed-columns (pcc).
DEFAULT_SCHEMA setting in sys.database_principles
Forced Parameterization
Vardecimal Storage Format
Figuring out the most popular queries in seconds
Scalable Shared Databases
Table/Stored Procedure Filter feature in SQL Management Studio
Trace flags
Number after a GO repeats the batch
Security using schemas
Encryption using built in encryption functions, views and base tables with triggers
In Management Studio, you can put a number after a GO end-of-batch marker to cause the batch to be repeated that number of times:
PRINT 'X'
GO 10
Will print 'X' 10 times. This can save you from tedious copy/pasting when doing repetitive stuff.
A lot of SQL Server developers still don't seem to know about the OUTPUT clause (SQL Server 2005 and newer) on the DELETE, INSERT and UPDATE statement.
It can be extremely useful to know which rows have been INSERTed, UPDATEd, or DELETEd, and the OUTPUT clause allows to do this very easily - it allows access to the "virtual" tables called inserted and deleted (like in triggers):
DELETE FROM (table)
OUTPUT deleted.ID, deleted.Description
WHERE (condition)
If you're inserting values into a table which has an INT IDENTITY primary key field, with the OUTPUT clause, you can get the inserted new ID right away:
INSERT INTO MyTable(Field1, Field2)
OUTPUT inserted.ID
VALUES (Value1, Value2)
And if you're updating, it can be extremely useful to know what changed - in this case, inserted represents the new values (after the UPDATE), while deleted refers to the old values before the UPDATE:
UPDATE (table)
SET field1 = value1, field2 = value2
OUTPUT inserted.ID, deleted.field1, inserted.field1
WHERE (condition)
If a lot of info will be returned, the output of OUTPUT can also be redirected to a temporary table or a table variable (OUTPUT INTO #myInfoTable).
Extremely useful - and very little known!
Marc
sp_msforeachtable: Runs a command with '?' replaced with each table name.
e.g.
exec sp_msforeachtable "dbcc dbreindex('?')"
You can issue up to 3 commands for each table
exec sp_msforeachtable
#Command1 = 'print ''reindexing table ?''',
#Command2 = 'dbcc dbreindex(''?'')',
#Command3 = 'select count (*) [?] from ?'
Also, sp_MSforeachdb
Connection String extras:
MultipleActiveResultSets=true;
This makes ADO.Net 2.0 and above read multiple, forward-only, read-only results sets on a single database connection, which can improve performance if you're doing a lot of reading. You can turn it on even if you're doing a mix of query types.
Application Name=MyProgramName
Now when you want to see a list of active connections by querying the sysprocesses table, your program's name will appear in the program_name column instead of ".Net SqlClient Data Provider"
TableDiff.exe
Table Difference tool allows you to discover and reconcile differences between a source and destination table or a view. Tablediff Utility can report differences on schema and data. The most popular feature of tablediff is the fact that it can generate a script that you can run on the destination that will reconcile differences between the tables.
Link
A less known TSQL technique for returning rows in random order:
-- Return rows in a random order
SELECT
SomeColumn
FROM
SomeTable
ORDER BY
CHECKSUM(NEWID())
In Management Studio, you can quickly get a comma-delimited list of columns for a table by :
In the Object Explorer, expand the nodes under a given table (so you will see folders for Columns, Keys, Constraints, Triggers etc.)
Point to the Columns folder and drag into a query.
This is handy when you don't want to use heinous format returned by right-clicking on the table and choosing Script Table As..., then Insert To... This trick does work with the other folders in that it will give you a comma-delimited list of names contained within the folder.
Row Constructors
You can insert multiple rows of data with a single insert statement.
INSERT INTO Colors (id, Color)
VALUES (1, 'Red'),
(2, 'Blue'),
(3, 'Green'),
(4, 'Yellow')
If you want to know the table structure, indexes and constraints:
sp_help 'TableName'
HashBytes() to return the MD2, MD4, MD5, SHA, or SHA1 hash of its input.
Figuring out the most popular queries
With sys.dm_exec_query_stats, you can figure out many combinations of query analyses by a single query.
Link
with the commnad
select * from sys.dm_exec_query_stats
order by execution_count desc
The spatial results tab can be used to create art.
enter link description here http://michaeljswart.com/wp-content/uploads/2010/02/venus.png
EXCEPT and INTERSECT
Instead of writing elaborate joins and subqueries, these two keywords are a much more elegant shorthand and readable way of expressing your query's intent when comparing two query results. New as of SQL Server 2005, they strongly complement UNION which has already existed in the TSQL language for years.
The concepts of EXCEPT, INTERSECT, and UNION are fundamental in set theory which serves as the basis and foundation of relational modeling used by all modern RDBMS. Now, Venn diagram type results can be more intuitively and quite easily generated using TSQL.
I know it's not exactly hidden, but not too many people know about the PIVOT command. I was able to change a stored procedure that used cursors and took 2 minutes to run into a speedy 6 second piece of code that was one tenth the number of lines!
useful when restoring a database for Testing purposes or whatever. Re-maps the login ID's correctly:
EXEC sp_change_users_login 'Auto_Fix', 'Mary', NULL, 'B3r12-36'
Drop all connections to the database:
Use Master
Go
Declare #dbname sysname
Set #dbname = 'name of database you want to drop connections from'
Declare #spid int
Select #spid = min(spid) from master.dbo.sysprocesses
where dbid = db_id(#dbname)
While #spid Is Not Null
Begin
Execute ('Kill ' + #spid)
Select #spid = min(spid) from master.dbo.sysprocesses
where dbid = db_id(#dbname) and spid > #spid
End
Table Checksum
Select CheckSum_Agg(Binary_CheckSum(*)) From Table With (NOLOCK)
Row Checksum
Select CheckSum_Agg(Binary_CheckSum(*)) From Table With (NOLOCK) Where Column = Value
I'm not sure if this is a hidden feature or not, but I stumbled upon this, and have found it to be useful on many occassions. You can concatonate a set of a field in a single select statement, rather than using a cursor and looping through the select statement.
Example:
DECLARE #nvcConcatonated nvarchar(max)
SET #nvcConcatonated = ''
SELECT #nvcConcatonated = #nvcConcatonated + C.CompanyName + ', '
FROM tblCompany C
WHERE C.CompanyID IN (1,2,3)
SELECT #nvcConcatonated
Results:
Acme, Microsoft, Apple,
If you want the code of a stored procedure you can:
sp_helptext 'ProcedureName'
(not sure if it is hidden feature, but I use it all the time)
A stored procedure trick is that you can call them from an INSERT statement. I found this very useful when I was working on an SQL Server database.
CREATE TABLE #toto (v1 int, v2 int, v3 char(4), status char(6))
INSERT #toto (v1, v2, v3, status) EXEC dbo.sp_fulubulu(sp_param1)
SELECT * FROM #toto
DROP TABLE #toto
In SQL Server 2005/2008 to show row numbers in a SELECT query result:
SELECT ( ROW_NUMBER() OVER (ORDER BY OrderId) ) AS RowNumber,
GrandTotal, CustomerId, PurchaseDate
FROM Orders
ORDER BY is a compulsory clause. The OVER() clause tells the SQL Engine to sort data on the specified column (in this case OrderId) and assign numbers as per the sort results.
Useful for parsing stored procedure arguments: xp_sscanf
Reads data from the string into the argument locations specified by each format argument.
The following example uses xp_sscanf
to extract two values from a source
string based on their positions in the
format of the source string.
DECLARE #filename varchar (20), #message varchar (20)
EXEC xp_sscanf 'sync -b -fproducts10.tmp -rrandom', 'sync -b -f%s -r%s',
#filename OUTPUT, #message OUTPUT
SELECT #filename, #message
Here is the result set.
-------------------- --------------------
products10.tmp random
Return Date Only
Select Cast(Floor(Cast(Getdate() As Float))As Datetime)
or
Select DateAdd(Day, 0, DateDiff(Day, 0, Getdate()))
dm_db_index_usage_stats
This allows you to know if data in a table has been updated recently even if you don't have a DateUpdated column on the table.
SELECT OBJECT_NAME(OBJECT_ID) AS DatabaseName, last_user_update,*
FROM sys.dm_db_index_usage_stats
WHERE database_id = DB_ID( 'MyDatabase')
AND OBJECT_ID=OBJECT_ID('MyTable')
Code from: http://blog.sqlauthority.com/2009/05/09/sql-server-find-last-date-time-updated-for-any-table/
Information referenced from:
SQL Server - What is the date/time of the last inserted row of a table?
Available in SQL 2005 and later
Here are some features I find useful but a lot of people don't seem to know about:
sp_tables
Returns a list of objects that can be
queried in the current environment.
This means any object that can appear
in a FROM clause, except synonym
objects.
Link
sp_stored_procedures
Returns a list of stored procedures in
the current environment.
Link
Find records which date falls somewhere inside the current week.
where dateadd( week, datediff( week, 0, TransDate ), 0 ) =
dateadd( week, datediff( week, 0, getdate() ), 0 )
Find records which date occurred last week.
where dateadd( week, datediff( week, 0, TransDate ), 0 ) =
dateadd( week, datediff( week, 0, getdate() ) - 1, 0 )
Returns the date for the beginning of the current week.
select dateadd( week, datediff( week, 0, getdate() ), 0 )
Returns the date for the beginning of last week.
select dateadd( week, datediff( week, 0, getdate() ) - 1, 0 )
Not so much a hidden feature but setting up key mappings in Management Studio under Tools\Options\Keyboard:
Alt+F1 is defaulted to sp_help "selected text" but I cannot live without the adding Ctrl+F1 for sp_helptext "selected text"
Persisted-computed-columns
Computed columns can help you shift the runtime computation cost to data modification phase. The computed column is stored with the rest of the row and is transparently utilized when the expression on the computed columns and the query matches. You can also build indexes on the PCC’s to speed up filtrations and range scans on the expression.
Link
There are times when there's no suitable column to sort by, or you just want the default sort order on a table and you want to enumerate each row. In order to do that you can put "(select 1)" in the "order by" clause and you'd get what you want. Neat, eh?
select row_number() over (order by (select 1)), * from dbo.Table as t
Simple encryption with EncryptByKey

Resources