Interestingly enough I found no post for this specific, but basic issue.
Goal: update the latest budgetid record docstatus = 0. Then I want to update the next-to-last budgetid record docstatus = 1. I am trying this within PHP but also testing in my SQL Server SEM and it is failing there, too.
My SQL Server statement:
select
budgetid, docstatus, datechanged
from
ccy_budget
where
activityid = 11111
order by
datechanged desc
limit 1,1;
Error that occurs in SEM is:
Incorrect syntax near 'limit'.
Yet in w3schools this [sample] sql works just fine:
SELECT *
FROM Customers
ORDER BY postalcode DESC
LIMIT 1,1;
Seems so simple, surely I am missing something fundamental.
Microsoft SQL Server 2008 R2 (RTM) - 10.50.1600.1 (X64)
Apr 2 2010 15:48:46
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.2 <X64> (Build 9200: ) (Hypervisor)
Equivalent syntax in SQL Server would be
select *
from table
order by somerow desc
offset 1 rows fetch next 1 rows only;
But the above is available from SQL Server 2012 on, so for your version, you have to some thing like below
;with cte
as
(
select *,row_number() over (order by postalcode desc) as rn
from table
)
select * from cte where rn=2
Related
I find a huge difference in performance when I run two equivalent queries, one with IN (<values>) and the other one with IN (SELECT <same values>).
As I managed to reproduce this problem in several versions of MS SQL Server, including...
Microsoft SQL Server 2014 (SP3-GDR) (KB4583463) - 12.0.6164.21 (X64)
Nov 1 2020 04:25:14 Copyright (c) Microsoft Corporation Standard
Edition (64-bit) on Windows NT 6.3 (Build 9600: ) (Hypervisor)
Microsoft SQL Server 2017 (RTM-CU23) (KB5000685) - 14.0.3381.3 (X64)
Feb 9 2021 12:08:50 Copyright (C) 2017 Microsoft Corporation
Developer Edition (64-bit) on Windows Server 2016 Standard 10.0
(Build 14393: ) (Hypervisor)
Microsoft SQL Server 2019 (RTM-CU10)
(KB5001090) - 15.0.4123.1 (X64) Mar 22 2021 18:10:24 Copyright
(C) 2019 Microsoft Corporation Developer Edition (64-bit) on Windows
Server 2019 Standard 10.0 (Build 17763: ) (Hypervisor)
...I found this does not depend on the SQL Server version, and does not depend on the specific database, but on the specific database server instance. I mean: if I find this performance issue in some database, I am able to reproduce it in every database in that same database instance.
That makes me think there is some sort of setting/configuration that would allow me to get the same (good) performance in the databases where this IN (SELECT...) runs so slow. This problem is producing several deadlocks and timeouts in some of our databases.
Script to reproduce the problem (again, being able to reproduce it depends on some setting that I don't know but I want to find):
I create a test table with the following script:
It inserts 10.000.000 "Penguins", and they are linked in couples (i.e: 5.000.000 couples of Penguins)
--Create test table
CREATE TABLE Penguins
(
Id INT NOT NULL Primary Key,
PartnerId INT NOT NULL
);
--Create index
CREATE NONCLUSTERED INDEX [IX_Penguins_PartnerId] ON [Penguins]
(
[PartnerId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);
--Insert 10,000,000 test rows
WITH x AS (SELECT n FROM (VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) v(n))
INSERT INTO Penguins(Id, PartnerId)
SELECT internal.Number, CASE When internal.Number % 2 = 0 THEN internal.Number + 1 ELSE internal.Number - 1 END
FROM
(SELECT ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n + 10000*tensthousands.n + 100000*hundredthousands.n + 1000000*millions.n Number
FROM x ones, x tens, x hundreds, x thousands, x tensthousands, x hundredthousands, x millions
) internal
ORDER BY 1
Then I run this query, which in some databases takes 12 seconds:
I want to see the data of 3 specific penguins (4540734, 4540736, 6394285) and their partners:
--Run query
declare #p0 as Table (
Id INT NOT NULL PRIMARY KEY
)
INSERT INTO #p0 (ID) VALUES (4540734)
INSERT INTO #p0 (ID) VALUES (4540736)
SELECT *
FROM Penguins
INNER JOIN Penguins Partner ON Partner.Id = Penguins.PartnerId
WHERE
Penguins.Id IN (
SELECT Id FROM #p0
)
Or
Penguins.Id = 6394285
But in those same databases, if instead of using a SELECT ... I specify the list of values, I get the same results but the query takes less than 1 second:
--Run query
SELECT *
FROM Penguins
INNER JOIN Penguins Partner ON Partner.Id = Penguins.PartnerId
WHERE
Penguins.Id IN (
--By defining the values as a list, it runs in less than 1 second
4540734,4540736
)
Or
Penguins.Id = 6394285
Bear in mind that I get this difference only if I combine the "IN (SELECT...)" with the "OR Penguins.Id = 6394285"
Execution plan of slow execution of IN (SELECT ... in the servers where it runs slow:
Execution plan of fast execution of IN (SELECT ... in the servers where it runs fast:
Execution plan of fast execution, using IN (values):
This question is not about how to modify the query to improve its performance, but to identify why the execution plan is so different in different database server instances.
Could you please help me identify what this performance difference depends on? This behaviour surprised many experienced database developers I talked with.
Thank you.
Strange situation... New physical severs, new install of SQL Server 2019 Enterprise version :
Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64) Sep 24 2019 13:48:23 Copyright (C) 2019 Microsoft Corporation Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2019 Standard 10.0 (Build 17763: ).
Testing the performance by creating the first database like this :
CREATE DATABASE DB_BENCH
GO
DECLARE #SQL NVARCHAR(max) = N'';
SELECT #SQL = #SQL + N'ALTER DATABASE DB_BENCH MODIFY FILE (NAME = ''' + name + N''', SIZE = 10 GB, FILEGROWTH = 64 MB);'
FROM DB_BENCH.sys.database_files;
SET #SQL = #SQL + N'ALTER DATABASE DB_BENCH SET RECOVERY SIMPLE;'
EXEC (#SQL);
GO
And the objects in the database like this :
USE DB_BENCH
GO
SET NOCOUNT ON;
GO
CREATE TABLE T_TIME_INTERVAL_TIV
(TIV_ID INT NOT NULL IDENTITY PRIMARY KEY,
TIV_GROUP INT,
TIV_DEBUT DATETIME2(0),
TIV_FIN DATETIME2(0))
GO
TRUNCATE TABLE T_TIME_INTERVAL_TIV;
GO
BULK INSERT T_TIME_INTERVAL_TIV
FROM "C:\DATA_SQL\intervals.txt"
WITH (KEEPIDENTITY ,
FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n');
GO
CREATE VIEW V
AS
SELECT TIV_GROUP AS id, TIV_DEBUT AS intime, TIV_FIN AS outtime
FROM T_TIME_INTERVAL_TIV
GO
The intervals.txt datafile contains 1 million lines.
You can have it at :
https://1drv.ms/t/s!AqvZfiQYoNpBiCD65D4zaRbch5s-?e=UicEYu
The query that produce the bug :
WITH T1 As
(SELECT id, intime
FROM #T
UNION ALL
SELECT id, outtime FROM #T),
T2 As
(SELECT ROW_NUMBER() OVER(PARTITION BY id ORDER BY intime) NN, id, intime
FROM T1 T1_1),
T3 As
(SELECT T2_1.NN - ROW_NUMBER() OVER(PARTITION BY T2_1.id ORDER BY T2_1.intime,T2_2.intime) NN1,
T2_1.id, T2_1.intime intime, T2_2.intime outtime
FROM T2 T2_1
INNER JOIN T2 T2_2
ON T2_1.id=T2_2.id
And T2_1.NN=T2_2.NN-1
WHERE EXISTS (SELECT *
FROM V S
WHERE S.id=T2_1.id
AND (S.intime < T2_2.intime AND S.outtime>T2_1.intime))
OR T2_1.intime = T2_2.intime)
SELECT id, MIN(intime) intime, MAX(outtime) outtime
FROM T3
GROUP BY id, NN1
ORDER BY id, intime, outtime;
We tested this query on 2 different servers... with the same SQL Server installation.
The result is always :
Msg 601, Level 12, State 1, Line ...
Could not continue scan with NOLOCK due to data movement.
With an installation of :
SQL Server 2019 (RTM) - 15.0.2000.5 (X64) Sep 24 2019 13:48:23 Developer Edition (64-bit) on Windows Server 2019 Standard 10.0 (Build 17763: ) (Hypervisor)
There is no problem...
We tested the databases with DBCC CHECKDB () WITH DATA_PURITY. No errors.
Can you reproduce with your different editions/patches (CU) and give me your results for which are alright or not ?
If some of you have already the bug I will add an entry in SQL Azure feedback...
Thanks
After many investigation, we found that it is really a SQL Server bug.
The bug disappear when we execute a :
UPDATE STATISTICS T_TIME_INTERVAL_TIV WITH FULLSCAN;
Or when whe "hint" the query with OPTION (MAXDOP 1)
Sometime a stack dump appear (not Always) showing this type of messages :
A time-out occurred while waiting for buffer latch -- type 4, bp 0000029BA883BDC0, page 9:407, stat 0x10b, database id: 2, allocation unit Id: 422212465393664/140737488683008, task 0x0000029B86723848 : 14, waittime 300 seconds, flags 0x1a, owning task 0x0000029B86713848. Not continuing to wait.
Which is the tempdb. A Stack Dump is systematically recorded on file.
We will call the MS hotline as soon as possible.
Thanks to all of you.
This is the simplest example with which I could reproduce the issue. As such it looks a little bit contrived, but bear with me.
declare #t table(number int)
insert into #t values (1), (2)
declare #sum bigint = 0
select #sum = #sum + number
from (select top 2 number from #t order by number) subquery
order by number desc
select #sum
Here's the query on the data explorer.
I would expect this to return 3, the sum of the values in the table #t. Instead, however, it returns 1.
Doing any of the following will cause the query to correctly return 3:
make #t.number and #sum have the same type (by making #sum an int or #t.number a bigint).
removing the outer order by
removing the inner order by
making both order bys sort in the same direction by adding desc to the inner one or removing it from the outer one
removing the subquery (i.e. just selecting from #t)
None of these things strike me as something that should change the behavior of this query.
Swapping the sort orders (descending in the subquery, ascending on the outside) will make the query return 2 instead of 1.
A similar thing happens with strings instead of numbers, so this isn't constrained to int and bigint.
This happens both with SQL Server 2014 and 2016, or to be precise
Microsoft SQL Server 2014 - 12.0.2000.8 (X64)
Feb 20 2014 20:04:26
Copyright (c) Microsoft Corporation
Developer Edition (64-bit) on Windows NT 6.3 <X64> (Build 10586: )
and
Microsoft SQL Server 2016 (RTM-CU1) (KB3164674) - 13.0.2149.0 (X64)
Jul 11 2016 22:05:22
Copyright (c) Microsoft Corporation
Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2012 R2 Standard 6.3 <X64> (Build 9600: )
(the latter being the data explorer).
What's going on?
The answer seems to be that you are/were relying on undocumented behaviour which changed in Sql Server 2012.
Per the documentation:
https://msdn.microsoft.com/en-us/library/ms187330.aspx
SELECT #local_variable is typically used to return a single value into the variable. However, when expression is the name of a column, it can return multiple values. If the SELECT statement returns more than one value, the variable is assigned the last value that is returned.
It is not documented what happens if the destination variable (to be assigned to) is part of the source expression. It seems this behaviour has changed. In earlier versions the variable would be assigned once for each row, but that doesn't seem to occur any more.
This is most visible for a lot of functions where the "group concat" trick ceased to work:
SELECT #sentence = #sentence + ' ' + word from SENTENCE_WORDS order by position
These have generally to be replaced by the xml concat trick.
set #sentence = (
select word as "text()", ' ' as "text()"
from SENTENCE_WORDS
order by position
for xml path(''), root('root'), type
).value('(/root)[1]', 'nvarchar(max)')
Remove the second ORDER BY (I.e. "order by number desc").
You are using an undocumented feature of T-SQL (I believe it's called ROW concatenation?) which is not guaranteed to work in future versions of SQL. It's a little hacky, but very useful none-the-less! As you've discovered, it breaks when you use the ORDER BY clause. This is a known issue of using Row concatenation.
The application I am using is written by dinosaurs, using *= and =* operators in sql queries. These queries do joins on 10 or so tables, and I have about 500 such queries. And all I have time for is to upgrade the existing software as-is instead of writing any code. So, updating the query builder functions in all of them is not feasible.
Sadly, I am supposed to upgrade the application to use SQL server 2014, instead of 2005. And in 2014, the backward compatibility for a lot of operators is removed. So, I need to enable some kind of compatibility option in SQL server 2014, if it is available. But, I don't know if there is such a thing, and if it is, how to enable it.
Found a query to find the current compatibility setting. Posting it here, in case someone else needs it.
WITH compatversions AS (
SELECT 65 AS MajorVersion ,'SQL Server 6.5' AS ServerVersion
UNION
SELECT 70,'SQL Server 7.0'
UNION
SELECT 80 , 'SQL Server 2000'
UNION
SELECT 90 , 'SQL Server 2005'
UNION
SELECT 100 , 'SQL Server 2008/R2'
UNION
SELECT 110 , 'SQL Server 2012'
UNION
SELECT 120 , 'SQL Server 2014'
)
SELECT TOP 3 ServerVersion,MajorVersion
,ServerVersion + ' ('+ CONVERT(VARCHAR(3),MajorVersion) +')' AS DropDownFormat
FROM compatversions
WHERE MajorVersion IN (
SELECT TOP 3 MajorVersion
FROM compatversions
WHERE MajorVersion <= CONVERT(INT,CAST(##microsoftversion/ 0x1000000 AS VARCHAR(3)) + '0')
ORDER BY MajorVersion DESC)
ORDER BY MajorVersion ASC;
Apparently, SQL server 2014 supports up to 2008 compatibility.
SQL Server 2008 Full-Text Search (FTS) is extremely slow in this scenario:
Query 1:
SELECT [...] FROM ContentItem CI WHERE
(**EXISTS** (SELECT TOP 1 * FROM **CONTAINSTABLE**([**Table1**], *, '"[search_string]*"') FT
WHERE FT.[Key] = CI.ContentItem_Id))
ORDER BY [...]
Results: super fast on SQL 2005 and SQL 2008
Query 2:
SELECT [...] FROM ContentItem CI WHERE
(**EXISTS** (SELECT TOP 1 * FROM **CONTAINSTABLE**([**Table2**], *, '"[search_string]*"') FT
WHERE FT.[Key] = CI.ContentItem_Id))
ORDER BY [...]
Results: super fast on SQL 2005 and SQL 2008
Query 3:
SELECT [...] FROM ContentItem CI WHERE
(**EXISTS** (SELECT TOP 1 * FROM **CONTAINSTABLE**([**Table1**], *, '"[search_string]*"') FT
WHERE FT.[Key] = CI.ContentItem_Id)
**OR EXISTS** (SELECT TOP 1 * FROM **CONTAINSTABLE**([**Table2**], *, '"[search_string]*"') FT
WHERE FT.[Key] = CI.ContentItem_Id))
ORDER BY [...]
Results: super fast on SQL 2005 (about a second), but extremely slow (3 min+) on SQL 2008
I'm aware of performance issues with SQL 2008 FTS (even on stackoverflow), but haven't find any reasonable solution yet.
Can you rewrite Query 3 to
SELECT ... WHERE EXISTS ... CONTAINSTABLE(Table1...)
UNION
SELECT ... WHERE EXISTS ... CONTAINSTABLE(Table2...)
ORDER BY ...
?
UNION ALL may be faster than UNION, but possibly result in duplicate records.
You probably get a bad plan. If you can post plan, it will help diagnose the problem. In case a bad join was choose, you can use query hint to solve the problem.
We just upgraded to SQL 2008 and ran into this issue. I found if I put this at the bottom of the query, it worked great for me: OPTION (MAXDOP 1)