Transact sql query perfomance advice - sql-server

I have 2 select queries. The first performs selection for grid paging(25 rows per page) (using TOP #pagesize*#pagenum EXCEPT TOP #pagesize*(#pagenum-1) construction). Second query total count of rows.
So, will WITH AS construction increase performance versus two separated queries, if so why? Note, that query contains multiple columns, INNER JOINs and WHERE conditions.

WITH part for itself does not help performance. It makes the query easier to understand.
If I understand correctly, the count is for all the rows. You can save the second call if you get all the rows to the client, but in most cases it will be more expensive.
Using top X to get only the last X/Y rows is a bad idea. You should do add an auto number and select the rows where the auto number is in the desired range.
SELECT ROW_NUMBER() OVER (ORDER BY [Something] DESC) FROM ...
WHERE [RowNumber] BETWEEN 10 AND 20

Alternatively, if you use ORDER BY ... OFFSET instead of TOP ... ORDER BY you can use COUNT(*) OVER () to get all rows regardless of paging. Else you have to isolate your data using WITH (as you do) and get paging with the 2nd table as well as anything else (row number, page, total pages and so on.
Example without offset:
DECLARE #page INT = 1, #rows INT = 5
;WITH data AS (SELECT * FROM mytable where id = 454545) --possible filters
,rows ([page], [pages], [rows]) AS
(
SELECT #page, CEILING(CAST(COUNT(*) AS float)/#rows), COUNT(*) FROM data
)
SELECT TOP (#rows) *
FROM ( SELECT row_number() OVER (ORDER BY data.id) rowNumber, * FROM rows, data ) pagination
WHERE rowNumber > (#page - 1) * #rows
order by rowNumber
Example with offset:
DECLARE #page INT = 1, #rows INT = 5
SELECT
row_number() OVER (order by id ) rowNumber,#page Page,(CEILING(CasT(COUNT(*) OVER () as float)/#rows)) Pages,
COUNT(*) OVER () Rows, *
from mytable where id = 454545 --possible filters
order by rowNumber
OFFset (#page-1)*#rows rows
FETCH NEXT #rows rows ONLY
In both cases make sure the order by is absolute else your paging is not guaranteed

Related

Pagination in SQL Server 2012 stored procedure with top distinct(x) records

I want to create stored procedure with pagination along with top 100 against a subset of a table. For example in a table(ex:employee table) with more than 3,000,000 records, I want to take the top 100,000 records and do the pagination. I'm able to do the pagination using below script, but I want to take the top 100,000 records and do the pagination.
DECLARE #currentPageNo int,#takeData int
SET #currentPageNo =1
SET #takeData = 10
SELECT DISTINCT emp.empid,emp.name,s.Salary,
FROM Employee emp
LEFT OUTER JOIN salary S ON emp.empid=S.empid
where emp.empid=12
ORDER BY emp.empid desc
OFFSET (#currentPageNo - 1) * #takeData ROWS
FETCH NEXT #takeData ROWS ONLY
I need some suggestions on how this can be achieved. I'm using SQL Server 2012.
In order to perform pagination properly you will need to ensure the order of the result set and assign row numbers, for this you can use ROWNUMBER() OVER()
You will also need to know the total count of the entire result set. This will tell you how many total pages you have. You can do this in a secondary query or add another column for Total Rows using COUNT() OVER()
Here's an example using your variables:
DECLARE #currentPageNo int,#takeData int
SET #currentPageNo =0
SET #takeData = 10
SELECT TOP(#currentPageNo) * FROM
(
SELECT ROW_NUMBER() OVER(ORDER BY [ColumnName]) RowNumber, *
FROM (
SELECT DISTINCT emp.empid,emp.name,s.Salary,COUNT(emp.empid) OVER() TotalRows
FROM Employee emp
LEFT OUTER JOIN salary S ON emp.empid=S.empid
where emp.empid=12
) Q1
) Q2
WHERE ( #takeData<>0 AND RowNumber BETWEEN ((#currentPageNo * #takeData) + 1) AND ((#currentPageNo +1) * #takeData))

how to get certain sql results

i'm looking to get certain sql results from a query depending on where they are positioned, for example, consider this code
SELECT * FROM Product ORDER BY id asc
which could return at least 100 or so results.
the question is though, how can i get the first 1 - 10 results of that, and then in another different, separate query, how can i get the results that are 11 - 20 or even get the results that are positioned 51 - 60 of that query?
Use a CTE to get the row number and then query by the row column
with your_query as(
SELECT ROW_NUMBER() OVER(ORDER BY ID ASC) AS Row, *
FROM Product
)
select * from your_query
where Row >=5 and Row<=10
There are a number of ways, here's one approach using ROW_NUMBER:
DECLARE #StartRow INTEGER = 11
DECLARE #EndRow INTEGER = 20
;WITH Data AS
(
SELECT TOP(#EndRow) ROW_NUMBER() OVER (ORDER BY id) AS RowNo, *
FROM Product
)
SELECT *
FROM Data
WHERE RowNo BETWEEN #StartRow AND #EndRow
ORDER BY Id

Anyway to get a value similar to ##ROWCOUNT when TOP is used?

If I have a SQL statement such as:
SELECT TOP 5
*
FROM Person
WHERE Name LIKE 'Sm%'
ORDER BY ID DESC
PRINT ##ROWCOUNT
-- shows '5'
Is there anyway to get a value like ##ROWCOUNT that is the actual count of all of the rows that match the query without re-issuing the query again sans the TOP 5?
The actual problem is a much more complex and intensive query that performs beautifully since we can use TOP n or SET ROWCOUNT n but then we cannot get a total count which is required to display paging information in the UI correctly. Presently we have to re-issue the query with a #Count = COUNT(ID) instead of *.
Whilst this doesn't exactly meet your requirement (in that the total count isn't returned as a variable), it can be done in a single statement:
;WITH rowCTE
AS
(
SELECT *
,ROW_NUMBER() OVER (ORDER BY ID DESC) AS rn1
,ROW_NUMBER() OVER (ORDER BY ID ASC) AS rn2
FROM Person
WHERE Name LIKE 'Sm%'
)
SELECT *
,(rn1 + rn2) - 1 as totalCount
FROM rowCTE
WHERE rn1 <=5
The totalCount column will have the total number of rows matching the where filter.
It would be interesting to see how this stacks up performance-wise against two queries on a decent-sized data-set.
you'll have to run another COUNT() query:
SELECT TOP 5
*
FROM Person
WHERE Name LIKE 'Sm%'
ORDER BY ID DESC
DECLARE #r int
SELECT
#r=COUNT(*)
FROM Person
WHERE Name LIKE 'Sm%'
select #r
Something like this may do it:
SELECT TOP 5
*
FROM Person
cross join (select count(*) HowMany
from Person
WHERE Name LIKE 'Sm%') tot
WHERE Name LIKE 'Sm%'
ORDER BY ID DESC
The subquery returns one row with one column containing the full count; the cross join includes it with all rows returned by the "main" query"; and "SELECT *" would include new column HowMany.
Depending on your needs, the next step might be to filter out that column from your return set. One way would be to load the data from the query into a temp table, and then return just the desired columns, and get rowcount from the HowMany column from any row.

Getting a Subset of Records along with Total Record Count

I'm working on returning a recordset from SQL Server 2008 to do some pagination. I'm only returning 15 records at a time, but I need to have the total number of matches along with the subset of records. I've used two different queries with mixed results depending on where in the larger group I need to pull the subset. Here's a sample:
SET NOCOUNT ON;
WITH tempTable AS (
SELECT
FirstName
, LastName
, ROW_NUMBER() OVER(ORDER BY FirstName ASC) AS RowNumber
FROM People
WHERE
Active = 1
)
SELECT
tempTable.*
, (SELECT Max(RowNumber) FROM tempTable) AS Records
FROM tempTable
WHERE
RowNumber >= 1
AND RowNumber <= 15
ORDER BY
FirstName
This query works really fast when I'm returning items on the low end of matches, like records 1 through 15. However, when I start returning records 1000 - 1015, the processing will go from under a second to more than 15 seconds.
So I changed the query to the following instead:
SET NOCOUNT ON;
WITH tempTable AS (
SELECT * FROM (
SELECT
FirstName
, LastName
, ROW_NUMBER() OVER(ORDER BY FirstName ASC) AS RowNumber
, COUNT(*) OVER(PARTITION BY NULL) AS Records
FROM People
WHERE
Active = 1
) derived
WHERE RowNumber >= 1 AND RowNumber <= 15
)
SELECT
tempTable.*
FROM tempTable
ORDER BY
FirstName
That query runs the high number returns in 2-3 seconds, but also runs the low number queries in 2-3 seconds as well. Because it's doing the count for each of 70,000+ rows, it makes every request take longer instead of just the large row numbers.
So I need to figure out how to get a good row count, as well as only return a subset of items at any point in the resultset without suffering such a huge penalty. I could handle a 2-3 second penalty for the high row numbers, but 15 is too much, and I'm not willing to suffer slow loads on the first few pages a person views.
NOTE: I know that I don't need the CTE in the second example, but this is just a simple example. In production I'm doing further joins on the tempTable after I've filtered it down to the 15 rows I need.
Here is what I have done (and its just as fast, no matter which records I return):
--Parameters include:
#pageNum int = 1,
#pageSize int = 0,
DECLARE
#pageStart int,
#pageEnd int
SELECT
#pageStart = #pageSize * #pageNum - (#pageSize - 1),
#pageEnd = #pageSize * #pageNum;
SET NOCOUNT ON;
WITH tempTable AS (
SELECT
ROW_NUMBER() OVER (ORDER BY FirstName ASC) AS RowNumber,
FirstName
, LastName
FROM People
WHERE Active = 1
)
SELECT
(SELECT COUNT(*) FROM tempTable) AS TotalRows,
*
FROM tempTable
WHERE #pageEnd = 0
OR RowNumber BETWEEN #pageStart AND #pageEnd
ORDER BY RowNumber
I've handled a situation a bit similar to this in the past by not bothering to determine a definite row count, but using the query plan to give me an estimated row count, a bit like the first item in this link describes:
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=108658
The intention was then to deliver whatever rows have been asked for within the range (from say 900-915) and then returning the estimated row count, like
rows 900-915 of approx. 990
which avoided having to count all rows. Once the user moves beyond that point, I just showed
rows 1000-1015 of approx. 1015
i.e. just taking the last requested row as my new estimate.

How do I select last 5 rows in a table without sorting?

I want to select the last 5 records from a table in SQL Server without arranging the table in ascending or descending order.
This is just about the most bizarre query I've ever written, but I'm pretty sure it gets the "last 5" rows from a table without ordering:
select *
from issues
where issueid not in (
select top (
(select count(*) from issues) - 5
) issueid
from issues
)
Note that this makes use of SQL Server 2005's ability to pass a value into the "top" clause - it doesn't work on SQL Server 2000.
Suppose you have an index on id, this will be lightning fast:
SELECT * FROM [MyTable] WHERE [id] > (SELECT MAX([id]) - 5 FROM [MyTable])
The way your question is phrased makes it sound like you think you have to physically resort the data in the table in order to get it back in the order you want. If so, this is not the case, the ORDER BY clause exists for this purpose. The physical order in which the records are stored remains unchanged when using ORDER BY. The records are sorted in memory (or in temporary disk space) before they are returned.
Note that the order that records get returned is not guaranteed without using an ORDER BY clause. So, while any of the the suggestions here may work, there is no reason to think they will continue to work, nor can you prove that they work in all cases with your current database. This is by design - I am assuming it is to give the database engine the freedom do as it will with the records in order to obtain best performance in the case where there is no explicit order specified.
Assuming you wanted the last 5 records sorted by the field Name in ascending order, you could do something like this, which should work in either SQL 2000 or 2005:
select Name
from (
select top 5 Name
from MyTable
order by Name desc
) a
order by Name asc
You need to count number of rows inside table ( say we have 12 rows )
then subtract 5 rows from them ( we are now in 7 )
select * where index_column > 7
select * from users
where user_id >
( (select COUNT(*) from users) - 5)
you can order them ASC or DESC
But when using this code
select TOP 5 from users order by user_id DESC
it will not be ordered easily.
select * from table limit 5 offset (select count(*) from table) - 5;
Without an order, this is impossible. What defines the "bottom"? The following will select 5 rows according to how they are stored in the database.
SELECT TOP 5 * FROM [TableName]
Well, the "last five rows" are actually the last five rows depending on your clustered index. Your clustered index, by definition, is the way that he rows are ordered. So you really can't get the "last five rows" without some order. You can, however, get the last five rows as it pertains to the clustered index.
SELECT TOP 5 * FROM MyTable
ORDER BY MyCLusteredIndexColumn1, MyCLusteredIndexColumnq, ..., MyCLusteredIndexColumnN DESC
Search 5 records from last records you can use this,
SELECT *
FROM Table Name
WHERE ID <= IDENT_CURRENT('Table Name')
AND ID >= IDENT_CURRENT('Table Name') - 5
If you know how many rows there will be in total you can use the ROW_NUMBER() function.
Here's an examble from MSDN (http://msdn.microsoft.com/en-us/library/ms186734.aspx)
USE AdventureWorks;
GO
WITH OrderedOrders AS
(
SELECT SalesOrderID, OrderDate,
ROW_NUMBER() OVER (ORDER BY OrderDate) AS 'RowNumber'
FROM Sales.SalesOrderHeader
)
SELECT *
FROM OrderedOrders
WHERE RowNumber BETWEEN 50 AND 60;
In SQL Server 2012 you can do this :
Declare #Count1 int ;
Select #Count1 = Count(*)
FROM [Log] AS L
SELECT
*
FROM [Log] AS L
ORDER BY L.id
OFFSET #Count - 5 ROWS
FETCH NEXT 5 ROWS ONLY;
Try this, if you don't have a primary key or identical column:
select [Stu_Id],[Student_Name] ,[City] ,[Registered],
RowNum = row_number() OVER (ORDER BY (SELECT 0))
from student
ORDER BY RowNum desc
You can retrieve them from memory.
So first you get the rows in a DataSet, and then get the last 5 out of the DataSet.
There is a handy trick that works in some databases for ordering in database order,
SELECT * FROM TableName ORDER BY true
Apparently, this can work in conjunction with any of the other suggestions posted here to leave the results in "order they came out of the database" order, which in some databases, is the order they were last modified in.
select *
from table
order by empno(primary key) desc
fetch first 5 rows only
Last 5 rows retrieve in mysql
This query working perfectly
SELECT * FROM (SELECT * FROM recharge ORDER BY sno DESC LIMIT 5)sub ORDER BY sno ASC
or
select sno from(select sno from recharge order by sno desc limit 5) as t where t.sno order by t.sno asc
When number of rows in table is less than 5 the answers of Matt Hamilton and msuvajac is Incorrect.
Because a TOP N rowcount value may not be negative.
A great example can be found Here.
i am using this code:
select * from tweets where placeID = '$placeID' and id > (
(select count(*) from tweets where placeID = '$placeID')-2)
In SQL Server, it does not seem possible without using ordering in the query.
This is what I have used.
SELECT *
FROM
(
SELECT TOP 5 *
FROM [MyTable]
ORDER BY Id DESC /*Primary Key*/
) AS T
ORDER BY T.Id ASC; /*Primary Key*/
DECLARE #MYVAR NVARCHAR(100)
DECLARE #step int
SET #step = 0;
DECLARE MYTESTCURSOR CURSOR
DYNAMIC
FOR
SELECT col FROM [dbo].[table]
OPEN MYTESTCURSOR
FETCH LAST FROM MYTESTCURSOR INTO #MYVAR
print #MYVAR;
WHILE #step < 10
BEGIN
FETCH PRIOR FROM MYTESTCURSOR INTO #MYVAR
print #MYVAR;
SET #step = #step + 1;
END
CLOSE MYTESTCURSOR
DEALLOCATE MYTESTCURSOR
Thanks to #Apps Tawale , Based on his answer, here's a bit of another (my) version,
To select last 5 records without an identity column,
select top 5 *,
RowNum = row_number() OVER (ORDER BY (SELECT 0))
from [dbo].[ViewEmployeeMaster]
ORDER BY RowNum desc
Nevertheless, it has an order by, but on RowNum :)
Note(1): The above query will reverse the order of what we get when we run the main select query.
So to maintain the order, we can slightly go like:
select *, RowNum2 = row_number() OVER (ORDER BY (SELECT 0))
from (
select top 5 *, RowNum = row_number() OVER (ORDER BY (SELECT 0))
from [dbo].[ViewEmployeeMaster]
ORDER BY RowNum desc
) as t1
order by RowNum2 desc
Note(2): Without an identity column, the query takes a bit of time in case of large data
Get the count of that table
select count(*) from TABLE
select top count * from TABLE where 'primary key row' NOT IN (select top (count-5) 'primary key row' from TABLE)
If you do not want to arrange the table in ascending or descending order. Use this.
select * from table limit 5 offset (select count(*) from table) - 5;

Resources