I'm working on a sql query that is passed a list of values as a parameter, like
select *
from ProductGroups
where GroupID in (24,12,7,14,65)
This list is constructed of relations used througout the database, and must be kept in this order.
I would like to order the results by this list. I only need the first result, but it could be the one with GroupId 7 in this case.
I can't query like
order by (24,12,7,14,65).indexOf(GroupId)
Does anyone know how to do this?
Additional info:
Building a join works and running it in the mssql query editor, but...
Due to limitiations of the software sending the query to mssql, I have to pass it to some internal query builder as 1 parameter, thus "24,12,7,14,65". And I don't know upfront how many numbers there will be in this list, could be 2, could be 20.
You can also order by on a CASE:
select *
from ProductGroups
where GroupID in (24,12,7,14,65)
order by case GroupId
when 7 then 1 -- First in ordering
when 14 then 2 -- Second
else 3
end
Use a table variable or temporary table with an identity column, feed in your values and join to that, e.g.
declare #rank table (
ordering int identity(1,1)
, number int
)
insert into #rank values (24)
insert into #rank values (12)
insert into #rank values (7)
insert into #rank values (14)
insert into #rank values (65)
select pg.*
from ProductGroups pg
left outer join
#rank r
on pg.GroupId = r.number
order by
r.ordering
I think I might have found a possible solution (but it's ugly):
select *
from ProductGroups
where GroupID in (24,12,7,14,65)
order by charindex(
','+cast(GroupID as varchar)+',' ,
','+'24,12,7,14,65'+',')
this will order the rows by the position they occur in the list. And I can pass the string like I need too.
Do a join with a temporary table, in which you have the values that you want to filter by as rows. Add a column to it that has the order that you want as the second column, and sort by it.
Related
I'm trying to move away from OFFSET/FETCH pagination to Keyset Pagination (also known as Seek Method). Since I'm just started, there are many questions I have in my mind but this is one of many where I try to get the pagination right along with Filter.
So I have 2 tables
aspnet_users
having columns
PK
UserId uniquidentifier
Fields
UserName NVARCHAR(256) NOT NULL,
AffiliateTag varchar(50) NULL
.....other fields
aspnet_membership
having columns
PK+FK
UserId uniquidentifier
Fields
Email NVARCHAR(256) NOT NULL
.....other fields
Indexes
Non Clustered Index on Table aspnet_users (UserName)
Non Clustered Index on Table aspnet_users (AffiliateTag)
Non Clustered Index on Table aspnet_membership(Email)
I have a page that will list the users (based on search term) with page size set to 20. And I want to search across multiple columns so instead of doing OR I find out having a separate query for each and then Union them will make the index use correctly.
so have the stored proc that will take search term and optionally UserName and UserId of last record for next page.
Create proc [dbo].[sp_searchuser]
#take int,
#searchTerm nvarchar(max) NULL,
#lastUserName nvarchar(256)=NULL,
#lastUserId nvarchar(256)=NULL
AS
IF(#lastUserName IS NOT NULL AND #lastUserId IS NOT NULL)
Begin
select top (#take) *
from
(
select u.UserId, u.UserName, u.AffiliateTag, m.Email
from aspnet_Users as u
inner join aspnet_Membership as m
on u.UserId=m.UserId
where u.UserName like #searchTerm
UNION
select u.UserId, u.UserName, u.AffiliateTag, m.Email
from aspnet_Users as u
inner join aspnet_Membership as m
on u.UserId=m.UserId
where u.AffiliateTag like convert(varchar(50), #searchTerm)
) as u1
where u1.UserName > #lastUserName
OR (u1.UserName=#lastUserName And u1.UserId > convert(uniqueidentifier, #lastUserId))
order by u1.UserName
End
Else
Begin
select top (#take) *
from
(
select u.UserId, u.UserName, u.AffiliateTag, m.Email
from aspnet_Users as u
inner join aspnet_Membership as m
on u.UserId=m.UserId
where u.UserName like #searchTerm
UNION
select u.UserId, u.UserName, u.AffiliateTag, m.Email
from aspnet_Users as u
inner join aspnet_Membership as m
on u.UserId=m.UserId
where u.AffiliateTag like convert(varchar(50), #searchTerm)
) as u1
order by u1.UserName
End
Now to get the result for first page with search term mua
exec [sp_searchuser] 20, 'mua%'
it uses both indexes created one for UserName column and another for AffiliateTag column which is good
But the problem is I find the inner union queries return all the matching rows
like in this case, the execution plan shows
UserName Like SubQuery
Number of Rows Read= 5
Actual Number of Rows= 4
AffiliateTag Like SubQuery
Number of Rows Read= 465
Actual Number of Rows= 465
so in total inner queries return 469 matching rows
and then outer query take out 20 for final result reset. So really reading more data than needed.
And when go to next page
exec [sp_searchuser] 20, 'mua%', 'lastUserName', 'lastUserId'
the execution plan shows
UserName Like SubQuery
Number of Rows Read= 5
Actual Number of Rows= 4
AffiliateTag Like SubQuery
Number of Rows Read= 465
Actual Number of Rows= 445
in total inner queries return 449 matching rows
so either with or without pagination, it reads more data than needed.
My expectation is to somehow limit the inner queries so it does not return all matching rows.
You might be interested in the Logical Processing Order, which determines when the objects defined in one step are made available to the clauses in subsequent steps. The Logical Processing Order steps are:
FROM
ON
JOIN
WHERE
GROUP BY
WITH CUBE or WITH ROLLUP
HAVING
SELECT
DISTINCT
ORDER BY
TOP
Of course, as noted the docs:
The actual physical execution of the statement is determined by the
query processor and the order may vary from this list.
meaning that sometimes some statements can start before previous complete.
In your case, you query looks like:
some data extraction
sort by user_name
get TOP records
There is no way to reduce the rows in the data extraction part as to have a deterministic result (we actually may need to order by user_name, user_id to have such) we need to get all matching rows, sort them and then get the desired rows.
For example, image the first query returning 20 names starting with 'Z'. And the second query to returned only one name starting with 'A'. If you stop somehow the execution and skip the second query, you will get wrong results - 20 names starting with 'Z' instead one starting with 'A' and 19 with 'Z'.
In such cases, I prefer to use dynamic T-SQL statements in order to get better execution times and reduce the code length. You are saying:
And I want to search across multiple columns so instead of doing OR I
find out having a separate query for each and then Union them will
make the index use correctly.
When you are using UNION you are performing double reads to your tables. In your cases, you are reading the aspnet_Membership table twice and the aspnet_Users twice (yes, here you are using two different indexes but I believe they are not covering and you end up performing look ups to extract the users name and email.
I guess you have started with covering indexed like in the example below:
DROP TABLE IF EXISTS [dbo].[StackOverflow];
CREATE TABLE [dbo].[StackOverflow]
(
[UserID] INT PRIMARY KEY
,[UserName] NVARCHAR(128)
,[AffiliateTag] NVARCHAR(128)
,[UserEmail] NVARCHAR(128)
,[a] INT
,[b] INT
,[c] INT
,[z] INT
);
CREATE INDEX IX_StackOverflow_UserID_UserName_AffiliateTag_I_UserEmail ON [dbo].[StackOverflow]
(
[UserID]
,[UserName]
,[AffiliateTag]
)
INCLUDE ([UserEmail]);
GO
INSERT INTO [dbo].[StackOverflow] ([UserID], [UserName], [AffiliateTag], [UserEmail])
SELECT TOP (1000000) ROW_NUMBER() OVER(ORDER BY t1.number)
,CONCAT('UserName',ROW_NUMBER() OVER(ORDER BY t1.number))
,CONCAT('AffiliateTag', ROW_NUMBER() OVER(ORDER BY t1.number))
,CONCAT('UserEmail', ROW_NUMBER() OVER(ORDER BY t1.number))
FROM master..spt_values t1
CROSS JOIN master..spt_values t2;
GO
So, for the following query:
SELECT TOP 20 [UserID]
,[UserName]
,[AffiliateTag]
,[UserEmail]
FROM [dbo].[StackOverflow]
WHERE [UserName] LIKE 'UserName200%'
OR [AffiliateTag] LIKE 'UserName200%'
ORDER BY [UserName];
GO
The issue here is we are reading all the rows even we are using the index.
What's good is that the index is covering and we are not performing look ups. Depending on the search criteria it may perform better than your approach.
If the performance is bad, we can use a trigger to UNPIVOT the original data and record in a separate table. It may look like this (it will be better to use attribute_id rather than the text like me):
DROP TABLE IF EXISTS [dbo].[StackOverflowAttributes];
CREATE TABLE [dbo].[StackOverflowAttributes]
(
[UserID] INT
,[AttributeName] NVARCHAR(128)
,[AttributeValue] NVARCHAR(128)
,PRIMARY KEY([UserID], [AttributeName], [AttributeValue])
);
GO
CREATE INDEX IX_StackOverflowAttributes_AttributeValue ON [dbo].[StackOverflowAttributes]
(
[AttributeValue]
)
INSERT INTO [dbo].[StackOverflowAttributes] ([UserID], [AttributeName], [AttributeValue])
SELECT [UserID]
,'Name'
,[UserName]
FROM [dbo].[StackOverflow]
UNION
SELECT [UserID]
,'AffiliateTag'
,[AffiliateTag]
FROM [dbo].[StackOverflow];
and the query before will looks like:
SELECT TOP 20 U.[UserID]
,U.[UserName]
,U.[AffiliateTag]
,U.[UserEmail]
FROM [dbo].[StackOverflowAttributes] A
INNER JOIN [dbo].[StackOverflow] U
ON A.[UserID] = U.[UserID]
WHERE A.[AttributeValue] LIKE 'UserName200%'
ORDER BY U.[UserName];
Now, we are reading only a part of the the index rows and after that performing a lookup.
In order to compare performance it will be better to use:
SET STATISTICS IO, TIME ON;
as it will give you how pages are read from the indexes. The result can be visualized here.
I have a simple query that is taking in an XML string and inserting one or more rows of data depending on what was selected on the UI.
There is a field on the records that determine their order and by default, these new fields are added to the end of the table.
If 3 records exist with the OrderID of 1,2,3 respectively and I add 3 more records through the XML string, those records should get 4,5,6 as their orderID. It doesn't matter which 3 values in the new set get which order, just that they continue the incrementation.
Here was my attempt, which works fine if there is a single item but if there are multiple, #newOrder is always the same so the count is invalid with #newOrder+1.
-- Get the next order based on the last entry
DECLARE #newOrder INT = (SELECT TOP 1 DepartmentOrder FROM c.department WHERE dID = #dID ORDER BY DepartmentOrder DESC)
-- Add new department
INSERT INTO c.department (dID,DepartmentID,DepartmentOrder)
SELECT #dID,
ParamValues.x1.value('departmentID[1]', 'int'),
ISNULL(#newOrder,0)+1
FROM #departments.nodes('/departments/department') AS ParamValues(x1)
Any thoughts?
You can use ROW_NUMBER to generate the new values:
INSERT INTO c.department (dID,DepartmentID,DepartmentOrder)
SELECT #dID,
ParamValues.x1.value('departmentID[1]', 'int'),
ISNULL(#newOrder,0)+ROW_NUMBER() OVER (ORDER BY ParamValues.x1)
FROM #departments.nodes('/departments/department') AS ParamValues(x1)
By using the XML column x1 in the ORDER BY clause, you are specifying that the DepartmentOrder should be ordered the same way the nodes appear in the xml.
I am doing a bulk insert into a table using SELECT and UNION. I need the order of the SELECT values to be unchanged when calling the INSERT, but it seems that the values are being inserted in an ascending order, rather than the order I specify.
For example, the below insert statement
declare #QuestionOptionMapping table
(
[ID] [int] IDENTITY(1,1)
, [QuestionOptionID] int
, [RateCode] varchar(50)
)
insert into #QuestionOptionMapping (
RateCode
)
select
'PD0116'
union
select
'PL0090'
union
select
'PL0091'
union
select
'DD0026'
union
select
'DD0025'
SELECT * FROM #QuestionOptionMapping
renders the data as
(5 row(s) affected)
ID QuestionOptionID RateCode
----------- ---------------- --------------------------------------------------
1 NULL DD0025
2 NULL DD0026
3 NULL PD0116
4 NULL PL0090
5 NULL PL0091
(5 row(s) affected)
How can the select of the inserted data return the same order as when it was inserted?
SQL Server stores your rows as an unordered set. The data points may or may not be contiguous, and they may or may not be in the "order" the data was specified in your insert statements.
When you query the data, the engine will retrieve the rows in the most efficient order, as determined by the optimizer. There is no guarantee that the order will be the same every time you query the data.
The only way to guarantee the order of your result set is to include an explicit ORDER BY clause with your SELECT statement.
See this answer for a much more in depth discussion as to why this the case. Default row order in SELECT query - SQL Server 2008 vs SQL 2012
By using the SELECT/UNION option for your INSERT statement, you're creating an unordered set that SQL Server ingests as a set, not as a series of inputs. Separate your inserts into discrete statements if you need them to have the IDENTITY values applied in order. Better yet, if the row numbering matters, don't leave it to chance. Explicitly number the rows on insert.
SQL tables do represent unordered sets. However, the identity column on an insert will follow the ordering of the order by.
Your data is getting out of order because of the duplicate elimination in the union. However, I would suggest writing the query to explicitly sort the data:
insert into #QuestionOptionMapping (RateCode)
select ratecode
from (values (1, 'PD0116'),
(2, 'PL0090'),
(3, 'PL0091'),
(4, 'DD0026'),
(5, 'DD0025')
) v(ord, ratecode)
order by ord;
Then be sure to use order by for the select:
select qom.*
from #QuestionOptionMapping qom
order by id;
Note that this also uses the values() table constructor, which is a very handy syntax.
If you're not selecting from tables?
Then you could insert VALUES, instead of a select with unions.
insert into #QuestionOptionMapping (RateCode) values
('PD0116')
,('PL0090')
,('PL0091')
,('DD0026')
,('DD0025')
Or in your query, change all the UNION to UNION ALL.
The difference between a UNION and a UNION ALL is that a UNION will remove duplicate rows.
While UNION ALL just stiches the resultsets from the selects together.
And for UNION to find those duplicates, internally it first has to sort them.
But a UNION ALL doesn't care about uniqueness, so it doesn't need to sort.
A 3th option would be to simply change from 1 insert statement to multiple insert statements.
One insert per value. Thus avoiding UNION completely.
But that anti-golfcoding method is also the most wordy.
Your problem is you are not putting them in in the order you think. UNION is distinct values only and it will typically sort the values to facilitate the distinct. Run the select statement alone and you will see.
If you insert using values then order is preserved:
insert into #QuestionOptionMapping (RateCode) values
('PD0116'), ('PL0090'), ('PL0091'), ('DD0026'), ('DD0025')
select * from #QuestionOptionMapping order by ID
I want to obtain one single row but it is returning 3 rows. The form_audit table has 3 rows with the same REF_NO,
How to get one distinct row?
Hope this will give you some idea how it works
CREATE TABLE #tblBackers (
amountBacked MONEY,
backersAccountID INT,
playerBacked INT,
Dates DATETIME)
INSERT INTO #tblBackers VALUES (25,12345,99999,GETDATE())
INSERT INTO #tblBackers VALUES (25,12345,99999,GETDATE())
INSERT INTO #tblBackers VALUES (25,12345,99699,GETDATE())
INSERT INTO #tblBackers VALUES (25,12345,99999,GETDATE())
INSERT INTO #tblBackers VALUES (25,98765,88888,GETDATE())
INSERT INTO #tblBackers VALUES (25,76543,77777,GETDATE())
GO
SELECT DISTINCT * FROM #tblBackers
SELECT DISTINCT TOP 1 * FROM #tblBackers
And use the ORDER BY to get the latest record.
If you only want one record per ref_no, then consider adding a group by clause on that field.
select
fa.ref_no
/*, other stuff*/
from
FORM_AUDIT fa
/* other joins*/
group by
fa.ref_no;
Keep in mind that this group by clause will aggregate all records that share the same ref_no into a single record in the result set. That means that you can no longer include fields like fh.* and fcd.* in the select list directly, because you have no guarantee that each of those fields has only one value per row in your result set. For every such field that you want to include in your select list, you must either:
Include that field in your group by clause, keeping in mind that doing so will no longer necessarily give you exactly one row per distinct ref_no; now you'll get one row per distinct combination of ref_no and whatever else you add to the group by clause, or
Use one of SQL Server's aggregate functions to transform the set of zero-to-many values in the field you're adding into a single value. Aggregate functions are things like max(), sum(), count(), etc. There's a complete list at the link.
Good luck!
Using SSRS with SQL Server 2008 R2 (Visual Studio environment).
I am trying to produce a stepped down report based on a level/value in a table on sql server. The level act as a indent position with sort_value been the recursive parent in the report.
Sample of table in SQL Server:
Sample of output required
OK, I've come up with a solution but please note the following before you proceed.
1. The process relies on the data being in the correct order, as per your sample data.
2. If this is your real data structure, I strongly recommend you review it.
OK, So the first things I did was recreate your table exactly as per example. I called the table Stepped as I couldn't think of anything else!
The following code can then be used as your dataset in SSRS but you can obviously just run the T-SQL directly to see the output.
-- Create a copy of the data with a row number. This means the input data MUST be in the correct order.
DECLARE #t TABLE(RowN int IDENTITY(1,1), Sort_Order int, [Level] int, Qty int, Currency varchar(20), Product varchar(20))
INSERT INTO #t (Sort_Order, [Level], Qty, Currency, Product)
SELECT * FROM Stepped
-- Update the table so each row where the sort_order is NULL will take the sort order from the row above
UPDATE a SET Sort_Order = b.Sort_Order
FROM #t a
JOIN #t b on a.RowN = b.rowN+1
WHERE a.Sort_Order is null and b.Sort_Order is not null
-- repeat this until we're done.
WHILE ##ROWCOUNT >0
BEGIN
UPDATE a SET Sort_Order = b.Sort_Order
FROM #t a
JOIN #t b on a.RowN = b.rowN+1
WHERE a.Sort_Order is null and b.Sort_Order is not null
END
-- Now we can select from our new table sorted by both sort oder and level.
-- We also separate out the products based on their level.
SELECT
CASE Level WHEN 1 THEN Product ELSE NULL END as ProdLvl_1
, CASE Level WHEN 2 THEN Product ELSE NULL END as ProdLvl_2
, CASE Level WHEN 3 THEN Product ELSE NULL END as ProdLvl_3
, QTY
, Currency
FROM #t s
ORDER BY Sort_Order, Level
The output looks like this...
You may also want to consider swapping out the final statement for this.
-- Alternatively use this style and use a single column in the report.
-- This is better if the number of levels can change.
SELECT
REPLICATE('--', Level-1) + Product as Product
, QTY
, Currency
FROM #t s
ORDER BY Sort_Order, Level
As this will give you a single column for 'product' indented like this.