How can i search for the same input parameter - sql-server

My input parameter is :
exec [dbo].[MULTI_VALUED_USER_INPUT_PARAMETER] N'2040,2041,2044,2047,2048'
I need to search in a data base where i have to search in those inputs like this
Product IN ('2040','2041','2044','2047','204048')
How can i convert the input parameter to put it inside the " Product" search one at a time
UPDATE
I need to split the input to be list of strings to be able to search in them

You can use a dynamic query and pass the IDs in the query directly but this option is vulnerable to SQL Injection. Therefore, you should split up the IDs first and then join it with your actual table. Your procedure should be like this:
CREATE PROCEDURE [MULTI_VALUED_USER_INPUT_PARAMETER] #Ids NVARCHAR(1000)
AS
--DECLARE #Ids NVARCHAR(1000) = N'2040,2041,2044,2047,2048'
;WITH CTE
AS(
SELECT CAST('<N>' + REPLACE(#Ids,',','</N><N>') + '</N>' AS XML) [XMLids]
)
SELECT T.a.value('.', 'INT') AS ID,yt.*
FROM CTE
CROSS APPLY CTE.XMLids.nodes('/N') AS T(a)
INNER JOIN dbo.YourTable yt
ON T.a.value('.', 'INT') = yt.Product
NOTE: I assumed that the IDs are INT by default, if not, please update the code. You can replace the JOIN with IN clause easily if required.

you are passing a string like this
N'2040,2041,2044,2047,2048'
and you are expecting it to be parsed as comma seperated values..so you need to parse the input first..
you could use split string available from here and do the parse like below in your proc..
select * from split_String(#String,',')
and pass the above input in the place where you are passing string in stored proc
split string source code:
CREATE FUNCTION dbo.SplitStrings_Numbers
(
#List NVARCHAR(MAX),
#Delimiter NVARCHAR(255)
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
SELECT Item = SUBSTRING(#List, Number,
CHARINDEX(#Delimiter, #List + #Delimiter, Number) - Number)
FROM dbo.Numbers
WHERE Number <= CONVERT(INT, LEN(#List))
AND SUBSTRING(#Delimiter + #List, Number, LEN(#Delimiter)) = #Delimiter
);
GO
Numbers table :
CREATE TABLE Numbers
(
Number INT NOT NULL,
CONSTRAINT PK_Numbers
PRIMARY KEY CLUSTERED (Number)
WITH FILLFACTOR = 100
)
INSERT INTO Numbers
SELECT
(a.Number * 256) + b.Number AS Number
FROM
(
SELECT number
FROM master..spt_values
WHERE
type = 'P'
AND number <= 255
) a (Number),
(
SELECT number
FROM master..spt_values
WHERE
type = 'P'
AND number <= 255
) b (Number)
GO

What I understood is that you want to execute that store procedure with different parameters, which have to be included in the IN clause you wrote.
You can do this:
DECLARE #Product INT = 0
-- Iterate over all products
WHILE ( 1 = 1)
BEGIN
-- Get next productID
--Since you didn't show us how is your product tables I had to make up a little,
SELECT TOP 1 #Product = Product
FROM Products
WHERE Product > #Product
ORDER BY Product
-- Exit loop if no more products
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC [dbo].[MULTI_VALUED_USER_INPUT_PARAMETER] #Product
END

Related

Substring is slow with while loop in SQL Server

One of my table column stores ~650,000 characters (each value of the column contains entire table). I know its bad design however, Client will not be able to change it.
I am tasked to convert the column into multiple columns.
I chose to use dbo.DelimitedSplit8K function
Unfortunately, it can only handle 8k characters at max.
So I decided to split the column into 81 8k batches using while loop and store the same in a variable table (temp or normal table made no improvement)
DECLARE #tab1 table ( serialnumber int, etext nvarchar(1000))
declare #scriptquan int = (select MAX(len (errortext)/8000) from mytable)
DECLARE #Counter INT
DECLARE #A bigint = 1
DECLARE #B bigint = 8000
SET #Counter=1
WHILE ( #Counter <= #scriptquan + 1)
BEGIN
insert into #tab1 select ItemNumber, Item from dbo.mytable cross apply dbo.DelimitedSplit8K(substring(errortext, #A, #B), CHAR(13)+CHAR(10))
SET #A = #A + 8000
SET #B = #B + 8000
SET #Counter = #Counter + 1
END
This followed by using below code
declare #tab2 table (Item nvarchar(max),itemnumber int, Colseq varchar(10)) -- declare table variable
;with cte as (
select [etext] ,ItemNumber, Item from #tab1 -- insert table name
cross apply dbo.DelimitedSplit8K(etext,' ')) -- insert table columns name that contains text
insert into #tab2 Select Item,itemnumber, 'a'+ cast (ItemNumber as varchar) colseq
from cte -- insert values to table variable
;WITH Tbl(item, colseq) AS(
select item, colseq from #tab2
),
CteRn AS(
SELECT item, colseq,
Rn = ROW_NUMBER() OVER(PARTITION BY colseq ORDER BY colseq)
FROM Tbl
)
SELECT
a1 Time,a2 Number,a3 Type,a4 Remarks
FROM CteRn r
PIVOT(
MAX(item)
FOR colseq IN(a1,a2,a3,a4)
)p
where a3 = 'error'
gives the desired output. However, just the loop takes 15 minutes to complete and overall query completes by 27 minutes. Is there any way I can make it faster? Total row count in my table is 2. So I don't think Index can help.
Client uses Azure SQL Database so I can't choose PowerShell or Python to accomplish this either.
Please let me know if more information is needed. I tried my best to mention everything I could.

Is it possible to iterate over a list of numbers in tsql?

I have a comma separated list of personIDs: 1265, 8632.
What I want to do is something I'd like to indicate by a chunk of pseudo-code, which does not work, of course.
declare #max int = (select count(*) from dbo.tablePersons)-1
declare #cnt = 0
BEGIN
SELECT personID FROM dbo.tablePersons
WHERE (1265, 8632)[#cnt]
IS NOT IN SELECT personID FROM dbo.tablePersons
SET #cnt = #cnt + 1
END
I want to iterate over the list 1265, 8632 and check whether the IDs in the list are NOT there in SELECT personID FROM dbo.tablePersons. The goal is to find all those IDs in my list that are there in SELECT personID FROM dbo.tablePersons.
Given the fact that this pseudo-code is not viable -is there a kind of a workaround?
You can try this:
DECLARE #mockperson TABLE(ID INT, PersName VARCHAR(100));
INSERT INTO #mockperson VALUES
(1,'pers 1')
,(2,'pers 2');
DECLARE #yourlist VARCHAR(100)='1,999,2';
--The query splits the given list and checks for numbers NOT IN the person table
WITH Casted AS
(
SELECT CAST('<x>' + REPLACE(#yourlist,',','</x><x>') + '</x>' AS XML) AS TheListAsXml
)
SELECT x.value('text()[1]','int')
FROM Casted
CROSS APPLY TheListAsXml.nodes('/x') AS A(x)
WHERE x.value('text()[1]','int') NOT IN(SELECT ID FROM #mockperson);
Starting with v2016
With STRING_SPLIT() (v2016+) this would be the same approach, just easier
SELECT *
FROM STRING_SPLIT(#yourlist,',') AS A
WHERE A.value NOT IN(SELECT ID FROM #mockperson);

Counting special characters in SQL

Looking for a SQL query that returns the count of all the special characters used in a particular column. Suppose in a column there are 5 rows and each uses two special characters. I am looking for some query that gives 10 as the result.
I tried to get Special characters rows, but how to get counts of each special characters for all databases in same table ( I have 10 databases in same table and same column )
Select * From tablename Where Name like '%[^a-z ,-.''0-9]%'
So I use a numbers table (virtual created in the CTE) and cross apply with your table taking each character 1 at a time and seeing if it matches your match criteria. I then count the number of rows returned to get the number of special characters:
SET UP
--Create Test Table
CREATE TABLE #Test
(
[Name] VARCHAR(50)
)
-- Insert Sample Data
INSERT INTO #Test
VALUES ('dgas-!gfsdfg'), ('d^0jdn'), (',dfsd%gfs_da')
Query
declare #match varchar(50) = '[^a-z ,-.''0-9]'
;WITH Numbers
AS
(
-- Generate Numbers table
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS N
FROM
(VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10)) x(n)
CROSS APPLY
(VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10)) y(n)
)
SELECT COUNT(*)
FROM Numbers
CROSS APPLY #Test
WHERE N < DATALENGTH(Name) + 1 -- Only need to get each character
AND SUBSTRING(Name, N, 1) LIKE #match -- does it match our expression
For my data his returns 4

Create a comma separated string with numbers 1 to x, where x is read from the record

I have a table document with a field steps. This is an integer field and can contain a number between 1 and 1000.
Now a new field is added (followedsteps) which must contain the numbers from 1 to [the number from field steps], comma separated.
So when the field steps contains the number 5, I want this string 1,2,3,4,5 to be set in the new column followedsteps.
The field steps is not null-able, lowest value is 1.
Is there an (easy) way to do this?
It's a one time migration.
As you are going to perform this only one time, it will be better to generate first the sequences:
IF OBJECT_ID('tempdb..#DataSource') IS NOT NULL
BEGIN;
DROP TABLE #DataSource;
END;
CREATE TABLE #DataSource
(
[ID] INT
,[Sequence] VARCHAR(MAX)
);
DECLARE #MaximumID INT = 1000; -- in your case: SELECT MAX(steps) FROM document
WITH DataSource AS
(
SELECT 1 AS num
UNION ALL
SELECT num+1
FROM DataSource
WHERE num+1<=#MaximumID
)
INSERT INTO #DataSource
SELECT A.[num]
,DS.[Sequence]
FROM DataSource A
CROSS APPLY
(
SELECT STUFF
(
(
SELECT ',' + CAST(B.[num] AS VARCHAR(12))
FROM DataSource B
WHERE A.[num] >= B.[num]
ORDER BY B.[num]
FOR XML PATH(''), TYPE
).value('.', 'VARCHAR(MAX)')
,1
,1
,''
)
) DS ([Sequence])
option (maxrecursion 32767)
The code above creates a temporary table with data you need to perform the update:
Then in transaction, perform the update by [ID]:
BEGIN TRAN
UPDATE document
SET followedsteps = [Sequence]
FROM document A
INNER JOIN ##DataSource B
ON A.[steps] = b.[id]
COMMIT TRAN

SQL Server query with pagination and count

I want to make a database query with pagination. So, I used a common-table expression and a ranked function to achieve this. Look at the example below.
declare #table table (name varchar(30));
insert into #table values ('Jeanna Hackman');
insert into #table values ('Han Fackler');
insert into #table values ('Tiera Wetherbee');
insert into #table values ('Hilario Mccray');
insert into #table values ('Mariela Edinger');
insert into #table values ('Darla Tremble');
insert into #table values ('Mammie Cicero');
insert into #table values ('Raisa Harbour');
insert into #table values ('Nicholas Blass');
insert into #table values ('Heather Hayashi');
declare #pagenumber int = 2;
declare #pagesize int = 3;
declare #total int;
with query as
(
select name, ROW_NUMBER() OVER(ORDER BY name ASC) as line from #table
)
select top (#pagesize) name from query
where line > (#pagenumber - 1) * #pagesize
Here, I can specify the #pagesize and #pagenumber variables to give me just the records that I want. However, this example (that comes from a stored procedure) is used to make a grid pagination in a web application. This web application requires to show the page numbers. For instance, if a have 12 records in the database and the page size is 3, then I'll have to show 4 links, each one representing a page.
But I can't do this without knowing how many records are there, and this example just gives me the subset of records.
Then I changed the stored procedure to return the count(*).
declare #pagenumber int = 2;
declare #pagesize int = 3;
declare #total int;
with query as
(
select name, ROW_NUMBER() OVER(ORDER BY name ASC) as line, total = count(*) over()from #table
)
select top (#pagesize) name, total from query
where line > (#pagenumber - 1) * #pagesize
So, along with each line, it will show the total number of records. But I didn't like it.
My question is if there's a better way (performance) to do this, maybe setting the #total variable without returning this information in the SELECT. Or is this total column something that won't harm the performance too much?
Thanks
Assuming you are using MSSQL 2012, you can use Offset and Fetch which cleans up server-side paging greatly. We've found performance is fine, and in most cases better. As far as getting the total column count, just use the window function below inline...it will not include the limits imposed by 'offset' and 'fetch'.
For Row_Number, you can use window functions the way you did, but I would recommend that you calculate that client side as (pagenumber*pagesize + resultsetRowNumber), so if you're on the 5th page of 10 results and on the third row you would output row 53.
When applied to an Orders table with about 2 million orders, I found the following:
FAST VERSION
This ran in under a second. The nice thing about it is that you can do your filtering in the common table expression once and it applies both to the paging process and the count. When you have many predicates in the where clause, this keeps things simple.
declare #skipRows int = 25,
#takeRows int = 100,
#count int = 0
;WITH Orders_cte AS (
SELECT OrderID
FROM dbo.Orders
)
SELECT
OrderID,
tCountOrders.CountOrders AS TotalRows
FROM Orders_cte
CROSS JOIN (SELECT Count(*) AS CountOrders FROM Orders_cte) AS tCountOrders
ORDER BY OrderID
OFFSET #skipRows ROWS
FETCH NEXT #takeRows ROWS ONLY;
SLOW VERSION
This took about 10 sec, and it was the Count(*) that caused the slowness. I'm surprised this is so slow, but I suspect it's simply calculating the total for each row. It's very clean though.
declare #skipRows int = 25,
#takeRows int = 100,
#count int = 0
SELECT
OrderID,
Count(*) Over() AS TotalRows
FROM Location.Orders
ORDER BY OrderID
OFFSET #skipRows ROWS
FETCH NEXT #takeRows ROWS ONLY;
CONCLUSION
We've gone through this performance tuning process before and actually found that it depended on the query, predicates used, and indexes involved. For instance, the second we introduced a view it chugged, so we actually query off the base table and then join up the view (which includes the base table) and it actually performs very well.
I would suggest having a couple of straight-forward strategies and applying them to high-value queries that are chugging.
DECLARE #pageNumber INT = 1 ,
#RowsPerPage INT = 20
SELECT *
FROM TableName
ORDER BY Id
OFFSET ( ( #pageNumber - 1 ) * #RowsPerPage ) ROWS
FETCH NEXT #RowsPerPage ROWS ONLY;
What if you calculate the count beforehand?
declare #pagenumber int = 2;
declare #pagesize int = 3;
declare #total int;
SELECT #total = count(*)
FROM #table
with query as
(
select name, ROW_NUMBER() OVER(ORDER BY name ASC) as line from #table
)
select top (#pagesize) name, #total total from query
where line > (#pagenumber - 1) * #pagesize
Another way, is to calculate max(line). Check the link
Return total records from SQL Server when using ROW_NUMBER
UPD:
For single query, check marc_s's answer on the link above.
with query as
(
select name, ROW_NUMBER() OVER(ORDER BY name ASC) as line from #table
)
select top (#pagesize) name,
(SELECT MAX(line) FROM query) AS total
from query
where line > (#pagenumber - 1) * #pagesize
#pagenumber=5
#pagesize=5
Create a common table expression and write logic like this
Between ((#pagenumber-1)*(#pagesize))+1 and (#pagenumber *#pagesize)
There are many way we can achieve pagination: I hope this information is useful to you and others.
Example 1: using offset-fetch next clause. introduce in 2005
declare #table table (name varchar(30));
insert into #table values ('Jeanna Hackman');
insert into #table values ('Han Fackler');
insert into #table values ('Tiera Wetherbee');
insert into #table values ('Hilario Mccray');
insert into #table values ('Mariela Edinger');
insert into #table values ('Darla Tremble');
insert into #table values ('Mammie Cicero');
insert into #table values ('Raisa Harbour');
insert into #table values ('Nicholas Blass');
insert into #table values ('Heather Hayashi');
declare #pagenumber int = 1
declare #pagesize int = 3
--this is a CTE( common table expression and this is introduce in 2005)
with query as
(
select ROW_NUMBER() OVER(ORDER BY name ASC) as line, name from #table
)
--order by clause is required to use offset-fetch
select * from query
order by name
offset ((#pagenumber - 1) * #pagesize) rows
fetch next #pagesize rows only
Example 2: using row_number() function and between
declare #table table (name varchar(30));
insert into #table values ('Jeanna Hackman');
insert into #table values ('Han Fackler');
insert into #table values ('Tiera Wetherbee');
insert into #table values ('Hilario Mccray');
insert into #table values ('Mariela Edinger');
insert into #table values ('Darla Tremble');
insert into #table values ('Mammie Cicero');
insert into #table values ('Raisa Harbour');
insert into #table values ('Nicholas Blass');
insert into #table values ('Heather Hayashi');
declare #pagenumber int = 2
declare #pagesize int = 3
SELECT *
FROM
(select ROW_NUMBER() OVER (ORDER BY PRODUCTNAME) AS RowNum, * from Products)
as Prodcut
where RowNum between (((#pagenumber - 1) * #pageSize )+ 1)
and (#pagenumber * #pageSize )
I hope these will be helpful to all
I don't like other solutions for being too complex, so here is my version.
Execute three select queries in one go and use output parameters for getting the count values. This query returns the total count, the filter count, and the page rows. It supports sorting, searching, and filtering the source data. It's easy to read and modify.
Let's say you have two tables with one-to-many relationship, items and their prices changed over time so the example query is not too trivial.
create table shop.Items
(
Id uniqueidentifier not null primary key,
Name nvarchar(100) not null,
);
create table shop.Prices
(
ItemId uniqueidentifier not null,
Updated datetime not null,
Price money not null,
constraint PK_Prices primary key (ItemId, Updated),
constraint FK_Prices_Items foreign key (ItemId) references shop.Items(Id)
);
Here is the query:
select #TotalCount = count(*) over()
from shop.Items i;
select #FilterCount = count(*) over()
from shop.Items i
outer apply (select top 1 p.Price, p.Updated from shop.Prices p where p.ItemId = i.Id order by p.Updated desc) as p
where (#Search is null or i.Name like '%' + #Search + '%')/**where**/;
select i.Id as ItemId, i.Name, p.Price, p.Updated
from shop.Items i
outer apply (select top 1 p.Price, p.Updated from shop.Prices p where p.ItemId = i.Id order by p.Updated desc) as p
where (#Search is null or i.Name like '%' + #Search + '%')/**where**/
order by /**orderby**/i.Id
offset #SkipCount rows fetch next #TakeCount rows only;
You need to provide the following parameters to the query:
#SkipCount - how many records to skip, calculated from the page number.
#TakeCount - how many records to return, calculated from or equal to the page size.
#Search - a text to search for in some columns, provided by the grid search box.
#TotalCount - the total number of records in the data source, the output parameter.
#FilterCount - the number of records after the search and filtering operations, the output parameter.
You can replace /**orderby**/ comment with the list of columns and their ordering directions if the grid must support sorting the rows by columns. you get this info from the grid and translate it to an SQL expression. We still need to order the records by some column initially, I usually use ID column for that.
If the grid must support filtering data by each column individually, you can replace /**where**/ comment with an SQL expression for that.
If the user is not searching and filtering the data, but only clicks through the grid pages, this query doesn't change at all and the database server executes it very quickly.

Resources