I am working on a query that I need to modify so that a string is passed to in(). The view table is being used by some other view table and ultimately by a stored procedure. The string values must be in ' '.
select region, county, name
from vw_main
where state - 'MD'
and building_id in ('101', '102') -- pass the string into in()
The values for the building_id will be entered at the stored procedure level upon its execution.
Please check below scripts which will give you answer.
Way 1: Split CSV value using XML and directly use select query in where condition
DECLARE #StrBuildingIDs VARCHAR(1000)
SET #StrBuildingIDs = '101,102'
SELECT
vm.region,
vm.county,
vm.name
FROM vw_main vm
WHERE vm.state = 'MD'
AND vm.building_id IN
(
SELECT
l.value('.','VARCHAR(20)') AS Building_Id
FROM
(
SELECT CAST('<a>' + REPLACE(#StrBuildingIDs,',','</a><a>') + '</a>') AS BuildIDXML
) x
CROSS APPLY x.BuildIDXML.nodes('a') Split(l)
)
Way 2: Split CSV value using XML, Create Variable Table and use that in where condition
DECLARE #StrBuildingIDs VARCHAR(1000)
SET #StrBuildingIDs = '101,102'
DECLARE #TblBuildingID TABLE(BuildingId INT)
INSERT INTO #TblBuildingID(BuildingId)
SELECT
l.value('.','VARCHAR(20)') AS Building_Id
FROM
(
SELECT CAST('<a>' + REPLACE(#StrBuildingIDs,',','</a><a>') + '</a>') AS BuildIDXML
) x
CROSS APPLY x.BuildIDXML.nodes('a') Split(l)
SELECT
vm.region,
vm.county,
vm.name
FROM vw_main AS vm
WHERE vm.state = 'MD'
AND vm.building_id IN
(
SELECT
BuildingId
FROM #TblBuildingID
)
Way 3: Split CSV value using XML, Create Variable Table and use that in INNER JOIN
Assuming the input string is not end-user input, you can do this. That is, derived or pulled from another table or other controlled source.
DECLARE #in nvarchar(some length) = N'''a'',''b'',''c'''
declare #stmt nvarchar(4000) = N'
select region, county, name
from vw_main
where state = ''MD''
and building_id in ({instr})'
set #stmt = replace(#stmt, N'{instr}', #instr)
exec sp_executesql #stmt=#stmt;
If the input is from an end-user, this is safer:
declare # table (a int, b char)
insert into #(a, b) values (1,'A'), (2, 'B')
declare #str varchar(50) = 'A,B'
select t.* from # t
join (select * from string_split(#str, ',')) s(b)
on t.b = s.b
You may like it better anyway, since there's no dynamic sql involved. However you must be running SQL Server 2016 or higher.
Related
In t-sql my dilemma is that I have to parse a potentially long string (up to 500 characters) for any of over 230 possible values and remove them from the string for reporting purposes. These values are a column in another table and they're all upper case and 4 characters long with the exception of two that are 5 characters long.
Examples of these values are:
USFRI
PROME
AZCH
TXJS
NYDS
XVIV. . . . .
Example of string before:
"Offered to XVIV and USFRI as back ups. No response as of yet."
Example of string after:
"Offered to and as back ups. No response as of yet."
Pretty sure it will have to be a UDF but I'm unable to come up with anything other than stripping ALL the upper case characters out of the string with PATINDEX which is not the objective.
This is unavoidably cludgy but one way is to split your string into rows, once you have a set of words the rest is easy; Simply re-aggregate while ignoring the matching values*:
with t as (
select 'Offered to XVIV and USFRI as back ups. No response as of yet.' s
union select 'Another row AZCH and TXJS words.'
), v as (
select * from (values('USFRI'),('PROME'),('AZCH'),('TXJS'),('NYDS'),('XVIV'))v(v)
)
select t.s OriginalString, s.Removed
from t
cross apply (
select String_Agg(j.[value], ' ') within group(order by Convert(tinyint,j.[key])) Removed
from OpenJson(Concat('["',replace(s, ' ', '","'),'"]')) j
where not exists (select * from v where v.v = j.[value])
)s;
* Requires a fully-supported version of SQL Server.
build a function to do the cleaning of one sentence, then call that function from your query, something like this SELECT Col1, dbo.fn_ReplaceValue(Col1) AS cleanValue, * FROM MySentencesTable. Your fn_ReplaceValue will be something like the code below, you could also create the table variable outside the function and pass it as parameter to speed up the process, but this way is all self contained.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION fn_ReplaceValue(#sentence VARCHAR(500))
RETURNS VARCHAR(500)
AS
BEGIN
DECLARE #ResultVar VARCHAR(500)
DECLARE #allValues TABLE (rowID int, sValues VARCHAR(15))
DECLARE #id INT = 0
DECLARE #ReplaceVal VARCHAR(10)
DECLARE #numberOfValues INT = (SELECT COUNT(*) FROM MyValuesTable)
--Populate table variable with all values
INSERT #allValues
SELECT ROW_NUMBER() OVER(ORDER BY MyValuesCol) AS rowID, MyValuesCol
FROM MyValuesTable
SET #ResultVar = #sentence
WHILE (#id <= #numberOfValues)
BEGIN
SET #id = #id + 1
SET #ReplaceVal = (SELECT sValue FROM #allValues WHERE rowID = #id)
SET #ResultVar = REPLACE(#ResultVar, #ReplaceVal, SPACE(0))
END
RETURN #ResultVar
END
GO
I suggest creating a table (either temporary or permanent), and loading these 230 string values into this table. Then use it in the following delete:
DELETE
FROM yourTable
WHERE col IN (SELECT col FROM tempTable);
If you just want to view your data sans these values, then use:
SELECT *
FROM yourTable
WHERE col NOT IN (SELECT col FROM tempTable);
I need to read a pipe delimited file where we have an array repeating 30 times. I need to access these array of elements and change the sequence and send in the output file.
E.g.
Tanya|1|Pen|2|Book|3|Eraser
Raj|11|Eraser|22|Bottle
In the above example, first field is the Customer name. After that we have an array of items ordered - Order ID and Item name.
Could you please suggest how to read these array elements individually to process these further?
You will be using copy activity in Azure Data Factory pipeline in which the source will be DelimitedText dataset and sink will be JSON dataset. If your source and destination files are located in Azure Blob Storage, create a Linked Service to connect the files with Azure Data Factory.
The source file dataset properties will look like this. You need to select Column Delimiter as Pipe (|).
The sink file with JSON type dataset settings will look like as shown in below. Just mention the output location path where the file will be saved after conversion.
In copy data activity sink tab, select the File pattern as Array of files. Trigger the pipeline.
The sample input and output shown below.
If you can control the output file format then you're better off using two (or three) delimiters, one for the person and the other for the order items. If you can get a third then use that to split the order item from the order line.
Assuming the data format:
Person Name | 1:M [Order]
And each Order is
order line | item name
You can simply ingest the entire row into a single nvarchar(max) column and then use SQL to break out the data you need.
The following is one such example.
declare #tbl table (d nvarchar(max));
insert into #tbl values('Tanya|1|Pen|2|Book|3|Eraser'),('Raj|11|Eraser|22|Bottle');
declare #base table (id int, person varchar(100),total_orders int, raw_orders varchar(max));
declare #output table (id int, person varchar(100),item_id int, item varchar(100));
with a as
(
select
CHARINDEX('|',d) idx
,d
,ROW_NUMBER() over (order by d) as id /*Or newid()*/
from #tbl
), b as
(
select
id
,SUBSTRING(d,0,idx) person
,SUBSTRING(d,idx+1,LEN(d)-idx+1) order_array
from a
), c as
(
select id, person, order_array
,(select count(1) from string_split(order_array,'|')) /2 orders
from b
)
insert into #base (id,person,total_orders,raw_orders)
select id,person,orders,order_array from c
declare #total_persons int = (select count(1) from #base);
declare #person_enu int = 1;
while #person_enu <= #total_persons
BEGIN
declare #total_orders int = (select total_orders from #base where id = #person_enu);
declare #raw_orders nvarchar(max) = (select raw_orders from #base where id = #person_enu);
declare #order_enu int = 1;
declare #i int = 1;
print CONCAT('Person ', #person_enu, '. Total orders: ', #total_orders);
while #order_enu <= #total_orders
begin
--declare #id int = (select value from string_split(#raw_orders,'|',1) where ordinal = #i);
--declare #val varchar(100) = (select value from string_split(#raw_orders,'|',1) where ordinal = #i+1);
--print concat('Will process order ',#order_enu);
--print concat('ID:',#i, ' Value:', #i+1)
--print concat('ID:',#id, ' Value:', #val)
INSERT INTO #output (id,person,item_id,item)
select b.id,b.person,n.value [item_id], v.value [item] from #base b
cross apply string_split(b.raw_orders,'|',1) n
cross apply string_split(b.raw_orders,'|',1) v
where b.id = #person_enu and n.ordinal = #i and v.ordinal = #i+1;
set #order_enu +=1;
set #i+=2;
end
set #person_enu += 1;
END
select * from #output;
I am passing string values from my code like '12th Standard/Ordinary National Diploma,Higher National Diploma' to SQL query, but I am not getting any values and nothing showing any result.
My SQL query:
declare #qua varchar(250),#final varchar(250),#Qualification varchar(250)
set #Qualification= '12th Standard/Ordinary National Diploma,Higher National Diploma'
set #qua =replace(#Qualification,',',''',''')
set #final= ''''+#qua+''''
select * from mytablename in(#final)
Result: Data is not displaying
Thank you in advance.
Instead do it using a table variable like
declare #tbl table(qual varchar(250));
insert into #tbl
select '12th Standard/Ordinary National Diploma'
union
select 'Higher National Diploma';
select * from mytablename where somecolumn in(select qual from #tbl);
Despite trying to put quote marks in there, you're still only passing a single string to the IN. The string just contains embedded quotes and SQL Server is looking for that single long string.
You also don't seem to be comparing a column for the IN.
Your best bet is to pass in multiple string variables, but if that's not possible then you'll have to write a function that parses a single string into a resultset and use that. For example:
SELECT
Column1, -- Because we never use SELECT *
Column2
FROM
MyTableName
WHERE
qualification IN (SELECT qualification FROM dbo.fn_ParseString(#qualifications))
You can insert all your search criteria in one table and then can easily do a lookup on the main table, example below:
DECLARE #MyTable TABLE (Name VARCHAR(10), Qualification VARCHAR(50))
DECLARE #Search TABLE (Qualifications VARCHAR(50))
INSERT INTO #MyTable VALUES ('User1','12th Standard'), ('User2','Some Education'),
('User3','Ordinary National Diploma'), ('User4','Some Degree'),
('User5','Higher National Diploma')
INSERT INTO #Search VALUES ('12th Standard'),('Ordinary National Diploma'),('Higher National Diploma')
SELECT MT.*
FROM #MyTable MT
INNER JOIN (SELECT Qualifications FROM #Search) S ON S.Qualifications = MT.Qualification
As previous said, you are passing a string with commas, not comma separated values. It needs to be split up into separate values.
You can do this by passing the qualification string into XML which you can use to turn it into separate rows of data.
The IN parameter will then accept the data as separate values.
DECLARE #Qualifications as varchar(150) = '12th Standard/Ordinary National Diploma,Higher National Diploma'
Declare #Xml XML;
SET #Xml = N'<root><r>' + replace(#Qualifications, char(44),'</r><r>') + '</r></root>';
select *
from MyTableName
Where MyTableName.Qualification in
(select r.value('.','varchar(max)') as item
from #Xml.nodes('//root/r') as records(r))
Alternatively you can create a table-valued function that splits according to input like in your case its ',' and then INNER JOIN with the returnColumnname and that particular column that you want to filter
SELECT COLUMNS, . . . .
FROM MyTableName mtn
INNER JOIN dbo.FNASplitToTable(#qualifications, ',') csvTable
ON csvTable.returnColumnName = mtn.somecolumn
Table Valued function might be like:
CREATE FUNCTION dbo.FNASplitToTable (#string varchar(MAX), #splitType CHAR(1))
RETURNS #result TABLE(Value VARCHAR(100))
AS
BEGIN
DECLARE #x XML
SELECT #x = CAST('<A>' + REPLACE(#string, #splitType, '</A><A>') + '</A>' AS XML)
INSERT INTO #result
SELECT LTRIM(t.value('.', 'VARCHAR(100)')) AS inVal
FROM #x.nodes('/A') AS x(t)
RETURN
END
GO
I have a stored proc in the following format
create PROCEDURE [dbo].[test proc]
#identifier varchar(20),
#issuerName varchar(max),
#max_records int=1000
AS
BEGIN
declare #select nvarchar(30)
SELECT #identifier as '#identifier'
, (
SELECT
MoodysOrgID as '#MoodysOrgID'
,ReportDate as '#ReportDate'
,m.UpdateTime as '#UpdateTime'
,m.FileCreationDate as '#FileCreationDate'
from mfm_financial_ratios m
inner join mfm_financial_ratios_coa c on c.AcctNo = m.AcctNo
where ReportDate in (select distinct top (#max_records) reportdate from mfm_financial_ratios where MoodysOrgID = m.MoodysOrgID)
and m.MoodysOrgID=(select top 1 IssuerID_Moodys as id from loans where LIN=#identifier or LoanXID=#identifier
and ParentName_Moodys=#issuerName and IssuerID_Moodys is not null)
order by ReportDate desc
FOR XML PATH('FinRatios'), TYPE
)
FOR XML PATH('FinRatiosHistory')
END
but i would like to make by query execute as dynamic sql
and my stored proc looks like
create PROCEDURE [dbo].[test proc]
#identifier varchar(20),
#issuerName varchar(max),
#max_records int=1000
AS
BEGIN
declare #select nvarchar(30)
set #select = N'SELECT #identifier as '#identifier'
, (
SELECT
MoodysOrgID as '#MoodysOrgID'
,ReportDate as '#ReportDate'
,m.UpdateTime as '#UpdateTime'
,m.FileCreationDate as '#FileCreationDate'
from mfm_financial_ratios m
inner join mfm_financial_ratios_coa c on c.AcctNo = m.AcctNo
where ReportDate in (select distinct top (#max_records) reportdate from mfm_financial_ratios where MoodysOrgID = m.MoodysOrgID)
and m.MoodysOrgID=(select top 1 IssuerID_Moodys as id from loans where LIN=#identifier or LoanXID=#identifier
and ParentName_Moodys=#issuerName and IssuerID_Moodys is not null)
order by ReportDate desc
FOR XML PATH('FinRatios'), TYPE
)
FOR XML PATH('FinRatiosHistory')'
exec #select
END
The following stored proc gives issues because of the comma used in it .Can someone let me know what you be the correct way of doing it
The problem are not the commas. You mostly have two problems: one, you're not escaping the quotes correctly. And two, you're not concatenating your variables correctly. Here's an example of both:
For concatenating variables: In your first select line, you cannot do this:
SELECT #identifier as '#identifier'
because sql does not know what to do with #identifier that way. You should concatenate the variable this way:
SELECT #identifier as ' + #identifier + '.. everything else goes here
Also, when you will have to concatenate max_records, since it's an int variable you should cast it to varchar first, like this:
select distinct top (' + cast(#max_records as varchar(10) + ') ....
Whenever you're using a variable in the middle of the string (such as #max_records) you HAVE to concatenate it in order for SQL to know it's a variable and not just a string. You didn't do it with max_records, #issuerName, etc.
For escaping quotes: You need to escape your single quotes when you don't want your select string to unexpectedly end. For example here:
FOR XML PATH('FinRatiosHistory')'
You should escape them with double quotes (google escaping single quotes sql if you don't get it)
FOR XML PATH(''FinRatiosHistory'')'
I have a SourceTable and a table variable #TQueries containing various T-SQL predicates that target SourceTable.
The expected result is to dynamically generate SELECT statements that return a list of Id's as specified by the predicates in #TQueries. Each dynamically generated SELECT statement also needs to execute in a particular order, and the final set of values needs to be unique and the ordering must be preserved.
Fortunately, there's a limit to how many values need to be retrieved and how many dynamic queries need to be generated. The Id list should contain at most 10 Ids, and we don't expect more than 7 queries.
The following is a sample of this setup, not the actual data/database:
-- Set up some test data, this is quick and dirty just to provide some data to test against
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SourceTable]') AND type in (N'U'))
BEGIN
-- Create a numbers table, sorta
SELECT TOP 20
IDENTITY(INT,1,1) AS Id,
ABS(CHECKSUM(NewId())) % 100 AS [SomeValue]
INTO [SourceTable]
FROM sysobjects a
END
DECLARE #TQueries TABLE (
[Ordinal] INT,
[WherePredicate] NVARCHAR(MAX),
[OrderByPredicate] NVARCHAR(MAX)
);
-- Simulate SELECTs with different order by that get different data due to varying WHERE clauses and ORDER conditions
INSERT INTO #TQueries VALUES ( 1, N'[Id] IN (6,11,13,7,10,3,15)', '[SomeValue] ASC' ) -- Sort Asc
INSERT INTO #TQueries VALUES ( 2, N'[Id] IN (9,15,14,20,17)', '[SomeValue] DESC' ) -- Sort Desc
INSERT INTO #TQueries VALUES ( 3, N'[Id] IN (20,10,1,16,11,19,9,15,17,6,2,3,13)', 'NEWID()' ) -- Sort Random
My main issue has been avoiding the use of a CURSOR or iterating through the rows one by one. The closest I've come to a set operation that meets this criteria is using a table variable to store the results of each query or a massive CTE.
Suggestions and comments are welcome.
Here's a solution that builds a single statement both to run all the queries and to return the results.
It uses a similar approach as in your answer when iterating over the #TQueries table, i.e. it also uses {...} tokens where column values from #TQuery should go, and it puts the values there with nested REPLACE() calls.
Other than that, it heavily depends on ranking functions, and I'm not sure if doesn't really abuse them. You'd need to test this method before deciding if it's better or worse than the one you've got so far.
DECLARE #QueryTemplate nvarchar(max), #FinalSQL nvarchar(max);
SET #QueryTemplate =
N'SELECT
[Id],
QueryRank = {Ordinal},
RowRank = ROW_NUMBER() OVER (ORDER BY {OrderByPredicate})
FROM [dbo].[SourceTable]
WHERE {WherePredicate}
';
SET #FinalSQL =
N'WITH AllData AS (
' +
SUBSTRING(
(
SELECT
'UNION ALL ' +
REPLACE(REPLACE(REPLACE(#QueryTemplate,
'{Ordinal}' , [Ordinal] ),
'{OrderByPredicate}', [OrderByPredicate]),
'{WherePredicate}' , [WherePredicate] )
FROM #TQueries
ORDER BY [Ordinal]
FOR XML PATH (''), TYPE
).value('.', 'nvarchar(max)'),
11, -- starting just after the first 'UNION ALL '
CAST(0x7FFFFFFF AS int) -- max int; no need to specify the exact length
) +
'),
RankedData AS (
SELECT
[Id],
QueryRank,
RowRank,
ValueRank = ROW_NUMBER() OVER (PARTITION BY [Id] ORDER BY QueryRank)
FROM AllData
)SELECT TOP (#top)
[Id]
FROM RankedData
WHERE ValueRank = 1
ORDER BY
QueryRank,
RowRank
';
PRINT #FinalSQL;
EXECUTE sp_executesql #FinalSQL, N'#top int', 10;
Basically, every subquery gets these auxiliary columns:
QueryRank – a constant value (within the subquery's result set) derived from [Ordinal];
RowRank – a ranking assigned to a row based on the [OrderByPredicate].
The result sets are UNIONed and then every entry of every unique value is again ranked (ValueRank) based on the query ranking.
When pulling the final result set, duplicates are suppressed (by the condition ValueRank = 1), and QueryRank and RowRank are used in the ORDER BY clause to preserve the original row order.
I used EXECUTE sp_executesql #query instead of EXECUTE (#query), because the former allows you to add parameters to the query. In particular, I parametrised the number of results to return (the argument of TOP). But you could certainly concatenate that value into the dynamic script directly, just like other things, if you prefer EXECUTE () over EXECUTE sq_executesql.
If you like, you can try this query at SQL Fiddle. (Note: the SQL Fiddle version replaces the #TQueries table variable with the TQueries table.)
This is what I've managed to piece together cobbled from my original response and improved upon by comments from #AndriyM
DECLARE #sql_prefix NVARCHAR(MAX);
SET #sql_prefix =
N'DECLARE #TResults TABLE (
[Ordinal] INT IDENTITY(1,1),
[ContentItemId] INT
);
DECLARE #max INT, #top INT;
SELECT #max = 10;';
DECLARE #sql_insert_template NVARCHAR(MAX), #sql_body NVARCHAR(MAX);
SET #sql_insert_template =
N'SELECT #top = #max - COUNT(*) FROM #TResults;
INSERT INTO #TResults
SELECT TOP (#top) [Id]
FROM [dbo].[SourceTable]
WHERE
{WherePredicate}
AND NOT EXISTS (
SELECT 1
FROM #TResults AS [tr]
WHERE [tr].[ContentItemId] = [SourceTable].[Id]
)
ORDER BY {OrderByPredicate};';
WITH Query ([Ordinal],[SqlCommand]) AS (
SELECT
[Ordinal],
REPLACE(REPLACE(#sql_insert_template, '{WherePredicate}', [WherePredicate]), '{OrderByPredicate}', [OrderByPredicate])
FROM #TQueries
)
SELECT
#sql_body = #sql_prefix + (
SELECT [SqlCommand]
FROM Query
ORDER BY [Ordinal] ASC
FOR XML PATH(''),TYPE).value('.', 'varchar(max)') + CHAR(13)+CHAR(10)
+N' SELECT * FROM #TResults ORDER BY [Ordinal]';
EXEC(#sql_body);
The basic idea is to use a table variable to hold the results of each query. I create a template for the SQL and replace the values in the template based on what is stored in #TQueries.
Once the entire script is completed I run it with EXEC.