TSQL: How insert separator between each character in a string - sql-server

I have a string like this:
Apple
I want to include a separator after each character so the end result will turn out like this:
A,p,p,l,e
In C#, we have one liner method to achieve the above with Regex.Replace('Apple', ".{1}", "$0,");
I can only think of looping each character with charindex to append the separator but seems a little complicated. Is there any elegant way and simpler way to achieve this?

Thanks HABO for the suggestions. I'm able to generate the result that I want using the code but takes a little bit of time to really understand how the code work.
After some searching, I manage to found one useful article to insert empty spaces between each character and it's easier for me to understand.
I modify the code a little to define and include desire separator instead of fixing it to space as the separator:
DECLARE #pos INT = 2 -- location where we want first space
DECLARE #result VARCHAR(100) = 'Apple'
DECLARE #separator nvarchar(5) = ','
WHILE #pos < LEN(#result)+1
BEGIN
SET #result = STUFF(#result, #pos, 0, #separator);
SET #pos = #pos+2;
END
select #result; -- Output: A,p,p,l,e
Reference

In following SQL scripts, I get each character using SUBSTRING() function using with a number table (basically I used spt_values view here for simplicity) and then I concatenate them via two different methods, you can choose one
If you are using SQL Server 2017, we have a new SQL string aggregation function
First script uses string_agg function
declare #str nvarchar(max) = 'Apple'
SELECT
string_agg( substring(#str,number,1) , ',') Within Group (Order By number)
FROM master..spt_values n
WHERE
Type = 'P' and
Number between 1 and len(#str)
If you are working with a previous version, you can use string concatenation using FOR XML Path and SQL Stuff function as follows
declare #str nvarchar(max) = 'Apple'
; with cte as (
SELECT
number,
substring(#str,number,1) as L
FROM master..spt_values n
WHERE
Type = 'P' and
Number between 1 and len(#str)
)
SELECT
STUFF(
(
SELECT
',' + L
FROM cte
order by number
FOR XML PATH('')
), 1, 1, ''
)
Both solution yields the same result, I hope it helps

If you have SQL Server 2017 and a copy of ngrams8k it's ultra simple:
declare #word varchar(100) = 'apple';
select newString = string_agg(token, ',') within group (order by position)
from dbo.ngrams8k(#word,1);
For pre-2017 systems it's almost as simple:
declare #word varchar(100) = 'apple';
select newstring =
( select token + case len(#word)+1-position when 1 then '' else ',' end
from dbo.ngrams8k(#word,1)
order by position
for xml path(''))

One ugly way to do it is to split the string into characters, ideally using a numbers table, and reassemble it with the desired separator.
A less efficient implementation uses recursion in a CTE to split the characters and insert the separator between pairs of characters as it goes:
declare #Sample as VarChar(20) = 'Apple';
declare #Separator as Char = ',';
with Characters as (
select 1 as Position, Substring( #Sample, 1, 1 ) as Character
union all
select Position + 1,
case when Position & 1 = 1 then #Separator else Substring( #Sample, Position / 2 + 1, 1 ) end
from Characters
where Position < 2 * Len( #Sample ) - 1 )
select Stuff( ( select Character + '' from Characters order by Position for XML Path( '' ) ), 1, 0, '' ) as Result;
You can replace the select Stuff... line with select * from Characters; to see what's going on.

Try this
declare #var varchar(50) ='Apple'
;WITH CTE
AS
(
SELECT
SeqNo = 1,
MyStr = #var,
OpStr = CAST('' AS VARCHAR(50))
UNION ALL
SELECT
SeqNo = SeqNo+1,
MyStr = MyStR,
OpStr = CAST(ISNULL(OpStr,'')+SUBSTRING(MyStR,SeqNo,1)+',' AS VARCHAR(50))
FROM CTE
WHERE SeqNo <= LEN(#var)
)
SELECT
OpStr = LEFT(OpStr,LEN(OpStr)-1)
FROM CTE
WHERE SeqNo = LEN(#Var)+1

Related

Print Substring before and after slash '/' from a string

I've a task to print substring from a string based on the occurence of slash '/' position in a string .Below i'm trying to show you one sample example how i want to get .
Declare #My_string = 'abc/def gh /ijk l/m/no p/qr
From the above string i want to print the substring value based on the position of slash occourence.
For Example: Sometimes i was asked to print the substring after the 2nd occourence of slah then i've to display ijk l and Sometimes i was asked to print the substring after the 3rd occourence then m should display if it is 1st then def gh vice versa .There is no specific occourence position it may vary based on need .How can i achieve it .Any help will be very thankful
Note: I want to achieve without using function .i know that it can be done by using split function .Is that possible to get without using it
One way to do it without using a function is to use a recursive cte.
This is not the way I would recommend splitting a string but since you insist on not using a function it's a reasonable alternative.
If you change your mind about splitting string function, you should read Aaron Bertrand's Split strings the right way – or the next best way and choose a split string function that would be easy to modify to return the item number as well.
Sample data:
DECLARE #My_string varchar(100) = 'abc/def gh /ijk l/m/no p/qr',
#SlashIndex int = 3
The CTE:
;WITH CTE AS
(
SELECT LEFT(#My_String, CHARINDEX('/', #My_String)-1) As Value,
0 As SlashIndex,
RIGHT(#My_String, LEN(#My_String) - CHARINDEX('/', #My_String)) As String
WHERE CHARINDEX('/', #My_String) > 0
OR LEN (#My_String) > 0
UNION ALL
SELECT LEFT(String, CASE WHEN CHARINDEX('/', String) > 0 THEN CHARINDEX('/', String) ELSE LEN(String) +1 END -1),
SlashIndex + 1,
RIGHT(String, LEN(String) - CASE WHEN CHARINDEX('/', String) > 0 THEN CHARINDEX('/', String) ELSE LEN(String) END)
FROM CTE
WHERE CHARINDEX('/', String) > 0
OR LEN(String) > 0
)
The Query:
SELECT Value
FROM CTE
WHERE SlashIndex = #SlashIndex
Result: m
You can see a live demo on rextester.
A simple way to get the 3rd position via XML
Example
Declare #My_string varchar(max)= 'abc/def gh /ijk l/m/no p/qr'
Select Pos3 = convert(xml,'<x>'+replace(#My_String,'/','</x><x>')+'</x>').value('/x[3]','varchar(100)')
Returns
Pos3
ijk l
created below function which will take 2 input first - your position and second will be your string and return the required output -
create function dbo.GetSubsting ( #StatPoint int , #Input_string varchar(1000) )
returns varchar(1000)
as
begin
Declare #len int , #idx int =1 , #section int = 1 , #output varchar(1000) = ''
DECLARE #tab table ( idx int identity(1,1 ), Val char(1) , section int )
select #len = len(#Input_string)
while #idx <= #len
begin
insert into #tab
select substring( #Input_string , #idx ,1) , #section
if substring( #Input_string , #idx ,1) = '/'
set #section = #section + 1
set #idx = #idx + 1
end
select #output = #output + Val from #tab where section = #StatPoint
select #output = replace(#output , '/' , '')
RETURN #output
end
go
select dbo.GetSubsting(3, 'abc/def gh /ijk l/m/no p/qr')
--OUTPUT
-- ijk l

SQL Server : select all after specific character

How I can select
"ALT1" if value is "W61N03D20V0-WHIH-ALT1"
"ALT2" if for "W61N03D20V0-WHIH-ALT2"
"SW" for "W61N03D20V0-WHIH-SW"
"Default" for "W61N26D1YA1-VICU" (without prefix)
"Defailt" for "W61N27D21V2-AZTD"
In other words I'm looking for a way extract last part after second suffix, but if I have't second suffix - then default
Thanks for advice
Try it like this:
First you "split" the string on its minus signs with the XML trick.
Then you read the third node from you XML - voila!
CREATE TABLE #tbl(content VARCHAR(100));
INSERT INTO #tbl VALUES('W61N03D20V0-WHIH-ALT1')
,('W61N03D20V0-WHIH-SW')
,('W61N26D1YA1-VICU');
WITH SplittedAsXml AS
(
SELECT CAST('<x>' + REPLACE(content,'-','</x><x>') + '</x>' AS XML) AS Content
FROM #tbl
)
SELECT ISNULL(Content.value('/x[3]','varchar(max)'),'default') AS TheThirdPart
FROM SplittedAsXml;
DROP TABLE #tbl;
The result
ALT1
SW
default
Going this ways would also give you the chance to get the other parts in one go just querying /x[1] and /x[2] too
I did it using the built-in substring() function:
declare #str VARCHAR(40) = 'W61N03D20V0-WHIH-ALT1' -- also works for the other examples
declare #sep VARCHAR(1) = '-'
declare #middleToEnd VARCHAR(40) = substring(#str, charindex(#sep, #str) + 1, len(#str))
declare #pos INT = charindex(#sep, #middleToEnd)
declare #lastPart VARCHAR(40) =
CASE WHEN #pos = 0
THEN 'Default'
ELSE substring(#middleToEnd, #pos + 1, len(#middleToEnd))
END
select #lastPart
For best performance, you can solve it with this one-liner(calculation is one line)
SELECT
COALESCE(STUFF(col,1,NULLIF(CHARINDEX('-',col, CHARINDEX('-',col)+1), 0),''),'Default')
FROM (values
('W61N03D20V0-WHIH-ALT1'),('W61N03D20V0-WHIH-ALT2'),
('W61N03D20V0-WHIH-SW'),('W61N26D1YA1-VICU'),
('W61N27D21V2-AZTD')) x(col)
Result:
ALT1
ALT2
SW
Default
Default
If I understand what you are asking for, the following does what you need:
-- fake table
WITH SomeTable AS (
SELECT 'W61N03D20V0-WHIH-ALT1' AS Field1
UNION ALL
SELECT 'W61N03D20V0-WHIH-SW'
UNION ALL
SELECT 'W61N26D1YA1-VICU'
)
-- select
SELECT
CASE CHARINDEX('-WHIH-', Field1)
WHEN 0 THEN 'Default'
ELSE SUBSTRING(Field1, CHARINDEX('-WHIH-', Field1) + 6, LEN(Field1) - (CHARINDEX('-WHIH-', Field1) + 5))
END
FROM SomeTable
Use can use a CASE expression to check whether the string starts with W61N03D20V0-WHIH.
If it starts with it use a combination of RIGHT, REVERSE and CHARINDEX functions to get last part from the string, else Default.
Query
select case when [your_column_name] like 'W61N03D20V0-WHIH%'
then right([your_column_name], charindex('-', reverse([your_column_name]), 1) - 1)
else 'Default' end as new_column_name
from your_table_name;
SQl Fiddle demo

How can I complete this Excel function in SQL Server?

I have approximately 30,000 records where I need to split the Description field and so far I can only seem to achieve this in Excel. An example Description would be:
1USBCP 2RJ45C6 1DVI 1DP 3MD 3MLP HANDS
Below is my Excel function:
=TRIM(MID(SUBSTITUTE($G309," ",REPT(" ",LEN($G309))),((COLUMNS($G309:G309)-1)*LEN($G309))+1,LEN($G309)))
This is then dragged across ten Excel columns, and splits the description field at each space.
I have seen many questions asked about splitting a string in SQL but they only seem to cover one space, not multiple spaces.
There is no easy function in SQL server to split strings. At least I don't know it. I use usually some trick that I found somewhere in the Internet some time ago. I modified it to your example.
The trick is that first we try to figure out how many columns do we need. We can do it by checking how many empty strings we have in the string. The easiest way is lenght of string - lenght of string without empty string.
After that for each string we try to find start and end of each word by position. At the end we cut simply string by start and end position and assign to coulmns. The details are in the query. Have fun!
CREATE TABLE test(id int, data varchar(100))
INSERT INTO test VALUES (1,'1USBCP 2RJ45C6 1DVI 1DP 3MD 3MLP HANDS')
INSERT INTO test VALUES (2,'Shorter one')
DECLARE #pivot varchar(8000)
DECLARE #select varchar(8000)
SELECT
#pivot=coalesce(#pivot+',','')+'[col'+cast(number+1 as varchar(10))+']'
FROM
master..spt_values where type='p' and
number<=(SELECT max(len(data)-len(replace(data,',',''))) FROM test)
SELECT
#select='
select p.*
from (
select
id,substring(data, start+2, endPos-Start-2) as token,
''col''+cast(row_number() over(partition by id order by start) as varchar(10)) as n
from (
select
id, data, n as start, charindex('','',data,n+2) endPos
from (select number as n from master..spt_values where type=''p'') num
cross join
(
select
id, '' '' + data +'' '' as data
from
test
) m
where n < len(data)-1
and substring(odata,n+1,1) = '','') as data
) pvt
Pivot ( max(token)for n in ('+#pivot+'))p'
EXEC(#select)
Here you can find example in SQL Fiddle
I didn't notice that you want to get rid of multiple blank spaces.
To do it please create some function that preprare your data :
CREATE FUNCTION dbo.[fnRemoveExtraSpaces] (#Number AS varchar(1000))
Returns Varchar(1000)
As
Begin
Declare #n int -- Length of counter
Declare #old char(1)
Set #n = 1
--Begin Loop of field value
While #n <=Len (#Number)
BEGIN
If Substring(#Number, #n, 1) = ' ' AND #old = ' '
BEGIN
Select #Number = Stuff( #Number , #n , 1 , '' )
END
Else
BEGIN
SET #old = Substring(#Number, #n, 1)
Set #n = #n + 1
END
END
Return #number
END
After that use the new version that removes extra spaces.
DECLARE #pivot varchar(8000)
DECLARE #select varchar(8000)
SELECT
#pivot=coalesce(#pivot+',','')+'[col'+cast(number+1 as varchar(10))+']'
FROM
master..spt_values where type='p' and
number<=(SELECT max(len(dbo.fnRemoveExtraSpaces(data))-len(replace(dbo.fnRemoveExtraSpaces(data),' ',''))) FROM test)
SELECT
#select='
select p.*
from (
select
id,substring(data, start+2, endPos-Start-2) as token,
''col''+cast(row_number() over(partition by id order by start) as varchar(10)) as n
from (
select
id, data, n as start, charindex('' '',data,n+2) endPos
from (select number as n from master..spt_values where type=''p'') num
cross join
(
select
id, '' '' + dbo.fnRemoveExtraSpaces(data) +'' '' as data
from
test
) m
where n < len(data)-1
and substring(data,n+1,1) = '' '') as data
) pvt
Pivot ( max(token)for n in ('+#pivot+'))p'
EXEC(#select)
I am probably not understanding your question, but all that you are doing in that formula, can be done almost exactly the same in SQL. I see someone has already answered but to my mind, how can it be necessary to do all that when you can do this. I might be wrong. But here goes.
declare #test as varchar(100)
set #test='abcd1234567'
select right(#test,2)
, left(#test,2)
, len(#test)
, case when len(#test)%2>0
then left(right(#test,round(len(#test)/2,0)+1),1)
else left(right(#test,round(len(#test)/2,0)+1),2) end
Results
67 ab 11 2
So right, left, length and mid can all be achieved.
If the spaces are the "substring" dividers, then: I dont remember well the actual syntax for do-while inside selects of sql, neither have i actually done that per se, but I don't see why it should not be possible. If it doesn't work then you need a temporary table and if that does not work you need a cursor. The cursor would be an external loop around this one to fetch and process a single string at a time. Or you can do something more clever. I am just a novice.
declare #x varchar(1)
declare #n integer
declare #i integer
declare #str varchar(100) -- this is your description. Fetch it and assign it. if in a cursor just use column-name
set #x = null
set #n = 0
set #i = 0
while n < len(#str)
while NOT #x = " "
begin
set #x = left(right(#str,n),1)
n = n+1
end
--insert into or update #temptable blablabla here.
Use i and n to locate substring and then left(right()) it out. or you can SELECT it, but that is a messy procedure if the number of substrings are long. Continue with:
set i = n
set #str = right(#str, i) -- this includes the " ". left() it out at will.
end
Now, a final comment, there should perhaps be a third loop checking for if you are at the last "substring" because I see now this code will throw error when it gets to the end. or "add" an empty space at the end to #str, that will also work. But my time is up. This is a suggestion at least.

split alpha and numeric using sql

I have a table and it has a 3 columns. The first column is the data that contains value(numeric) and unit(percentage and etc..), the second column is numeric column, the third is Unit column. What I want to do is split the numeric and the unit from the first column then put those split-ted data to its designated column.
Here is my table:
I tried this function:SO link here..., it really does splitting alpha and numeric but then I'm new in using SQL Function, my problem there is the parameter must be in string STRING, so what I did is change it to Sub Query but it gives me error.
Sample COde:
SQL FUNCTION:
create function [dbo].[GetNumbersFromText](#String varchar(2000))
returns table as return
(
with C as
(
select cast(substring(S.Value, S1.Pos, S2.L) as int) as Number,
stuff(s.Value, 1, S1.Pos + S2.L, '') as Value
from (select #String+' ') as S(Value)
cross apply (select patindex('%[0-9]%', S.Value)) as S1(Pos)
cross apply (select patindex('%[^0-9]%', stuff(S.Value, 1, S1.Pos, ''))) as S2(L)
union all
select cast(substring(S.Value, S1.Pos, S2.L) as int),
stuff(S.Value, 1, S1.Pos + S2.L, '')
from C as S
cross apply (select patindex('%[0-9]%', S.Value)) as S1(Pos)
cross apply (select patindex('%[^0-9]%', stuff(S.Value, 1, S1.Pos, ''))) as S2(L)
where patindex('%[0-9]%', S.Value) > 0
)
select Number
from C
)
SELECT STATEMENT with SUB Query:
declare #S varchar(max)
select number from GetNumbersFromText(Select SomeColm From Table_Name) option (maxrecursion 0)
BTW, im using sql server 2005.
Thanks!
If the numeric part is always at the beginning, then you can use this:
PATINDEX('%[0-9][^0-9]%', ConcUnit)
to get the index of the last digit.
Thus, this:
DECLARE #str VARCHAR(MAX) = '4000 ug/ML'
SELECT LEFT(#str, PATINDEX('%[0-9][^0-9]%', #str )) AS Number,
LTRIM(RIGHT(#str, LEN(#str) - PATINDEX('%[0-9][^0-9]%', #str ))) As Unit
gives you:
Number Unit
-------------
4000 ug/ML
EDIT:
If numeric data include double values as well, then you can use this:
SELECT LEN(#str) - PATINDEX ('%[^0-9][0-9]%', REVERSE(#str))
to get the index of the last digit.
Thus, this:
SELECT LEFT(#str, LEN(#str) - PATINDEX ('%[^0-9][0-9]%', REVERSE(#str)))
gives you the numeric part.
And this:
SELECT LEFT(#str, LEN(#str) - PATINDEX ('%[^0-9][0-9]%', REVERSE(#str))) AS Numeric,
CASE
WHEN CHARINDEX ('%', #str) <> 0 THEN LTRIM(RIGHT(#str, LEN(#str) - CHARINDEX ('%', #str)))
ELSE LTRIM(RIGHT(#str, PATINDEX ('%[^0-9][0-9]%', REVERSE(#str))))
END AS Unit
gives you both numberic and unit part.
Here are some tests that I made with the data you have posted:
Input:
DECLARE #str VARCHAR(MAX) = '50 000ug/ML'
Output:
Numeric Unit
------------
50 000 ug/ML
Input:
DECLARE #str VARCHAR(MAX) = '99.5%'
Output:
Numeric Unit
------------
99.5
Input:
DECLARE #str VARCHAR(MAX) = '4000 . 35 % ug/ML'
Output:
Numeric Unit
------------------
4000 . 35 ug/ML
Here is my answer. Check output in SQLFiddle for the same.
create TABLE temp
(
string NVARCHAR(50)
)
INSERT INTO temp (string)
VALUES
('4000 ug\ml'),
('2000 ug\ml'),
('%'),
('ug\ml')
SELECT subsrtunit,LEFT(subsrtnumeric, PATINDEX('%[^0-9]%', subsrtnumeric+'t') - 1)
FROM (
SELECT subsrtunit = SUBSTRING(string, posofchar, LEN(string)),
subsrtnumeric = SUBSTRING(string, posofnumber, LEN(string))
FROM (
SELECT string, posofchar = PATINDEX('%[^0-9]%', string),
posofnumber = PATINDEX('%[0-9]%', string)
FROM temp
) d
) t
Updated Version to handle 99.5 ug\ml
create TABLE temp
(
string NVARCHAR(50)
)
INSERT INTO temp (string)
VALUES
('4000 ug\ml'),
('2000 ug\ml'),
('%'),
('ug\ml'),
('99.5 ug\ml')
SELECT subsrtunit,LEFT(subsrtnumeric, PATINDEX('%[^0-9.]%', subsrtnumeric+'t') - 1)
FROM (
SELECT subsrtunit = SUBSTRING(string, posofchar, LEN(string)),
subsrtnumeric = SUBSTRING(string, posofnumber, LEN(string))
FROM (
SELECT string, posofchar = PATINDEX('%[^0-9.]%', string),
posofnumber = PATINDEX('%[0-9.]%', string)
FROM temp
) d
) t
Updated Version: To handle 1 000 ug\ml,20 000ug\ml
create TABLE temp
(
string NVARCHAR(50)
)
INSERT INTO temp (string)
VALUES
('4000 ug\ml'),
('2000 ug\ml'),
('%'),
('ug\ml'),
('99.5 ug\ml'),
('1 000 ug\ml'),
('20 000ug\ml')
SELECT substring(replace(subsrtunit,' ',''),PATINDEX('%[0-9.]%', replace(subsrtunit,' ',''))+1,len(subsrtunit)),
LEFT(replace(subsrtnumeric,' ',''), PATINDEX('%[^0-9.]%', replace(subsrtnumeric,' ','')+'t') - 1)
FROM (
SELECT subsrtunit = SUBSTRING(string, posofchar, LEN(string)),
subsrtnumeric = SUBSTRING(string, posofnumber, LEN(string))
FROM (
SELECT string, posofchar = PATINDEX('%[^0-9.]%', replace(string,' ','')),
posofnumber = PATINDEX('%[0-9.]%', replace(string,' ',''))
FROM temp
) d
) t
Check out SQLFiddle for the same.
Would something like this work? Based on the shown data it looks like it would.
Apply it to your data set as a select and if you like the results then you can make an update from it.
WITH cte as (SELECT 'ug/mL' ConcUnit, 500 as [Numeric], '' as Unit
UNION ALL SELECT '2000 ug/mL', NULL, '')
SELECT
[ConcUnit] as [ConcUnit],
[Numeric] as [Original Numeric],
[Unit] as [Original Unit],
CASE WHEN ConcUnit LIKE '% %' THEN
SUBSTRING(ConcUnit, 1, CHARINDEX(' ', ConcUnit) - 1)
ELSE [Numeric] END as [New Numeric],
CASE WHEN ConcUnit LIKE '% %'
THEN SUBSTRING(ConcUnit, CHARINDEX(' ', ConcUnit) + 1, LEN(ConcUnit))
ELSE ConcUnit END as [New Unit]
FROM cte
change #concunit & #unitx Respectively
DECLARE #concunit varchar(10)='45.5%'
DECLARE #unitx varchar(10)='%'
BEGIN
SELECT RTRIM(SUBSTRING( #concunit , 1 , CHARINDEX( #unitx , #concunit
) - 1
)) AS Number,
RTRIM(SUBSTRING( #concunit , CHARINDEX( #unitx , #concunit
) , LEN( #concunit
) - (CHARINDEX( #unitx , #concunit
) - 1)
)) AS Unit
end
I had the same dilemma, but in my case the alpha's were in front of the numerics.
So using the logic that #Giorgos Betsos added to his answer, I just reversed it.
I.e., when your input is :
abc123
You can split it like this:
declare #input varchar(30) = 'abc123'
select
replace(#input,reverse(LEFT(reverse(#input), PATINDEX('%[0-9][^0-9]%', reverse(#input) ))),'') Alpha
, reverse(LEFT(reverse(#input), PATINDEX('%[0-9][^0-9]%', reverse(#input) ))) Numeric
Results :

User defined function replacing WHERE col IN(...)

I have created a user defined function to gain performance with queries containing 'WHERE col IN (...)' like this case:
SELECT myCol1, myCol2
FROM myTable
WHERE myCol3 IN (100, 200, 300, ..., 4900, 5000);
The queries are generated from an web application and are in some cases much more complex.
The function definition looks like this:
CREATE FUNCTION [dbo].[udf_CSVtoIntTable]
(
#CSV VARCHAR(MAX),
#Delimiter CHAR(1) = ','
)
RETURNS
#Result TABLE
(
[Value] INT
)
AS
BEGIN
DECLARE #CurrStartPos SMALLINT;
SET #CurrStartPos = 1;
DECLARE #CurrEndPos SMALLINT;
SET #CurrEndPos = 1;
DECLARE #TotalLength SMALLINT;
-- Remove space, tab, linefeed, carrier return
SET #CSV = REPLACE(#CSV, ' ', '');
SET #CSV = REPLACE(#CSV, CHAR(9), '');
SET #CSV = REPLACE(#CSV, CHAR(10), '');
SET #CSV = REPLACE(#CSV, CHAR(13), '');
-- Add extra delimiter if needed
IF NOT RIGHT(#CSV, 1) = #Delimiter
SET #CSV = #CSV + #Delimiter;
-- Get total string length
SET #TotalLength = LEN(#CSV);
WHILE #CurrStartPos < #TotalLength
BEGIN
SET #CurrEndPos = CHARINDEX(#Delimiter, #CSV, #CurrStartPos);
INSERT INTO #Result
VALUES (CAST(SUBSTRING(#CSV, #CurrStartPos, #CurrEndPos - #CurrStartPos) AS INT));
SET #CurrStartPos = #CurrEndPos + 1;
END
RETURN
END
The function is intended to be used like this (or as an INNER JOIN):
SELECT myCol1, myCol2
FROM myTable
WHERE myCol3 IN (
SELECT [Value]
FROM dbo.udf_CSVtoIntTable('100, 200, 300, ..., 4900, 5000', ',');
Do anyone have some optimiztion idears of my function or other ways to improve performance in my case?
Is there any drawbacks that I have missed?
I am using MS SQL Server 2005 Std and .NET 2.0 framework.
I'm not sure of the performance increase, but I would use it as an inner join and get away from the inner select statement.
Using a UDF in a WHERE clause or (worse) a subquery is asking for trouble. The optimizer sometimes gets it right, but often gets it wrong and evaluates the function once for every row in your query, which you don't want.
If your parameters are static (they appear to be) and you can issue a multistatement batch, I'd load the results of your UDF into a table variable, then use a join against the table variable to do your filtering. This should work more reliably.
that loop will kill performance!
create a table like this:
CREATE TABLE Numbers
(
Number int not null primary key
)
that has rows containing values 1 to 8000 or so and use this function:
CREATE FUNCTION [dbo].[FN_ListAllToNumberTable]
(
#SplitOn char(1) --REQUIRED, the character to split the #List string on
,#List varchar(8000) --REQUIRED, the list to split apart
)
RETURNS
#ParsedList table
(
RowNumber int
,ListValue varchar(500)
)
AS
BEGIN
/*
DESCRIPTION: Takes the given #List string and splits it apart based on the given #SplitOn character.
A table is returned, one row per split item, with a columns named "RowNumber" and "ListValue".
This function workes for fixed or variable lenght items.
Empty and null items will be included in the results set.
PARAMETERS:
#List varchar(8000) --REQUIRED, the list to split apart
#SplitOn char(1) --OPTIONAL, the character to split the #List string on, defaults to a comma ","
RETURN VALUES:
a table, one row per item in the list, with a column name "ListValue"
TEST WITH:
----------
SELECT * FROM dbo.FN_ListAllToNumTable(',','1,12,123,1234,54321,6,A,*,|||,,,,B')
DECLARE #InputList varchar(200)
SET #InputList='17;184;75;495'
SELECT
'well formed list',LEFT(#InputList,40) AS InputList,h.Name
FROM Employee h
INNER JOIN dbo.FN_ListAllToNumTable(';',#InputList) dt ON h.EmployeeID=dt.ListValue
WHERE dt.ListValue IS NOT NULL
SET #InputList='17;;;184;75;495;;;'
SELECT
'poorly formed list join',LEFT(#InputList,40) AS InputList,h.Name
FROM Employee h
INNER JOIN dbo.FN_ListAllToNumTable(';',#InputList) dt ON h.EmployeeID=dt.ListValue
SELECT
'poorly formed list',LEFT(#InputList,40) AS InputList, ListValue
FROM dbo.FN_ListAllToNumTable(';',#InputList)
**/
/*this will return empty rows, and row numbers*/
INSERT INTO #ParsedList
(RowNumber,ListValue)
SELECT
ROW_NUMBER() OVER(ORDER BY number) AS RowNumber
,LTRIM(RTRIM(SUBSTRING(ListValue, number+1, CHARINDEX(#SplitOn, ListValue, number+1)-number - 1))) AS ListValue
FROM (
SELECT #SplitOn + #List + #SplitOn AS ListValue
) AS InnerQuery
INNER JOIN Numbers n ON n.Number < LEN(InnerQuery.ListValue)
WHERE SUBSTRING(ListValue, number, 1) = #SplitOn
RETURN
END /*Function FN_ListAllToNumTable*/
I have other versions that do not return empty or null rows, ones that return just the item and not the row number, etc. Look in the header comment to see how to use this as part of a JOIN, which is much faster than in a where clause.
The CLR solution did not give me an good performance so I will use a recursive query. So here is the definition of the SP I will use (mostly based on Erland Sommarskogs examples):
CREATE FUNCTION [dbo].[priudf_CSVtoIntTable]
(
#CSV VARCHAR(MAX),
#Delimiter CHAR(1) = ','
)
RETURNS
#Result TABLE
(
[Value] INT
)
AS
BEGIN
-- Remove space, tab, linefeed, carrier return
SET #CSV = REPLACE(#CSV, ' ', '');
SET #CSV = REPLACE(#CSV, CHAR(9), '');
SET #CSV = REPLACE(#CSV, CHAR(10), '');
SET #CSV = REPLACE(#CSV, CHAR(13), '');
WITH csvtbl(start, stop) AS
(
SELECT start = CONVERT(BIGINT, 1),
stop = CHARINDEX(#Delimiter, #CSV + #Delimiter)
UNION ALL
SELECT start = stop + 1,
stop = CHARINDEX(#Delimiter, #CSV + #Delimiter, stop + 1)
FROM csvtbl
WHERE stop > 0
)
INSERT INTO #Result
SELECT CAST(SUBSTRING(#CSV, start, CASE WHEN stop > 0 THEN stop - start ELSE 0 END) AS INT) AS [Value]
FROM csvtbl
WHERE stop > 0
OPTION (MAXRECURSION 1000)
RETURN
END
Thank for the input, I have to admit that I have made som bad research before I started my work. I found that Erland Sommarskog has written a lot of this problem on his webpage, after your responeses and after reading his page I decided that I will try to make a CLR to solve this.
I tried a recursive query, this resulted in good performance but I will try CLR function anyway.

Resources