Related
I have a need to create a function the will return nth element of a delimited string.
For a data migration project, I am converting JSON audit records stored in a SQL Server database into a structured report using SQL script. Goal is to deliver a sql script and a sql function used by the script without any code.
(This is a short-term fix will be used while a new auditing feature is added the ASP.NET/MVC application)
There is no shortage of delimited string to table examples available.
I've chosen a Common Table Expression example http://www.sqlperformance.com/2012/07/t-sql-queries/split-strings
Example: I want to return 67 from '1,222,2,67,888,1111'
This is the easiest answer to rerieve the 67 (type-safe!!):
SELECT CAST('<x>' + REPLACE('1,222,2,67,888,1111',',','</x><x>') + '</x>' AS XML).value('/x[4]','int')
In the following you will find examples how to use this with variables for the string, the delimiter and the position (even for edge-cases with XML-forbidden characters)
The easy one
This question is not about a string split approach, but about how to get the nth element. The easiest, fully inlineable way would be this IMO:
This is a real one-liner to get part 2 delimited by a space:
DECLARE #input NVARCHAR(100)=N'part1 part2 part3';
SELECT CAST(N'<x>' + REPLACE(#input,N' ',N'</x><x>') + N'</x>' AS XML).value('/x[2]','nvarchar(max)')
Variables can be used with sql:variable() or sql:column()
Of course you can use variables for delimiter and position (use sql:column to retrieve the position directly from a query's value):
DECLARE #dlmt NVARCHAR(10)=N' ';
DECLARE #pos INT = 2;
SELECT CAST(N'<x>' + REPLACE(#input,#dlmt,N'</x><x>') + N'</x>' AS XML).value('/x[sql:variable("#pos")][1]','nvarchar(max)')
Edge-Case with XML-forbidden characters
If your string might include forbidden characters, you still can do it this way. Just use FOR XML PATH on your string first to replace all forbidden characters with the fitting escape sequence implicitly.
It's a very special case if - additionally - your delimiter is the semicolon. In this case I replace the delimiter first to '#DLMT#', and replace this to the XML tags finally:
SET #input=N'Some <, > and &;Other äöü#€;One more';
SET #dlmt=N';';
SELECT CAST(N'<x>' + REPLACE((SELECT REPLACE(#input,#dlmt,'#DLMT#') AS [*] FOR XML PATH('')),N'#DLMT#',N'</x><x>') + N'</x>' AS XML).value('/x[sql:variable("#pos")][1]','nvarchar(max)');
UPDATE for SQL-Server 2016+
Regretfully the developers forgot to return the part's index with STRING_SPLIT. But, using SQL-Server 2016+, there is JSON_VALUE and OPENJSON.
With JSON_VALUE we can pass in the position as the index' array.
For OPENJSON the documentation states clearly:
When OPENJSON parses a JSON array, the function returns the indexes of the elements in the JSON text as keys.
A string like 1,2,3 needs nothing more than brackets: [1,2,3].
A string of words like this is an example needs to be ["this","is","an"," example"].
These are very easy string operations. Just try it out:
DECLARE #str VARCHAR(100)='Hello John Smith';
DECLARE #position INT = 2;
--We can build the json-path '$[1]' using CONCAT
SELECT JSON_VALUE('["' + REPLACE(#str,' ','","') + '"]',CONCAT('$[',#position-1,']'));
--See this for a position safe string-splitter (zero-based):
SELECT JsonArray.[key] AS [Position]
,JsonArray.[value] AS [Part]
FROM OPENJSON('["' + REPLACE(#str,' ','","') + '"]') JsonArray
In this post I tested various approaches and found, that OPENJSON is really fast. Even much faster than the famous "delimitedSplit8k()" method...
UPDATE 2 - Get the values type-safe
We can use an array within an array simply by using doubled [[]]. This allows for a typed WITH-clause:
DECLARE #SomeDelimitedString VARCHAR(100)='part1|1|20190920';
DECLARE #JsonArray NVARCHAR(MAX)=CONCAT('[["',REPLACE(#SomeDelimitedString,'|','","'),'"]]');
SELECT #SomeDelimitedString AS TheOriginal
,#JsonArray AS TransformedToJSON
,ValuesFromTheArray.*
FROM OPENJSON(#JsonArray)
WITH(TheFirstFragment VARCHAR(100) '$[0]'
,TheSecondFragment INT '$[1]'
,TheThirdFragment DATE '$[2]') ValuesFromTheArray
Here is my initial solution...
It is based on work by Aaron Bertrand http://www.sqlperformance.com/2012/07/t-sql-queries/split-strings
I simply changed the return type to make it a scalar function.
Example:
SELECT dbo.GetSplitString_CTE('1,222,2,67,888,1111',',',4)
CREATE FUNCTION dbo.GetSplitString_CTE
(
#List VARCHAR(MAX),
#Delimiter VARCHAR(255),
#ElementNumber int
)
RETURNS VARCHAR(4000)
AS
BEGIN
DECLARE #result varchar(4000)
DECLARE #Items TABLE ( position int IDENTITY PRIMARY KEY,
Item VARCHAR(4000)
)
DECLARE #ll INT = LEN(#List) + 1, #ld INT = LEN(#Delimiter);
WITH a AS
(
SELECT
[start] = 1,
[end] = COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, #ld), 0), #ll),
[value] = SUBSTRING(#List, 1,
COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, #ld), 0), #ll) - 1)
UNION ALL
SELECT
[start] = CONVERT(INT, [end]) + #ld,
[end] = COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, [end] + #ld), 0), #ll),
[value] = SUBSTRING(#List, [end] + #ld,
COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, [end] + #ld), 0), #ll)-[end]-#ld)
FROM a
WHERE [end] < #ll
)
INSERT #Items SELECT [value]
FROM a
WHERE LEN([value]) > 0
OPTION (MAXRECURSION 0);
SELECT #result=Item
FROM #Items
WHERE position=#ElementNumber
RETURN #result;
END
GO
How about:
CREATE FUNCTION dbo.NTH_ELEMENT (#Input NVARCHAR(MAX), #Delim CHAR = '-', #N INT = 0)
RETURNS NVARCHAR(MAX)
AS
BEGIN
RETURN (SELECT VALUE FROM STRING_SPLIT(#Input, #Delim) ORDER BY (SELECT NULL) OFFSET #N ROWS FETCH NEXT 1 ROW ONLY)
END
On Azure SQL Database, and on SQL Server 2022, STRING_SPLIT now has an optional ordinal parameter. If the parameter is omitted, or 0 is passed, then the function acts as it did before, and just returns a value column and the order is not guaranteed. If you pass the parameter with the value 1 then the function returns 2 columns, value, and ordinal which (unsurprisingly) provides the ordinal position of the value within the string.
So, if you wanted the 4th delimited value from the string '1,222,2,67,888,1111' you could do the following:
SELECT [value]
FROM STRING_SPLIT('1,222,2,67,888,1111',',',1)
WHERE ordinal = 4;
If the value was in a column, it would look like this:
SELECT SS.[value]
FROM dbo.YourTable YT
CROSS APPLY STRING_SPLIT(YT.YourColumn,',',1) SS
WHERE SS.ordinal = 4;
#a - the value (f.e. 'a/bb/ccc/dddd/ee/ff/....')
#p - the desired position (1,2,3...)
#d - the delimeter ( '/' )
trim(substring(replace(#a,#d,replicate(' ',len(#a))),(#p-1)*len(#a)+1,len(#a)))
only problem is - if desired part has trailing or leading blanks they get trimmed.
Completely Based on article from https://exceljet.net/formula/split-text-with-delimiter
In a rare moment of lunacy I just thought that split is far easier if we use XML to parse it out for us:
(Using the variables from #Gary Kindel's answer)
declare #xml xml
set #xml = '<split><el>' + replace(#list,#Delimiter,'</el><el>') + '</el></split>'
select
el = split.el.value('.','varchar(max)')
from #xml.nodes('/split/el') split(el))
This lists all elements of the string, split by the specified character.
We can use an xpath test to filter out empty values, and a further xpath test to restrict this to the element we're interested in. In full Gary's function becomes:
alter FUNCTION dbo.GetSplitString_CTE
(
#List VARCHAR(MAX),
#Delimiter VARCHAR(255),
#ElementNumber int
)
RETURNS VARCHAR(max)
AS
BEGIN
-- escape any XML https://dba.stackexchange.com/a/143140/65992
set #list = convert(VARCHAR(MAX),(select #list for xml path(''), type));
declare #xml xml
set #xml = '<split><el>' + replace(#list,#Delimiter,'</el><el>') + '</el></split>'
declare #ret varchar(max)
set #ret = (select
el = split.el.value('.','varchar(max)')
from #xml.nodes('/split/el[string-length(.)>0][position() = sql:variable("#elementnumber")]') split(el))
return #ret
END
you can put this select into UFN. if you need you can customize it for specifying delimiter as well. in that case your ufn will have two input. number Nth and delimiter to use.
DECLARE #tlist varchar(max)='10,20,30,40,50,60,70,80,90,100'
DECLARE #i INT=1, #nth INT=3
While len(#tlist) <> 0
BEGIN
IF #i=#nth
BEGIN
select Case when charindex(',',#tlist) <> 0 Then LEFT(#tlist,charindex(',',#tlist)-1)
Else #tlist
END
END
Select #tlist = Case when charindex(',',#tlist) <> 0 Then substring(#tlist,charindex(',',#tlist)+1,len(#tlist))
Else ''
END
SELECT #i=#i+1
END
Alternatively, one can use xml, nodes() and ROW_NUMBER. We can order the elements based on their document order. For example:
DECLARE #Input VARCHAR(100) = '1a,2b,3c,4d,5e,6f,7g,8h'
,#Number TINYINT = 3
DECLARE #XML XML;
DECLARE #value VARCHAR(100);
SET #XML = CAST('<x>' + REPLACE(#Input,',','</x><x>') + '</x>' AS XML);
WITH DataSource ([rowID], [rowValue]) AS
(
SELECT ROW_NUMBER() OVER (ORDER BY T.c ASC)
,T.c.value('.', 'VARCHAR(100)')
FROM #XML.nodes('./x') T(c)
)
SELECT #value = [rowValue]
FROM DataSource
WHERE [rowID] = #Number;
SELECT #value;
I would rather create a temp table with an identity column and fill it up with output from the SPLIT function.
CREATE TABLE #tblVals(Id INT IDENTITY(1,1), Val NVARCHAR(100))
INSERT INTO #tblVals (Val)
SELECT [value] FROM STRING_SPLIT('Val1-Val3-Val2-Val5', '-')
SELECT * FROM #tblVals
Now you can easily do something like below.
DECLARE #val2 NVARCHAR(100) = (SELECT TOP 1 Val FROM #tblVals WHERE Id = 2)
See the snapshot below:
You can use STRING_SPLIT with ROW_NUMBER:
SELECT value, idx FROM
(
SELECT
value,
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) idx
FROM STRING_SPLIT('Lorem ipsum dolor sit amet.', ' ')
) t
WHERE idx=2
returns second element (idx=2): 'ipsum'
We have the answer over below url.
DECLARE # AS VARCHAR(MAX) = 'Pawan1,Pawan2,Pawan4,Pawan3'
SELECT VALUE FROM
(
SELECT VALUE , ROW_NUMBER() OVER (ORDER BY (SELECT null)) rnk FROM STRING_SPLIT(#, ',')
)x where rnk = 3
GO
https://msbiskills.com/2018/06/15/sql-puzzle-multiple-ways-to-split-a-string-and-get-nth-row-xml-advanced-sql/
I don't have enough reputation to comment, so I am adding an answer. Please adjust as appropriate.
I have a problem with Gary Kindel's answer for cases where there is nothing between the two delimiters
If you do
select * from dbo.GetSplitString_CTE('abc^def^^ghi','^',3)
you get
ghi
instead of an empty string
If you comment out the
WHERE LEN([value]) > 0
line, you get the desired result
I cannot comment on Gary's solution because of my low reputation
I know Gary was referencing another link.
I have struggled to understand why we need this variable
#ld INT = LEN(#Delimiter)
I also don't understand why charindex has to start at the position of length of delimiter, #ld
I tested with many examples with a single character delimiter, and they work. Most of the time, delimiter character is a single character. However, since the developer included the ld as length of delimiter, the code has to work for delimiters that have more than one character
In this case, the following case will fail
11,,,22,,,33,,,44,,,55,,,
I cloned from the codes from this link. http://codebetter.com/raymondlewallen/2005/10/26/quick-t-sql-to-parse-a-delimited-string/
I have tested various scenarios including the delimiters that have more than one character
alter FUNCTION [dbo].[split1]
(
#string1 VARCHAR(8000) -- List of delimited items
, #Delimiter VARCHAR(40) = ',' -- delimiter that separates items
, #ElementNumber int
)
RETURNS varchar(8000)
AS
BEGIN
declare #position int
declare #piece varchar(8000)=''
declare #returnVal varchar(8000)=''
declare #Pattern varchar(50) = '%' + #Delimiter + '%'
declare #counter int =0
declare #ld int = len(#Delimiter)
declare #ls1 int = len (#string1)
declare #foundit int = 0
if patindex(#Pattern , #string1) = 0
return ''
if right(rtrim(#string1),1) <> #Delimiter
set #string1 = #string1 + #Delimiter
set #position = patindex(#Pattern , #string1) + #ld -1
while #position > 0
begin
set #counter = #counter +1
set #ls1 = len (#string1)
if (#ls1 >= #ld)
set #piece = left(#string1, #position - #ld)
else
break
if (#counter = #ElementNumber)
begin
set #foundit = 1
break
end
if len(#string1) > 0
begin
set #string1 = stuff(#string1, 1, #position, '')
set #position = patindex(#Pattern , #string1) + #ld -1
end
else
set #position = -1
end
if #foundit =1
set #returnVal = #piece
else
set #returnVal = ''
return #returnVal
you can create simple table variable and use it as below
Declare #tbl_split Table (Id INT IDENTITY(1,1), VAL VARCHAR(50))
INSERT #tbl_split SELECT VALUE
FROM string_split('999999:01', ':')
Select val from #tbl_split
WHERE Id=2
SELECT REPLACE('<strong>100</strong><b>.00 GB', '%^(^-?\d*\.{0,1}\d+$)%', '');
I want to replace any markup between two parts of the number with above regex, but it does not seem to work. I'm not sure if it is regex syntax that's wrong because I tried simpler one such as '%[^0-9]%' just to test but it didn't work either. Does anyone know how can I achieve this?
You can use PATINDEX
to find the first index of the pattern (string's) occurrence. Then use STUFF to stuff another string into the pattern(string) matched.
Loop through each row. Replace each illegal characters with what you want. In your case replace non numeric with blank. The inner loop is if you have more than one illegal character in a current cell that of the loop.
DECLARE #counter int
SET #counter = 0
WHILE(#counter < (SELECT MAX(ID_COLUMN) FROM Table))
BEGIN
WHILE 1 = 1
BEGIN
DECLARE #RetVal varchar(50)
SET #RetVal = (SELECT Column = STUFF(Column, PATINDEX('%[^0-9.]%', Column),1, '')
FROM Table
WHERE ID_COLUMN = #counter)
IF(#RetVal IS NOT NULL)
UPDATE Table SET
Column = #RetVal
WHERE ID_COLUMN = #counter
ELSE
break
END
SET #counter = #counter + 1
END
Caution: This is slow though! Having a varchar column may impact. So using LTRIM RTRIM may help a bit. Regardless, it is slow.
Credit goes to this StackOverFlow answer.
EDIT
Credit also goes to #srutzky
Edit (by #Tmdean)
Instead of doing one row at a time, this answer can be adapted to a more set-based solution. It still iterates the max of the number of non-numeric characters in a single row, so it's not ideal, but I think it should be acceptable in most situations.
WHILE 1 = 1 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, '')
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 BREAK;
END;
You can also improve efficiency quite a lot if you maintain a bit column in the table that indicates whether the field has been scrubbed yet. (NULL represents "Unknown" in my example and should be the column default.)
DECLARE #done bit = 0;
WHILE #done = 0 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table
WHERE COALESCE(Scrubbed_Column, 0) = 0)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, ''),
Scrubbed_Column = 0
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 SET #done = 1;
-- if Scrubbed_Column is still NULL, then the PATINDEX
-- must have given 0
UPDATE table
SET Scrubbed_Column = CASE
WHEN Scrubbed_Column IS NULL THEN 1
ELSE NULLIF(Scrubbed_Column, 0)
END;
END;
If you don't want to change your schema, this is easy to adapt to store intermediate results in a table valued variable which gets applied to the actual table at the end.
Instead of stripping out the found character by its sole position, using Replace(Column, BadFoundCharacter, '') could be substantially faster. Additionally, instead of just replacing the one bad character found next in each column, this replaces all those found.
WHILE 1 = 1 BEGIN
UPDATE dbo.YourTable
SET Column = Replace(Column, Substring(Column, PatIndex('%[^0-9.-]%', Column), 1), '')
WHERE Column LIKE '%[^0-9.-]%'
If ##RowCount = 0 BREAK;
END;
I am convinced this will work better than the accepted answer, if only because it does fewer operations. There are other ways that might also be faster, but I don't have time to explore those right now.
In a general sense, SQL Server does not support regular expressions and you cannot use them in the native T-SQL code.
You could write a CLR function to do that. See here, for example.
For those looking for a performant and easy solution and are willing to enable CLR:
CREATE database TestSQLFunctions
go
use TestSQLFunctions
go
ALTER database TestSQLFunctions set trustworthy on
EXEC sp_configure 'clr enabled', 1
RECONFIGURE WITH OVERRIDE
go
CREATE ASSEMBLY [SQLFunctions]
AUTHORIZATION [dbo]
FROM 0x4D5A90000300000004000000FFFF0000B800000000000000400000000000000000000000000000000000000000000000000000000000000000000000800000000E1FBA0E00B409CD21B8014CCD21546869732070726F6772616D2063616E6E6F742062652072756E20696E20444F53206D6F64652E0D0D0A2400000000000000504500004C0103004BE8B85F0000000000000000E00022200B013000000800000006000000000000C2270000002000000040000000000010002000000002000004000000000000000600000000000000008000000002000000000000030060850000100000100000000010000010000000000000100000000000000000000000702700004F000000004000009803000000000000000000000000000000000000006000000C00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000080000000000000000000000082000004800000000000000000000002E74657874000000C8070000002000000008000000020000000000000000000000000000200000602E72737263000000980300000040000000040000000A0000000000000000000000000000400000402E72656C6F6300000C0000000060000000020000000E00000000000000000000000000004000004200000000000000000000000000000000A4270000000000004800000002000500682000000807000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003A022D02022A020304281000000A2A1E02281100000A2A0042534A4201000100000000000C00000076342E302E33303331390000000005006C00000018020000237E000084020000CC02000023537472696E6773000000005005000004000000235553005405000010000000234755494400000064050000A401000023426C6F620000000000000002000001471500000900000000FA0133001600000100000013000000020000000200000003000000110000000F00000001000000030000000000CA01010000000000060025014F02060092014F02060044001D020F006F02000006006C00E20106000801E2010600D400E20106007901E20106004501E20106005E01E20106008300E2010600580030020600360030020600B700E20106009E00B0010600AA02DB010A00F300FC010A001F00FC010E00C3027E02000000000100000000000100010001001000C3029D0241000100010050200000000096002E001A0001005F2000000000861817020600040000000100BD0200000200F40100000300B102090017020100110017020600190017020A0029001702100031001702100039001702100041001702100049001702100051001702100059001702100061001702150069001702100071001702100079001702100089001702060099002E001A0081001702060020007B0010012E000B002A002E00130033002E001B0052002E0023005B002E002B006D002E0033006D002E003B006D002E0043005B002E004B0073002E0053006D002E005B006D002E0063008B002E006B00B5002E007300C2000480000001000000000000000000000000009D020000040000000000000000000000210016000000000004000000000000000000000021000A00000000000400000000000000000000002100DB010000000000000000003C4D6F64756C653E0053797374656D2E44617461006D73636F726C696200446174614163636573734B696E64005265706C61636500477569644174747269627574650044656275676761626C6541747472696275746500436F6D56697369626C6541747472696275746500417373656D626C795469746C6541747472696275746500417373656D626C7954726164656D61726B417474726962757465005461726765744672616D65776F726B41747472696275746500417373656D626C7946696C6556657273696F6E41747472696275746500417373656D626C79436F6E66696775726174696F6E4174747269627574650053716C46756E6374696F6E41747472696275746500417373656D626C794465736372697074696F6E41747472696275746500436F6D70696C6174696F6E52656C61786174696F6E7341747472696275746500417373656D626C7950726F6475637441747472696275746500417373656D626C79436F7079726967687441747472696275746500417373656D626C79436F6D70616E794174747269627574650052756E74696D65436F6D7061746962696C6974794174747269627574650053797374656D2E52756E74696D652E56657273696F6E696E670053514C46756E6374696F6E732E646C6C0053797374656D0053797374656D2E5265666C656374696F6E007061747465726E004D6963726F736F66742E53716C5365727665722E536572766572002E63746F720053797374656D2E446961676E6F73746963730053797374656D2E52756E74696D652E496E7465726F7053657276696365730053797374656D2E52756E74696D652E436F6D70696C6572536572766963657300446562756767696E674D6F6465730053797374656D2E546578742E526567756C617245787072657373696F6E730053514C46756E6374696F6E73004F626A656374007265706C6163656D656E7400696E70757400526567657800000000000000003A1617E607071B47B964858BCD87458B00042001010803200001052001011111042001010E04200101020600030E0E0E0E08B77A5C561934E0890801000800000000001E01000100540216577261704E6F6E457863657074696F6E5468726F7773010801000200000000001101000C53514C46756E6374696F6E73000005010000000017010012436F7079726967687420C2A920203230323000002901002434346436386231632D393735312D343938612D396665352D32316666333934303738303900000C010007312E302E302E3000004D01001C2E4E45544672616D65776F726B2C56657273696F6E3D76342E352E320100540E144672616D65776F726B446973706C61794E616D65142E4E4554204672616D65776F726B20342E352E32808F010001005455794D6963726F736F66742E53716C5365727665722E5365727665722E446174614163636573734B696E642C2053797374656D2E446174612C2056657273696F6E3D342E302E302E302C2043756C747572653D6E65757472616C2C205075626C69634B6579546F6B656E3D623737613563353631393334653038390A4461746141636365737301000000000000982700000000000000000000B2270000002000000000000000000000000000000000000000000000A4270000000000000000000000005F436F72446C6C4D61696E006D73636F7265652E646C6C0000000000FF25002000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001001000000018000080000000000000000000000000000001000100000030000080000000000000000000000000000001000000000048000000584000003C03000000000000000000003C0334000000560053005F00560045005200530049004F004E005F0049004E0046004F0000000000BD04EFFE00000100000001000000000000000100000000003F000000000000000400000002000000000000000000000000000000440000000100560061007200460069006C00650049006E0066006F00000000002400040000005400720061006E0073006C006100740069006F006E00000000000000B0049C020000010053007400720069006E006700460069006C00650049006E0066006F0000007802000001003000300030003000300034006200300000001A000100010043006F006D006D0065006E007400730000000000000022000100010043006F006D00700061006E0079004E0061006D006500000000000000000042000D000100460069006C0065004400650073006300720069007000740069006F006E0000000000530051004C00460075006E006300740069006F006E00730000000000300008000100460069006C006500560065007200730069006F006E000000000031002E0030002E0030002E003000000042001100010049006E007400650072006E0061006C004E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000004800120001004C006500670061006C0043006F007000790072006900670068007400000043006F0070007900720069006700680074002000A90020002000320030003200300000002A00010001004C006500670061006C00540072006100640065006D00610072006B00730000000000000000004A00110001004F0072006900670069006E0061006C00460069006C0065006E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000003A000D000100500072006F0064007500630074004E0061006D00650000000000530051004C00460075006E006300740069006F006E00730000000000340008000100500072006F006400750063007400560065007200730069006F006E00000031002E0030002E0030002E003000000038000800010041007300730065006D0062006C0079002000560065007200730069006F006E00000031002E0030002E0030002E0030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000C000000C43700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
WITH PERMISSION_SET = SAFE
go
CREATE FUNCTION RegexReplace(
#input nvarchar(max),
#pattern nvarchar(max),
#replacement nvarchar(max)
) RETURNS nvarchar (max)
AS EXTERNAL NAME SQLFunctions.[SQLFunctions.Regex].Replace;
go
-- outputs This is a test
SELECT dbo.RegexReplace('This is a test 12345','[0-9]','')
Content of the DLL:
I stumbled across this post looking for something else but thought I'd mention a solution I use which is far more efficient - and really should be the default implementation of any function when used with a set based query - which is to use a cross applied table function. Seems the topic is still active so hopefully this is useful to someone.
Example runtime on a few of the answers so far based on running recursive set based queries or scalar function, based on 1m rows test set removing the chars from a random newid, ranges from 34s to 2m05s for the WHILE loop examples and from 1m3s to {forever} for the function examples.
Using a table function with cross apply achieves the same goal in 10s. You may need to adjust it to suit your needs such as the max length it handles.
Function:
CREATE FUNCTION [dbo].[RemoveChars](#InputUnit VARCHAR(40))
RETURNS TABLE
AS
RETURN
(
WITH Numbers_prep(Number) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,Numbers(Number) AS
(
SELECT TOP (ISNULL(LEN(#InputUnit),0))
row_number() OVER (ORDER BY (SELECT NULL))
FROM Numbers_prep a
CROSS JOIN Numbers_prep b
)
SELECT
OutputUnit
FROM
(
SELECT
substring(#InputUnit,Number,1)
FROM Numbers
WHERE substring(#InputUnit,Number,1) like '%[0-9]%'
ORDER BY Number
FOR XML PATH('')
) Sub(OutputUnit)
)
Usage:
UPDATE t
SET column = o.OutputUnit
FROM ##t t
CROSS APPLY [dbo].[RemoveChars](t.column) o
Here is a function I wrote to accomplish this based off of the previous answers.
CREATE FUNCTION dbo.RepetitiveReplace
(
#P_String VARCHAR(MAX),
#P_Pattern VARCHAR(MAX),
#P_ReplaceString VARCHAR(MAX),
#P_ReplaceLength INT = 1
)
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #Index INT;
-- Get starting point of pattern
SET #Index = PATINDEX(#P_Pattern, #P_String);
while #Index > 0
begin
--replace matching charactger at index
SET #P_String = STUFF(#P_String, PATINDEX(#P_Pattern, #P_String), #P_ReplaceLength, #P_ReplaceString);
SET #Index = PATINDEX(#P_Pattern, #P_String);
end
RETURN #P_String;
END;
[Gist][1]
[1]: https://gist.github.com/jkdba/ca13fe8f2a9855c4bdbfd0a5d3dfcda2
Edit:
Originally I had a recursive function here which does not play well with sql server as it has a 32 nesting level limit which would result in an error like the below any time you attempt to make 32+ replacements with the function. Instead of trying to make a server level change to allow more nesting (which could be dangerous like allow never ending loops) switching to a while loop makes a lot more sense.
Maximum stored procedure, function, trigger, or view nesting level exceeded (limit 32).
Wrapping the solution inside a SQL function could be useful if you want to reuse it.
I'm even doing it at the cell level, that's why I'm putting this as a different answer:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(300))
RETURNS VARCHAR(300)
BEGIN
DECLARE #str VARCHAR(300) = #string;
DECLARE #Pattern VARCHAR (20) = '%[^a-zA-Z0-9]%';
DECLARE #Len INT;
SELECT #Len = LEN(#String);
WHILE #Len > 0
BEGIN
SET #Len = #Len - 1;
IF (PATINDEX(#Pattern,#str) > 0)
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
A more speedy approach for large strings would look something like this:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(MAX))
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #str VARCHAR(MAX) = #string;
DECLARE #Pattern VARCHAR (MAX) = '%[^a-zA-Z0-9]%';
WHILE PATINDEX(#Pattern,#str) > 0
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
RETURN #str
END
I've created this function to clean up a string that contained non numeric characters in a time field. The time contained question marks when they did not added the minutes, something like this 20:??. Function loops through each character and replaces the ? with a 0 :
CREATE FUNCTION [dbo].[CleanTime]
(
-- Add the parameters for the function here
#intime nvarchar(10)
)
RETURNS nvarchar(5)
AS
BEGIN
-- Declare the return variable here
DECLARE #ResultVar nvarchar(5)
DECLARE #char char(1)
-- Add the T-SQL statements to compute the return value here
DECLARE #i int = 1
WHILE #i <= LEN(#intime)
BEGIN
SELECT #char = CASE WHEN substring(#intime,#i,1) like '%[0-9:]%' THEN substring(#intime,#i,1) ELSE '0' END
SELECT #ResultVar = concat(#ResultVar,#char)
set #i = #i + 1
END;
-- Return the result of the function
RETURN #ResultVar
END
I think this solution is faster and simple. I use always CTE/recursive because WHILE is so slow on SQL Server.
I use it in projects I work with and large databases.
/*
Function: dbo.kSql_ReplaceRegExp
Create Date: 20.02.2021
Author: Karcan Ozbal
Description: The given string value will be replaced according to the given regexp/pattern.
Parameter(s): #Value : Value/Text to REPLACE.
#RegExp : The regexp/pattern to be used for REPLACE operation.
Usage: select dbo.kSql_ReplaceRegExp('2T3EST5','%[0-9]%')
Output: 'TEST'
*/
ALTER FUNCTION [dbo].[kSql_ReplaceRegExp](
#Value nvarchar(max),
#RegExp nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #Result nvarchar(max)
;WITH CTE AS (
SELECT NUM = 1, VALUE = #Value, IDX = PATINDEX(#RegExp, #Value)
UNION ALL
SELECT NUM + 1, VALUE = REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''), IDX = PATINDEX(#RegExp, REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''))
FROM CTE
WHERE IDX > 0
)
SELECT TOP(1) #Result = VALUE
FROM CTE
ORDER BY NUM DESC
OPTION (maxrecursion 0)
RETURN #Result
END
If you are doing this just for a parameter coming into a Stored Procedure, you can use the following:
declare #badIndex int
set #badIndex = PatIndex('%[^0-9]%', #Param)
while #badIndex > 0
set #Param = Replace(#Param, Substring(#Param, #badIndex, 1), '')
set #badIndex = PatIndex('%[^0-9]%', #Param)
I thought this was clearer:
ALTER FUNCTION [dbo].[func_ReplaceChars](
#Value nvarchar(max),
#Chars nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #cLen int = len(#Chars);
DECLARE #curChar int = 0;
WHILE #curChar<#cLen
BEGIN
set #Value = replace(#Value,substring(#Chars,#curChar,1),'');
set #curChar = #curChar + 1;
END;
RETURN #Value
END
I'm using this code similar to several codes above:
DROP FUNCTION [dbo].[fnCleanString]
GO
CREATE FUNCTION [dbo].[fnCleanString] (#input VARCHAR(max), #Pattern
VARCHAR (20))
RETURNS VARCHAR(max)
BEGIN
DECLARE #str VARCHAR(max) = #input;
DECLARE #Len INT;
DECLARE #INDEX INT;
SELECT #Len = LEN(#input);
WHILE #Len > 0
BEGIN
SET #INDEX = PATINDEX(#Pattern,#str);
IF (#INDEX > 0)
BEGIN
SET #str=REPLACE(#str,SUBSTRING(#str,#INDEX, 1), '');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
You can use it like this:
SELECT CleanName = dbo.[fnCleanString](Name, '%[0-9]%') from YourTable
I think a simpler and faster approach is iterate by each character of the alphabet:
DECLARE #i int
SET #i = 0
WHILE(#i < 256)
BEGIN
IF char(#i) NOT IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.')
UPDATE Table SET Column = replace(Column, char(#i), '')
SET #i = #i + 1
END
SELECT REPLACE('<strong>100</strong><b>.00 GB', '%^(^-?\d*\.{0,1}\d+$)%', '');
I want to replace any markup between two parts of the number with above regex, but it does not seem to work. I'm not sure if it is regex syntax that's wrong because I tried simpler one such as '%[^0-9]%' just to test but it didn't work either. Does anyone know how can I achieve this?
You can use PATINDEX
to find the first index of the pattern (string's) occurrence. Then use STUFF to stuff another string into the pattern(string) matched.
Loop through each row. Replace each illegal characters with what you want. In your case replace non numeric with blank. The inner loop is if you have more than one illegal character in a current cell that of the loop.
DECLARE #counter int
SET #counter = 0
WHILE(#counter < (SELECT MAX(ID_COLUMN) FROM Table))
BEGIN
WHILE 1 = 1
BEGIN
DECLARE #RetVal varchar(50)
SET #RetVal = (SELECT Column = STUFF(Column, PATINDEX('%[^0-9.]%', Column),1, '')
FROM Table
WHERE ID_COLUMN = #counter)
IF(#RetVal IS NOT NULL)
UPDATE Table SET
Column = #RetVal
WHERE ID_COLUMN = #counter
ELSE
break
END
SET #counter = #counter + 1
END
Caution: This is slow though! Having a varchar column may impact. So using LTRIM RTRIM may help a bit. Regardless, it is slow.
Credit goes to this StackOverFlow answer.
EDIT
Credit also goes to #srutzky
Edit (by #Tmdean)
Instead of doing one row at a time, this answer can be adapted to a more set-based solution. It still iterates the max of the number of non-numeric characters in a single row, so it's not ideal, but I think it should be acceptable in most situations.
WHILE 1 = 1 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, '')
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 BREAK;
END;
You can also improve efficiency quite a lot if you maintain a bit column in the table that indicates whether the field has been scrubbed yet. (NULL represents "Unknown" in my example and should be the column default.)
DECLARE #done bit = 0;
WHILE #done = 0 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table
WHERE COALESCE(Scrubbed_Column, 0) = 0)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, ''),
Scrubbed_Column = 0
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 SET #done = 1;
-- if Scrubbed_Column is still NULL, then the PATINDEX
-- must have given 0
UPDATE table
SET Scrubbed_Column = CASE
WHEN Scrubbed_Column IS NULL THEN 1
ELSE NULLIF(Scrubbed_Column, 0)
END;
END;
If you don't want to change your schema, this is easy to adapt to store intermediate results in a table valued variable which gets applied to the actual table at the end.
Instead of stripping out the found character by its sole position, using Replace(Column, BadFoundCharacter, '') could be substantially faster. Additionally, instead of just replacing the one bad character found next in each column, this replaces all those found.
WHILE 1 = 1 BEGIN
UPDATE dbo.YourTable
SET Column = Replace(Column, Substring(Column, PatIndex('%[^0-9.-]%', Column), 1), '')
WHERE Column LIKE '%[^0-9.-]%'
If ##RowCount = 0 BREAK;
END;
I am convinced this will work better than the accepted answer, if only because it does fewer operations. There are other ways that might also be faster, but I don't have time to explore those right now.
In a general sense, SQL Server does not support regular expressions and you cannot use them in the native T-SQL code.
You could write a CLR function to do that. See here, for example.
For those looking for a performant and easy solution and are willing to enable CLR:
CREATE database TestSQLFunctions
go
use TestSQLFunctions
go
ALTER database TestSQLFunctions set trustworthy on
EXEC sp_configure 'clr enabled', 1
RECONFIGURE WITH OVERRIDE
go
CREATE ASSEMBLY [SQLFunctions]
AUTHORIZATION [dbo]
FROM 0x4D5A90000300000004000000FFFF0000B800000000000000400000000000000000000000000000000000000000000000000000000000000000000000800000000E1FBA0E00B409CD21B8014CCD21546869732070726F6772616D2063616E6E6F742062652072756E20696E20444F53206D6F64652E0D0D0A2400000000000000504500004C0103004BE8B85F0000000000000000E00022200B013000000800000006000000000000C2270000002000000040000000000010002000000002000004000000000000000600000000000000008000000002000000000000030060850000100000100000000010000010000000000000100000000000000000000000702700004F000000004000009803000000000000000000000000000000000000006000000C00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000080000000000000000000000082000004800000000000000000000002E74657874000000C8070000002000000008000000020000000000000000000000000000200000602E72737263000000980300000040000000040000000A0000000000000000000000000000400000402E72656C6F6300000C0000000060000000020000000E00000000000000000000000000004000004200000000000000000000000000000000A4270000000000004800000002000500682000000807000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003A022D02022A020304281000000A2A1E02281100000A2A0042534A4201000100000000000C00000076342E302E33303331390000000005006C00000018020000237E000084020000CC02000023537472696E6773000000005005000004000000235553005405000010000000234755494400000064050000A401000023426C6F620000000000000002000001471500000900000000FA0133001600000100000013000000020000000200000003000000110000000F00000001000000030000000000CA01010000000000060025014F02060092014F02060044001D020F006F02000006006C00E20106000801E2010600D400E20106007901E20106004501E20106005E01E20106008300E2010600580030020600360030020600B700E20106009E00B0010600AA02DB010A00F300FC010A001F00FC010E00C3027E02000000000100000000000100010001001000C3029D0241000100010050200000000096002E001A0001005F2000000000861817020600040000000100BD0200000200F40100000300B102090017020100110017020600190017020A0029001702100031001702100039001702100041001702100049001702100051001702100059001702100061001702150069001702100071001702100079001702100089001702060099002E001A0081001702060020007B0010012E000B002A002E00130033002E001B0052002E0023005B002E002B006D002E0033006D002E003B006D002E0043005B002E004B0073002E0053006D002E005B006D002E0063008B002E006B00B5002E007300C2000480000001000000000000000000000000009D020000040000000000000000000000210016000000000004000000000000000000000021000A00000000000400000000000000000000002100DB010000000000000000003C4D6F64756C653E0053797374656D2E44617461006D73636F726C696200446174614163636573734B696E64005265706C61636500477569644174747269627574650044656275676761626C6541747472696275746500436F6D56697369626C6541747472696275746500417373656D626C795469746C6541747472696275746500417373656D626C7954726164656D61726B417474726962757465005461726765744672616D65776F726B41747472696275746500417373656D626C7946696C6556657273696F6E41747472696275746500417373656D626C79436F6E66696775726174696F6E4174747269627574650053716C46756E6374696F6E41747472696275746500417373656D626C794465736372697074696F6E41747472696275746500436F6D70696C6174696F6E52656C61786174696F6E7341747472696275746500417373656D626C7950726F6475637441747472696275746500417373656D626C79436F7079726967687441747472696275746500417373656D626C79436F6D70616E794174747269627574650052756E74696D65436F6D7061746962696C6974794174747269627574650053797374656D2E52756E74696D652E56657273696F6E696E670053514C46756E6374696F6E732E646C6C0053797374656D0053797374656D2E5265666C656374696F6E007061747465726E004D6963726F736F66742E53716C5365727665722E536572766572002E63746F720053797374656D2E446961676E6F73746963730053797374656D2E52756E74696D652E496E7465726F7053657276696365730053797374656D2E52756E74696D652E436F6D70696C6572536572766963657300446562756767696E674D6F6465730053797374656D2E546578742E526567756C617245787072657373696F6E730053514C46756E6374696F6E73004F626A656374007265706C6163656D656E7400696E70757400526567657800000000000000003A1617E607071B47B964858BCD87458B00042001010803200001052001011111042001010E04200101020600030E0E0E0E08B77A5C561934E0890801000800000000001E01000100540216577261704E6F6E457863657074696F6E5468726F7773010801000200000000001101000C53514C46756E6374696F6E73000005010000000017010012436F7079726967687420C2A920203230323000002901002434346436386231632D393735312D343938612D396665352D32316666333934303738303900000C010007312E302E302E3000004D01001C2E4E45544672616D65776F726B2C56657273696F6E3D76342E352E320100540E144672616D65776F726B446973706C61794E616D65142E4E4554204672616D65776F726B20342E352E32808F010001005455794D6963726F736F66742E53716C5365727665722E5365727665722E446174614163636573734B696E642C2053797374656D2E446174612C2056657273696F6E3D342E302E302E302C2043756C747572653D6E65757472616C2C205075626C69634B6579546F6B656E3D623737613563353631393334653038390A4461746141636365737301000000000000982700000000000000000000B2270000002000000000000000000000000000000000000000000000A4270000000000000000000000005F436F72446C6C4D61696E006D73636F7265652E646C6C0000000000FF25002000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001001000000018000080000000000000000000000000000001000100000030000080000000000000000000000000000001000000000048000000584000003C03000000000000000000003C0334000000560053005F00560045005200530049004F004E005F0049004E0046004F0000000000BD04EFFE00000100000001000000000000000100000000003F000000000000000400000002000000000000000000000000000000440000000100560061007200460069006C00650049006E0066006F00000000002400040000005400720061006E0073006C006100740069006F006E00000000000000B0049C020000010053007400720069006E006700460069006C00650049006E0066006F0000007802000001003000300030003000300034006200300000001A000100010043006F006D006D0065006E007400730000000000000022000100010043006F006D00700061006E0079004E0061006D006500000000000000000042000D000100460069006C0065004400650073006300720069007000740069006F006E0000000000530051004C00460075006E006300740069006F006E00730000000000300008000100460069006C006500560065007200730069006F006E000000000031002E0030002E0030002E003000000042001100010049006E007400650072006E0061006C004E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000004800120001004C006500670061006C0043006F007000790072006900670068007400000043006F0070007900720069006700680074002000A90020002000320030003200300000002A00010001004C006500670061006C00540072006100640065006D00610072006B00730000000000000000004A00110001004F0072006900670069006E0061006C00460069006C0065006E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000003A000D000100500072006F0064007500630074004E0061006D00650000000000530051004C00460075006E006300740069006F006E00730000000000340008000100500072006F006400750063007400560065007200730069006F006E00000031002E0030002E0030002E003000000038000800010041007300730065006D0062006C0079002000560065007200730069006F006E00000031002E0030002E0030002E0030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000C000000C43700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
WITH PERMISSION_SET = SAFE
go
CREATE FUNCTION RegexReplace(
#input nvarchar(max),
#pattern nvarchar(max),
#replacement nvarchar(max)
) RETURNS nvarchar (max)
AS EXTERNAL NAME SQLFunctions.[SQLFunctions.Regex].Replace;
go
-- outputs This is a test
SELECT dbo.RegexReplace('This is a test 12345','[0-9]','')
Content of the DLL:
I stumbled across this post looking for something else but thought I'd mention a solution I use which is far more efficient - and really should be the default implementation of any function when used with a set based query - which is to use a cross applied table function. Seems the topic is still active so hopefully this is useful to someone.
Example runtime on a few of the answers so far based on running recursive set based queries or scalar function, based on 1m rows test set removing the chars from a random newid, ranges from 34s to 2m05s for the WHILE loop examples and from 1m3s to {forever} for the function examples.
Using a table function with cross apply achieves the same goal in 10s. You may need to adjust it to suit your needs such as the max length it handles.
Function:
CREATE FUNCTION [dbo].[RemoveChars](#InputUnit VARCHAR(40))
RETURNS TABLE
AS
RETURN
(
WITH Numbers_prep(Number) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,Numbers(Number) AS
(
SELECT TOP (ISNULL(LEN(#InputUnit),0))
row_number() OVER (ORDER BY (SELECT NULL))
FROM Numbers_prep a
CROSS JOIN Numbers_prep b
)
SELECT
OutputUnit
FROM
(
SELECT
substring(#InputUnit,Number,1)
FROM Numbers
WHERE substring(#InputUnit,Number,1) like '%[0-9]%'
ORDER BY Number
FOR XML PATH('')
) Sub(OutputUnit)
)
Usage:
UPDATE t
SET column = o.OutputUnit
FROM ##t t
CROSS APPLY [dbo].[RemoveChars](t.column) o
Here is a function I wrote to accomplish this based off of the previous answers.
CREATE FUNCTION dbo.RepetitiveReplace
(
#P_String VARCHAR(MAX),
#P_Pattern VARCHAR(MAX),
#P_ReplaceString VARCHAR(MAX),
#P_ReplaceLength INT = 1
)
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #Index INT;
-- Get starting point of pattern
SET #Index = PATINDEX(#P_Pattern, #P_String);
while #Index > 0
begin
--replace matching charactger at index
SET #P_String = STUFF(#P_String, PATINDEX(#P_Pattern, #P_String), #P_ReplaceLength, #P_ReplaceString);
SET #Index = PATINDEX(#P_Pattern, #P_String);
end
RETURN #P_String;
END;
[Gist][1]
[1]: https://gist.github.com/jkdba/ca13fe8f2a9855c4bdbfd0a5d3dfcda2
Edit:
Originally I had a recursive function here which does not play well with sql server as it has a 32 nesting level limit which would result in an error like the below any time you attempt to make 32+ replacements with the function. Instead of trying to make a server level change to allow more nesting (which could be dangerous like allow never ending loops) switching to a while loop makes a lot more sense.
Maximum stored procedure, function, trigger, or view nesting level exceeded (limit 32).
Wrapping the solution inside a SQL function could be useful if you want to reuse it.
I'm even doing it at the cell level, that's why I'm putting this as a different answer:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(300))
RETURNS VARCHAR(300)
BEGIN
DECLARE #str VARCHAR(300) = #string;
DECLARE #Pattern VARCHAR (20) = '%[^a-zA-Z0-9]%';
DECLARE #Len INT;
SELECT #Len = LEN(#String);
WHILE #Len > 0
BEGIN
SET #Len = #Len - 1;
IF (PATINDEX(#Pattern,#str) > 0)
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
A more speedy approach for large strings would look something like this:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(MAX))
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #str VARCHAR(MAX) = #string;
DECLARE #Pattern VARCHAR (MAX) = '%[^a-zA-Z0-9]%';
WHILE PATINDEX(#Pattern,#str) > 0
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
RETURN #str
END
I've created this function to clean up a string that contained non numeric characters in a time field. The time contained question marks when they did not added the minutes, something like this 20:??. Function loops through each character and replaces the ? with a 0 :
CREATE FUNCTION [dbo].[CleanTime]
(
-- Add the parameters for the function here
#intime nvarchar(10)
)
RETURNS nvarchar(5)
AS
BEGIN
-- Declare the return variable here
DECLARE #ResultVar nvarchar(5)
DECLARE #char char(1)
-- Add the T-SQL statements to compute the return value here
DECLARE #i int = 1
WHILE #i <= LEN(#intime)
BEGIN
SELECT #char = CASE WHEN substring(#intime,#i,1) like '%[0-9:]%' THEN substring(#intime,#i,1) ELSE '0' END
SELECT #ResultVar = concat(#ResultVar,#char)
set #i = #i + 1
END;
-- Return the result of the function
RETURN #ResultVar
END
I think this solution is faster and simple. I use always CTE/recursive because WHILE is so slow on SQL Server.
I use it in projects I work with and large databases.
/*
Function: dbo.kSql_ReplaceRegExp
Create Date: 20.02.2021
Author: Karcan Ozbal
Description: The given string value will be replaced according to the given regexp/pattern.
Parameter(s): #Value : Value/Text to REPLACE.
#RegExp : The regexp/pattern to be used for REPLACE operation.
Usage: select dbo.kSql_ReplaceRegExp('2T3EST5','%[0-9]%')
Output: 'TEST'
*/
ALTER FUNCTION [dbo].[kSql_ReplaceRegExp](
#Value nvarchar(max),
#RegExp nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #Result nvarchar(max)
;WITH CTE AS (
SELECT NUM = 1, VALUE = #Value, IDX = PATINDEX(#RegExp, #Value)
UNION ALL
SELECT NUM + 1, VALUE = REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''), IDX = PATINDEX(#RegExp, REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''))
FROM CTE
WHERE IDX > 0
)
SELECT TOP(1) #Result = VALUE
FROM CTE
ORDER BY NUM DESC
OPTION (maxrecursion 0)
RETURN #Result
END
If you are doing this just for a parameter coming into a Stored Procedure, you can use the following:
declare #badIndex int
set #badIndex = PatIndex('%[^0-9]%', #Param)
while #badIndex > 0
set #Param = Replace(#Param, Substring(#Param, #badIndex, 1), '')
set #badIndex = PatIndex('%[^0-9]%', #Param)
I thought this was clearer:
ALTER FUNCTION [dbo].[func_ReplaceChars](
#Value nvarchar(max),
#Chars nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #cLen int = len(#Chars);
DECLARE #curChar int = 0;
WHILE #curChar<#cLen
BEGIN
set #Value = replace(#Value,substring(#Chars,#curChar,1),'');
set #curChar = #curChar + 1;
END;
RETURN #Value
END
I'm using this code similar to several codes above:
DROP FUNCTION [dbo].[fnCleanString]
GO
CREATE FUNCTION [dbo].[fnCleanString] (#input VARCHAR(max), #Pattern
VARCHAR (20))
RETURNS VARCHAR(max)
BEGIN
DECLARE #str VARCHAR(max) = #input;
DECLARE #Len INT;
DECLARE #INDEX INT;
SELECT #Len = LEN(#input);
WHILE #Len > 0
BEGIN
SET #INDEX = PATINDEX(#Pattern,#str);
IF (#INDEX > 0)
BEGIN
SET #str=REPLACE(#str,SUBSTRING(#str,#INDEX, 1), '');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
You can use it like this:
SELECT CleanName = dbo.[fnCleanString](Name, '%[0-9]%') from YourTable
I think a simpler and faster approach is iterate by each character of the alphabet:
DECLARE #i int
SET #i = 0
WHILE(#i < 256)
BEGIN
IF char(#i) NOT IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.')
UPDATE Table SET Column = replace(Column, char(#i), '')
SET #i = #i + 1
END
I have a need to create a function the will return nth element of a delimited string.
For a data migration project, I am converting JSON audit records stored in a SQL Server database into a structured report using SQL script. Goal is to deliver a sql script and a sql function used by the script without any code.
(This is a short-term fix will be used while a new auditing feature is added the ASP.NET/MVC application)
There is no shortage of delimited string to table examples available.
I've chosen a Common Table Expression example http://www.sqlperformance.com/2012/07/t-sql-queries/split-strings
Example: I want to return 67 from '1,222,2,67,888,1111'
This is the easiest answer to rerieve the 67 (type-safe!!):
SELECT CAST('<x>' + REPLACE('1,222,2,67,888,1111',',','</x><x>') + '</x>' AS XML).value('/x[4]','int')
In the following you will find examples how to use this with variables for the string, the delimiter and the position (even for edge-cases with XML-forbidden characters)
The easy one
This question is not about a string split approach, but about how to get the nth element. The easiest, fully inlineable way would be this IMO:
This is a real one-liner to get part 2 delimited by a space:
DECLARE #input NVARCHAR(100)=N'part1 part2 part3';
SELECT CAST(N'<x>' + REPLACE(#input,N' ',N'</x><x>') + N'</x>' AS XML).value('/x[2]','nvarchar(max)')
Variables can be used with sql:variable() or sql:column()
Of course you can use variables for delimiter and position (use sql:column to retrieve the position directly from a query's value):
DECLARE #dlmt NVARCHAR(10)=N' ';
DECLARE #pos INT = 2;
SELECT CAST(N'<x>' + REPLACE(#input,#dlmt,N'</x><x>') + N'</x>' AS XML).value('/x[sql:variable("#pos")][1]','nvarchar(max)')
Edge-Case with XML-forbidden characters
If your string might include forbidden characters, you still can do it this way. Just use FOR XML PATH on your string first to replace all forbidden characters with the fitting escape sequence implicitly.
It's a very special case if - additionally - your delimiter is the semicolon. In this case I replace the delimiter first to '#DLMT#', and replace this to the XML tags finally:
SET #input=N'Some <, > and &;Other äöü#€;One more';
SET #dlmt=N';';
SELECT CAST(N'<x>' + REPLACE((SELECT REPLACE(#input,#dlmt,'#DLMT#') AS [*] FOR XML PATH('')),N'#DLMT#',N'</x><x>') + N'</x>' AS XML).value('/x[sql:variable("#pos")][1]','nvarchar(max)');
UPDATE for SQL-Server 2016+
Regretfully the developers forgot to return the part's index with STRING_SPLIT. But, using SQL-Server 2016+, there is JSON_VALUE and OPENJSON.
With JSON_VALUE we can pass in the position as the index' array.
For OPENJSON the documentation states clearly:
When OPENJSON parses a JSON array, the function returns the indexes of the elements in the JSON text as keys.
A string like 1,2,3 needs nothing more than brackets: [1,2,3].
A string of words like this is an example needs to be ["this","is","an"," example"].
These are very easy string operations. Just try it out:
DECLARE #str VARCHAR(100)='Hello John Smith';
DECLARE #position INT = 2;
--We can build the json-path '$[1]' using CONCAT
SELECT JSON_VALUE('["' + REPLACE(#str,' ','","') + '"]',CONCAT('$[',#position-1,']'));
--See this for a position safe string-splitter (zero-based):
SELECT JsonArray.[key] AS [Position]
,JsonArray.[value] AS [Part]
FROM OPENJSON('["' + REPLACE(#str,' ','","') + '"]') JsonArray
In this post I tested various approaches and found, that OPENJSON is really fast. Even much faster than the famous "delimitedSplit8k()" method...
UPDATE 2 - Get the values type-safe
We can use an array within an array simply by using doubled [[]]. This allows for a typed WITH-clause:
DECLARE #SomeDelimitedString VARCHAR(100)='part1|1|20190920';
DECLARE #JsonArray NVARCHAR(MAX)=CONCAT('[["',REPLACE(#SomeDelimitedString,'|','","'),'"]]');
SELECT #SomeDelimitedString AS TheOriginal
,#JsonArray AS TransformedToJSON
,ValuesFromTheArray.*
FROM OPENJSON(#JsonArray)
WITH(TheFirstFragment VARCHAR(100) '$[0]'
,TheSecondFragment INT '$[1]'
,TheThirdFragment DATE '$[2]') ValuesFromTheArray
Here is my initial solution...
It is based on work by Aaron Bertrand http://www.sqlperformance.com/2012/07/t-sql-queries/split-strings
I simply changed the return type to make it a scalar function.
Example:
SELECT dbo.GetSplitString_CTE('1,222,2,67,888,1111',',',4)
CREATE FUNCTION dbo.GetSplitString_CTE
(
#List VARCHAR(MAX),
#Delimiter VARCHAR(255),
#ElementNumber int
)
RETURNS VARCHAR(4000)
AS
BEGIN
DECLARE #result varchar(4000)
DECLARE #Items TABLE ( position int IDENTITY PRIMARY KEY,
Item VARCHAR(4000)
)
DECLARE #ll INT = LEN(#List) + 1, #ld INT = LEN(#Delimiter);
WITH a AS
(
SELECT
[start] = 1,
[end] = COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, #ld), 0), #ll),
[value] = SUBSTRING(#List, 1,
COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, #ld), 0), #ll) - 1)
UNION ALL
SELECT
[start] = CONVERT(INT, [end]) + #ld,
[end] = COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, [end] + #ld), 0), #ll),
[value] = SUBSTRING(#List, [end] + #ld,
COALESCE(NULLIF(CHARINDEX(#Delimiter,
#List, [end] + #ld), 0), #ll)-[end]-#ld)
FROM a
WHERE [end] < #ll
)
INSERT #Items SELECT [value]
FROM a
WHERE LEN([value]) > 0
OPTION (MAXRECURSION 0);
SELECT #result=Item
FROM #Items
WHERE position=#ElementNumber
RETURN #result;
END
GO
How about:
CREATE FUNCTION dbo.NTH_ELEMENT (#Input NVARCHAR(MAX), #Delim CHAR = '-', #N INT = 0)
RETURNS NVARCHAR(MAX)
AS
BEGIN
RETURN (SELECT VALUE FROM STRING_SPLIT(#Input, #Delim) ORDER BY (SELECT NULL) OFFSET #N ROWS FETCH NEXT 1 ROW ONLY)
END
On Azure SQL Database, and on SQL Server 2022, STRING_SPLIT now has an optional ordinal parameter. If the parameter is omitted, or 0 is passed, then the function acts as it did before, and just returns a value column and the order is not guaranteed. If you pass the parameter with the value 1 then the function returns 2 columns, value, and ordinal which (unsurprisingly) provides the ordinal position of the value within the string.
So, if you wanted the 4th delimited value from the string '1,222,2,67,888,1111' you could do the following:
SELECT [value]
FROM STRING_SPLIT('1,222,2,67,888,1111',',',1)
WHERE ordinal = 4;
If the value was in a column, it would look like this:
SELECT SS.[value]
FROM dbo.YourTable YT
CROSS APPLY STRING_SPLIT(YT.YourColumn,',',1) SS
WHERE SS.ordinal = 4;
#a - the value (f.e. 'a/bb/ccc/dddd/ee/ff/....')
#p - the desired position (1,2,3...)
#d - the delimeter ( '/' )
trim(substring(replace(#a,#d,replicate(' ',len(#a))),(#p-1)*len(#a)+1,len(#a)))
only problem is - if desired part has trailing or leading blanks they get trimmed.
Completely Based on article from https://exceljet.net/formula/split-text-with-delimiter
In a rare moment of lunacy I just thought that split is far easier if we use XML to parse it out for us:
(Using the variables from #Gary Kindel's answer)
declare #xml xml
set #xml = '<split><el>' + replace(#list,#Delimiter,'</el><el>') + '</el></split>'
select
el = split.el.value('.','varchar(max)')
from #xml.nodes('/split/el') split(el))
This lists all elements of the string, split by the specified character.
We can use an xpath test to filter out empty values, and a further xpath test to restrict this to the element we're interested in. In full Gary's function becomes:
alter FUNCTION dbo.GetSplitString_CTE
(
#List VARCHAR(MAX),
#Delimiter VARCHAR(255),
#ElementNumber int
)
RETURNS VARCHAR(max)
AS
BEGIN
-- escape any XML https://dba.stackexchange.com/a/143140/65992
set #list = convert(VARCHAR(MAX),(select #list for xml path(''), type));
declare #xml xml
set #xml = '<split><el>' + replace(#list,#Delimiter,'</el><el>') + '</el></split>'
declare #ret varchar(max)
set #ret = (select
el = split.el.value('.','varchar(max)')
from #xml.nodes('/split/el[string-length(.)>0][position() = sql:variable("#elementnumber")]') split(el))
return #ret
END
you can put this select into UFN. if you need you can customize it for specifying delimiter as well. in that case your ufn will have two input. number Nth and delimiter to use.
DECLARE #tlist varchar(max)='10,20,30,40,50,60,70,80,90,100'
DECLARE #i INT=1, #nth INT=3
While len(#tlist) <> 0
BEGIN
IF #i=#nth
BEGIN
select Case when charindex(',',#tlist) <> 0 Then LEFT(#tlist,charindex(',',#tlist)-1)
Else #tlist
END
END
Select #tlist = Case when charindex(',',#tlist) <> 0 Then substring(#tlist,charindex(',',#tlist)+1,len(#tlist))
Else ''
END
SELECT #i=#i+1
END
Alternatively, one can use xml, nodes() and ROW_NUMBER. We can order the elements based on their document order. For example:
DECLARE #Input VARCHAR(100) = '1a,2b,3c,4d,5e,6f,7g,8h'
,#Number TINYINT = 3
DECLARE #XML XML;
DECLARE #value VARCHAR(100);
SET #XML = CAST('<x>' + REPLACE(#Input,',','</x><x>') + '</x>' AS XML);
WITH DataSource ([rowID], [rowValue]) AS
(
SELECT ROW_NUMBER() OVER (ORDER BY T.c ASC)
,T.c.value('.', 'VARCHAR(100)')
FROM #XML.nodes('./x') T(c)
)
SELECT #value = [rowValue]
FROM DataSource
WHERE [rowID] = #Number;
SELECT #value;
I would rather create a temp table with an identity column and fill it up with output from the SPLIT function.
CREATE TABLE #tblVals(Id INT IDENTITY(1,1), Val NVARCHAR(100))
INSERT INTO #tblVals (Val)
SELECT [value] FROM STRING_SPLIT('Val1-Val3-Val2-Val5', '-')
SELECT * FROM #tblVals
Now you can easily do something like below.
DECLARE #val2 NVARCHAR(100) = (SELECT TOP 1 Val FROM #tblVals WHERE Id = 2)
See the snapshot below:
You can use STRING_SPLIT with ROW_NUMBER:
SELECT value, idx FROM
(
SELECT
value,
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) idx
FROM STRING_SPLIT('Lorem ipsum dolor sit amet.', ' ')
) t
WHERE idx=2
returns second element (idx=2): 'ipsum'
We have the answer over below url.
DECLARE # AS VARCHAR(MAX) = 'Pawan1,Pawan2,Pawan4,Pawan3'
SELECT VALUE FROM
(
SELECT VALUE , ROW_NUMBER() OVER (ORDER BY (SELECT null)) rnk FROM STRING_SPLIT(#, ',')
)x where rnk = 3
GO
https://msbiskills.com/2018/06/15/sql-puzzle-multiple-ways-to-split-a-string-and-get-nth-row-xml-advanced-sql/
I don't have enough reputation to comment, so I am adding an answer. Please adjust as appropriate.
I have a problem with Gary Kindel's answer for cases where there is nothing between the two delimiters
If you do
select * from dbo.GetSplitString_CTE('abc^def^^ghi','^',3)
you get
ghi
instead of an empty string
If you comment out the
WHERE LEN([value]) > 0
line, you get the desired result
I cannot comment on Gary's solution because of my low reputation
I know Gary was referencing another link.
I have struggled to understand why we need this variable
#ld INT = LEN(#Delimiter)
I also don't understand why charindex has to start at the position of length of delimiter, #ld
I tested with many examples with a single character delimiter, and they work. Most of the time, delimiter character is a single character. However, since the developer included the ld as length of delimiter, the code has to work for delimiters that have more than one character
In this case, the following case will fail
11,,,22,,,33,,,44,,,55,,,
I cloned from the codes from this link. http://codebetter.com/raymondlewallen/2005/10/26/quick-t-sql-to-parse-a-delimited-string/
I have tested various scenarios including the delimiters that have more than one character
alter FUNCTION [dbo].[split1]
(
#string1 VARCHAR(8000) -- List of delimited items
, #Delimiter VARCHAR(40) = ',' -- delimiter that separates items
, #ElementNumber int
)
RETURNS varchar(8000)
AS
BEGIN
declare #position int
declare #piece varchar(8000)=''
declare #returnVal varchar(8000)=''
declare #Pattern varchar(50) = '%' + #Delimiter + '%'
declare #counter int =0
declare #ld int = len(#Delimiter)
declare #ls1 int = len (#string1)
declare #foundit int = 0
if patindex(#Pattern , #string1) = 0
return ''
if right(rtrim(#string1),1) <> #Delimiter
set #string1 = #string1 + #Delimiter
set #position = patindex(#Pattern , #string1) + #ld -1
while #position > 0
begin
set #counter = #counter +1
set #ls1 = len (#string1)
if (#ls1 >= #ld)
set #piece = left(#string1, #position - #ld)
else
break
if (#counter = #ElementNumber)
begin
set #foundit = 1
break
end
if len(#string1) > 0
begin
set #string1 = stuff(#string1, 1, #position, '')
set #position = patindex(#Pattern , #string1) + #ld -1
end
else
set #position = -1
end
if #foundit =1
set #returnVal = #piece
else
set #returnVal = ''
return #returnVal
you can create simple table variable and use it as below
Declare #tbl_split Table (Id INT IDENTITY(1,1), VAL VARCHAR(50))
INSERT #tbl_split SELECT VALUE
FROM string_split('999999:01', ':')
Select val from #tbl_split
WHERE Id=2
I have a column that has values formatted like a,b,c,d. Is there a way to count the number of commas in that value in T-SQL?
The first way that comes to mind is to do it indirectly by replacing the comma with an empty string and comparing the lengths
Declare #string varchar(1000)
Set #string = 'a,b,c,d'
select len(#string) - len(replace(#string, ',', ''))
Quick extension of cmsjr's answer that works for strings with more than one character.
CREATE FUNCTION dbo.CountOccurrencesOfString
(
#searchString nvarchar(max),
#searchTerm nvarchar(max)
)
RETURNS INT
AS
BEGIN
return (LEN(#searchString)-LEN(REPLACE(#searchString,#searchTerm,'')))/LEN(#searchTerm)
END
Usage:
SELECT * FROM MyTable
where dbo.CountOccurrencesOfString(MyColumn, 'MyString') = 1
You can compare the length of the string with one where the commas are removed:
len(value) - len(replace(value,',',''))
The answer by #csmjr has a problem in some instances.
His answer was to do this:
Declare #string varchar(1000)
Set #string = 'a,b,c,d'
select len(#string) - len(replace(#string, ',', ''))
This works in most scenarios, however, try running this:
DECLARE #string VARCHAR(1000)
SET #string = 'a,b,c,d ,'
SELECT LEN(#string) - LEN(REPLACE(#string, ',', ''))
For some reason, REPLACE gets rid of the final comma but ALSO the space just before it (not sure why). This results in a returned value of 5 when you'd expect 4. Here is another way to do this which will work even in this special scenario:
DECLARE #string VARCHAR(1000)
SET #string = 'a,b,c,d ,'
SELECT LEN(REPLACE(#string, ',', '**')) - LEN(#string)
Note that you don't need to use asterisks. Any two-character replacement will do. The idea is that you lengthen the string by one character for each instance of the character you're counting, then subtract the length of the original. It's basically the opposite method of the original answer which doesn't come with the strange trimming side-effect.
Building on #Andrew's solution, you'll get much better performance using a non-procedural table-valued-function and CROSS APPLY:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
/* Usage:
SELECT t.[YourColumn], c.StringCount
FROM YourDatabase.dbo.YourTable t
CROSS APPLY dbo.CountOccurrencesOfString('your search string', t.[YourColumn]) c
*/
CREATE FUNCTION [dbo].[CountOccurrencesOfString]
(
#searchTerm nvarchar(max),
#searchString nvarchar(max)
)
RETURNS TABLE
AS
RETURN
SELECT (DATALENGTH(#searchString)-DATALENGTH(REPLACE(#searchString,#searchTerm,'')))/NULLIF(DATALENGTH(#searchTerm), 0) AS StringCount
Declare #string varchar(1000)
DECLARE #SearchString varchar(100)
Set #string = 'as as df df as as as'
SET #SearchString = 'as'
select ((len(#string) - len(replace(#string, #SearchString, ''))) -(len(#string) -
len(replace(#string, #SearchString, ''))) % 2) / len(#SearchString)
Accepted answer is correct ,
extending it to use 2 or more character in substring:
Declare #string varchar(1000)
Set #string = 'aa,bb,cc,dd'
Set #substring = 'aa'
select (len(#string) - len(replace(#string, #substring, '')))/len(#substring)
Darrel Lee I think has a pretty good answer. Replace CHARINDEX() with PATINDEX(), and you can do some weak regex searching along a string, too...
Like, say you use this for #pattern:
set #pattern='%[-.|!,'+char(9)+']%'
Why would you maybe want to do something crazy like this?
Say you're loading delimited text strings into a staging table, where the field holding the data is something like a varchar(8000) or nvarchar(max)...
Sometimes it's easier/faster to do ELT (Extract-Load-Transform) with data rather than ETL (Extract-Transform-Load), and one way to do this is to load the delimited records as-is into a staging table, especially if you may want an simpler way to see the exceptional records rather than deal with them as part of an SSIS package...but that's a holy war for a different thread.
If we know there is a limitation on LEN and space, why cant we replace the space first?
Then we know there is no space to confuse LEN.
len(replace(#string, ' ', '-')) - len(replace(replace(#string, ' ', '-'), ',', ''))
Use this code, it is working perfectly.
I have create a sql function that accept two parameters, the first param is the long string that we want to search into it,and it can accept string length up to 1500 character(of course you can extend it or even change it to text datatype).
And the second parameter is the substring that we want to calculate the number of its occurance(its length is up to 200 character, of course you can change it to what your need). and the output is an integer, represent the number of frequency.....enjoy it.
CREATE FUNCTION [dbo].[GetSubstringCount]
(
#InputString nvarchar(1500),
#SubString NVARCHAR(200)
)
RETURNS int
AS
BEGIN
declare #K int , #StrLen int , #Count int , #SubStrLen int
set #SubStrLen = (select len(#SubString))
set #Count = 0
Set #k = 1
set #StrLen =(select len(#InputString))
While #K <= #StrLen
Begin
if ((select substring(#InputString, #K, #SubStrLen)) = #SubString)
begin
if ((select CHARINDEX(#SubString ,#InputString)) > 0)
begin
set #Count = #Count +1
end
end
Set #K=#k+1
end
return #Count
end
In SQL 2017 or higher, you can use this:
declare #hits int = 0
set #hits = (select value from STRING_SPLIT('F609,4DFA,8499',','));
select count(#hits)
Improved version based on top answer and other answers:
Wrapping the string with delimiters ensures that LEN works properly. Making the replace character string one character longer than the match string removes the need for division.
CREATE FUNCTION dbo.MatchCount(#value nvarchar(max), #match nvarchar(max))
RETURNS int
BEGIN
RETURN LEN('[' + REPLACE(#value,#match,REPLICATE('*', LEN('[' + #match + ']') - 1)) + ']') - LEN('['+#value+']')
END
DECLARE #records varchar(400)
SELECT #records = 'a,b,c,d'
select LEN(#records) as 'Before removing Commas' , LEN(#records) - LEN(REPLACE(#records, ',', '')) 'After Removing Commans'
The following should do the trick for both single character and multiple character searches:
CREATE FUNCTION dbo.CountOccurrences
(
#SearchString VARCHAR(1000),
#SearchFor VARCHAR(1000)
)
RETURNS TABLE
AS
RETURN (
SELECT COUNT(*) AS Occurrences
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY O.object_id) AS n
FROM sys.objects AS O
) AS N
JOIN (
VALUES (#SearchString)
) AS S (SearchString)
ON
SUBSTRING(S.SearchString, N.n, LEN(#SearchFor)) = #SearchFor
);
GO
---------------------------------------------------------------------------------------
-- Test the function for single and multiple character searches
---------------------------------------------------------------------------------------
DECLARE #SearchForComma VARCHAR(10) = ',',
#SearchForCharacters VARCHAR(10) = 'de';
DECLARE #TestTable TABLE
(
TestData VARCHAR(30) NOT NULL
);
INSERT INTO #TestTable
(
TestData
)
VALUES
('a,b,c,de,de ,d e'),
('abc,de,hijk,,'),
(',,a,b,cde,,');
SELECT TT.TestData,
CO.Occurrences AS CommaOccurrences,
CO2.Occurrences AS CharacterOccurrences
FROM #TestTable AS TT
OUTER APPLY dbo.CountOccurrences(TT.TestData, #SearchForComma) AS CO
OUTER APPLY dbo.CountOccurrences(TT.TestData, #SearchForCharacters) AS CO2;
The function can be simplified a bit using a table of numbers (dbo.Nums):
RETURN (
SELECT COUNT(*) AS Occurrences
FROM dbo.Nums AS N
JOIN (
VALUES (#SearchString)
) AS S (SearchString)
ON
SUBSTRING(S.SearchString, N.n, LEN(#SearchFor)) = #SearchFor
);
I finally write this function that should cover all the possible situations, adding a char prefix and suffix to the input. this char is evaluated to be different to any of the char conteined in the search parameter, so it can't affect the result.
CREATE FUNCTION [dbo].[CountOccurrency]
(
#Input nvarchar(max),
#Search nvarchar(max)
)
RETURNS int AS
BEGIN
declare #SearhLength as int = len('-' + #Search + '-') -2;
declare #conteinerIndex as int = 255;
declare #conteiner as char(1) = char(#conteinerIndex);
WHILE ((CHARINDEX(#conteiner, #Search)>0) and (#conteinerIndex>0))
BEGIN
set #conteinerIndex = #conteinerIndex-1;
set #conteiner = char(#conteinerIndex);
END;
set #Input = #conteiner + #Input + #conteiner
RETURN (len(#Input) - len(replace(#Input, #Search, ''))) / #SearhLength
END
usage
select dbo.CountOccurrency('a,b,c,d ,', ',')
Declare #MainStr nvarchar(200)
Declare #SubStr nvarchar(10)
Set #MainStr = 'nikhildfdfdfuzxsznikhilweszxnikhil'
Set #SubStr = 'nikhil'
Select (Len(#MainStr) - Len(REPLACE(#MainStr,#SubStr,'')))/Len(#SubStr)
this T-SQL code finds and prints all occurrences of pattern #p in sentence #s. you can do any processing on the sentence afterward.
declare #old_hit int = 0
declare #hit int = 0
declare #i int = 0
declare #s varchar(max)='alibcalirezaalivisualization'
declare #p varchar(max)='ali'
while #i<len(#s)
begin
set #hit=charindex(#p,#s,#i)
if #hit>#old_hit
begin
set #old_hit =#hit
set #i=#hit+1
print #hit
end
else
break
end
the result is:
1
6
13
20
I ended up using a CTE table for this,
CREATE TABLE #test (
[id] int,
[field] nvarchar(500)
)
INSERT INTO #test ([id], [field])
VALUES (1, 'this is a test string http://url, and https://google.com'),
(2, 'another string, hello world http://example.com'),
(3, 'a string with no url')
SELECT *
FROM #test
;WITH URL_count_cte ([id], [url_index], [field])
AS
(
SELECT [id], CHARINDEX('http', [field], 0)+1 AS [url_index], [field]
FROM #test AS [t]
WHERE CHARINDEX('http', [field], 0) != 0
UNION ALL
SELECT [id], CHARINDEX('http', [field], [url_index])+1 AS [url_index], [field]
FROM URL_count_cte
WHERE CHARINDEX('http', [field], [url_index]) > 0
)
-- total urls
SELECT COUNT(1)
FROM URL_count_cte
-- urls per row
SELECT [id], COUNT(1) AS [url_count]
FROM URL_count_cte
GROUP BY [id]
Using this function, you can get the number of repetitions of words in a text.
/****** Object: UserDefinedFunction [dbo].[fn_getCountKeywords] Script Date: 22/11/2021 17:52:00 ******/
DROP FUNCTION IF EXISTS [dbo].[fn_getCountKeywords]
GO
/****** Object: UserDefinedFunction [dbo].[fn_getCountKeywords] Script Date: 2211/2021 17:52:00 ******/
SET ANSI_NULLS OFF
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: m_Khezrian
-- Create date: 2021/11/22-17:52
-- Description: Return Count Keywords In Input Text
-- =============================================
Create OR Alter Function [dbo].[fn_getCountKeywords]
(#Text nvarchar(max)
,#Keywords nvarchar(max)
)
RETURNS #Result TABLE
(
[ID] int Not Null IDENTITY PRIMARY KEY
,[Keyword] nvarchar(max) Not Null
,[Cnt] int Not Null Default(0)
)
/*With ENCRYPTION*/ As
Begin
Declare #Key nvarchar(max);
Declare #Cnt int;
Declare #I int;
Set #I = 0 ;
--Set #Text = QUOTENAME(#Text);
Insert Into #Result
([Keyword])
Select Trim([value])
From String_Split(#Keywords,N',')
Group By [value]
Order By Len([value]) Desc;
Declare CntKey_Cursor Insensitive Cursor For
Select [Keyword]
From #Result
Order By [ID];
Open CntKey_Cursor;
Fetch Next From CntKey_Cursor Into #Key;
While (##Fetch_STATUS = 0) Begin
Set #Cnt = 0;
While (PatIndex(N'%'+#Key+'%',#Text) > 0) Begin
Set #Cnt += 1;
Set #I += 1 ;
Set #Text = Stuff(#Text,PatIndex(N'%'+#Key+'%',#Text),len(#Key),N'{'+Convert(nvarchar,#I)+'}');
--Set #Text = Replace(#Text,#Key,N'{'+Convert(nvarchar,#I)+'}');
End--While
Update #Result
Set [Cnt] = #Cnt
Where ([Keyword] = #Key);
Fetch Next From CntKey_Cursor Into #Key;
End--While
Close CntKey_Cursor;
Deallocate CntKey_Cursor;
Return
End
GO
--Test
Select *
From dbo.fn_getCountKeywords(
N'<U+0001F4E3> MARKET IMPACT Euro area Euro CPIarea annual inflation up to 3.0% MaCPIRKET forex'
,N'CPI ,core,MaRKET , Euro area'
)
Go
Reference https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-ver15
Example:
SELECT s.*
,s.[Number1] - (SELECT COUNT(Value)
FROM string_split(s.[StringColumn],',')
WHERE RTRIM(VALUE) <> '')
FROM TableName AS s
Applies to: SQL Server 2016 (13.x) and later
You can use the following stored procedure to fetch , values.
IF EXISTS (SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID(N'[dbo].[sp_parsedata]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[sp_parsedata]
GO
create procedure sp_parsedata
(#cid integer,#st varchar(1000))
as
declare #coid integer
declare #c integer
declare #c1 integer
select #c1=len(#st) - len(replace(#st, ',', ''))
set #c=0
delete from table1 where complainid=#cid;
while (#c<=#c1)
begin
if (#c<#c1)
begin
select #coid=cast(replace(left(#st,CHARINDEX(',',#st,1)),',','') as integer)
select #st=SUBSTRING(#st,CHARINDEX(',',#st,1)+1,LEN(#st))
end
else
begin
select #coid=cast(#st as integer)
end
insert into table1(complainid,courtid) values(#cid,#coid)
set #c=#c+1
end
The Replace/Len test is cute, but probably very inefficient (especially in terms of memory).
A simple function with a loop will do the job.
CREATE FUNCTION [dbo].[fn_Occurences]
(
#pattern varchar(255),
#expression varchar(max)
)
RETURNS int
AS
BEGIN
DECLARE #Result int = 0;
DECLARE #index BigInt = 0
DECLARE #patLen int = len(#pattern)
SET #index = CHARINDEX(#pattern, #expression, #index)
While #index > 0
BEGIN
SET #Result = #Result + 1;
SET #index = CHARINDEX(#pattern, #expression, #index + #patLen)
END
RETURN #Result
END
Perhaps you should not store data that way. It is a bad practice to ever store a comma delimited list in a field. IT is very inefficient for querying. This should be a related table.