Related
I have a stored procedure which is supposed to query based on user selections. The stored procedure takes three GUID parameters (user selected) and holds each GUID in it's own variable called '#GUID_X' where X would be an integer in the range of [1-15).
The stored procedure needs to query for these GUID values and since there is a pattern to the variable names, I'd like to use a loop to iterate through the variables to access their values and then insert each value into the query.
So far, I've tried concatenating '#GUID_' + 'counter number' and ended up with a list of values such as:
#FoodGUID_3
#FoodGUID_2
#FoodGUID_1
When in reality, I was expecting a list that looked like:
ABABABA1-FGH-1234-ZX12-123456EDVERT
GHDF1234-CVB-4312-CV15-678912NK1284
VBNM5678-BVC-2134-BN18-852741DC1434
Why can't I access the actual GUID values of the declared #FoodGUID_X args? What would be a better way of approaching this problem? Ideally the approach wouldn't be to use an 'if' clause for each arg since this might be expensive!
For ref. here if my full code:
DECLARE #foodGUID_1 NVARCHAR(360) = N'ABABABA1-FGH-1234-ZX12-123456EDVERT',
#foodGUID_2 NVARCHAR(360) = N'GHDF1234-CVB-4312-CV15-678912NKOMLP',
#foodGUID_3 NVARCHAR(360) = N'VBNM5678-BVC-2134-BN18-852741DCFVGB'
DECLARE #SelectSQL NVARCHAR(2500) = 'Query string + '
DECLARE #NumOfFoodsSelected INT = 3
DECLARE #counter INT = #NumOfFoodsSelected
DECLARE #currentGUID NVARCHAR(360)
SET #currentGUID = '#DeviceGUID_'+ CONVERT(NVARCHAR(15), #counter)
PRINT #currentGUID
WHILE (#counter != 1)
BEGIN
SET #counter = #counter - 1
SET #currentGUID = '#DeviceGUID_'+ CONVERT(NVARCHAR(15), #counter)
PRINT #currentGUID
END
Thanks in advance for any and all help! :)
you can use UDT as parameter on your stored procedure to store your GUIDs.
So your script will look like this.
declare #goodGuids table (
id int identity(1, 1),
foodguid NVARCHAR(360)
);
insert into #goodGuids(foodguid) values
(N'ABABABA1-FGH-1234-ZX12-123456EDVERT'),
(N'GHDF1234-CVB-4312-CV15-678912NKOMLP'),
(N'VBNM5678-BVC-2134-BN18-852741DCFVGB')
select 'DeviceGUID_' + foodguid from #goodGuids
A table value constructor can be used to access the names and values:
DECLARE #foodGUID_1 NVARCHAR(360) = N'ABABABA1-FGH-1234-ZX12-123456EDVERT',
#foodGUID_2 NVARCHAR(360) = N'GHDF1234-CVB-4312-CV15-678912NKOMLP',
#foodGUID_3 NVARCHAR(360) = N'VBNM5678-BVC-2134-BN18-852741DCFVGB';
select VariableValue
from ( values
( '#foodGUID_1', #foodGUID_1 ),
( '#foodGUID_2', #foodGUID_2 ),
( '#foodGUID_3', #foodGUID_3 )
) as foodGUIDs( VariableName, VariableValue )
where VariableName = '#foodGUID_2';
The where clause could be where VariableName = '#foodGUID_' + Cast( #Counter as VarChar(2) ) for brute force looping through the values. The value can be retrieved in a variable using select #foodGUID = VariableValue from ...
The table value constructor can be used in a variety of queries as a derived table, e.g. to insert multiple rows into a table or as one side of a join in a select.
Problem
A stored procedure is receiving list of variables and values, and the delimiter. This stored procedure needs to insert those in a table.
--Example table
create table #tempo
(
Variable1 int,
Variable2 int,
Variable3 int
)
These are the parameters to the stored procedure:
declare #variableList varchar(100)
declare #valueList varchar(100)
declare #separator char(1)
set #variableList = 'Variable1#Variable2#Variable3'
set #valueList = '1111#2222#3333'
set #separator = '#'
Result
What I want to achieve is this:
select * from #tempo
+---------+---------+---------+
|Variable1|Variable2|Variable3|
+---------+---------+---------+
|1111 |2222 |3333 |
+---------+---------+---------+
One way to do it
I can use a loop and build dynamic SQL but I want to avoid it. Other than the obvious reasons for not using dynamic SQL, the loop structure is hard to maintain, explain and testing can become an issue too.
Ideal way
I am thinking about a more elegant way to do this, for example with string_split or coalesce etc. But cannot figure out a way without using dynamic SQL or loops.
If you always have same set of column names then it is very easy to do with pivoting, but if columns are changing then you can use the same script with dynamically adjusted list of variables, provided as a parameter or from direct reading from temp table:
INSERT INTO #tempo SELECT *
FROM (
SELECT [value], rv = 'Variable' + CAST(Row_Number() OVER ( ORDER BY (SELECT 1)) as VARCHAR)
FROM STRING_SPLIT(#valueList,#separator)
) AS src
PIVOT (MAX([value]) FOR rv IN (Variable1,Variable2,Variable3)) AS pvt;
You can always try pivoting out the data. This is just the select, but could easily have an insert wrapped into it.
We use a split string with a row ID to allow matching of two split data sets. Function is :
CREATE FUNCTION [dbo].[Split] (#RowData NVARCHAR(MAX), #SplitOn NVARCHAR(5))
RETURNS #RtnValue TABLE (Id INT IDENTITY(1, 1), Data NVARCHAR(100))
AS
BEGIN
DECLARE #Cnt INT;
SET #Cnt = 1;
WHILE (CHARINDEX(#SplitOn, #RowData) > 0)
BEGIN
INSERT INTO #RtnValue (Data)
SELECT Data = LTRIM(RTRIM(SUBSTRING(#RowData, 1, CHARINDEX(#SplitOn, #RowData) - 1)));
SET #RowData = SUBSTRING(#RowData, CHARINDEX(#SplitOn, #RowData) + 1, LEN(#RowData));
SET #Cnt = #Cnt + 1;
END;
INSERT INTO #RtnValue (Data)
SELECT Data = LTRIM(RTRIM(#RowData));
RETURN;
END;
You can then join the two sets together to give some key value pairs, and from there pivot out the data to give the format you requested. If you replace the last select with a select from any of the previous cte's then you can see how the logic unfolds.
DECLARE #variableList VARCHAR(100);
DECLARE #valueList VARCHAR(100);
DECLARE #separator CHAR(1);
SET #variableList = 'Variable1,Variable2,Variable3';
SET #valueList = '1111, 2222, 3333';
SET #separator = ',';
WITH cteVar AS (SELECT Id, Data FROM dbo.Split(#variableList, #separator) )
, cteVal AS (SELECT Id, Data FROM dbo.Split(#valueList, #separator) )
, cteData AS
(SELECT cteVar.Data VariableData
, cteVal.Data ValueData
FROM cteVar
JOIN cteVal ON cteVal.Id = cteVar.Id)
, ctePivot AS
(SELECT *
FROM cteData
PIVOT ( MAX(ValueData)
FOR VariableData IN ([Variable1], [Variable2], [Variable3])) AS PivotTable)
SELECT *
FROM ctePivot;
This is quite a long approach to it but hopefully it well help you understand the steps involved. Its worth looking at the Pivot function in general anyway, its well documented.
SELECT REPLACE('<strong>100</strong><b>.00 GB', '%^(^-?\d*\.{0,1}\d+$)%', '');
I want to replace any markup between two parts of the number with above regex, but it does not seem to work. I'm not sure if it is regex syntax that's wrong because I tried simpler one such as '%[^0-9]%' just to test but it didn't work either. Does anyone know how can I achieve this?
You can use PATINDEX
to find the first index of the pattern (string's) occurrence. Then use STUFF to stuff another string into the pattern(string) matched.
Loop through each row. Replace each illegal characters with what you want. In your case replace non numeric with blank. The inner loop is if you have more than one illegal character in a current cell that of the loop.
DECLARE #counter int
SET #counter = 0
WHILE(#counter < (SELECT MAX(ID_COLUMN) FROM Table))
BEGIN
WHILE 1 = 1
BEGIN
DECLARE #RetVal varchar(50)
SET #RetVal = (SELECT Column = STUFF(Column, PATINDEX('%[^0-9.]%', Column),1, '')
FROM Table
WHERE ID_COLUMN = #counter)
IF(#RetVal IS NOT NULL)
UPDATE Table SET
Column = #RetVal
WHERE ID_COLUMN = #counter
ELSE
break
END
SET #counter = #counter + 1
END
Caution: This is slow though! Having a varchar column may impact. So using LTRIM RTRIM may help a bit. Regardless, it is slow.
Credit goes to this StackOverFlow answer.
EDIT
Credit also goes to #srutzky
Edit (by #Tmdean)
Instead of doing one row at a time, this answer can be adapted to a more set-based solution. It still iterates the max of the number of non-numeric characters in a single row, so it's not ideal, but I think it should be acceptable in most situations.
WHILE 1 = 1 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, '')
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 BREAK;
END;
You can also improve efficiency quite a lot if you maintain a bit column in the table that indicates whether the field has been scrubbed yet. (NULL represents "Unknown" in my example and should be the column default.)
DECLARE #done bit = 0;
WHILE #done = 0 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table
WHERE COALESCE(Scrubbed_Column, 0) = 0)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, ''),
Scrubbed_Column = 0
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 SET #done = 1;
-- if Scrubbed_Column is still NULL, then the PATINDEX
-- must have given 0
UPDATE table
SET Scrubbed_Column = CASE
WHEN Scrubbed_Column IS NULL THEN 1
ELSE NULLIF(Scrubbed_Column, 0)
END;
END;
If you don't want to change your schema, this is easy to adapt to store intermediate results in a table valued variable which gets applied to the actual table at the end.
Instead of stripping out the found character by its sole position, using Replace(Column, BadFoundCharacter, '') could be substantially faster. Additionally, instead of just replacing the one bad character found next in each column, this replaces all those found.
WHILE 1 = 1 BEGIN
UPDATE dbo.YourTable
SET Column = Replace(Column, Substring(Column, PatIndex('%[^0-9.-]%', Column), 1), '')
WHERE Column LIKE '%[^0-9.-]%'
If ##RowCount = 0 BREAK;
END;
I am convinced this will work better than the accepted answer, if only because it does fewer operations. There are other ways that might also be faster, but I don't have time to explore those right now.
In a general sense, SQL Server does not support regular expressions and you cannot use them in the native T-SQL code.
You could write a CLR function to do that. See here, for example.
For those looking for a performant and easy solution and are willing to enable CLR:
CREATE database TestSQLFunctions
go
use TestSQLFunctions
go
ALTER database TestSQLFunctions set trustworthy on
EXEC sp_configure 'clr enabled', 1
RECONFIGURE WITH OVERRIDE
go
CREATE ASSEMBLY [SQLFunctions]
AUTHORIZATION [dbo]
FROM 0x4D5A90000300000004000000FFFF0000B800000000000000400000000000000000000000000000000000000000000000000000000000000000000000800000000E1FBA0E00B409CD21B8014CCD21546869732070726F6772616D2063616E6E6F742062652072756E20696E20444F53206D6F64652E0D0D0A2400000000000000504500004C0103004BE8B85F0000000000000000E00022200B013000000800000006000000000000C2270000002000000040000000000010002000000002000004000000000000000600000000000000008000000002000000000000030060850000100000100000000010000010000000000000100000000000000000000000702700004F000000004000009803000000000000000000000000000000000000006000000C00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000080000000000000000000000082000004800000000000000000000002E74657874000000C8070000002000000008000000020000000000000000000000000000200000602E72737263000000980300000040000000040000000A0000000000000000000000000000400000402E72656C6F6300000C0000000060000000020000000E00000000000000000000000000004000004200000000000000000000000000000000A4270000000000004800000002000500682000000807000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003A022D02022A020304281000000A2A1E02281100000A2A0042534A4201000100000000000C00000076342E302E33303331390000000005006C00000018020000237E000084020000CC02000023537472696E6773000000005005000004000000235553005405000010000000234755494400000064050000A401000023426C6F620000000000000002000001471500000900000000FA0133001600000100000013000000020000000200000003000000110000000F00000001000000030000000000CA01010000000000060025014F02060092014F02060044001D020F006F02000006006C00E20106000801E2010600D400E20106007901E20106004501E20106005E01E20106008300E2010600580030020600360030020600B700E20106009E00B0010600AA02DB010A00F300FC010A001F00FC010E00C3027E02000000000100000000000100010001001000C3029D0241000100010050200000000096002E001A0001005F2000000000861817020600040000000100BD0200000200F40100000300B102090017020100110017020600190017020A0029001702100031001702100039001702100041001702100049001702100051001702100059001702100061001702150069001702100071001702100079001702100089001702060099002E001A0081001702060020007B0010012E000B002A002E00130033002E001B0052002E0023005B002E002B006D002E0033006D002E003B006D002E0043005B002E004B0073002E0053006D002E005B006D002E0063008B002E006B00B5002E007300C2000480000001000000000000000000000000009D020000040000000000000000000000210016000000000004000000000000000000000021000A00000000000400000000000000000000002100DB010000000000000000003C4D6F64756C653E0053797374656D2E44617461006D73636F726C696200446174614163636573734B696E64005265706C61636500477569644174747269627574650044656275676761626C6541747472696275746500436F6D56697369626C6541747472696275746500417373656D626C795469746C6541747472696275746500417373656D626C7954726164656D61726B417474726962757465005461726765744672616D65776F726B41747472696275746500417373656D626C7946696C6556657273696F6E41747472696275746500417373656D626C79436F6E66696775726174696F6E4174747269627574650053716C46756E6374696F6E41747472696275746500417373656D626C794465736372697074696F6E41747472696275746500436F6D70696C6174696F6E52656C61786174696F6E7341747472696275746500417373656D626C7950726F6475637441747472696275746500417373656D626C79436F7079726967687441747472696275746500417373656D626C79436F6D70616E794174747269627574650052756E74696D65436F6D7061746962696C6974794174747269627574650053797374656D2E52756E74696D652E56657273696F6E696E670053514C46756E6374696F6E732E646C6C0053797374656D0053797374656D2E5265666C656374696F6E007061747465726E004D6963726F736F66742E53716C5365727665722E536572766572002E63746F720053797374656D2E446961676E6F73746963730053797374656D2E52756E74696D652E496E7465726F7053657276696365730053797374656D2E52756E74696D652E436F6D70696C6572536572766963657300446562756767696E674D6F6465730053797374656D2E546578742E526567756C617245787072657373696F6E730053514C46756E6374696F6E73004F626A656374007265706C6163656D656E7400696E70757400526567657800000000000000003A1617E607071B47B964858BCD87458B00042001010803200001052001011111042001010E04200101020600030E0E0E0E08B77A5C561934E0890801000800000000001E01000100540216577261704E6F6E457863657074696F6E5468726F7773010801000200000000001101000C53514C46756E6374696F6E73000005010000000017010012436F7079726967687420C2A920203230323000002901002434346436386231632D393735312D343938612D396665352D32316666333934303738303900000C010007312E302E302E3000004D01001C2E4E45544672616D65776F726B2C56657273696F6E3D76342E352E320100540E144672616D65776F726B446973706C61794E616D65142E4E4554204672616D65776F726B20342E352E32808F010001005455794D6963726F736F66742E53716C5365727665722E5365727665722E446174614163636573734B696E642C2053797374656D2E446174612C2056657273696F6E3D342E302E302E302C2043756C747572653D6E65757472616C2C205075626C69634B6579546F6B656E3D623737613563353631393334653038390A4461746141636365737301000000000000982700000000000000000000B2270000002000000000000000000000000000000000000000000000A4270000000000000000000000005F436F72446C6C4D61696E006D73636F7265652E646C6C0000000000FF25002000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001001000000018000080000000000000000000000000000001000100000030000080000000000000000000000000000001000000000048000000584000003C03000000000000000000003C0334000000560053005F00560045005200530049004F004E005F0049004E0046004F0000000000BD04EFFE00000100000001000000000000000100000000003F000000000000000400000002000000000000000000000000000000440000000100560061007200460069006C00650049006E0066006F00000000002400040000005400720061006E0073006C006100740069006F006E00000000000000B0049C020000010053007400720069006E006700460069006C00650049006E0066006F0000007802000001003000300030003000300034006200300000001A000100010043006F006D006D0065006E007400730000000000000022000100010043006F006D00700061006E0079004E0061006D006500000000000000000042000D000100460069006C0065004400650073006300720069007000740069006F006E0000000000530051004C00460075006E006300740069006F006E00730000000000300008000100460069006C006500560065007200730069006F006E000000000031002E0030002E0030002E003000000042001100010049006E007400650072006E0061006C004E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000004800120001004C006500670061006C0043006F007000790072006900670068007400000043006F0070007900720069006700680074002000A90020002000320030003200300000002A00010001004C006500670061006C00540072006100640065006D00610072006B00730000000000000000004A00110001004F0072006900670069006E0061006C00460069006C0065006E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000003A000D000100500072006F0064007500630074004E0061006D00650000000000530051004C00460075006E006300740069006F006E00730000000000340008000100500072006F006400750063007400560065007200730069006F006E00000031002E0030002E0030002E003000000038000800010041007300730065006D0062006C0079002000560065007200730069006F006E00000031002E0030002E0030002E0030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000C000000C43700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
WITH PERMISSION_SET = SAFE
go
CREATE FUNCTION RegexReplace(
#input nvarchar(max),
#pattern nvarchar(max),
#replacement nvarchar(max)
) RETURNS nvarchar (max)
AS EXTERNAL NAME SQLFunctions.[SQLFunctions.Regex].Replace;
go
-- outputs This is a test
SELECT dbo.RegexReplace('This is a test 12345','[0-9]','')
Content of the DLL:
I stumbled across this post looking for something else but thought I'd mention a solution I use which is far more efficient - and really should be the default implementation of any function when used with a set based query - which is to use a cross applied table function. Seems the topic is still active so hopefully this is useful to someone.
Example runtime on a few of the answers so far based on running recursive set based queries or scalar function, based on 1m rows test set removing the chars from a random newid, ranges from 34s to 2m05s for the WHILE loop examples and from 1m3s to {forever} for the function examples.
Using a table function with cross apply achieves the same goal in 10s. You may need to adjust it to suit your needs such as the max length it handles.
Function:
CREATE FUNCTION [dbo].[RemoveChars](#InputUnit VARCHAR(40))
RETURNS TABLE
AS
RETURN
(
WITH Numbers_prep(Number) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,Numbers(Number) AS
(
SELECT TOP (ISNULL(LEN(#InputUnit),0))
row_number() OVER (ORDER BY (SELECT NULL))
FROM Numbers_prep a
CROSS JOIN Numbers_prep b
)
SELECT
OutputUnit
FROM
(
SELECT
substring(#InputUnit,Number,1)
FROM Numbers
WHERE substring(#InputUnit,Number,1) like '%[0-9]%'
ORDER BY Number
FOR XML PATH('')
) Sub(OutputUnit)
)
Usage:
UPDATE t
SET column = o.OutputUnit
FROM ##t t
CROSS APPLY [dbo].[RemoveChars](t.column) o
Here is a function I wrote to accomplish this based off of the previous answers.
CREATE FUNCTION dbo.RepetitiveReplace
(
#P_String VARCHAR(MAX),
#P_Pattern VARCHAR(MAX),
#P_ReplaceString VARCHAR(MAX),
#P_ReplaceLength INT = 1
)
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #Index INT;
-- Get starting point of pattern
SET #Index = PATINDEX(#P_Pattern, #P_String);
while #Index > 0
begin
--replace matching charactger at index
SET #P_String = STUFF(#P_String, PATINDEX(#P_Pattern, #P_String), #P_ReplaceLength, #P_ReplaceString);
SET #Index = PATINDEX(#P_Pattern, #P_String);
end
RETURN #P_String;
END;
[Gist][1]
[1]: https://gist.github.com/jkdba/ca13fe8f2a9855c4bdbfd0a5d3dfcda2
Edit:
Originally I had a recursive function here which does not play well with sql server as it has a 32 nesting level limit which would result in an error like the below any time you attempt to make 32+ replacements with the function. Instead of trying to make a server level change to allow more nesting (which could be dangerous like allow never ending loops) switching to a while loop makes a lot more sense.
Maximum stored procedure, function, trigger, or view nesting level exceeded (limit 32).
Wrapping the solution inside a SQL function could be useful if you want to reuse it.
I'm even doing it at the cell level, that's why I'm putting this as a different answer:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(300))
RETURNS VARCHAR(300)
BEGIN
DECLARE #str VARCHAR(300) = #string;
DECLARE #Pattern VARCHAR (20) = '%[^a-zA-Z0-9]%';
DECLARE #Len INT;
SELECT #Len = LEN(#String);
WHILE #Len > 0
BEGIN
SET #Len = #Len - 1;
IF (PATINDEX(#Pattern,#str) > 0)
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
A more speedy approach for large strings would look something like this:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(MAX))
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #str VARCHAR(MAX) = #string;
DECLARE #Pattern VARCHAR (MAX) = '%[^a-zA-Z0-9]%';
WHILE PATINDEX(#Pattern,#str) > 0
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
RETURN #str
END
I've created this function to clean up a string that contained non numeric characters in a time field. The time contained question marks when they did not added the minutes, something like this 20:??. Function loops through each character and replaces the ? with a 0 :
CREATE FUNCTION [dbo].[CleanTime]
(
-- Add the parameters for the function here
#intime nvarchar(10)
)
RETURNS nvarchar(5)
AS
BEGIN
-- Declare the return variable here
DECLARE #ResultVar nvarchar(5)
DECLARE #char char(1)
-- Add the T-SQL statements to compute the return value here
DECLARE #i int = 1
WHILE #i <= LEN(#intime)
BEGIN
SELECT #char = CASE WHEN substring(#intime,#i,1) like '%[0-9:]%' THEN substring(#intime,#i,1) ELSE '0' END
SELECT #ResultVar = concat(#ResultVar,#char)
set #i = #i + 1
END;
-- Return the result of the function
RETURN #ResultVar
END
I think this solution is faster and simple. I use always CTE/recursive because WHILE is so slow on SQL Server.
I use it in projects I work with and large databases.
/*
Function: dbo.kSql_ReplaceRegExp
Create Date: 20.02.2021
Author: Karcan Ozbal
Description: The given string value will be replaced according to the given regexp/pattern.
Parameter(s): #Value : Value/Text to REPLACE.
#RegExp : The regexp/pattern to be used for REPLACE operation.
Usage: select dbo.kSql_ReplaceRegExp('2T3EST5','%[0-9]%')
Output: 'TEST'
*/
ALTER FUNCTION [dbo].[kSql_ReplaceRegExp](
#Value nvarchar(max),
#RegExp nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #Result nvarchar(max)
;WITH CTE AS (
SELECT NUM = 1, VALUE = #Value, IDX = PATINDEX(#RegExp, #Value)
UNION ALL
SELECT NUM + 1, VALUE = REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''), IDX = PATINDEX(#RegExp, REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''))
FROM CTE
WHERE IDX > 0
)
SELECT TOP(1) #Result = VALUE
FROM CTE
ORDER BY NUM DESC
OPTION (maxrecursion 0)
RETURN #Result
END
If you are doing this just for a parameter coming into a Stored Procedure, you can use the following:
declare #badIndex int
set #badIndex = PatIndex('%[^0-9]%', #Param)
while #badIndex > 0
set #Param = Replace(#Param, Substring(#Param, #badIndex, 1), '')
set #badIndex = PatIndex('%[^0-9]%', #Param)
I thought this was clearer:
ALTER FUNCTION [dbo].[func_ReplaceChars](
#Value nvarchar(max),
#Chars nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #cLen int = len(#Chars);
DECLARE #curChar int = 0;
WHILE #curChar<#cLen
BEGIN
set #Value = replace(#Value,substring(#Chars,#curChar,1),'');
set #curChar = #curChar + 1;
END;
RETURN #Value
END
I'm using this code similar to several codes above:
DROP FUNCTION [dbo].[fnCleanString]
GO
CREATE FUNCTION [dbo].[fnCleanString] (#input VARCHAR(max), #Pattern
VARCHAR (20))
RETURNS VARCHAR(max)
BEGIN
DECLARE #str VARCHAR(max) = #input;
DECLARE #Len INT;
DECLARE #INDEX INT;
SELECT #Len = LEN(#input);
WHILE #Len > 0
BEGIN
SET #INDEX = PATINDEX(#Pattern,#str);
IF (#INDEX > 0)
BEGIN
SET #str=REPLACE(#str,SUBSTRING(#str,#INDEX, 1), '');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
You can use it like this:
SELECT CleanName = dbo.[fnCleanString](Name, '%[0-9]%') from YourTable
I think a simpler and faster approach is iterate by each character of the alphabet:
DECLARE #i int
SET #i = 0
WHILE(#i < 256)
BEGIN
IF char(#i) NOT IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.')
UPDATE Table SET Column = replace(Column, char(#i), '')
SET #i = #i + 1
END
SELECT REPLACE('<strong>100</strong><b>.00 GB', '%^(^-?\d*\.{0,1}\d+$)%', '');
I want to replace any markup between two parts of the number with above regex, but it does not seem to work. I'm not sure if it is regex syntax that's wrong because I tried simpler one such as '%[^0-9]%' just to test but it didn't work either. Does anyone know how can I achieve this?
You can use PATINDEX
to find the first index of the pattern (string's) occurrence. Then use STUFF to stuff another string into the pattern(string) matched.
Loop through each row. Replace each illegal characters with what you want. In your case replace non numeric with blank. The inner loop is if you have more than one illegal character in a current cell that of the loop.
DECLARE #counter int
SET #counter = 0
WHILE(#counter < (SELECT MAX(ID_COLUMN) FROM Table))
BEGIN
WHILE 1 = 1
BEGIN
DECLARE #RetVal varchar(50)
SET #RetVal = (SELECT Column = STUFF(Column, PATINDEX('%[^0-9.]%', Column),1, '')
FROM Table
WHERE ID_COLUMN = #counter)
IF(#RetVal IS NOT NULL)
UPDATE Table SET
Column = #RetVal
WHERE ID_COLUMN = #counter
ELSE
break
END
SET #counter = #counter + 1
END
Caution: This is slow though! Having a varchar column may impact. So using LTRIM RTRIM may help a bit. Regardless, it is slow.
Credit goes to this StackOverFlow answer.
EDIT
Credit also goes to #srutzky
Edit (by #Tmdean)
Instead of doing one row at a time, this answer can be adapted to a more set-based solution. It still iterates the max of the number of non-numeric characters in a single row, so it's not ideal, but I think it should be acceptable in most situations.
WHILE 1 = 1 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, '')
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 BREAK;
END;
You can also improve efficiency quite a lot if you maintain a bit column in the table that indicates whether the field has been scrubbed yet. (NULL represents "Unknown" in my example and should be the column default.)
DECLARE #done bit = 0;
WHILE #done = 0 BEGIN
WITH q AS
(SELECT ID_Column, PATINDEX('%[^0-9.]%', Column) AS n
FROM Table
WHERE COALESCE(Scrubbed_Column, 0) = 0)
UPDATE Table
SET Column = STUFF(Column, q.n, 1, ''),
Scrubbed_Column = 0
FROM q
WHERE Table.ID_Column = q.ID_Column AND q.n != 0;
IF ##ROWCOUNT = 0 SET #done = 1;
-- if Scrubbed_Column is still NULL, then the PATINDEX
-- must have given 0
UPDATE table
SET Scrubbed_Column = CASE
WHEN Scrubbed_Column IS NULL THEN 1
ELSE NULLIF(Scrubbed_Column, 0)
END;
END;
If you don't want to change your schema, this is easy to adapt to store intermediate results in a table valued variable which gets applied to the actual table at the end.
Instead of stripping out the found character by its sole position, using Replace(Column, BadFoundCharacter, '') could be substantially faster. Additionally, instead of just replacing the one bad character found next in each column, this replaces all those found.
WHILE 1 = 1 BEGIN
UPDATE dbo.YourTable
SET Column = Replace(Column, Substring(Column, PatIndex('%[^0-9.-]%', Column), 1), '')
WHERE Column LIKE '%[^0-9.-]%'
If ##RowCount = 0 BREAK;
END;
I am convinced this will work better than the accepted answer, if only because it does fewer operations. There are other ways that might also be faster, but I don't have time to explore those right now.
In a general sense, SQL Server does not support regular expressions and you cannot use them in the native T-SQL code.
You could write a CLR function to do that. See here, for example.
For those looking for a performant and easy solution and are willing to enable CLR:
CREATE database TestSQLFunctions
go
use TestSQLFunctions
go
ALTER database TestSQLFunctions set trustworthy on
EXEC sp_configure 'clr enabled', 1
RECONFIGURE WITH OVERRIDE
go
CREATE ASSEMBLY [SQLFunctions]
AUTHORIZATION [dbo]
FROM 0x4D5A90000300000004000000FFFF0000B800000000000000400000000000000000000000000000000000000000000000000000000000000000000000800000000E1FBA0E00B409CD21B8014CCD21546869732070726F6772616D2063616E6E6F742062652072756E20696E20444F53206D6F64652E0D0D0A2400000000000000504500004C0103004BE8B85F0000000000000000E00022200B013000000800000006000000000000C2270000002000000040000000000010002000000002000004000000000000000600000000000000008000000002000000000000030060850000100000100000000010000010000000000000100000000000000000000000702700004F000000004000009803000000000000000000000000000000000000006000000C00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000080000000000000000000000082000004800000000000000000000002E74657874000000C8070000002000000008000000020000000000000000000000000000200000602E72737263000000980300000040000000040000000A0000000000000000000000000000400000402E72656C6F6300000C0000000060000000020000000E00000000000000000000000000004000004200000000000000000000000000000000A4270000000000004800000002000500682000000807000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003A022D02022A020304281000000A2A1E02281100000A2A0042534A4201000100000000000C00000076342E302E33303331390000000005006C00000018020000237E000084020000CC02000023537472696E6773000000005005000004000000235553005405000010000000234755494400000064050000A401000023426C6F620000000000000002000001471500000900000000FA0133001600000100000013000000020000000200000003000000110000000F00000001000000030000000000CA01010000000000060025014F02060092014F02060044001D020F006F02000006006C00E20106000801E2010600D400E20106007901E20106004501E20106005E01E20106008300E2010600580030020600360030020600B700E20106009E00B0010600AA02DB010A00F300FC010A001F00FC010E00C3027E02000000000100000000000100010001001000C3029D0241000100010050200000000096002E001A0001005F2000000000861817020600040000000100BD0200000200F40100000300B102090017020100110017020600190017020A0029001702100031001702100039001702100041001702100049001702100051001702100059001702100061001702150069001702100071001702100079001702100089001702060099002E001A0081001702060020007B0010012E000B002A002E00130033002E001B0052002E0023005B002E002B006D002E0033006D002E003B006D002E0043005B002E004B0073002E0053006D002E005B006D002E0063008B002E006B00B5002E007300C2000480000001000000000000000000000000009D020000040000000000000000000000210016000000000004000000000000000000000021000A00000000000400000000000000000000002100DB010000000000000000003C4D6F64756C653E0053797374656D2E44617461006D73636F726C696200446174614163636573734B696E64005265706C61636500477569644174747269627574650044656275676761626C6541747472696275746500436F6D56697369626C6541747472696275746500417373656D626C795469746C6541747472696275746500417373656D626C7954726164656D61726B417474726962757465005461726765744672616D65776F726B41747472696275746500417373656D626C7946696C6556657273696F6E41747472696275746500417373656D626C79436F6E66696775726174696F6E4174747269627574650053716C46756E6374696F6E41747472696275746500417373656D626C794465736372697074696F6E41747472696275746500436F6D70696C6174696F6E52656C61786174696F6E7341747472696275746500417373656D626C7950726F6475637441747472696275746500417373656D626C79436F7079726967687441747472696275746500417373656D626C79436F6D70616E794174747269627574650052756E74696D65436F6D7061746962696C6974794174747269627574650053797374656D2E52756E74696D652E56657273696F6E696E670053514C46756E6374696F6E732E646C6C0053797374656D0053797374656D2E5265666C656374696F6E007061747465726E004D6963726F736F66742E53716C5365727665722E536572766572002E63746F720053797374656D2E446961676E6F73746963730053797374656D2E52756E74696D652E496E7465726F7053657276696365730053797374656D2E52756E74696D652E436F6D70696C6572536572766963657300446562756767696E674D6F6465730053797374656D2E546578742E526567756C617245787072657373696F6E730053514C46756E6374696F6E73004F626A656374007265706C6163656D656E7400696E70757400526567657800000000000000003A1617E607071B47B964858BCD87458B00042001010803200001052001011111042001010E04200101020600030E0E0E0E08B77A5C561934E0890801000800000000001E01000100540216577261704E6F6E457863657074696F6E5468726F7773010801000200000000001101000C53514C46756E6374696F6E73000005010000000017010012436F7079726967687420C2A920203230323000002901002434346436386231632D393735312D343938612D396665352D32316666333934303738303900000C010007312E302E302E3000004D01001C2E4E45544672616D65776F726B2C56657273696F6E3D76342E352E320100540E144672616D65776F726B446973706C61794E616D65142E4E4554204672616D65776F726B20342E352E32808F010001005455794D6963726F736F66742E53716C5365727665722E5365727665722E446174614163636573734B696E642C2053797374656D2E446174612C2056657273696F6E3D342E302E302E302C2043756C747572653D6E65757472616C2C205075626C69634B6579546F6B656E3D623737613563353631393334653038390A4461746141636365737301000000000000982700000000000000000000B2270000002000000000000000000000000000000000000000000000A4270000000000000000000000005F436F72446C6C4D61696E006D73636F7265652E646C6C0000000000FF25002000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001001000000018000080000000000000000000000000000001000100000030000080000000000000000000000000000001000000000048000000584000003C03000000000000000000003C0334000000560053005F00560045005200530049004F004E005F0049004E0046004F0000000000BD04EFFE00000100000001000000000000000100000000003F000000000000000400000002000000000000000000000000000000440000000100560061007200460069006C00650049006E0066006F00000000002400040000005400720061006E0073006C006100740069006F006E00000000000000B0049C020000010053007400720069006E006700460069006C00650049006E0066006F0000007802000001003000300030003000300034006200300000001A000100010043006F006D006D0065006E007400730000000000000022000100010043006F006D00700061006E0079004E0061006D006500000000000000000042000D000100460069006C0065004400650073006300720069007000740069006F006E0000000000530051004C00460075006E006300740069006F006E00730000000000300008000100460069006C006500560065007200730069006F006E000000000031002E0030002E0030002E003000000042001100010049006E007400650072006E0061006C004E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000004800120001004C006500670061006C0043006F007000790072006900670068007400000043006F0070007900720069006700680074002000A90020002000320030003200300000002A00010001004C006500670061006C00540072006100640065006D00610072006B00730000000000000000004A00110001004F0072006900670069006E0061006C00460069006C0065006E0061006D0065000000530051004C00460075006E006300740069006F006E0073002E0064006C006C00000000003A000D000100500072006F0064007500630074004E0061006D00650000000000530051004C00460075006E006300740069006F006E00730000000000340008000100500072006F006400750063007400560065007200730069006F006E00000031002E0030002E0030002E003000000038000800010041007300730065006D0062006C0079002000560065007200730069006F006E00000031002E0030002E0030002E0030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000C000000C43700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
WITH PERMISSION_SET = SAFE
go
CREATE FUNCTION RegexReplace(
#input nvarchar(max),
#pattern nvarchar(max),
#replacement nvarchar(max)
) RETURNS nvarchar (max)
AS EXTERNAL NAME SQLFunctions.[SQLFunctions.Regex].Replace;
go
-- outputs This is a test
SELECT dbo.RegexReplace('This is a test 12345','[0-9]','')
Content of the DLL:
I stumbled across this post looking for something else but thought I'd mention a solution I use which is far more efficient - and really should be the default implementation of any function when used with a set based query - which is to use a cross applied table function. Seems the topic is still active so hopefully this is useful to someone.
Example runtime on a few of the answers so far based on running recursive set based queries or scalar function, based on 1m rows test set removing the chars from a random newid, ranges from 34s to 2m05s for the WHILE loop examples and from 1m3s to {forever} for the function examples.
Using a table function with cross apply achieves the same goal in 10s. You may need to adjust it to suit your needs such as the max length it handles.
Function:
CREATE FUNCTION [dbo].[RemoveChars](#InputUnit VARCHAR(40))
RETURNS TABLE
AS
RETURN
(
WITH Numbers_prep(Number) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,Numbers(Number) AS
(
SELECT TOP (ISNULL(LEN(#InputUnit),0))
row_number() OVER (ORDER BY (SELECT NULL))
FROM Numbers_prep a
CROSS JOIN Numbers_prep b
)
SELECT
OutputUnit
FROM
(
SELECT
substring(#InputUnit,Number,1)
FROM Numbers
WHERE substring(#InputUnit,Number,1) like '%[0-9]%'
ORDER BY Number
FOR XML PATH('')
) Sub(OutputUnit)
)
Usage:
UPDATE t
SET column = o.OutputUnit
FROM ##t t
CROSS APPLY [dbo].[RemoveChars](t.column) o
Here is a function I wrote to accomplish this based off of the previous answers.
CREATE FUNCTION dbo.RepetitiveReplace
(
#P_String VARCHAR(MAX),
#P_Pattern VARCHAR(MAX),
#P_ReplaceString VARCHAR(MAX),
#P_ReplaceLength INT = 1
)
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #Index INT;
-- Get starting point of pattern
SET #Index = PATINDEX(#P_Pattern, #P_String);
while #Index > 0
begin
--replace matching charactger at index
SET #P_String = STUFF(#P_String, PATINDEX(#P_Pattern, #P_String), #P_ReplaceLength, #P_ReplaceString);
SET #Index = PATINDEX(#P_Pattern, #P_String);
end
RETURN #P_String;
END;
[Gist][1]
[1]: https://gist.github.com/jkdba/ca13fe8f2a9855c4bdbfd0a5d3dfcda2
Edit:
Originally I had a recursive function here which does not play well with sql server as it has a 32 nesting level limit which would result in an error like the below any time you attempt to make 32+ replacements with the function. Instead of trying to make a server level change to allow more nesting (which could be dangerous like allow never ending loops) switching to a while loop makes a lot more sense.
Maximum stored procedure, function, trigger, or view nesting level exceeded (limit 32).
Wrapping the solution inside a SQL function could be useful if you want to reuse it.
I'm even doing it at the cell level, that's why I'm putting this as a different answer:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(300))
RETURNS VARCHAR(300)
BEGIN
DECLARE #str VARCHAR(300) = #string;
DECLARE #Pattern VARCHAR (20) = '%[^a-zA-Z0-9]%';
DECLARE #Len INT;
SELECT #Len = LEN(#String);
WHILE #Len > 0
BEGIN
SET #Len = #Len - 1;
IF (PATINDEX(#Pattern,#str) > 0)
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
A more speedy approach for large strings would look something like this:
CREATE FUNCTION [dbo].[fnReplaceInvalidChars] (#string VARCHAR(MAX))
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #str VARCHAR(MAX) = #string;
DECLARE #Pattern VARCHAR (MAX) = '%[^a-zA-Z0-9]%';
WHILE PATINDEX(#Pattern,#str) > 0
BEGIN
SELECT #str = STUFF(#str, PATINDEX(#Pattern,#str),1,'');
END
RETURN #str
END
I've created this function to clean up a string that contained non numeric characters in a time field. The time contained question marks when they did not added the minutes, something like this 20:??. Function loops through each character and replaces the ? with a 0 :
CREATE FUNCTION [dbo].[CleanTime]
(
-- Add the parameters for the function here
#intime nvarchar(10)
)
RETURNS nvarchar(5)
AS
BEGIN
-- Declare the return variable here
DECLARE #ResultVar nvarchar(5)
DECLARE #char char(1)
-- Add the T-SQL statements to compute the return value here
DECLARE #i int = 1
WHILE #i <= LEN(#intime)
BEGIN
SELECT #char = CASE WHEN substring(#intime,#i,1) like '%[0-9:]%' THEN substring(#intime,#i,1) ELSE '0' END
SELECT #ResultVar = concat(#ResultVar,#char)
set #i = #i + 1
END;
-- Return the result of the function
RETURN #ResultVar
END
I think this solution is faster and simple. I use always CTE/recursive because WHILE is so slow on SQL Server.
I use it in projects I work with and large databases.
/*
Function: dbo.kSql_ReplaceRegExp
Create Date: 20.02.2021
Author: Karcan Ozbal
Description: The given string value will be replaced according to the given regexp/pattern.
Parameter(s): #Value : Value/Text to REPLACE.
#RegExp : The regexp/pattern to be used for REPLACE operation.
Usage: select dbo.kSql_ReplaceRegExp('2T3EST5','%[0-9]%')
Output: 'TEST'
*/
ALTER FUNCTION [dbo].[kSql_ReplaceRegExp](
#Value nvarchar(max),
#RegExp nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #Result nvarchar(max)
;WITH CTE AS (
SELECT NUM = 1, VALUE = #Value, IDX = PATINDEX(#RegExp, #Value)
UNION ALL
SELECT NUM + 1, VALUE = REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''), IDX = PATINDEX(#RegExp, REPLACE(VALUE, SUBSTRING(VALUE,IDX,1),''))
FROM CTE
WHERE IDX > 0
)
SELECT TOP(1) #Result = VALUE
FROM CTE
ORDER BY NUM DESC
OPTION (maxrecursion 0)
RETURN #Result
END
If you are doing this just for a parameter coming into a Stored Procedure, you can use the following:
declare #badIndex int
set #badIndex = PatIndex('%[^0-9]%', #Param)
while #badIndex > 0
set #Param = Replace(#Param, Substring(#Param, #badIndex, 1), '')
set #badIndex = PatIndex('%[^0-9]%', #Param)
I thought this was clearer:
ALTER FUNCTION [dbo].[func_ReplaceChars](
#Value nvarchar(max),
#Chars nvarchar(50)
)
RETURNS nvarchar(max)
AS
BEGIN
DECLARE #cLen int = len(#Chars);
DECLARE #curChar int = 0;
WHILE #curChar<#cLen
BEGIN
set #Value = replace(#Value,substring(#Chars,#curChar,1),'');
set #curChar = #curChar + 1;
END;
RETURN #Value
END
I'm using this code similar to several codes above:
DROP FUNCTION [dbo].[fnCleanString]
GO
CREATE FUNCTION [dbo].[fnCleanString] (#input VARCHAR(max), #Pattern
VARCHAR (20))
RETURNS VARCHAR(max)
BEGIN
DECLARE #str VARCHAR(max) = #input;
DECLARE #Len INT;
DECLARE #INDEX INT;
SELECT #Len = LEN(#input);
WHILE #Len > 0
BEGIN
SET #INDEX = PATINDEX(#Pattern,#str);
IF (#INDEX > 0)
BEGIN
SET #str=REPLACE(#str,SUBSTRING(#str,#INDEX, 1), '');
END
ELSE
BEGIN
BREAK;
END
END
RETURN #str
END
You can use it like this:
SELECT CleanName = dbo.[fnCleanString](Name, '%[0-9]%') from YourTable
I think a simpler and faster approach is iterate by each character of the alphabet:
DECLARE #i int
SET #i = 0
WHILE(#i < 256)
BEGIN
IF char(#i) NOT IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.')
UPDATE Table SET Column = replace(Column, char(#i), '')
SET #i = #i + 1
END
Is there a way to make a TSQL variable constant?
No, but you can create a function and hardcode it in there and use that.
Here is an example:
CREATE FUNCTION fnConstant()
RETURNS INT
AS
BEGIN
RETURN 2
END
GO
SELECT dbo.fnConstant()
One solution, offered by Jared Ko is to use pseudo-constants.
As explained in SQL Server: Variables, Parameters or Literals? Or… Constants?:
Pseudo-Constants are not variables or parameters. Instead, they're simply views with one row, and enough columns to support your constants. With these simple rules, the SQL Engine completely ignores the value of the view but still builds an execution plan based on its value. The execution plan doesn't even show a join to the view!
Create like this:
CREATE SCHEMA ShipMethod
GO
-- Each view can only have one row.
-- Create one column for each desired constant.
-- Each column is restricted to a single value.
CREATE VIEW ShipMethod.ShipMethodID AS
SELECT CAST(1 AS INT) AS [XRQ - TRUCK GROUND]
,CAST(2 AS INT) AS [ZY - EXPRESS]
,CAST(3 AS INT) AS [OVERSEAS - DELUXE]
,CAST(4 AS INT) AS [OVERNIGHT J-FAST]
,CAST(5 AS INT) AS [CARGO TRANSPORT 5]
Then use like this:
SELECT h.*
FROM Sales.SalesOrderHeader h
JOIN ShipMethod.ShipMethodID const
ON h.ShipMethodID = const.[OVERNIGHT J-FAST]
Or like this:
SELECT h.*
FROM Sales.SalesOrderHeader h
WHERE h.ShipMethodID = (SELECT TOP 1 [OVERNIGHT J-FAST] FROM ShipMethod.ShipMethodID)
My workaround to missing constans is to give hints about the value to the optimizer.
DECLARE #Constant INT = 123;
SELECT *
FROM [some_relation]
WHERE [some_attribute] = #Constant
OPTION( OPTIMIZE FOR (#Constant = 123))
This tells the query compiler to treat the variable as if it was a constant when creating the execution plan. The down side is that you have to define the value twice.
No, but good old naming conventions should be used.
declare #MY_VALUE as int
There is no built-in support for constants in T-SQL. You could use SQLMenace's approach to simulate it (though you can never be sure whether someone else has overwritten the function to return something else…), or possibly write a table containing constants, as suggested over here. Perhaps write a trigger that rolls back any changes to the ConstantValue column?
Prior to using a SQL function run the following script to see the differences in performance:
IF OBJECT_ID('fnFalse') IS NOT NULL
DROP FUNCTION fnFalse
GO
IF OBJECT_ID('fnTrue') IS NOT NULL
DROP FUNCTION fnTrue
GO
CREATE FUNCTION fnTrue() RETURNS INT WITH SCHEMABINDING
AS
BEGIN
RETURN 1
END
GO
CREATE FUNCTION fnFalse() RETURNS INT WITH SCHEMABINDING
AS
BEGIN
RETURN ~ dbo.fnTrue()
END
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = dbo.fnTrue()
IF #Value = 1
SELECT #Value = dbo.fnFalse()
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using function'
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
DECLARE #FALSE AS BIT = 0
DECLARE #TRUE AS BIT = ~ #FALSE
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = #TRUE
IF #Value = 1
SELECT #Value = #FALSE
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using local variable'
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = 1
IF #Value = 1
SELECT #Value = 0
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using hard coded values'
GO
If you are interested in getting optimal execution plan for a value in the variable you can use a dynamic sql code. It makes the variable constant.
DECLARE #var varchar(100) = 'some text'
DECLARE #sql varchar(MAX)
SET #sql = 'SELECT * FROM table WHERE col = '''+#var+''''
EXEC (#sql)
For enums or simple constants, a view with a single row has great performance and compile time checking / dependency tracking ( cause its a column name )
See Jared Ko's blog post https://blogs.msdn.microsoft.com/sql_server_appendix_z/2013/09/16/sql-server-variables-parameters-or-literals-or-constants/
create the view
CREATE VIEW ShipMethods AS
SELECT CAST(1 AS INT) AS [XRQ - TRUCK GROUND]
,CAST(2 AS INT) AS [ZY - EXPRESS]
,CAST(3 AS INT) AS [OVERSEAS - DELUXE]
, CAST(4 AS INT) AS [OVERNIGHT J-FAST]
,CAST(5 AS INT) AS [CARGO TRANSPORT 5]
use the view
SELECT h.*
FROM Sales.SalesOrderHeader
WHERE ShipMethodID = ( select [OVERNIGHT J-FAST] from ShipMethods )
Okay, lets see
Constants are immutable values which are known at compile time and do not change for the life of the program
that means you can never have a constant in SQL Server
declare #myvalue as int
set #myvalue = 5
set #myvalue = 10--oops we just changed it
the value just changed
Since there is no build in support for constants, my solution is very simple.
Since this is not supported:
Declare Constant #supplement int = 240
SELECT price + #supplement
FROM what_does_it_cost
I would simply convert it to
SELECT price + 240/*CONSTANT:supplement*/
FROM what_does_it_cost
Obviously, this relies on the whole thing (the value without trailing space and the comment) to be unique. Changing it is possible with a global search and replace.
There are no such thing as "creating a constant" in database literature. Constants exist as they are and often called values. One can declare a variable and assign a value (constant) to it. From a scholastic view:
DECLARE #two INT
SET #two = 2
Here #two is a variable and 2 is a value/constant.
SQLServer 2022 (currently only as Preview available) is now able to Inline the function proposed by SQLMenace, this should prevent the performance hit described by some comments.
CREATE FUNCTION fnConstant() RETURNS INT AS BEGIN RETURN 2 END GO
SELECT is_inlineable FROM sys.sql_modules WHERE [object_id]=OBJECT_ID('dbo.fnConstant');
is_inlineable
1
SELECT dbo.fnConstant()
ExecutionPlan
To test if it also uses the value coming from the Function, I added a second function returning value "1"
CREATE FUNCTION fnConstant1()
RETURNS INT
AS
BEGIN
RETURN 1
END
GO
Create Temp Table with about 500k rows with Value 1 and 4 rows with Value 2:
DROP TABLE IF EXISTS #temp ;
create table #temp (value_int INT)
DECLARE #counter INT;
SET #counter = 0
WHILE #counter <= 500000
BEGIN
INSERT INTO #temp VALUES (1);
SET #counter = #counter +1
END
SET #counter = 0
WHILE #counter <= 3
BEGIN
INSERT INTO #temp VALUES (2);
SET #counter = #counter +1
END
create index i_temp on #temp (value_int);
Using the describe plan we can see that the Optimizer expects 500k values for
select * from #temp where value_int = dbo.fnConstant1(); --Returns 500001 rows
Constant 1
and 4 rows for
select * from #temp where value_int = dbo.fnConstant(); --Returns 4rows
Constant 2
Robert's performance test is interesting. And even in late 2022, the scalar functions are much slower (by an order of magnitude) than variables or literals. A view (as suggested mbobka) is somewhere in-between when used for this same test.
That said, using a loop like that in SQL Server is not something I'd ever do, because I'd normally be operating on a whole set.
In SQL 2019, if you use schema-bound functions in a set operation, the difference is much less noticeable.
I created and populated a test table:
create table #testTable (id int identity(1, 1) primary key, value tinyint);
And changed the test so that instead of looping and changing a variable, it queries the test table and returns true or false depending on the value in the test table, e.g.:
insert #testTable(value)
select case when value > 127
then #FALSE
else #TRUE
end
from #testTable with(nolock)
I tested 5 scenarios:
hard-coded values
local variables
scalar functions
a view
a table-valued function
running the test 10 times, yielded the following results:
scenario
min
max
avg
scalar functions
233
259
240
hard-coded values
236
265
243
local variables
235
278
245
table-valued function
243
272
253
view
244
267
254
Suggesting to me, that for set-based work in (at least) 2019 and better, there's not much in it.
set nocount on;
go
-- create test data table
drop table if exists #testTable;
create table #testTable (id int identity(1, 1) primary key, value tinyint);
-- populate test data
insert #testTable (value)
select top (1000000) convert(binary (1), newid())
from sys.all_objects a
, sys.all_objects b
go
-- scalar function for True
drop function if exists fnTrue;
go
create function dbo.fnTrue() returns bit with schemabinding as
begin
return 1
end
go
-- scalar function for False
drop function if exists fnFalse;
go
create function dbo.fnFalse () returns bit with schemabinding as
begin
return 0
end
go
-- table-valued function for booleans
drop function if exists dbo.tvfBoolean;
go
create function tvfBoolean() returns table with schemabinding as
return
select convert(bit, 1) as true, convert(bit, 0) as false
go
-- view for booleans
drop view if exists dbo.viewBoolean;
go
create view dbo.viewBoolean with schemabinding as
select convert(bit, 1) as true, convert(bit, 0) as false
go
-- create table for results
drop table if exists #testResults
create table #testResults (id int identity(1,1), test int, elapsed bigint, message varchar(1000));
-- define tests
declare #tests table(testNumber int, description nvarchar(100), sql nvarchar(max))
insert #tests values
(1, N'hard-coded values', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then 0
else 1
end
from #testTable t')
, (2, N'local variables', N'
declare #FALSE as bit = 0
declare #TRUE as bit = 1
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then #FALSE
else #TRUE
end
from #testTable t'),
(3, N'scalar functions', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then dbo.fnFalse()
else dbo.fnTrue()
end
from #testTable t'),
(4, N'view', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when value > 127
then b.false
else b.true
end
from #testTable t with(nolock), viewBoolean b'),
(5, N'table-valued function', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when value > 127
then b.false
else b.true
end
from #testTable with(nolock), dbo.tvfBoolean() b')
;
declare #testNumber int, #description varchar(100), #sql nvarchar(max)
declare #testRuns int = 10;
-- execute tests
while #testRuns > 0 begin
set #testRuns -= 1
declare testCursor cursor for select testNumber, description, sql from #tests;
open testCursor
fetch next from testCursor into #testNumber, #description, #sql
while ##FETCH_STATUS = 0 begin
declare #TimeStart datetime2(7) = sysdatetime();
execute sp_executesql #sql;
declare #TimeEnd datetime2(7) = sysdatetime()
insert #testResults(test, elapsed, message)
select #testNumber, datediff_big(ms, #TimeStart, #TimeEnd), #description
fetch next from testCursor into #testNumber, #description, #sql
end
close testCursor
deallocate testCursor
end
-- display results
select test, message, count(*) runs, min(elapsed) as min, max(elapsed) as max, avg(elapsed) as avg
from #testResults
group by test, message
order by avg(elapsed);
The best answer is from SQLMenace according to the requirement if that is to create a temporary constant for use within scripts, i.e. across multiple GO statements/batches.
Just create the procedure in the tempdb then you have no impact on the target database.
One practical example of this is a database create script which writes a control value at the end of the script containing the logical schema version. At the top of the file are some comments with change history etc... But in practice most developers will forget to scroll down and update the schema version at the bottom of the file.
Using the above code allows a visible schema version constant to be defined at the top before the database script (copied from the generate scripts feature of SSMS) creates the database but used at the end. This is right in the face of the developer next to the change history and other comments, so they are very likely to update it.
For example:
use tempdb
go
create function dbo.MySchemaVersion()
returns int
as
begin
return 123
end
go
use master
go
-- Big long database create script with multiple batches...
print 'Creating database schema version ' + CAST(tempdb.dbo.MySchemaVersion() as NVARCHAR) + '...'
go
-- ...
go
-- ...
go
use MyDatabase
go
-- Update schema version with constant at end (not normally possible as GO puts
-- local #variables out of scope)
insert MyConfigTable values ('SchemaVersion', tempdb.dbo.MySchemaVersion())
go
-- Clean-up
use tempdb
drop function MySchemaVersion
go