Delay in sql server query result with leading and trailing character removed - sql-server

I have a query in SQL Server 2012 that takes 8 seconds to retrieve 1158 rows. In the query I have to join on two fields, I have to trim leading and trailing 0s while joining.
below Query : It takes 8 seconds to get 1158 rows
Select value1, value2
from TableA LEFT JOIN TableB
ON
SUBSTRING(REVERSE(SUBSTRING(REVERSE(TableA.[policy#]),PATINDEX('%[^' + '0' + ' ]%',REVERSE(TableA.[policy#])),100)),PATINDEX('%[^' + '0' + ' ]%',REVERSE(SUBSTRING(REVERSE(TableA.[policy#]),PATINDEX('%[^' + '0' + ' ]%',REVERSE(TableA.[policy#])),100))),100)
= SUBSTRING(REVERSE(SUBSTRING(REVERSE(TableB.[policy#]),PATINDEX('%[^' + '0' + ' ]%',REVERSE(TableB.[policy#])),100)),PATINDEX('%[^' + '0' + ' ]%',REVERSE(SUBSTRING(REVERSE(TableB.[policy#]),PATINDEX('%[^' + '0' + ' ]%',REVERSE(TableB.[policy#])),100))),100)
WHERE {some conditions}
Instead of writing the ugly code for removing leading and trailing 0s I have created two functions to do the same
ALTER FUNCTION [dbo].[L_TRIM](#String VARCHAR(MAX), #Char varchar(5))
RETURNS VARCHAR(MAX)
BEGIN
RETURN SUBSTRING(#String,PATINDEX('%[^' + #Char + ' ]%',#String),100)
END
ALTER FUNCTION [dbo].[R_TRIM](#String VARCHAR(MAX), #Char varchar(5))
RETURNS VARCHAR(MAX)
BEGIN
RETURN REVERSE(SUBSTRING(REVERSE(#String),PATINDEX('%[^' + #Char + ' ]%',REVERSE(#String)),100))
END
Below Query : It takes 1 minute 40 seconds to get 1158 rows
Select value1, value2
from TableA LEFT JOIN TableB
ON
dbo.L_trim(dbo.R_trim( TableA.[policy#], '0'),'0') = dbo.L_trim(dbo.R_trim(TableB.[policy#], '0'),'0')
WHERE {some conditions}
Can someone guide me how can I retrieve the rows in lesser time without writing ugly code ?

Related

Need help adding WHERE clause in pre-written SQL statement

Preface: my SQL is rudimentary. I received a SQL query from a vendor, it selects and exports every single employee comment and other data from a few different DBs as CSV meant for import, it was written by them but they're not helping with this request. The query is pulling so much data it makes a large time consuming file for import. So I want to add to / modify the query to have a "WHERE date > whateverdate" to narrow my results to recent data. For example, I want to pull only comments entered in the past 2 days.
The column I'm looking to add the clause for is the column "A.CMS502", defined as datetime. I believe this is the only relevant column in this query. An example date in this column is "2003-10-06 17:05:21.000". I am using SQL Server 2008 if it helps. Is it possible here? Thank you.
SELECT
'ID,Acct/LnNbr,NoteCreatedDate,CollectorId,ApplytoAll,Note'
UNION ALL
SELECT
ID + ',' + ID + ',' + NoteCreatedDate + ',' + CollectorId + ',' + 'No' + ',' + Note
FROM
(SELECT
CASE WHEN SUBSTRING(A.CMS301,LEN(A.CMS301),1) = 'S'
THEN SUBSTRING(A.CMS301,1,LEN(A.CMS301) - 1)
ELSE A.CMS301
END + '-' +
CASE WHEN SUBSTRING(A.CMS301,LEN(A.CMS301),1) = 'S'
THEN 'S' ELSE 'L'
END AS [ID],
REPLACE(CONVERT(VARCHAR, A.CMS501, 10), '-', '') AS [NoteCreatedDate],
CASE WHEN U.CMS1201 IS NOT NULL
THEN U.CMS1205 + ' ' + U.CMS1204
ELSE (SELECT CMS1205 + ' ' + CMS1204 FROM sysUSER WHERE CMS1201 = 'PSUSER')
END AS CollectorId,
CAST(A.CMS512 AS NVARCHAR(MAX)) AS [Note]
FROM
ACTIVITY AS A
LEFT JOIN
sysUSER AS U ON A.CMS503 = U.CMS1201
WHERE
A.CMS504 NOT IN (411,500,511,711,804,900,901,903,907,2000,999777)
AND A.CMS504 NOT BETWEEN 1102 AND 1199) AS S
Try this, this will output last 2 days.
SELECT 'ID,Acct/LnNbr,NoteCreatedDate,CollectorId,ApplytoAll,Note'
UNION ALL
SELECT ID + ',' + ID + ',' + NoteCreatedDate + ',' + CollectorId + ',' + 'No' + ',' + Note
FROM
(
SELECT CASE WHEN SUBSTRING(A.CMS301,LEN(A.CMS301),1) = 'S' THEN SUBSTRING(A.CMS301,1,LEN(A.CMS301) - 1) ELSE A.CMS301 END
+ '-' + CASE WHEN SUBSTRING(A.CMS301,LEN(A.CMS301),1) = 'S' THEN 'S' ELSE 'L'
END AS [ID]
,REPLACE(CONVERT(varchar,A.CMS501,10),'-','') AS [NoteCreatedDate]
,CASE WHEN U.CMS1201 IS NOT NULL THEN U.CMS1205 + ' ' + U.CMS1204 ELSE
(SELECT CMS1205 + ' ' + CMS1204 FROM sysUSER WHERE CMS1201 = 'PSUSER')
END AS CollectorId
,CAST(A.CMS512 AS nvarchar(max)) AS [Note]
FROM ACTIVITY AS A
LEFT JOIN sysUSER AS U
ON A.CMS503 = U.CMS1201
WHERE A.CMS504 NOT IN (411,500,511,711,804,900,901,903,907,2000,999777)
AND A.CMS504 NOT BETWEEN 1102 AND 1199
AND A.CMS502 >= DATEADD(D, -2, GETDATE())
) AS S

SQL Server build dynamic sql

I have a temp table called #temp, and I need to get all the CDate column from that table, to build a string.
The CDate list in that table is (20171209, 20171210....20171223)
I expected to see
'A.[20171209] as [20171209], A.[20171210] as [20171210],
A.[20171211] as [20171211], A.[20171212] as [20171212],
A.[20171213] as [20171213], A.[20171214] as [20171214],
A.[20171215] as [20171215], A.[20171216] as [20171216],
A.[20171217] as [20171217], A.[20171218] as [20171218],
A.[20171219] as [20171219], A.[20171220] as [20171220],
A.[20171221] as [20171221], A.[20171222] as [20171222],
A.[20171223] as [20171223], '
however the result I got is missing the first date , ie 'A.[20171209] as [20171209]'
Here is my code:
SELECT
#col2 = ISNULL(#col2 + 'A.' + QUOTENAME(CDate) + ' as ' + QUOTENAME(CDate) + ', ' , '')
FROM
(SELECT DISTINCT CDate FROM #temp) AS tmp;
Your current approach will not work in some cases, it is an undocumented feature, always use For Xml path to concatenating the rows into csv.
SET #col2 = stuff((SELECT ', A.' + Quotename(CDate) + ' as '
+ Quotename(CDate)
FROM (SELECT DISTINCT CDate
FROM #temp) a
FOR xml path('')),1,1,'')

Generate column name dynamically in sql server

Please look at the below query..
select name as [Employee Name] from table name.
I want to generate [Employee Name] dynamically based on other column value.
Here is the sample table
s_dt dt01 dt02 dt03
2015-10-26
I want dt01 value to display as column name 26 and dt02 column value will be 26+1=27
I'm not sure if I understood you correctly. If I'am going into the wrong direction, please add comments to your question to make it more precise.
If you really want to create columns per sql you could try a variation of this script:
DECLARE #name NVARCHAR(MAX) = 'somename'
DECLARE #sql NVARCHAR(MAX) = 'ALTER TABLE aps.tbl_Fabrikkalender ADD '+#name+' nvarchar(10) NULL'
EXEC sys.sp_executesql #sql;
To retrieve the column name from another query insert the following between the above declares and fill the placeholders as needed:
SELECT #name = <some colum> FROM <some table> WHERE <some condition>
You would need to dynamically build the SQL as a string then execute it. Something like this...
DECLARE #s_dt INT
DECLARE #query NVARCHAR(MAX)
SET #s_dt = (SELECT DATEPART(dd, s_dt) FROM TableName WHERE 1 = 1)
SET #query = 'SELECT s_dt'
+ ', NULL as dt' + RIGHT('0' + CAST(#s_dt as VARCHAR), 2)
+ ', NULL as dt' + RIGHT('0' + CAST((#s_dt + 1) as VARCHAR), 2)
+ ', NULL as dt' + RIGHT('0' + CAST((#s_dt + 2) as VARCHAR), 2)
+ ', NULL as dt' + RIGHT('0' + CAST((#s_dt + 3) as VARCHAR), 2)
+ ' FROM TableName WHERE 1 = 1)
EXECUTE(#query)
You will need to replace WHERE 1 = 1 in two places above to select your data, also change TableName to the name of your table and it currently puts NULL as the dynamic column data, you probably want something else there.
To explain what it is doing:
SET #s_dt is selecting the date value from your table and returning only the day part as an INT.
SET #query is dynamically building your SELECT statement based on the day part (#s_dt).
Each line is taking #s_dt, adding 0, 1, 2, 3 etc, casting as VARCHAR, adding '0' to the left (so that it is at least 2 chars in length) then taking the right two chars (the '0' and RIGHT operation just ensure anything under 10 have a leading '0').
It is possible to do this using dynamic SQL, however I would also consider looking at the pivot operators to see if they can achieve what you are after a lot more efficiently.
https://technet.microsoft.com/en-us/library/ms177410(v=sql.105).aspx

Is CHAR(14) not allowed in SQL Server T-SQL patindex range?

What's the problem with CHAR(13) or perhaps CHAR(14) in TSQL patindex?
As soon as I include CHAR(14) in a pattern, I get no records found.
Searching for an answer, I just found my own question (unanswered) from 2009 (here: http://www.sqlservercentral.com/Forums/Topic795063-338-1.aspx).
Here is another simple test, to show what I mean:
/* PATINDEX TEST */
DECLARE #msg NVARCHAR(255)
SET #msg = 'ABC' + NCHAR(13) + NCHAR(9) + 'DEF'
DECLARE #unwanted NVARCHAR(50)
-- unwanted chars in a "chopped up" string
SET #unwanted = N'%[' + NCHAR(1) + '-' + NCHAR(13) + NCHAR(14) + '-' + NCHAR(31) + ']%'
SELECT patindex(#unwanted, #msg)
-- Result: 4
-- NOW LET THE unwanted string includ the whole range from 1 to 31
SET #unwanted = '%['+NCHAR(1)+'-'+NCHAR(31)+']%' -- -- As soon as Char(14) is included, we get no match with patindex!
SELECT patindex(#unwanted, #msg)
-- Result: 0
It is permitted.
You need to bear in mind that the ranges are based on collation sort order not character codes however so perhaps in your default collation it sorts in a position that you do not expect.
What is your database's default collation?
What does the following return?
;WITH CTE(N) AS
(
SELECT 1 UNION ALL
SELECT 9 UNION ALL
SELECT 13 UNION ALL
SELECT 14 UNION ALL
SELECT 31
)
SELECT N
FROM CTE
ORDER BY NCHAR(N)
For me it returns
N
-----------
1
14
31
9
13
So both characters 9 and 13 are outside the range 1-31. Hence
'ABC' + NCHAR(13) + NCHAR(9) + 'DEF' NOT LIKE N'%['+NCHAR(1)+N'-'+NCHAR(31)+N']%'
Which explains the results in your question. Character 14 doesn't enter into it.
You can use a binary collate clause to get it to sort more as you were expecting. e.g.
SELECT patindex(#unwanted COLLATE Latin1_General_100_BIN, #msg)
Returns 4 in the second query too.

Replace duplicate spaces with a single space in T-SQL

I need to ensure that a given field does not have more than one space (I am not concerned about all white space, just space) between characters.
So
'single spaces only'
needs to be turned into
'single spaces only'
The below will not work
select replace('single spaces only',' ',' ')
as it would result in
'single spaces only'
I would really prefer to stick with native T-SQL rather than a CLR based solution.
Thoughts?
Even tidier:
select string = replace(replace(replace(' select single spaces',' ','<>'),'><',''),'<>',' ')
Output:
select single spaces
This would work:
declare #test varchar(100)
set #test = 'this is a test'
while charindex(' ',#test ) > 0
begin
set #test = replace(#test, ' ', ' ')
end
select #test
If you know there won't be more than a certain number of spaces in a row, you could just nest the replace:
replace(replace(replace(replace(myText,' ',' '),' ',' '),' ',' '),' ',' ')
4 replaces should fix up to 16 consecutive spaces (16, then 8, then 4, then 2, then 1)
If it could be significantly longer, then you'd have to do something like an in-line function:
CREATE FUNCTION strip_spaces(#str varchar(8000))
RETURNS varchar(8000) AS
BEGIN
WHILE CHARINDEX(' ', #str) > 0
SET #str = REPLACE(#str, ' ', ' ')
RETURN #str
END
Then just do
SELECT dbo.strip_spaces(myText) FROM myTable
This is somewhat brute force, but will work
CREATE FUNCTION stripDoubleSpaces(#prmSource varchar(max)) Returns varchar(max)
AS
BEGIN
WHILE (PATINDEX('% %', #prmSource)>0)
BEGIN
SET #prmSource = replace(#prmSource ,' ',' ')
END
RETURN #prmSource
END
GO
-- Unit test --
PRINT dbo.stripDoubleSpaces('single spaces only')
single spaces only
update mytable
set myfield = replace (myfield, ' ', ' ')
where charindex(' ', myfield) > 0
Replace will work on all the double spaces, no need to put in multiple replaces. This is the set-based solution.
It can be done recursively via the function:
CREATE FUNCTION dbo.RemSpaceFromStr(#str VARCHAR(MAX)) RETURNS VARCHAR(MAX) AS
BEGIN
RETURN (CASE WHEN CHARINDEX(' ', #str) > 0 THEN
dbo.RemSpaceFromStr(REPLACE(#str, ' ', ' ')) ELSE #str END);
END
then, for example:
SELECT dbo.RemSpaceFromStr('some string with many spaces') AS NewStr
returns:
NewStr
some string with many spaces
Or the solution based on method described by #agdk26 or #Neil Knight (but safer)
both examples return output above:
SELECT REPLACE(REPLACE(REPLACE('some string with many spaces'
, ' ', ' ' + CHAR(7)), CHAR(7) + ' ', ''), ' ' + CHAR(7), ' ') AS NewStr
--but it remove CHAR(7) (Bell) from string if exists...
or
SELECT REPLACE(REPLACE(REPLACE('some string with many spaces'
, ' ', ' ' + CHAR(7) + CHAR(7)), CHAR(7) + CHAR(7) + ' ', ''), ' ' + CHAR(7) + CHAR(7), ' ') AS NewStr
--but it remove CHAR(7) + CHAR(7) from string
How it works:
Caution:
Char/string used to replace spaces shouldn't exist on begin or end of string and stand alone.
Here is a simple function I created for cleaning any spaces before or after, and multiple spaces within a string. It gracefully handles up to about 108 spaces in a single stretch and as many blocks as there are in the string. You can increase that by factors of 8 by adding additional lines with larger chunks of spaces if you need to. It seems to perform quickly and has not caused any problems in spite of it's generalized use in a large application.
CREATE FUNCTION [dbo].[fnReplaceMultipleSpaces] (#StrVal AS VARCHAR(4000))
RETURNS VARCHAR(4000)
AS
BEGIN
SET #StrVal = Ltrim(#StrVal)
SET #StrVal = Rtrim(#StrVal)
SET #StrVal = REPLACE(#StrVal, ' ', ' ') -- 16 spaces
SET #StrVal = REPLACE(#StrVal, ' ', ' ') -- 8 spaces
SET #StrVal = REPLACE(#StrVal, ' ', ' ') -- 4 spaces
SET #StrVal = REPLACE(#StrVal, ' ', ' ') -- 2 spaces
SET #StrVal = REPLACE(#StrVal, ' ', ' ') -- 2 spaces (for odd leftovers)
RETURN #StrVal
END
Method #1
The first method is to replace extra spaces between words with an uncommon symbol combination as a temporary marker. Then you can replace the temporary marker symbols using the replace function rather than a loop.
Here is a code example that replaces text within a String variable.
DECLARE #testString AS VARCHAR(256) = ' Test text with random* spacing. Please normalize this spacing!';
SELECT REPLACE(REPLACE(REPLACE(#testString, ' ', '*^'), '^*', ''), '*^', ' ');
Execution Time Test #1: In ten runs of this replacement method, the average wait time on server replies was 1.7 milliseconds and total execution time was 4.6 milliseconds.
Execution Time Test #2: The average wait time on server replies was 1.7 milliseconds and total execution time was 3.7 milliseconds.
Method #2
The second method is not quite as elegant as the first, but also gets the job done. This method works by nesting four (or optionally more) replace statements that replace two blank spaces with one blank space.
DECLARE #testString AS VARCHAR(256) = ' Test text with random* spacing. Please normalize this spacing!';
SELECT REPLACE(REPLACE(REPLACE(REPLACE(#testString,' ',' '),' ',' '),' ',' '),' ',' ')
Execution Time Test #1: In ten runs of this replacement method, the average wait time on server replies was 1.9 milliseconds and total execution time was 3.8 milliseconds.
Execution Time Test #2: The average wait time on server replies was 1.8 milliseconds and total execution time was 4.8 milliseconds.
Method #3
The third method of replacing extra spaces between words is to use a simple loop. You can do a check on extra spaces in a while loop and then use the replace function to reduce the extra spaces with each iteration of the loop.
DECLARE #testString AS VARCHAR(256) = ' Test text with random* spacing. Please normalize this spacing!';
WHILE CHARINDEX(' ',#testString) > 0
SET #testString = REPLACE(#testString, ' ', ' ')
SELECT #testString
Execution Time Test #1: In ten runs of this replacement method, the average wait time on server replies was 1.8 milliseconds and total execution time was 3.4 milliseconds.
Execution Time Test #2: The average wait time on server replies was 1.9 milliseconds and total execution time was 2.8 milliseconds.
This is the solution via multiple replace, which works for any strings (does not need special characters, which are not part of the string).
declare #value varchar(max)
declare #result varchar(max)
set #value = 'alpha beta gamma delta xyz'
set #result = replace(replace(replace(replace(replace(replace(replace(
#value,'a','ac'),'x','ab'),' ',' x'),'x ',''),'x',''),'ab','x'),'ac','a')
select #result -- 'alpha beta gamma delta xyz'
You can try this:
select Regexp_Replace('single spaces only','( ){2,}', ' ') from dual;
Just Adding Another Method-
Replacing Multiple Spaces with Single Space WITHOUT Using REPLACE in SQL Server-
DECLARE #TestTable AS TABLE(input VARCHAR(MAX));
INSERT INTO #TestTable VALUES
('HAPPY NEWYEAR 2020'),
('WELCOME ALL !');
SELECT
CAST('<r><![CDATA[' + input + ']]></r>' AS XML).value('(/r/text())[1] cast as xs:token?','VARCHAR(MAX)')
AS Expected_Result
FROM #TestTable;
--OUTPUT
/*
Expected_Result
HAPPY NEWYEAR 2020
WELCOME ALL !
*/
Found this while digging for an answer:
SELECT REPLACE(
REPLACE(
REPLACE(
LTRIM(RTRIM('1 2 3 4 5 6'))
,' ',' '+CHAR(7))
,CHAR(7)+' ','')
,CHAR(7),'') AS CleanString
where charindex(' ', '1 2 3 4 5 6') > 0
The full answer (with explanation) was pulled from: http://techtipsbysatish.blogspot.com/2010/08/sql-server-replace-multiple-spaces-with.html
On second look, seems to be just a slightly different version of the selected answer.
Please Find below code
select trim(string_agg(value,' ')) from STRING_SPLIT(' single spaces only ',' ')
where value<>' '
This worked for me..
Hope this helps...
With the "latest" SQL Server versions (Compatibility level 130) you could also use string_split and string_agg.
string_split can return an ordinal column when provided with a third argument. (https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-ver16#enable_ordinal). So we can preserve the order of the string_split.
Using a common table expression:
with cte(value) as (select value from string_split(' a b c d e ', ' ', 1) where value <> '' order by ordinal offset 0 rows)
select string_agg(value, ' ') from cte
a b c d e results in a b c d e
I use FOR XML PATH solution to replace multiple spaces into single space
The idea is to replace spaces with XML tags
Then split XML string into string fragments without XML tags
Finally concatenating those string values by adding single space characters between two
Here is how final UDF function can be called
select dbo.ReplaceMultipleSpaces(' Sample text with multiple space ')

Resources