I have been asked to do a bit of work on a SQL Server 2005 query that has 26 (!) nested Replace function calls.
There are two issues with this query.
The first is that the 26 replace functions are not enough, a few more are needed, but the query is hitting some size limitations (it's a stored procedure and its making using of sp_MSforEachDB).
The second problem is that the query is just not legible at this stage.
I would like to use table variable with a lookup, but the issue is that the replace query is being used to replace values in one column with values from another column. It does however all work, apparent from not replacing enough strings.
Here is a simplified example to hopefully better explain what I am trying to do.
Table to be converted
ID MainText Ptext1 Ptext2 Ptext3
1 Have a %Ptxt1 %Ptxt2 Apple Tonight
2 %Ptxt1 likes %Ptxt2 at %Ptxt3 Sally Cake Midnight
The desired result for 1 is "Have a Apple Tonight", the desired result for 2 is "Sally Likes Cake at Midnight".
The current SQL Looks a bit like
EXEC sp_MSForEachDB 'IF ''?'' LIKE ''%Database%''
Select Replace(Replace(Replace([Maintext], '%Ptxt1' , [Ptext1]),'%Ptxt2',[Ptext2]),'Ptxt3', [Ptext3]) from dbo.mytable
Please forgive any mistakes in the example.
I have seen nice examples of people using Table variables to store parameters for the replace function - but I haven't seen any where a column reference is used instead of a static string, is this possible ?
Call your base table: MyData.
Create a new table named MyValues (id, Token, data_value). the ID is FK to you base table. for each record in MyData, insert the values of field names into MyValues.
So, MyValues looks like
Id Token data_value
==== ====== ==========
1 Ptxt1 Apple
1 Ptxt2 Tonight
2 Ptxt1 Sally
2 Ptxt2 Cake
2 Ptxt3 Midnight
and so on...
Then you can join MyTable with MyValues.
Now, one way of solving the problem, is looping on MyData and inner loop on MyValues.
declare #i int
for each value in MyData
begin
#i++
read MyDaya.MainText into #mainText
for each value in MyValues where ID = Id of the MainData
begin
read MyValues.data_value into #actualValue
read MyValues.Token into #token
#mainText = Replace(#mainText, #toekn, #actualValue)
Update MyData with mainText or do what ever appropriate
end
end
This is pseudo-code, I hope you can implement that in T- SQL
Related
I want to create a report in MS SQL Server BIDS (SSMS and Visual Studio). The user would enter a list of email addresses as a parameter. So #pEmails would be 'foo#bluh.com', 'bar#meh.org', etc. These email addresses may or may not be in a table.
I can simply do:
and Table.Email in (#pEmails)
and that works, except I need to return the email address if it's NOT found as well. So the results would be something like:
|email |found in table|
|------------|--------------|
|foo#bluh.com| Y |
|bar#meh.org | N |
I was thinking I could take the list of values entered as the #pEmails parameter and create a temp table with them, which I could then left join with, but my attempts to do so have not worked out.
declare #pEmails table (EmailAddress varchar(255));
insert into #pEmails values (#ReportParameter1);
select
*
from
#pEmails
The above works if only a single value is put into #ReportParameter1, but not if multiples are in it.
I am using SQL Server 2008. Any suggestions on how best to proceed?
As has been stated, you need some kind of split function, for analysis on the performance of various methods Split strings the right way – or the next best way is an excellent read. Once you have your function, you then need to define your query parameter as a string, rather than a table:
So your query would actually become:
DECLARE #pEmails TABLE (EmailAddress varchar(255));
INSERT #pEmails (EmailAddress)
SELECT Value
FROM dbo.Split(#pEmallString);
Then go to your dataset properties, and instead of passing the multivalue parameter #pEmails to the dataset, instead create a new one #pEmailString, and set the value as an expression, which should be:
=Join(Parameters!pEmails.Value, ",")
This turns your multivalue parameter into a single comma delimited string. It seems pretty backwards that you need to convert it to a delimited string, only to then split it in SQL, unfortunately I don't know of a better way.
Here are some learnings on this topic (standing on the shoulders of the information elsewhere in this thread).
Set a parameter (select 'multiple values' checkbox):
InputList
Establish dataset query:
SELECT *
INTO #InputTemp
FROM STRING_SPLIT(#InputListJoin, ',')
SELECT value as ValueName
FROM #InputTemp T2
WHERE NOT EXISTS (
SELECT MyValue
FROM MyTable T1
WHERE T1.MyValue = T2.value
)
Establish dataset parameters:
Name: #InputList | Value: [#InputList]
Name: #InputListJoin | Value(expression): =Join(Parameters!InputList.Value,",")
The element names can be changed as needed.
Somewhat on topic, other details that might be helpful:
[#InputList.IsMultiValue] --> true/false whether your parameter is multi-value (not whether there are multiple values)
[#InputList.Count] --> count of items in input list (excludes blank lines)
=Parameters!InputList.Value(2) --> return third value from list (counting from zero)
I've read countless posts on this topic but I can't seem to get any of the recommendations to apply to my particular situation (which isn't different than others...)
I have an SSRS report. Dataset 1 is using a stored procedure and in the where clause I have
and (#param is null or alias.column in
(select Item from dbo.ufnSplit(#param,',')))
I borrowed the dbo.ufnSplit function from this post here: https://stackoverflow.com/a/512300/22194
FUNCTION [dbo].[ufnSplit]
(#RepParam nvarchar(max), #Delim char(1)= ',')
RETURNS #Values TABLE (Item nvarchar(max))AS
--based on John Sansoms StackOverflow answer:
--https://stackoverflow.com/a/512300/22194
BEGIN
DECLARE #chrind INT
DECLARE #Piece nvarchar(100)
SELECT #chrind = 1
WHILE #chrind > 0
BEGIN
SELECT #chrind = CHARINDEX(#Delim,#RepParam)
IF #chrind > 0
SELECT #Piece = LEFT(#RepParam,#chrind - 1)
ELSE
SELECT #Piece = #RepParam
INSERT #Values(Item) VALUES(#Piece)
SELECT #RepParam = RIGHT(#RepParam,LEN(#RepParam) - #chrind)
IF LEN(#RepParam) = 0 BREAK
END
RETURN
END
In dataset 2 I am getting the values that I want to pass to dataset 1
select distinct list from table
My parameter for #param is configured to look at dataset 2 for available values
My issue is that if I select a single value from my parameter dropdown for #param, the report works. If I select multiple values from the dropdown, I only return data for the first value selected.
My values in dataset 2 do not contain any ,'s
Did I miss anything for fail to provide enough information? I'm open to criticism, feedback, do's and don'ts for this, I've struggled with this issue for some time, and by no means a SQL expert :)
Cheers,
MD
Update So SQL Profiler is showing me this:
exec sp... #param=N'value1,value2 ,value3 '
Questions are:
1. Shouldn't every value be wrapped in single quotes?
2. What's with the N before the list?
3. Guessing the trailing spaces need to be trimmed out
When you select multiple values from a parameter dropdown list they are stored in an array. In order to convert that to a string that you can pass to SQL you can use the Join function. Go to your dataset properties and then to the Parameters tab. Replace the Parameter Value with this expression:
=Join(Parameters!param.Value, ",")
It should look like this:
Now your split function will get one comma separated string like it's supposed to. I would also suggest having the split function trim off spaces from the values after it has separated them.
So I figured it out and wanted to post my results here in hopes it helps someone else.
Bad data. One trailing space was blowing up my entire result set, and I didn't notice it until I ran through several scenarios (choosing many combinations of parameters)
My result set had trailing spaces - once I did an rtrim on it I didn't have to do any fancy join/split's in SSRS.
I am trying to create function which can replace certain words with hyperlink in sql. When I call the function as query in sql, its takes a really long time to execute the query, more than 2-3min. I assumed this is because, the tag_libary table has around 600,000 records and iterating through large number, would consume a lot of processing time.
CREATE FUNCTION dbo.ReplaceTags(#body VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
BEGIN
SELECT #body = REPLACE(#body,name,''+name+'')
FROM Tag_Library
RETURN #body
END
article table (id, title, body)
1, Story1, At the same time there is a list consisting of: DUCHS, EUROC, GLSPE and WODST. Only two of the tags have covered with the prices in the last three months - GROSV at 99.11 on 8 October and JUBIL at 0s on 11 September.
tag_library table (id, name)
1,DRYDN33
2,DUCHS
3,DRYDN33
4,DRYDN15
5,EUROC
6,DRYDN15
7,GROSV
Hence, I am writing to seek some advice, if there is a way to make this sql function optimal or would it be better to change this function, into a insert trigger?
Please advice, if possible.
Yust a thought, I did not test it:
Change your query to this one:
SELECT
#body = REPLACE(#body,name,''+name+'')
FROM
Tag_Library
WHERE
#body LIKE '%' + name + '%'
This should filter the Tag_Library table to those records which are present in the input string and the SQL Server do not have to process lots of unnecessary records (replaces). BUT It will not prevent to do a full table / index scan to check the table!
You can improve this solution by storing the required tags in a table per articles (and update that table via triggers when the source records/tables are changed). In this case you can use joins to filter the Tag_Library table (instead of the LIKE operator), but it reqires extra codes to maintain the dictionary.
You're focusing on the wrong thing. The problem is that this is a scalar function, and they perform miserably. You should change it to a table-valued function that returns a single row and use APPLY.
See, for example:
http://dataeducation.com/scalar-functions-inlining-and-performance-an-entertaining-title-for-a-boring-post/
I want to use a clause along the lines of "CASE WHEN ... THEN 1 ELSE 0 END" in a select statement. The tricky part is that I need it to work with "value IN #List".
If I hard code the list it works fine - and it performs well:
SELECT
CASE WHEN t.column_a IN ( 'value a', 'value b' ) THEN 1 ELSE 0 END AS priority
, t.column_b
, t.column_c
FROM
table AS t
ORDER BY
priority DESC
What I would like to do is:
-- #AvailableValues would be a list (array) of strings.
DECLARE
#AvailableValues ???
SELECT
#AvailableValues = ???
FROM
lookup_table
SELECT
CASE WHEN t.column_a IN #AvailableValues THEN 1 ELSE 0 END AS priority
, t.column_b
, t.column_c
FROM
table AS t
ORDER BY
priority DESC
Unfortunately, it seems that SQL Server doesn't do this - you can't use a variable with an IN clause. So this leaves me with some other options:
Make '#AvailableValues' a comma-delimited string and use a LIKE statement. This does not perform well.
Use an inline SELECT statement against 'lookup_table' in place of the variable. Again, doesn't perform well (I think) because it has to lookup the table on each row.
Write a function wrapping around the SELECT statement in place of the variable. I haven't tried this yet (will try it now) but it seems that it will have the same problem as a direct SELECT statement.
???
Are there any other options? Performance is very important for the query - it has to be really fast as it feeds a real-time search result page (i.e. no caching) for a web site.
Are there any other options here? Is there a way to improve the performance of one of the above options to get good performance?
Thanks in advance for any help given!
UPDATE: I should have mentioned that the 'lookup_table' in the example above is already a table variable. I've also updated the sample queries to better demonstrate how I'm using the clause.
UPDATE II: It occurred to me that the IN clause is operating off an NVARCHAR/NCHAR field (due to historical table design reasons). If I was to make changes that dealt with integer fields (i.e through PK/FK relationship constraints) could this have much impact on performance?
You can use a variable in an IN clause, but not in the way you're trying to do. For instance, you could do this:
declare #i int
declare #j int
select #i = 10, #j = 20
select * from YourTable where SomeColumn IN (#i, #j)
The key is that the variables cannot represent more than one value.
To answer your question, use the inline select. As long as you don't reference an outer value in the query (which could change the results on a per-row basis), the engine will not repeatedly select the same data from the table.
Based on your update and assuming the lookup table is small, I suggest trying something like the following:
DECLARE #MyLookup table
(SomeValue nvarchar(100) not null)
SELECT
case when ml.SomeValue is not null then 1 else 0 end AS Priority
,t.column_b
,t.column_c
from MyTable t
left outer join #MyLookup ml
on ml.SomeValue = t.column_a
order by case when ml.SomeValue is not null then 1 else 0 end desc
(You can't reference the column alias "Priority" in the ORDER BY clause. Alternatively, you could use the ordinal position like so:
order by 1 desc
but that's generally not recommended.)
As long as the lookup table is small , this really should run fairly quickly -- but your comment implies that it's a pretty big table, and that could slow down performance.
As for n[Var]char vs. int, yes, integers would be faster, if only because the CPU has fewer bytes to juggle around... which shoud only be a problem when processing a lot of rows, so it might be worth trying.
I solved this problem by using a CHARINDEX function. I wanted to pass the string in as a single parameter. I created a string with leading and trailing commas for each value I wanted to test for. Then I concatenated a leading and trailing commas to the string I wanted to see if was "in" the parameter. At the end I checked for CHARINDEX > 0
DECLARE #CTSPST_Profit_Centers VARCHAR (256)
SELECT #CTSPST_Profit_Centers = ',CS5000U37Y,CS5000U48B,CS5000V68A,CS5000V69A,CS500IV69A,CS5000V70S,CS5000V79B,CS500IV79B,'
SELECT
CASE
WHEN CHARINDEX(','+ISMAT.PROFIT_CENTER+',' ,#CTSPST_Profit_Centers) > 0 THEN 'CTSPST'
ELSE ISMAT.DESIGN_ID + ' 1 CPG'
END AS DESIGN_ID
You can also do it in the where clause
WHERE CHARINDEX(','+ISMAT.PROFIT_CENTER+',',#CTSPST_Profit_Centers) > 0
If you were trying to compare numbers you'd need to convert the number to a text string for the CHARINDEX function to work.
This might be along the lines of what you need.
Note that this assumes that you have permissions and the input data has been sanitized.
From Running Dynamic Stored Procedures
CREATE PROCEDURE MyProc (#WHEREClause varchar(255))
AS
-- Create a variable #SQLStatement
DECLARE #SQLStatement varchar(255)
-- Enter the dynamic SQL statement into the
-- variable #SQLStatement
SELECT #SQLStatement = "SELECT * FROM TableName WHERE " + #WHEREClause
-- Execute the SQL statement
EXEC(#SQLStatement)
I have an odd case where when I look at the data through my SQL scripts I have one value, but if I look at the data directly SELECT * FROM table I get another value.
My first thought was parameter sniffing, but that didn't fix the issue. I'm not doing anything with the value at hand, except getting it with a stored procedure.
Example of the stored procedure.
CREATE PROCEDURE example
(
#iRefProjectID int
)
AS
-- Prevent Parameter sniffing
DECLARE #projectID int
SET #projectID = #iRefProjectID
SELECT iEntryType
FROM table
WHERE iEntryType IN (1,5,6)
AND iProjectID = #projectID
RETURN
GO
Now one of the rows so extracted contains a '2', which when I look at it through the SP it is a '1'. It shouldn't have been picked at all being 2 != 1 || 5 || 6. Suddenly the 2 becomes a 1 and then "1" == 1.
Where should I look to kill this bug.
The rows in question
SELECT * FROM table
3264427 2003-11-25 00:00:00.000 **2** Udligning til afregning F83907 100625.00
Exec SP
3264427 2003-11-25 00:00:00.000 -100625.00 Udligning til afregning F83907 **1**
Ahh .. found something. This rather looks like a Join bug.
There is no such bug.
You probably have 2 tables with same name in different schemas. Example: dbo.table and [DOMAIN\User].table
Best practice is always qualify objects to avoid incorrect schema resolution.
There are other options such as:
different databases
different servers
table is actually an unrefreshed view
dirty read (as per Yves M. comment) because you have changed isolation level