Related
I’m using static SQL for 99% of the time, but a recent scenario led me to write a dynamic SQL and I want to make sure I didn’t miss anything before this SQL is released to production.
The tables’ names are a combination of a prefix, a 2 letters variable and a suffix and column name is a prefix + 2 letters variable.
First I’ve checked that #p_param is 2 letters length and is “whitelisted”:
IF (LEN(#p_param) = 2 and (#p_param = ‘aa’ or #p_param = ‘bb’ or #p_param = ‘cc’ or #p_param = ‘dd’ or #p_param = ‘aa’)
BEGIN
set #p_table_name = 'table_' + #p_param + '_suffix';
set #sql = 'update ' + QUOTENAME(#p_table_name) + ' set column_name = 2 where id in (1,2,3,4);';
EXEC sp_executesql #sql;
--Here I’m checking the second parameter that I will create the column name with
IF (LEN(#p_column) = 2 and (#p_column = 'ce' or #p_column = 'pt')
BEGIN
Set #column_name = 'column_name_' + #p_column_param;
set #second_sql = 'update ' + QUOTENAME(#p_table_name) + ' set ' +
QUOTENAME(#column_name) + ' = 2 where id in (#p_some_param);';
EXEC sp_executesql #second_sql, N'#p_some_param NVARCHAR(200)', #p_some_param = #p_some_param;
END
END
Is this use case safe? Are there any pitfalls I should be a ware of?
Seems like you've lost some things in the translation to meaningless names to prepare your query to post here, so it's kinda hard to tell. However, the overall approach seems OK to me.
Using a whitelist with QUOTENAME for the identifiers will protect you from SQL injections using the identifiers parameters, and passing the value parameters as a parameter to sp_executeSql will protect you from SQL injections using the value parameters, so I would say you are doing fine on that front.
There are a couple of things I would change, though.
In addition to testing your tables and columns names against a hard coded white list, I would also test then against information_schema.columns, just to make sure that the procedure will not raise an error in case a table or column is missing.
Also, Your whitelist conditions can be improved - Instead of:
IF (LEN(#p_param) = 2 and (#p_param = ‘aa’ or #p_param = ‘bb’ or #p_param = ‘cc’ or #p_param = ‘dd’ or #p_param = ‘aa’)
You can simply write:
IF #p_param IN('aa', 'bb', 'cc','dd')
This sp_send_dbmail script works in one of our processes. It attaches an Excel file filled with whatever the query is. It knows to do this because of the extension on the file's name (.xls).
However, it changes a varchar(50) field into a number field, and removes the leading zeroes. This is a known annoyance dealt with in a million ways that won't work for my process.
EXEC msdb.dbo.sp_send_dbmail
#profile_name = #profileName
,#recipients = #emailRecipientList
,#subject = #subject
,#importance = #importance
,#body = #emailMsg
,#body_format = 'html'
,#query = #QuerySQL
,#execute_query_database = #QueryDB
,#attach_query_result_as_file = 1
,#query_attachment_filename = #QueryExcelFileName
,#query_result_header = 1
,#query_result_width = #QueryWidth
,#query_result_separator = #QuerySep
,#query_result_no_padding = 1
Examples of problem below: this simple query changes the StringNumber column from varchar to number in Excel, and removes the zeroes.
SELECT [RowID],[Verbage], StringNumber FROM [dbo].[tblTestStringNumber]
In SQL Server (desired format):
After in Excel (leading zeroes missing):
Now, there might be a way. I only say this because in SQL Server 2016 results pane, if you right click in upper left hand corner, it gives the option of "Open in Excel"
And. . . . drum roll . . . the dataset opens in Excel and the leading zeroes are still there!
If you start a number with a single quote (') in Excel, it will interpret it as a string, so a common solution is to change the query to add one in:
SELECT [RowID]
,[Verbage]
, StringNumber = '''' + [StringNumber]
FROM [dbo].[tblTestStringNumber]
And Excel will usually not display the single quote because it knows that it's a way to cast to type string.
#JustJohn I think it will work fine:
SELECT [RowID]
,[Verbage]
, '="' + [StringNumber]+ '"' StringNumber
FROM [dbo].[tblTestStringNumber]
I've been tasked with fixing some SQL code that doesn't work. The query reads from a view against a predicate. The query right now looks like so.
SELECT TOP (100) Beginn
FROM V_LLAMA_Seminare
//Removal of the following line makes the query successful, keeping it breaks it
where Beginn > (select cast (getdate() as smalldatetime))
order by Beginn desc
When I run the above query, I am greeted with the following error.
Msg 295, Level 16, State 3, Line 1
Conversion failed when converting character string to smalldatetime data type.
I decided to remove the WHERE clause, and now it runs returning 100 rows.
At first, I thought that behind the scenes, SQL Server was somehow including my predicate when bringing back the View . But then I investigated how the View was being created, especially the Beginn field, and at no point does it return a String.
Long story short, the column that becomes the Beginn field is a BIGINT timestamp like 201604201369.... The original user transforms this BIGINT to a smalldatetime using the following magic.
....
CASE WHEN ma.datum_dt = 0
THEN null
ELSE CONVERT(smalldatetime, SUBSTRING(CAST(ma.datum_dt AS varchar(max)),0,5) + '-' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),5,2) + '-' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),7,2) + ' ' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),9,2) +':'+
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),11,2) +':' +
RIGHT(CAST(ma.datum_dt AS varchar(max)),2)) END AS Beginn
...
My last attempt at finding the problem was to query the view and run the function ISDATE over the Beginn column and see if it returned a 0 which it never did.
So my question is two fold, "Why does a predicate break something" and two "Where on earth is this string error coming from when the Beginn value is being formed from a BIGINT".
Any help is greatly appreciated.
This problem is culture related...
Try this and then change the first SET LANGUAGE to GERMAN
SET LANGUAGE ENGLISH;
DECLARE #bi BIGINT=20160428001600;
SELECT CASE WHEN #bi = 0
THEN null
ELSE CONVERT(datetime, SUBSTRING(CAST(#bi AS varchar(max)),0,5) + '-' +
SUBSTRING(CAST(#bi AS varchar(max)),5,2) + '-' +
SUBSTRING(CAST(#bi AS varchar(max)),7,2) + ' ' +
SUBSTRING(CAST(#bi AS varchar(max)),9,2) +':'+
SUBSTRING(CAST(#bi AS varchar(max)),11,2) +':' +
RIGHT(CAST(#bi AS varchar(max)),2)) END AS Beginn
It is a very bad habit to think, that date values look the same everywhere (Oh no, my small application will never go international ...)
Try to stick to culture independent formats like ODBC or ISO
EDIT
A very easy solution for you actually was to replace the blank with a "T"
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),7,2) + 'T' +
Then it's ISO 8601 and will convert...
The solution was found after looking through #Shnugo's comment. When I took my query which contained the Bigint->Datetime conversion logic, and put it into a CTE with "TOP 100000000" to avoid any implicit conversion actions, my query worked. Here is what my view looks like now with some unimportant parts omitted.
---Important part---
CREATE VIEW [dbo].[V_SomeView] AS
WITH CTE AS (
SELECT TOP 1000000000 ma.id AS MA_ID,
---Important part---
vko.extkey AS ID_VKO,
vko.text AS Verkaufsorganisation,
fi.f7000 AS MDM_Nr,
vf.f7105 AS SAPKdnr,
CASE WHEN ma.datum_dt = 0 --Conversion logic
CASE WHEN ma.endedatum_dt = 0 --Conversion logic
CONVERT(NVARCHAR(MAX),art.text) AS Art,
.....
FROM [ucrm].[dbo].[CRM_MA] ma,
[ucrm].[dbo].[CRM_fi] fi,
[ucrm].[dbo].[CRM_vf] vf,
[ucrm].[dbo].[CRM_ka] vko,
[ucrm].[dbo].[CRM_ka] art,
[ucrm].[dbo].[CRM_ka] kat
where ma.loskz = 0
and fi.loskz = 0
and vf.loskz = 0
and fi.F7029 = 0
and vf.F7023 = 0
...
GROUP BY ma.id,
vko.extkey,
vko.text,
fi.f7000 ,
vf.f7105,
ma.datum_dt,
ma.endedatum_dt,
....
)
select * FROM CTE;
I am not a SQL expert so please forgive me if this is SQL 101 :).
In a select statement there are 2 replace functions. They look for a Servername and it's admin share d$ by it's UNC path. Example '\SERVERNAME\d$'
It then replaces '\SERVERNAME\d$' with 'D:'.
Here is the query currently:
select Replace(p.Path,'\\SERVERNAME\d$','D:') as searchpath
,p.path as fullpath
,s.ShareName
,s.SharePath
,p.Member
,p.Access
From Paths As p
Left Outer Join Shares as s on
Replace(p.Path,'\\SERVERNAME\d$','D:') Like s.SharePath + '\%'
Up until now it has always been d$.
Today my needs have changed and I need the query to find ANY servername UNC path admin share regardless of share letter (c$, d$, e$, f$...etc) and replace it with it's respective drive letter (D:, E:, F:... etc).
My thought is replace function could find the $ and look one character to the left of it to get the proper share letter, then use that for the replace. The issue I have, not being a SQL professional, is that I know SQL can likley do what I need it to do...I just don't know how to get there. I've googled and found some examples, but haven't had any luck in getting them to work.
Any help would be greatly appreciated.
You can use a combination of STUFF, PATINDEX, LEN to get what you want.
Sample Query
DECLARE #ReplaceChar VARCHAR(100) = '[prefixcharacters]\\SERVERNAME\d$[postcharacter]'
DECLARE #SearchString VARCHAR(100) = '\\SERVERNAME\_$'
SELECT
STUFF(#ReplaceChar,PATINDEX('%' + #SearchString + '%',#ReplaceChar),LEN(#SearchString),
UPPER(SUBSTRING(#ReplaceChar,PATINDEX('%' + #SearchString + '%',#ReplaceChar) + LEN(#SearchString) - 2,1)) + ':') as searchpath
WHERE PATINDEX('%' + #SearchString + '%',#ReplaceChar) > 0
Output
[prefixcharacters]D:[postcharacter]
Alternate Query
You can shorten the query if you want to get the previous character before $ as per your title. Something like this
DECLARE #ReplaceChar VARCHAR(100) = '[prefixcharacters]\\SERVERNAME\d$[postcharacter]'
DECLARE #SearchString VARCHAR(100) = '\\SERVERNAME\_$'
SELECT
STUFF(#ReplaceChar,
PATINDEX('%'+#SearchString+'%',#ReplaceChar),
LEN(#SearchString),
UPPER(SUBSTRING(#ReplaceChar,CHARINDEX('$',#ReplaceChar) -1,1)) + ':')
WHERE PATINDEX('%'+#SearchString+'%',#ReplaceChar) > 0
In this query
STUFF replaces your pattern with with the character before $ + ':'
Start of pattern is identified by PATINDEX('%'+#SearchString+'%',#ReplaceChar)
D is identified by getting the charindex of '$' and then getting the previous character using SUBSTRING
What about ¸
select Replace(SUBSTRING(p.path, 14, Len(#spath)-14),'$',':') as searchpath
,p.path as fullpath
,s.ShareName
,s.SharePath
,p.Member
,p.Access
From Paths As p
Left Outer Join Shares as s on
Replace(SUBSTRING(p.path, 14, Len(#spath)-14),'$',':') Like s.SharePath + '\%
select as searchpath
DECLARE #str nvarchar (100)
SET #str = '\\SERVERNAME\d$'
IF #str LIKE '\\SERVERNAME\_$'
SET #str = UPPER(SUBSTRING(#str, 14, 1)) + ':'
SELECT #str
Starting from previous, something like
select UPPER(SUBSTRING(p.path, 14, 1)) + ':' as searchpath
,p.path as fullpath
,s.ShareName
,s.SharePath
,p.Member
,p.Access
From Paths As p
Left Outer Join Shares as s on
SUBSTRING(p.path, 14, 1) + ':' Like s.SharePath + '\%'
I am no mysql expert either :)
Based on the logic you mentioned in the last part of the question, I have used concat and substring to get to the drive letter in the column.
Hope this helps
select replace(path, concat(substring(path, 1, locate('$', path) - 2), substring(path, locate('$', path) - 1, 1) , '$'), concat(substring(path, locate('$', path) - 1, 1) , ':')) as searchpath ...
The remaining part of the query would be the same.
We have a web application that uses SQL Server 2008 as the database. Our users are able to do full-text searches on particular columns in the database. SQL Server's full-text functionality does not seem to provide support for hit highlighting. Do we need to build this ourselves or is there perhaps some library or knowledge around on how to do this?
BTW the application is written in C# so a .Net solution would be ideal but not necessary as we could translate.
Expanding on Ishmael's idea, it's not the final solution, but I think it's a good way to start.
Firstly we need to get the list of words that have been retrieved with the full-text engine:
declare #SearchPattern nvarchar(1000) = 'FORMSOF (INFLECTIONAL, " ' + #SearchString + ' ")'
declare #SearchWords table (Word varchar(100), Expansion_type int)
insert into #SearchWords
select distinct display_term, expansion_type
from sys.dm_fts_parser(#SearchPattern, 1033, 0, 0)
where special_term = 'Exact Match'
There is already quite a lot one can expand on, for example the search pattern is quite basic; also there are probably better ways to filter out the words you don't need, but it least it gives you a list of stem words etc. that would be matched by full-text search.
After you get the results you need, you can use RegEx to parse through the result set (or preferably only a subset to speed it up, although I haven't yet figured out a good way to do so). For this I simply use two while loops and a bunch of temporary table and variables:
declare #FinalResults table
while (select COUNT(*) from #PrelimResults) > 0
begin
select top 1 #CurrID = [UID], #Text = Text from #PrelimResults
declare #TextLength int = LEN(#Text )
declare #IndexOfDot int = CHARINDEX('.', REVERSE(#Text ), #TextLength - dbo.RegExIndexOf(#Text, '\b' + #FirstSearchWord + '\b') + 1)
set #Text = SUBSTRING(#Text, case #IndexOfDot when 0 then 0 else #TextLength - #IndexOfDot + 3 end, 300)
while (select COUNT(*) from #TempSearchWords) > 0
begin
select top 1 #CurrWord = Word from #TempSearchWords
set #Text = dbo.RegExReplace(#Text, '\b' + #CurrWord + '\b', '<b>' + SUBSTRING(#Text, dbo.RegExIndexOf(#Text, '\b' + #CurrWord + '\b'), LEN(#CurrWord) + 1) + '</b>')
delete from #TempSearchWords where Word = #CurrWord
end
insert into #FinalResults
select * from #PrelimResults where [UID] = #CurrID
delete from #PrelimResults where [UID] = #CurrID
end
Several notes:
1. Nested while loops probably aren't the most efficient way of doing it, however nothing else comes to mind. If I were to use cursors, it would essentially be the same thing?
2. #FirstSearchWord here to refers to the first instance in the text of one of the original search words, so essentially the text you are replacing is only going to be in the summary. Again, it's quite a basic method, some sort of text cluster finding algorithm would probably be handy.
3. To get RegEx in the first place, you need CLR user-defined functions.
It looks like you could parse the output of the new SQL Server 2008 stored procedure sys.dm_fts_parser and use regex, but I haven't looked at it too closely.
You might be missing the point of the database in this instance. Its job is to return the data to you that satisfies the conditions you gave it. I think you will want to implement the highlighting probably using regex in your web control.
Here is something a quick search would reveal.
http://www.dotnetjunkies.com/PrintContent.aspx?type=article&id=195E323C-78F3-4884-A5AA-3A1081AC3B35
Some details:
search_kiemeles=replace(lcase(search),"""","")
do while not rs.eof 'The search result loop
hirdetes=rs("hirdetes")
data=RegExpValueA("([A-Za-zöüóőúéáűíÖÜÓŐÚÉÁŰÍ0-9]+)",search_kiemeles) 'Give back all the search words in an array, I need non-english characters also
For i=0 to Ubound(data,1)
hirdetes = RegExpReplace(hirdetes,"("&NoAccentRE(data(i))&")","<em>$1</em>")
Next
response.write hirdetes
rs.movenext
Loop
...
Functions
'All Match to Array
Function RegExpValueA(patrn, strng)
Dim regEx
Set regEx = New RegExp ' Create a regular expression.
regEx.IgnoreCase = True ' Set case insensitivity.
regEx.Global = True
Dim Match, Matches, RetStr
Dim data()
Dim count
count = 0
Redim data(-1) 'VBSCript Ubound array bug workaround
if isnull(strng) or strng="" then
RegExpValueA = data
exit function
end if
regEx.Pattern = patrn ' Set pattern.
Set Matches = regEx.Execute(strng) ' Execute search.
For Each Match in Matches ' Iterate Matches collection.
count = count + 1
Redim Preserve data(count-1)
data(count-1) = Match.Value
Next
set regEx = nothing
RegExpValueA = data
End Function
'Replace non-english chars
Function NoAccentRE(accent_string)
NoAccentRE=accent_string
NoAccentRE=Replace(NoAccentRE,"a","§")
NoAccentRE=Replace(NoAccentRE,"á","§")
NoAccentRE=Replace(NoAccentRE,"§","[aá]")
NoAccentRE=Replace(NoAccentRE,"e","§")
NoAccentRE=Replace(NoAccentRE,"é","§")
NoAccentRE=Replace(NoAccentRE,"§","[eé]")
NoAccentRE=Replace(NoAccentRE,"i","§")
NoAccentRE=Replace(NoAccentRE,"í","§")
NoAccentRE=Replace(NoAccentRE,"§","[ií]")
NoAccentRE=Replace(NoAccentRE,"o","§")
NoAccentRE=Replace(NoAccentRE,"ó","§")
NoAccentRE=Replace(NoAccentRE,"ö","§")
NoAccentRE=Replace(NoAccentRE,"ő","§")
NoAccentRE=Replace(NoAccentRE,"§","[oóöő]")
NoAccentRE=Replace(NoAccentRE,"u","§")
NoAccentRE=Replace(NoAccentRE,"ú","§")
NoAccentRE=Replace(NoAccentRE,"ü","§")
NoAccentRE=Replace(NoAccentRE,"ű","§")
NoAccentRE=Replace(NoAccentRE,"§","[uúüű]")
end function