We are deploying EF Core migrations using SQL scripts to our staging environment.
When running the SQL scripts having Stored procedures, Views and functions we ran into problems of unrecognized characters.
The solution was to parse the SQL Scripts to use Dynamic script by surrounding the CREATE or ALTER statements with EXEC('...').
Using powershell [regex]::replace we did this format but we still have an issue with single quotes inside these statements.
Below are the powershell scripts:
## Replace CREATE statements ( SP, View, Func )
$sql = [regex]::replace($sql, "BEGIN\s+(CREATE (?:PROCEDURE|VIEW|FUNCTION).+?)END;", "BEGIN`nEXEC('`$1');`nEND;", "ignorecase,singleline")
## Replace ALTER statements ( SP, View, Func )
$sql = [regex]::replace($sql, "BEGIN\s+(ALTER (?:PROCEDURE|VIEW|FUNCTION).+?)END;", "BEGIN`nEXEC('`$1');`nEND;", "ignorecase,singleline")
How can we extend these to escape every single quote with another single quote only for these statements?
I was looking to add another .replace() function to the $1 param but no luck.
Example SQL Script
IF NOT EXISTS(SELECT * FROM [TestHelper].[__EFMigrationsHistory] WHERE [MigrationId] = N'20191004135334_db_sp-CsvImport')
BEGIN
CREATE PROCEDURE [TestHelper].[CsvImportService]
#DvseImportId INT,
#Path NVARCHAR(255)
AS
BEGIN
-- DROP TEMP TABLE IF EXITSTS
DROP TABLE IF EXISTS [TestHelper].[_tmpDvseImportData]
-- CREATE TEMP TABLE
CREATE TABLE [TestHelper].[_tmpDvseImportData]
(
DataSupplierID INT
, DataSupplier NVARCHAR(255)
, ArticleNumber NVARCHAR(255)
, ArticleNumberNorm NVARCHAR(255)
, GenNo INT
, GenDescription NVARCHAR(255)
, State BIT
)
-- BULK INSERT DATA FROM FILESHARE
EXEC('BULK INSERT [TestHelper].[_tmpDvseImportData] FROM ''' + #Path + ''' WITH ( FIELDTERMINATOR = '';'',ROWTERMINATOR = ''\n'')')
-- STEP 1: FORMAT DATA
-- REMOVE DOUBLE QUOTES = CHAR(34)
UPDATE [TestHelper].[_tmpDvseImportData]
SET DataSupplier = REPLACE(DataSupplier, CHAR(34), '')
, ArticleNumber = REPLACE(ArticleNumber, CHAR(34), '')
, ArticleNumberNorm = REPLACE(ArticleNumberNorm, CHAR(34), '')
, GenDescription = REPLACE(GenDescription, CHAR(34), '')
-- STEP 2: REMOVE DUPLICATES
...
-- Drop table
DROP TABLE [TestHelper].[_tmpDvseImportData]
END
END;
GO
IF NOT EXISTS(SELECT * FROM [TestHelper].[__EFMigrationsHistory] WHERE [MigrationId] = N'20191008093824_db_view-DvseSuppliersWithMetaData')
BEGIN
CREATE VIEW [TestHelper].[DvseSuppliersWithMetaData] AS
SELECT supplier.Id
, supplier.[Key]
, supplier.Code
, supplier.[Name]
, (SELECT COUNT(*) FROM [TestHelper].[DvseArticle] WHERE SupplierId = supplier.Id) as TotalArticles
, (SELECT COUNT(*) FROM [TestHelper].[DvseArticle] WHERE SupplierId = supplier.Id AND IsActive = 1) as TotalActive
FROM [TestHelper].[DvseSupplier] AS supplier
GROUP BY supplier.Id
, supplier.[Key]
, supplier.Code
, supplier.[Name]
END;
GO
In Windows PowerShell within the .Replace method, you can use a script block. The script block can then manipulate the matched results. It isn't as clean as script-block substitution offered by PowerShell Core.
$sbCreate = {param ($m) "BEGIN`nEXEC('$($m.Groups[1].Value -replace "'","''")');`nEND;"}
$sbAlter = {param ($m) "BEGIN`nEXEC('$($m.Groups[1].Value -replace "'","''")');`nEND;" }
$sql = [regex]::replace($sql, "BEGIN\s+(CREATE (?:PROCEDURE|VIEW|FUNCTION).+?)END;", $sbCreate, "ignorecase,singleline")
$sql = [regex]::replace($sql, "BEGIN\s+(ALTER (?:PROCEDURE|VIEW|FUNCTION).+?)END;", $sbAlter, "ignorecase,singleline")
Explanation:
In each script block, parameter $m is the matched object. Since you want to access the first unnamed capture group (1), you can retrieve that value using $m.Groups[1].Value. The sub-expression operator $() is used to that we can access the properties of $m and use the -replace operation within the replacement string.
If you are you using PS version 6 or above, you should be able to use a scriptblock substitution to achieve what you want.
For example, given that you have saved your example string as a variable called sqlString, you could do the following:
$regex = [regex]::new("BEGIN\s+(CREATE (?:PROCEDURE|VIEW|FUNCTION).+?)END;", 'SingleLine')
$sqlString -replace $regex, {"BEGIN`nEXEC('$($_.Value.Replace("'", "''"))');`nEND;"}
Related
I'm creating a stored procedure in Snowflake that will eventually be called by a task.
However I'm getting the following error:
Multiple SQL statements in a single API call are not supported; use one API call per statement instead
And not sure how approach the advised solution within my Javascript implementation.
Here's what I have
CREATE OR REPLACE PROCEDURE myStoreProcName()
RETURNS VARCHAR
LANGUAGE javascript
AS
$$
var rs = snowflake.execute( { sqlText:
`set curr_date = '2015-01-01';
CREATE OR REPLACE TABLE myTableName AS
with cte1 as (
SELECT
*
FROM Table1
where date = $curr_date
)
,cte2 as (
SELECT
*
FROM Table2
where date = $curr_date
)
select * from
cte1 as 1
inner join cte2 as 2
on(1.key = 2.key)
`
} );
return 'Done.';
$$;
You could write your own helper function(idea of user: waldente):
this.executeMany=(s) => s.split(';').map(sqlText => snowflake.createStatement({sqlText}).execute());
executeMany('set curr_date = '2015-01-01';
CREATE OR REPLACE TABLE ...');
The last statement should not contain ; it also may fail if there is ; in one of DDL which was not intended as separator.
You can't have:
var rs = snowflake.execute( { sqlText:
`set curr_date = '2015-01-01';
CREATE OR REPLACE TABLE myTableName AS
...
`
Instead you need to call execute twice (or more). Each for a different query ending in ;.
I am working on a query that I need to modify so that a string is passed to in(). The view table is being used by some other view table and ultimately by a stored procedure. The string values must be in ' '.
select region, county, name
from vw_main
where state - 'MD'
and building_id in ('101', '102') -- pass the string into in()
The values for the building_id will be entered at the stored procedure level upon its execution.
Please check below scripts which will give you answer.
Way 1: Split CSV value using XML and directly use select query in where condition
DECLARE #StrBuildingIDs VARCHAR(1000)
SET #StrBuildingIDs = '101,102'
SELECT
vm.region,
vm.county,
vm.name
FROM vw_main vm
WHERE vm.state = 'MD'
AND vm.building_id IN
(
SELECT
l.value('.','VARCHAR(20)') AS Building_Id
FROM
(
SELECT CAST('<a>' + REPLACE(#StrBuildingIDs,',','</a><a>') + '</a>') AS BuildIDXML
) x
CROSS APPLY x.BuildIDXML.nodes('a') Split(l)
)
Way 2: Split CSV value using XML, Create Variable Table and use that in where condition
DECLARE #StrBuildingIDs VARCHAR(1000)
SET #StrBuildingIDs = '101,102'
DECLARE #TblBuildingID TABLE(BuildingId INT)
INSERT INTO #TblBuildingID(BuildingId)
SELECT
l.value('.','VARCHAR(20)') AS Building_Id
FROM
(
SELECT CAST('<a>' + REPLACE(#StrBuildingIDs,',','</a><a>') + '</a>') AS BuildIDXML
) x
CROSS APPLY x.BuildIDXML.nodes('a') Split(l)
SELECT
vm.region,
vm.county,
vm.name
FROM vw_main AS vm
WHERE vm.state = 'MD'
AND vm.building_id IN
(
SELECT
BuildingId
FROM #TblBuildingID
)
Way 3: Split CSV value using XML, Create Variable Table and use that in INNER JOIN
Assuming the input string is not end-user input, you can do this. That is, derived or pulled from another table or other controlled source.
DECLARE #in nvarchar(some length) = N'''a'',''b'',''c'''
declare #stmt nvarchar(4000) = N'
select region, county, name
from vw_main
where state = ''MD''
and building_id in ({instr})'
set #stmt = replace(#stmt, N'{instr}', #instr)
exec sp_executesql #stmt=#stmt;
If the input is from an end-user, this is safer:
declare # table (a int, b char)
insert into #(a, b) values (1,'A'), (2, 'B')
declare #str varchar(50) = 'A,B'
select t.* from # t
join (select * from string_split(#str, ',')) s(b)
on t.b = s.b
You may like it better anyway, since there's no dynamic sql involved. However you must be running SQL Server 2016 or higher.
I am newbie to postgresql and I need a postgresql version of this function - could you help me please?
CREATE FUNCTION [dbo].[getpersonname]
(#commid INT)
RETURNS VARCHAR(255)
AS
BEGIN
DECLARE #personnname varchar(255)
SELECT
#personnname = COALESCE(#personname + ',', '') +
ISNULL(CONVERT(VARCHAR(50), pers.personnname ),'')
FROM
dbo.comlink comli
INNER JOIN
dbo.person pers ON personid= comli_personid
WHERE
comli.comli_commid = #commid
RETURN #personnname
END
If I understand that code correctly it returns all names separated with commas.
That can easily be done with a simple SQL statement in Postgres:
create FUNCTION get_person_name(p_commid int)
RETURNS text
AS
$$
SELECT string_agg(pers.personnname, ',')
FROM dbo.comlink comli
JOIN dbo.person pers ON personid = comli_personid
WHERE comli.comli_commid= p_commid;
$$
language sql;
I've created a full-text indexed column on a table.
I have a stored procedure to which I may pass the value of a variable "search this text". I want to search for "search", "this" and "text" within the full-text column. The number of words to search would be variable.
I could use something like
WHERE column LIKE '%search%' OR column LIST '%this%' OR column LIKE '%text%'
But that would require me to use dynamic SQL, which I'm trying to avoid.
How can I use my full-text search to find each of the words, presumably using CONTAINS, and without converting the whole stored procedure to dynamic SQL?
If you say you definitely have SQL Table Full Text Search Enabled, Then you can use query like below.
select * from table where contains(columnname,'"text1" or "text2" or "text3"' )
See link below for details
Full-Text Indexing Workbench
So I think I came up with a solution. I created the following scalar function:
CREATE FUNCTION [dbo].[fn_Util_CONTAINS_SearchString]
(
#searchString NVARCHAR(MAX),
#delimiter NVARCHAR(1) = ' ',
#ANDOR NVARCHAR(3) = 'AND'
)
RETURNS NVARCHAR(MAX)
AS
BEGIN
IF #searchString IS NULL OR LTRIM(RTRIM(#searchString)) = '' RETURN NULL
-- trim leading/trailing spaces
SET #searchString = LTRIM(RTRIM(#searchString))
-- remove double spaces (prevents empty search terms)
WHILE CHARINDEX(' ', #searchString) > 0
BEGIN
SET #searchString = REPLACE(#searchString,' ',' ')
END
-- reformat
SET #searchString = REPLACE(#searchString,' ','" ' + #ANDOR + ' "') -- replace spaces with " AND " (quote) AND (quote)
SET #searchString = ' "' + #searchString + '" ' -- surround string with quotes
RETURN #searchString
END
I can get my results:
DECLARE #ftName NVARCHAR (1024) = dbo.fn_Util_CONTAINS_SearchString('value1 value2',default,default)
SELECT * FROM Table WHERE CONTAINS(name,#ftName)
I would appreciate any comments/suggestions.
For your consideration.
I understand your Senior wants to avoid dynamic SQL, but it is my firm belief that Dynamic SQL is NOT evil.
In the example below, you can see that with a few parameters (or even defaults), and a 3 lines of code, you can:
1) Dynamically search any source
2) Return desired or all elements
3) Rank the Hit rate
The SQL
Declare #SearchFor varchar(max) ='Daily,Production,default' -- any comma delim string
Declare #SearchFrom varchar(150) ='OD' -- table or even a join statment
Declare #SearchExpr varchar(150) ='[OD-Title]+[OD-Class]' -- Any field or even expression
Declare #ReturnCols varchar(150) ='[OD-Nr],[OD-Title]' -- Any field(s) even with alias
Set #SearchFor = 'Sign(CharIndex('''+Replace(Replace(Replace(#SearchFor,' , ',','),', ',''),',',''','+#SearchExpr+'))+Sign(CharIndex(''')+''','+#SearchExpr+'))'
Declare #SQL varchar(Max) = 'Select * from (Select Distinct'+#ReturnCols+',Hits='+#SearchFor+' From '+#SearchFrom + ') A Where Hits>0 Order by Hits Desc'
Exec(#SQL)
Returns
OD-Nr OD-Title Hits
3 Daily Production Summary 2
6 Default Settings 1
I should add that my search string is comma delimited, but you can change to space.
Another note CharIndex can be substanitally faster that LIKE. Take a peek at
http://cc.davelozinski.com/sql/like-vs-substring-vs-leftright-vs-charindex
I have this my stored procedure:
create procedure [dbo].[Sp_AddPermission]
#id nvarchar(max)
as
declare #words varchar(max), #sql nvarchar(max)
set #words = #id
set #sql = 'merge admin AS target
using (values (''' + replace(replace(#words,';','),('''),'-',''',') + ')) AS source(uname, [add], [edit], [delete], [view],Block)
on target.uname = source.uname
when matched then update set [add] = source.[add], [edit] = source.[edit], [delete] = source.[delete], [view] = source.[view], [Block]=source.[Block];'
exec(#sql);
When executing it, this error is shown:
The MERGE statement attempted to UPDATE or DELETE the same row more
than once. This happens when a target row matches more than one source
row. A MERGE statement cannot UPDATE/DELETE the same row of the target
table multiple times. Refine the ON clause to ensure a target row
matches at most one source row, or use the GROUP BY clause to group
the source rows.
How to resolve this?
Regards
Baiju
The problem is obvious: you a generating a source table with multiple values for the same uname.
Why are you using a merge for this? I think a simple update would do, and I don't think update will return an error when you have multiple identical keys in source:
update t
set [add] = source.[add],
[edit] = source.[edit],
[delete] = source.[delete],
[view] = source.[view],
[Block]=source.[Block]
from target t join
(values(. . . )) s(uname, [add], [edit], [delete], [view],Block)
on t.uname = s.uname;
But, you could fix this if you like by choosing an arbitrary row for the update (which is what the above does):
update t
set [add] = source.[add],
[edit] = source.[edit],
[delete] = source.[delete],
[view] = source.[view],
[Block]=source.[Block]
from target t join
(select s.*,
row_number() over (partition by uname order by uname) as seqnum
from (values(. . . )) s(uname, [add], [edit], [delete], [view],Block)
) s
on t.uname = s.uname and s.seqnum = 1;
Of course, this approach can also be used with the merge.