Snowflake Stored procedure Copy into Temporary table - snowflake-cloud-data-platform

Question is related to Snowflake and Snowsql. That said I'm trying to within a stored proc create a temp table and then 'copy into' this temp table from azure blob storage.
I'm manually executed the snow sql statements and they work fine.
Statement 1:CREATE TEMPORARY TABLE DB.TABLES.LINE_DETAILS_INCREMENTAL LIKE DB.TABLES.LINE_DETAILS;
Statement 2:
COPY INTO DB.TABLES.LINE_DETAILS_INCREMENTAL FROM (SELECT * FROM #DB.BASE.Azure/S/LINE_DETAILS_INCREMENTAL )
force = false file_format = (type = csv field_delimiter = '|' encoding = 'Windows 1252' skip_header = 0);
But when I encapsulate this into a stored proc and try to run it it gives error:-
"JavaScript compilation error: Uncaught SyntaxError: Unexpected identifier in SP_DELETE_LINE_DETAILS at ' var insert_clause = 'COPY INTO DB.TABLES.LINE_DETAILS_INCREMENTAL FROM (SELECT * FROM #Feeds_DB.BASE.Azure/S/LINE_DETAILS_INCREMENTAL ) force = true file_format = (type = csv field_delimiter = '|' encoding = 'Windows1252' skip_header = 0) On_error = continue;'' position 288 ".
Code of the stored proc is:-
CREATE or replace procedure "DB"."TABLES"."SP_DELETE_INSERT_3DAYS_INTO_LINE_DETAILS"()
returns varchar(1000)
language javascript
as
$$
try{
var create_clause = 'CREATE TEMPORARY TABLE DB.TABLES.LINE_DETAILS_INCREMENTAL LIKE DB.TABLES.PS_TRANSACTION_LINE_DETAILS;'
var create_stmt = snowflake.createStatement({sqlText: create_clause});
var create_res = create_stmt.execute();
var insert_clause = 'COPY INTO DB.TABLES.LINE_DETAILS_INCREMENTAL FROM (SELECT * FROM #Tableau_Feeds_DB.BASE_TABLES.JDAStagingAzure/POS/PS_TRANSACTION_LINE_DETAILS_INCREMENTAL ) force = true file_format = (type = csv field_delimiter = '|' encoding = 'Windows1252' skip_header = 0) On_error = continue;'
var insert_stmt = snowflake.createStatement({sqlText: insert_clause});
var insert_res = insert_stmt.execute();
var select_clause = 'select distinct TO_CHAR(TO_DATE(CREATE_DATE)) as CREATE_DATE from DB.TABLES.LINE_DETAILS_INCREMENTAL order by CREATE_DATE';
var select_stmt = snowflake.createStatement({sqlText: select_clause});
var select_res = select_stmt.execute();
while (select_res.next())
{
date_ip = select_res.getColumnValue(1);
var desc_user_sql = `delete from DB.TABLES.LINE_DETAILS where TO_DATE(CREATE_DATE) = :1;`
var desc_user_stmt = snowflake.createStatement({sqlText: desc_user_sql, binds: [date_ip]});
var desc_user_sql2 = `INSERT INTO DB.TABLES.PS_TRANSACTION_LINE_DETAILS select * from DB.TABLES.PS_TRANSACTION_LINE_DETAILS_INCREMENTAL where TO_DATE(CREATE_DATE) = :1;`
var desc_user_stmt2 = snowflake.createStatement({sqlText: desc_user_sql2, binds: [date_ip]});
try{
desc_user_stmt.execute();
desc_user_stmt2.execute();
}
catch(err)
{
return "Error inserting records: " +err;
}
}
return "Data has been insert in success!";
}
catch(err){
return "Error whileselecting Roles : " +err;
}
return 0;
$$

I think the issue you have here is that you're using single quote marks both to start/end your string, and within the string itself.
For example, you have the following phrase in your string:
encoding = 'Windows1252'
I would suggest escaping the additional quotation marks with a backslash, like so:
encoding = \'Windows1252\'
Do this for all additional quotation marks and you should be fine.
Let me know if you still face issues after!

Try changing single quotes as below
var insert_clause = `COPY INTO DB.TABLES.LINE_DETAILS_INCREMENTAL FROM (SELECT * FROM #Tableau_Feeds_DB.BASE_TABLES.JDAStagingAzure/POS/PS_TRANSACTION_LINE_DETAILS_INCREMENTAL ) force = true file_format = (type = csv field_delimiter = '|' encoding = 'Windows1252' skip_header = 0) On_error = continue;`
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html#line-continuation

Related

Snowflake's get_ddl displays extra single quotes everywhere

In Snowflake, when I create a store proc like so
create procedure stack_overflow_question(select_table varchar)
returns varchar
language sql
as
declare
select_statement varchar;
begin
select_statement := '
SELECT * FROM ' || :select_table || '
';
end;
Then, later when I use select get_ddl('procedure', 'stack_overflow_question(varchar)'); function to make edits to the store proc, the result of this function call has extra single quotes.
Here is the result
CREATE OR REPLACE PROCEDURE "STACK_OVERFLOW_QUESTION"("SELECT_TABLE" VARCHAR(16777216))
RETURNS VARCHAR(16777216)
LANGUAGE SQL
EXECUTE AS OWNER
AS 'declare
select_statement varchar;
begin
select_statement := ''
SELECT * FROM '' || :select_table || ''
'';
end';
Note the difference between the two! The extra single quotes. Also double quotes in the name of the store proc.
Is there something that I can do to prevent this from happening? I am using Snowsight - but don't think that this actually is the problem. Also, I am using snowflake as the language for store procs.
Any ideas?
I wrote a UDF that you can wrap around get_ddl that will convert the DDL from using doubled single quotes to single quotes and wrap the body with $$:
create or replace function CODE_DDL_TO_TEXT(CODE_TEXT string)
returns string
language javascript
as
$$
var lines = CODE_TEXT.split("\n");
var out = "";
var startCode = new RegExp("^AS '$", "ig");
var endCode = new RegExp("^'\;$", "ig");
var inCode = false;
var isChange = false;
var s;
for (i = 0; i < lines.length; i++){
isChange = false;
if(!inCode) {
inCode = startCode.test(lines[i]);
if(inCode) {
isChange = true;
out += "AS $" + "$\n";
}
}
if (endCode.test(lines[i])){
out += "$" + "$;";
isChange = true;
inCode = false;
}
if(!isChange){
if(inCode){
s = lines[i].replace(/''/g, "'") + "\n";
s = s.replace(/\\\\/g, "\\");
out += s;
} else {
out += lines[i] + "\n";
}
}
}
return out;
$$;
You can then call the UDF by wrapping it around the get_ddl function. Here is an example of fishing its own DDL out of get_ddl:
select CODE_DDL_TO_TEXT(get_ddl('function', 'CODE_DDL_TO_TEXT(string)'));
Edit:
You can also use this SQL to reconstruct a stored procedure from the INFORMATION_SCHEMA:
select 'create or replace procedure ' || PROCEDURE_NAME || ARGUMENT_SIGNATURE ||
'\nreturns ' || DATA_TYPE ||
'\nlanguage ' || PROCEDURE_LANGUAGE ||
'\nas $' || '$\n' ||
PROCEDURE_DEFINITION ||
'\n$' || '$;'
from INFORMATION_SCHEMA.PROCEDURES
;
This only returns body -
SELECT PROCEDURE_DEFINITION
FROM INFORMATION_SCHEMA.PROCEDURES
WHERE PROCEDURE_SCHEMA = 'SCHEMA_NAME' AND PROCEDURE_NAME = upper('stack_overflow_question');

Can I use a variable in a stage path?

I'm using a COPY INTO statement to load some tables into S3:
COPY INTO 's3://sandbox-staging/US/'
FROM US
storage_integration = sandbox
FILE_FORMAT = (
type = 'parquet'
)
header = true
overwrite = true;
I have to do a migration like this for every state. To save some time and protect against human error, I'd love to set the table name as a variable, so that I can use it in both the COPY INTO and FROM clauses. For example:
SET loc = 'US_NY';
SET staging_path = 's3://sandbox-staging/' || $loc || '/';
COPY INTO $staging_path
FROM table($loc)
storage_integration = sandbox
FILE_FORMAT = (
type = 'parquet'
)
header = true
overwrite = true;
The FROM clause works, it's the COPY INTO I can't seem to get right. In the same sense that there's a table function for table literals, is there any literal function I can use for staging paths?
You can try using a variable with execute immediate to dynamically generate the command. https://docs.snowflake.com/en/sql-reference/sql/execute-immediate.html
SET loc = 'US_NY';
SET staging_path = '''s3://sandbox-staging/' || $loc || '/''' ;
SET copy_command=
'COPY INTO ' || $staging_path ||
' FROM ' || $loc ||
' storage_integration = sandbox
FILE_FORMAT = (
type = \'parquet\'
)
header = true
overwrite = true;';
EXECUTE IMMEDIATE $copy_command;
To view the copy command code you can run:
SELECT $copy_command;
Output:
COPY INTO 's3://sandbox-staging/US_NY/' FROM US_NY storage_integration = sandbox FILE_FORMAT = ( type = 'parquet' ) header = true overwrite = true;
Going to the original requirement - running this for all states - this is tailor made for a SQL generator.
create or replace table TABLE_LIST(NAME string);
insert into TABLE_LIST (NAME) values ('US_NY'), ('US_CA'), ('US_NC'), ('US_FL');
select $$
COPY INTO 's3://sandbox-staging/$$ || NAME || $$/'$$ || $$
FROM $$ || NAME || $$
storage_integration = sandbox
FILE_FORMAT = (
type = 'parquet'
)
header = true
overwrite = true
$$ as SQL_COMMAND
from TABLE_LIST;
That will generate all the SQL commands in a table. If you want to automate running them, you can use a stored procedure to do that. For running generated SQL statements, there's one available to do that already.
https://snowflake.pavlik.us/index.php/2019/08/22/executing-multiple-sql-statements-in-a-stored-procedure/
You can then call it like this:
call RunBatchSQL($$
select 'COPY INTO ''s3://sandbox-staging/' || NAME || '\'' ||
' FROM ' || NAME ||
' storage_integration = sandbox
FILE_FORMAT = (
type = ''parquet''
)
header = true
overwrite = true'
as SQL_COMMAND
from TABLE_LIST;
$$);
It may be cleaner to write a stored procedure from scratch, but this allows you to run any generated SQL statements.

How To Get Snowflake Stored Procedure DDL with $$

We prefer coding Snowflake stored procedures and javascript UDF's using the $$ notation. It is easier as we do not have to worry about escaping every single quote in our code. However, when we retrieve the DDL using GET_DDL, Snowflake removes the $$ and places the body of the SP in single quotes and also escapes every single quote.
Is there a way to get the SP DDL from Snowflake in the $$ format?
Example, below is the SP we create. Notice the $$ sign and that we do not hae
CREATE OR REPLACE PROCEDURE "SP_TEST"()
RETURNS VARCHAR(16777216)
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
var result = snowflake.execute({sqlText: "select 'Hello World!' as test;"})
result.next();
return String(result.getColumnValue(1));
$$;
Then, when we retrieve the DDL from Snowflake using SELECT GET_DDL('PROCEDURE','SP_TEST()'), we get the following. Notice the $$ has been replaced by single quotes and that all other single quotes are now escaped.
CREATE OR REPLACE PROCEDURE "SP_TEST"()
RETURNS VARCHAR(16777216)
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS '
var result = snowflake.execute({sqlText: "select ''Hello World!'' as test;"})
result.next();
return String(result.getColumnValue(1));
';
Thanks
I've not found a built-in way to do it, but you can use a UDF to do this. This should work provided that the $$ start and end of the code section are always on lines by themselves:
create or replace function CODE_DDL_TO_TEXT(CODE_TEXT string)
returns string
language javascript
as
$$
var lines = CODE_TEXT.split("\n");
var out = "";
var startCode = new RegExp("^AS '$", "ig");
var endCode = new RegExp("^'\;$", "ig");
var inCode = false;
var isChange = false;
var s;
for (i = 0; i < lines.length; i++){
isChange = false;
if(!inCode) {
inCode = startCode.test(lines[i]);
if(inCode) {
isChange = true;
out += "AS $" + "$\n";
}
}
if (endCode.test(lines[i])){
out += "$" + "$;";
isChange = true;
inCode = false;
}
if(!isChange){
if(inCode){
s = lines[i].replace(/''/g, "'") + "\n";
s = s.replace(/\\\\/g, "\\");
out += s;
} else {
out += lines[i] + "\n";
}
}
}
return out;
$$;
select CODE_DDL_TO_TEXT(get_ddl('function', 'CODE_DDL_TO_TEXT(string)'));
execute immediate $$
DECLARE
PROCS RESULTSET;
ALL_PROCS RESULTSET;
DB_NAME STRING DEFAULT 'PROC_TEST';
TABLE_NAME STRING;
QUERY STRING;
ALL_QUERY STRING;
COUNT INT DEFAULT 0;
OUTPUT STRING DEFAULT '';
c1 CURSOR FOR select DATABASE_NAME from snowflake.information_schema.databases;
BEGIN
TABLE_NAME := CONCAT(DB_NAME, '.INFORMATION_SCHEMA.SCHEMATA');
OPEN c1;
FOR rec IN c1 DO
QUERY := 'SELECT PROCEDURE_CATALOG,PROCEDURE_NAME,PROCEDURE_OWNER,PROCEDURE_DEFINITION FROM ' || rec.DATABASE_NAME || '.INFORMATION_SCHEMA.procedures';
OUTPUT := OUTPUT || QUERY || '\n UNION ALL \n';
--PROCS := (EXECUTE IMMEDIATE :QUERY);
-- ALL_PROCS := ALL_PROCS + PROCS;
END FOR;
--ALL_PROCS := (EXECUTE IMMEDIATE :OUTPUT);
--RETURN table(ALL_PROCS);
RETURN OUTPUT;
END;
$$;

JDBC getmetadata of an oracle procedure inside package in a schema

In the oracle database there is a schema. Inside schema there is a package which contains different methods. How to retrieve the metadata of the procedure using getProcedureColumn() function in DatabaseMetaDataclass?
I have tried to get metadata using getProcedureColumns(catalog,schemaname,procedurename,columnnamepattern) it works fine when the procedure is located inside a schema. When a procedure is located inside a package in a schema it is not retrieving.
This will print out all the column information for a specific procedure in a package. Change parameters with real values.
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
Connection conn =
DriverManager.getConnection ("jdbc:oracle:thin:#<server>:<port>:<sid>", "<username>", "<password>");
DatabaseMetaData metadata = conn.getMetaData();
String packageName = "<your package name>";
String schemaName = "<schema name>";
String procedureName = "<procedure name>";
ResultSet rs = metadata.getProcedureColumns(
packageName,
schemaName,
procedureName,
"%");
while(rs.next()) {
// get stored procedure metadata
String procedureCatalog = rs.getString(1);
String procedureSchema = rs.getString(2);
procedureName = rs.getString(3);
String columnName = rs.getString(4);
short columnReturn = rs.getShort(5);
int columnDataType = rs.getInt(6);
String columnReturnTypeName = rs.getString(7);
int columnPrecision = rs.getInt(8);
int columnByteLength = rs.getInt(9);
short columnScale = rs.getShort(10);
short columnRadix = rs.getShort(11);
short columnNullable = rs.getShort(12);
String columnRemarks = rs.getString(13);
System.out.println("stored Procedure name="+procedureName);
System.out.println("procedureCatalog=" + procedureCatalog);
System.out.println("procedureSchema=" + procedureSchema);
System.out.println("procedureName=" + procedureName);
System.out.println("columnName=" + columnName);
System.out.println("columnReturn=" + columnReturn);
System.out.println("columnDataType=" + columnDataType);
System.out.println("columnReturnTypeName=" + columnReturnTypeName);
System.out.println("columnPrecision=" + columnPrecision);
System.out.println("columnByteLength=" + columnByteLength);
System.out.println("columnScale=" + columnScale);
System.out.println("columnRadix=" + columnRadix);
System.out.println("columnNullable=" + columnNullable);
System.out.println("columnRemarks=" + columnRemarks);
}

Easy way to convert exec sp_executesql to a normal query?

When dealing with debugging queries using Profiler and SSMS, its pretty common for me to copy a query from Profiler and test them in SSMS. Because I use parameterized sql, my queries are all sent as exec sp_executesql queries.
exec sp_executesql
N'/*some query here*/',
N'#someParameter tinyint',
# someParameter =2
I'll take this and convert it into a normal query for ease of editing (intellisense, error checking, line numbers, etc):
DECLARE #someParameter tinyint
SET #someParameter = 2
/*some query here*/
Of course, the bigger and more complex the query, the harder to do this. And when you're going back and forth multiple times, it can be a pain in the ass and soak up lots of time.
Is there an easy (e.g., macro command) way to convert muh executesql into something more convenient?
I spent a little time making an simple script that did this for me. It's a WIP, but I stuck a (very ugly) webpage in front of it and it's now hosted here if you want to try it:
http://execsqlformat.herokuapp.com/
Sample input:
exec sp_executesql
N'SELECT * FROM AdventureWorks.HumanResources.Employee
WHERE ManagerID = #level',
N'#level tinyint',
#level = 109;
And the output:
BEGIN
DECLARE #level tinyint;
SET #level = 109;
SELECT * FROM AdventureWorks.HumanResources.Employee
WHERE ManagerID = #level
END
The formatting of the actual SQL statement once I've plucked it from the input is done using the API at http://sqlformat.appspot.com
I was looking for something similar so I use this in LinqPad, just copy sp_executesql statement to the clipboard and run the code in LinqPad. It outputs the SQL statement.
void Main()
{
ConvertSql(System.Windows.Forms.Clipboard.GetText()).Dump();
}
private static string ConvertSql(string origSql)
{
string tmp = origSql.Replace("''", "~~");
string baseSql;
string paramTypes;
string paramData = "";
int i0 = tmp.IndexOf("'") + 1;
int i1 = tmp.IndexOf("'", i0);
if (i1 > 0)
{
baseSql = tmp.Substring(i0, i1 - i0);
i0 = tmp.IndexOf("'", i1 + 1);
i1 = tmp.IndexOf("'", i0 + 1);
if (i0 > 0 && i1 > 0)
{
paramTypes = tmp.Substring(i0 + 1, i1 - i0 - 1);
paramData = tmp.Substring(i1 + 1);
}
}
else
{
throw new Exception("Cannot identify SQL statement in first parameter");
}
baseSql = baseSql.Replace("~~", "'");
if (!String.IsNullOrEmpty(paramData))
{
string[] paramList = paramData.Split(",".ToCharArray());
foreach (string paramValue in paramList)
{
int iEq = paramValue.IndexOf("=");
if (iEq < 0)
continue;
string pName = paramValue.Substring(0, iEq).Trim();
string pVal = paramValue.Substring(iEq + 1).Trim();
baseSql = baseSql.ReplaceWholeWord(pName, pVal);
}
}
return baseSql;
}
public static class StringExtensionsMethods
{
/// <summary>
/// Replaces the whole word.
/// </summary>
/// <param name="s">The s.</param>
/// <param name="word">The word.</param>
/// <param name="replacement">The replacement.</param>
/// <returns>String.</returns>
public static String ReplaceWholeWord(this String s, String word, String replacement)
{
var firstLetter = word[0];
var sb = new StringBuilder();
var previousWasLetterOrDigit = false;
var i = 0;
while (i < s.Length - word.Length + 1)
{
var wordFound = false;
var c = s[i];
if (c == firstLetter)
if (!previousWasLetterOrDigit)
if (s.Substring(i, word.Length).Equals(word))
{
wordFound = true;
var wholeWordFound = true;
if (s.Length > i + word.Length)
{
if (Char.IsLetterOrDigit(s[i + word.Length]))
wholeWordFound = false;
}
sb.Append(wholeWordFound ? replacement : word);
i += word.Length;
}
if (wordFound) continue;
previousWasLetterOrDigit = Char.IsLetterOrDigit(c);
sb.Append(c);
i++;
}
if (s.Length - i > 0)
sb.Append(s.Substring(i));
return sb.ToString();
}
}
I spent a little time and created a small modification of Matt Roberts / Wangzq solutions without DECLAREs section, you can try it on .NET Fiddle or download LINQPad 5 file.
Input:
exec sp_executesql N'UPDATE MyTable SET [Field1] = #0, [Field2] = #1',N'#0 nvarchar(max) ,#1 int',#0=N'String',#1=0
Output:
UPDATE MyTable SET [Field1] = N'String', [Field2] = 0
Code:
using System;
using System.Linq;
using System.Text.RegularExpressions;
public class Program
{
public static void Main()
{
var sql = #"exec sp_executesql N'UPDATE MyTable SET [Field1] = #0, [Field2] = #1',N'#0 nvarchar(max) ,#1 int',#0=N'String',#1=0";
Console.WriteLine(ConvertSql(sql));
}
public static string ConvertSql(string origSql)
{
var re = new Regex(#"exec*\s*sp_executesql\s+N'([\s\S]*)',\s*N'(#[\s\S]*?)',\s*([\s\S]*)", RegexOptions.IgnoreCase); // 1: the sql, 2: the declare, 3: the setting
var match = re.Match(origSql);
if (match.Success)
{
var sql = match.Groups[1].Value.Replace("''", "'");
//var declare = match.Groups[2].Value;
var setting = match.Groups[3].Value + ',';
// to deal with comma or single quote in variable values, we can use the variable name to split
var re2 = new Regex(#"#[^',]*?\s*=");
var variables = re2.Matches(setting).Cast<Match>().Select(m => m.Value).ToArray();
var values = re2.Split(setting).Where(s=>!string.IsNullOrWhiteSpace(s)).Select(m => m.Trim(',').Trim().Trim(';')).ToArray();
for (int i = variables.Length-1; i>=0; i--)
{
sql = Regex.Replace(sql, "(" + variables[i].Replace("=", "")+")", values[i], RegexOptions.Singleline);
}
return sql;
}
return #"Unknown sql query format.";
}
}
Another solution which replaces the parameter values directly in the query
(not exactly what you asked for but it might prove useful to others):
https://code.msdn.microsoft.com/windowsdesktop/spExecuteSql-parser-1a9cd7bc
I goes from:
exec sp_executesql N'UPDATE Task SET Status = #p0, Updated = #p1 WHERE Id = #p2 AND Status = #p3 AND Updated = #p4',N'#p0 int,#p1 datetime,#p2 int,#p3 int,#p4 datetime',#p0=1,#p1='2015-02-07 21:36:30.313',#p2=173990,#p3=2,#p4='2015-02-07 21:35:32.830'
to:
UPDATE Task SET Status = 1, Updated = '2015-02-07 21:36:30.313' WHERE Id = 173990 AND Status = 2 AND Updated = '2015-02-07 21:35:32.830'
which makes it easier to understand.
The console application on that page can be used by passing a file parameter or copying the sp_executesql in the clipboard, running the app and then pasting the resulting SQL from the clipboard.
Update:
An SQL formatter can also be added to that solution for easier readability:
http://www.nuget.org/packages/PoorMansTSQLFormatter/
newSql = ConvertSql(Clipboard.GetText());
var formattedSql = SqlFormattingManager.DefaultFormat(newSql);
Clipboard.SetText(formattedSql);
You can use this Azur data studio extension. it based on #Matt Roberts repo.
https://github.com/PejmanNik/sqlops-spexecutesql-to-sql/releases/tag/0.0.1
Sql Prompt got this feature recently (2017-02-06). Select the text and look for "Inline EXEC" in the context menu. Gotta love Prompt :)
I am not aware of an existing Add-In that can do this. But you could create one :)
A few regular expressions and some string concatenation and after that sell it to Vinko and other souls looking for this functionality.
If you're feeling like diving into this, here is some information on creating an SSMS addin:
http://sqlblogcasts.com/blogs/jonsayce/archive/2008/01/15/building-a-sql-server-management-studio-addin.aspx
I faced with this problem too and wrote simple application for solving it - ClipboardSqlFormatter. This is a tray application that listens clipboard input events and tries to detect and convert dynamic sql to static sql.
Anything you need is to copy dynamic sql (from sql profiler for example) and paste to text editor - pasted sql will be a static sql :)
For example, if copied sql is:
exec sp_executesql N' SELECT "obj"."CreateDateTime", "obj"."LastEditDateTime" FROM LDERC
"doc" INNER JOIN LDObject "obj" ON ("doc"."ID" = "obj"."ID") LEFT OUTER JOIN LDJournal
"ContainerID.jrn" ON ("doc"."JournalID" = "ContainerID.jrn"."ID") WHERE ( "doc"."ID"
= #V0 AND ( "doc"."StateID" <> 5 AND "ContainerID.jrn"."Name" <> ''Hidden journal''
) ) ',N'#V0 bigint',#V0=6815463'
then pasted sql will be:
SELECT "obj"."CreateDateTime"
,"obj"."LastEditDateTime"
FROM LDERC "doc"
INNER JOIN LDObject "obj" ON ("doc"."ID" = "obj"."ID")
LEFT OUTER JOIN LDJournal "ContainerID.jrn" ON ("doc"."JournalID" = "ContainerID.jrn"."ID")
WHERE (
"doc"."ID" = 6815463
AND (
"doc"."StateID" <> 5
AND "ContainerID.jrn"."Name" <> 'Hidden journal'
)
)
Conclusion: I note this still gets a little attention so I'll add details here for what my eventual solution was.
It turns out that nothing beats doing it for yourself. I created a simple console app that parsed my stored procedure and spit out what I wanted. By adding it to the list of external tools, and passing the current filename as an argument, I could use the following to strip out and rearrange what I needed.
In use, I'd add a new sql file, paste in the sql, save it, then run the external tool. After it completes, the IDE asks me to reload the file. Poof, no more stored procedure.
I do note that this may not work with every executesql statement, so you'll have to modify if it does not meet your needs.
class Program
{
const string query = "query";
const string decls = "decls";
const string sets = "sets";
static void Main(string[] args)
{
try
{
var text = File.ReadAllText(args[0]);
if(string.IsNullOrEmpty(text))
{
Console.WriteLine("File is empty. Try saving it before using the hillbilly sproc decoder");
}
var regex = new Regex(#"exec sp_executesql N'(?<" + query + ">.*)',N'(?<" + decls + ">[^']*)',(?<" + sets + ">.*)", RegexOptions.Singleline);
var match = regex.Match(text);
if(!match.Success || match.Groups.Count != 4)
{
Console.WriteLine("Didn't capture that one. Shit.");
Console.Read();
return;
}
var sb = new StringBuilder();
sb.Append("DECLARE ").AppendLine(match.Groups[decls].Value);
foreach(var set in match.Groups[sets].Value.Split(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
sb.Append("SET ").AppendLine(set);
sb.AppendLine(match.Groups[query].Value.Replace("''", "'"));
File.WriteAllText(args[0], sb.ToString());
}
catch(Exception ex)
{
Console.WriteLine("S*t blew up, yo");
Console.WriteLine(ex.ToString());
Console.WriteLine("Press a key to exit");
Console.Read();
}
}
}
Here is simple UI that i use to inspect NHibernate queries. Some regex, parsing and sqlformat.org API to beautify sql is used.
<html>
<head>
<script src="https://code.jquery.com/jquery-3.4.1.min.js"></script>
<script type="text/javascript">
$(function(){
$("#btn-format").on("click", () =>{
var insql = $("#textarea-in").val();
var regex = new RegExp("exec sp_executesql N'(?<command>.+?(?='(,N'$)?))'(,\s*N'(?<types>.+?(?=',))',\s*(?<vals>.+))?");
var groups = insql.replace(/\n|\r/g, "").match(regex).groups;
var outsql = "";
if (groups.types)
{
var types = groups.types.match(/#[^\s]+ \w+(\([\w\d,]+\))?/g);
for (const typeDeclaration of types) {
outsql = outsql + 'declare ' + typeDeclaration + '\n';
}
outsql = outsql + '\n';
for (const setVal of groups.vals.split(',')) {
outsql = outsql + 'set ' + setVal + '\n';
}
outsql = outsql + '\n';
}
$.ajax({
url: 'https://sqlformat.org/api/v1/format',
type: 'POST',
dataType: 'json',
crossDomain: true,
data: {
sql: groups.command, reindent: 1
},
success: (data) => {
outsql = outsql + data.result;
$("#textarea-out").val(outsql);
},
error: () =>{
outsql = outsql + '-- No format happened. See browser console for details \n';
outsql = outsql + groups.command;
$("#textarea-out").val(outsql);
}
});
})
});
</script>
</head>
<body>
<textarea id="textarea-in" style="width: 100%; height: 48%;" class="form-control" placeholder="type 'exec sp_executesql...' here"></textarea>
<br/>
<button id="btn-format">Format</button>
<br/>
<textarea id="textarea-out" style="width: 100%; height: 48%;" class="form-control"></textarea>
</body>
Test in Fiddle
Note: Will not work, if you have single quotes in query
I have improved one of previous answers particularly refining regular expression in order to support queries without parameters. Here is my option in form of PowerShell script, which uses Windows clipboard both as input and as output:
$regex = "(?s)^exec sp_executesql N'(?<query>.*?[^'])'(?:,N'(?<decls>.*?[^'])',(?<sets>.*))?$"
$inputText = [string](Get-Clipboard)
if (!($inputText -match $regex)) {
return
}
$resultBuilder = [System.Text.StringBuilder]::new()
if ($matches["decls"]) {
$resultBuilder.AppendLine("DECLARE $($matches["decls"])")
$matches["sets"].Split(',') | ForEach-Object {
$resultBuilder.AppendLine("SET $_");
}
}
$resultBuilder.AppendLine($matches["query"].Replace("''", "'"));
Set-Clipboard $resultBuilder.ToString()

Resources