I have an unusual situation - first time!
I'm trying to write a pre-deployment script for an SSDT Project in Visual Studio, which will prepare a load of data for migration. There's a user-defined table-valued function (TF) in the new version of the database called [MySchema].[MyFunc] that will really help with the migration script... but it doesn't exist yet in the old database that will be upgraded and migrated. There are many other objects in the new version, but I just need to use this one to help with the migration. The function doesn't have any dependencies on any other new (or existing) objects, it's totally self-contained.
I'm hoping to compile [MySchema].[MyFunc] as part of the pre-deployment, so that I can use it. The function lives in .\MySchema\Functions\MyFunc.sql
I've attempted the following...
ATTEMPT 1
This fails with Incorrect syntax near CREATE (Note: CREATE is the first line of file .\MySchema\Functions\MyFunc.sql):
IF object_id('[MySchema].[MyFunc]', 'TF') IS NULL
BEGIN
:r .\MySchema\Functions\MyFunc.sql
END
ATTEMPT 2
This fails with Incorrect syntax near 'GO' and Incorrect syntax near ':' Expecting CONVERSATION:
GO
IF object_id('[MySchema].[MyFunc]', 'TF') IS NULL
BEGIN
:r .\MySchema\Functions\MyFunc.sql
GO
END
ATTEMPT 3
Copy and paste the entire CREATE FUNCTION statement into my pre-deployment script:
CREATE FUNCTION [MySchema].[MyFunc]
(
#p1 VARCHAR(255)
)
RETURNS #returntable TABLE
(
some_col NVARCHAR(MAX)
)
AS
BEGIN
-- some function code here
RETURN
END
But this fails with CREATE FUNCTION must be the only statement in the batch. I've tried inserting GO before & after this, but I get similar results to ATTEMPT 2. I've also tried with ; instead of GO.
ATTEMPT 4
I tried to use an iTVF, but it didn't help
ATTEMPT 5
I considered, for a few brief moments, taking the code out of my function and just using it without a function... but I need to use that code around 20 times in the migration script. So I dismissed this option. It will produce a different result every time (due to changing parameters), so I can't just put it in a CTE or similar and re-use it each time.
PLEASE NOTE
Answers should be SSDT project specific. This isn't code being run in SSMS. It's in Visual Studio and will be run as part of a Publish process. If you're not sure what SSDT or a pre-deployment script is, please don't answer :-) Any answers not based on SSDT are irrelevant to this scenario. Thanks!
If you're targeting SQL Server 2016 or later two possible solutions come to mind:
in a seperate batch before including your function code try:
DROP FUNCTION IF EXISTS [MySchema].[MyFunc];
revise the content of the MyFunc.sql file such that it starts with:
CREATE OR ALTER FUNCTION [MySchema].[MyFunc]
...
References:
DROP FUNCTION (Transact-SQL)
CREATE FUNCTION (Transact-SQL)
You can't have GO inside BEGIN/END. I don't see why Attempt 3 doesn't work, most probably because of other code you have in the same script. Normally you can create objects in the pre-script. Depending on the version of SQL Server you can either use IF EXISTS or change the code to DROP the function if it exists and then always CREATE it instead of the opposite approach you are trying to do.
Another way is to simply put data population logic to the pre-script itself. You can do something like:
IF NOT EXISTS (SELECT * FROM SomeTable)
BEGIN
INSERT INTO SomeTable ...
END
In such case you'll be able to re-run this script without duplicating the data.
Related
SQL Server has Deferred Name Resolution feature, read here for details:
https://msdn.microsoft.com/en-us/library/ms190686(v=sql.105).aspx
In that page, all it's talking is stored procedure so it seems Deferred Name Resolution only works for stored procedures and not for functions and I did some testing.
create or alter function f2(#i int)
returns table
as
return (select fff from xxx)
go
Note the table xxx does not exist. When I execute the above CREATE statement, I got the following message:
Msg 208, Level 16, State 1, Procedure f2, Line 4 [Batch Start Line 22]
Invalid object name 'xxx'.
It seems that SQL Server instantly found the non-existent table xxx and it proved Deferred Name Resolution doesn't work for functions. However when I slightly change it as follows:
create or alter function f1(#i int)
returns int
as
begin
declare #x int;
select #x = fff from xxx;
return #x
end
go
I can successfully execute it:
Commands completed successfully.
When executing the following statement:
select dbo.f1(3)
I got this error:
Msg 208, Level 16, State 1, Line 34
Invalid object name 'xxx'.
So here it seems the resolution of the table xxx was deferred. The most important differences between these two cases is the return type. However I can't explain when Deferred Name Resolution will work for functions and when not. Can anyone help me to understand this? Thanks in advance.
It feels like you were looking for understanding of why your particular example didn't work. Quassnoi's answer was correct but didn't offer a reason so I went searching and found this MSDN Social answer by Erland Sommarskog. The interesting part:
However, it does not extend to views and inline-table functions. For
stored procedures and scalar functions, all SQL Server stores in the
database is the text of the module. But for views and inline-table
functions (which are parameterised view by another name) SQL Server
stores metadata about the columns etc. And that is not possible if the
table is missing.
Hope that helps with understanding why :-)
EDIT:
I did take some time to confirm Quassnoi's comment that sys.columns as well as several other tables did contain some metadata about the inline function so I am unsure if there is other metadata not written. However I thought I would add a few other notes I was able to find that may help explain in conjunction.
First a quote from Wayne Sheffield's blog:
In the MTVF, you see only an operation called “Table Valued Function”. Everything that it is doing is essentially a black box – something is happening, and data gets returned. For MTVFs, SQL can’t “see” what it is that the MTVF is doing since it is being run in a separate context. What this means is that SQL has to run the MTVF as it is written, without being able to make any optimizations in the query plan to optimize it.
Then from the SQL Server 2016 Exam 70-761 by Itzik Ben-Gan (Skill 3.1):
The reason that it's called an inline function is because SQL Server inlines, or expands, the inner query definition, and constructs an internal query directly against the underlying tables.
So it seems the inline function essentially returns a query and is able to optimize it with the outer query, not allowing the black-box approach and thus not allowing deferred name resolution.
What you have in your first example is an inline function (it does not have BEGIN/END).
Inline functions can only be table-valued.
If you used a multi-statement table-valued function for you first example, like this:
CREATE OR ALTER FUNCTION
fn_test(#a INT)
RETURNS #ret TABLE
(
a INT
)
AS
BEGIN
INSERT
INTO #ret
SELECT a
FROM xxx
RETURN
END
, it would compile alright and fail at runtime (if xxx would not exist), same as a stored procedure or a scalar UDF would.
So yes, DNR does work for all multi-statement functions (those with BEGIN/END), regardless of their return type.
In Sql Developer (SqlDev) (the Oracle tool) I most of the time use ';' to indicate the end of a batch.
Also we use the ';' when having a larger script with lots of batches in it. For example a script where the first batch creates a table, the second inserts data in that table, the third does a join with the just created table and another table, etc.
On SqlDev the script (with the different batches in it) works fine. But when we copied the exact script to SQL Server Management Studio (SMSS) and ran it, it gave errors that the table (of the third batch where the created table is joined) does not exist.
How can I make the scipt run on SMSS without the script failing?
In SQL server you can use 'GO' to split a batch or block of statement.
something like below.
ALTER TABLE [dbo].[Security] ADD CONSTRAINT [DF_Security_ImportSettings] DEFAULT ((11111011111111111.)) FOR [ImportSettings]
GO
ALTER TABLE [dbo].[Security] ADD CONSTRAINT [DF_Security_PricingType] DEFAULT ((-1)) FOR [PricingType]
GO
ALTER TABLE [dbo].[Security] ADD CONSTRAINT [DF_Security_AutoUpdateCustomPricing] DEFAULT ((1)) FOR [AutoUpdateCustomPricing]
GO
Go is the Keyword you are looking for ..
Example..
insert into t1
select 1
go
alter table t1
add abc int
this is also configurable in SSMS(i haven't tested though) to ; or some other word..
It appears that in SQL Server Management Studio (SMSS) sometimes it is needed to use 'GO' instead of ';'.
The ';' works differently in SQL Server Management Studio (SSMS) compared to the use of ';' in for example SQl Developer (SQLDev) (the Oracle tool).
In SQLDev the ';' acts as a end of batch indicator, where SSMS doesn't see it when using DLL. Instead SMSS first looks at the whole script and thinks of smart ways to run it. Which means all is run parallel, where some batches are dependent of others, but they are not run properply and gives failure of the script.
In my situation it meant I had to use 'GO' to tell the DBMS to run the first, second and third sequantial instead of parallel. I changed all the ';' with GO in the script (in fact it has a whole lot more batches in it) and that did the trick. I'm not sure it is completely right to do it this way, but at least it worked. :)
Also see:
What is the use of GO in SQL Server Management Studio & Transact SQL?
When do I need to use Begin / End Blocks and the Go keyword in SQL Server?
I'm trying to do something like:
"If it exists, use it. If not, create it."
"If it exists, delete it. If not, create it."
One place it's definitely choking is the use it command - because if it DOES NOT EXIST - it chokes on the use command EVEN THOUGH that command will not run.
Here's more explanation:
I have a SQL Server script where I create a database and then I use the database.
The script will not run
because the use database command is invalid
because the database does not exist
but it will exist after the first command executes
but it doesn't matter because it doesn't exist NOW so the script will not run.
How do I put code in there that tries to use a database that might not exist?
How do I put code in there that will cause an error if run directly but WILL NOT RUN unless conditions are appropriate.
Please see the attached images.
Here's the code so you don't have to type it...
-- SQL SERVER: We can't run this script because CFPT does not exist.
-- ME: But it WILL exist after the first command runs
-- SQL SERVER: That does not matter - at THIS point in the code... it does not exist... tough luck
-- CREATE THE DATABASE
create database CFPT
-- USE THE DATABASE
USE CFPT
use master
drop database CFPT
Second code snippet:
-- SQL SERVER: We can't run this script because CFPT does not exist.
select db_id('CFPT') -- this just lets us see what the IF statement is going to have to deal with
IF db_id('CFPT') is null
begin
print 'DESIRED DB DOES NOT EXIST'
return
end
else
begin
use CFPT -- this line of code makes the whole script just not run.
end;
-- doesn't want to work - chokes on the use databasename (when the database does not exist)
(EDIT 1 start ////////////////////////////////////////////////////////////////////////////////////)
A third image was added with this edit - The SECOND image shows that the if/then/else statement will not work. The 3rd image shows that the database CFPT is not in the database list (left side of image) and the select statement was run (top highlighed code) and the results of that select (bottom red circle)
How do I get the if/then/else statement to work? (Because the THEN will not run if the conditions are not favorable shall-we-say)
(for some reason the red wavy lines are not showing up - they should be but they aren't - hmmm)
(EDIT 1 end ////////////////////////////////////////////////////////////////////////////////////)
(EDIT 2 start ////////////////////////////////////////////////////////////////////////////////////)
In relation to this question - trying to segregate commands that would normally fail but will not be attempted to be executed unless conditions are just right..... (see 4th image below) I'm segregating some commands with an IF statement (IF 1=2) but SQL Server is going into that IF statement even though the condition is false. Why is that?
(EDIT 2 end ////////////////////////////////////////////////////////////////////////////////////)
Try this ...
-- CREATE THE DATABASE
create database CFPT
GO
-- USE THE DATABASE
USE CFPT
use master
drop database CFPT
The GO command is a batch terminator, it separates the command to create the database from the command to use it.
See https://msdn.microsoft.com/en-us/library/ms188037.aspx
and
What is the use of GO in SQL Server Management Studio & Transact SQL?
I'm trying to create a wrapper in T-SQL for a procedure where I'm not sure what the data types are. I can run the wrapper without an INSERT INTO statement and I get the data just fine, but I need to have it in a table.
Whenever I use the INSERT INTO I get an error:
Column name or number of supplied values does not match table definition
I've parsed back through my code and can't see where any column names don't match up, so I'm thinking that it has to be a data type. I've looked through the procedure I'm wrapping to see if I can find what the data types are, but some aren't defined there; I've referenced the tables they pull some data from to find the definitions; I've run SQL_VARIANT_PROPERTY on all of the data to see what data type it is (although some of them come up null).
Is there some better way for me to track down exactly where the error is?
I think you can find out your stored procedure result schema, using sp_describe_first_result_set (available from SQL2012) and FMTONLY. Something like this:
EXEC sp_describe_first_result_set
#tsql = N'SET FMTONLY OFF; EXEC yourProcedure <params are embedded here>'
More details can be found here.
However, if I remember correctly, this works only if your procedure used deterministic schemas (no SELECT INTO #tempTable or similar things).
One trick to find out the schema of your result is to actually materialize the result into ad-hoc created table. However, this is not easy since SELECT INTO does not work with EXEC procedure. One work-around is this:
1) Define a linked-server to the instance itself. E.g. loopback
2) Execute your procedure like this (for SQL 2008R2):
SELECT * INTO tempTableToHoldDataAndStructure
FROM OPENQUERY(' + #LoopBackServerName + ', ''set fmtonly off exec ' + #ProcedureFullName + ' ' + #ParamsStr
where
#LoopBackServerName = 'loopback'
#ProcedureFullName = loopback.database.schema.procedure_name
#ParamsStr = embedded parameters
For SQL2012 I think the execution might fail if RESULT SETS are not provided (i.e. schema definition of the expected result, which is kind of a chicken-egg problem in this case):
' WITH RESULT SETS (( ' + #ResultSetStr + '))'');
Okay, I have a solution to my problem. It's tedious, but tedious I can do. Randomly guessing is what drives me crazy. The procedure I'm wrapping dumps 51 columns. I already know I can get it to work without putting anything into a table. So I decided to comment out part of the select statement in the procedure I'm wrapping so it's only selecting 1 column. (First I made a copy of that procedure so I don't screw up the original; then I referenced the copy from my wrapper). Saved both, ran it, and it worked. So far so good. I could have done it line by line, but I'm more of a binary kind of guy, so I went about halfway down--now I'm including about 25 columns in both the select statement and my table--and it's still working. Repeat procedure until it doesn't work any more, then backtrack until it does again. My error was in identifying one of the data types followed by "IDENTITY". I'm not sure what will happen when I leave that out, but at least my wrapper works.
I remember reading a while back that randomly SQL Server can slow down and / or take a stupidly long time to execute a stored procedure when it is written like:
CREATE PROCEDURE spMyExampleProc
(
#myParameterINT
)
AS
BEGIN
SELECT something FROM myTable WHERE myColumn = #myParameter
END
The way to fix this error is to do this:
CREATE PROCEDURE spMyExampleProc
(
#myParameterINT
)
AS
BEGIN
DECLARE #newParameter INT
SET #newParameter = #myParameter
SELECT something FROM myTable WHERE myColumn = #newParameter
END
Now my question is firstly is it bad practice to follow the second example for all my stored procedures? This seems like a bug that could be easily prevented with little work, but would there be any drawbacks to doing this and if so why?
When I read about this the problem was that the same proc would take varying times to execute depending on the value in the parameter, if anyone can tell me what this problem is called / why it occurs I would be really grateful, I cant seem to find the link to the post anywhere and it seems like a problem that could occur for our company.
The problem is "parameter sniffing" (SO Search)
The pattern with #newParameter is called "parameter masking" (also SO Search)
You could always use the this masking pattern but it isn't always needed. For example, a simple select by unique key, with no child tables or other filters should behave as expected every time.
Since SQL Server 2008, you can also use the OPTIMISE FOR UNKNOWN (SO). Also see Alternative to using local variables in a where clause and Experience with when to use OPTIMIZE FOR UNKNOWN