the copy into command returns an output dataset.
CTAS can create a table from the results of a query.
combining the two, we would expect to get the list of loaded files into a new table.
CREATE TABLE MY_LOADED_FILES
AS
COPY INTO mytable
FROM #my_int_stage;
However, this returns:
SQL compilation error: syntax error line 3 at position 0 unexpected 'copy'.
What am I doing wrong?
It doesn't look like you can put a COPY INTO statement inside another statement unfortunately. There is a way to do this however by using the result_scan function to return the results of the previous query.
copy into test_database.public.test_table from #my_int_stage;
create temporary table test_database.public.test_table_results as (
select * from table(result_scan(LAST_QUERY_ID()))
);
Of course you need to make sure the second query runs in the same session as the copy statement and also that it is run directly after the copy statement. Alternatively you can use the query id with the result_scan.
If you want to see which files were loaded why don't you just look at the copy_history of the table?
I am trying to create an SSIS package for Stored Procedure Deployment and Backup for our project.
I have some .sql file, each file contains one stored procedure definition and the name of the file is the stored procedure name itself.
I am trying to do the following by using SSIS
Read all files names one by one
Find the definition of each stored procedure if it exists in the database
If exists, then save the definition with the same name in a different folder (In my case it's a ROLLBACK folder)
For all new SP it save in a same file named DropNewSp.sql with DROP STORED PROCEDURE command.
After completing the backup process, execute all files in the destination database.
I am able to generate the desire .sql files, but I am faceing the following problem
1. The package also generated unwanted blank .sql file for all new Stored Procedure
2. The execution process failed if the stored procedure has some dependency on subsequent stored proc
In this answer, I will provide the main steps with some references to get more information on how to achieve each step. Even if I agree with the comments mentioned above that this is not the job of SSIS.
Add a foreach loop container that loop over .sql files and store the file name inside a variable:
Looping over files with the Foreach Loop
Lesson 2-2: Add and configure the Foreach Loop container
Load multiple source files
Add an Expression Task to retrieve the file name from the File Full Path (variable)
#{User::FileNameWithoutExtension] = SUBSTRING (#[User::FullFilePath], LEN( #[User::FullFilePath] ) - FINDSTRING( REVERSE( #[User::FullFilePath] ), "\\", 1) + 2, LEN (RIGHT( #[User::FullFilePath], FINDSTRING( REVERSE( #[User::FullFilePath] ), "\\", 1 ) - 1 ) ) - FINDSTRING( REVERSE( #[User::FullFilePath] ), ".", 1 ) )
Add an Execute SQL Task inside the foreach loop container to check if the stored procedure is found in the database:
SELECT COUNT(*) FROM sys.objects WHERE type = 'P' AND name = ?)
Pass the procedure name as parameter to the execute sql task:
Passing Variables to and from an SSIS task
Store the count result inside a variable of type integer using Resultsets:
SSIS Basics: Using the Execute SQL Task to Generate Result Sets
Using precedence constraints with expressions add 2 paths from the execute sql task
the first using expression #[User::Count] == 0
the second using expression #[User::Count] > 0
Other references:
Working with Precedence Constraints in SQL Server Integration Services
Defining Workflow in SSIS using Precedence Constraints
On the second path add an Execute SQL Task to get the procedure definition using the same approach above:
SELECT OBJECT_DEFINITION (OBJECT_ID(N'<databasename>.<schemaname>.' + CAST(? as VARCHAR(100))));
And store the result inside a variable using a result set.
Add a Script Task to write the procedure definition into the destination file
On the first path add a File system task to move the file into the directory specified
Add another foreach loop to read new files and execute the content.
I have a table that contains just under a million rows. I'm building a form using SSIS that asks for user input and uses the values as parameters to build a view from source data. I'm having trouble getting SSIS to create the view from a variable.
The purpose of this 'tool' is to provide a dialogue that programmatically builds a view and later an update statement based upon parameters defined via a form that will execute an SSIS package. A number of the ppl on my team know 0 SQL. Therefore this circumvents any SQL knowledge. Creating an entirely standalone app is not ideal as it would require too much additional overhead on my side and would deviate from a number of our existing processes that currently use SSIS/SQL to achieve similar results.
With that here is what I've tried/trying.
I have an SSIS package that contains 'Execute SQL Task'
This task brings up a form with 5 inputs (variables)
var1,var2,var3,var4,var5.
some vars are strings others are doubles, ints etc... (they all vary)
You populate the fields and hit okay.
These variables are passed to an 'Execute Package Task'.
Inside this package (Package B)
the vars are used in an 'Execute SQL Task'.
This task is attempting to take the users input and create a view with a where clause containing 4 other variables.
example:
Create View ? AS Select col1,col2,col3,col4 WHERE
col1 = ?
AND col2 =?
AND col3 =?.........
First it appears that using ? in the create view is invalid.
The error being:
Error: 0x0 at Build_Query: Incorrect syntax near '#P1'.
Error: 0xC002F210 at Build_Query, Execute SQL Task: Executing the query "CREATE VIEW ? as Select * from S_t_equip_template
..." failed with the following error: "The batch could not be analyzed because of compile errors.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established
correctly.
Task failed: Build_Query
If I use the create view variable as an expression and remove the variables/paramerters for the where statements, I can create the view no problem.
However the where statements throw errors once I add them back in. I've tried evaluating these as an expression in the 'Execute SQL Task' but as these are of various types I get the error:
[Execute SQL Task] Error: Executing the query "
CREATE VIEW testing AS SELECT
P.label,P.uniq..." failed with the following error: "The metadata could not
be determined because statement 'CREATE VIEW testingagain AS SELECT
P.label,P.uniqueid,C.label as Child_Label,C.uniqueid as Child_uni' does not
support metadata discovery.". Possible failure reasons: Problems with the
query, "ResultSet" property not set correctly, parameters not set correctly,
or connection not established correctly.
No idea what is going on. Any help would be appreciated
I've googled the error and found some info but the other use-cases are so different that it's difficult to understand the actual cause, or another work around.
AS requested (simplified example):
I've created a package variable (datatype string) called: View_Name
Execute SQL Task:
CREATE VIEW #[User::View_Name] AS SELECT
* from table1
where col1 = 100;
Specifically it does not like that I use a variable here.
If I set the View name everything works until: I move on to my Where clauses that contain variables.
Create a variable called type (datatype int)
I map the variable/parameter in my sql task
Example:
CREATE VIEW tempTable AS SELECT
* from table1
where col1 = ?;
This won't work, same error.
If i attempt to do the above via an expression or expressions I get the following error
[Execute SQL Task] Error: Executing the query "CREATE VIEW test_45678 AS SELECT P.label,P.uniquei..." failed with the following error: "Must declare the scalar variable "#".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Generally having to due with the variables cannot be evaluated this way. I'm guessing I'd need to evaluate each piece individually and build the expression piece by piece. That's fine but very inefficient and not maintainable.
I had some fundamental misunderstanding in my attempt to evaluate my expressions. Specifically syntax issues and having to cast each variable to a string.
My SQLstatement variable
ON C.pid = P.ID
where
C.width >="+(DT_WSTR, 8)#[User::Width] +"-"+ (DT_WSTR, 8)#[User::Range]+........
The final expression looks like so:
"Create View "+#[User::View_Name] + " AS SELECT " + #[User::SQLStatement]
I'm trying to do something like:
"If it exists, use it. If not, create it."
"If it exists, delete it. If not, create it."
One place it's definitely choking is the use it command - because if it DOES NOT EXIST - it chokes on the use command EVEN THOUGH that command will not run.
Here's more explanation:
I have a SQL Server script where I create a database and then I use the database.
The script will not run
because the use database command is invalid
because the database does not exist
but it will exist after the first command executes
but it doesn't matter because it doesn't exist NOW so the script will not run.
How do I put code in there that tries to use a database that might not exist?
How do I put code in there that will cause an error if run directly but WILL NOT RUN unless conditions are appropriate.
Please see the attached images.
Here's the code so you don't have to type it...
-- SQL SERVER: We can't run this script because CFPT does not exist.
-- ME: But it WILL exist after the first command runs
-- SQL SERVER: That does not matter - at THIS point in the code... it does not exist... tough luck
-- CREATE THE DATABASE
create database CFPT
-- USE THE DATABASE
USE CFPT
use master
drop database CFPT
Second code snippet:
-- SQL SERVER: We can't run this script because CFPT does not exist.
select db_id('CFPT') -- this just lets us see what the IF statement is going to have to deal with
IF db_id('CFPT') is null
begin
print 'DESIRED DB DOES NOT EXIST'
return
end
else
begin
use CFPT -- this line of code makes the whole script just not run.
end;
-- doesn't want to work - chokes on the use databasename (when the database does not exist)
(EDIT 1 start ////////////////////////////////////////////////////////////////////////////////////)
A third image was added with this edit - The SECOND image shows that the if/then/else statement will not work. The 3rd image shows that the database CFPT is not in the database list (left side of image) and the select statement was run (top highlighed code) and the results of that select (bottom red circle)
How do I get the if/then/else statement to work? (Because the THEN will not run if the conditions are not favorable shall-we-say)
(for some reason the red wavy lines are not showing up - they should be but they aren't - hmmm)
(EDIT 1 end ////////////////////////////////////////////////////////////////////////////////////)
(EDIT 2 start ////////////////////////////////////////////////////////////////////////////////////)
In relation to this question - trying to segregate commands that would normally fail but will not be attempted to be executed unless conditions are just right..... (see 4th image below) I'm segregating some commands with an IF statement (IF 1=2) but SQL Server is going into that IF statement even though the condition is false. Why is that?
(EDIT 2 end ////////////////////////////////////////////////////////////////////////////////////)
Try this ...
-- CREATE THE DATABASE
create database CFPT
GO
-- USE THE DATABASE
USE CFPT
use master
drop database CFPT
The GO command is a batch terminator, it separates the command to create the database from the command to use it.
See https://msdn.microsoft.com/en-us/library/ms188037.aspx
and
What is the use of GO in SQL Server Management Studio & Transact SQL?
I have a database and have a Sql script to add some fields to a table called "Products" in the database.
But when i am executing this script, I am getting the following error:
Cannot find the object "Products" because it does not exist or you do not have permissions
Why is the error occurring and what should I do to resolve it?
I found a reason why this would happen. The user had the appropriate permissions, but the stored procedure included a TRUNCATE statement:
TRUNCATE TableName
Since TRUNCATE deletes items without logging, you (apparently) need elevated permissions to execute a stored procedure that contains it. We changed the statement to:
DELETE FROM TableName
...and the error went away!
Are you sure that you are executing the script against the correct database? In SQL Server Management studio you can change the database you are running the query against in a drop-down box on one of the toolbars, or you can start your query with this:
USE SomeDatabase
It can also happen due to a typo in referencing a table such as [dbo.Product] instead of [dbo].[Product].
Does the user you're executing this script under even see that table??
select top 1 * from products
Do you get any output for this??
If yes: does this user have the permission to modify the table, i.e. execute DDL scripts like ALTER TABLE etc.? Typically, regular users don't have this elevated permissions.
Look for any DDL operation in the script.
Maybe the user does not have access rights to run changes.
In my case it was SET IDENTITY_INSERT tblTableName ON
You can either add db_ddladmin for the whole database or for just the table to solve this issue (or change the script)
-- give the non-ddladmin user INSERT/SELECT as well as ALTER:
GRANT ALTER, INSERT, SELECT ON dbo.tblTableName TO user_name;
It could also be possible that you have created the "Products" in your login schema and you were trying to execute the same in a different schema (probably dbo)
Steps to resolve this issue
1)open the management studio
2) Locate the object in the explorer and identify the schema under which your object is? ( it is the text before your object name ). In the image below its the "dbo" and my object name is action status
if you see it like "yourcompanydoamin\yourloginid" then you should
you can modify the permission on that specific schema and not any other schema.
you may refer to "Ownership and User-Schema Separation in SQL Server"
I've been trying to copy a table from PROD to DEV but get an error:
"Cannot find the object X because it does not exist or you do not have permissions."
However, the table did exist, and I was running as sa so I did have permissions.
The problem was actually with CONTRAINTS. I'd renamed the table on DEV to be old_XXX months ago. But when I tried to copy the original one over from PROD, the Defaut Constraint names clashed.
The error message was misleading
You can right click the procedure, choose properties and see which permissions are granted to your login ID. You can then manually check off the "Execute" and alter permission for the proc.
Or to script this it would be:
GRANT EXECUTE ON OBJECT::dbo.[PROCNAME]
TO [ServerInstance\user];
GRANT ALTER ON OBJECT::dbo.[PROCNAME]
TO [ServerInstance\user];
This could be a permission issue. The user needs at least ALTER permission to truncate a table.
Another option is to call DELETE FROM instead of TRUNCATE TABLE, but this operation is slower because it writes to the Log file, whereas TRUNCATE does not write to the log file.
The minimum permission required is ALTER on table_name. TRUNCATE TABLE
permissions default to the table owner, members of the sysadmin fixed
server role, and the db_owner and db_ddladmin fixed database roles,
and are not transferable. However, you can incorporate the TRUNCATE
TABLE statement within a module, such as a stored procedure, and grant
appropriate permissions to the module using the EXECUTE AS clause.
Sharing my case, hope that will help.
In my situation inside MY_PROJ.Database->MY_PROJ.Database.sqlproj I had to put this:
<Build Include="dbo\Tables\MyTableGeneratingScript.sql" />
In my case I was running under a different user than the one I was expecting.
My code passed 'DRIVER={SQL Server};SERVER=...;DATABASE=...;Trusted_Connection=false;User Id=XXX;Password=YYY' as the connection string to pypyodbc.connect(), but it ended up connecting with the credentials of the Windows user that ran the script instead of the User Id= from the connection string.
(I verified this using the SQL Server Profiler and by putting an invalid uid/password combination in the connection string - which didn't result in an expected error).
I decided not to dig into this further, since switching to this better way of connecting fixed the issue:
conn = pypyodbc.connect(driver='{SQL Server}', server='servername',
database='dbname', uid='userName', pwd='Password')
In my case the sql server version on my localhost is higher than that on the production server and hence some new variables were added to the generated script from the localhost. This caused errors in creating the table in the first place.
Since the creation of the table failed, subsequent query on the "NON EXISITING" table also failed.
Luckily, in among the long list of the sql errors, I found this "OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF" to be the new varialbe in the script causing my issue. I did a search and replace and the error went away.
Hope it helps someone.
The TRUNCATE statement was my first problem, glad to find the solution here. But I was using SSIS and trying to load data from another database, and it failed with the same error on any table that used IDENTITY to create an auto-incrementing ID. If I was scripting it myself I'd first need to use the command SET IDENTITY_INSERT tablename ON, and then SET IDENTITY_INSERT tablename OFF when the table update was done. But this requires ALTER permissions on the table, which I do not have. Hence the error message in SSIS on the table load (even though the previous step had just deleted all the data out of the table.)
You receive this error, when you use an ORM like GORM (https://gorm.io/) in Go for example.
When you try to create a struct and accidentally pass the ID (primary key) although it's inserted automatically.
Rich features IDE like Visual Studio Code make this mistake happen easily:
if tx := db.Create(&myStruct{
Ts: Time.Now(),
ID: 42,
}); tx.Error != nil {
t.Fatal(tx.Error)
}
You can still use auto-filling by Visual Studio Code, but delete your entry for your model's primary keys:
if tx := db.Create(&myStruct{
Ts: Time.Now(),
}); tx.Error != nil {
t.Fatal(tx.Error)
}