INSERT INTO results_table EXEC sproc from within a scheduled job - sql-server

I'm trying to setup a scheduled job that with one step that would insert the results from a sproc into a table.
INSERT INTO results_table EXEC sproc
The job executes and reports a success. Yet nothing gets inserted into a table. When I execute the same script from the SSMS the results are inserted. What might cause the problem?
** EDIT the job is owned by sa and the step is executed as dbo. All the runs in the history are reported as finished successfully. I've tried changing the step to
INSERT INTO results_table(field_names) (SELECT values FROM table GROUP BY column_name)
and it behaves in a similar way
** EDIT the problem only occurs when I select from the master database. Selecting from other tables works fine.

Check if you are inserting in the Master database or the the database that you want to insert. Or call the SP with database instanse inside the job step
Insert Into Results_Table
EXEC <DBNAME>.<SchemaName>.<ProcedureName>

Have you tried inserting the results of the stored procedure into a temp table first then inserting them into your results_table? I would suggest that as well as this article which reviews this concept in-depth: http://www.sommarskog.se/share_data.html

The problem was that in the scheduled job the stored procedure was executed not in the context of master database.

Related

SQL Server 2016 Create Trigger to start after New Table is finished updating

I have a table that is pushed to me from another SQL Server. The table is dropped after it is rotated to a "Current Day" table (Current Data is rotated to prev day before this).
Currently we have jobs that are running to do the "rotating" that are set at a specific time. I had originally created a trigger but clearly a trigger won't work (as I figured out from the comments) since the DDL operation wont continue its flow until after this trigger is complete... It also looks like this is just not possible since I don't have control over the group that is pushing the data to us.
Resolution : I went to the org that pushes the data and requested they add a step that inserts a record into a TableLog table and I am doing my trigger off of that insert instead.
CREATE TRIGGER InsertTest
ON [pace].[Table_Load_Log]
after insert
AS
if exists(select Table_name from inserted where inserted.Table_name = 'POE_Task_Details_SE_TEMP')
BEGIN
--drop table dbo.newtable
exec dbo.sp_start_job N'Make Pace Tables From Temp Table Push’
END
GO
There is no way to do this with a trigger. If you need to know when a CRUD operation on a table is complete, you would need to execute a command after the CRUD operation in the same process that launches it.

Statement execution in stored procedure in SQL Server

I have a stored procedure (simplified version) as below. In general it will push the pending items to email queue and update the status of those items to pushed. This stored procedure has been called in a 60 seconds interval.
ALTER PROCEDURE [dbo].[CreateEmailQueue]
AS
BEGIN
insert into EmailQueue
select *
from actionstatus
where actionstatus='Pending'
update actionstatus
set actionstatus = 'Pushed'
where actionstatus = 'Pending'
END
It works fine but I found one exception record. The exception's action status has been changed to "Pushed" but the record did not appear in the EmailQueue. I suspected that the action status record has been generated after the insert but before the update. But I am not 100% sure as I assume that SQL Server always execute a stored procedure in an isolation block so the data change won't affect the data set inside the execution. Could someone help explain the actual process inside the stored procedure, review this stored procedure and advise if it is OK, or it should be put in a transaction block? and, any performance or locking issue for doing this in transaction? Thanks for your input.
You must use transactins inside stored procedure. SQL Server don't generate transactions for stored procedures calls automatically. You can simply check it by execute code something like taht:
insert ...
--this will falls
select 1/0
Insertion will be done and than you'll have division by zero error

Adding insert into linked server inside a trigger

I'm trying to run a trigger that allows me to insert the inserted data on my local table to the linked server's table. This is what I did:
use [medb]
ALTER TRIGGER [dbo].[trigger1] ON [dbo].[tbl1]
AFTER INSERT
AS
BEGIN
INSERT into openquery(DEV, 'tbl_remotetbl') select * from inserted
END
but it is giving this error:
Cannot process the object "tbl_remotetbl". The OLE DB provider
"MSDASQL" for linked server "DEV" indicates that either the object has
no columns or the current user does not have permissions on that
object.
What seems to be my problem?
Note: I am using SQL Server 2008 R2
Did you try running the command outside the trigger ? Did it work ?
here is the syntax I'm using in my openquery:
INSERT INTO OPENQUERY(LinkedServer_Name,
'select remote_field_1,remote_field_2 from remote_Schema.remote_table')
select local_column1,local_column2
FROM local_Table
Now, with that being said, making this statement work inside a trigger, is something I couldn't do. Above statement woks perfectly when executed by it self. But once it is placed in a trigger, the entire transaction related to that trigger fails. I mean even the insert statement that fires the trigger does not go through, and the main table does not insert the data which was meant to be inserted in it.
i ran into the same issue, and spent many hours trying to figure out how to make an openquery statement work inside update/insert/delete triggers, with no success....
so, here's an alternate solution, maybe this can fix your issue, this is in a scenario where i need to pass data from MSSQL to a MySQL DB.
first, create a holding table, so you can store temporary info that's inserted into a table that will only hold data that i need to pass to MySQL
create table holding_table (ID int, value2 int)
trigger will insert data to the holding table, instead of sending it directly to MySQL
ALTER TRIGGER [dbo].[temp_data_to_mysql]
ON [dbo].[source_table]
FOR insert
AS
BEGIN
INSERT into holding_table (ID,Stock)select a, b from inserted
END
GO
after that, you can just create a task in the SQL server agent, so it can execute your stored procedure every N minutes.
hope it helps, im aware that this is a workaround, but after some investigation and testing, i was unable to make openquery work called within a trigger process..

Concurrency problems with temp tables in consequent batches?

I have sometimes a problem when running a script. I have the probelm when using an application (that I didn't write and therefore cannot debug) that launches the scripts. This app isn't returning the full error from SQL Server, but just the error description, so I don't know exactly where th error comes.
I have the error only using this tool (it is a tool that sends the queries directly to SQL Server, using a DAC component), if I run the query manuallyin management studio I don't have the error. (This error moreover occurs only on a particular database).
My query is something like:
SELECT * INTO #TEMP_TABLE
FROM ANOTHER_TABLE
GO
--some other commands here
GO
INSERT INTO SOME_OTHER_TABLE(FIELD1,FIELD2)
SELECT FIELDA, FIELDB
FROM #TEMP_TABLE
GO
DROP TABLE #TEMP_TABLE
GO
The error I get is #TEMP_TABLE is not a valid object
So somehow i suspect that the DROP statement is executed before the INSERT statement.
But AFAIK when a GO is there the next statement is not executed until the previous has been completed.
Now I suspoect that this is not true with temp tables... Or do you have another ideas?
Your problem is most likely caused by either an end of session prior to the DROP TABLE causing SQL Server to automatically drop the table or the DROP TABLE is being executed in a different session than the other code (that created and used the temporary table) causing the table not to be visible.
I am assuming that stored procedures are not involved here, because it looks like you are just executing batches, since local temporary tables are also dropped when a stored proc is exited.
There is a good description of local temporary table behavior in this article on Temporary Tables in SQL Server:
You get housekeeping with Local Temporary tables; they are
automatically dropped when they go out of scope, unless explicitly
dropped by using DROP TABLE. Their scope is more generous than a table
Variable so you don't have problems referencing them within batches or
in dynamic SQL. Local temporary tables are dropped automatically at
the end of the current session or procedure. Dropping it at the end of
the procedure that created it can cause head-scratching: a local
temporary table that is created within a stored procedure or session
is dropped when it is finished so it cannot be referenced by the
process that called the stored procedure that created the table. It
can, however, be referenced by any nested stored procedures executed
by the stored procedure that created the table. If the nested
procedure references a temporary table and two temporary tables with
the same name exist at that time, which table is the query is resolved
against?
I would start up SQL Profiler and verify if your tool uses one connection to execute all batches, or if it disconnects/reconnects. Also it could be using a connection pool.
Anyway, executing SQL batches from a file is so simple that you might develop your own tool very quickly and be better off.

Errors: "INSERT EXEC statement cannot be nested." and "Cannot use the ROLLBACK statement within an INSERT-EXEC statement." How to solve this?

I have three stored procedures Sp1, Sp2 and Sp3.
The first one (Sp1) will execute the second one (Sp2) and save returned data into #tempTB1 and the second one will execute the third one (Sp3) and save data into #tempTB2.
If I execute the Sp2 it will work and it will return me all my data from the Sp3, but the problem is in the Sp1, when I execute it it will display this error:
INSERT EXEC statement cannot be nested
I tried to change the place of execute Sp2 and it display me another error:
Cannot use the ROLLBACK statement
within an INSERT-EXEC statement.
This is a common issue when attempting to 'bubble' up data from a chain of stored procedures. A restriction in SQL Server is you can only have one INSERT-EXEC active at a time. I recommend looking at How to Share Data Between Stored Procedures which is a very thorough article on patterns to work around this type of problem.
For example a work around could be to turn Sp3 into a Table-valued function.
This is the only "simple" way to do this in SQL Server without some giant convoluted created function or executed sql string call, both of which are terrible solutions:
create a temp table
openrowset your stored procedure data into it
EXAMPLE:
INSERT INTO #YOUR_TEMP_TABLE
SELECT * FROM OPENROWSET ('SQLOLEDB','Server=(local);TRUSTED_CONNECTION=YES;','set fmtonly off EXEC [ServerName].dbo.[StoredProcedureName] 1,2,3')
Note: You MUST use 'set fmtonly off', AND you CANNOT add dynamic sql to this either inside the openrowset call, either for the string containing your stored procedure parameters or for the table name. Thats why you have to use a temp table rather than table variables, which would have been better, as it out performs temp table in most cases.
OK, encouraged by jimhark here is an example of the old single hash table approach: -
CREATE PROCEDURE SP3 as
BEGIN
SELECT 1, 'Data1'
UNION ALL
SELECT 2, 'Data2'
END
go
CREATE PROCEDURE SP2 as
BEGIN
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
INSERT INTO #tmp1
EXEC SP3
else
EXEC SP3
END
go
CREATE PROCEDURE SP1 as
BEGIN
EXEC SP2
END
GO
/*
--I want some data back from SP3
-- Just run the SP1
EXEC SP1
*/
/*
--I want some data back from SP3 into a table to do something useful
--Try run this - get an error - can't nest Execs
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
DROP TABLE #tmp1
CREATE TABLE #tmp1 (ID INT, Data VARCHAR(20))
INSERT INTO #tmp1
EXEC SP1
*/
/*
--I want some data back from SP3 into a table to do something useful
--However, if we run this single hash temp table it is in scope anyway so
--no need for the exec insert
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
DROP TABLE #tmp1
CREATE TABLE #tmp1 (ID INT, Data VARCHAR(20))
EXEC SP1
SELECT * FROM #tmp1
*/
My work around for this problem has always been to use the principle that single hash temp tables are in scope to any called procs. So, I have an option switch in the proc parameters (default set to off). If this is switched on, the called proc will insert the results into the temp table created in the calling proc. I think in the past I have taken it a step further and put some code in the called proc to check if the single hash table exists in scope, if it does then insert the code, otherwise return the result set. Seems to work well - best way of passing large data sets between procs.
This trick works for me.
You don't have this problem on remote server, because on remote server, the last insert command waits for the result of previous command to execute. It's not the case on same server.
Profit that situation for a workaround.
If you have the right permission to create a Linked Server, do it.
Create the same server as linked server.
in SSMS, log into your server
go to "Server Object
Right Click on "Linked Servers", then "New Linked Server"
on the dialog, give any name of your linked server : eg: THISSERVER
server type is "Other data source"
Provider : Microsoft OLE DB Provider for SQL server
Data source: your IP, it can be also just a dot (.), because it's localhost
Go to the tab "Security" and choose the 3rd one "Be made using the login's current security context"
You can edit the server options (3rd tab) if you want
Press OK, your linked server is created
now your Sql command in the SP1 is
insert into #myTempTable
exec THISSERVER.MY_DATABASE_NAME.MY_SCHEMA.SP2
Believe me, it works even you have dynamic insert in SP2
I found a work around is to convert one of the prods into a table valued function. I realize that is not always possible, and introduces its own limitations. However, I have been able to always find at least one of the procedures a good candidate for this. I like this solution, because it doesn't introduce any "hacks" to the solution.
I encountered this issue when trying to import the results of a Stored Proc into a temp table, and that Stored Proc inserted into a temp table as part of its own operation. The issue being that SQL Server does not allow the same process to write to two different temp tables at the same time.
The accepted OPENROWSET answer works fine, but I needed to avoid using any Dynamic SQL or an external OLE provider in my process, so I went a different route.
One easy workaround I found was to change the temporary table in my stored procedure to a table variable. It works exactly the same as it did with a temp table, but no longer conflicts with my other temp table insert.
Just to head off the comment I know that a few of you are about to write, warning me off Table Variables as performance killers... All I can say to you is that in 2020 it pays dividends not to be afraid of Table Variables. If this was 2008 and my Database was hosted on a server with 16GB RAM and running off 5400RPM HDDs, I might agree with you. But it's 2020 and I have an SSD array as my primary storage and hundreds of gigs of RAM. I could load my entire company's database to a table variable and still have plenty of RAM to spare.
Table Variables are back on the menu!
I recommend to read this entire article. Below is the most relevant section of that article that addresses your question:
Rollback and Error Handling is Difficult
In my articles on Error and Transaction Handling in SQL Server, I suggest that you should always have an error handler like
BEGIN CATCH
IF ##trancount > 0 ROLLBACK TRANSACTION
EXEC error_handler_sp
RETURN 55555
END CATCH
The idea is that even if you do not start a transaction in the procedure, you should always include a ROLLBACK, because if you were not able to fulfil your contract, the transaction is not valid.
Unfortunately, this does not work well with INSERT-EXEC. If the called procedure executes a ROLLBACK statement, this happens:
Msg 3915, Level 16, State 0, Procedure SalesByStore, Line 9 Cannot use the ROLLBACK statement within an INSERT-EXEC statement.
The execution of the stored procedure is aborted. If there is no CATCH handler anywhere, the entire batch is aborted, and the transaction is rolled back. If the INSERT-EXEC is inside TRY-CATCH, that CATCH handler will fire, but the transaction is doomed, that is, you must roll it back. The net effect is that the rollback is achieved as requested, but the original error message that triggered the rollback is lost. That may seem like a small thing, but it makes troubleshooting much more difficult, because when you see this error, all you know is that something went wrong, but you don't know what.
I had the same issue and concern over duplicate code in two or more sprocs. I ended up adding an additional attribute for "mode". This allowed common code to exist inside one sproc and the mode directed flow and result set of the sproc.
what about just store the output to the static table ? Like
-- SubProcedure: subProcedureName
---------------------------------
-- Save the value
DELETE lastValue_subProcedureName
INSERT INTO lastValue_subProcedureName (Value)
SELECT #Value
-- Return the value
SELECT #Value
-- Procedure
--------------------------------------------
-- get last value of subProcedureName
SELECT Value FROM lastValue_subProcedureName
its not ideal, but its so simple and you don't need to rewrite everything.
UPDATE:
the previous solution does not work well with parallel queries (async and multiuser accessing) therefore now Iam using temp tables
-- A local temporary table created in a stored procedure is dropped automatically when the stored procedure is finished.
-- The table can be referenced by any nested stored procedures executed by the stored procedure that created the table.
-- The table cannot be referenced by the process that called the stored procedure that created the table.
IF OBJECT_ID('tempdb..#lastValue_spGetData') IS NULL
CREATE TABLE #lastValue_spGetData (Value INT)
-- trigger stored procedure with special silent parameter
EXEC dbo.spGetData 1 --silent mode parameter
nested spGetData stored procedure content
-- Save the output if temporary table exists.
IF OBJECT_ID('tempdb..#lastValue_spGetData') IS NOT NULL
BEGIN
DELETE #lastValue_spGetData
INSERT INTO #lastValue_spGetData(Value)
SELECT Col1 FROM dbo.Table1
END
-- stored procedure return
IF #silentMode = 0
SELECT Col1 FROM dbo.Table1
Declare an output cursor variable to the inner sp :
#c CURSOR VARYING OUTPUT
Then declare a cursor c to the select you want to return.
Then open the cursor.
Then set the reference:
DECLARE c CURSOR LOCAL FAST_FORWARD READ_ONLY FOR
SELECT ...
OPEN c
SET #c = c
DO NOT close or reallocate.
Now call the inner sp from the outer one supplying a cursor parameter like:
exec sp_abc a,b,c,, #cOUT OUTPUT
Once the inner sp executes, your #cOUT is ready to fetch. Loop and then close and deallocate.
If you are able to use other associated technologies such as C#, I suggest using the built in SQL command with Transaction parameter.
var sqlCommand = new SqlCommand(commandText, null, transaction);
I've created a simple Console App that demonstrates this ability which can be found here:
https://github.com/hecked12/SQL-Transaction-Using-C-Sharp
In short, C# allows you to overcome this limitation where you can inspect the output of each stored procedure and use that output however you like, for example you can feed it to another stored procedure. If the output is ok, you can commit the transaction, otherwise, you can revert the changes using rollback.
On SQL Server 2008 R2, I had a mismatch in table columns that caused the Rollback error. It went away when I fixed my sqlcmd table variable populated by the insert-exec statement to match that returned by the stored proc. It was missing org_code. In a windows cmd file, it loads result of stored procedure and selects it.
set SQLTXT= declare #resets as table (org_id nvarchar(9), org_code char(4), ^
tin(char9), old_strt_dt char(10), strt_dt char(10)); ^
insert #resets exec rsp_reset; ^
select * from #resets;
sqlcmd -U user -P pass -d database -S server -Q "%SQLTXT%" -o "OrgReport.txt"

Resources