sp_rename failing when called from inside another stored procedure - sql-server

Question

First post from a self-taught data warehouse guy. I've done lots of searching and reading to get where I am now, but can't get past this sticking point.
Background: as part of our nightly ETL job, we have to copy many tables from many remote DBs (linked servers) into staging-area DBs. After table copies have finished, I continue with the transformation from the staging area DBs into production tables.
Since the remote DBs all have identical schema, I made a stored procedure in the production DB to do the work. The stored procedure accepts parameters of the remote database name and the table name. In the nightly job, SQL Server Agent runs an SSIS package; the package contains one (retry-looping) SSIS task for each remote database; all the tasks run concurrently; each task uses a variable to pass the DB name to SQL file; then the SQL file calls the stored procedure once for each table.
Example remote table and local staging-area table:
Remote: [FLTA].[cstone].[csdbo].[CLIENT]
Local: [FLTAL].dbo.[FLTA CLIENT]
The stored procedure is pretty simple, dropping the old table and using SELECT to make a fresh copy from the remote DB. It looks approximately like this:
CREATE PROCEDURE dbo.spTableCopyNew
(#p VARCHAR(50), #Tablename VARCHAR(50))
AS
-- Drop the existing table
EXEC('IF OBJECT_ID(''[' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +']'', ''U'') IS NOT NULL
DROP TABLE [' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +']'
)
-- Copy the new table
EXEC('SELECT * into [' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +']
FROM [' + #p + '].[cstone].[csdbo].[' + #Tablename +']'
)
GO
The SQL looks roughly like this:
-- Set local variables for the remote server connection, the local database name, and the table prefix
DECLARE #Prefix varchar(50)
-- Accept the variables passed in from the SSIS task
SET #Prefix = ?
-- Copy the two tables
EXEC Datawarehouse.dbo.spTableCopy #Prefix, 'CLIENT'
EXEC Datawarehouse.dbo.spTableCopy #Prefix, 'PATIENT'
Maintenance is a breeze: when we need to grab a new table from all the remote databases, I just add it to the "productionLoad.sql" file.
It works really well...except when it doesn't.
Due to un-figured-out-yet reasons, sometimes a table fails to copy. And since I'm dropping the existing table before copying the new one, this will sometimes break things further down the line. My SSIS tasks will retry up to three times per remote DB, so occasional failures are no big deal. But if the same remote DB has three failures in one night, I'm gonna have a bad time.
My current attempt at a solution is to copy the remote table to a temp table, then ONLY AFTER that copy is successful, drop the local table and rename the temp table to the "real" table. Which brings me to the problem:
I can't get sp_rename to work when called from a stored procedure, to rename tables that exist in a different database than the stored procedure. I've created new variables to resolve expressions, then send those variables to sp_rename, since I can't pass expressions into that stored procedure.
Here's my attempt at a new stored procedure:
CREATE PROCEDURE dbo.spTableCopy
(#p VARCHAR(50), #Tablename VARCHAR(50))
AS
BEGIN
EXEC('USE [' + #p + 'L]')
-- Create variables for schema and table names
-- Since sp_rename can accept variables, but not expresssions containing variables.
DECLARE #RemoteTable VARCHAR(50) = '[' + #p + '].[cstone].[csdbo].[' + #Tablename +']'
DECLARE #LocalTableTemp VARCHAR(50) = '[' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +'_temp]'
DECLARE #LocalTable VARCHAR(50) = '' + #p + ' ' + #Tablename + ''
-- Check for previous temp table and drop it
EXEC('IF OBJECT_ID(''[' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +'_temp]'', ''U'') IS NOT NULL
DROP TABLE [' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +'_temp]'
)
-- Copy the new table
EXEC('SELECT * into ' + #LocalTableTemp + '
FROM ' + #RemoteTable + ''
)
-- Drop the existing table
EXEC('IF OBJECT_ID(''[' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +']'', ''U'') IS NOT NULL
DROP TABLE [' + #p + 'L].dbo.[' + #p + ' ' + #Tablename +']'
)
-- Rename temp table to real table
EXEC sp_rename #LocalTableTemp, #LocalTable
END
GO
This all works when executing it as normal SQL code, but when I make it into a stored procedure, sp_rename fails (everything else works). The final table [FLTAL CLIENT_temp] is there and contains the right data.
sp_rename returns the following error:
Msg 290, Level 16, State 2, Procedure sp_rename, Line 318
Invalid EXECUTE statement using object "Object", method "LockMatchID".
I've fought with this way too long.
Am I just screwing up the syntax?
Can I get sp_rename to work on other DBs with "USE?"
If not, will it work if I make a copy of my sp_tableCopy in every staging-area DB?
If not, will catch-try work inside this stored procedure, even if I call this stored procedure many times concurrently?
What else can I do to recover from failed table copies?
My alternate solution that I haven't pursued yet: after the temp table is successfully created, to TRUNC the existing table and insert everything from the temp table into the real table. That seems messy though.
P.S. Our IT guys are "looking into" the nature of the copy failures.

Answer 1

Try This...
USE
EXEC ..sp_rename '..', '<target_table>'

Answer 2

USE sourcedb
EXEC targetdatabase..sp_rename 'schema.oldtable', 'target_table'