The application, I have been currently working with has different schema names for its tables, for example Table1 can have multiple existence say A.Table1 and B.Table1. All my stored procedures are stored under dbo. I'm writing the below stored procedures using dynamic SQL. I'm currently using SQL Server 2008 R2 and soon it will be migrated to SQL Server 2012.
create procedure dbo.usp_GetDataFromTable1
#schemaname varchar(100),
#userid bigint
as
begin
declare #sql nvarchar(4000)
set #sql='select a.EmailID from '+#schemaname+'.Table1 a where a.ID=#user_id';
exec sp_executesql #sql, N'#user_id bigint', #user_id=#userid
end
Now my questions are,
1. Is this type of approach affects the performance of my stored procedure?
2. If performance is affected, then how to write procedures for this kind of scenario?
The best way around this would be a redesign, if at all possible.
You can even implement this retrospectively by adding a new column to replace the schema, for example: Profile, then merge all tables from each schema into one in a single schema (e.g. dbo).
Then your procedure would appear as follows:
create procedure dbo.usp_GetDataFromTable1
#profile int,
#userid bigint
as
begin
select a.EmailID from dbo.Table1 a
where a.ID = #user_id
and a.Profile = #profile
end
I have used an int for the profile column, but if you use a varchar you could even keep your schema name for the profile value, if that helps to make things clearer.
I would look at a provisioning approach, where you dynamically create the tables and stored procedures as part of some up-front process. I'm not 100% sure of your scenario, but perhaps this could be when you add a new user. Then, you can call these SP's by convention in the application.
For example, new user creation calls an SP which creates c.Table and c.GetDetails SP.
then in the app you can call c.GetDetails based on "c" being a property of the user definition.
This gets you around any security concerns from using dynamic SQL. It's still dynamic, but is built once up front.
Dynamic schema and same table structure is quite unusual, but you can still obtain what you want using something like this:
declare #sql nvarchar(4000)
declare #schemaName VARCHAR(20) = 'schema'
declare #tableName VARCHAR(20) = 'Table'
-- this will fail, as the whole string will be 'quoted' within [..]
-- declare #tableName VARCHAR(20) = 'Category; DELETE FROM TABLE x;'
set #sql='select * from ' + QUOTENAME(#schemaName) + '.' + QUOTENAME(#tableName)
PRINT #sql
-- #user_id is not used here, but it can if the query needs it
exec sp_executesql #sql, N'#user_id bigint', #user_id=0
So, QUOTENAME should keep on the safe side regarding SQL injection.
1. Performance - dynamic SQL cannot benefit from some performance improvements (I think procedure associated statistics or something similar), so there is a performance risk.
However, for simple things that run on rather small amount of data (tens of millions at most) and for data that is not heavily changes (inserts and deletes), I don't think you will have noticeable problems.
2. Alternative -bukko has suggested a solution. Since all tables have the same structure, they can be merged. If it becomes huge, good indexing and partitioning should be able to reduce query execution times.
There is a work around for this if you know what schemas you are going to be using. You stated here that schema name is created on signup, we use this approach on login. I have a view which I add or remove unions from on session startup/dispose. Example below.
CREATE VIEW [engine].[vw_Preferences]
AS
SELECT TOP (0) CAST (NULL AS NVARCHAR (255)) AS SessionID,
CAST (NULL AS UNIQUEIDENTIFIER) AS [PreferenceGUID],
CAST (NULL AS NVARCHAR (MAX)) AS [Value]
UNION ALL SELECT 'ZZZ_7756404F411B46138371B45FB3EA6ADB', * FROM ZZZ_7756404F411B46138371B45FB3EA6ADB.Preferences
UNION ALL SELECT 'ZZZ_CE67D221C4634DC39664975494DB53B2', * FROM ZZZ_CE67D221C4634DC39664975494DB53B2.Preferences
UNION ALL SELECT 'ZZZ_5D6FB09228D941AC9ECD6C7AC47F6779', * FROM ZZZ_5D6FB09228D941AC9ECD6C7AC47F6779.Preferences
UNION ALL SELECT 'ZZZ_5F76B619894243EB919B87A1E4408D0C', * FROM ZZZ_5F76B619894243EB919B87A1E4408D0C.Preferences
UNION ALL SELECT 'ZZZ_A7C5ED1CFBC843E9AD72281702FCC2B4', * FROM ZZZ_A7C5ED1CFBC843E9AD72281702FCC2B4.Preferences
The first select top 0 row is a fall back so I always have a default definition, and a static table definition. You can select from the view and filter by a session id with
SELECT PreferenceGUID, Value
FROM engine.vw_Preferences
WHERE SessionID = 'ZZZ_5D6FB09228D941AC9ECD6C7AC47F6779';
The interesting part here though is how the execution plan is generated when you have static values inside a view. the unions that would not produce results are not evaluated by the code, leaving a basic execution plan without any joins or unions...
You can test this, it and it is just as efficient as reading directly from the table (to within a margin of error so minor nobody would care). It is even possible to replace the write back processes by using "instead" triggers and then building dynamic sql in the background. The dynamic sql is less efficient on writes but it means you can update any table via the view, usually only possible with a single table view.
Dynamic Sql usually effects both performance and security, most of the times for the worst. However, since you can't parameterize identifiers, this is probably the only way for you unless you are willing to duplicate your stored procedures for each schema:
create procedure dbo.usp_GetDataFromTable1
#schemaname varchar(100),
#userid bigint
as
begin
if #schemaname = 'a'
begin
select EmailID from a.Table1 where ID = #user_id
end
else if schemaname = 'b'
begin
select EmailID from b.Table1 where ID = #user_id
end
end
The only reason I can think of for doing this is satisfying multiple tenants. You're close but the approach you are taking is wrong.
There are 3 solutions for multi-tenancy which I'm aware of: Database per tenant, single database schema per tenant, or single database single schema (aka, tenant by row).
Two of these have already been mentioned by other users here. The one that hasn't really been detailed is schema per tenant which is what it looks like you fall under. For this approach you need to change the way you see the database. The database at this point is just a container for schemas. Each schema can have their own design, stored procs, triggers, queues, functions, etc. The main goal is data isolation. You don't want tenant A seeing tenant Bs stuff. The advantage of the schema per tenant approach is you can be more flexible with tenant specific database changes. It also allows you to scale easier than a database per tenant approach.
Answer: Instead of writing dynamic SQL to take into account the schema using the DBO user you should instead create the same stored proc for each schema (create procedure example: schema_name.stored_proc_name). In order to run the stored proc for a schema you'll need to impersonate a user that is tied to the schema in question. It would look something like this:
execute as user = 'tenantA'
exec sp_testing
revert --revert will take us back to the original user, most likely DBO in your case.
Data collation across all tenants is a little harder. The only solution that I'm aware of is to run using the DBO user and "union all" the results across all schemas separately, kind of tedious if you have a ton of schemas.
Related
My desired end result is to simply be able to SELECT from a Stored Procedure. I've searched the Internet and unfortunately the Internet said this can't be done and that you first need to create a Temp Table to store the data. My problem is that you must first define the columns in the Temp Table before Executing the STORED Procedure. This is just time consuming. I simply want to take the data from the stored procedure and just stick it into a Temp Table.
What is the FASTEST route to achieve this from a coding perspective? To put it simply it's time consuming to first have to lookup the returned fields from a Stored Procedure and then write them all out.
Is there some sort of tool that can just build the CREATE Table Statement based on the Stored Procedure? See screenshot for clarification.
Most of the Stored Procedures I'm dealing with have 50+ fields. I don't look forward to defining each of these fields manually.
Here is good SO Post that got me this far but not what I was hoping. This still takes too much time. What are experienced SQL Server guys doing? I've only just recently made the jump from Oracle to SQL Server and I see that Temp Tables are a big deal in SQL Server from what I can tell.
You have several options to ease your task. However, these won't be fully automatic. Be aware that these won't work if there's dynamic sql in the procedure's code. You might be able to format the result from the functions to increase the automation allowing you to copy and paste easily.
SELECT * FROM sys.dm_exec_describe_first_result_set_for_object(OBJECT_ID('report.MyStoredProcedureWithAnyColumns'), 0) ;
SELECT * FROM sys.dm_exec_describe_first_result_set(N'EXEC report.MyStoredProcedureWithAnyColumns', null, 0) ;
EXEC sp_describe_first_result_set #tsql = N'EXEC report.MyStoredProcedureWithAnyColumns';
GO
If you don't mind ##temp table and some dynamic SQL
NOTE: As Luis Cazares correctly pointed out... the ##temp runs the risk of collision due to concurrency concerns
Example
Declare #SQL varchar(max) = 'Exec [dbo].[prc-App-Lottery-Search] ''8117'''
Declare #temp varchar(500) = '##myTempTable'
Set #SQL = '
If Object_ID(''tempdb..'+#temp+''') Is Not NULL Drop Table '+#temp+';
Create Table '+#temp+' ('+stuff((Select concat(',',quotename(Name),' ',system_type_name)
From sys.dm_exec_describe_first_result_set(#SQL,null,null ) A
Order By column_ordinal
For XML Path ('')),1,1,'') +')
Insert '+#temp+' '+#SQL+'
'
Exec(#SQL)
Select * from ##myTempTable
I am new to SQL Server. I wrote a simple stored procedure that returns rows with data condition.
Here is my code:
CREATE PROCEDURE test.newArtists
#LastUpdated smalldatetime
AS
BEGIN
SELECT *
FROM ARTIST
WHERE GENERATED > #LastUpdated OR MODIFIED > #LastUpdated;
END
When I execute this stored procedure, it returns 0 rows. like ...
DECLARE #temp DATETIME;
SET #temp = CONVERT (DATETIME, '2016/12/05');
EXEC test.newArtists #LastUpdated = #temp;
However, when I execute the query without using procedure, it returns about 5,000 rows.
DECLARE #temp DATETIME;
SET #temp = CONVERT (DATETIME, '2016/12/05');
SELECT *
FROM ARTIST
WHERE GENERATED > #temp OR MODIFIED > #temp;
I just do not understand whey those two returns different results.
Thanks for explanations!
===================================================
I find the problem. Thanks.
I'am using test schema.
So I connected to test. However, When I use SELECT * FROM ARTIST it does not search test.ARTIST, but ARTIST table which belongs to dbo.
Summary.
The basic problem here is I got 2 tables with name ARTIST.
However, I still do not understand why it automatically look for dbo schema.
I got little experience with MYSQL, but when I connect to a certain schema, it only find objects inside of the schema. Is it normal or should I do some work to set prefix?
Thanks for answers though b
It is always good practice to refer to database objects by a schema
name and the object name, separated by a period (.).
object referred to without an explicit schema name ... will be located by searching the default schema first, followed by the dbo schema
Source: SQL Server Best Practices – Implementation of Database Object Schemas
Dbo is the default schema, it will always go to that one first if the schema is not specified. It is a good practice to always specify the schema in the query especially if you are using more than one. That way it won't waste time looking in the wrong schema first.
For decades i've used VIEWs as a synonym:
CREATE VIEW dbo.Banks AS
SELECT *
FROM OtherDatabase.dbo.Banks
i do this so i can abstract where the "real" table is. And when it changes, it's as simple as altering the view:
And this works well. It's doesn't cause the optimizer any issues, and i have been able to edit the view as required.
Synonyms
Starting with SQL Server 2005, Microsoft introduced synonyms:
CREATE SYNONYM dbo.Banks FOR OtherDatabase.dbo.Banks
It seems to work identically to the VIEW approach. Every execution plan i've looked at behaves identically.
Unfortunately it seems that synonyms are unable to provide one of their basic functions, functionality i need:
Provides a layer of abstraction that protects a client application from changes made to the name or location of the base object
You are not able to change where a synonym points. Because there is no ALTER SYNONYM statement, you first have to drop the synonym and then re-create the synonym with the same name, but point the synonym to the new location.
Do they have any redeeming quality?
Practically speaking, this isn't going to happen. i will just never do it. i won't use a mechanism that requires me to drop objects from a database in order to change a setting. i'm certainly not going to delete all the easily alterable VIEWs, replacing them with SYNONYMs, and have to explain to everyone why making everything harder is "better".
So my question is, is there anything i am losing by using views?
every execution plan looks identical to synonyms
i can easily change the "view synonym" at any time
Is there a virtue to a table or view synonym that i'm missing?
Aside from having to call RefreshAllViews in case i forgot that i made a table change somewhere
Even stored procedures
i don't even use synonyms for stored procedures:
CREATE PROCEDURE dbo.GetUSDNoonRateAsOf #tradeDate datetime AS
EXECUTE OtherDatabase.dbo.GetUSDNoonRateAsOf #tradeDate
Is there a value in synonyms that i am missing?
Update: RefreshAllViews procedure
We have a standard procedure in every database. Reordering, or inserting, columns wreaks havoc on views; so they have to be "refreshed".
CREATE PROCEDURE [dbo].[RefreshAllViews] AS
-- This sp will refresh all views in the catalog.
-- It enumerates all views, and runs sp_refreshview for each of them
SET NOCOUNT ON
DECLARE abc CURSOR FOR
SELECT TABLE_NAME AS ViewName
FROM INFORMATION_SCHEMA.VIEWS
ORDER BY newid()
OPEN abc
DECLARE #ViewName varchar(128)
--DECLARE #ParmDefinition NVARCHAR(500)
-- Build select string once
DECLARE #SQLString nvarchar(2048)
--SET #SQLString = N'EXECUTE sp_RefreshView #View'
--SET #ParmDefinition = N'#View nvarchar(128)'
FETCH NEXT FROM abc
INTO #ViewName
WHILE ##FETCH_STATUS = 0
BEGIN
IF #ViewName <> 'IndexServerNodes'
BEGIN
SET #SQLString = 'EXECUTE sp_RefreshView '+#ViewName
PRINT #SQLString
EXECUTE sp_ExecuteSQL #SQLString--, #ParmDefinition, #View = #ViewName
END
FETCH NEXT FROM abc
INTO #ViewName
END
CLOSE abc
DEALLOCATE abc
God knows why SQL Server doesn't do it for me.
A synonym is a much more transparent redirect. I prefer them over views because views need to be maintained. When you use SELECT * especially.
I'm not sure I buy that the lack of ALTER SYNONYM is a real blocker. The drop/create of a synonym is a very simple metadata operation, and will be very fast. Omitting error handling for brevity:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN TRANSACTION;
DROP SYNONYM ...
CREATE SYNONYM ...
COMMIT TRANSACTION;
Similarly, for stored procedures, if your base stored procedure interface changes (say, you add a parameter), you have to also change the wrapper procedure - not so with a synonym.
One downside is that you can create, say, an instead of trigger on a view, but you can't on a synonym. There are other operations you can't perform via a synonym (mostly DDL). And of course IntelliSense may not function correctly, depending on version.
Not being able to memorize the syntax seems like a made-up excuse to me. There are no fancy options or with clauses; just a 2-part name for the synonym, and a 2-, 3- or 4-part name for the object it refers to:
CREATE SYNONYM dbo.Something FOR Server.Database.dbo.SomethingElse;
If you can't memorize that, how did you create the synonym in the first place?
I also have a suggestion to thoroughly simplify your stored procedure (and prevent it from failing when any view is not in the dbo schema, or the procedure is executed by someone whose default schema is not the same as the view's schema, or the view has an ' or space in its name, or otherwise breaks any of the rules for identifiers (you can find them on this page)):
CREATE PROCEDURE [dbo].[RefreshAllViews]
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sql NVARCHAR(MAX) = N'';
SELECT #sql += '
EXEC sp_refreshview ' + CHAR(39)
+ QUOTENAME(REPLACE(s.name,'''',''''''))
+ '.' + QUOTENAME(REPLACE(v.name,'''','''''')) + CHAR(39) + ';'
FROM sys.views AS v
INNER JOIN sys.schemas AS s
ON v.[schema_id] = s.[schema_id];
PRINT #sql;
EXEC sp_executesql #sql;
END
GO
At the very least, if you're going to keep the cursor, stop using the terrible default options (declare the cursor as LOCAL FAST_FORWARD), and use sys.views instead of INFORMATION_SCHEMA.
God knows why SQL Server doesn't do it for me.
Because SQL Server is software, and it isn't perfect - especially when it comes to dependencies. The main problem is that you are violating a best practice by using SELECT * in your views in the first place. shrug If you would accept your hang-ups about synonyms, you won't have to worry about that.
If a view references a table, and you subsequently add columns to that table, you must modify the view in order to “pick up” the new column—even if you use SELECT *. Synonyms will “pick up” those columns automatically. Here’s a sample script:
-- Set things up
CREATE TABLE Foo
(
Id int not null
,data varchar(10) not null
)
GO
INSERT Foo values (1,'one'),(2,'Two')
GO
CREATE SYNONYM synFoo for Foo
GO
CREATE VIEW vFooDelim as select Id, Data from Foo
GO
CREATE VIEW vFooStar as select * from Foo
GO
select * from Foo
select * from synFoo
select * from vFooDelim
select * from vFooStar
then,
-- Add a column
ALTER TABLE Foo
add MoreData datetime default getdate()
GO
select * from Foo
select * from synFoo
select * from vFooDelim
select * from vFooStar
GO
(don’t forget to)
-- Clean things up
DROP Synonym synFoo
DROP VIEW vFooDelim
DROP VIEW vFooStar
DROP TABLE Foo
A significantly more obscure situation (that we do all the time here), if you have to set up a reference in a database to an object in another database, you don’t necessarily know what columns are in that table (dynamic denormalized) are or will be, and you don’t know the name of the database at the time you write your code (one database per client, but only once they sign the contract) (usually), using synonyms can be a godsend. At the time of database creation, just dynamicaly build and run CREATE SYNONYM myTable FOR <DatabaseName>.<schema>.MyTable, and you are done—no matter what columns get added for which client in the future.
Synonyms are useful for situations where you're working with lots of disparate data sources/multiple databases etc, or doing data migrations.
I've never really found cause to use them in new, greenfield developments.
I want to write a stored proc which will use a parameter, which will be the table name.
E.g:
#tablename << Parameter
SELECT * FROM #tablename
How is this possible?
I wrote this:
set ANSI_NULLS ON
set QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[GetAllInterviewQuestions]
#Alias varchar = null
AS
BEGIN
Exec('Select * FROM Table as ' #Alias)
END
But it says incorrect syntax near #Alias.
Well, firstly you've omitted the '+' from your string. This way of doing things is far from ideal, but you can do
DECLARE #SQL nvarchar(max)
SELECT #SQL = 'SELECT * FROM ' + QuoteName(#Alias)
Exec(#SQL)
I'd strongly suggest rethinking how you do this, however. Generating Dynamic SQL often leads to SQL Injection vulnerabilities as well as making it harder for SQL Server (and other DBs) to work out the best way to process your query. If you have a stored procedure that can return any table, you're really getting virtually no benefit from it being a stored procedure in the first place as it won't be able to do much in the way of optimizations, and you're largely emasculating the security benefits too.
You'll have to do it like this:
exec('select * from '+#tablename+' where...')
But make sure you fully understand the risks, like SQL injection attacks. In general, you shouldn't ever have to use something like this if the DB is well designed.
Don't you mean
Exec('SELECT * FROM ' + #tableName)
Also, the error you get is because you've forgotten a + before #Alias.
Often, having to parameterize the table name indicates you should re-think your database schema. If you are pulling interview questions from many different tables, it is probably better to create one table with a column distinguishing between the questions in whatever way the different tables would have.
Most implementations of SQL do not allow you to specify structural elements - table names, column names, order by columns, etc. - via parameters; you have to use dynamic SQL to parameterize those aspects of a query.
However, looking at the SQL, you have:
Exec('SELECT * FROM Table AS ' #Alias)
Surely, this would mean that the code will only ever select from a table called 'Table', and you would need to concatenate the #Alias with it -- and in many SQL dialects, concatenation is indicated by '||':
Exec('SELECT * FROM Table AS ' || #Alias)
This still probably doesn't do what you want - but it might not generate a syntax error when the procedure is created (but it would probably generate an error at runtime).
simple problem, but perhaps no simple solution, at least I can't think of one of the top of my head but then I'm not the best at finding the best solutions.
I have a stored proc, this stored proc does (in a basic form) a select on a table, envision this:
SELECT * FROM myTable
okay, simple enough, except the table name it needs to search on isn't known, so we ended up with something pretty similiar to this:
-- Just to give some context to the variables I'll be using
DECLARE #metaInfoID AS INT
SET #metaInfoID = 1
DECLARE #metaInfoTable AS VARCHAR(200)
SELECT #metaInfoTable = MetaInfoTableName FROM MetaInfos WHERE MetaInfoID = #MetaInfoID
DECLARE #sql AS VARCHAR(200)
SET #sql = 'SELECT * FROM ' + #metaInfoTable
EXEC #sql
So, I, recognize this is ultimately bad, and can see immediately where I can perform a sql injection attack. So, the question is, is there a way to achieve the same results without the construction of the dynamic sql? or am I going to have to be super, super careful in my client code?
You have to use dynamic sql if you don't know the table name up front. But yes, you should validate the value before attempting to use it in an SQL statement.
e.g.
IF EXISTS(SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME=#metaInfoTable)
BEGIN
-- Execute the SELECT * FROM #metaInfoTable dynamic sql
END
This will make sure a table with that name exists. There is an overhead to doing this obviously as you're querying INFORMATION_SCHEMA. You could instead validate the #metaInfoTable contains only certain characters:
-- only run dynamic sql if table name value contains 0-9,a-z,A-Z, underscores or spaces (enclose table name in square brackets, in case it does contain spaces)
IF NOT #metaInfoTable LIKE '%^[0-9a-zA-Z_ ]%'
BEGIN
-- Execute the SELECT * FROM #metaInfoTable dynamic sql
END
Given the constraints described, I'd suggest 2 ways, with slight variations in performance an architecture.
Choose At the Client & Re-Architect
I'd suggest that you should consider a small re-architecture as much as possible to force the caller/client to decide which table to get its data from. It's a code smell to hold table names in another table.
I am taking an assumption here that #MetaInfoID is being passed from a webapp, data access block, etc. That's where the logic of which table to perform the SELECT on should be housed. I'd say that the client should know which stored procedure (GetCustomers or GetProducts) to call based on that #MetaInfoID. Create new method in your DAL like GetCustomersMetaInfo() and GetProductsMetaInfo() and GetInvoicesMetaInfo() which call into their appropriate sprocs (with no dynamic SQL needed, and no maintenance of a meta table in the DB).
Perhaps try to re-architect the system a little bit.
In SQL Server
If you absolutely have to do this lookup in the DB, and depending on the number of tables that you have, you could perform a handful of IF statements (as many as needed) like:
IF #MetaInfoID = 1
SELECT * FROM Customers
IF #MetaInfoID =2
SELECT * FROM Products
-- etc
That would probably become to be a nightmare to maintain.
Perhaps you could write a stored procedure for each MetaInfo. In this way, you gain the advantage of pre-compilation, and no SQL injection can occur here. (imagine if someone sabotaged the MetaInfoTableName column)
IF #MetaInfoID = 1
EXEC GetAllCustomers
IF #MetaInfoID = 2
EXEC GetAllProducts