I have N amounts of tables like this:
foo_1_data
(
id int,
some_foo_data_1 varchar(100),
some_foo_data_2 char(5)
)
foo_2_data
(
id int,
some_foo_data_1 varchar(100),
some_foo_data_2 char(5)
)
bar_1_data
(
id int,
some_bar_data_1 decimal(10,2),
some_bar_data_2 datetime
)
bar_2_data
(
id int,
some_bar_data_1 decimal(10,2),
some_bar_data_2 datetime
)
The tables are based on different data - foo_data and bar_data - and are user generated and there is an ID - foo_N_data and bar_N_data - set.
Now I want to implement validation on records on these tables through a Stored Procedure to follow certain business rules on the receiving end, which we don't have control over.
Example:
some_foo_data_1 is a varchar(100) is a name and in our system it can be
between 0 and 100 characters but on the receiving end it needs to be
between 2 and 70 characters.
some_bar_data_1 is a decimal(10.2) and it needs to be between 0 and
100 (let's say it's a percentage)
We tried and implement Stored Procedures called, for example, sp_rule_name, that takes the original table and column and validated it through Dynamic SQL like this:
EXEC('SELECT ' + #column_to_validate + ' FROM ' + #table_to_validate + ' WHERE ...')
This creates re-usable code between the different validation procedures (one for each different data set - one for foo_data and one for bar_data) but the code is hard to understand and maintain because of the mixture of T-SQL and Dynamic SQL. We also tried using functions on the fields but dropped it because of slow performance.
Is there an efficient and re-usable way to validate data from dynamic tablas?
Note: We know how the table structure will look like of both foo and bar but we don't know the N value so we can't call it directly.
Once I had faced a similar situation and I had to rely on dynamic SQL to get the job done. But troubleshooting it and even reading it was a pain.
I did a small change and that helped in solving the troubleshooting/reading problem especially if it's only SELECT on the underlying table
See live demo
create proc sp_rule_name #tblName varchar(100)
as
begin
declare #sql varchar(max);
set #sql =' CREATE VIEW tempView AS SELECT * FROM '+ #tblName+ ' ;'
exec (#sql)
-- after this normal SQL syntax using tempView object for all sort of validations. No need for dynamic SQL
select * from tempView;
end
go
create table foo_1_data (
id int,
some_foo_data_1 varchar(100),
some_foo_data_2 char(5)
);
insert into foo_1_data values
(1,'345453','56666');
execute sp_rule_name N'foo_1_data';
Related
In T-SQL, I can create a table variable using syntax like
DECLARE #table AS TABLE (id INT, col VARCHAR(20))
For now, if I want to create an exact copy of a real table in the database, I do something like this
SELECT *
FROM INFOMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'MY_TABLE_NAME'
to check the column datatype and also max length, and start to create the #table variable, naming the variable, datatype and max_length one by one which is not very effective. May I know if there is any simpler way to do it like
DECLARE #table AS TABLE = SOME_REAL_TABLE_IN_DATABASE
Furthermore, is there any way to retrieve the column name, data type and max length of the column and use it directly in the declaration like
DECLARE #table AS TABLE (#col1_specs)
Thank you in advance.
EDIT:
Thanks for the answers and comments, we can do that for #table_variable but only in dynamic SQL and it is not good for maintainability. However, we can do that using #temp_table.
Based on the answer by Ezlo, we can do something like this :
SELECT TABLE.* INTO #TEMP_TABLE FROM TABLE
For more information, please refer to this answer.
Difference between temp table and table variable (stackoverflow)
Difference between temp table and table variable (dba.stackexchange)
Object names and data types (tables, columns, etc.) can't be parameterized (can't come from variables). This means you can't do the following (which would be required to copy a table structure, for example):
DECLARE #TableName VARCHAR(50) = 'Employees'
SELECT
T.*
FROM
#TableName AS T
The only workaround is to use dynamic SQL:
DECLARE #TableName VARCHAR(50) = 'Employees'
DECLARE #DynamicSQL VARCHAR(MAX) = '
SELECT
T.*
FROM
' + QUOTENAME(#TableName) + ' AS T '
EXEC (#DynamicSQL)
However, variables (scalar and table variables) declared outside the dynamic SQL won't be accessible inside as they lose scope:
DECLARE #VariableOutside INT = 10
DECLARE #DynamicSQL VARCHAR(MAX) = 'SELECT #VariableOutside AS ValueOfVariable'
EXEC (#DynamicSQL)
Msg 137, Level 15, State 2, Line 1
Must declare the scalar variable "#VariableOutside".
This means that you will have to declare your variable inside the dynamic SQL:
DECLARE #DynamicSQL VARCHAR(MAX) = 'DECLARE #VariableOutside INT = 10
SELECT #VariableOutside AS ValueOfVariable'
EXEC (#DynamicSQL)
Result:
ValueOfVariable
10
Which brings me to my conclusion: if you want to dynamically create a copy of an existing table as a table variable, all the access of your table variable will have to be inside a dynamic SQL script, which is a huge pain and has some cons (harder to maintain and read, more prone to error, etc.).
A common approach is to work with temporary tables instead. Doing a SELECT * INTO to create them will inherit the table's data types. You can add an always false WHERE condition (like WHERE 1 = 0) if you don't want the actual rows to be inserted.
IF OBJECT_ID('tempdb..#Copy') IS NOT NULL
DROP TABLE #Copy
SELECT
T.*
INTO
#Copy
FROM
YourTable AS T
WHERE
1 = 0
The answer for both questions is simple NO.
Although, I agree with you that T-SQL should change in this way.
In the first case, it means having a command to clone a table structure.
Of course, there is a possibility to make your own T-SQL extension by using SQLCLR.
create proc City_Info
#StateRef nvarchar(20)
as
begin
declare #StateCod nvarchar(3);
declare #Check int;
select #StateCod = StateCod from State_Cod where State_Nam = #StateRef
create table C0NCAT(#StateCod' ,'City')(Sno int identity(1,1))
end
Can Anyone tell how can i fetch a Particular Name from Column and Make table using Procedure in mssql?
First of all it looks like classic example of SELECT * FROM sales + #yymm
This is a variation of the previous case, where there is a suite of tables that actually do describe the same entity. All tables have the same columns, and the name includes some partitioning component, typically year and sometimes also month. New tables are created as a new year/month begins.
In this case, writing one stored procedure per table is not really feasible. Not the least, because the user may want to specify a date range for a search, so even with one procedure per table you would still need a dynamic dispatcher.
If you still want to go this way you could use Dynamic-SQL.
create proc City_Info
#StateRef nvarchar(20)
as
begin
declare #StateCod nvarchar(3);
declare #Check int;
select #StateCod = StateCod from State_Cod where State_Nam = #StateRef;
DECLARE #sql NVARCHAR(MAX) =
'create table '
+ QUOTENAME(C0NCAT(#StateCod ,'City'))
+ '(Sno int identity(1,1))';
EXEC sp_executesql #sql
end
It looks like #temptables created using dynamic SQL via the EXECUTE string method have a different scope and can't be referenced by "fixed" SQLs in the same stored procedure.
However, I can reference a temp table created by a dynamic SQL statement in a subsequence dynamic SQL but it seems that a stored procedure does not return a query result to a calling client unless the SQL is fixed.
A simple 2 table scenario:
I have 2 tables. Let's call them Orders and Items. Order has a Primary key of OrderId and Items has a Primary Key of ItemId. Items.OrderId is the foreign key to identify the parent Order. An Order can have 1 to n Items.
I want to be able to provide a very flexible "query builder" type interface to the user to allow the user to select what Items he want to see. The filter criteria can be based on fields from the Items table and/or from the parent Order table. If an Item meets the filter condition including and condition on the parent Order if one exists, the Item should be return in the query as well as the parent Order.
Usually, I suppose, most people would construct a join between the Item table and the parent Order tables. I would like to perform 2 separate queries instead. One to return all of the qualifying Items and the other to return all of the distinct parent Orders. The reason is two fold and you may or may not agree.
The first reason is that I need to query all of the columns in the parent Order table and if I did a single query to join the Orders table to the Items table, I would be repoeating the Order information multiple times. Since there are typically a large number of items per Order, I'd like to avoid this because it would result in much more data being transfered to a fat client. Instead, as mentioned, I would like to return the two tables individually in a dataset and use the two tables within to populate a custom Order and child Items client objects. (I don't know enough about LINQ or Entity Framework yet. I build my objects by hand). The second reason I would like to return two tables instead of one is because I already have another procedure that returns all of the Items for a given OrderId along with the parent Order and I would like to use the same 2-table approach so that I could reuse the client code to populate my custom Order and Client objects from the 2 datatables returned.
What I was hoping to do was this:
Construct a dynamic SQL string on the Client which joins the orders table to the Items table and filters appropriate on each table as specified by the custom filter created on the Winform fat-client app. The SQL build on the client would have looked something like this:
TempSQL = "
INSERT INTO #ItemsToQuery
OrderId, ItemsId
FROM
Orders, Items
WHERE
Orders.OrderID = Items.OrderId AND
/* Some unpredictable Order filters go here */
AND
/* Some unpredictable Items filters go here */
"
Then, I would call a stored procedure,
CREATE PROCEDURE GetItemsAndOrders(#tempSql as text)
Execute (#tempSQL) --to create the #ItemsToQuery table
SELECT * FROM Items WHERE Items.ItemId IN (SELECT ItemId FROM #ItemsToQuery)
SELECT * FROM Orders WHERE Orders.OrderId IN (SELECT DISTINCT OrderId FROM #ItemsToQuery)
The problem with this approach is that #ItemsToQuery table, since it was created by dynamic SQL, is inaccessible from the following 2 static SQLs and if I change the static SQLs to dynamic, no results are passed back to the fat client.
3 around come to mind but I'm look for a better one:
1) The first SQL could be performed by executing the dynamically constructed SQL from the client. The results could then be passed as a table to a modified version of the above stored procedure. I am familiar with passing table data as XML. If I did this, the stored proc could then insert the data into a temporary table using a static SQL that, because it was created by dynamic SQL, could then be queried without issue. (I could also investigate into passing the new Table type param instead of XML.) However, I would like to avoid passing up potentially large lists to a stored procedure.
2) I could perform all the queries from the client.
The first would be something like this:
SELECT Items.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
SELECT Orders.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
This still provides me with the ability to reuse my client sided object-population code because the Orders and Items continue to be returned in two different tables.
I have a feeling to, that I might have some options using a Table data type within my stored proc, but that is also new to me and I would appreciate a little bit of spoon feeding on that one.
If you even scanned this far in what I wrote, I am surprised, but if so, I woul dappreciate any of your thoughts on how to accomplish this best.
You first need to create your table first then it will be available in the dynamic SQL.
This works:
CREATE TABLE #temp3 (id INT)
EXEC ('insert #temp3 values(1)')
SELECT *
FROM #temp3
This will not work:
EXEC (
'create table #temp2 (id int)
insert #temp2 values(1)'
)
SELECT *
FROM #temp2
In other words:
Create temp table
Execute proc
Select from temp table
Here is complete example:
CREATE PROC prTest2 #var VARCHAR(100)
AS
EXEC (#var)
GO
CREATE TABLE #temp (id INT)
EXEC prTest2 'insert #temp values(1)'
SELECT *
FROM #temp
1st Method - Enclose multiple statements in the same Dynamic SQL Call:
DECLARE #DynamicQuery NVARCHAR(MAX)
SET #DynamicQuery = 'Select * into #temp from (select * from tablename) alias
select * from #temp
drop table #temp'
EXEC sp_executesql #DynamicQuery
2nd Method - Use Global Temp Table:
(Careful, you need to take extra care of global variable.)
IF OBJECT_ID('tempdb..##temp2') IS NULL
BEGIN
EXEC (
'create table ##temp2 (id int)
insert ##temp2 values(1)'
)
SELECT *
FROM ##temp2
END
Don't forget to delete ##temp2 object manually once your done with it:
IF (OBJECT_ID('tempdb..##temp2') IS NOT NULL)
BEGIN
DROP Table ##temp2
END
Note: Don't use this method 2 if you don't know the full structure on database.
I had the same issue that #Muflix mentioned. When you don't know the columns being returned, or they are being generated dynamically, what I've done is create a global table with a unique id, then delete it when I'm done with it, this looks something like what's shown below:
DECLARE #DynamicSQL NVARCHAR(MAX)
DECLARE #DynamicTable VARCHAR(255) = 'DynamicTempTable_' + CONVERT(VARCHAR(36), NEWID())
DECLARE #DynamicColumns NVARCHAR(MAX)
--Get "#DynamicColumns", example: SET #DynamicColumns = '[Column1], [Column2]'
SET #DynamicSQL = 'SELECT ' + #DynamicColumns + ' INTO [##' + #DynamicTable + ']' +
' FROM [dbo].[TableXYZ]'
EXEC sp_executesql #DynamicSQL
SET #DynamicSQL = 'IF OBJECT_ID(''tempdb..##' + #DynamicTable + ''' , ''U'') IS NOT NULL ' +
' BEGIN DROP TABLE [##' + #DynamicTable + '] END'
EXEC sp_executesql #DynamicSQL
Certainly not the best solution, but this seems to work for me.
I would strongly suggest you have a read through http://www.sommarskog.se/arrays-in-sql-2005.html
Personally I like the approach of passing a comma delimited text list, then parsing it with text to table function and joining to it. The temp table approach can work if you create it first in the connection. But it feel a bit messier.
Result sets from dynamic SQL are returned to the client. I have done this quite a lot.
You're right about issues with sharing data through temp tables and variables and things like that between the SQL and the dynamic SQL it generates.
I think in trying to get your temp table working, you have probably got some things confused, because you can definitely get data from a SP which executes dynamic SQL:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + ''''
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
Also:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * INTO #temp FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + '''; SELECT * FROM #temp;'
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
I have problem compilin this code..can anyone tell whats wrong with the syntax
CREATE PROCEDURE spGenericInsert
(
#insValueStr nvarchar(200)
#tblName nvarchar(10)
)
AS
BEGIN
DECLARE #insQueryStr nvarchar(400)
DECLARE #insPrimaryKey nvarchar(10)
DECLARE #rowCountVal integer
DECLARE #prefix nvarchar(5)
IF #tblName='HW_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM HW_Master_DB)
ELSE IF #TableName='SW_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM SW_Master_DB)
ELSE IF #TableName='INV_Allocation_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM INV_Allocation_DB)
ELSE IF #TableName='REQ_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM REQ_Master_DB)
IF #tblName = 'DEFECT_LOG'
SET #prefix='DEF_'
ELSE IF #tblName='INV_Allocation_DB'
SET #prefix='INV_'
ELSE IF #tblName='REQ_Master_DB'
SET #prefix='REQ_'
ELSE IF #tblName='SW_Master_DB'
SET #prefix='SWI_'
ELSE IF #tblName='HW_Master_DB'
SET #prefix='HWI_'
SET #insPrimaryKey= #prefix + RIGHT(replicate('0',5)+ convert(varchar(5),#rowCountVal),5) -- returns somethin like 'DEF_00005'
SET #insQueryStr= 'INSERT INTO ' + #tblName + ' VALUES (' + #insPrimaryKey + ',' + #insValueStr + ')'
EXEC(#insQueryStr)
END
I know about Integer Identity columns.. but i have to use a AlphaNumeric ID in the tables in inserting new values in a highly multi-user intranet system.
The records will not be deleted from the table. So problem is that of maintain synchronous insertion of records with ID field automatically generated.
Any suggestions how that can be done.
Take your pick:
#TableName isn't defined
#tblName vs. #TableName
I cannot immediately see what's wrong with the syntax (the sharp eye of Jonathan Lonowski has solved that already), but there are some things wrong with the code:
You create dynamic SQL, so your code is vunerable to SQL-injection attacks. Both the input parameters are used in a dangerous way. Solve this by creating a stored procedure for every table. So you don't have to generate SQL anymore.
There is no check if the table is not in the list used.
Your primary key generation algorithm can/will create duplicate keys in a multi-user scenario, or if rows are deleted from the table. Solve by using an identity column or some other feature from the database you are using.
Honestly, you seem to be making a headache for yourself. Check out integer identities and IDENTITY syntax.
Unless you are truly required to use keys in the "DEF_00005" format, they will make your life a lot easier.
CREATE TABLE DemoTable (
Key INT IDENTITY(1,1) NOT NULL PRIMARY KEY,
Value VARCHAR(200)
);
INSERT INTO DemoTable (Value) VALUES ('Something');
SELECT * FROM DemoTable;
| Key | Value |
|-----|-----------|
| 1 | Something |
Aside from missing lots of semicolons, you're going to have to give us more to go on.
Actually, SQL Server might not need semicolons, so ignore that...
But here is a good place to start learning about stored prcedures in SQL server. You can search Google for some more as well.
The following code generates the primaey key for the new record to be inserted and inserts the record into a table, whose name and the values to be inserted are given as parameters to the stored procedure. I am getting a runtime error. I am using Visual Studio 2005 to work with SQL Server 2005 Express Edition
ALTER PROCEDURE spGenericInsert
(
#insValueStr nvarchar(300),
#tblName nvarchar(10)
)
AS
DECLARE #sql nvarchar(400)
DECLARE #params nvarchar(200)
DECLARE #insPrimaryKey nvarchar(10)
DECLARE #rowCountVal integer
DECLARE #prefix nvarchar(5)
--following gets the rowcount of the table--
SELECT #rowCountVal = ISNULL(SUM(spart.rows), 0)
FROM sys.partitions spart
WHERE spart.object_id = object_id(#tblName) AND spart.index_id < 2
SET #rowCountVal = #rowCountVal+1
--Following Creates the Primary Key--
IF #tblName = 'DEFECT_LOG'
SET #prefix='DEF_'
ELSE IF #tblName='INV_Allocation_DB'
SET #prefix='INV_'
ELSE IF #tblName='REQ_Master_DB'
SET #prefix='REQ_'
ELSE IF #tblName='SW_Master_DB'
SET #prefix='SWI_'
ELSE IF #tblName='HW_Master_DB'
SET #prefix='HWI_'
SET #insPrimaryKey= #prefix + RIGHT(replicate('0',5)+ convert(varchar(5),#rowCountVal),5) -- returns somethin like 'DEF_00005'
-- Following is for inserting into the table --
SELECT #sql = N' INSERT INTO #tableName VALUES ' +
N' ( #PrimaryKey , #ValueStr )'
SELECT #params = N'#tableName nvarchar(10), ' +
N'#PrimaryKey nvarchar(10), ' +
N'#ValueStr nvarchar(300)'
EXEC sp_executesql #sql, #params, #tableName=#tblName, #PrimaryKey=#insPrimaryKey, #ValueStr=#insValueStr
Output Message:
Running [dbo].[spGenericInsert] ( #insValueStr = 2,"Hi",1/1/1987, #tblName = DEFECT_LOG ).
Must declare the table variable "#tableName".
No rows affected.
(0 row(s) returned)
#RETURN_VALUE = 0
Finished running [dbo].[spGenericInsert].
You are going to have to concatenate the table name directly into the string, as this cannot be parameterized:
SELECT #sql = N' INSERT INTO [' + #tblName + '] VALUES ' +
N' ( #PrimaryKey , #ValueStr )'
SELECT #params = N'#PrimaryKey nvarchar(10), ' +
N'#ValueStr nvarchar(300)'
To prevent injection attacks, you should white-list this table name. This also isn't robust if the table has other non-nullable columns, etc.
note: Personally, though, I don't think this is a good use of TSQL; it might be more appropriate to construct the command in the client (C# or whatever), and execute it as a parameterized command. There are use-cases for dynamic-SQL, but I'm not sure this is a good example of one.
Better yet, use your preferred ORM tool (LINQ-to-SQL, NHibernate, LLBLGen, Entity Framework, etc) to do all this for you, and concentrate on your actual problem domain.
White list essentially means make sure that the table being passed in is a valid table that you want them to be able to insert into. Let's just say for arguments sake that table name is user provided, the user could then start inserting records into system tables.
You can do a white list check by bouncing the table name of the sysobjects table:
select * from sysobjects where name=#tblname and xType='U'
However as Marc suggested this is not a good use of TSQL, and your better off handling this in the app tier as a paramatized query.
Agree with Marc- overall this is an extremely poor idea. Generic inserts/updates or deletes cause problems for the database eventually.
Another point is that this process will have problems when two users run simulutaneously against the same table as they will try to insert the same Primary Key.