Creating tables based off of SchemaColumn & SchemaTable composite table - sql-server

I wanted to create a new set of tables in a db using a stored procedure and a build table.
The build table includes the following columns and some sample rows:
tblNm colNm colTyp colLen colReq colWarning colUni colComUni
account personID Decimal NULL 0 0 0 0
account studentNum String 15 0 0 0 0
I was considering using multiple cursors as a form of nested looping, but I cannot figure out how to define the column parameters in the nested procedure because cursors only return one value.
I am considering to build an alter statement that parses these values. How could I do this?

You can solve the problem by using two cursors or one cursor. Two cursors will make the code more readable. One cursor will be more efficient.
Two cursors
The code below demonstrates how to use two cursors to iterate through the tables and columns.
DECLARE #tblNm VARCHAR(MAX)
DECLARE cTables CURSOR FOR
SELECT tblNm
FROM CompositeSchema
GROUP BY tblNm
ORDER BY tblNm
OPEN cTables
FETCH cTables INTO #tblNm
WHILE ##FETCH_STATUS=0
BEGIN
PRINT 'Processing table '+#tblNm
-- Start of code to execute for each table
DECLARE #sqlCreateTable VARCHAR(MAX)
SET #sqlCreateTable = 'CREATE TABLE ['+#tblNm+'] ('
DECLARE #colNm VARCHAR(MAX),#colTyp VARCHAR(MAX),#colLen INT,#colReq BIT,#colWarning BIT,#colUni BIT,#colComUni BIT
DECLARE #isFirst BIT
SET #isFirst = 1
DECLARE cCols CURSOR FOR
SELECT colNm,colTyp,colLen,colReq,colWarning,colUni,colComUni
FROM CompositeSchema
WHERE tblNm=#tblNm
ORDER BY colComUni DESC,colNm ASC
OPEN cCols
FETCH cCols INTO #colNm,#colTyp,#colLen,#colReq,#colWarning,#colUni,#colComUni
WHILE ##FETCH_STATUS=0
BEGIN
PRINT 'Processing column ['+#tblNm+'].['+#colNm+']'
-- Start of code to process each column (simplified!)
IF #isFirst=0
SET #sqlCreateTable = #sqlCreateTable+','
SET #isFirst = 0
SET #sqlCreateTable = #sqlCreateTable+'['+#colNm+'] '+#colTyp
IF NOT #colLen IS NULL
SET #sqlCreateTable = #sqlCreateTable+'('+CAST(#colLen AS VARCHAR)+')'
-- End of code to process each column
FETCH cCols INTO #colNm,#colTyp,#colLen,#colReq,#colWarning,#colUni,#colComUni
END
CLOSE cCols
DEALLOCATE cCols
SET #sqlCreateTable = #sqlCreateTable+')'
PRINT #sqlCreateTable
-- EXEC(#sqlCreateTable)
-- End of code to execute for each table
FETCH cTables INTO #tblNm
END
CLOSE cTables
DEALLOCATE cTables
One cursor
In this case we use just one cursor. We keep track of what the current table is that we are processing in the #currentTblNm variable. Whenever the variable changes, we create all columns at once.
DECLARE #currentTblNm VARCHAR(MAX),#sqlCreateTable VARCHAR(MAX)
SET #currentTblNm = ''
DECLARE #tblNm VARCHAR(MAX),#colNm VARCHAR(MAX),#colTyp VARCHAR(MAX),#colLen INT,#colReq BIT,#colWarning BIT,#colUni BIT,#colComUni BIT
DECLARE #isFirst BIT
SET #isFirst = 1
DECLARE cCols CURSOR FOR
SELECT tblNm,colNm,colTyp,colLen,colReq,colWarning,colUni,colComUni
FROM CompositeSchema
ORDER BY tblNm ASC,colComUni DESC,colNm ASC
OPEN cCols
FETCH cCols INTO #tblNm,#colNm,#colTyp,#colLen,#colReq,#colWarning,#colUni,#colComUni
WHILE ##FETCH_STATUS=0
BEGIN
IF #currentTblNm<>#tblNm
BEGIN
IF #sqlCreateTable<>''
BEGIN
SET #sqlCreateTable = #sqlCreateTable+')'
PRINT #sqlCreateTable
--EXEC (#sqlCreateTable)
END
SET #isFirst = 1
SET #sqlCreateTable = 'CREATE TABLE ['+#tblNm+'] ('
SET #currentTblNm = #tblNm
PRINT 'Processing table ['+#tblNm+']'
END
-- Start of code to process each column (simplified!)
IF #isFirst=0
SET #sqlCreateTable = #sqlCreateTable+','
SET #isFirst = 0
SET #sqlCreateTable = #sqlCreateTable+'['+#colNm+'] '+#colTyp
IF NOT #colLen IS NULL
SET #sqlCreateTable = #sqlCreateTable+'('+CAST(#colLen AS VARCHAR)+')'
-- End of code to process each column
FETCH cCols INTO #tblNm,#colNm,#colTyp,#colLen,#colReq,#colWarning,#colUni,#colComUni
END
CLOSE cCols
DEALLOCATE cCols
IF #sqlCreateTable<>''
BEGIN
SET #sqlCreateTable = #sqlCreateTable+')'
PRINT #sqlCreateTable
-- EXEC(#sqlCreateTable)
END
Both pieces of code, Two cursors and One cursor are simplified. The logic to properly create all of the constraints (like primary key, unique constraints, foreign keys etc.), the logic to properly map the column data types, and not to forget, the making the distinction between creating a new table and altering an existing table is beyond the scope of this post.
Worth mentioning is that you could also use declarative SQL code with FOR XML to create the table structure. This is possible and would be able to generate the CREATE TABLE statements with a much better performance. From experience I know that this code will be much harder to maintain and you might run into the limitations of declarative SQL.

Related

SQL Server stored procedure , loop through table and get values?

In the code shown below, I loop through a table (in this case the disciplines table) and want to add records to another table, I'm rather new to this
How can I retrieve values from those tables for the insert command?
CREATE PROCEDURE cdisc
-- check disciplines
AS
BEGIN
DECLARE test CURSOR FAST_FORWARD FOR
SELECT *
FROM tbl_portfolio
WHERE show = 'Ja' ;
WHILE ##FETCH_STATUS = 0
BEGIN
DECLARE disc CURSOR FAST_FORWARD FOR
SELECT *
FROM disciplines;
WHILE ##fetch_status = 0
BEGIN
INSERT INTO ProjectDisciplines (Proj_id, discipline_ID)
VALUES (disc!ID, test!id)
END
CLOSE disc
DEALLOCATE disc
END
CLOSE test
DEALLOCATE test
END

How to optimize cursor in a stored procedure

I'm having problems with a stored procedure that iterates over a table, it works fine with a few hundred rows however when the table is over the thousands it saturates the memory and crashes.
The procedure should iterate row by row and fill a column with a value which is calculated from another column in the row. I suspect it is the cursor that crashes the procedure and in other questions I've read to use a while loop but I'm no expert in sql and the examples I tried from those answers didn't work.
CREATE PROCEDURE [dbo].[GenerateNewHashes]
AS
BEGIN
SET NOCOUNT ON;
DECLARE #module BIGINT = 382449983
IF EXISTS(SELECT 1 FROM dbo.telephoneSource WHERE Hash IS NULL)
BEGIN
DECLARE hash_cursor CURSOR FOR
SELECT a.telephone, a.Hash
FROM dbo.telephoneSource AS a
OPEN hash_cursor
FETCH FROM hash_cursor
WHILE ##FETCH_STATUS = 0
BEGIN
UPDATE dbo.telephoneSource
SET Hash = CAST(telephone AS BIGINT) % #module
WHERE CURRENT OF hash_cursor
FETCH NEXT FROM hash_cursor
END
CLOSE hash_cursor
DEALLOCATE hash_cursor
END
END
Basically the stored procedure is intended to fill a new column called Hash that was added to the existing table, when the script that updates the table ends the new column is filled with NULL values and then this stored procedure is supposed to fill each null value with the operation telephone number (which is a bigint) % module variable (big int as well).
Is there anything besides changing to a while loop that I can do to make it use less memory or just don't crash? Thanks in advance.
You could do the following:
WHILE 1=1
BEGIN
UPDATE TOP (10000) dbo.telephoneSource
SET Hash = CAST(telephone AS BIGINT)%#module
WHERE Hash IS NULL;
IF ##ROWCOUNT = 0
BEGIN
BREAK;
END;
END;
This will update Hash as long as there are NULL values and will exit once there have been no records updated.
Adding a filtered index could be useful as well:
CREATE NONCLUSTERED INDEX IX_telephoneSource_Hash_telephone
ON dbo.telephoneSource (Hash)
INCLUDE (telephone)
WHERE Hash IS NULL;
It will speed up lookups in order to update it. But this might be not needed.
Here is example of code to do it in loops from my comment above with out using a cursor, and if you add where your field you are updating IS NOT NULL into the inner loop it wont update ones that were already done (in case you need to restart the process or something.
I didnt include your specific tables in there but if you need me to I can add it in there.
DECLARE #PerBatchCount as int
DECLARE #MAXID as bigint
DECLARE #WorkingOnID as bigint
Set #PerBatchCount = 1000
--Find range of IDs to process using yoru tablename
SELECT #WorkingOnID = MIN(ID), #MAXID = MAX(ID)
FROM YouTableHere WITH (NOLOCK)
WHILE #WorkingOnID <= #MAXID
BEGIN
-- do an update on all the ones that exist in the offer table NOW
--DO YOUR UPDATE HERE
-- include this where clause where ID is your PK you are looping through
WHERE ID BETWEEN #WorkingOnID AND (#WorkingOnID + #PerBatchCount -1)
set #WorkingOnID = #WorkingOnID + #PerBatchCount
END
SET NOCOUNT OFF;
I would simply add computed column:
ALTER TABLE dbo.telephoneSource
ADD Hash AS (CAST(telephone AS BIGINT)%382449983) PERSISTED;

SQL server Select variables showing NULL

in the code below when I run it in Degug mode I can see the variables contain values, however when I select them they show NULL, any ideas? I need to eventually do an Update back to the table [dbo].[HistData]
with the values where RecordID = some number. Any ideas welcome.
-- Declare the variables to store the values returned by FETCH.
DECLARE #HD_TckrPercent decimal(6,3) -- H2 in above formula
DECLARE #HD_CloseLater decimal(9,2) -- F2 in above formula
DECLARE #HD_CloseEarlier decimal(9,2) -- F3 in above formula
DECLARE #RowsNeeded INT
DECLARE #RecordCOUNT INT
SET #RowsNeeded = 2
set #RecordCOUNT = 0 -- to initialize it
DECLARE stocks_cursor CURSOR FOR
SELECT top (#RowsNeeded) [TCKR%], [Stock_Close] FROM [dbo].[HistData]
ORDER BY [RecordID]
OPEN stocks_cursor
-- Perform the first fetch and store the values in variables.
-- Note: The variables are in the same order as the columns
-- in the SELECT statement.
-- Check ##FETCH_STATUS to see if there are any more rows to fetch.
WHILE ##FETCH_STATUS = 0
BEGIN
-- Concatenate and display the current values in the variables.
-- This is executed as long as the previous fetch succeeds.
set #RecordCOUNT = (#RecordCOUNT + 1)
Print #HD_CloseLater
IF #RecordCOUNT = 1
BEGIN
FETCH NEXT FROM stocks_cursor
INTO #HD_TckrPercent, #HD_CloseLater
END
ELSE
BEGIN
FETCH NEXT FROM stocks_cursor
INTO #HD_TckrPercent, #HD_CloseEarlier
END
Select #HD_TckrPercent
Select #HD_CloseLater
Select #HD_CloseEarlier
END
CLOSE stocks_cursor
DEALLOCATE stocks_cursor
GO

sql copy specific columns from one table to another table

In the sql server I have a Codes table in DevDB. It has 10 columns. There is a foreign key column as ParentCodeID, int column.
I need to copy 5 columns excluding the int column to the TestDB. But I need to include "33" as the ParentcodeID to this column.
There are like 250+ records. I need to do this automatically.
Please help!!!!
Try this,
INSERT INTO TestDB.dbo.Codes(Col1,Col2,Col3,....)
SELECT 33,col1,col2.... FROM DevDB.dbo.tblCode where ParentCodeID = xx
You can automate even the generation of the script to avoid corrections in future if new columns would be added. Here is the code:
declare #tblName varchar(100),#FieldList varchar(max),#FieldListReplaced varchar(max),#sql varchar(max),#Field varchar(100),#ReplacedField varchar(100),#ReplaceWithValue varchar(100)
set #tblName='Codes'
set #ReplacedField='ParentcodeID'
set #ReplaceWithValue='33'
set #FieldList=','
declare cur_fieldlist cursor for
select [Name] from sys.all_columns
where object_id=(Select id from sysobjects where name=#tblName) and
is_computed=0 and (is_identity=0)
order by column_id
open cur_fieldlist
fetch next from cur_fieldlist into #Field
while ##fetch_status = 0
begin
set #FieldList=#FieldList+#Field+','
fetch next from cur_fieldlist into #Field
end
close cur_fieldlist
deallocate cur_fieldlist
set #FieldListReplaced=replace(#FieldList,','+#ReplacedField+',',','+#ReplaceWithValue+',') /*to avoid replacement for several fields like ParentcodeID, ParentcodeID2, BigParentcodeID, BigParentcodeID2 etc. */
set #FieldList=substring(#FieldList,2,(len(#FieldList)-2))
set #FieldListReplaced=substring(#FieldListReplaced,2,(len(#FieldListReplaced)-2))
set #sql='INSERT INTO TestDB.dbo.'+#tblName+' ('+#FieldList+')
SELECT '+#FieldListReplaced+' FROM DevDB.dbo.'+#tblName+' WHERE 1=1 /* put your conditions here */'
print (#sql) --for checking
exec (#sql)

performance problem sql server 2005 update sentence

I have a table "OFICIAL3" with 500k rows. and 30 columns. and table INSIS with 150k rows and 20 columns.
OFICIAL3.NUMERO_TITULO has an index.
INSIS.NumeroDocumento has an index too.
update sentence take long time. this process will take 9 hours in my machine
my machine is a core 2 duo 2.GHZ and 2GB RAM
ALTER PROCEDURE [dbo].[CompletarDatos] AS
declare #cantidad int;
declare #CONTADOR int;
declare #NRO_TITULO VARCHAR(600);
declare #POYECTO VARCHAR(200);
DECLARE #I_PROYECTO VARCHAR(500);
DECLARE #I_AREA_INT VARCHAR(500);
SET NOCOUNT ON
BEGIN
SET #cantidad =(select count(*) from OFICIAL3)
SET #CONTADOR=1
declare CURSORITO cursor for
select NUMERO_TITULO from OFICIAL3
open CURSORITO
fetch next from CURSORITO
into #NRO_TITULO
while ##fetch_status = 0
begin
SET #CONTADOR=#CONTADOR+1
PRINT 'ROW='+CONVERT(NVARCHAR(30),#CONTADOR)+' NRO TITULO='+#NRO_TITULO
SET #I_PROYECTO = (SELECT PROYECTO FROM INSIS WHERE NumeroDocumento=#NRO_TITULO)
SET #I_AREA_INT = (SELECT I_AREA_INTERVENCION FROM INSIS WHERE NumeroDocumento=#NRO_TITULO)
UPDATE OFICIAL3 SET PROYECT=#I_PROYECTO , COD_AREA=#I_AREA_INT WHERE NUMERO_TITULO=#NRO_TITULO
fetch next from CURSORITO into #NRO_TITULO
end
-- cerramos el cursor
close CURSORITO
deallocate CURSORITO
END
Assuming OFICIAL4 is a typo, this should work as a single update:
UPDATE o
SET PROYECT = i.PROYECTO,
COD_AREA = i.I_AREA_INTERVENCION
FROM OFICIAL3 o
INNER JOIN
INSIS i
ON o.NUMERO_TITULO = i.NumeroDocumento
As others have commented, an approach that avoids the CURSOR is vastly preferable from a performance point of view. Another thought is that a covering index on `INSIS (NumeroDocumento, PROYECTO, I_AREA_INTERVENCION) would speed things up further for this query.
Is there any way you can do this without a cursor? Removing the iteration should help it considerably.

Resources