I have a function in Microsoft SQL that creates auto sql lines, a (Querybuilder). In this case I have this line in one of my Cursors That looks something like this:
set #Query = #Query + ' AND ('+ (case when(#CountryDelivery = 1) then 'Delivery' else 'pickup' end) +'.CountryUK '+ (case(#CountryExcl) when 1 then ' <> ''' else ' = ''' end) + #country + ''')'
This will make my result look something like:
"AND (country = 'USA') AND (Country = 'Canada') ......"ETC, all depends how many countries the user stores in his datatable.
My challenge is that I want the "AND" statement to be removed at the start of the Query but not if the user add more countries.
This is my full function:
SET #Query = #Query + ' ('
declare Accountring_Country1Cursor Cursor for
select Country, Exclude, IsDelivery, ID from dbo.Accounting_Country1 where Service_ID = #Service_ID
OPEN Accountring_Country1Cursor
FETCH NEXT FROM Accountring_Country1Cursor
INTO #country, #CountryExcl, #CountryDelivery, #country_ID
WHILE ##FETCH_STATUS = 0
BEGIN
set #Query = #Query + ' AND ('+ (case when(#CountryDelivery = 1) then 'Delivery' else 'pickup' end) +'.CountryUK '+ (case(#CountryExcl) when 1 then ' <> ''' else ' = ''' end) + #country + ''')'
FETCH NEXT FROM Accountring_Country1Cursor
INTO #country, #CountryExcl, #CountryDelivery,#country_ID
END
CLOSE Accountring_Country1Cursor
DEALLOCATE Accountring_Country1Cursor
SET #Query = #Query + ') '
RETURN #Query
END
Try this:
declare #filter varchar(max);
set #filter = '(1 = 1)';
declare Accountring_Country1Cursor Cursor for select Country, Exclude, IsDelivery, ID from dbo.Accounting_Country1 where Service_ID = #Service_ID
OPEN Accountring_Country1Cursor
FETCH NEXT FROM Accountring_Country1Cursor INTO #country, #CountryExcl, #CountryDelivery, #country_ID
WHILE ##FETCH_STATUS = 0
BEGIN
set #filter = #filter + ' AND ('+ (case when(#CountryDelivery = 1) then 'Delivery' else 'pickup' end) +'.CountryUK '+ (case(#CountryExcl) when 1 then ' <> ''' else ' = ''' end) + #country + ''')'
FETCH NEXT FROM Accountring_Country1Cursor INTO #country, #CountryExcl, #CountryDelivery,#country_ID
END
CLOSE Accountring_Country1Cursor
DEALLOCATE Accountring_Country1Cursor
SET #Query = #Query + '(' + #filter + ') '
RETURN #Query
Basically, the default value for the filter is true. Then you can always add a new ' AND (something)', because it will be valid SQL. And after you're done building the filter, you just append it to your #Query variable, all at once.
Or, in a more straightforward way, you can just take a substring of your final query:
set #Query = '(' + substring(#Query, 4, len(#Query))
Or, slightly more complicated but more versatile:
set #Query = substring(#Query, 0, 1) + substring(#Query, 4, len(#Query))
Related
We've around 7 million records in Azure SQL DB. We have created a stored procedure to fetch paginated records. Following is the script:
CREATE PROCEDURE [dbo].[spGetCommodityReport]
(
#pgNum AS INT = 1,
#recPerPg AS INT = 100,
#factory AS NVARCHAR(max) = null,
#filter AS NVARCHAR(max) = null,
#isoWYF as VARCHAR(50) = null,
#isoWYT as VARCHAR(50) = null,
#columns as NVARCHAR(max) = null,
#measureCol AS NVARCHAR(max) = null
)
AS
BEGIN
SET NOCOUNT ON
DECLARE
#where AS NVARCHAR(2000),
#SQL AS NVARCHAR(max),
--sanitize local variables
SET #factory = ISNULL(#factory,'')
SET #filter = ISNULL(#filter,'')
SET #isoWYF = ISNULL(#isoWYF,'')
SET #isoWYT = ISNULL(#isoWYT,'')
SET #measureCol = ISNULL(#measureCol, '[food_qty]')
SET #where = '[cal_year_week] BETWEEN ' +char(39)+ #isoWYF +char(39) + ' AND ' + char(39) + #isoWYT + char(39)
IF(#factory != '')
BEGIN
SET #where = '[factory] IN ' + #factory + ' AND ' + #where
END
-- apply filters if requested
IF(#filter != '')
BEGIN
SET #where = #where + ' AND ' + #filter
END
--Prepare dynamic query
SET #SQL = 'SELECT ((totRec/' + CAST(#recPerPg AS VARCHAR(100)) + ') +1) [totPg],' + CAST( #PgNum AS VARCHAR(100)) +' [currPg],'+ CAST(#RecPerPg AS VARCHAR(100)) +' [recPerPg], * FROM ('
SET #SQL = #SQL + 'SELECT * '
SET #SQL = #SQL + ' from
(SELECT
count(*) over() [totRec]
,[rec_no]
,[factory]
,[city]
,[food]
,[drink]
,[ingre]
,[cal_year_week]
,' + #measureCol +
',[audit_region]
FROM [dbo].[TBLFOOD] WITH (NOLOCK) WHERE'
+ #where
SET #SQL = #SQL + ') as SRC PIVOT(SUM('+ #measureCol +') for [cal_year_week] in ('+ #columns +')) as PVT) AS FOODTBL '
SET #SQL = #SQL + ' ORDER BY factory'
if(#recPerPg > 0)
BEGIN
SET #SQL = #SQL + ' OFFSET ' + CAST( #recPerPg * (#pgNum - 1) AS VARCHAR(100)) + ' ROWS '
SET #SQL = #SQL + ' FETCH NEXT ' + CAST( #recPerPg AS VARCHAR(100)) + ' ROWS ONLY;'
END
print(#SQL)
EXEC(#SQL)
END
In SYS API we enrich the response and our EXP API is being consumed by front-end app. We don't have Process API in this project.
We're doing transformation which is discussed here: Transformation of Payload to custom object using DataWeave 2.0
This is working as expected in dev environment where we have around 50k records. When we promote this to higher environment where we have around 7 million records, we're getting timed-out exceptions. Tried a few things, increase the response time-out in both EXP API and SYS API but no luck.
While ran following statement in DB (having 7 million records), it returns 100 records as per criteria in less than a second:
EXEC [dbo].[spGetCommodityReport] #factory = '0415', #filter = " city != '0099' ", #isoWYF = "[202201]" , #isoWYT = "[202203]", #columns = "[202201], [202202], [202203]"
My question:
What is the best way to fix this time-out issue?
I have created a script to compare the tables of two databases to find differences. I want to be able to manually set two variables (one for each database) and have the 'USE' statement use the variable value for one of the databases but it changes context and does work correctly.
This is what I am trying to use to populating a variable (#Use) with the USE code to execute
--set #Use = 'USE ' + #NewProdDB + ';SELECT name FROM sys.tables'
--EXEC (#Use)
This is the entire query:
--========================================================================================================
-- Used to find table values in the test upgrade system database not found the newly upgraded system
-- database. Make sure change the system database names before running the query.
-- Also, select the upgraded database to run against
--========================================================================================================
DECLARE #NewProdDB VARCHAR(100)
DECLARE #TestUpgradeDB VARCHAR(100)
DECLARE #Columns VARCHAR (MAX)
DECLARE #Query VARCHAR(MAX)
DECLARE #ResultsTest VARCHAR(MAX)
DECLARE #SetResultsCursor NVARCHAR(MAX)
DECLARE #Fetcher NVARCHAR(MAX)
DECLARE #Use NVARCHAR(50)
DECLARE #ListOfTables VARCHAR(100)
DECLARE #WTF NVARCHAR(max)
DECLARE #rescanCode nvarchar(max)
/* Enter Upgraded system DB here */
SET #NewProdDB = 'zSBSSYS'
/* Enter Test Upgrade system DB here */
SET #TestUpgradeDB = 'TESTzSBSSYS'
--set #Use = 'USE ' + #NewProdDB + ';SELECT name FROM sys.tables'
--EXEC (#Use)
SET NOCOUNT ON
set #rescanCode = 'DECLARE recscan CURSOR FOR
SELECT TABLE_NAME FROM ' + #TestUpgradeDB + '.INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME NOT LIKE ''Mbf%'' AND TABLE_TYPE = ''BASE TABLE''
ORDER BY TABLE_NAME'
exec (#rescanCode)
OPEN recscan
FETCH NEXT FROM recscan INTO #ListOfTables
WHILE ##fetch_status = 0
BEGIN
BEGIN TRY
/* START Table column query */
Declare #Table varchar(60)
set #Table = #ListOfTables
set #table = (SELECT top 1 name FROM sys.tables where name = #Table)
DECLARE
#sql1 nvarchar(max) = ''
,#INTOColumns NVARCHAR(MAX)=''
,#Where nvarchar(max) = ''
,#ColumnID Int
,#Column Varchar(8000)
,#Type Varchar(8000)
,#Length Int
,#del1 varchar(2) = ','
,#CRLF bit = 1
,#aliasfield varchar(5) = '[a].'
DECLARE CSR_Attributes CURSOR FOR
SELECT
[ColumnID] = Col.column_id,
[Column] = Col.Name,
[Type] = [types].name,
[Length] = Col.max_length
FROM sys.objects Obj, sys.columns Col, sys.types [types]
WHERE Obj.OBJECT_ID = Col.OBJECT_ID AND Obj.TYPE = 'U'
AND Col.system_type_id = [types].system_type_id
AND Obj.name = #table
AND Col.name <> 'tstamp'
ORDER BY Obj.name, Col.column_id, Col.name
OPEN CSR_Attributes
FETCH NEXT FROM CSR_Attributes INTO #ColumnID, #Column, #Type, #Length
WHILE ##FETCH_STATUS = 0
BEGIN
set #sql1 += #column + #del1
set #Where += 'b.' + #column + '=a.' + #column + ' and '
set #INTOColumns += '#INTOcol,'
FETCH NEXT FROM CSR_Attributes INTO #ColumnID, #Column, #Type, #Length
END
CLOSE CSR_Attributes
DEALLOCATE CSR_Attributes
SET #Columns = SUBSTRING(#sql1,1,len(#sql1)-1) -- get rid of last comma
SET #Where = SUBSTRING(#Where,1,len(#Where)-4) -- get rid of last 'and'
SET #INTOColumns = SUBSTRING(#INTOColumns,1,len(#INTOColumns)-1) -- get rid of last comma
/* END Table column query */
/* Create SELECT statement here */
SET #ResultsTest='SELECT TOP 1 ' + #Columns + '
FROM ' + #TestUpgradeDB + '..' + #Table + ' A
WHERE NOT EXISTS
(SELECT ' + #Columns + '
FROM ' + #NewProdDB + '..' + #Table + ' B WHERE ' + #Where + ')'
SET #SetResultsCursor = 'DECLARE DataReturned CURSOR FOR ' + #ResultsTest
exec (#SetResultsCursor)
OPEN DataReturned
SET #Fetcher = 'DECLARE #INTOcol NVARCHAR(Max)
FETCH NEXT FROM DataReturned INTO ' + #INTOColumns
exec (#Fetcher)
if ##FETCH_STATUS = 0
begin
CLOSE DataReturned
DEALLOCATE DataReturned
SET #Query='SELECT ' + #Columns + '
FROM ' + #TestUpgradeDB + '..' + #Table + ' A WHERE NOT EXISTS
(SELECT ' + #Columns + ' FROM ' + #NewProdDB + '..' + #Table + ' B WHERE ' + #Where + ')'
select #Table
exec (#Query)
end
else
begin
CLOSE DataReturned
DEALLOCATE DataReturned
end
end try
begin catch
--SELECT ERROR_MESSAGE() AS ErrorMessage;
end catch
FETCH NEXT FROM recscan INTO #ListOfTables
end
CLOSE recscan
DEALLOCATE recscan
SET NOCOUNT OFF
You can't do this:
DECLARE #DB varchar(10) = 'beep';
DECLARE #USE varchar(50) = 'USE '+ #DB;
EXEC sp_sqlexec #USE;
SELECT blah FROM bloop
Database context is only used during the #USE sql, then reverts
You can do this:
DECLARE #DB varchar(10) = 'beep';
DECLARE #SQL varchar(50) = 'USE '+#DB+'; SELECT blah FROM bloop;'
EXEC sp_sqlexec #USE;
That aside, look into SQL Server Data Tools (SSDT)
It does exactly what it seems you are trying to do, schema and/or data compare between databases, and it's very easy to use.
hello this my dynamic query and this procedure I did tested is working.
but Does not bring data to the server-side (entity)
visual studio 2012
framework 4.5
entity store procedure
public IEnumerable<spGetInvoiceDetailSearch_Result> GetInvoiceDetailedSearch(InvoiceModel item)
{
return DALContext.GetInvoiceDetailedSearch(item);
}
ALTER PROCEDURE [dbo].[spGetInvoiceDetailSearch] #InvoiceItemID INT
,#InvoiceTypeID INT
,#VesselID INT
,#PaidBy NVARCHAR(50)
,#InvoiceNo NVARCHAR(50)
,#CompanyID INT
,#InvoiceFromDate DATE
,#InvoiceToDate DATE
,#FromDueDate DATE
,#ToDueDate DATE
,#FromAmount DECIMAL(18, 4)
,#ToAmount DECIMAL(18, 4)
,#DueDateType NVARCHAR(50)
AS
BEGIN
DECLARE #SQLQuery AS NVARCHAR(4000)
SELECT #SQLQuery =
'SELECT dbo.Invoices.InvoiceID, dbo.Invoices.CompanyID, dbo.Invoices.VesselID, dbo.Invoices.InvoiceNo, dbo.Invoices.DueDate, dbo.Invoices.Amount,
dbo.Invoices.Comment, dbo.Invoices.IsPaid, dbo.Invoices.PaymentDate, dbo.Invoices.PaidBy, dbo.Invoices.Period, dbo.Invoices.InvoiceDate,
dbo.Invoices.InvoiceCurrencyCode, dbo.Invoices.InvoiceAmount, dbo.Invoices.IsReceived, dbo.Invoices.IsProforma, dbo.Invoices.InvoiceTypeID,
dbo.Invoices.IsDeleted, dbo.Invoices.Parity, dbo.Invoices.DueDateType, dbo.Vessels.Name AS VesselName, dbo.InvoiceVsInvoiceItems.ItemPrice as ItemPrice,
dbo.InvoiceVsInvoiceItems.InvoiceItemID as InvoiceItemID, dbo.InvoiceVsInvoiceItems.VAT as VAT, dbo.InvoiceVsInvoiceItems.ItemType as ItemType, dbo.InvoiceItems.Name AS InvoiceItemName,
dbo.Companies.Name AS CompanyName, dbo.InvoiceTypes.Name AS InvoiceTypeName
FROM dbo.Invoices LEFT OUTER JOIN
dbo.Companies ON dbo.Invoices.CompanyID = dbo.Companies.CompanyID LEFT OUTER JOIN
dbo.InvoiceTypes ON dbo.Invoices.InvoiceTypeID = dbo.InvoiceTypes.InvoiceTypeID LEFT OUTER JOIN
dbo.InvoiceVsInvoiceItems ON dbo.Invoices.InvoiceID = dbo.InvoiceVsInvoiceItems.InvoiceID LEFT OUTER JOIN
dbo.InvoiceItems ON dbo.InvoiceVsInvoiceItems.InvoiceVsInvoiceItemID = dbo.InvoiceItems.InvoiceItemID LEFT OUTER JOIN
dbo.Vessels ON dbo.Invoices.VesselID = dbo.Vessels.VesselID WHERE
dbo.Invoices.IsDeleted != 1
and dbo.Vessels.IsDeleted != 1
and dbo.Companies.IsDeleted != 1 '
SET FMTONLY OFF
IF #InvoiceItemID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.InvoiceItems.InvoiceItemID= ''' + CAST(#InvoiceItemID AS NVARCHAR(50)) + ''''
END
IF #InvoiceTypeID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.InvoiceTypeID= ''' + CAST(#InvoiceTypeID AS NVARCHAR(50)) + ''''
END
IF #VesselID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.VesselID= ''' + CAST(#VesselID AS NVARCHAR(50)) + ''''
END
IF #PaidBy IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + 'AND dbo.Invoices.PaidBy = ''' + CAST(#PaidBy AS NVARCHAR(50)) + ''''
END
IF #InvoiceNo IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + 'AND dbo.Invoices.InvoiceNo = ''' + CAST(#InvoiceNo AS NVARCHAR(50)) + ''''
END
IF #CompanyID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.CompanyID = ''' + CAST(#CompanyID AS NVARCHAR(50)) + ''''
END
IF #FromAmount IS NOT NULL AND #ToAmount IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.Amount BETWEEN ''' + CAST(#FromAmount AS NVARCHAR(100)) + ''' AND ''' + CAST(#ToAmount AS NVARCHAR(100)) + ''''
END
IF #DueDateType IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + 'AND dbo.Invoices.DueDateType = ''' + CAST(#DueDateType AS NVARCHAR(50)) + ''''
END
IF #InvoiceFromDate IS NOT NULL AND #InvoiceToDate IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.InvoiceDate Between ''' + CAST(#InvoiceFromDate AS NVARCHAR(100)) + ''' AND ''' + CAST(#InvoiceToDate AS NVARCHAR(100)) + ''''
END
IF #FromDueDate IS NOT NULL AND #ToDueDate IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.DueDate Between ''' + CAST(#FromDueDate AS NVARCHAR(100)) + ''' AND ''' + CAST(#ToDueDate AS NVARCHAR(100)) + ''''
END
EXECUTE (#SQLQuery)
END
and end question
my table date type : date format but
server shows it like datetime how can I do to change it to date format..
thank you
regards
ALTER PROCEDURE [dbo].[spGetInvoiceDetailSearch] #InvoiceItemID INT
,#InvoiceTypeID INT
,#VesselID INT
,#PaidBy NVARCHAR(50)
,#InvoiceNo NVARCHAR(50)
,#CompanyID INT
,#InvoiceFromDate DATE
,#InvoiceToDate DATE
,#FromDueDate DATE
,#ToDueDate DATE
,#FromAmount DECIMAL(18, 4)
,#ToAmount DECIMAL(18, 4)
,#DueDateType NVARCHAR(50)
AS
BEGIN
DECLARE #SQLQuery AS NVARCHAR(4000)
SELECT #SQLQuery =
'SELECT dbo.Invoices.InvoiceID, dbo.Invoices.CompanyID, dbo.Invoices.VesselID, dbo.Invoices.InvoiceNo, dbo.Invoices.DueDate, dbo.Invoices.Amount,
dbo.Invoices.Comment, dbo.Invoices.IsPaid, dbo.Invoices.PaymentDate, dbo.Invoices.PaidBy, dbo.Invoices.Period, dbo.Invoices.InvoiceDate,
dbo.Invoices.InvoiceCurrencyCode, dbo.Invoices.InvoiceAmount, dbo.Invoices.IsReceived, dbo.Invoices.IsProforma, dbo.Invoices.InvoiceTypeID,
dbo.Invoices.IsDeleted, dbo.Invoices.Parity, dbo.Invoices.DueDateType, dbo.Vessels.Name AS VesselName, dbo.InvoiceVsInvoiceItems.ItemPrice as ItemPrice,
dbo.InvoiceVsInvoiceItems.InvoiceItemID as InvoiceItemID, dbo.InvoiceVsInvoiceItems.VAT as VAT, dbo.InvoiceVsInvoiceItems.ItemType as ItemType, dbo.InvoiceItems.Name AS InvoiceItemName,
dbo.Companies.Name AS CompanyName, dbo.InvoiceTypes.Name AS InvoiceTypeName
FROM dbo.Invoices LEFT OUTER JOIN
dbo.Companies ON dbo.Invoices.CompanyID = dbo.Companies.CompanyID LEFT OUTER JOIN
dbo.InvoiceTypes ON dbo.Invoices.InvoiceTypeID = dbo.InvoiceTypes.InvoiceTypeID LEFT OUTER JOIN
dbo.InvoiceVsInvoiceItems ON dbo.Invoices.InvoiceID = dbo.InvoiceVsInvoiceItems.InvoiceID LEFT OUTER JOIN
dbo.InvoiceItems ON dbo.InvoiceVsInvoiceItems.InvoiceVsInvoiceItemID = dbo.InvoiceItems.InvoiceItemID LEFT OUTER JOIN
dbo.Vessels ON dbo.Invoices.VesselID = dbo.Vessels.VesselID WHERE
dbo.Invoices.IsDeleted != 1
and dbo.Vessels.IsDeleted != 1
and dbo.Companies.IsDeleted != 1 '
SET FMTONLY OFF
IF #InvoiceItemID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.InvoiceItems.InvoiceItemID= ' + CAST(#InvoiceItemID AS NVARCHAR(50)) + ''
END
IF #InvoiceTypeID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.InvoiceTypeID= ' + CAST(#InvoiceTypeID AS NVARCHAR(50)) + ''
END
IF #VesselID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.VesselID= ' + CAST(#VesselID AS NVARCHAR(50)) + ''
END
IF #PaidBy IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.PaidBy = ''' + CAST(#PaidBy AS NVARCHAR(50)) + ''''
END
IF #InvoiceNo IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.InvoiceNo = ''' + CAST(#InvoiceNo AS NVARCHAR(50)) + ''''
END
IF #CompanyID > 0
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.CompanyID = ' + CAST(#CompanyID AS NVARCHAR(50)) + ''
END
IF #FromAmount IS NOT NULL AND #ToAmount IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.Amount BETWEEN ''' + CAST(#FromAmount AS NVARCHAR(100)) + ''' AND ''' + CAST(#ToAmount AS NVARCHAR(100)) + ''''
END
IF #DueDateType IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.DueDateType = ''' + CAST(#DueDateType AS NVARCHAR(50)) + ''''
END
IF #InvoiceFromDate IS NOT NULL AND #InvoiceToDate IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.InvoiceDate Between ''' + CAST(#InvoiceFromDate AS NVARCHAR(100)) + ''' AND ''' + CAST(#InvoiceToDate AS NVARCHAR(100)) + ''''
END
IF #FromDueDate IS NOT NULL AND #ToDueDate IS NOT NULL
BEGIN
SET #SQLQuery = #SQLQuery + ' AND dbo.Invoices.DueDate Between ''' + CAST(#FromDueDate AS NVARCHAR(100)) + ''' AND ''' + CAST(#ToDueDate AS NVARCHAR(100)) + ''''
END
PRINT (#SQLQuery)
END
First of all, debugging it's very easy. Replace EXEC(#SQLQUery) with print and then you see your actual query.
You had some sintax error ( some places where AND was missing a space in front) and also you have some interger that were treated as strings.
Try my updated procedure.
It seems that procedure is getting called properly but no rows are getting returned, to debug the exact problem you can write actual hardcoded query returning 1 or more records instead of dynamic query.
So after doing that there are two possibilities
procedure call via edmx returns data, that means parameter values are causing some problem.
Any Data is not returned.
To solve any of the problem you need to check corresponding sql query which is getting generated while calling SP via Enitity Framework.
I am looking for a script which finds and replaces all fields of type string within a DB with specified text.
The script would for example take the following parameters:
Search for: null
Replace with: empty-string
The primary string data types in SQL Server: Varchar, NVarchar, Text.
This script would then comb through all string based table data and look for in this case null and replace it with a empty string.
Ok I've put together the following code in the meantime.
-- Specify 'dbo' for all tables
DECLARE #schemaName VARCHAR(5) = 'dbo'
BEGIN
DECLARE #tableName VARCHAR(255) -- table name
DECLARE #tableID INT -- table id (aka syst.table.object_id)
DECLARE table_cursor CURSOR FOR
SELECT T.object_id AS TableID, T.name AS TableName FROM sys.tables T
INNER JOIN sys.schemas S ON S.schema_id = T.schema_id
WHERE S.name = #schemaName
OPEN table_cursor
FETCH NEXT FROM table_cursor INTO #tableID, #tableName
WHILE ##FETCH_STATUS = 0
BEGIN
-- construct each tables queries
DECLARE #totalColumnsFound INT = (SELECT COUNT(*) FROM sys.columns C WHERE OBJECT_ID = #tableID
-- text and nvarchar column data types chosen for me (if you need more like ntext, varcahr see sys.types for their ids)
AND (C.system_type_id = 35 OR c.system_type_id = 231))
IF (#totalColumnsFound > 0)
BEGIN
DECLARE #tableUpdateQuery VARCHAR(MAX) = 'update ' + #schemaName + '.' + #tableName + ' set ';
DECLARE #columnName VARCHAR(255) -- column name
DECLARE column_cursor CURSOR FOR
SELECT C.name AS ColumnName FROM sys.columns C WHERE OBJECT_ID = #tableID
-- text and nvarchar column data types chosen for me (if you need more like ntext, varcahr see sys.types for their ids)
AND (C.system_type_id = 35 OR c.system_type_id = 231)
OPEN column_cursor
FETCH NEXT FROM column_cursor INTO #columnName
WHILE ##FETCH_STATUS = 0
BEGIN
-- construct the columns for the update query, piece by piece.
-- This is also where you can apply your logic for how to handle the string update.
-- I am trimming string and updating nulls to empty strings here.
SET #tableUpdateQuery = #tableUpdateQuery + ' ' + #columnName + ' = ltrim(rtrim(isnull(' + #columnName + ',''''))),'
FETCH NEXT FROM column_cursor INTO #columnName
END
CLOSE column_cursor
DEALLOCATE column_cursor
-- trim last comma from string
SET #tableUpdateQuery = LEFT(#tableUpdateQuery, LEN(#tableUpdateQuery) - 1)
/** debuging purposes **
print 'Updating table --> ' + #tableName
print #tableUpdateQuery
print ' '
*/
-- execute dynamic sql
EXEC(#tableUpdateQuery)
END
FETCH NEXT FROM table_cursor INTO #tableID, #tableName
END
CLOSE table_cursor
DEALLOCATE table_cursor
END
--GO
this should help you:
/*
Author: sqiller
Description: Searches for a value to replace in all columns from all tables
USE: EXEC dbo.usp_Update_AllTAbles 'work', 'sqiller', 1
#search = Value to look for Replace
#newvalue = the value that will replace #search
#Test = If set to 1, it will only PRINT the UPDATE statement instead of EXEC, useful to see
what is going to update before.
*/
CREATE PROCEDURE dbo.usp_Update_AllTAbles(
#search varchar(100),
#newvalue varchar(100),
#Test bit)
AS
BEGIN
IF NOT EXISTS (select 1 from INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 'Tables_to_Update')
BEGIN
CREATE TABLE dbo.Tables_to_Update(
Table_name varchar(100),
Column_name varchar(100),
recordsToUpdate int
)
END
DECLARE #table varchar(100)
DECLARE #column varchar(100)
DECLARE #SQL varchar(max)
SELECT TABLE_SCHEMA+'.'+TABLE_NAME as Table_Name, 0 as Processed INTO #tables from information_schema.tables WHERE TABLE_TYPE != 'VIEW'
WHILE EXISTS (select * from #tables where processed = 0)
BEGIN
SELECT top 1 #table = table_name from #tables where processed = 0
SELECT column_name, 0 as Processed INTO #columns from information_schema.columns where TABLE_SCHEMA+'.'+TABLE_NAME = #table
WHILE EXISTS (SELECT * from #columns where processed = 0)
BEGIN
SELECT top 1 #column = COLUMN_NAME from #columns where processed = 0
SET #SQL = 'INSERT INTO Tables_to_Update
select '''+ #table +''', '''+ #column +''', count(*) from '+#table+ ' where '+ #column +' like ''%'+ #search +'%'''
EXEC(#SQL)
IF EXISTS (SELECT * FROM Tables_to_Update WHERE Table_name = #table)
BEGIN
SET #SQL = 'UPDATE '+ #table + ' SET '+ #column + ' = REPLACE('''+#column+''','''+#search+''','''+ #newvalue +''') WHERE '+ #column + ' like ''%'+#search+'%'''
--UPDATE HERE
IF (#Test = 1)
BEGIN
PRINT #SQL
END
ELSE
BEGIN
EXEC(#SQL)
END
END
UPDATE #columns SET Processed = 1 where COLUMN_NAME = #column
END
DROP TABLE #columns
UPDATE #tables SET Processed = 1 where table_name = #table
END
SELECT * FROM Tables_to_Update where recordsToUpdate > 0
END
The following will find and replace a string in every database (excluding system databases) on every table on the instance you are connected to:
Simply change 'Search String' to whatever you seek and 'Replace String' with whatever you want to replace it with.
--Getting all the databases and making a cursor
DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('master','model','msdb','tempdb') -- exclude these databases
DECLARE #databaseName nvarchar(1000)
--opening the cursor to move over the databases in this instance
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #databaseName
WHILE ##FETCH_STATUS = 0
BEGIN
PRINT #databaseName
--Setting up temp table for the results of our search
DECLARE #Results TABLE(TableName nvarchar(370), RealColumnName nvarchar(370), ColumnName nvarchar(370), ColumnValue nvarchar(3630))
SET NOCOUNT ON
DECLARE #SearchStr nvarchar(100), #ReplaceStr nvarchar(100), #SearchStr2 nvarchar(110)
SET #SearchStr = 'Search String'
SET #ReplaceStr = 'Replace String'
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%','''')
DECLARE #TableName nvarchar(256), #ColumnName nvarchar(128)
SET #TableName = ''
--Looping over all the tables in the database
WHILE #TableName IS NOT NULL
BEGIN
DECLARE #SQL nvarchar(2000)
SET #ColumnName = ''
DECLARE #result NVARCHAR(256)
SET #SQL = 'USE ' + #databaseName + '
SELECT #result = MIN(QUOTENAME(TABLE_SCHEMA) + ''.'' + QUOTENAME(TABLE_NAME))
FROM [' + #databaseName + '].INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = ''BASE TABLE'' AND TABLE_CATALOG = ''' + #databaseName + '''
AND QUOTENAME(TABLE_SCHEMA) + ''.'' + QUOTENAME(TABLE_NAME) > ''' + #TableName + '''
AND OBJECTPROPERTY(
OBJECT_ID(
QUOTENAME(TABLE_SCHEMA) + ''.'' + QUOTENAME(TABLE_NAME)
), ''IsMSShipped''
) = 0'
EXEC master..sp_executesql #SQL, N'#result nvarchar(256) out', #result out
SET #TableName = #result
PRINT #TableName
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
DECLARE #ColumnResult NVARCHAR(256)
SET #SQL = '
SELECT #ColumnResult = MIN(QUOTENAME(COLUMN_NAME))
FROM [' + #databaseName + '].INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(''[' + #databaseName + '].' + #TableName + ''', 2)
AND TABLE_NAME = PARSENAME(''[' + #databaseName + '].' + #TableName + ''', 1)
AND DATA_TYPE IN (''char'', ''varchar'', ''nchar'', ''nvarchar'')
AND TABLE_CATALOG = ''' + #databaseName + '''
AND QUOTENAME(COLUMN_NAME) > ''' + #ColumnName + ''''
PRINT #SQL
EXEC master..sp_executesql #SQL, N'#ColumnResult nvarchar(256) out', #ColumnResult out
SET #ColumnName = #ColumnResult
PRINT #ColumnName
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC
(
'USE ' + #databaseName + '
SELECT ''' + #TableName + ''',''' + #ColumnName + ''',''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630)
FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
)
END
END
END
--Declaring another temporary table
DECLARE #time_to_update TABLE(TableName nvarchar(370), RealColumnName nvarchar(370))
INSERT INTO #time_to_update
SELECT TableName, RealColumnName FROM #Results GROUP BY TableName, RealColumnName
DECLARE #MyCursor CURSOR;
BEGIN
DECLARE #t nvarchar(370)
DECLARE #c nvarchar(370)
--Looping over the search results
SET #MyCursor = CURSOR FOR
SELECT TableName, RealColumnName FROM #time_to_update GROUP BY TableName, RealColumnName
--Getting my variables from the first item
OPEN #MyCursor
FETCH NEXT FROM #MyCursor
INTO #t, #c
WHILE ##FETCH_STATUS = 0
BEGIN
-- Updating the old values with the new value
DECLARE #sqlCommand varchar(1000)
SET #sqlCommand = '
USE ' + #databaseName + '
UPDATE [' + #databaseName + '].' + #t + ' SET ' + #c + ' = REPLACE(' + #c + ', ''' + #SearchStr + ''', ''' + #ReplaceStr + ''')
WHERE ' + #c + ' LIKE ''' + #SearchStr2 + ''''
PRINT #sqlCommand
BEGIN TRY
EXEC (#sqlCommand)
END TRY
BEGIN CATCH
PRINT ERROR_MESSAGE()
END CATCH
--Getting next row values
FETCH NEXT FROM #MyCursor
INTO #t, #c
END;
CLOSE #MyCursor ;
DEALLOCATE #MyCursor;
END;
DELETE FROM #time_to_update
DELETE FROM #Results
FETCH NEXT FROM db_cursor INTO #databaseName
END
CLOSE db_cursor
DEALLOCATE db_cursor
Note: this isn't ideal, nor is it optimized
Here is another answer, similar to above (and hopefully more readable/efficient), since I recently had a similar requirement and this is how I solved it.
CREATE OR ALTER PROCEDURE UPDATE_ALL_COLUMNS
#TableNameSearchFilter NVARCHAR(100),
#TableSchema NVARCHAR(100),
#TestValue NVARCHAR(100),
#NewValue NVARCHAR(100)
AS
BEGIN
DECLARE #NRCOLUMNS INT;
DECLARE #i INT = 0;
DECLARE #COLUMN NVARCHAR(100) = '';
DECLARE #SQL NVARCHAR(MAX) = '';
DECLARE #TableToUpdate NVARCHAR(256) = '';
DECLARE #insertingNULL BIT;
IF (#NewValue IS NULL) SET #insertingNULL = 1
ELSE SET #insertingNULL = 0;
WHILE #TableToUpdate IS NOT NULL
BEGIN
SELECT #TableToUpdate = MIN(TABLE_NAME)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE #TableNameSearchFilter
AND TABLE_SCHEMA = #TableSchema
AND TABLE_NAME > #TableToUpdate;
WITH CTE1 AS
(
SELECT ROW_NUMBER() OVER (ORDER BY ORDINAL_POSITION) AS RN
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #TableToUpdate
AND TABLE_SCHEMA = #TableSchema
AND (#insertingNULL = 0 OR (#insertingNULL = 1 AND IS_NULLABLE = 'YES'))
)
SELECT #i = MIN(RN), #NRCOLUMNS = MAX(RN) FROM CTE1;
WHILE (#i <= #NRCOLUMNS AND #TableToUpdate IS NOT NULL)
BEGIN
WITH CTE AS
(
SELECT *, ROW_NUMBER() OVER (ORDER BY ORDINAL_POSITION) AS RN
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #TableToUpdate
AND TABLE_SCHEMA = #TableSchema
AND (#insertingNULL = 0 OR (#insertingNULL = 1 AND IS_NULLABLE = 'YES'))
)
SELECT #COLUMN = COLUMN_NAME
FROM CTE
WHERE RN = #i;
SET #SQL = #SQL +
N'UPDATE D SET ' + #COLUMN + N' = ' + ISNULL(N'''' + #NewValue + N'''', N'NULL')
+ N' FROM ' + #TableSchema + N'.' + #TableToUpdate + N' D WHERE CAST(D.' + #COLUMN + ' AS NVARCHAR) = ' + ISNULL(N'''' + #TestValue + N'''', N'NULL') + ';'
+ NCHAR(13) + NCHAR(10);
SET #i = #i + 1;
END;
END;
--PRINT SUBSTRING(#SQL, 1, 4000)
--PRINT SUBSTRING(#SQL, 4001, 8000)
--PRINT SUBSTRING(#SQL, 8001, 12000)
--PRINT SUBSTRING(#SQL, 12001, 16000)
--PRINT SUBSTRING(#SQL, 16001, 20000)
--PRINT SUBSTRING(#SQL, 20001, 24000)
EXEC (#SQL)
END
GO
As a usage example:
EXEC UPDATE_ALL_COLUMNS '%temp%', 'dbo', '', NULL
Parameters:
#TableNameSearchFilter - this will be used with the LIKE operator to find all the tables from your database whose names that match this value;
#TableSchema - the schema of the table (usually dbo)
#TestValue - the value to search for in ALL of the columns (and rows) of each found table;
#NewValue - the value to replace #TestValue with. Can also be NULL.
Explanation:
The EXEC statement will find ALL tables whose names contain the word 'temp', on the 'dbo' schema of your database, then search for the value '' (empty string) in ALL columns of ALL of the found tables, then replace this value with a NULL.
Obviously, if you have long(er) column/table names or the update value, make sure to update the limits on the parameters.
Make sure to first comment the last line (EXEC (#SQL)) and uncomment the lines with PRINT, just to get an idea for what the procedure does and how the final statements look like.
This is not going to work (most likely) if you want to search for the NULL value (i.e. to have #TestValue as NULL). Nevertheless, it can be easily changed to accomplish this as well, by replacing the equal sign from the WHERE clause (in the dynamic query) with IS NULL and removing the rest of the line, when #TestValue IS NULL.
Can be easily adapted to search for columns of only certain types (like VARCHAR etc).
The procedure accounts for inserting NULL values, and will only do so in NULLABLE columns.
Need help with this SQL Server 2000 procedure. The problem is made difficult because I'm testing procedure via Oracle SQL Developer.
I'm running the procedure to iterate column with new sequence of numbers in Varchar format for those who have null values.
But I keep getting error, so a) I may have done a wrong approach b) syntax is incorrect due to version used. I'm primarily Oracle user.
Error I keep getting: SQL Error: Incorrect syntax near the keyword 'End'. which isn't helpful enough to fix it out. The End refers to the very last 'End' in the procedure.
Any help would be greatly appreciated.
Here's the Procedure.
ALTER PROCEDURE [dbo].[OF_AUTOSEQUENCE] #JvarTable Varchar(250), #varColumn Varchar(250), #optIsString char(1), #optInterval int AS
/*
Procedure OF_AUTOSEQUENCE
Created by Joshua [Surname omitted]
When 20100902
Purpose To fill up column with new sequence numbers
Arguments varTable - Table name
varColumn - Column name
optIsString - Option: is it string or numeric, either use T(rue) or F(alse)
optInterval - Steps in increment in building new sequence (Should be 1 (one))
Example script to begin procedure
EXECUTE [dbo].[OF_AUTOSEQUENCE] 'dbo.EH_BrownBin', 'Match', 'T', 1
Any questions about this, please send email to
[business email omitted]
*/
declare
#topseed int,
#stg_topseed varchar(100),
#Sql_string nvarchar(4000),
#myERROR int,
#myRowCount int
set #Sql_string = 'Declare MyCur CURSOR FOR select ' + #varColumn + ' from ' + #JvarTable + ' where ' + #varColumn + ' is null'
Exec sp_executesql #Sql_string
SET NOCOUNT ON
Begin
if #optIsString = 'T'
Begin
set #Sql_string = 'select top 1 ' + #varColumn + ' from ' + #JvarTable + ' order by convert(int, ' + #varColumn + ') desc'
set #stg_topseed = #Sql_string
set #topseed = convert(int, #stg_topseed)
ENd
else
Begin
set #Sql_string = 'select top 1 ' + #varColumn + ' from ' + #JvarTable + ' order by ' + #varColumn + ' desc'
set #topseed = #Sql_string
ENd
-- SELECT #myERROR = ##ERROR, #myRowCOUNT = ##ROWCOUNT
-- IF #myERROR != 0 GOTO HANDLE_ERROR
open MyCur
fetch next from MyCur
WHILE ##FETCH_STATUS = 0
set #topseed = #topseed + #optInterval
if #optIsString = 'T'
begin
set #Sql_string = 'update ' + #JvarTable + ' set ' + #varColumn + ' = cast((' + #topseed + ') as char) where current of ' + MyCur
exec (#Sql_string)
ENd
else
begin
set #Sql_string = 'update ' + #JvarTable + ' set ' + #varColumn + ' = ' + #topseed + ' where current of ' + MyCur
exec (#Sql_string)
ENd
fetch next from MyCur
ENd
-- SELECT #myERROR = ##ERROR, #myRowCOUNT = ##ROWCOUNT
-- IF #myERROR != 0 GOTO HANDLE_ERROR
--HANDLE_ERROR:
--print #myERROR
CLOSE MyCur
DEALLOCATE MyCur
End
you're missing a begin right after the WHILE. You indented like you want a block (multiple statements) in the while loop, and even have a end for the while, but no begin.
make it:
...
open MyCur
fetch next from MyCur
WHILE ##FETCH_STATUS = 0
begin --<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<add this
set #topseed = #topseed + #optInterval
if #optIsString = 'T'
begin
set #Sql_string = 'update ' + #JvarTable + ' set ' + #varColumn + ' = cast((' + #topseed + ') as char) where current of ' + MyCur
exec (#Sql_string)
ENd
else
begin
set #Sql_string = 'update ' + #JvarTable + ' set ' + #varColumn + ' = ' + #topseed + ' where current of ' + MyCur
exec (#Sql_string)
ENd
fetch next from MyCur
ENd
...