Convert a SQL function into a stored procedure - sql-server

I am having trouble converting an UDF into a stored procedure.
Here is what I've got: this is the stored procedure that calls the function (I am using it to search for and remove all UNICODE characters that are not between 32 and 126):
ALTER PROCEDURE [dbo].[spRemoveUNICODE]
#FieldList varchar(250) = '',
#Multiple int = 0,
#TableName varchar(100) = ''
AS
BEGIN
SET NOCOUNT ON;
DECLARE #SQL VARCHAR(MAX), #counter INT = 0
IF #Multiple > 0
BEGIN
DECLARE #Field VARCHAR(100)
SELECT splitdata
INTO #TempValue
FROM dbo.fnSplitString(#FieldList,',')
WHILE (SELECT COUNT(*) FROM #TempValue) >= 1
BEGIN
DECLARE #Column VARCHAR(100) = (SELECT TOP 1 splitdata FROM #TempValue)
SET #SQL = 'UPDATE ' + #TableName + ' SET ' + #Column + ' = dbo.RemoveNonASCII(' + #Column + ')'
EXEC (#SQL)
--print #SQL
SET #counter = #counter + 1
PRINT #column + ' was checked for ' + #counter + ' rows.'
DELETE FROM #TempValue
WHERE splitdata = #Column
END
END
ELSE IF #Multiple = 0
BEGIN
SET #SQL = 'UPDATE ' + #TableName + ' SET ' + #FieldList + ' = dbo.RemoveNonASCII(' + #FieldList + ')'
EXEC (#SQL)
--print #SQL
SET #counter = #counter + 1
PRINT #column + ' was checked for ' + #counter + ' rows.'
END
END
And here is the UDF that I created to help with the update (RemoveNonASCII):
ALTER FUNCTION [dbo].[RemoveNonASCII]
(#nstring nvarchar(max))
RETURNS varchar(max)
AS
BEGIN
-- Variables
DECLARE #Result varchar(max) = '',#nchar nvarchar(1), #position int
-- T-SQL statements to compute the return value
set #position = 1
while #position <= LEN(#nstring)
BEGIN
set #nchar = SUBSTRING(#nstring, #position, 1)
if UNICODE(#nchar) between 32 and 127
set #Result = #Result + #nchar
set #position = #position + 1
set #Result = REPLACE(#Result,'))','')
set #Result = REPLACE(#Result,'?','')
END
if (#Result = '')
set #Result = null
-- Return the result
RETURN #Result
END
I've been trying to convert it into a stored procedure. I want to track how many rows actually get updated when this is run. Right now it just says that all rows, however many I run this on, are updated. I want to know if say only half of them had bad characters. The stored procedure is already set up so that it tells me which column it is looking at, I want to include how many rows were updated. Here is what I've tried so far:
DECLARE #Result varchar(max) = '',#nchar nvarchar(1), #position int, #nstring nvarchar(max), #counter int = 0, #CountRows int = 0, #Length int
--select Notes from #Temp where Notes is not null order by Notes OFFSET #counter ROWS FETCH NEXT 1 ROWS ONLY
set #nstring = (select Notes from #Temp where Notes is not null order by Notes OFFSET #counter ROWS FETCH NEXT 1 ROWS ONLY)
set #Length = LEN(#nstring)
if #Length = 0 set #Length = 1
-- Add the T-SQL statements to compute the return value here
set #position = 1
while #position <= #Length
BEGIN
print #counter
print #CountRows
select #nstring
set #nchar = SUBSTRING(#nstring, #position, 1)
if UNICODE(#nchar) between 32 and 127
begin
print unicode(#nchar)
set #Result = #Result + #nchar
set #counter = #counter + 1
end
if UNICODE(#nchar) not between 32 and 127
begin
set #CountRows = #CountRows + 1
end
set #position = #position + 1
END
print 'Rows found with invalid UNICODE: ' + convert(varchar,#CountRows)
Right now I'm purposely creating a temp table and adding a bunch of notes and then adding in a bunch of invalid characters.
I created a list of 700+ Notes and then updated 2 of them with some invalid characters (outside the 32 - 127). There are a few that are null and a few that are not null, but that doesn't have anything in them. What happens is that I get 0 updates.
Rows found with invalid UNICODE: 0
Though it does see that the UNICODE for the one that it pulls is 32.
Obviously I'm missing something I just don't see what it is.

Here is a set based solution to handle your bulk replacements. Instead of a slow scalar function this is utilizing an inline table valued function. These are far faster than their scalar ancestors. I am using a tally table here. I keep this as a view on my system like this.
create View [dbo].[cteTally] as
WITH
E1(N) AS (select 1 from (values (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))dt(n)),
E2(N) AS (SELECT 1 FROM E1 a, E1 b), --10E+2 or 100 rows
E4(N) AS (SELECT 1 FROM E2 a, E2 b), --10E+4 or 10,000 rows max
cteTally(N) AS
(
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM E4
)
select N from cteTally
If you are interested about tally tables here is an excellent article on the topic. http://www.sqlservercentral.com/articles/T-SQL/62867/
create function RemoveNonASCII
(
#SearchVal nvarchar(max)
) returns table as
RETURN
with MyValues as
(
select substring(#SearchVal, N, 1) as MyChar
, t.N
from cteTally t
where N <= len(#SearchVal)
and UNICODE(substring(#SearchVal, N, 1)) between 32 and 127
)
select distinct MyResult = STUFF((select MyChar + ''
from MyValues mv2
order by mv2.N
--for xml path('')), 1, 0, '')
FOR XML PATH(''),TYPE).value('.','NVARCHAR(MAX)'), 1, 0, '')
from MyValues mv
;
Now instead of being forced to call this every single row you can utilize cross apply. The performance benefit of just this portion of your original question should be pretty huge.
I also eluded to your string splitter also being a potential performance issue. Here is an excellent article with a number of very fast set based string splitters. http://sqlperformance.com/2012/07/t-sql-queries/split-strings
The last step here would be eliminate the first loop in your procedure. This can be done also but I am not entirely certain what your code is doing there. I will look closer and see what I can find out. In the meantime parse through this and feel free to ask questions about any parts you don't understand.

Here is what I've got working based on the great help from Sean Lange:
How I call the Stored Procedure:
exec spRemoveUNICODE #FieldList='Notes,Notes2,Notes3,Notes4,Notes5',#Multiple=1,#TableName='#Temp'
The #Temp table is created:
create table #Temp (ID int,Notes nvarchar(Max),Notes2 nvarchar(max),Notes3 nvarchar(max),Notes4 nvarchar(max),Notes5 nvarchar(max))
Then I fill it with comments from 5 fields from a couple of different tables that range in length from NULL to blank (but not null) to 5000 characters.
I then insert some random characters like this:
update #Temp
set Notes2 = SUBSTRING(Notes2,1,LEN(Notes2)/2) + N'￿㹊潮Ņ᯸ࢹᖈư㹨ƶ槹鎤⻄ƺ綐ڌ⸀ƺ삸)䀤ƍ샄)Ņᛡ鎤ꗘᖃᒨ쬵Ğᘍ鎤ᐜᏰ>֔υ赸Ƹ쳰డ촜)鉀௿촜)쮜)Ἡ屰山舰霡ࣆ 耏Аం畠Ư놐ᓜતᏛ֔Ꮫ֨Ꮫ꯼ᓜƒ 邰఍厰ఆ邰఍드)抉鎤듄)繟Ĺ띨)᯸ࢹ䮸ࣉ᯸ࢹ䮸ࣉ샰)ԌƏ￿

Related

Merging more than one table into one existing table

This is the table creation and insertion query
If not exists(select * from sysobjects where name='hrs')
Create table hrs(hr int)
declare #cnt int =1
while #cnt <= 12
begin
insert into hrs values(#cnt)
set #cnt=#cnt+1
end
The above code gives the output like
but I just want that
declare #cnt1 int = 1
while #cnt1<=12
begin
EXEC('select he'+#cnt1+' = case when hr = 1 then '+#cnt1+' end from hrs')
set #cnt1=#cnt1+1
end
The above code returns the 12 different table but i just want the all records in one table (without creating any new table).
So, how can i do this?
Please help me.
Thanks.
Here the all column are created dynamically through loop
Here are the full query
declare #s varchar(MAX)=''
declare #j int = 1
while #j<=12
begin
if #j = 12
Set #s = #s+'he'+convert(varchar,#j)+'=MAX(case when hr='+convert(varchar,#j)+' then '+convert(varchar,#j)+' end)'
else
set #s = #s+'he'+convert(varchar,#j)+'=MAX(case when hr='+convert(varchar,#j)+' then '+convert(varchar,#j)+' end),'
set #j=#j+1
end
set #s = 'select '+#s+' from hrs'
exec(#s)
Your query doesn't make a lot of sense, but you can build a list of columns and then exec that:
declare #columns nvarchar(max)
declare #cnt int = 1
while #cnt <= 12
begin
set #columns = isnull(#columns + ', ', '') + 'He' + cast(#cnt as nvarchar) +
' = sum(case when hr = ' + cast(#cnt as nvarchar) + ' then hr end)'
end
declare #sql nvarchar(max) = 'select ' + #columns ' + from hr'
exec (#sql)

updating Null Values in multiple columns of a table (SQL Server)

I have 64 columns and I am trying to automate the loop process. The loop runs but it shows 0 affected rows. If I update the table, column by column, it works.
Any idea why its showing 0 affected rows and what can be done ?
update temp set col1 = 'C' where col1 IS Null; -- works (276 rows affected)--
declare #count as int;
declare #name as varchar(max);
set #count = 2;
while #count < (SELECT Count(*) FROM INFORMATION_SCHEMA.Columns where TABLE_NAME = 'temp')+1
Begin
Set #name = (select name from (select colorder, name from (SELECT *
FROM syscolumns WHERE id=OBJECT_ID('temp')) colnames) as cl where colorder = #count)
Print #name
update temp set #name = 'C' where #name IS Null;
SET #count = #count + 1;
END;
You need to use dynamic sql to update the different columns during runtime as below.
Note: I just added/modified the dynamic sql part.
declare #count as int;
declare #name as varchar(max)
declare #sql nvarchar (1000)
set #count = 2
while #count < (SELECT Count(*) FROM INFORMATION_SCHEMA.Columns where TABLE_NAME = 'temp')+1
Begin
Set #name = (select name from (select colorder, name from (SELECT *
FROM syscolumns WHERE id=OBJECT_ID('temp')) colnames) as cl where colorder = #count)
Print #name
set #sql = N'update temp set ' + #name + '= ''C'' where ' + #name + ' is null ';
exec sp_executesql #sql
SET #count = #count + 1
END;

Conversion Failed String to Integer in SQL

I have stored the OrganisationIds 1 ,2 in #String variables.i want to convert it into Integer.Can anyone please help?
Below is my code..
DECLARE #RowCount INT
Declare #String varchar(100)
declare #OrganizationIds int
SELECT #RowCount = COUNT(*) FROM #RawData
WHILE (#RowCount>0)
BEGIN
set #String=convert(varchar,#OrganizationIds)+','
If (#RowCount>0)
Begin
PRINT 'Loop Sequence : ' + convert(varchar,#RowCount) + ' '
set #OrganizationIds = (SELECT OrgId FROM #RawData WHERE ROWID = #RowCount)
PRINT 'Orgid Inside Loop:' + Convert(varchar,#OrganizationIds)
End
Set #RowCount = #RowCount-1
Set #OrganizationIds = convert(varchar,#OrganizationIds)
PRINT 'Orgid Outside Loop:'+ convert(varchar,#OrganizationIds)
set #String=#String + Convert(varchar,#OrganizationIds)
END
PRINT 'String Value Outside Loop: ' + #String
Declare #TempData Table
(
OrganizationID int
)
insert into #TempData(OrganizationID)
EXEC GetFormsData_Organization #String
I believe you have to use the CAST function instead of convert. Try if that solves the problem.

Need to insert breaks in strings of very column with ' ' or ','

I have a table that has one Column but over 100,000 rows
Col_Name
qwchijhuirhxnihdiuyfnx
dhjhfiurhncnmxmzjcoinrds
xnbxknsiiuncirnxknrxnxz
I need to insert a '.' or '$' or some marker after every 3rd character
Example of result needed:
Col_Name
qwc.hij.hui.rhx.nih.diu.yfn.x
dhj.hfi.urh.ncn.mxm.zjc.oin.rds.
xnb.xkn.sii.unc.irn.xkn.rxn.xz
I originally solved this with:
INSERT INTO New_Table
(
c1
,c2
,c3
)
SELECT
substring(CAST(Col_Name AS VARCHAR(MAX)),1,3) as C1
,substring(CAST(Col_Name AS VARCHAR(MAX)),4,3) as C2
,substring(CAST(Col_Name AS VARCHAR(MAX)),7,3) as C3
From Table_Name
This causes problems later in the script so the data must remain in one column but could be inserted into a new table as long as it was a new table with just one column
Here's a sqlfiddle starting point you can refactor http://sqlfiddle.com/#!6/ab6dd/1/0 using function and while loop.
You may be able to do something more efficient with regular expressions or SQLCLR if you need speed.
CREATE FUNCTION dotify (#input varchar(MAX))
RETURNS varchar(MAX)
AS
BEGIN
DECLARE #output varchar(MAX) = ''
declare #index int = 0
declare #length int
set #length = len(#input)
while #index <= #length
begin
SET #output = #output + substring(#input, #index, 1)
if (#index % 3) = 0 AND #index > 0
BEGIN
SET #output = #output +'.'
END
set #index = #index + 1
end
return(#output)
END
GO
select TOP 10000 col_name, dbo.dotify(col_name) FROM old_table
You can use TOP to limit the processing time to a few seconds so you can easily profile efficiency changes you make.

Create test data in SQL Server

Does anyone have or know of a SQL script that will generate test data for a given table?
Ideally it will look at the schema of the table and create row(s) with test data based on the datatype for each column.
If this doesn't exist, would anyone else find it useful? If so I'll pull my finger out and write one.
Well I thought I would pull my finger out and write myself a light weight data generator:
declare #select varchar(max), #insert varchar(max), #column varchar(100),
#type varchar(100), #identity bit, #db nvarchar(100)
set #db = N'Orders'
set #select = 'select '
set #insert = 'insert into ' + #db + ' ('
declare crD cursor fast_forward for
select column_name, data_type,
COLUMNPROPERTY(
OBJECT_ID(
TABLE_SCHEMA + '.' + TABLE_NAME),
COLUMN_NAME, 'IsIdentity') AS COLUMN_ID
from Northwind.INFORMATION_SCHEMA.COLUMNS
where table_name = #db
open crD
fetch crD into #column, #type, #identity
while ##fetch_status = 0
begin
if #identity = 0 or #identity is null
begin
set #insert = #insert + #column + ', '
set #select = #select +
case #type
when 'int' then '1'
when 'varchar' then '''test'''
when 'nvarchar' then '''test'''
when 'smalldatetime' then 'getdate()'
when 'bit' then '0'
else 'NULL'
end + ', '
end
fetch crD into #column, #type, #identity
end
set #select = left(#select, len(#select) - 1)
set #insert = left(#insert, len(#insert) - 1) + ')'
exec(#insert + #select)
close crD
deallocate crD
Given any table, the script will create one record with some arbitrary values for the types; int, varchar, nvarchar, smalldatetime and bit. The case statement could be replaced with a function. It won't travel down dependencies but it will skip any seeded columns.
My motivation for creating this is to test my NHibernate mapping files against a table with some 50 columns so I was after a quick a simple script which can be re-used.
Have you tried ApexSQL Generate: https://www.apexsql.com/sql_tools_generate.aspx ?
I stumbled upon it during my own search for the similar thing, and it did the job quite well. It’s not free, but you get a free trial with all features available, so you can try before you buy.
I think it will suite your needs quite well, since it keeps track of your relations between tables, column types and even constraints (for a more complex databases).
One thing I liked (and needed, actually) was that it has built-in values for actual names, addresses etc. It helps so much when querying created test data and not get a random strings.
Also, you can export to SQL (or few other formats) and use the created data at any time to repopulate the database.
There is a program from red gate software which will do this for you. It's called SQL Data Generator.
We need step by step create query for tables need entry data. i used below codes, step by step for insert test data:
1. Create a table :
CREATE TABLE dbo.TestTableSize
(
MyKeyField VARCHAR(10) NOT NULL,
MyDate1 DATETIME NOT NULL,
MyDate2 DATETIME NOT NULL,
MyDate3 DATETIME NOT NULL,
MyDate4 DATETIME NOT NULL,
MyDate5 DATETIME NOT NULL
)
2. Variable Declarations
DECLARE #RowCount INT
DECLARE #RowString VARCHAR(10)
DECLARE #Random INT
DECLARE #Upper INT
DECLARE #Lower INT
DECLARE #InsertDate DATETIME
3.Set on time :
SET #Lower = -730
SET #Upper = -1
SET #RowCount = 0
4.Populate the Table :
WHILE #RowCount < 3000000
BEGIN
5.Preparing Values
SET #RowString = CAST(#RowCount AS VARCHAR(10))
SELECT #Random = ROUND(((#Upper - #Lower -1) * RAND() + #Lower), 0)
SET #InsertDate = DATEADD(dd, #Random, GETDATE())
6. Write insert statment :
INSERT INTO TestTableSize
(MyKeyField
,MyDate1
,MyDate2
,MyDate3
,MyDate4
,MyDate5)
VALUES
(REPLICATE('0', 10 - DATALENGTH(#RowString)) + #RowString
, #InsertDate
,DATEADD(dd, 1, #InsertDate)
,DATEADD(dd, 2, #InsertDate)
,DATEADD(dd, 3, #InsertDate)
,DATEADD(dd, 4, #InsertDate))
SET #RowCount = #RowCount + 1
END
7. Complete code :
DECLARE #RowCount INT
DECLARE #RowString VARCHAR(10)
DECLARE #Random INT
DECLARE #Upper INT
DECLARE #Lower INT
DECLARE #InsertDate DATETIME
SET #Lower = -730
SET #Upper = -1
SET #RowCount = 0
WHILE #RowCount < 3000000
BEGIN
SET #RowString = CAST(#RowCount AS VARCHAR(10))
SELECT #Random = ROUND(((#Upper - #Lower -1) * RAND() + #Lower), 0)
SET #InsertDate = DATEADD(dd, #Random, GETDATE())
INSERT INTO TestTableSize
(MyKeyField
,MyDate1
,MyDate2
,MyDate3
,MyDate4
,MyDate5)
VALUES
(REPLICATE('0', 10 - DATALENGTH(#RowString)) + #RowString
, #InsertDate
,DATEADD(dd, 1, #InsertDate)
,DATEADD(dd, 2, #InsertDate)
,DATEADD(dd, 3, #InsertDate)
,DATEADD(dd, 4, #InsertDate))
SET #RowCount = #RowCount + 1
END
Some flavours of Visual Studio have data generation built in.
If you use database projects in it you can create data generation plans. Here's the MSDN article
I used following way it basically copies data from itself , the data grows exponentially with every execution.Claveat is that You have to have some sample data at first and also you have to execute the query eg I had 327680 rows of data when i started with 10 rows of data .by executing the query just 16 times.Execute one more time and i will hage 655360 rows of data!
insert into mytable select [col1], [col2], [col3] from mytable

Resources