Use master.sys.fn_varbintohexsubstring in computed column - sql-server

In my sql server table, I want to add a computed column that is a hash of other columns in the same table. Below is my table structure.
Address:
AddressID(int , PK)
AddressLine1 (nvarchar)
AddressLine2 (nvarchar)
City (nvarchar)
State (nvarchar)
AddressHash(computed column)
Below is what I want to get in my computed column:
MASTER.SYS.FN_VARBINTOHEXSUBSTRING(0, HASHBYTES('SHA1',COALESCE(AddressLine1, N'') + COALESCE(AddressLine2, N'') + COALESCE(City, N'') + COALESCE(State, N'')), 1, 0)
If I right-click the table and go to design and enter the above for "Formula" under "Computed Column Specification", I get the following error:
- Unable to modify table.
A user-defined function name cannot be prefixed with a database name in this context.
So I thought I would use a user defined function to calculate the hash and map that udf to formula.
Below is the code that I am using to create UDF:
CREATE FUNCTION udfHashAddress
(
#pAddressLine1 nvarchar(50), #pAddressLine2 nvarchar(50), #pCity nvarchar(50), #pState nvarchar(50))
)
RETURNS nvarchar(max) -- not sure what the correct size would be
WITH SCHEMABINDING
AS
BEGIN
DECLARE #result nvarchar(max)
SELECT #result = MASTER.SYS.FN_VARBINTOHEXSUBSTRING(0, HASHBYTES('SHA1',COALESCE(#pAddressLine1, N'') + COALESCE(#pAddressLine2, N'') + COALESCE(#pCity, N'') + COALESCE(#pState, N'')), 1, 0)
RETURN #result
END
GO
But I get the following error with the above code:
*Cannot schema bind function 'udfHashAddress' because name 'MASTER.SYS.FN_VARBINTOHEXSUBSTRING' is invalid for schema binding. Names must be in two-part format and an object cannot reference itself.*
When I removed the "MASTER" db prefix I got this error:
*Cannot schema bind function 'udfHashAddress' because it references system object 'SYS.FN_VARBINTOHEXSUBSTRING'.*
Am I missing something here? Would appreciate any assistance/pointers.

Since you're using SQL Server 2008, have you tried simply:
CONVERT(VARCHAR(MAX), HASHBYTES('SHA1','string'), 2);
This will return upper case instead of lower case letters, but you can fix that with LOWER() if it's important.
Here is an example with your actual schema (created in tempdb on purpose):
USE tempdb;
GO
CREATE TABLE dbo.[Address]
(
AddressID INT PRIMARY KEY,
AddressLine1 NVARCHAR(64),
AddressLine2 NVARCHAR(64),
City NVARCHAR(64),
[State] NVARCHAR(64),
AddressHash AS LOWER(CONVERT(VARCHAR(4000), HASHBYTES('SHA1',
COALESCE(AddressLine1, N'') + COALESCE(AddressLine2, N'')
+ COALESCE(City, N'') + COALESCE([State], N'')), 2))
--PERSISTED -- you could also persist it if desired
);
INSERT dbo.[Address]
VALUES(1, 'foo', 'bar', 'blat', 'splunge'),
(2, 'bar', 'foo', 'blag', 'splmger');
SELECT *, master.dbo.fn_varbintohexsubstring
(0,
HASHBYTES
(
'SHA1',
COALESCE(AddressLine1, N'') + COALESCE(AddressLine2, N'')
+ COALESCE(City, N'') + COALESCE([State], N'')
), 1, 0)
FROM dbo.[Address];
GO
DROP TABLE dbo.[Address];

Related

SQL Server - parameter with unknown number of multiple values

I'm creating a grid that has two columns: Name and HotelId. The problem is that data for this grid should be sent with a single parameter of VARCHAR type and should look like this:
#Parameter = 'Name1:5;Name2:10;Name3:6'
As you can see, the parameter contains Name and a number that represents ID value and you can have multiple such entries, separated by ";" symbol.
My first idea was to write a query that creates a temp table that will have two columns and populate it with data from the parameter.
How could I achieve this? It seems like I need to split the parameter two times: by the ";" symbol for each row and then by ":" symbol for each column.
How should I approach this?
Also, if there is any other more appropriate solution, I'm open to suggestions.
First Drop the #temp table if Exists...
IF OBJECT_ID('tempdb..#temp', 'U') IS NOT NULL
/*Then it exists*/
DROP TABLE #temp
Then create #temp table
CREATE TABLE #temp (v1 VARCHAR(100))
Declare all the #Paramter....
DECLARE #Parameter VARCHAR(50)
SET #Parameter= 'Name1:5;Name2:10;Name3:6'
DECLARE #delimiter nvarchar(1)
SET #delimiter= N';';
Here, Inserting all #parameter value into #temp table using ';' separated..
INSERT INTO #temp(v1)
SELECT * FROM(
SELECT v1 = LTRIM(RTRIM(vals.node.value('(./text())[1]', 'nvarchar(4000)')))
FROM (
SELECT x = CAST('<root><data>' + REPLACE(#Parameter, #delimiter, '</data><data>') + '</data></root>' AS XML).query('.')
) v
CROSS APPLY x.nodes('/root/data') vals(node)
)abc
After inserting the value into #temp table..get all the value into ':' seprated...
select Left(v1, CHARINDEX(':', v1)-1) as Name , STUFF(v1, 1, CHARINDEX(':', v1), '') as HotelId FROM #temp
Then you will get this type of Output

Queries with Dynamic Parameters - better ways?

I have the following stored procedure that is quite extensive because of the dynamic #Name parameter and the sub query.
Is there a better more efficient way to do this?
CREATE PROCEDURE [dbo].[spGetClientNameList]
#Name varchar(100)
AS
BEGIN
SET NOCOUNT ON;
SELECT
*
FROM
(
SELECT
ClientID,
FirstName + ' ' + LastName as Name
FROM
Client
) a
where a.Name like '%' + #Name + '%'
Shamelessly stealing from two recent articles by Aaron Bertrand:
Follow-up #1 on leading wildcard seeks - Aaron Bertrand
One way to get an index seek for a leading %wildcard - Aaron Bertrand
The jist is to create something that we can use that resembles a trigram (or trigraph) in PostgreSQL.
Aaron Bertrand also includes a disclaimer as follows:
"Before I start to show how my proposed solution would work, let me be absolutely clear that this solution should not be used in every single case where LIKE '%wildcard%' searches are slow. Because of the way we're going to "explode" the source data into fragments, it is likely limited in practicality to smaller strings, such as addresses or names, as opposed to larger strings, like product descriptions or session abstracts."
test setup: http://rextester.com/IIMT54026
Client table
create table dbo.Client (
ClientId int not null primary key clustered
, FirstName varchar(50) not null
, LastName varchar(50) not null
);
insert into dbo.Client (ClientId, FirstName, LastName) values
(1, 'James','')
, (2, 'Aaron','Bertrand')
go
Function used by Aaron Bertrand to explode string fragments (modified for input size):
create function dbo.CreateStringFragments(#input varchar(101))
returns table with schemabinding
as return
(
with x(x) as (
select 1 union all select x+1 from x where x < (len(#input))
)
select Fragment = substring(#input, x, len(#input)) from x
);
go
Table to store fragments for FirstName + ' ' + LastName:
create table dbo.Client_NameFragments (
ClientId int not null
, Fragment varchar(101) not null
, constraint fk_ClientsNameFragments_Client
foreign key(ClientId) references dbo.Client
on delete cascade
);
create clustered index s_cat on dbo.Client_NameFragments(Fragment, ClientId);
go
Loading the table with fragments:
insert into dbo.Client_NameFragments (ClientId, Fragment)
select c.ClientId, f.Fragment
from dbo.Client as c
cross apply dbo.CreateStringFragments(FirstName + ' ' + LastName) as f;
go
Creating trigger to maintain fragments:
create trigger dbo.Client_MaintainFragments
on dbo.Client
for insert, update as
begin
set nocount on;
delete f from dbo.Client_NameFragments as f
inner join deleted as d
on f.ClientId = d.ClientId;
insert dbo.Client_NameFragments(ClientId, Fragment)
select i.ClientId, fn.Fragment
from inserted as i
cross apply dbo.CreateStringFragments(i.FirstName + ' ' + i.LastName) as fn;
end
go
Quick trigger tests:
/* trigger tests --*/
insert into dbo.Client (ClientId, FirstName, LastName) values
(3, 'Sql', 'Zim')
update dbo.Client set LastName = 'unknown' where LastName = '';
delete dbo.Client where ClientId = 3;
--select * from dbo.Client_NameFragments order by ClientId, len(Fragment) desc
/* -- */
go
New Procedure:
create procedure [dbo].[Client_getNameList] #Name varchar(100) as
begin
set nocount on;
select
ClientId
, Name = FirstName + ' ' + LastName
from Client c
where exists (
select 1
from dbo.Client_NameFragments f
where f.ClientId = c.ClientId
and f.Fragment like #Name+'%'
)
end
go
exec [dbo].[Client_getNameList] #Name = 'On Bert'
returns:
+----------+----------------+
| ClientId | Name |
+----------+----------------+
| 2 | Aaron Bertrand |
+----------+----------------+
I guess search operation on Concatenated column wont take Indexes sometimes. I got situation like above and I replaced the Concatenated search with OR which gave me better performance most of the times.
Create Non Clustered Indexes on FirstName and LastName if not present.
Check the performance after modifying the above Procedure like below
CREATE PROCEDURE [dbo].[spGetClientNameList]
#Name varchar(100)
AS
BEGIN
SET NOCOUNT ON;
SELECT
ClientID,
FirstName + ' ' + LastName as Name
FROM
Client
WHERE FirstName LIKE '%' + #Name + '%'
OR LastName LIKE '%' + #Name + '%'
END
And do check execution plans to verify those Indexes are used or not.
The problem really comes down to having to compute the column (concat the first name and last name), that pretty much forces sql server into doing a full scan of the table to determine what is a match and what isn't. If you're not allowed to add indexes or alter the table, you'll have to change the query around (supply firstName and lastName separately). If you are, you could add a computed column and index that:
Create Table client (
ClientId INT NOT NULL PRIMARY KEY IDENTITY(1,1)
,FirstName VARCHAR(100)
,LastName VARCHAR(100)
,FullName AS FirstName + ' ' + LastName
)
Create index FullName ON Client(FullName)
This will at least speed your query up by doing index seeks instead of full table scans. Is it worth it? It's difficult to say without looking at how much data there is, etc.
where a.Name like '%' + #Name + '%'
This statement never can use index. In this situation it's beter to use Full Text Search
if you can restrict your like to
where a.Name like #Name + '%'
it will use index automaticaly. Moreover you can use REVERSE() function to index statement like :
where a.Name like '%' + #Name

How to find exact row where SQL query fail

I'm using SQL Server 2008 R2. I have 2 tables old and new and about 500k rows.
I need to convert data from old to new. Some columns were changed. For example in old table many columns are of type varchar and in new table int.
I'm executing query like this:
INSERT INTO new (xxx)
SELECT FROM old (yyy)
And get following error:
Msg 245, Level 16, State 1, Line 4
Conversion failed when converting the nvarchar value 'Tammi ' to data type int.
This error shows, that in old table are some rows with wrong data in columns. (Human factor).
But how can I find these wrong rows? Is it possible?
How can I find in what column wrong data is present?
This is a pain. But, to find values that cannot be converted to ints, try this:
select yyyy
from old
where yyyy like '%[^0-9]%';
In SQL Server 2012+, you can use try_convert():
select yyyy
from old
where try_convert(int, yyyy) is null;
Could you execute the code that this T-SQL statement generates (just change the table name):
DECLARE #TableName SYSNAME = 'DataSource'
SELECT 'SELECT * FROM ' + #TableName + ' WHERE ' +
STUFF
(
(
SELECT 'OR ISNUMERIC([' + name + '] + ''.e0'') = 0 '
FROM sys.columns
WHERE object_id = OBJECT_ID(#TableName)
FOR XML PATH(''), TYPE
).value('.', 'VARCHAR(MAX)')
,1
,3
,''
);
For, example, if we have the following table:
IF OBJECT_ID('DataSource') IS NOT NULL
BEGIN
DROP TABLE DataSource;
END;
GO
CREATE TABLE DataSource
(
A VARCHAR(12)
,B VARCHAR(12)
,C VARCHAR(12)
);
GO
INSERT DataSource ([A], [B], [C])
VALUES ('1', '2', '3')
,('0.5', '4', '2')
,('1', '2', 'A');
GO
The script will generate this statement:
SELECT * FROM DataSource WHERE ISNUMERIC([A] + '.e0') = 0 OR ISNUMERIC([B] + '.e0') = 0 OR ISNUMERIC([C] + '.e0') = 0
returning two of the rows (because A and 0.5 cannot be converted to int):

How to bulk import lat, long from csv file into sql server as spatial data type?

How can I bulk import lat, long from csv file into sql server as spatial data type, not two separate columns of type double?
Given input .csv file with two columns:
Latitude, Longitude
want to create sql db that has one column corresponding to a spatial data type.
Did some research and found this article:
Convert Latitude/Longitude (Lat/Long) to Geography Point
So as given in article, I've created table and inserted some test data using given script:
CREATE TABLE [dbo].[Landmark] (
[ID] INT IDENTITY(1, 1),
[LandmarkName] VARCHAR(100),
[Location] VARCHAR(50),
[Latitude] FLOAT,
[Longitude] FLOAT
)
GO
INSERT INTO [dbo].[Landmark] ( [LandmarkName], [Location], [Latitude], [Longitude] )
VALUES ( 'Statue of Liberty', 'New York, USA', 40.689168,-74.044563 ),
( 'Eiffel Tower', 'Paris, France', 48.858454, 2.294694),
( 'Leaning Tower of Pisa', 'Pisa, Italy', 43.72294, 10.396604 ),
( 'Great Pyramids of Giza', 'Cairo, Egypt', 29.978989, 31.134632 ),
( 'Sydney Opera House', 'Syndey, Australia', -33.856651, 151.214967 ),
( 'Taj Mahal', 'Agra, India', 27.175047, 78.042042 ),
( 'Colosseum', 'Rome, Italy', 41.890178, 12.492378 )
GO
And then added calculated column using this query:
ALTER TABLE [dbo].[Landmark]
ADD [GeoLocation] AS geography::STPointFromText('POINT(' + CAST([Longitude] AS VARCHAR(20)) + ' ' + CAST([Latitude] AS VARCHAR(20)) + ')', 4326)
GO
Now when I query table using
SELECT *
FROM dbo.Landmark
I'm getting calculated spatial results too:
And results in axis:
Hopefully I understood you right.
Update:
I'm not sure if this will satisfy you. It's quite dirty, but it does the job:
That's how I formatted CSV file. I've used same structure as in previous example:
Statue of Liberty| New York, USA| 40.689168|-74.044563
Eiffel Tower| Paris, France| 48.858454| 2.294694
Leaning Tower of Pisa| Pisa, Italy| 43.72294| 10.396604
Great Pyramids of Giza| Cairo, Egypt| 29.978989| 31.134632
Sydney Opera House| Syndey, Australia| -33.856651| 151.214967
Taj Mahal| Agra, India| 27.175047| 78.042042
Colosseum| Rome, Italy| 41.890178| 12.492378
Columns seperator is | symbol and rows seperator is break symbol.
So what I did is, I used OPENROWSET to open CSV file and format this into rows instead having one long string( that's how OPENROWSET read my csv file, unfortunately). I've used this SplitString function:
https://stackoverflow.com/a/19935594/3680098
Now I need to turn these buddies into columns instead of one string. I've used this answer provided on SO: https://stackoverflow.com/a/15108499/3680098
Summing things up, that's the query:
SELECT LTRIM(RTRIM(xmlValue.value('/values[1]/value[1]','nvarchar(100)'))) AS LandmarkName
, LTRIM(RTRIM(xmlValue.value('/values[1]/value[2]','nvarchar(100)'))) AS Location
, LTRIM(RTRIM(xmlValue.value('/values[1]/value[3]','nvarchar(20)'))) AS Lon
, LTRIM(RTRIM(xmlValue.value('/values[1]/value[4]','nvarchar(20)'))) AS Lat
, GEOGRAPHY::STPointFromText('POINT(' + xmlValue.value('/values[1]/value[4]','nvarchar(20)') + ' ' + xmlValue.value('/values[1]/value[3]','nvarchar(20)') + ')', 4326)
FROM dbo.SplitString((SELECT Document.* FROM OPENROWSET(BULK N'C:\Temp\test.csv', SINGLE_CLOB) AS Document), CHAR(10)) AS T
CROSS APPLY (SELECT CAST('<values><value>' + REPLACE(T.Value, '|', '</value><value>') + '</value></values>' AS XML)) AS T1(xmlValue);
It generates me required data as in my first screenshot and it seems just fine.
So what I need to do, is to create my table and insert these into it:
CREATE TABLE [dbo].[Landmark] (
[ID] INT IDENTITY(1, 1),
[LandmarkName] VARCHAR(100),
[Location] VARCHAR(50),
[GeoLocation] GEOGRAPHY
)
GO
INSERT INTO dbo.Landmark (LandmarkName, Location, GeoLocation)
SELECT LTRIM(RTRIM(xmlValue.value('/values[1]/value[1]','nvarchar(100)'))) AS LandmarkName
, LTRIM(RTRIM(xmlValue.value('/values[1]/value[2]','nvarchar(100)'))) AS Location
, GEOGRAPHY::STPointFromText('POINT(' + xmlValue.value('/values[1]/value[4]','nvarchar(20)') + ' ' + xmlValue.value('/values[1]/value[3]','nvarchar(20)') + ')', 4326)
FROM dbo.SplitString((SELECT Document.* FROM OPENROWSET(BULK N'C:\Temp\test.csv', SINGLE_CLOB) AS Document), CHAR(10)) AS T
CROSS APPLY (SELECT CAST('<values><value>' + REPLACE(T.Value, '|', '</value><value>') + '</value></values>' AS XML)) AS T1(xmlValue)
Results:

Error trying to insert data using trigger T-SQL

Below is a trigger used on one of our SQL tables for any insert/update action. 99/100 times this trigger works just fine however every now and then we receive this error message:
Cannot insert the value NULL into column 'TransactionDate', table
'AgentResourcesU01.dbo.TransactionLog'; column does not allow nulls.
INSERT fails. The statement has been terminated.
As you can see from the Insert statement, the columns in our transaction log table are TransactionDate, Operator, TableName, Action, TableString and UserId. I set the variable #transDate in the opening SELECT statement so as it appears to me, there should be no way a NULL gets in there unless it's bad data coming in.
Any thoughts?
BEGIN
SELECT #symetraNumber = SymetraNumber, #lastChangeOperator = LastChangeOperator, #transDate = LastChangeDate, #entityType = EntityType,
#firstName = FirstName, #lastName = LastName, #suffix = NameSuffix, #corpName = CorporateName, #controlId = ControlId
FROM inserted
IF #firstName IS NULL SET #firstName = 'NULL'
IF #lastName IS NULL SET #lastName = 'NULL'
IF #suffix IS NULL SET #suffix = 'NULL'
IF #corpName IS NULL SET #corpName = 'NULL'
IF #controlId IS NULL SET #controlId = 'NULL'
SET #tableString = 'SymNum:' + #symetraNumber + ' EntType:' + #entityType + ' Fname:' + #firstName + ' Lname:' + #lastname + ' Suff:' + #suffix +
' CorpName:' + #corpName + ' ctrlId:' + #controlId
INSERT INTO TransactionLog (TransactionDate, Operator, TableName, Action, TableString, UserId)
VALUES (#transDate, 'Op', #tableName, #action, #tableString, #lastChangeOperator)
END
To demonstrate Marc's point, you can do this in a set-based way, without all these nasty variables and IF checks:
INSERT dbo.TransactionLog
(
TransactionDate,
Operator,
TableName,
Action,
TableString,
UserId
)
SELECT
LastChangeDate,
'Op',
#TableName,
#action,
'SymNum:' + COALESCE(SymetraNumber, 'NULL')
+ ' EntType:' + COALESCE(EntityType, 'NULL')
+ ' Fname:' + COALESCE(FirstName, 'NULL')
+ ' Lname:' + COALESCE(LastName, 'NULL')
+ ' Suff:' + COALESCE(NameSuffix, 'NULL')
+ ' CorpName:' + COALESCE(CorporateName, 'NULL')
+ ' ctrlId:' + COALESCE(ControlId, 'NULL'),
LastChangeOperator
FROM inserted;
If LastChangeDate in the underlying table is NULLable, either mark it as NOT NULL, fix the problem where NULL is getting inserted, or both. The trigger shouldn't have to know about this constraint but you can work around it by doing something like this (if the value is NULL, set it to right now):
...
UserId
)
SELECT
COALESCE(LastChangeDate, CURRENT_TIMESTAMP),
'Op',
...
[1] I assume that you have an INSERT/UPDATE/DELETE trigger and when somebody try to delete rows from the base table then the inserted table from trigger will have zero rows. Look at this example:
CREATE TABLE MyTableWithTrigger (
MyID INT IDENTITY PRIMARY KEY,
LastUpdateDate DATETIME NOT NULL
);
GO
CREATE TABLE TransactionLog (
TransactionLogID INT IDENTITY PRIMARY KEY,
CreateDate DATETIME NOT NULL DEFAULT GETDATE(),
LastUpdateDate DATETIME NOT NULL,
MyID INT NOT NULL
);
GO
CREATE TRIGGER trIUD_MyTableWithTrigger_Audit
ON MyTableWithTrigger
AFTER INSERT, UPDATE, DELETE
AS
BEGIN
DECLARE #LastUpdateDate DATETIME, #MyID INT;
SELECT #LastUpdateDate=i.LastUpdateDate, #MyID=i.MyID
FROM inserted i;
INSERT TransactionLog (LastUpdateDate, MyID)
VALUES (#LastUpdateDate, #MyID)
END;
GO
PRINT 'Test 1'
INSERT MyTableWithTrigger (LastUpdateDate)
VALUES ('2011-01-01');
DELETE MyTableWithTrigger;
SELECT * FROM MyTableWithTrigger;
SELECT * FROM TransactionLog;
Output:
Msg 515, Level 16, State 2, Procedure trIUD_MyTableWithTrigger_Audit, Line 10
Cannot insert the value NULL into column 'LastUpdateDate', table 'Test.dbo.TransactionLog'; column does not allow nulls. INSERT fails.
The statement has been terminated.
[2] To correct your trigger (see Marc's comment) please read pages 192 - 199 from Alex Kuznetsov's book (Defensive Database Programming with SQL Server: Amazon, Red Gate - PDF).

Resources