In SQL Server, is there any quick way to find out null column names for a particular record other than using CASE expression?
For eg:
I have one record whose values are like:
id|FName|LName|Dept
1 |NULL |Smith|NULL
Expected result is: FName, Dept
So, every time I will have only one record and I need to find the list of NULL columns for it.
if it's just a result set then you can't do it in sql-server.
but you can use other language making a query tool to do it like below C# code :
void Main(){
var nullColumnsName = GetNullColumnsName("select 1 id,null FName,'Smith' LName ,null Dept");
Console.WriteLine(nullColumnsName); //result : FName,Dept
}
IEnumerable<string> GetNullColumnsName(string sql)
{
using (var cnn = Connection)
{
if(cnn.State == ConnectionState.Closed) cnn.Open();
using(var cmd = cnn.CreateCommand()){
cmd.CommandText = sql;
using(var reader = cmd.ExecuteReader(CommandBehavior.SequentialAccess|CommandBehavior.SingleResult|CommandBehavior.SingleRow))
{
if(reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
var value = reader.GetValue(i);
if(value is DBNull)
yield return reader.GetName(i);
}
}
while (reader.Read()) {};
}
}
}
}
As per your this statement: "every time I will have only one record and I need to find the list of NULL columns for it"
You can store these values like follows:
Column | Values
FName | Null
LName | Smith
Dept | Null
and I assume that id is the primary key so as per your requirement if you want to store that then you can or if not, it's your choice.
SELECT Column FROM table
WHERE Values IS NULL;
You can use the following script. The script:
opens a cursor over all columns in the table definition
then, for each column, it queries all values in the column
if there is no non-null value in the column, it prints the name of the column
create table tmpTable (id int, FName varchar(32), LName varchar(32), Dept varchar(32))
insert into tmpTable values(1, null, 'Smith', null)
go
declare #column varchar(128)
declare #sql varchar(max)
declare c cursor for
select c.name from sys.tables t join sys.columns c on t.object_id = c.object_id
where t.name = 'tmpTable'
open c
fetch next from c into #column
while ##fetch_status = 0
begin
select #sql = 'if not exists (select * from tmpTable where ' + #column + ' is not null) begin print + ''' + #column + ''' end'
exec(#sql)
fetch next from c into #column
end
close c
deallocate c
go
drop table tmpTable
For your test case, this is the output of the script:
FName
Dept
Note: This script will work regardless how many rows you have in the table.
Edit:
I had written my answer before the following clarification was added "It is a result set". My solution works with a table (as I am querying the system catalogs) and will not cover the clarified scenario (using a result set).
Related
In t-sql my dilemma is that I have to parse a potentially long string (up to 500 characters) for any of over 230 possible values and remove them from the string for reporting purposes. These values are a column in another table and they're all upper case and 4 characters long with the exception of two that are 5 characters long.
Examples of these values are:
USFRI
PROME
AZCH
TXJS
NYDS
XVIV. . . . .
Example of string before:
"Offered to XVIV and USFRI as back ups. No response as of yet."
Example of string after:
"Offered to and as back ups. No response as of yet."
Pretty sure it will have to be a UDF but I'm unable to come up with anything other than stripping ALL the upper case characters out of the string with PATINDEX which is not the objective.
This is unavoidably cludgy but one way is to split your string into rows, once you have a set of words the rest is easy; Simply re-aggregate while ignoring the matching values*:
with t as (
select 'Offered to XVIV and USFRI as back ups. No response as of yet.' s
union select 'Another row AZCH and TXJS words.'
), v as (
select * from (values('USFRI'),('PROME'),('AZCH'),('TXJS'),('NYDS'),('XVIV'))v(v)
)
select t.s OriginalString, s.Removed
from t
cross apply (
select String_Agg(j.[value], ' ') within group(order by Convert(tinyint,j.[key])) Removed
from OpenJson(Concat('["',replace(s, ' ', '","'),'"]')) j
where not exists (select * from v where v.v = j.[value])
)s;
* Requires a fully-supported version of SQL Server.
build a function to do the cleaning of one sentence, then call that function from your query, something like this SELECT Col1, dbo.fn_ReplaceValue(Col1) AS cleanValue, * FROM MySentencesTable. Your fn_ReplaceValue will be something like the code below, you could also create the table variable outside the function and pass it as parameter to speed up the process, but this way is all self contained.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION fn_ReplaceValue(#sentence VARCHAR(500))
RETURNS VARCHAR(500)
AS
BEGIN
DECLARE #ResultVar VARCHAR(500)
DECLARE #allValues TABLE (rowID int, sValues VARCHAR(15))
DECLARE #id INT = 0
DECLARE #ReplaceVal VARCHAR(10)
DECLARE #numberOfValues INT = (SELECT COUNT(*) FROM MyValuesTable)
--Populate table variable with all values
INSERT #allValues
SELECT ROW_NUMBER() OVER(ORDER BY MyValuesCol) AS rowID, MyValuesCol
FROM MyValuesTable
SET #ResultVar = #sentence
WHILE (#id <= #numberOfValues)
BEGIN
SET #id = #id + 1
SET #ReplaceVal = (SELECT sValue FROM #allValues WHERE rowID = #id)
SET #ResultVar = REPLACE(#ResultVar, #ReplaceVal, SPACE(0))
END
RETURN #ResultVar
END
GO
I suggest creating a table (either temporary or permanent), and loading these 230 string values into this table. Then use it in the following delete:
DELETE
FROM yourTable
WHERE col IN (SELECT col FROM tempTable);
If you just want to view your data sans these values, then use:
SELECT *
FROM yourTable
WHERE col NOT IN (SELECT col FROM tempTable);
I wanted to check for null or empty GUID in stored procedure. I have used the below code:
CREATE PROCEDURE [dbo].[GetGuid]
#buildingID XML,
AS
DECLARE #fetchedBuildValue uniqueidentifier
SET #fetchedBuildValue = (SELECT data.item.value('./#ID', 'uniqueidentifier') AS PId
FROM #buildingID.nodes('/items/item') data(item));
SELECT
DemoUser.[ID],
DemoUser.[Suffix],
DemoUser.[BusinessPhoneNumber]
INNER JOIN
Relationship AS T_U ON T_U.Target = DemoUser.ID
AND T_U.TargetType = 0
AND T_U.SourceType = 6
INNER JOIN
DemoTenant ON T_U.Source = DemoTenant.ID
WHERE
(#firstname IS NULL OR DemoUser.FirstName LIKE '%' + #firstname + '%')
AND ((#fetchedBuildValue = '00000000-0000-0000-0000-000000000000'
AND Building.State = #state
AND Building.City = #city)
OR
(#fetchedBuildValue != '00000000-0000-0000-0000-000000000000'
AND Building.ID = #fetchedBuildValue))
The above stored procedure is not checking for empty guid. The guid that I am passing from code is in a xml as below:
<items><item ID="00000000-0000-0000-0000-000000000000" /></items>
However, if a valid guid is passed, it is working fine.
How can I add an empty check here so that when the guid is empty, my state and city where clause work instead of the guild column
I am looking for that performance 'sweet spot' when trying to update multiple columns for multiple rows...
Background.
I work in an MDM/abstract/hierarchical classification/functional SQL Server environment. This question pertains to a post data driven calculation process where I need to save the results. I pass JSON to a SQL function that will automatically create SQL for inserts/updates (and skip updates if the values match).
The tblDestination looks like
create table tblDestination as
(
sysPrimaryKey bigint identity(1,1)
, sysHeaderId bigint -- udt_ForeignKey
, sysLevel1Id bigint -- udt_ForeignKey (classification level 1)
, strText nvarchar(100) -- from here down: the values that need to be updated
, dtmDate datetime2(7)
, numNumeric flat
, flgFlag bit
, intInteger bigint
, sysRefKey bigint-- ForeignKey
, primary key non clustered (sysPrimaryKey)
)
/* note that the clustered index on this table exists, contains more than the columns listed above, and is physically modeled correctly. you may use any clustered/IX/UI indexes that you need to if you are testing */
#JSON looks like... (ARBITRARY# ranges between 2 and 100)
declare #JSON nvarchar(max)='{"ARBITRARY NAME 1":"3/1/2017","ARBITRARY NAME 2": "Value", "ARBITRARY NAME 3": 45.3}'
The function cursors through the incoming #JSON and builds insert or update statements.
Cursor local static forward_only read_only
for
select [key] as JSON_Key, [value] from openjson(#json)
while ##fetch
-- get the id for ARBITRARY ID plus a classification id
select #level1Id = level1Id, #level2id = level2Id
from tblLevel1 where ProgrammingName = #JSON_Key
-- get a ProgrammingName field for the previously retrieved level2Id
Select #ProgrammingName = /*INTEGER/FLAG/NUMERIC/TEXT/DATE/REFKEY
from tblLevel2 where level2id = #level2id
-- clear variables
set #numeric = null, #integer = null, #text = null etc..
-- check to see if insert or update is required
Select #DestinationID from tblDestination where HeaderId = #header and Level1 = #Level1
if #DestinationId is null
begin
If #ProgrammingName = 'Numeric' begin #Numeric = #JSON_Value end
else if #ProgrammingName = 'Integer' begin #Integer = #JSON_value end
etc..
-- dynamically build the updates here..
/*
'update tblDestination
Set numeric = ' +#numeric
+', flag = '+#flag
+', date = '+#date
.. etc
+'where HeaderId = '+#header + ' and level1Id = '+#Level1Id
end
IE:
Update tblDestination
Set numNumeric = NULL
, flgFlag = NULL
, dtmDate = '3/1/2017'
Where sysPrimaryKey = 33676224
*/
Finally... to the point of this post: has anyone here had experience with multiple row updates on multiple columns?
Something like:
Set TableNumeric
= CASE WHEN Level3Id = 33676224 then null
when leve3id = 33676225 then 3.2
when level3id = 33676226 then null
end
, tableDate = case when level3id = 33676224 then '3/1/2017'
when 33676225 then null
when 33676226 then null
end
where headerId = 23897
and IDs in (33676224, 33676225, 33676226)
I know that the speed varies for Insert statements (Number of Columns inserted vs Number of records), and have that part dialed in.
I am curious to know if anyone has found that 'sweet spot' for updates.
The sweet spot meaning:
How many CASES before I should make a new update block?
Is 'Update tbl set (Column = Case (When Id = ## then ColumnValue)^n END)^n' the proper approach to reduce the number of actual Updates being fired?
Is wrapping Updates in a transaction a faster option (and how many per COMMIT)?
Legibility of the update statement is irrelevant. Nobody will actually see the code.
I have isolated the single update statement chain to be approx 70%+ of the query cost in question (compared to all inserts and 20/80,50/50,80/20 %update/%inserts)
I have the following stored procedure that is quite extensive because of the dynamic #Name parameter and the sub query.
Is there a better more efficient way to do this?
CREATE PROCEDURE [dbo].[spGetClientNameList]
#Name varchar(100)
AS
BEGIN
SET NOCOUNT ON;
SELECT
*
FROM
(
SELECT
ClientID,
FirstName + ' ' + LastName as Name
FROM
Client
) a
where a.Name like '%' + #Name + '%'
Shamelessly stealing from two recent articles by Aaron Bertrand:
Follow-up #1 on leading wildcard seeks - Aaron Bertrand
One way to get an index seek for a leading %wildcard - Aaron Bertrand
The jist is to create something that we can use that resembles a trigram (or trigraph) in PostgreSQL.
Aaron Bertrand also includes a disclaimer as follows:
"Before I start to show how my proposed solution would work, let me be absolutely clear that this solution should not be used in every single case where LIKE '%wildcard%' searches are slow. Because of the way we're going to "explode" the source data into fragments, it is likely limited in practicality to smaller strings, such as addresses or names, as opposed to larger strings, like product descriptions or session abstracts."
test setup: http://rextester.com/IIMT54026
Client table
create table dbo.Client (
ClientId int not null primary key clustered
, FirstName varchar(50) not null
, LastName varchar(50) not null
);
insert into dbo.Client (ClientId, FirstName, LastName) values
(1, 'James','')
, (2, 'Aaron','Bertrand')
go
Function used by Aaron Bertrand to explode string fragments (modified for input size):
create function dbo.CreateStringFragments(#input varchar(101))
returns table with schemabinding
as return
(
with x(x) as (
select 1 union all select x+1 from x where x < (len(#input))
)
select Fragment = substring(#input, x, len(#input)) from x
);
go
Table to store fragments for FirstName + ' ' + LastName:
create table dbo.Client_NameFragments (
ClientId int not null
, Fragment varchar(101) not null
, constraint fk_ClientsNameFragments_Client
foreign key(ClientId) references dbo.Client
on delete cascade
);
create clustered index s_cat on dbo.Client_NameFragments(Fragment, ClientId);
go
Loading the table with fragments:
insert into dbo.Client_NameFragments (ClientId, Fragment)
select c.ClientId, f.Fragment
from dbo.Client as c
cross apply dbo.CreateStringFragments(FirstName + ' ' + LastName) as f;
go
Creating trigger to maintain fragments:
create trigger dbo.Client_MaintainFragments
on dbo.Client
for insert, update as
begin
set nocount on;
delete f from dbo.Client_NameFragments as f
inner join deleted as d
on f.ClientId = d.ClientId;
insert dbo.Client_NameFragments(ClientId, Fragment)
select i.ClientId, fn.Fragment
from inserted as i
cross apply dbo.CreateStringFragments(i.FirstName + ' ' + i.LastName) as fn;
end
go
Quick trigger tests:
/* trigger tests --*/
insert into dbo.Client (ClientId, FirstName, LastName) values
(3, 'Sql', 'Zim')
update dbo.Client set LastName = 'unknown' where LastName = '';
delete dbo.Client where ClientId = 3;
--select * from dbo.Client_NameFragments order by ClientId, len(Fragment) desc
/* -- */
go
New Procedure:
create procedure [dbo].[Client_getNameList] #Name varchar(100) as
begin
set nocount on;
select
ClientId
, Name = FirstName + ' ' + LastName
from Client c
where exists (
select 1
from dbo.Client_NameFragments f
where f.ClientId = c.ClientId
and f.Fragment like #Name+'%'
)
end
go
exec [dbo].[Client_getNameList] #Name = 'On Bert'
returns:
+----------+----------------+
| ClientId | Name |
+----------+----------------+
| 2 | Aaron Bertrand |
+----------+----------------+
I guess search operation on Concatenated column wont take Indexes sometimes. I got situation like above and I replaced the Concatenated search with OR which gave me better performance most of the times.
Create Non Clustered Indexes on FirstName and LastName if not present.
Check the performance after modifying the above Procedure like below
CREATE PROCEDURE [dbo].[spGetClientNameList]
#Name varchar(100)
AS
BEGIN
SET NOCOUNT ON;
SELECT
ClientID,
FirstName + ' ' + LastName as Name
FROM
Client
WHERE FirstName LIKE '%' + #Name + '%'
OR LastName LIKE '%' + #Name + '%'
END
And do check execution plans to verify those Indexes are used or not.
The problem really comes down to having to compute the column (concat the first name and last name), that pretty much forces sql server into doing a full scan of the table to determine what is a match and what isn't. If you're not allowed to add indexes or alter the table, you'll have to change the query around (supply firstName and lastName separately). If you are, you could add a computed column and index that:
Create Table client (
ClientId INT NOT NULL PRIMARY KEY IDENTITY(1,1)
,FirstName VARCHAR(100)
,LastName VARCHAR(100)
,FullName AS FirstName + ' ' + LastName
)
Create index FullName ON Client(FullName)
This will at least speed your query up by doing index seeks instead of full table scans. Is it worth it? It's difficult to say without looking at how much data there is, etc.
where a.Name like '%' + #Name + '%'
This statement never can use index. In this situation it's beter to use Full Text Search
if you can restrict your like to
where a.Name like #Name + '%'
it will use index automaticaly. Moreover you can use REVERSE() function to index statement like :
where a.Name like '%' + #Name
What I need to achieve is to send a list of unknown QTY of values to a Sql server NOT IN clause but can only achieve this with singular values. below is my Sql statement:
SELECT SorMaster.LastInvoice
, SorMaster.SalesOrder
, SorMaster.OrderStatus
, ArCustomer.RouteCode
, SorMaster.Customer
, SorMaster.CustomerName
, SorMaster.CustomerPoNumber
, SorMaster.OrderDate
, SorMaster.DateLastInvPrt
, ArInvoice.InvoiceBal1
, ArInvoice.TermsCode
FROM SorMaster AS SorMaster
INNER JOIN ArCustomer AS ArCustomer ON SorMaster.Customer = ArCustomer.Customer
INNER JOIN ArInvoice AS ArInvoice ON SorMaster.LastInvoice = ArInvoice.Invoice
WHERE (SorMaster.OrderStatus = '9')
AND (SorMaster.Branch LIKE 'J%')
AND (SorMaster.DocumentType = 'O')
AND (SorMaster.LastInvoice > #Last_Invoice)
AND (SorMaster.OrderDate > DATEADD(Month, - 4, GETDATE()))
AND (SorMaster.LastInvoice NOT IN (#ExclusionList))
ORDER BY SorMaster.LastInvoice
The #ExclusionList value is generated by this code as a string from a listbox:
Dim exclusion As String = ""
If MenuForm.ExclusionCB.Checked = True Then
For i = 0 To MenuForm.ExclusionLB.Items.Count - 2
exclusion = exclusion & MenuForm.ExclusionLB.Items(i) & ","
Next
exclusion = exclusion & MenuForm.ExclusionLB.Items(MenuForm.ExclusionLB.Items.Count - 1)
Else
exclusion = ""
End If
I have also tried sending the entire listbox as a collection.
Does anyone know how I can send more than one value (something like 1,2,3,4,5,6) and have sql understand that these is more than one? I won't have an issue with the SELECT statement changing, just as long as it returns the same information.
The reason I need this with the exception list, is our remote DB PK is on the Salesorder column and the local DB is on the LastInvoice column
Hope this makes sense. if you need more info, please let me know
You can send it as a string and use dynamic sql. Here's a simple example how to do that.
DECLARE #vals VARCHAR(50) = '1,2,3,4,5,6'
DECLARE #sql VARCHAR(MAX) = 'SELECT * FROM TABLE WHERE FIELD1 IN'
SET #sql = #sql + ' (' + #vals + ')'
-- #sql = 'SELECT * FROM TABLE WHERE FIELD1 IN (1,2,3,4,5,6)'
EXEC (#sql)