Compare XML to string? - sql-server

I have a table with an XML column. The vast majority of rows contain this data:
<X C="1"></X>
I would like to find any row among the millions that does not match this. So...
select *
from DataResults
where cast(baserentamount as varchar(250)) not like '<X C="1"></X>'
This returns every row. I assume there is something very simple wrong here?
Update: perhaps I should reverse the sense of this question - what is the easiest way to do the query I want, IE, match the entry in an XML column in a WHERE?

You should look at the actual XML in your table. If you insert that xml into a table it is not the same string you think it is.
declare #Something table(baserentamount xml)
insert #Something select '<X C="1"></X>'
select *
, cast(baserentamount as varchar(250))
from #Something
Therefore your query will very likely return every row. And it will certainly return the rows you want to exclude because the xml string has changed.

Ahhh, well there it is, the devil is in the details:
XML as presented:
<X C="1" />
XML after CAST:
<X C="1"/>
Not entirely obvious, nor obvious why this should be, but there it is, it's a WS problem.

Any approach to do this on string level is wrong (due to different XML layouts, which are semantically equal and therefore possible at any stage).
Try something along this:
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata XML);
INSERT INTO #tbl (xmldata) VALUES
(N'<X C="1"></X>')
,(N'<X A="huh?" C="1"></X>')
,(N'<X C="2"></X>')
,(N'<X C="1"></X><X C="2"></X>')
,(N'<Y C="1"></Y>')
,(N'<root>some other</root>')
;
SELECT t.*
,CASE WHEN t.xmldata.query(N'count(/*)>1').value('.','bit')=1 THEN 'X' END MoreThanOneNodeInLevel1
,CASE WHEN t.xmldata.query(N'count(/*[1]/#*)>1').value('.','bit')=1 THEN 'X' END MoreThanOnAttributeInFirstElement
,CASE WHEN t.xmldata.query(N'/X[1]/#C != "1"').value('.','bit')=1 THEN 'X' END Attribute_X_C_isNot1
,CASE WHEN t.xmldata.query(N'local-name(/*[1])!="X"').value('.','bit')=1 THEN 'X' END FirstElementIsNotNamedX
FROM #tbl t;
The result
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| ID | xmldata | MoreThanOneNodeInLevel1 | MoreThanOnAttributeInFirstElement | Attribute_X_C_isNot1 | FirstElementIsNotNamedX |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 1 | <X C="1" /> | NULL | NULL | NULL | NULL |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 2 | <X A="huh?" C="1" /> | NULL | X | NULL | NULL |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 3 | <X C="2" /> | NULL | NULL | X | NULL |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 4 | <X C="1" /><X C="2" /> | X | NULL | NULL | NULL |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 5 | <Y C="1" /> | NULL | NULL | NULL | X |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
| 6 | <root>some other</root> | NULL | NULL | NULL | X |
+----+-------------------------+-------------------------+-----------------------------------+----------------------+-------------------------+
As you can see, just the first row is NULL in all columns.
The idea in short:
Your XML has got
- 1 single element at level 1
- This element has got 1 single attribute
- The "C"-attribute's value in the first element="1"
- The local-name of the first element is "X"
In the sample data I added various "wrong" XML instances.
You might need to add more rules.
Hint: You can place the expression in a WHERE clause or use XML.exist().

The pure XML way to handle what you need would be to create a tiny XML Schema. After that you can validate that XML column against it.
SQL
USE tempdb;
GO
-- DDL and sample data population, start
IF EXISTS (SELECT * FROM sys.xml_schema_collections
WHERE name = N'MySchema'
AND schema_id = SCHEMA_ID(N'dbo'))
DROP XML SCHEMA COLLECTION dbo.MySchema;
CREATE XML SCHEMA COLLECTION dbo.MySchema
AS N'<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="X">
<xsd:complexType>
<xsd:attribute name="C" type="xsd:int" use="required" fixed="1"/>
</xsd:complexType>
</xsd:element>
</xsd:schema>'
GO
DECLARE #validationTbl TABLE (ID INT NOT NULL, Reason NVARCHAR(1024) NULL);
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata XML);
INSERT INTO #tbl (xmldata) VALUES
(N'<X C="1"></X>') -- good
,(N'<X C="1"/>') -- good
,(N'<X C="2"></X>') -- bad
,(N'<root>Miami</root>'); -- ugly
-- DDL and sample data population, end
-- Method #1
DECLARE #ID INT, #XML AS XML(dbo.MySchema)
, #RowCount INT = (SELECT COUNT(*) FROM #tbl);
WHILE #RowCount > 0
BEGIN
BEGIN TRY
SELECT #ID = ID, #XML = XMLData
FROM #tbl
ORDER BY ID DESC
OFFSET #RowCount - 1 ROWS FETCH NEXT 1 ROWS ONLY;
END TRY
BEGIN CATCH
INSERT INTO #validationTbl (ID, Reason)
VALUES (#ID, ERROR_MESSAGE());
END CATCH
SET #RowCount -= 1;
END;
-- test
SELECT * FROM #validationTbl;
-- Method #2
-- unfortunately, it stops at the very first error
-- TRY_CAST() shall swallow all XSD validation errors internally
-- and produce a NULL value for all invalid rows
/*
Validate Expressions (XQuery)
https://learn.microsoft.com/en-us/sql/xquery/validate-expressions-xquery?view=sql-server-ver15
https://learn.microsoft.com/en-us/sql/t-sql/functions/try-cast-transact-sql?view=sql-server-ver15
*/
SELECT TOP(4) *, TRY_CAST(xmldata AS XML(dbo.MySchema)) AS Result
FROM #tbl
ORDER BY ID;
Output
+----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | Reason |
+----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2 | XML Validation: Element or attribute 'C' was defined as fixed, the element value has to be equal to value of 'fixed' attribute specified in definition. Location: /*:X[1]/#*:C |
| 3 | XML Validation: Declaration not found for element 'root'. Location: /*:root[1] |
+----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Related

convert xml data stored in a table to individual name/value pairs stored as records in a different table sql

I'm sorry, I didn't fully explain what I'm looking for. Here is whatI am using:
The table contains xml in the following format"
<row>
<POLICYNUMBER>IFH6000258-04</POLICYNUMBER>
<POLICY_NO>CFH6000258</POLICY_NO>
<POLICY_MOD>03</POLICY_MOD>
</row>
declare #mystuff xml;
set #mystuff = (select top 1 * from TMP_APP for xml path)
INSERT INTO [dbo].[FredTest]
(
[xmlstuff]
)
VALUES
(
#mystuff
)
DECLARE #hldit table (ID int not null,xmldata xml)
* set #hldit = (select xmlstuff from FredTest)
Select entr.value('local-name(.)', 'VARCHAR(50)') as nme,
entr.value('(.)[1]', 'varchar(50)') dta
From #hldit
Cross apply
XmlData.nodes('/root/row/*') as xt(entr)
I get an error on the set(see *). I can't seem to populate the table.
I want to insert the output into a table with 2 columns,'name' and 'value' and have a separate record for each pair.
The output should look like;
Name Value
record 1 Policynumber IFH6000258-04
record 2 Policy_no CFH6000258
record 3 Policy_mod. 03
I feel like I am close. Everything runs but I can't populate the #hldit table. I hope this explains my problem.
Thanks
I am new to XML. I have to extract records, store them in an xml data field in a different table, then extract the xml data and store the name/value pairs in another table. I'm good up to creating the xml data. What I can't figure out is how to take the raw xml from the table, create a table with the name/value pairs stored as individual records. I have looked and tried everything out there but still now luck. I'm hoping the final rows in the new table would look like:
name value
--------------------------
firstname Fred
lastname jones
address 123 here street
instead of like this:
firstname="fred" lastname="jones address="123 here street"
I'll even take it in xml PATH format instead of RAW.
Thanks for any help you can give.
It seems I know what you are after.
Please try the following conceptual example.
It will work in MS SQL Server 2005 onwards.
SQL #1, attributes based
-- DDL and sample data population, start
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata XML);
INSERT INTO #tbl (xmldata) VALUES
(N'<root>
<r firstname="fred" lastname="jones" address="123 here street"/>
<r firstname="Laura" lastname="Bush" address="257 somewhere street"/>
</root>');
-- DDL and sample data population, end
SELECT c.value('local-name(.)','VARCHAR(30)') AS [Column]
, c.value('.', 'VARCHAR(100)') AS [Value]
FROM #tbl
CROSS APPLY xmldata.nodes('/root/r/#*') AS t(c);
Output
+-----------+----------------------+
| Column | Value |
+-----------+----------------------+
| firstname | fred |
| lastname | jones |
| address | 123 here street |
| firstname | Laura |
| lastname | Bush |
| address | 257 somewhere street |
+-----------+----------------------+
SQL #2, elements based
-- DDL and sample data population, start
DECLARE #destination TABLE (ID INT, pos INT, [Name] VARCHAR(30), [Value] VARCHAR(100));
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata XML);
INSERT INTO #tbl (xmldata) VALUES
(N'<row>
<POLICYNUMBER>IFH6000258-04</POLICYNUMBER>
<POLICY_NO>CFH6000258</POLICY_NO>
<POLICY_MOD>03</POLICY_MOD>
</row>');
-- DDL and sample data population, end
INSERT INTO #destination (ID, pos, [Name],[Value])
SELECT ID
, c.value('let $n := . return count(../*[. << $n])+1', 'INT') AS pos
, c.value('local-name(.)','VARCHAR(30)') AS [Name]
, c.value('.', 'VARCHAR(100)') AS [Value]
FROM #tbl
CROSS APPLY xmldata.nodes('/row/*') AS t(c);
-- test
SELECT * FROM #destination;
Output
+----+-----+--------------+---------------+
| ID | pos | Name | Value |
+----+-----+--------------+---------------+
| 1 | 1 | POLICYNUMBER | IFH6000258-04 |
| 1 | 2 | POLICY_NO | CFH6000258 |
| 1 | 3 | POLICY_MOD | 03 |
+----+-----+--------------+---------------+

How to display the "rows" which has value and NULL columns grouped separately

There is a table called 'member' which has 60 columns. Only some of the columns have values; 5 columns have values and the rest of 55 columns are NULL.
The problem is that the columns with values are scattered and I find it difficult to search the row to find the ones which have values.
I have tried a few suggestions which I received while posting this question. Below is one of them.
select * from table order by NULLIF(value,'') = '' DESC, value
I tried below
select * from member order by NULLIF(date_of_begin,'')
This doesn't satisfy my requirement when I want something like this. I use a select statement and segregate all columns having values to be displayed first and then all columns with NULL displayed last.
ID Member Name Age Gender Date of begin DOB DOC Extra
1 John 34 M 4/10/2019 NULL NULL NULL
2 Jack NULL M 4/11/2019 NULL NULL NULL
3 David 54 M 4/15/2019 NULL NULL NULL
4 Eric NULL M 4/16/2019 NULL NULL NULL
5 Ivan 45 M 4/10/2019 NULL NULL NULL
I want a select statement which will divide the above example with the below grouping. Age is NULL for Jack, so it should be placed in last column and displayed. This will make my work easier in finding which columns have NULL and which have values sorted.
For example-- >
Select * from member where id =2 IS NOT NULL (I need help here in this statement change)
Desired results:
ID Member Name Gender Date of begin DOB DOC Extra Age
2 Jack M 4/11/2019 NULL NULL NULL NULL
I'm not willing to swear this is worth the effort, but if you only need one ID at a time, or at least only a few, this will return only the non-null column values. If you wanted to go to even greater pain and try to pivot it back into rows, have at it, but this gets you a workable result set.
We're using CROSS APPLY to unpivot your results, and then dispose of the NULL values. You're data doesn't look like a row anymore, but the handful of data points you need are there for you.
Using the CROSS APPLY, in order to get all of the column values into one single column, I CAST them all as NVARCHAR data, so your results are all text fields.
In the example, I returned two ID values, just so you could see how that would look.
Data set up, in case anybody else wants to take a swing at this:
DECLARE #table TABLE(
ID INTEGER NOT NULL PRIMARY KEY
,Member_Name VARCHAR(5) NULL
,Age INTEGER NULL
,Gender VARCHAR(1) NULL
,Date_of_begin DATE NOT NULL
,DOB VARCHAR(4) NULL
,DOC VARCHAR(4) NULL
,Extra VARCHAR(4) NULL
);
INSERT INTO #table(ID,Member_Name,Age,Gender,Date_of_begin,DOB,DOC,Extra) VALUES (1,'John',34,'M','4/10/2019',NULL,NULL,NULL);
INSERT INTO #table(ID,Member_Name,Age,Gender,Date_of_begin,DOB,DOC,Extra) VALUES (2,'Jack',NULL,'M','4/11/2019',NULL,NULL,NULL);
INSERT INTO #table(ID,Member_Name,Age,Gender,Date_of_begin,DOB,DOC,Extra) VALUES (3,'David',54,'M','4/15/2019',NULL,NULL,NULL);
INSERT INTO #table(ID,Member_Name,Age,Gender,Date_of_begin,DOB,DOC,Extra) VALUES (4,'Eric',NULL,'M','4/16/2019',NULL,NULL,NULL);
INSERT INTO #table(ID,Member_Name,Age,Gender,Date_of_begin,DOB,DOC,Extra) VALUES (5,'Ivan',45,'M','4/10/2019',NULL,NULL,NULL);
Here's the query.
SELECT
c.*
FROM #table AS t
CROSS APPLY (VALUES ('ID',CAST(t.ID AS NVARCHAR(30))),
('Member_Name',CAST(t.Member_Name AS NVARCHAR(30))),
('Age',CAST(t.Age AS NVARCHAR(30))),
('Gender',CAST(t.Gender AS NVARCHAR(30))),
('Date_of_begin',CAST(t.Date_of_begin AS NVARCHAR(30))),
('DOB',CAST(t.DOB AS NVARCHAR(30))),
('DOC',CAST(t.DOC AS NVARCHAR(30))),
('Extra',CAST(t.Extra AS NVARCHAR(30)))
) c (ColName,ColValue)
WHERE t.ID IN (2,3)
AND c.ColValue IS NOT NULL
ORDER BY
t.ID,
CASE
WHEN c.ColName = 'ID' THEN 1
WHEN c.ColName = 'Member_Name' THEN 2
ELSE 3
END
Results:
+---------------+------------+
| ColName | ColValue |
+---------------+------------+
| ID | 2 |
| Member_Name | Jack |
| Gender | M |
| Date_of_begin | 2019-04-11 |
| ID | 3 |
| Member_Name | David |
| Age | 54 |
| Gender | M |
| Date_of_begin | 2019-04-15 |
+---------------+------------+
The solution I suggest is to store one record in n rows,
n is the number of columns,
then you can order the result as you wish using order by
or add the criteria based on NULL values
I am using the system sql server tables syscolumns and sysobjects
create table tab_mytable (id int ,colonne varchar(100),valeur varchar(100));
insert into tab_mytable(id,colonne) select c.colid,c.name from sysobjects o inner join
syscolumns c on o.id=c.id where o.name='member' order by c.colid
declare #cmd as varchar(1000)
declare #mytab as table (val varchar(100))
declare mycursor cursor for select colonne from tab_mytable
declare #colonne as varchar(100)
declare #val as varchar(100)
open mycursor
fetch mycursor into #colonne
while ##fetch_status=0
begin
set #cmd= 'select '+#colonne +' from member where id=2'
insert into #mytab exec(#cmd)
select #val=val from #mytab
set #cmd='update tab_mytable set valeur='''+#val+''' where colonne='''+#colonne+''''
exec(#cmd)
print #cmd
delete from #mytab
fetch mycursor into #colonne
end
close mycursor
deallocate mycursor
--Put here your Query to display teh results as you wish
select * from tab_mytable order by valeur
You can use also the UNPIVOT statement
But in that use you have to write all the column names, for the solution above, you won't write them in your script

SQL Dynamic Charindex

I have a field in a sql table but I need to parse it via charindex, but the lttle caveat is, I don't know how many pieces there are.
The field data would look like the following:
(Image: "filename=a.jpg"), (Image: "filename=b.jpg")
But the question I'm not sure how many filenames there will be in this string, so i need to dynamically build this out this could be 1 or this could be 100.
Any suggestions?
Thanks
Since you cannot know in advance how many values you will extract from each value, I would suggest to represent the results as records, not columns.
If you are using SQL Server 2016 or higher, you can use function STRING_SPLIT() to turn CSV parts to records. Then, SUBSTRING() and CHARINDEX() can be used to extract the relevant information:
declare #t table ([txt] varchar(200))
insert into #t VALUES ('(Image: "filename=a.jpg"),(Image: "filename=b.jpg")')
SELECT value, SUBSTRING(
value,
CHARINDEX('=', value) + 1,
LEN(value) - CHARINDEX('=', value) - 2
)
FROM #t t
CROSS APPLY STRING_SPLIT(t.txt , ',')
Demo on DB Fiddle:
DECLARE #t table ([txt] varchar(200))
INSERT INTO #t VALUES ('(Image: "filename=a.jpg"),(Image: "filename=b.jpg")')
SELECT value, SUBSTRING(
value,
CHARINDEX('=', value) + 1,
LEN(value) - CHARINDEX('=', value) - 2
)
FROM #t t
CROSS APPLY STRING_SPLIT(t.txt , ',')
GO
value | (No column name)
:------------------------ | :---------------
(Image: "filename=a.jpg") | a.jpg
(Image: "filename=b.jpg") | b.jpg
NB : this assumes that the value to extract is always located after the first equal sign and until 2 characters before the end of string. If the pattern is different, you may need to adapt the SUBSTRING()/CHARINDEX() calls.
The real issue is: This is breaking 1.NF. You should never ever store more than one piece of data in one cell. Such CSV-formats are a pain in the neck and you really should use a related side table to store your image hints one by one.
Nevertheless, this can be handled:
--A mockup table
DECLARE #mockup TABLE(ID INT IDENTITY,YourString VARCHAR(1000));
INSERT INTO #mockup VALUES
('(Image: "filename=a.jpg"), (Image: "filename=b.jpg") ')
,('(Image: "filename=aa.jpg"), (Image: "filename=bb.jpg"), (Image: "filename=cc.jpg"), (Image: "filename=dd.jpg"), (Image: "filename=ee.jpg")');
--Pick one element by its position:
DECLARE #position INT=2;
SELECT CAST('<x>' + REPLACE(t.YourString,',','</x><x>') + '</x>' AS XML)
.value('/x[position()=sql:variable("#position")][1]','nvarchar(max)')
FROM #mockup t;
The trick is, to transform the string to XML and use XQuery to fetch the needed element by its position. The intermediate XML looks like this:
<x>(Image: "filename=a.jpg")</x>
<x> (Image: "filename=b.jpg") </x>
You can use some more replacements and L/RTRIM() to get it cleaner.
Read table data
And if you want to create a clean side table and you need all data neatly separated, you can use a bit more of the same:
SELECT CAST('<x><y><z>'
+ REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
t.YourString,'(','') --no opening paranthesis
,')','') --no closing paranthesis
,'"','') --no quotes
,' ','') --no blanks
,'=','</z><z>') --Split at "="
,':','</z></y><y><z>') --Split at ":"
,',','</z></y></x><x><y><z>') --Split at ","
+ '</z></y></x>' AS XML)
FROM #mockup t;
This returns
<x>
<y>
<z>Image</z>
</y>
<y>
<z>filename</z>
<z>a.jpg</z>
</y>
</x>
<x>
<y>
<z>Image</z>
</y>
<y>
<z>filename</z>
<z>b.jpg</z>
</y>
</x>
And with this you would get a clean EAV-table (
WITH Casted AS
(
SELECT ID
,CAST('<x><y><z>'
+ REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
t.YourString,'(','')
,')','')
,'"','')
,' ','')
,'=','</z><z>')
,':','</z></y><y><z>')
,',','</z></y></x><x><y><z>')
+ '</z></y></x>' AS XML) AS CastedToXml
FROM #mockup t
)
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS ID
,ID AS oldId
,eachElement.value('y[1]/z[1]','varchar(max)') AS DataType
,eachElement.value('y[2]/z[1]','varchar(max)') AS ContentType
,eachElement.value('y[2]/z[2]','varchar(max)') AS Content
FROM Casted
CROSS APPLY CastedToXml.nodes('/x') A(eachElement)
The result
+----+-------+----------+-------------+---------+
| ID | oldId | DataType | ContentType | Content |
+----+-------+----------+-------------+---------+
| 1 | 1 | Image | filename | a.jpg |
+----+-------+----------+-------------+---------+
| 2 | 1 | Image | filename | b.jpg |
+----+-------+----------+-------------+---------+
| 3 | 2 | Image | filename | aa.jpg |
+----+-------+----------+-------------+---------+
| 4 | 2 | Image | filename | bb.jpg |
+----+-------+----------+-------------+---------+
| 5 | 2 | Image | filename | cc.jpg |
+----+-------+----------+-------------+---------+
| 6 | 2 | Image | filename | dd.jpg |
+----+-------+----------+-------------+---------+
| 7 | 2 | Image | filename | ee.jpg |
+----+-------+----------+-------------+---------+
I used a table value function
ALTER FUNCTION [dbo].[Fn_sqllist_to_table](#list AS VARCHAR(8000),
#delim AS VARCHAR(10))
RETURNS #listTable TABLE(
Position INT,
Value VARCHAR(8000))
AS
BEGIN
DECLARE #myPos INT
SET #myPos = 1
WHILE Charindex(#delim, #list) > 0
BEGIN
INSERT INTO #listTable
(Position,Value)
VALUES (#myPos,LEFT(#list, Charindex(#delim, #list) - 1))
SET #myPos = #myPos + 1
IF Charindex(#delim, #list) = Len(#list)
INSERT INTO #listTable
(Position,Value)
VALUES (#myPos,'')
SET #list = RIGHT(#list, Len(#list) - Charindex(#delim, #list))
END
IF Len(#list) > 0
INSERT INTO #listTable
(Position,Value)
VALUES (#myPos,#list)
RETURN
END
By calling it via
select * into #test from tableX as T
cross apply [Fn_sqllist_to_table](fieldname,'(')
and then just substringed the value into the final table

How to check what column in INSERT do not have the correct data type?

Imagine I have 200 columns in one INSERT statement, and I occasionally get an "Cannot convert" error for one of columns. Things is, I do not know which column causes this error.
Is there any way in T-SQL or mybatis to check WHICH column has the incorrect format? (I have just date, char, numeric). I can use ISNUMERIC, ISDATE for every column, but this is not so elegant.
I'm using mybatis in Java, so I cannot use any PreparedStatement or so.
You could build a query that tries to convert each of the suspected columns.
And limit the query to where one of the attempts to convert fails.
Mostly the bad data will be in CHAR's or VARCHAR's when trying to cast or convert them to a datetime or number type.
So you can limit your research to those.
Also, from the error you should see which value failed to convert to which type. Which can also help to limit which fields you research.
A simplified example using table variables:
declare #T1 table (id int identity(1,1) primary key, field1 varchar(30), field2 varchar(30), field3 varchar(30));
declare #T2 table (id int identity(1,1) primary key, field1_int int, field2_date date, field3_dec decimal(10,2));
insert into #T1 (field1, field2, field3) values
('1','2018-01-01','1.23'),
('not an int','2018-01-01','1.23'),
('1','not a date','1.23'),
('1','2018-01-01','not a decimal'),
(null,'2018-01-01','1.23'),
('1',null,'1.23'),
('1','2018-01-01',null)
;
select top 1000
id,
case when try_convert(int, field1) is null then field1 end as field1,
case when try_convert(date, field2) is null then field2 end as field2,
case when try_convert(decimal(10,4), field3) is null then field3 end as field3
from #T1
where
try_convert(int, coalesce(field1, '0')) is null
or try_convert(date, coalesce(field2, '1900-01-01')) is null
or try_convert(decimal(10,4), coalesce(field3, '0.0')) is null;
Returns:
id field1 field2 field3
-- ---------- ----------- -------------
2 not an int NULL NULL
3 NULL not a date NULL
4 NULL NULL not a decimal
If the origin data doesn't have to much bad data you could try to fix the origin data first.
Or use the try_convert for the problematic columns with bad data.
For example:
insert into #T2 (field1_int, field2_date, field3_dec)
select
try_convert(int, field1),
try_convert(date, field2),
try_convert(decimal(10,4), field3)
from #T1;
With larger imports - especially when you expect issues - a two-stepped approach is highly recommended.
import the data to a very tolerant staging table (all NVARCHAR(MAX))
check, evaluate, manipulate, correct whatever is needed and do the real insert from here
Here is a generic approach you might adapt to your needs. It will check all tables values against a type-map-table and output all values, which fail in TRY_CAST (needs SQL-Server 2012+)
A table to mockup the staging table (partly borrowed from LukStorms' answer - thx!)
CREATE TABLE #T1 (id INT IDENTITY(1,1) PRIMARY KEY
,fldInt VARCHAR(30)
,fldDate VARCHAR(30)
,fldDecimal VARCHAR(30));
GO
INSERT INTO #T1 (fldInt, fldDate, fldDecimal) values
('1','2018-01-01','1.23'),
('blah','2018-01-01','1.23'),
('1','blah','1.23'),
('1','2018-01-01','blah'),
(null,'2018-01-01','1.23'),
('1',null,'1.23'),
('1','2018-01-01',null);
--a type map (might be taken from INFORMATION_SCHEMA of an existing target table automatically)
DECLARE #type_map TABLE(ColumnName VARCHAR(100),ColumnType VARCHAR(100));
INSERT INTO #type_map VALUES('fldInt','int')
,('fldDate','date')
,('fldDecimal','decimal(10,2)');
--The staging table's name
DECLARE #TableName NVARCHAR(100)='#T1';
--dynamically created statements for each column
DECLARE #columnSelect NVARCHAR(MAX)=
(SELECT
' UNION ALL SELECT id ,''' + tm.ColumnName + ''',''' + tm.ColumnType + ''',' + QUOTENAME(tm.ColumnName)
+ ',CASE WHEN TRY_CAST(' + QUOTENAME(tm.ColumnName) + ' AS ' + tm.ColumnType + ') IS NULL THEN 0 ELSE 1 END ' +
'FROM ' + QUOTENAME(#TableName)
FROM #type_map AS tm
FOR XML PATH('')
);
-The final dynamically created statement
DECLARE #cmd NVARCHAR(MAX)=
'SELECT tbl.*
FROM
(
SELECT 0 AS id,'''' AS ColumnName,'''' AS ColumnType,'''' AS ColumnValue,0 AS IsValid WHERE 1=0 '
+ #columnSelect +
') AS tbl
WHERE tbl.IsValid = 0;'
--Execution with EXEC()
EXEC(#cmd);
The result:
+----+------------+---------------+-------------+---------+
| id | ColumnName | ColumnType | ColumnValue | IsValid |
+----+------------+---------------+-------------+---------+
| 2 | fldInt | int | blah | 0 |
+----+------------+---------------+-------------+---------+
| 5 | fldInt | int | NULL | 0 |
+----+------------+---------------+-------------+---------+
| 3 | fldDate | date | blah | 0 |
+----+------------+---------------+-------------+---------+
| 6 | fldDate | date | NULL | 0 |
+----+------------+---------------+-------------+---------+
| 4 | fldDecimal | decimal(10,2) | blah | 0 |
+----+------------+---------------+-------------+---------+
| 7 | fldDecimal | decimal(10,2) | NULL | 0 |
+----+------------+---------------+-------------+---------+
The statement created is like here:
SELECT tbl.*
FROM
(
SELECT 0 AS id,'' AS ColumnName,'' AS ColumnType,'' AS ColumnValue,0 AS IsValid WHERE 1=0
UNION ALL SELECT id
,'fldInt'
,'int'
,[fldInt]
,CASE WHEN TRY_CAST([fldInt] AS int) IS NULL THEN 0 ELSE 1 END
FROM [#T1]
UNION ALL SELECT id
,'fldDate'
,'date',[fldDate]
,CASE WHEN TRY_CAST([fldDate] AS date) IS NULL THEN 0 ELSE 1 END
FROM [#T1]
UNION ALL SELECT id
,'fldDecimal'
,'decimal(10,2)'
,[fldDecimal]
,CASE WHEN TRY_CAST([fldDecimal] AS decimal(10,2)) IS NULL THEN 0 ELSE 1 END
FROM [#T1]
) AS tbl
WHERE tbl.IsValid = 0;

Reverse order of a XML Column in SQL Server

In a SQL Server table, I have a XML column where status are happened (first is oldest, last current status).
I have to write a stored procedure that returns the statuses: newest first, oldest last.
This is what I wrote:
ALTER PROCEDURE [dbo].[GetDeliveryStatus]
#invoiceID nvarchar(255)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #xml xml
SET #xml = (SELECT statusXML
FROM Purchase
WHERE invoiceID = #invoiceID )
SELECT
t.n.value('text()[1]', 'nvarchar(50)') as DeliveryStatus
FROM
#xml.nodes('/statuses/status') as t(n)
ORDER BY
DeliveryStatus DESC
END
Example of value in the statusXML column:
<statuses>
<status>A</status>
<status>B</status>
<status>A</status>
<status>B</status>
<status>C</status>
</statuses>
I want the procedure to return:
C
B
A
B
A
with ORDER BY .... DESC it return ALPHABETIC reversed (C B B A A)
How should I correct my procedure ?
Create a sequence for the nodes based on the existing order then reverse it.
WITH [x] AS (
SELECT
t.n.value('text()[1]', 'nvarchar(50)') as DeliveryStatus
,ROW_NUMBER() OVER (ORDER BY t.n.value('..', 'NVARCHAR(100)')) AS [Order]
FROM
#xml.nodes('/statuses/status') as t(n)
)
SELECT
DeliveryStatus
FROM [x]
ORDER BY [x].[Order] DESC
... results ...
DeliveryStatus
C
B
A
B
A
There is no need to declare a variable first. You can (and you should!) read the needed values from your table column directly. Best was an inline table valued function (rather than a SP just to read something...)
Better performance
inlineable
You can query many InvoiceIDs at once
set-based
Try this (I drop the mock-table at the end - carefull with real data!):
CREATE TABLE Purchase(ID INT IDENTITY,statusXML XML, InvocieID INT, OtherValues VARCHAR(100));
INSERT INTO Purchase VALUES('<statuses>
<status>A</status>
<status>B</status>
<status>A</status>
<status>B</status>
<status>C</status>
</statuses>',100,'Other values of your row');
GO
WITH NumberedStatus AS
(
SELECT ID
,InvocieID
, ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS Nr
,stat.value('.','nvarchar(max)') AS [Status]
,OtherValues
FROM Purchase
CROSS APPLY statusXML.nodes('/statuses/status') AS A(stat)
WHERE InvocieID=100
)
SELECT *
FROM NumberedStatus
ORDER BY Nr DESC
GO
--Clean-Up
--DROP TABLE Purchase;
The result
+---+-----+---+---+--------------------------+
| 1 | 100 | 5 | C | Other values of your row |
+---+-----+---+---+--------------------------+
| 1 | 100 | 4 | B | Other values of your row |
+---+-----+---+---+--------------------------+
| 1 | 100 | 3 | A | Other values of your row |
+---+-----+---+---+--------------------------+
| 1 | 100 | 2 | B | Other values of your row |
+---+-----+---+---+--------------------------+
| 1 | 100 | 1 | A | Other values of your row |
+---+-----+---+---+--------------------------+

Resources