SQL Server split SELECT XML column as arbitrary individual columns - sql-server

Question

In my application, I have few pre-defined fields for an object and user can define custom fields. I am using XML data type to store the custom fields in a name value format.
e.g. I have Employees table that has FN, LN, Email as pre-defined columns and CustomFields as XML column to hold the user defined fields.
And different rows can contain different custom fields.
e.g. Row 1 -> John, Smith, jsmith#example.com,
<root>
<phone>123-123-1234</phone>
<country>USA</country>
</root>
and then Row 2 -> Smith, John, sjohn#example.com,
<root>
<age>50</age>
<sex>Male</sex>
</root>
And there can be any number of such custom fields defined for different employee records. The format will always be the same
<root><field>value</field></root>
How can I return Phone and Country as columns while selecting Row1 and return Age and Sex as columns while selecting Row2?

Answer 1

Take this temp table for all examples
CREATE TABLE #tbl (ID INT IDENTITY, FirstName VARCHAR(100),LastName VARCHAR(100),eMail VARCHAR(100),CustomFields XML);
INSERT INTO #tbl VALUES
('John','Smith','john.smith#test.com'
,'<root>
<phone>123-123-1234</phone>
<country>USA</country>
</root>')
, ('Jane','Miller','jane.miller#test.com'
,'<root>
<age>50</age>
<sex>Male</sex>
</root>');
Option 1
Assuming that there is a fix known set of custom fields.
This allows typesafe reading (age as INT)
all possible columns are returned, unused are NULL
Try this code
SELECT tbl.ID
,tbl.FirstName
,tbl.LastName
,tbl.eMail
,tbl.CustomFields.value('(/root/phone)[1]','nvarchar(max)') AS phone
,tbl.CustomFields.value('(/root/country)[1]','nvarchar(max)') AS country
,tbl.CustomFields.value('(/root/age)[1]','int') AS age
,tbl.CustomFields.value('(/root/sex)[1]','nvarchar(max)') AS sex
FROM #tbl AS tbl
This is the result
+----+-----------+----------+----------------------+--------------+---------+------+------+
| ID | FirstName | LastName | eMail | phone | country | age | sex |
+----+-----------+----------+----------------------+--------------+---------+------+------+
| 1 | John | Smith | john.smith#test.com | 123-123-1234 | USA | NULL | NULL |
+----+-----------+----------+----------------------+--------------+---------+------+------+
| 2 | Jane | Miller | jane.miller#test.com | NULL | NULL | 50 | Male |
+----+-----------+----------+----------------------+--------------+---------+------+------+
*/
Option 2
assuming you do not know the field names in advance you cannot name the output columns directly
But you can use generic names, read the data row-wise and do PIVOT
Try this:
SELECT p.*
FROM
(
SELECT tbl.FirstName
,tbl.LastName
,tbl.eMail
,N'Col_' + CAST(ROW_NUMBER() OVER(PARTITION BY tbl.ID ORDER BY (SELECT NULL)) AS NVARCHAR(max)) AS ColumnName
,A.cf.value('local-name(.)','nvarchar(max)') + ':' + A.cf.value('.','nvarchar(max)') AS cf
FROM #tbl AS tbl
CROSS APPLY tbl.CustomFields.nodes('/root/*') AS A(cf)
) AS x
PIVOT
(
MAX(cf) FOR ColumnName IN(Col_1,Col_2,Col_3,Col_4 /*add as many as you need*/)
) AS p
This is the result
+-----------+----------+----------------------+--------------------+-------------+-------+-------+
| FirstName | LastName | eMail | Col_1 | Col_2 | Col_3 | Col_4 |
+-----------+----------+----------------------+--------------------+-------------+-------+-------+
| Jane | Miller | jane.miller#test.com | age:50 | sex:Male | NULL | NULL |
+-----------+----------+----------------------+--------------------+-------------+-------+-------+
| John | Smith | john.smith#test.com | phone:[phone] | country:USA | NULL | NULL |
+-----------+----------+----------------------+--------------------+-------------+-------+-------+
Option 3
assuming you do not know the columns, but you need the columns correctly named
attention: be aware of the fact, that such an approach will never be allowed in ad-hoc-SQL such as VIEW or inline TVF which might be a great back draw...
This needs dynamic creation of a statement. I will create the statement of Option 1 but replace the fix list with a dynamically created list:
DECLARE #DynamicColumns NVARCHAR(MAX)=
(
SELECT ',tbl.CustomFields.value(''(/root/' + A.cf.value('local-name(.)','nvarchar(max)') + ')[1]'',''nvarchar(max)'') AS ' + A.cf.value('local-name(.)','nvarchar(max)')
FROM #tbl AS tbl
CROSS APPLY tbl.CustomFields.nodes('/root/*') AS A(cf)
FOR XML PATH('')
);
DECLARE #DynamicSQL NVARCHAR(MAX)=
' SELECT tbl.ID
,tbl.FirstName
,tbl.LastName
,tbl.eMail'
+ #DynamicColumns +
' FROM #tbl AS tbl;'
EXEC(#DynamicSQL);
The result would be the same as in Option 1, but with a completely dynamic approach.
Cleanup
DROP TABLE #tbl;