Encountering an issue while attempting to convert some special characters into proper XML. Instead, it's returning XML which is failing for the calling application.
This is part of a much larger chunk of code. I must use XML PATH, otherwise it will require refactoring the entire procedure, which is outside the scope of this work.
I have attempted to replace the values, and tried replacing the values both before and after. I'm unable to perform an extra pass on the subsequent returned XML
DROP TABLE IF EXISTS #MyData
GO
CREATE TABLE #MyData
(
RefId varchar(20) NOT NULL,
Company varchar(255) NOT NULL
);
GO
INSERT INTO #MyData (RefId, Company) VALUES ('SN7172', 'Goodle '+CHAR(0174));
INSERT INTO #MyData (RefId, Company) VALUES ('GW5282', 'NewtownApple '+CHAR(0169));
INSERT INTO #MyData (RefId, Company) VALUES ('ZE0018', 'Curlewky Appliances '+CHAR(0151));
INSERT INTO #MyData (RefId, Company) VALUES ('TW1037', 'Sparky Games '+CHAR(0153));
GO
SELECT (
SELECT RefId,
REPLACE(REPLACE(REPLACE(REPLACE(Company, char(0174),'®'), char(0169),'©'), char(0153),'™'), char(0151),'—') Company
FROM #MyData
FOR XML PATH('Catalog'), TYPE
)
FOR XML PATH('Catalogs'), TYPE
The SQL is being converted to ® instead of ®
Yet, the XML below shows the ampersand being converted, rather than the full string.
<Catalog>
<RefId>GW5282</RefId>
<Company>Newtown & Apple ©</Company>
</Catalog>
<Catalog>
<RefId>ZE0018</RefId>
<Company>Curlewky Appliances —</Company>
</Catalog>
<Catalog>
<RefId>TW1037</RefId>
<Company>Sparky Games ™</Company>
</Catalog>
</Catalogs>
Do I get this correctly? The XML your SQL-Server produces is correct (something like <Company>NewtownApple ©</Company>), but your consumer cannot deal with the © and therefore you want to use the entity instead. Correct?
One hack might be to convert the resulting XML and do the replacements there on string base:
SELECT REPLACE(REPLACE(REPLACE(REPLACE(CAST(
(
SELECT (
SELECT RefId, Company
FROM #MyData
FOR XML PATH('Catalog'), TYPE
)
FOR XML PATH('Catalogs'), TYPE) AS nvarchar(MAX)), char(0174),'®'), char(0169),'©'), char(0153),''), char(0151),'');
But do not try to cast the result back to XML. You would get the special characters back...
Related
I have a table that has mixed column types (int, varchar, & xml). The varchar columns have XML formatted data and the xml column has a large node of data. I'm trying to select all the varchar columns with the XML included to create an XML output file using a SQL Agent job step. The problem I'm having is the column defined as XML is padding a lot of spaces in the file and is truncating data in that column. Table is as follows:
CREATE TABLE [dbo].[MixedTYPEXML](
[col1] [varchar](4) NOT NULL,
[col2] [int] NOT NULL,
[xmlinText1] [varchar](190) NOT NULL,
[JOBP] [xml] NULL
)
GO
INSERT INTO [dbo].[MixedTypeXML]
([col1]
,[col2]
,[xmlinText1]
,[JOBP]
)
VALUES
('Prod'
,'35490'
,'<JOBP AllowExternal="1" name="JOBP.EXTRACTS_PHP_SSIS_POST_CORE#35490">'
,'<PreCon><conditions id="CONDITIONS" /></PreCon>'
)
GO
My select statement is
SELECT col2, xmlinText1, JOBP from dbo.MixedTypeXML
The expected results are below but the actual would be several thousand records. I'm only pulling one record with a small sample data example.
35490 <JOBP AllowExternal="1" name="JOBP.EXTRACTS_PHP_SSIS_POST_CORE#35490"> <PreCon><conditions id="CONDITIONS" /></PreCon>
This is just a subset of the columns I'm trying to extract. I basically need to extract all the columns in the table including the XML defined column to send the results to an output table on the local drive without padding spaces or truncating the data. The output file will be an XML formatted file. The JOBP column sample data is just a snippet. The actual data contains a few hundred lines of xml nodes.
I've looked at lots of posts on how to extract nodes and values but I'm not trying to extract specific data. I want the entire table to create an XML output file. Any ideas how to do that?
Based on the desired output, the entire output file shall be in XML format. It cannot be partially *.csv file mixed with XML elements.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (
[col1] VARCHAR(4) NOT NULL,
[col2] INT NOT NULL,
[xmlinText] VARCHAR(190) NOT NULL,
[JOBP] XML NULL
);
INSERT INTO #tbl ([col1] ,[col2] ,[xmlinText] ,[JOBP])
VALUES
(
'Prod'
,35490
,'<JOBP AllowExternal="1" name="JOBP.EXTRACTS_PHP_SSIS_POST_CORE#35490"/>'
,'<PreCon><conditions id="CONDITIONS" /></PreCon>'
)
-- DDL and sample data population, end
SELECT col1, col2
, TRY_CAST([xmlinText] AS XML) AS [xmlinText]
, JOBP
FROM #tbl
FOR XML PATH('row'), TYPE, ROOT('root');
Output
<root>
<row>
<col1>Prod</col1>
<col2>35490</col2>
<xmlinText>
<JOBP AllowExternal="1" name="JOBP.EXTRACTS_PHP_SSIS_POST_CORE#35490" />
</xmlinText>
<JOBP>
<PreCon>
<conditions id="CONDITIONS" />
</PreCon>
</JOBP>
</row>
</root>
None of the above answers worked for me. What I finally got to work was using the CONVERT string feature.
SELECT col1, col2
, CONVERT(VARCHAR(MAX), JOBP)
FROM #tbl
For some reason we found on the database characters like this: Ã
I can assume this character represent the character: é
Now I need to revise the whole table but checking all other characters to make sure there are no others.
Where I can find the relation of characters for example between this à and é? or probably find an SQL function that is already done to make those replacement.
I'm using SQL Server 2014
As mentioned by Daniel E, your dirty data might have been caused by the use of incorrect code pages (UTF-8 that was interpreted as ISO 8859-1).
One way to find entries with dirty data, is to use a "not exists" ("^") like expression with the list of valid characters in that expression. See example below.
declare #t table (name varchar(20))
insert into #t values ('touché')
insert into #t values ('encore touché')
insert into #t values ('reçu')
insert into #t values ('hello world')
select * from #t where name like '%[^a-zA-Z., -]%'
select * from #t where name like '%[^a-zA-Z.,èêé -]%' COLLATE Latin1_General_BIN
On SQL Server 2012 (SP1), what's the best way to take data that looks like this:
declare #t table (
id int,
data xml
)
insert into #t values (1, '<node/>'), (2, '<node/>')
id data
1 <node />
2 <node />
and turn it into
id data
1 <root><node /></root>
2 <root><node /></root>
?
Is there a way that's more efficient than just converting it to character data, adding the node start and end tag and then converting it back to xml? I'm dealing with this issue on larger data sets, so this is just a simple example.
One more approach (but my favorite was har07's answer)
declare #t table (
id int,
data xml
)
insert into #t values (1, '<node/>'), (2, '<node/>');
SELECT id
,(SELECT data AS [*] FOR XML PATH('root'),TYPE)
FROM #t;
You can use simple XQuery to add <root> parent in your query result. No back-and-forth conversion between XML and VARCHAR data types involved :
SELECT id, data.query('<root>{.}</root>') AS data
FROM #t
Quick test : http://rextester.com/RLZ30365
Brief explanation :
{} : indicates that texts in between should be treated as XQuery expression instead of literals
. : reference to current context node, which in this case is <node />
I'm working on a SSIS Package.
I have a table as below:
Table Name: Employee_table
EmployeID EmployeeName EmployeeDataXML
==============================================
1 Mark <Age>32</Age><Role>Manager</Role>
2 Albert <Age>31</Age><Role>Staff</Role>
==============================================
This table has to be exported into a flat file with name: Employeedata.dat
Content in the file should look like this:
<EmployeeID>1</EmployeeID><EmployeeName>Mark</EmplyeeName><EmployeeDataXML><Age>32</Age><Role>Manager</Role></EmployeeDataXML>
<EmployeeID>2</EmployeeID><EmployeeName>Albert</EmplyeeName><EmployeeDataXML><Age>31</Age><Role>Staff</Role></EmployeeDataXML>
Basically, the employeeid and employeename columns are not in xml format but still when the export happens they should be wrapped up in xml too.
Can someone guide me which is the best way to do it?
Do i need to use any transformation here?
Is there any control/task which is readily available?
Can writing a SQL Select Statement which could simply solves this?
Please guide.
Yes, a simple SELECT using FOR XML PATH should take care of this:
DECLARE #TestData TABLE
(
EmployeID INT NOT NULL,
EmployeeName NVARCHAR(50) NOT NULL,
EmployeeDataXML XML
);
INSERT INTO #TestData (EmployeID, EmployeeName, EmployeeDataXML)
VALUES (1, N'Mark', N'<Age>32</Age><Role>Manager</Role>');
INSERT INTO #TestData (EmployeID, EmployeeName, EmployeeDataXML)
VALUES (2, N'Albert', N'<Age>31</Age><Role>Staff</Role>');
SELECT EmployeID, EmployeeName, EmployeeDataXML
FROM #TestData
FOR XML PATH(N'Employee');
produces the following:
<Employee>
<EmployeID>1</EmployeID>
<EmployeeName>Mark</EmployeeName>
<EmployeeDataXML>
<Age>32</Age>
<Role>Manager</Role>
</EmployeeDataXML>
</Employee>
<Employee>
<EmployeID>2</EmployeID>
<EmployeeName>Albert</EmployeeName>
<EmployeeDataXML>
<Age>31</Age>
<Role>Staff</Role>
</EmployeeDataXML>
</Employee>
You didn't have the parent <Employee> element shown in the sample output, but I don't think the file would be usable without some element wrapping the field elements into a "row".
I have two databases.I insert data from DATABASE_2 TO DATABASE_1 however i need to convert some column.
I must convert Customer_Telephone_Number from varchar to bigint after that insert it.
So,
My Question is in below.
SET IDENTITY_INSERT DATABASE_1.dbo.CUSTOMER_TABLE ON
INSERT INTO DATABASE_1.dbo.CUSTOMER_TABLE
(
Customer_Id,
Customer_Telephone_Number
)
Select
Customer_Id,
Customer_Telephone_Number // This is varchar so i need to convert it to Big int.
from
DATABASE_2.DBO.CUSTOMER_TABLE
Any help will be appreciated.
Thanks.
If the data stored is without spaces or other non numeric symbols:
Select
Customer_Id,
CONVERT(BIGINT,Customer_Telephone_Number)
from
DATABASE_2.DBO.CUSTOMER_TABLE
For instance, if there is value with (222)-3333-333 it would fail. If the value was 2223333333 it would succeed