SQL limit for XML type - sql-server

I am actually working on a large database and I am querying the following data;
My Query:
SELECT ldd.LDistCD, ldd.LDistDescPay
FROM LDetail ldd
INNER JOIN LDist ld
ON ldd.ID = ld.ID
AND ld.ID = '019458'
AND ld.LDistType = 'F'
Result:
What I am doing next, is to loop across the results (27873) in my VB codes to concatenate the data in the following format;
LDistCD + '|' + LDistDescPay
Normally that would be a very time consuming time looping through all these rows. Hence to optimise the work, I am using the following query which should already concatenate the data for me;
SELECT stuff((SELECT ',' + ldd.LddLabourDistCD + '|' + ldd.LddLabourDistDescPay
FROM LDetail ldd
INNER JOIN LDist ld
ON ldd.ID = ld.ID
AND ld.ID = 019425 AND ld.LDistType = 'F'
FOR XML
PATH ('')), 1, 1, '')
Everything is working fine except for the result, whereby some data is being truncated!! Running the last query on MS SQL Server returns the concatenate result but it is not complete. I get the impression there's a limit to the result which is being exceeded.
Can anyone help on the issue please?
Difficult for me to upload the db or the result but just to tell you that the 27873 rows, when concatenated in one string, is not fitting in the result.

The truncation you are seeing is specific to SQL Server Management Studio (SSMS). If you go to the Query menu and select Query Options..., then go to Results and then to Grid, you should see on the right side a section for "Maximum Characters Retrieved". The "Non XML data" has a max of 65,535 (which should also be the default) and the "XML data" is a drop-down with options:
1 MB
2 MB (default)
5 MB
Unlimited
Since the XML datatype can bring back more than 65,535 characters, you can convert your output to XML (this is only needed when using SSMS; client libraries should pull back the full string):
SELECT CONVERT(XML,
stuff((SELECT ',' + ldd.LddLabourDistCD + '|' + ldd.LddLabourDistDescPay
FROM LDetail ldd
INNER JOIN LDist ld
ON ldd.ID = ld.ID
AND ld.ID = 019425
AND ld.LDistType = 'F'
FOR XML PATH ('')), 1, 1, '')
)

Related

SQL Alternative for FOR XML PATH

For some reason my query :
SELECT
MP_SQLQRY_ProjectArchive.ProjectShort,
MP_SQLQRY_ProjectArchive.ProjectName ,
MP_SQLQRY_ProjectArchive.ProjectManagerName ,
ProjectOwnerName,
ProjStateName,
ProjectKategoryShort,
ProjectTypeShort,
MP_SQLQRY_ProjectArchive.ProjectPriority,
DepartmentShort,
ProjectArchivedDate,
CONVERT(NVARCHAR(MAX),ERPAccountCodes.ERPAccountCodes) As ERPAccountCodes
FROM
MP_SQLQRY_ProjectArchive
INNER JOIN
SQLQRY_VALID_PROJECT_ORDER_INFO ON MP_SQLQRY_ProjectArchive.ProjectID = SQLQRY_VALID_PROJECT_ORDER_INFO.ProjectID
INNER JOIN
COM_tbl_MAD_Project ON MP_SQLQRY_ProjectArchive.ProjectID = COM_tbl_MAD_Project.ProjectID
LEFT JOIN
(SELECT
ProjectID,
CONVERT(Nvarchar(max), STUFF((SELECT ';' + [SAPNumber] AS [data()]
FROM [COM_tbl_MAD_ProjectERPAccountCode] t2
WHERE t2.[ProjectID] = t1.[ProjectID]
ORDER BY [SAPNumber]
FOR XML PATH('')), 1, 1, '')) AS ERPAccountCodes
FROM
[COM_tbl_MAD_ProjectERPAccountCode] t1
GROUP BY
[ProjectID], [SAPNumber]) AS ERPAccountCodes ON ERPAccountCodes.ProjectID = COM_tbl_MAD_Project.ProjectID
will not return the values for the ERPAccountCodes column. In SQL Server, it works fine and I get my values, which are seperated with a ; as I want.
But when I run it as extern connection with ADODB in access this single column is empty.
I suppose it is a problem with access or ADODB but I have no clue what to do about it
Is there an alternative to FOR XML PATH or is there a different way to do it?
For clarification, the values return some blank values. Not NULL, just these weird blank symbols when i debugged in VBA in access, altough I tried converting it into a NVarchar which theoretically should work just fine.
Left is an Excel created from the MS Access SQL call. Right is from SQL Server Management Studio:

SQL Server trigger reports 'inserted' table is missing

I am running a script against a SQL Server 2016 database that creates various tables, views and triggers. This same script has been working against dozens of other servers but it is getting an error against this one particular server.
All of the triggers seem to be created but when I check for invalid objects it reports all of them as invalid. The really strange part is, it says the problem is the "inserted" table (or "deleted" table, depending on the trigger) is missing.
I am checking for invalid objects using this query:
SELECT
QuoteName(OBJECT_SCHEMA_NAME(referencing_id)) + '.'
+ QuoteName(OBJECT_NAME(referencing_id)) AS ProblemObject,
o.type_desc,
ISNULL(QuoteName(referenced_server_name) + '.', '')
+ ISNULL(QuoteName(referenced_database_name) + '.', '')
+ ISNULL(QuoteName(referenced_schema_name) + '.', '')
+ QuoteName(referenced_entity_name) AS MissingReferencedObject
FROM
sys.sql_expression_dependencies sed
LEFT JOIN
sys.objects o ON sed.referencing_id = o.object_id
WHERE
(is_ambiguous = 0)
AND (OBJECT_ID(ISNULL(QuoteName(referenced_server_name) + '.', '')
+ ISNULL(QuoteName(referenced_database_name) + '.', '')
+ ISNULL(QuoteName(referenced_schema_name) + '.', '')
+ QuoteName(referenced_entity_name)) IS NULL)
ORDER BY
ProblemObject,
MissingReferencedObject
which I got from here
Find broken objects in SQL Server
The triggers are "instead of" triggers against the views that then modify the underlying tables. There is really nothing special about any of them.
Is there something wrong with my query and the objects aren't really invalid or is there something with the database? I am running the script as the database owner so I don't think it is a permissions issue.
Thanks
Run this query against the problem SQL Server, and against one where no problems are shown:
SELECT *
FROM
sys.sql_expression_dependencies sed
LEFT JOIN
sys.objects o ON sed.referencing_id = o.object_id
WHERE type = 'TR'
ORDER BY
type_desc,
referenced_entity_name
If the inserted / deleted tables show up in one list but not the other, then there is some difference causing this. I'd check identifier quoting rules to start.

Why are my results in SSRS different from SQL SMS?

I've never seen this happen before, so I'm curious if anybody knows why, and maybe how I can correct for it.
This particular part of my code:
SELECT inv_num, co_line,
STUFF((
SELECT '/' + rs2.inv_pro_description
FROM #ReportSet rs2
WHERE rs2.inv_num = rs.inv_num AND
rs2.co_line = rs.co_line
FOR XML PATH('')), 1, 1, '') as ipd_combo
FROM #ReportSet rs
WHERE inv_pro_seq IS NOT NULL AND inv_pro_description <> 'Less Previously Invoiced'
GROUP BY inv_num, co_line
results in a correct concatenation of my two results, e.g., "10% Advance/Fixed $5 Required" in this case. However, when utilizing this procedure in SSRS, my field contains instead "Fixed $5 Required/10% Advance".
I have grown accustomed to being able to predict my SSRS reports based on running SQL queries, so this confused me.
I don't really understand the STUFF / XML PATH code, of course. I've copied it from somebody else (on this website, naturally) so I know that might have something to do with it.
To get consistent ordering, add an order by clause to your inner select:
SELECT inv_num, co_line,
STUFF((
SELECT '/' + rs2.inv_pro_description
FROM #ReportSet rs2
WHERE rs2.inv_num = rs.inv_num AND
rs2.co_line = rs.co_line
ORDER BY rs2.inv_pro_seq --********
FOR XML PATH('')), 1, 1, '') as ipd_combo
FROM #ReportSet rs
WHERE inv_pro_seq IS NOT NULL AND inv_pro_description <> 'Less Previously Invoiced'
GROUP BY inv_num, co_line
The commented asterisks indicate the added line.
(I answered in comments, but adding this here for consistency/permanency).

Display file path of documents in Crystal Report

I have a crystal report that lists invoices and I would like to be able to show the file path of the invoices in another column of the report. The following query allows me to search for the documents based on their unique ID number. It then displays the file location of the one document I search for, however I would like to have this apply to all of the documents listed in the report. Can someone please help me out with this?
`DECLARE #entryID INT = 35793
SELECT dbo.toc.name AS DocumentName, dbo.doc.pagenum + 1 AS PageNum, dbo.vol.fixpath + '\' +
SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),1,2) + '\' +
SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),3,2) + '\' +
SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),5,2) + '\' +
CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2) + '.TIF' AS FullPathAndFilename
FROM dbo.doc
LEFT JOIN dbo.toc ON dbo.doc.tocid = dbo.toc.tocid
LEFT JOIN dbo.vol ON dbo.toc.vol_id = dbo.vol.vol_id
WHERE dbo.doc.tocid = #entryID
ORDER BY dbo.doc.pagenum`
The answer depends on how you currently retrieve data in the existing Crystal Report.
Option A: If your report data source is a "Command" (in other words you have written a SQL statement in the crystal report that retrieves the data you need), then you will want to modify that command to pull in this new information via a subquery. So for example if your current report SQL is something like "select x.* from foo as x", you would want it to be something like:
select x.*,
(SELECT dbo.vol.fixpath + '\'
+ SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),1,2) + '\'
+ SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),3,2) + '\'
+ SUBSTRING(CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2),5,2) + '\'
+ CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), dbo.doc.storeid),2)
+ '.TIF' AS FullPathAndFilename
FROM dbo.doc
LEFT JOIN dbo.toc ON dbo.doc.tocid = dbo.toc.tocid
LEFT JOIN dbo.vol ON dbo.toc.vol_id = dbo.vol.vol_id
WHERE dbo.doc.tocid = x.tocid //this line joining the new SQL to all report rows
)
from foo as x;
Option B: If instead you pull your data from Crystal using the join wizard, then it would probably be best to create a new view in your database that matches the SQL you provided (minus the last 2 lines), and then join your existing main report tables to this view in the Crystal join wizard.

How to import from XML file in SQL Server Import and Export Wizard?

I have data and schema for a table as a single XML file. How do I import this using the SQL Server Import and Export Wizard?
Should I use a "Flat File Source" as my Data Source? Or what?
[For info, I exported the XML from VistaDB, but I have not yet got as far as the point where the system which created the data might be an issue.]
As far as I know, you cannot do this by using a the import export wizard. Assuming you want the data to wind up all relational rather than as XML datatype, you'll need to create the table and use sp_xml_preparedocument and OPENXML.
See How to use OPENXML to load XML data into existing SQL Table?
As far as I know MS SQL Server Management Studio does not have a tool to upload XML to a table.
There is an option that involves a combination of OPENROWSET and XML processing, but it requires that the files reside in the server's file system.
I needed to load into a table a series of log files generated by a Java web Application but had no access to upload them to the server, I had the logs in my local machine. I managed to upload data in a two step process that is not too cumbersome but its definitely too slow for a permanent solution.
I created a table that consists of two columns: an autonumeric primary key, and a varchar(max).
I used the import data to upload the text files to the table so that each line in the file is a record in the table. The primary key coincidentally represents the line number.
So I could write something like:
select LineNumber, TextLine from [LogFile] order by LineNumber
I then prepared another table with an structure that matched the records in my XML.
My XML files had the particularity that each "value" tag was in its own text line, the opening and closing "record" tags where each on a separate line.
For example:
<log>
<record>
<date>2018-07-27T09:54:20</date>
<millis>1532706860250</millis>
<sequence>13587</sequence>
<logger>registroweb.ServReg</logger>
<level>INFO</level>
<class>somepackage.someclass</class>
<method>methodname</method>
<thread>11153</thread>
<message>some very long text</message>
<param>another long text</param>
</record>
...
</log>
This would mean that I could select all records where text_line = '<log>' would give me all opening record tags, but most importantly, self joining the table with t2.line_number = t1.line_number + 1 would always give me the line containing the date tag, line_number+2 would give millis, and so on.
So with the following query I was able to transform the flat linear table into a proper table:
insert into LogFileProcessed(
[date],
[millis],
[sequence],
[logger] ,
[level] ,
[class] ,
[method] ,
[thread] ,
[message],
[param]
)
select
--record.TextLine,
convert(datetime, replace(replace(ltrim(dte.TextLine), '<date>', ''), '</date>', ''), 126) [date],
convert(bigint, replace(replace(ltrim(mls.TextLine), '<millis>', ''), '</millis>', '')) [millis],
convert(bigint, replace(replace(ltrim(seq.TextLine), '<sequence>', ''), '</sequence>', '')) [sequence],
replace(replace(ltrim(logr.TextLine), '<logger>', ''), '</logger>', '') [logger],
replace(replace(ltrim(lvl.TextLine), '<level>', ''), '</level>', '') [level],
replace(replace(ltrim(cls.TextLine), '<class>', ''), '</class>', '') [class],
replace(replace(ltrim(mtd.TextLine), '<method>', ''), '</method>', '') [method],
replace(replace(ltrim(trd.TextLine), '<thread>', ''), '</thread>', '') [thread],
replace(replace(ltrim(msg.TextLine), '<message>', ''), '</message>', '') [message],
replace(replace(ltrim(prm.TextLine), '<param>', ''), '</param>', '') [param]
from LogFile record
left join LogFile dte on dte.LineNumber = record.LineNumber+1
left join LogFile mls on mls.LineNumber = record.LineNumber+2
left join LogFile seq on seq.LineNumber = record.LineNumber+3
left join LogFile logr on logr.LineNumber = record.LineNumber+4
left join LogFile lvl on lvl.LineNumber = record.LineNumber+5
left join LogFile cls on cls.LineNumber = record.LineNumber+6
left join LogFile mtd on mtd.LineNumber = record.LineNumber+7
left join LogFile trd on trd.LineNumber = record.LineNumber+8
left join LogFile msg on msg.LineNumber = record.LineNumber+9
left join LogFile prm on prm.LineNumber = record.LineNumber+10 and prm.TextLine <> '</record>' -- param is actually the only tag that is optional and some times is not present in the record.
where record.TextLine = '<record>'
order by 1, 2
Given the particular restrictions I have at the time and the structure of the files, this worked good enough for a one time task, allowing me to perform regular queries on the data without having to repeatedly run XML parsing or processing code.

Resources