In SQL Server 2005 I am trying to query a varchar(MAX) column which has some rows with text data that exceed the 8192. Yet, In Management Studio I have under Tools --> Options --> Query Results --> Results to Text --> Max numbers of characters displayed in each column = 8192, which is a maximum. Accordingly, it seems the truncation on these rows occurs only due to the limitation imposed by text output.
The only thing I see to get around this is to use a SUBSTRING function to grab say the first 8000 chars, then the next 8000 chars etc. etc. But this is ugly and error prone.
I should mention that SSIS and BCP are not options for me.
Does anyone have a better suggestion? Thanks!
You can export the data to a flat file which will not be truncated. To do this:
Right click the Database
Click Tasks -> Export Data
Select your Data Source (defaults should be fine)
Choose "Flat File Destination" for the Destination type.
Pick a file name for the output.
On the "Specify Table Copy or Query", choose "Write a query to specify the data to transfer"
Paste in your query
Remaining steps should be self explanatory. This will output the file to text and you can open it in your favorite text editor.
I also use XML but a slightly different method that gets around most of the issues with XML entitisation.
declare #VeryLongText nvarchar(max) = '';
SELECT top 100 #VeryLongText = #VeryLongText + '
' + OBJECT_DEFINITION(object_id)
FROM sys.all_objects
WHERE type='P' and is_ms_shipped=1
SELECT LEN(#VeryLongText)
SELECT #VeryLongText AS [processing-instruction(x)] FOR XML PATH('')
PRINT #VeryLongText /*WILL be truncated*/
Make sure that the "XML data" limit in SSMS is set sufficiently high!
Did you try this simple solution? Only 2 clicks away!
At the query window,
set query options to "Results to Grid", run your query
Right click on the results tab at the grid corner, save results as any files
You will get all the text you want to see in the file!!! I can see 130,556 characters for my result of a varchar(MAX) field
My solution was a bit round-about but got me there (as long as the output is less than 65535 characters):
In SQL Management Studio, set the limit for grid results to 65535 (Tools > Options > Query Results > SQL Server > Results to Grid > Non XML data)
Run the query, output to grid
Right-click the results, choose "Save Results As..." and save the results to a file
Open the file in notepad or similar to get the output
UPDATE: To demonstrate that this works, here's some SQL that selects a single 100,000 character column. If I save the grid output to a csv file, all 100,000 characters are there with no truncation.
DECLARE #test nvarchar(MAX), #i int, #line nvarchar(100)
SET #test = ''; SET #i = 100
WHILE #i < 100000
BEGIN
SET #test = #test + STUFF(REPLICATE('_', 98) + CHAR(13) + CHAR(10), 1, LEN(CAST(#i AS nvarchar)), CAST(#i AS nvarchar))
SET #i = #i + 100
END
SELECT #test
Notes:
It doesn't seem to make any difference what the character length setting is, as I orignally thought.
I'm using SQL 2008 R2 (both the server and Management Studio)
It doesn't seem to make a difference if the long column is stored in a local variable (as in this example), or selected from an actual table
I ran in to this trying to export XML. This is the solution I used:
Select the Result to Grid option, right click the link that shows up in the Results pane, then select Save Results As, choose the All Files file type, give the file a name and click Save. All the xml data is saved correctly to a file.
I'm using SSMS 10, and I could't get Torre's solution to work. The export wizard kept thinking the input column was an image:
The data type for "input column "XML_F52E2B61-18A1-11d1-B105-00805F49916B" (26)" is DT_IMAGE
In SSMS if you select data from a row it is limited to a small number of characters, but if you Edit data from a row, the full value will be there. It might not always be there but if you ctrl-a, ctrl-c then past it in an editor it will all be there.
If given a choice I would have the query return the data as "For XML Auto" or "For XML Raw" or "For XML explicit" that way the limitations are much higher and you can do much more with the outputed results.
I usually use XML to get huge debug string as output (using test harness from Luke):
declare #test nvarchar(max), #i int, #line nvarchar(100)
set #test = ''; set #i = 100
while #i < 100000
begin
set #test = #test + stuff(replicate('_', 98) + char(13) + char(10), 1, len(cast(#i as nvarchar)), cast(#i as nvarchar))
set #i = #i + 100
end
-- ctrl+d for "results to grid" then click the xml output
--select cast('<root>' + #test + '</root>' as xml)
-- revised
select #test for xml path(''), type;
Another workaround , use HeidiSql for this tricky queries. It does not have the limits in the field lenght.
The truncation you are talking about only happens in Management Studio. If you pull the column into another app, it will not be truncated.
There's no way you're using Query Analyzer to talk to SQL Server 2005. Do you mean Management Studio?
Related
I have a table with 270,000 rows and I wish to export two of the columns into XML.
I use
SELECT
[WordItem] as w, [Definition] as d
FROM
[dbo].[Dictionary]
FOR XML Path('Word')
to create the XML file, but when I "Save As", the file has random line breaks and which splits some of the XML tags.
Is there a way to fix or avoid this? I want to then convert the XML file to JSON and it's proving difficult because of this (Any advice on how to convert easily convert a large XML file to JSON is welcome too!)
In the meanwhile - as long as SQL Server hasn't got native JSON support - you might try it like this:
DECLARE #json VARCHAR(MAX)=
(
SELECT '{\n"root": {\n"Word": [' +
STUFF(
(
SELECT ',\n{\n"w": "' + [WordItem] + '",\n"d": "' + [Definition] + '"\n}'
FROM [dbo].[Dictionary]
FOR XML PATH('')
),1,1,'')
+ '\n]\n}\n}'
);
SELECT REPLACE(#json,'\n',CHAR(13)+CHAR(10));
If the generated string is bigger than 8192 characters you won't be able to see the full string in SSMS. There is a trick:
SELECT REPLACE(#json,'\n',CHAR(13)+CHAR(10))
FOR XML PATH('');
This will put the generated string in an XML viewer. In the query options you must set the DataGrid XML output size to unlimited. This is a way to get almost any size back...
I ran into this too. The inserted line breaks are not random. It just looks it due to the other (intended) line breaks. They don't show up in SQL Managment Studio's "XML Editor", but when you right click and save on the data, SQL Managment Studio inserts a \r\n after 2033 characters (awfully close to 2048). That seems to match what I was getting when just copy-pasting the "row" from the DataGrid into a text file. Setting the DataGrid XML output size from 2Mb to unlimited doesn't seem to fix this. Copy-paste from the "XML Editor" wasn't an option due to the large data size. "Results to text" and "Results to file" didn't help either.
I ended up replacing legitimate line breaks with ¶ and then replacing newlines with empty string, and ¶ back to new lines with regexs later when I actually wanted to use the XML as XML.
So, in sql:
REPLACE(REPLACE([Value], CHAR(10), '¶'), char(13), '')
then in javascript
parseXML = ( new window.DOMParser() ).parseFromString( data.replace(/\r\n/g, "").replace(/¶/g,"\r\n"), "text/xml");
Side note CAST('<![CDATA[' + [Value] + ']]>' AS XML) [Value] seems to remove the CDATA tag, and encode [Value] into xml. I was sorely confused when my CDATA tags disappeared.
I ran into the same issue today.
Try Azure Data Studio it does not have the same limitations as SSMS.
Works like a charm (at least on the built in preview}
I have a column (col1) with nvarchar(max).
I am trying to do
DECLARE #my_string NVARCHAR(max)
set #my_string = N'test'
UPDATE dbo.tab1
SET col1 = #my_string + ISNULL(col1, N'')
no luck , I have no idea why it is happening. #marc_s
The string value in col1 getting truncated after 250 characters. This happening in both SQL Server 2005 and 2008.
First of all - I'm not seeing this behavior you're reporting. How and why do you think your column gets truncated at 250 characters?? Are you using a tool to inspect the data that might be truncating the output??
You could check the length of the column:
SELECT col1, LEN(col1) FROM dbo.tab1
Is it really only 250 characters long???
Also: you're mixing VARCHAR (in #my_string) and NVARCHAR (col1) which can lead to messy results. Avoid this!
Next: if you want NVARCHAR(MAX), you need to cast your other strings to that format.
Try this:
DECLARE #my_string NVARCHAR(200)
set #my_string = N'test'
UPDATE dbo.tab1
SET col1 = CAST(#my_string AS NVARCHAR(MAX)) + ISNULL(col1, N'')
As I said - in my tests, I didn't need to do this - but maybe it works in your case?
Go to menu - Query --> Query options --> Results --> Text
There is an option Maximun number of characters displayed in each column and mine was defaulted to 256.
Once I set this to 1000 the problem was fixed.
Do you test in SSMS? If so, check in options Query Results > SQL Server > Results to Grid - Maximum characters retrieved > Non XML data. Is there a value 250 or similar?
I am entering error information into an ErrorLog table in my database. I have a utility class to do this:
ErrorHandler.Error("Something has broken!!\n\nDescription");
This works fine. However, when I try to access this table, the line breaks no longer seem to be present.
If I SELECT the table:
SELECT * from ErrorLog ORDER BY ErrorDate
there are no line breaks present in the log. This is kind of expected, as line breaks in one-line rows would break the formatting. However, If I copy the data out, the line break characters have been lost, and the data is all on one line.
How do I get line breaks in data at the end of my query when I put line breaks in? I don't know if the string has been stripped of line breaks when it enters the table, or if the viewer in SQL Server Management Studio has stripped out the line breaks.
The data type of the column into which error messages are put is nvarchar(Max), if that makes a difference.
EDIT: Unexpectedly, Pendri's solution didn't work.
Here is an excerpt of the string just before it passes into the SQL server:
POST /ipn/paymentResponse.ashx?installation=272&msgType=result HTTP/1.0\n\rContent-Length: 833\n\rContent-Type:
And here is the same string when I extract it from the grid viewer in SQL Server Management Studio:
POST /ipn/paymentResponse.ashx?installation=272&msgType=result HTTP/1.0 Content-Length: 833 Content-Type:
The place where the line break should be has been double spaced.
Any ideas?
No need to replace string input\output, you need just pick up correct option:
Tools -> Options...
> Query Results
> SQL Server
> Results to Grid
set "Retain CR\LF on copy or save" to true.
And don't forget to restart your management studio!
according Charles Gagnon answer
SSMS replaces linebreaks with spaces in the grid output. If you use Print to print the values (will go to your messages tab) then the carriage returns will be displayed there if they were stored with the data.
Example:
SELECT 'ABC' + CHAR(13) + CHAR(10) + 'DEF'
PRINT 'ABC' + CHAR(13) + CHAR(10) + 'DEF'
The first will display in a single cell in the grid without breaks, the second will print with a break to the messages pane.
A quick and easy way to print the values would be to select into a variable:
DECLARE #x varchar(100);
SELECT #x = 'ABC' + CHAR(13) + CHAR(10) + 'DEF';
PRINT #x;
Update a couple years later.
As described here, one solution to preserve viewing linebreaks in SSMS is to convert the output to XML:
SELECT * FROM (
SELECT * from ErrorLog ORDER BY ErrorDate
) AS [T(x)] FOR XML PATH
Fortunately, if you have SSMS 2012, this is no longer an issue, as line breaks are retained.
I echo David C's answer, except you should use the "TYPE" keyword so that you can click to open the data in a new window.
Note that any unsafe XML characters will not work well with either of our solutions.
Here is a proof of concept:
DECLARE #ErrorLog TABLE (ErrorText varchar(500), ErrorDate datetime);
INSERT INTO #ErrorLog (ErrorText, ErrorDate) VALUES
('This is a long string with a' + CHAR(13) + CHAR(10) + 'line break.', getdate()-1),
('Another long string with' + CHAR(13) + CHAR(10) + '<another!> line break.', getdate()-2);
SELECT
(
SELECT ErrorText AS '*'
FOR XML PATH(''), TYPE
) AS 'ErrorText',
ErrorDate
FROM #ErrorLog
ORDER BY ErrorDate;
I can confirm that the line breaks are preserved when copying out of a grid in SSMS 2012.
try using char(13) + char(10) instead of '\n' in your string (define a constant and concatenate to your sql)
Another simple solution is to click the "results to text" button in SSMS. Its not super clean, but gives you visibility to line breaks with about half a second of work.
For SQL Server 2008, there is no provision to set "Retain CR\LF on copy or save" to true.
For this issue, what I did is that, replace char(13) with "\r" and replace char(10) with "\n" like below.
REPLACE(REPlACE([COLUMN_NAME],char(13), '\r'),CHAR(10),'\n')
And in the code-behind again I've replaced "\r\n" with a break tag.
I worked out in this way as there was no option provided in SQL 2008 as mentioned above. This answer might be an alternative though.
Thanks
Software I'm working with uses a text field to store XML. From my searches online, the text datatype is supposed to hold 2^31 - 1 characters. Currently SQL Server is truncating the XML at 65,535 characters every time. I know this is caused by SQL Server, because if I add a 65,536th character to the column directly in Management Studio, it states that it will not update because characters will be truncated.
Is the max length really 65,535 or could this be because the database was designed in an earlier version of SQL Server (2000) and it's using the legacy text datatype instead of 2005's?
If this is the case, will altering the datatype to Text in SQL Server 2005 fix this issue?
that is a limitation of SSMS not of the text field, but you should use varchar(max) since text is deprecated
Here is also a quick test
create table TestLen (bla text)
insert TestLen values (replicate(convert(varchar(max),'a'), 100000))
select datalength(bla)
from TestLen
Returns 100000 for me
MSSQL 2000 should allow up to 2^31 - 1 characters (non unicode) in a text field, which is over 2 billion. Don't know what's causing this limitation but you might wanna try using varchar(max) or nvarchar(max). These store as many characters but allow also the regular string T-SQL functions (like LEN, SUBSTRING, REPLACE, RTRIM,...).
If you're able to convert the column, you might as well, since the text data type will be removed in a future version of SQL Server. See here.
The recommendation is to use varchar(MAX) or nvarchar(MAX). In your case, you could also use the XML data type, but that may tie you to certain database engines (if that's a consideration).
You should have a look at
XML Support in Microsoft SQL Server
2005
Beginning SQL Server 2005 XML
Programming
So I would rather try to use the data type appropriate for the use. Not make a datatype fit your use from a previous version.
Here's a little script I wrote for getting out all data
SELECT #data = N'huge data';
DECLARE #readSentence NVARCHAR (MAX) = N'';
DECLARE #dataLength INT = ( SELECT LEN (#data));
DECLARE #currIndex INT = 0;
WHILE #data <> #readSentence
BEGIN
DECLARE #temp NVARCHAR (MAX) = N'';
SET #temp = ( SELECT SUBSTRING (#data, #currIndex, 65535));
SELECT #temp;
SET #readSentence += #temp;
SET #currIndex += 65535;
END;
In this live SQL Server 2008 (build 10.0.1600) database, there's an Events table, which contains a text column named Details. (Yes, I realize this should actually be a varchar(MAX) column, but whoever set this database up did not do it that way.)
This column contains very large logs of exceptions and associated JSON data that I'm trying to access through SQL Server Management Studio, but whenever I copy the results from the grid to a text editor, it truncates it at 43679 characters.
I've read on various locations on the Internet that you can set your Maximum Characters Retrieved for XML Data in Tools > Options > Query Results > SQL Server > Results To Grid to Unlimited, and then perform a query such as this:
select Convert(xml, Details) from Events
where EventID = 13920
(Note that the data is column is not XML at all. CONVERTing the column to XML is merely a workaround I found from Googling that someone else has used to get around the limit SSMS has from retrieving data from a text or varchar(MAX) column.)
However, after setting the option above, running the query, and clicking on the link in the result, I still get the following error:
Unable to show XML. The following error happened:
Unexpected end of file has occurred. Line 5, position 220160.
One solution is to increase the number of characters retrieved from the server for XML data. To change this setting, on the Tools menu, click Options.
So, any idea on how to access this data? Would converting the column to varchar(MAX) fix my woes?
SSMS only allows unlimited data for XML data. This is not the default and needs to be set in the options.
One trick which might work in quite limited circumstances is simply naming the column in a special manner as below so it gets treated as XML data.
DECLARE #S varchar(max) = 'A'
SET #S = REPLICATE(#S,100000) + 'B'
SELECT #S as [XML_F52E2B61-18A1-11d1-B105-00805F49916B]
In SSMS (at least versions 2012 to current of 18.3) this displays the results as below
Clicking on it opens the full results in the XML viewer. Scrolling to the right shows the last character of B is preserved,
However this does have some significant problems. Adding extra columns to the query breaks the effect and extra rows all become concatenated with the first one. Finally if the string contains characters such as < opening the XML viewer fails with a parsing error.
A more robust way of doing this that avoids issues of SQL Server converting < to < etc or failing due to these characters is below (credit Adam Machanic here).
DECLARE #S varchar(max)
SELECT #S = ''
SELECT #S = #S + '
' + OBJECT_DEFINITION(OBJECT_ID) FROM SYS.PROCEDURES
SELECT #S AS [processing-instruction(x)] FOR XML PATH('')
I was able to get this to work...
SELECT CAST('<![CDATA[' + LargeTextColumn + ']]>' AS XML) FROM TableName;
One work-around is to right-click on the result set and select "Save Results As...". This exports it to a CSV file with the entire contents of the column. Not perfect but worked well enough for me.
Did you try this simple solution? Only 2 clicks away!
At the query window,
set query options to "Results to Grid", run your query
Right click on the results tab at the grid corner, save results as any files
You will get all the text you want to see in the file!!! I can see 130,556 characters for my result of a varchar(MAX) field
The simplest workaround I found is to backup the table and view the script. To do this
Right click your database and choose Tasks > Generate Scripts...
"Introduction" page click Next
"Choose Objects" page
Choose the Select specific database objects and select your table.
Click Next
"Set Scripting Options" page
Set the output type to Save scripts to a specific location
Select Save to file and fill in the related options
Click the Advanced button
Set General > Types of data to script to Data only or Schema and Data and click ok
Click Next
"Summary Page" click next
Your sql script should be generated based on the options you set in 4.2. Open this file up and view your data.
The data type TEXT is old and should not be used anymore, it is a pain to select data out of a TEXT column.
ntext, text, and image (Transact-SQL)
ntext, text, and image data types
will be removed in a future version of
Microsoft SQL Server. Avoid using
these data types in new development
work, and plan to modify applications
that currently use them. Use
nvarchar(max), varchar(max), and
varbinary(max) instead.
you need to use TEXTPTR (Transact-SQL) to retrieve the text data.
Also see this article on Handling The Text Data Type.
It sounds like the Xml may not be well formed. If that is the case, then you will not be able to cast it as Xml and given that, you are limited in how much text you can return in Management Studio. However, you could break up the text into smaller chunks like so:
With Tally As
(
Select ROW_NUMBER() OVER ( ORDER BY s1.object_id ) - 1 As Num
From sys.sysobjects As s1
Cross Join sys.sysobjects As s2
)
Select Substring(T1.textCol, T2.Num * 8000 + 1, 8000)
From Table As T1
Cross Join Tally As T2
Where T2.Num <= Ceiling(Len(T1.textCol) / 8000)
Order By T2.Num
You would then need to manually combine them again.
EDIT
It sounds like there are some characters in the text data that the Xml parser does not like. You could try converting those values to entities and then try the Convert(xml, data) trick. So something like:
Update Table
Set Data = Replace(Cast(Data As varchar(max)),'<','<')
(I needed to cast to varchar(max) because the replace function will not work on text columns. There should not be any reason you couldn't convert those text columns to varchar(max).)
You are out of luck, I think. THe problem is not a SQL level problem as all other answers seem to focus on, but simply one of the user interface. Management Studio is not meant to be a general purpose / generic data access interface. It is not there to be your interface, but your administrative area, and it has serious limitations handling binary data and large test data - because people using it within the specified usage profile will not run into this problem.
Presenting large text data is simply not the planned usage.
Your only choice would be a table valued function that takes the text input and cuts it rows for every line, so that Management Studio gets a list of rows, not a single row.
I prefer this simple XML hack which makes columns clickable in SSMS on a cell-by-cell basis. With this method, you can view your data quickly in SSMS’s tabular view and click on particular cells to see the full value when they are interesting. This is identical to the OP’s technique except that it avoids the XML errors.
SELECT
e.EventID
,CAST(REPLACE(REPLACE(e.Details, '&', '&'), '<', '<') AS XML) Details
FROM Events e
WHERE 1=1
AND e.EventID BETWEEN 13920 AND 13930
;
Starting from SSMS 18.2, you can now view up to 2 million characters in the grid results. Source
Allow more data to be displayed (Result to Text) and stored in cells
(Result to Grid). SSMS now allows up to 2M characters for both.
I verified this with the code below.
DECLARE #S varchar(max) = 'A'
SET #S = REPLICATE(#S,2000000) + 'B'
SELECT #S as a
declare #takeOver table(details nvarchar(max))
declare #json_auto nvarchar(max)
select #json_auto = (select distinct
From table_1 cg
inner join table_2 c
on cg.column_1= c.column_1and cg.isDeleted =0 and c.isdeleted = 0
inner join table_3 d
on c.column_2= d.column_2 and d.isdeleted = 0
where cg.Id= 1017
for Json Auto)
insert into #takeOver
values(#json_auto)
select * from #takeOver