Fairly new to SQL, but saved a huge query and when I try to open it's asking me for "encoding".
enter image description here
I chose auto-detect as default but it just opens up a new query with a few random symbols when my query was hundreds of lines long.
Any idea how to get it back or what option to choose here? Is there a chance I saved over the file by mistake?
On SSMS, select Tools/Options then expand (Text Editor), add the extension (sql) and map it to (SQL Query Editor).
This solution works if you are working with ASCII characters.
Related
I have dozens of SSRS Reports and the corresponding stored procedures that I am slowly cleaning up and optimizing, I am finding a number of data sets that have extra fields that are not used in the actual report, they are usually the result of a SELECT * that is slowing down the SP end of things significantly in a few places.
I am wondering if there is a quicker way to see which fields from the datasets are used/unused to more efficiently clean up the stored procedures. It seems like clicking through to each <<expr>> and checking them off is a ridiculous way to go about this.
I'll tell you, I wish I knew a tool that simplifies this for you.
And I don't off the top of my head.
But for sure I know you can search the text of the rdl and find these details.
I do this often when troubleshooting problems with existing reports (or SSIS packages).
The .rdl files are human-readable xml.
You can open any one file in a text editor and search the text - even Visual Studio if you "Open File" rather than use the Report project.
Given that, of course you can write a routine in your preferred programming language that
finds the query or proc in the data source of the Report
runs it (as metadata only) to get all the columns
search for each one in the text of the rdl
you can be more specific if you use xml queries to limit
the search to more realistic targets like display box Data Sources
Sorry I don't have a more convenient answer like an existing tool.
If I remember, I may look for one because this is a big problem for "corporate coders" like us.
If I can't find one, maybe I'll write the script in .net and come back and post it :)
Yes you can ! Use the following steps
right click the rdl file and
click the View Code . This will be an XML format
CTR + F to get a search text box
Enter the name of any field in the text box.
Use the Forward Arrow icon to see the number of occurrence of the searched field name
If the text field is in the dataset and Tablixcell value, then it shows it's being used in the report
If the text field is only in the dataset and not any tablixcell value, then it's not being used in the report.
I have a huge amount of trouble loading spreadsheets into a SQL Server database.
Currently, I'm using an SSIS package to load the data and I have had to make lots of adjustments to get the data to load:
All numbers must be formatted as text (otherwise they don't load properly).
Sometimes numbers must be preceded with single quote (') to get them to load.
If a column has a mix of number cells and text cells, the text cells must come first in the file (otherwise only numbers load and text comes in as NULL).
If a user changes a column name the file will not load.
If a user changes a tab name the file won't load.
If a user adds a new column (even at the end of a sheet) the file won't load.
Extra sheets in the file is not a problem, thankfully!
Dates seem sensitive whether or not they will load properly.
Connection strings to the Excel file must include "IMEX=1" or things are worse.
Scheduled SSIS jobs must be run as 32-bit even on 64-bit system.
I've been loading the data (usually 200,000-500,000 rows per file) into a table with all fields defined as nvarchar. Then, when loaded I transfer that data in the next step of the SSIS package to the working table with typed data fields.
All of the requirements that I must put on the user for how to format the Excel file is really a pain. We usually have to send the file back multiple times until all the formatting issues are correct before the file will load. I'd like to eliminate this thrash.
I know I'm not the only one that is facing this type of problem. So, I must ask...
What is a better alternative to Excel for loading data into a SQL Server database?
Or, am I going about this the wrong way? Should I be using something other than SSIS to load Excel spreadsheets?
You can try OpenRowSet:
SELECT *
INTO SomeTable
From OpenRowSet('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=\\servername\c$\filename.xls;HDR=YES;IMEX=1', [Sheet2$])
Not really a SQL answer, but an easy one:
You could require the users to copy and paste data to an excel spreadsheet where everything but the data fields to be included are locked. This will prevent many of the pain points described.
Ok I'm using SQL Server 2008 and have a table field of type VARCHAR(MAX). Problem is that when saving information using Hibernate, the contents of VARCHAR(MAX) field is getting truncated. I don't see any error messages on either the app server or database server.
The content of this field is just a plain text file. The size of this text file is 383KB.
This is what I have done so far to troubleshoot this problem:
Changed the database field from VARCHAR(MAX) to TEXT and same
problem occurs.
Used the SQL Server Profiler and I noticed that the full text
content is being
received by the database server, but for some reason the profiler freezes when trying
to view the SQL with the truncation problem. Like I said, just before it freezes, I
did noticed that the full text file content (383KB) are being received, so it seems
that it might be the database problem.
Has anyone encountered this problem before? Any ideas what causes this truncation?
NOTE: just want to mention that I'm just going into SQL Studio and just copying the TEXT field content and pasting it to Textpad. That's how I noticed it's getting truncated.
Thanks in advance.
Your problem is that you think Management Studio is going to present you with all of the data. It doesn't. Go to Tools > Options > Query Results > SQL Server. If you are using Results to Grid, change "Maximum Characters Retrieved" for "Non XML data" (just note that Results to Grid will eliminate any CR/LF). If you are using Results to Text, change "Maximum number of characters displayed in each column."
You may be tempted to enter more, but the maximum you can return within Management Studio is:
65535 for Results to Grid
8192 for Results to Text
If you really want to see all the data in Management Studio, you can try converting it to XML, but this has issues also. First set Results To Grid > XML data to 5 MB or unlimited, then do:
SELECT CONVERT(XML, column) FROM dbo.table WHERE...
Now this will produce a grid result where the link is actually clickable. This will open a new editor window (it won't be a query window, so won't have execute buttons, IntelliSense, etc.) with your data converted to XML. This means it will replace > with > etc. Here's a quick example:
SELECT CONVERT(XML, 'bob > sally');
Result:
When you click on the grid, you get this new window:
(It does kind of have IntelliSense, validating XML format, which is why you see the squigglies.)
BACK AT THE RANCH
If you just want to sanity check and don't really want to copy all 383K elsewhere, then don't! Just check using:
SELECT DATALENGTH(column) FROM dbo.table WHERE...
This should show you that your data was captured by the database, and the problem is the tool and your method of verification.
(I've since written a tip about this here.)
try using SELECT * FROM dbo.table for XML PATH
I had a similar situation. I have an excel sheet. A couple of columns in the sheet may have more than 255 chars, sometimes even 500. A simple way was to sort the rows of data, placing the rows with the most characters up on top. You actually need just one row. When SQL imports the data, it recognizes the field being more than 255 characters and imports the entire data :)
Otherwise, they suggested using regedit to change a specific value. Didn't want to do that.
Hope this helps
I want to export my big SSMS (SQL Server Management Studio) query result (2.5m lines, 9 fields) as .csv or comma-delimited .txt (with headings). (MS SQL Server 2005 Management Studio.)
So that I can then either read it line-by-line into VBA program (to do certain calculations on the data) or do queries on it in Excel (e.g. with Microsoft Query). The calculations are complicated and I prefer to do it somewhere else than SSMS.
If I choose ‘query result to text’ in SSMS and a small answer (few lines e.g. up to 200k) I could of course simply copy and paste to a text editor. For my large answer here I could of course copy and paste 200k or so lines at a time, 10 times, into a text editor like Ultra-Edit. (When I try all 2.5m at once, I get a memory warning inside SSMS.) But for the future I’d like a more elegant solution.
For ‘query result to file’, SSMS writes to an .rpt file always. (When you right-click in the results window and choose ‘save as’, it gives a memory error just like above.)
--> So it looks like my only option is to have SSMS output its result to a file i.e. .rpt and then afterwards, convert the .rpt to .txt.
I assume this .rpt is a Crystal Reports file? Or isn't it. I don’t have Crystal Reports on my PC, so I cannot use that to convert the file.
When opening the .rpt in Ultra-Edit it looks fine. However in Microsoft Query in Excel, the headings doesn’t want to show.
When I simply read & write the .rpt using VBA, the file halves in size. (330meg to 180meg). In Microsoft Query the headings do show now (though the first field name has a funny leading character, which has happened to me before in other totally different situations). I do seem to be able to do meaningful pivot tables on it in Excel.
However when I open this new file in Ultra-Edit, it shows Chinese characters! Could there still be some funny characters in it somewhere?
--> Is there perhaps a free (and simple/ safe) converter app available somewhere. Or should I just trust that this .txt is fine for reading into my VBA program.
Thanks
Simple way: In SQL Server Management Studio, go to the "Query" menu & select "Query Options…" > Results > Text > Change "Output Format" to "Comma Delimited". Now, run your query to export to a file, and once done rename the file from .rpt to .csv and it will open in Excel :).
Here is my solution.
Use Microsoft SQL Server Management Studio
Configure it to save Tab delimited .rpt files: Go to 'Query' > 'Query Options' > 'Results' > 'Text' > 'Output Format' and choose 'Tab delimited' (press OK)
Now, when you create a report, use the 'Save With Encoding...' menu, and select 'Unicode' (by default, it's 'UTF8')
You can now open the file with Excel, and everything will be in columns, with no escaping nor foreign characters issues (note the file may be bigger due to unicode encoding).
Well with the help of a friend I found my solution: Rpt files are plain text files generated in MS SQL Server Management Studio, but with UCS-2 Little Endian encoding instead of ANSI.
--> In Ultra-Edit the option ‘file, conversion options, unicode to ASCII’ did the trick. The text file reduces from 330meg to 180 meg, Microsoft Query in Excel can now see the columns, and VBA can read the file & process lines*.
P.s. Another alternative would have been to use MS Access (which can handle big results) and connect with ODBC to the database. However then I would have to use Jet-SQL which has fewer commands than the T-SQL of MS SQL Server Management Studio. Apparently one can create a new file as .adp in MS Access 2007 and then use T-SQL to a SQL Server back end. But in MS Access 2010 (on my PC) this option seems not to exist anymore.
You can use BCP
Open a command prompt, then type this:
SET Q="select * from user1.dbo.table1"
BCP.EXE %Q% queryout query.out -S ServerName -T -c -t
You can use -U -P (instead of -T) for SQL Authentication.
Your app have a problem with UNICODE. You can force a code page using -C {code page}. If in doubt, try 850.
-t will force tab as field delimiter, you can change it for comma -t,
The nice thing is you can call this directly from your VBA running shell command.
This is the recommended way I see you can do it.
My Source (Answer from DavidAir)
Pick "results to grid" then then right-click on the grid and select "Save Results As..." This will save a CSV.
Actually, there is a problem with that if some values contain commas - the resulting CSV is not properly escaped. The RPT file is actually quite nice as it contains fixed-width columns. If you have Excel, a relatively easy way of converting the result to CSV is to open the RPT file in Excel. This will bring up the text import wizard and Excel would do a pretty good job at guessing the columns. Go through the wizard and then save the results as CSV.
I recommend using the "SQL Server Import and Export Wizard" for a couple reasons:
The output file will not have a status message at the bottom like a .rpt file does (ie. "(100 rows affected)") which may mess up your data import
Ability to specify custom row and column delimiters of a length greater than 1 character
Ability to specify custom source to destination mapping (ie. column FirstName can be mapped to first_name in the CSV)
Ability to perform a direct transfer to any other database accessible from the SSMS machine
Ability to explicitly select your file encoding and locale
It can be accessed by right-clicking on your database in the management studio (you must right-click the database and not the table) and selecting Tasks > Export Data.
When asked for data source you can select the "SQL Server Native Client" and when asked to select a destination you can select "Flat File Destination".
You are then asked to specify a table or query to use.
You can find more info about the tool here:
https://learn.microsoft.com/en-us/sql/integration-services/import-export-data/start-the-sql-server-import-and-export-wizard?view=sql-server-2017
In my case, I execute a query on SSMS (before that press CTRL+SHIFT+F) the result open a window to save it as an rpt file, I couldn´t read it (no Crystal Report install in my computer) so...next time I runned the query I saved it as (all files) set with extension *.txt, and that´s it I was able to read it as text file.
First get your data in .rpt file by using any of above method.
Default .rpt with fixed space column. (262MB)
Comma delimited with Unicode. (52MB) - I used this.
Change file extension to .csv.
Open/Import it in excel and verify data. File type is 'Text Unicode'.
Save it as CSV (Comma Delimited), which reduced size to 25 MB.
I''ve a little problem while using SQL Server Full Text Search.
Let me explain,
I've a table with a BLOB inside (a PDF file).
I've created the full text index in that table like it should be.
I've the PDF iFilter from Adobe.
BUT, when I put some files in my table and execute a search like:
SELECT *
FROM MyTable
WHERE FREETEXT(*, N'thank');
It only returns the columns from my table (well, that's what I asked, right?).
But I wanted to return the sentence where the word 'thank' was found.
Is there any way to do this?
I've been fighting with this issue for almost 2 days...
Do you have any evidence that the PDF IFilter is working from within SQL Server at all?
Just as a test put an MS Word 2003 doc in there and see if it gets indexed properly.