I am stuck at a problem for which I cannot find any reason or solution.
I am running a SQL script to export some data to an Excel sheet. There is an application running on the other end which reads and processes the Excel sheet.
Problem: The column headers are being displayed at the bottom and the application is expecting them to be on the top row. I cannot change the functioning of the application.
This was working fine in SQL 2005, but we recently updated to SQL 2012 and this started happening.
I have not found anything over the internet to solve this issue.
This is the SQL script that I am executing
SELECT
#columnNames = COALESCE( #columnNames + ',', '') + '['+ column_name + ']',
#columnConvert = COALESCE( #columnConvert + ',', '') + 'convert(nvarchar(4000),'
+ '['+ column_name + ']' +
case
when data_type in ('datetime', 'smalldatetime') then ',121'
when data_type in ('numeric', 'decimal') then ',128'
when data_type in ('float', 'real', 'money', 'smallmoney') then ',2'
when data_type in ('datetime', 'smalldatetime') then ',120'
else ''
end + ') as ' + '['+ column_name + ']'
FROM tempdb.INFORMATION_SCHEMA.Columns
WHERE table_name = '##TempExportData'
-- execute select query to insert data and column names into new temp table
SELECT #sql = 'select ' + #columnNames + ' into ##TempExportData2 from (select ' + #columnConvert + ', ''2'' as [temp##SortID] from ##TempExportData union all select ''' + replace(replace(replace(#columnNames, ',', ''', '''),'[',''),']','') + ''', ''1'') t order by [temp##SortID]'
exec (#sql)
-- build full BCP query
DECLARE #bcpCommand VARCHAR(8000)
SET #bcpCommand = 'bcp " SELECT * from ##TempExportData2" queryout'
SET #bcpCommand = #bcpCommand + ' ' + #fullFileName + ' -T -w -S' + #serverInstance
EXEC master..xp_cmdshell #bcpCommand
Where TempExportData2 holds the data that along with column headers
I think I understand the problem: You are using the order by in the select into instead of in the final select statement.
You should know that data inside tables is considered unorderd and Sql Server (and any other rdbms I know, actually) does not guarantee the order of rows selected if the select statement does not contain an order by clause.
Therefor, you should add the [temp##SortID] column to your ##TempExportData2 table and use it to sort the last select statement:
SET #bcpCommand = 'bcp " SELECT * from ##TempExportData2 ORDER BY [temp##SortID]" queryout'
Since you don't need that column in the output query, so you might want to specify the column names in that select statement. However, if it's not causing damage to your application that reads the excel file or to the data it produces, I would suggest keeping the select * to make the query more readable.
Related
I am working on exporting data from one environment to another environment. I want to select the list of tables which has new set of records either inserted or modified.
Database has around 200 tables and only if 10 table records are impacted since yesterday, i want to filter only those tables. Some of these tables does not have createdate table column. It is harder to identify the record difference based on plain select query to the table.
How to find the list of tables which has new set of records impacted in SQL?
And if possible only those newly impacted records from the identified tables.
I tried with this query, however this query is not returning actual tables.
select * from sysobjects where id in (
select object_id
FROM sys.dm_db_index_usage_stats
WHERE last_user_update > getdate() - 1 )
If you've not got a timestamp or something to identify newly changed records such as auditing, utilising triggers or Change Data Capture enabled on those tables, it's quiet impossible to do.
However, reading your scenario is it not possible to ignore what has changed or been modified and just simply export those 200 tables from one environment to the other and override it on the destination location?
If not, then you might be only interested in comparing data rather than identifying newly changed records to identify which tables did not match. You can do that using EXCEPT
See below example of comparing two databases with the same table names and schema creating a dynamic SQL statement column using EXCEPT from both databases on the fly and running them in a while loop; inserting each table name that was effected into a temp table.
DECLARE #Counter AS INT
, #Query AS NVARCHAR(MAX)
IF OBJECT_ID('tempdb..#CompareRecords') IS NOT NULL DROP TABLE #CompareRecords
IF OBJECT_ID('tempdb..#TablesNotMatched') IS NOT NULL DROP TABLE #TablesNotMatched
CREATE TABLE #TablesNotMatched (ObjectName NVARCHAR(200))
SELECT
ROW_NUMBER() OVER( ORDER BY (SELECT 1)) AS RowNr
, t.TABLE_CATALOG
, t.TABLE_SCHEMA
, t.TABLE_NAME
, Query = 'IF' + CHAR(13)
+ '(' + CHAR(13)
+ ' SELECT' + CHAR(13)
+ ' COUNT(*) + 1' + CHAR(13)
+ ' FROM' + CHAR(13)
+ ' (' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' EXCEPT' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t2.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' ) AS sq' + CHAR(13)
+ ') > 1' + CHAR(13)
+ 'SELECT ' + QUOTENAME(QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME), '''''') + ' AS TableNameRecordsNotMatched'
INTO #CompareRecords
FROM <UAT_DATABASE>.INFORMATION_SCHEMA.TABLES AS t
LEFT JOIN <PROD_DATABASE>.INFORMATION_SCHEMA.TABLES AS t2 ON t.TABLE_SCHEMA = t2.TABLE_SCHEMA
AND t.TABLE_NAME = t2.TABLE_NAME
WHERE t.TABLE_TYPE = 'BASE TABLE'
SET #Counter = (SELECT MAX(RowNr) FROM #CompareRecords)
WHILE #Counter > 0
BEGIN
SET #Query = (SELECT cr.Query FROM #CompareRecords AS cr WHERE cr.RowNr = #Counter)
INSERT INTO #TablesNotMatched
EXECUTE sp_executesql #Query
SET #Counter = #Counter - 1
END
SELECT
*
FROM #TablesNotMatched
Note when using EXCEPT both tables have to have the exact same column count and type.
I hope this slightly helps.
I am trying to calculate total row size for a table, whose formula is
row_size = Fixed data size + variable data size + null bitmap +4.
for this I am trying to find the average variable data size through query in SQL. Is there a query that can be used in SQL which would achieve this?
not a complete answer, but how about something like this
build dynamic sql from the INFORMATION_SCHEMA.COLUMNS table -
you could maybe introduce your own flag to flag variable/non-variable length fields.
Building the SQL using a cursor is an option too
--data lengths of each table
DECLARE #SQLa as nvarchar(max) ='';
select #SQLa = #SQLa +
'SELECT COALESCE(DATALENGTH(['+ Column_Name +']), 0) as dlen, '''
+ Column_Name
+ ''' colname, '''
+ Table_name
+ ''' tabname FROM ['
+ Table_name
+ '] UNION ALL ' FROM INFORMATION_SCHEMA.COLUMNS
SELECT #SQLa = LEFT(#SQLa, LEN(#SQLa) -10)
SELECT #SQLa = 'SELECT DQ.tabname, SUM(DQ.dlen) as TotalLen FROM (' + #SQLa + ') DQ GROUP BY tabname'
select #SQLa;
EXEC sp_executesql #sqla;
I'm inserting data from one table with twenty columns & 300 rows to another table. Mid-run, I get error
Msg 8152, Level 16, State 14, Line 1
String or binary data would be truncated.
I know what the error means, but how do I find which value, or at least which row, is generating this error?
The INSERT statement is something straightforward like this:
Insert into Employee (col1, col2, col3,..., col20)
select col1, col2, col3,..., col20 from TEMP_EMPL
Thanks.
According to this answer it's not directly possible. There is also an old support ticket to add the name of the column to this error message.
To find the reason you can disable this error by setting
set ANSI_WARNINGS OFF
insert the data and compare the maximum length of the column with current maximum length of the data. After that you can limit your search to columns that are filled to the maximum. For example you can use the following code (just change #TableName and #TableSchema):
DECLARE #TableName sysname = 'test', #TableSchema sysname = 'dbo'
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL = STUFF((SELECT
'
UNION ALL
select ' + QUOTENAME(Table_Name,'''') + ' AS Table_Name, ' +
QUOTENAME(Column_Name,'''') + ' AS ColumnName, MAX(' +
CASE WHEN DATA_TYPE IN ('XML','HierarchyID','Geometry','Geography','text','ntext')
THEN 'DATALENGTH(' ELSE 'LEN(' END + QUOTENAME(Column_Name) +
')) as [Max Length], ' + QUOTENAME(C.DATA_TYPE,'''') + ' AS Data_Type, ' +
CAST(COALESCE(C.CHARACTER_MAXIMUM_LENGTH, C.NUMERIC_SCALE,0) AS VARCHAR(10)) +
' AS Data_Width FROM ' + QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(Table_Name)
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE TABLE_NAME = #TableName
AND table_schema = #TableSchema
--AND DATA_TYPE NOT IN ('XML','HierarchyID','Geometry','Geography')
ORDER BY COLUMN_NAME
FOR XML PATH(''),Type).value('.','varchar(max)'),1,11,'')
--print #SQL
EXECUTE (#SQL)
I have a procedure that does a "select * from view" and this output contains an inverted comma in one of the columns. I want to do a replace() to remove the extra inverted comma but I cannot change the "select *" as this is a dynamic query that is used in different scenarios.
As evident, the following query does not work
select replace(*, '"', '') from ReportRegistry
Some help appreciated.
I wanted to know if there is any way I can remove the quotes without knowing the column names. I want them to be removed from all columns. The columns names are different in different cases.
The view is defined within the procedure differently in different cases. The dynamic colum names are added as text and them executed using sp_executesql.
DECLARE #ColNames NVARCHAR(MAX) = ''
--Dynamically build a list of column names for the view, separated by commas.
SELECT #ColNames = #ColNames +
CASE
--Use the REPLACE function for "String" type data types. Did I leave any data types out?
--Change "x" to the character(s) you want to replace.
WHEN c.DATA_TYPE IN ('VARCHAR', 'CHAR', 'NVARCHAR', 'NCHAR') THEN 'REPLACE([' + c.COLUMN_NAME + '], ''x'', '''') AS [' + c.COLUMN_NAME + '],'
--All other data types.
ELSE '[' + c.COLUMN_NAME + '],'
END
--In addition to user tables, views are included in INFORMATION_SCHEMA.COLUMNS (and in INFORMATION_SCHEMA.TABLES).
FROM INFORMATION_SCHEMA.COLUMNS c
WHERE c.TABLE_NAME = 'ReportRegistry'
--Retain original order of the view's columns.
ORDER BY c.ORDINAL_POSITION
--Remove last comma
SET #ColNames = LEFT(#ColNames, LEN(#ColNames) - 1)
EXEC ('SELECT ' + #ColNames + ' FROM ReportRegistry ')
select replace(TargetColumnName, '"', '')
,Column2
,Column3
,..... and so on.....
from ReportRegistry
declare #tbl varchar(max) = 'table_or_view_name'
declare #cols varchar(max) =
(select case when data_type='varchar' then 'REPLACE('+column_name+',''"'','''')' else column_name end+char(10)+','
from information_schema.columns
where table_name = #tbl for xml path(''))
declare #sql varchar(max)='select '+left(#cols,len(#cols)-1)+' from '+#tbl
print #sql
exec(#sql)
I have a SQL Server table with 100 columns. I would like to do this to each column:
UPDATE 2011INFODATA
SET [Attribute_Code_1] = ltrim(rtrim(Attribute_Code_1))
WHERE len(ltrim(rtrim(Attribute_Code_1))) > 0
My question is how I can use the informatin_schema.columns to dynamically generate or insert the various column names into the above SQL?
I could of course do it by hand for each column and get the same result. But that would be no fun and I want to use this same sql for many tables.
Something along these lines will do it.
DECLARE #Sql nvarchar(max)
SET #Sql = '';
SELECT #Sql = #Sql + '
UPDATE 2011INFODATA
SET ' + QUOTENAME(COLUMN_NAME) + ' = ltrim(rtrim(' + QUOTENAME(COLUMN_NAME) + '))
WHERE len(ltrim(rtrim(' + QUOTENAME(COLUMN_NAME) + '))) > 0'
FROM information_schema.columns
WHERE TABLE_NAME = '2011INFODATA' AND DATA_TYPE LIKE '%char'
EXEC sp_executesql #Sql
Edit: updated sample SQL to incorporate helpful comments from #billinkc, #Martin Smith, #Aaron Bertrand.