Batch file to insert Excel data into SQL Server table - sql-server

I'm trying to write a batch file that takes information from an Excel spreadsheet and adds new rows into my SQL Server.
I think what I have currently is nearly there. As it runs and says a line has been updated, but it hasn't.
This is my batch file:
#echo off
copy \\RDS-2012-C\2012_N_Drive\Operations\Reports\Metrics\sql2.xlsx C:\ACCESS_SQLSVR\import_location
sqlcmd -S RDS-2012-G\RDS_SQL -d Master -i C:\ACCESS_SQLSVR\import_location\sesame_import.sql
pause
And this is what it runs:
EXEC sp_configure 'Show Advanced Options', 1;
RECONFIGURE
GO
EXEC sp_configure 'Ad Hoc Distributed Queries', 1;
RECONFIGURE
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'AllowInProcess', 1;
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'DynamicParameters', 1;
GO
SELECT * FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0;Database=C:\ACCESS_SQLSVR\import_location\sql2.xlsx;HDR=YES;IMEX=1',
'SELECT * FROM [SQL$]')
GO
Do I simply need a different command instead of SELECT?
Also, the full database and table in the SQL Server is called:
Sesame_FTest.dbo.SQL
Running this when the table doesn't exist results in the table being created and filled with the Excel information. But running it a second time with a new Excel doesn't update like I thought it would.

Try it like this . . .
------ INSERT INTO NON-EXISTING TABLES
USE [YourDatabase]
GO
SELECT * INTO tbl_Mapping
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;HDR=YES;Database=C:\Users\rshuell001\Desktop\State Street\Final Deliverables\Raw Data\Mapping.xlsx',
'SELECT * FROM [Mapping$]')
SELECT * INTO tbl_Data
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;HDR=YES;Database=C:\Users\rshuell001\Desktop\State Street\Final Deliverables\Raw Data\Data.xlsx',
'SELECT * FROM [Data$]')
OR . . .
------ INSERT INTO EXISTING TABLES
USE [YourDatabase]
GO
Insert INTO tbl_Mapping
SELECT * FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;HDR=YES;Database=C:\Users\rshuell001\Desktop\State Street\Final Deliverables\Raw Data\Mapping.xlsx',
'SELECT * FROM [Mapping$]')
Insert INTO tbl_Data
SELECT * FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;HDR=YES;Database=C:\Users\rshuell001\Desktop\State Street\Final Deliverables\Raw Data\Data.xlsx',
'SELECT * FROM [Data$]')

Related

SQL query from import wizard

I imported an Excel (.xlsx) file into a table in SQL Server using the import wizard.
I want to get the query used to import so that I can store it and incorporate it in a SQL Server stored procedure. How can I get that query?
You can't get the specific query used by the wizard, you can only save the SSIS package for later use. You can do this with other methods, via sql, which you can include in your stored procedure as explained in the official documentation.
OPENROWSET
sp_configure 'show advanced options', 1;
RECONFIGURE;
GO
sp_configure 'Ad Hoc Distributed Queries', 1;
RECONFIGURE;
GO
USE ImportFromExcel;
GO
SELECT * INTO Data_dq
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0; Database=C:\Temp\Data.xlsx', [Sheet1$]);
GO
BULK INSERT
USE ImportFromExcel;
GO
BULK INSERT Data_bi FROM 'C:\Temp\data.csv'
WITH (
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
);
GO

Export xlsx Files out of a SQL Server database

I'm trying to save Excel files in my SQL Server database as BLOB and export them.
Importing the file was quiet easy, but when I'm trying to export them to the file system, the file is damaged and unreadable.
I was quite shocked, because I thought Microsoft wouldn't have much problems importing and exporting their own file types.
INSERT INTO TestBlob(tbName, tbDesc, tbBin)
SELECT
'testfile.xlsx', 'testfile', BulkColumn
FROM
OPENROWSET (Bulk 'C:\temp\testfile.xlsx', Single_Blob) AS tb
I already tried to export it by bcp but I just got a corrupted file.
Do I have to provide a special format file for xlsx files? Are there any other ways to get the xlsx files out of the database?
Thanks in advance
BR. RF
EXEC sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
EXEC sp_configure 'Ad Hoc Distributed Queries', 1;
GO
RECONFIGURE;
GO
USE [AdventureWorks];
GO
INSERT INTO OPENROWSET ('Microsoft.Jet.OLEDB.4.0', 'Excel 8.0;Database=c:\contact.xls;',
'SELECT * FROM [Sheet1$]')
SELECT TOP 5 FirstName, LastName
FROM Person.Contact
GO
https://blog.sqlauthority.com/2008/01/08/sql-server-2005-export-data-from-sql-server-2005-to-microsoft-excel-datasheet/
Or . . .
1 Export data to existing EXCEL file from SQL Server table
insert into OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=D:\testing.xls;',
'SELECT * FROM [SheetName$]') select * from SQLServerTable
2 Export data from Excel to new SQL Server table
select *
into SQLServerTable FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=D:\testing.xls;HDR=YES',
'SELECT * FROM [Sheet1$]')
3 Export data from Excel to existing SQL Server table
Insert into SQLServerTable Select * FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=D:\testing.xls;HDR=YES',
'SELECT * FROM [SheetName$]')
4 If you dont want to create an EXCEL file in advance and want to export data to it, use
EXEC sp_makewebtask
#outputfile = 'd:\testing.xls',
#query = 'Select * from Database_name..SQLServerTable',
#colheaders =1,
#FixedFont=0,#lastupdated=0,#resultstitle='Testing details'
(Now you can find the file with data in tabular format)
5 To export data to new EXCEL file with heading(column names), create the following procedure
create procedure proc_generate_excel_with_columns
(
#db_name varchar(100),
#table_name varchar(100),
#file_name varchar(100)
)
as
--Generate column names as a recordset
declare #columns varchar(8000), #sql varchar(8000), #data_file varchar(100)
select
#columns=coalesce(#columns+',','')+column_name+' as '+column_name
from
information_schema.columns
where
table_name=#table_name
select #columns=''''''+replace(replace(#columns,' as ',''''' as '),',',',''''')
--Create a dummy file to have actual data
select #data_file=substring(#file_name,1,len(#file_name)-charindex('\',reverse(#file_name)))+'\data_file.xls'
--Generate column names in the passed EXCEL file
set #sql='exec master..xp_cmdshell ''bcp " select * from (select '+#columns+') as t" queryout "'+#file_name+'" -c'''
exec(#sql)
--Generate data in the dummy file
set #sql='exec master..xp_cmdshell ''bcp "select * from '+#db_name+'..'+#table_name+'" queryout "'+#data_file+'" -c'''
exec(#sql)
--Copy dummy file to passed EXCEL file
set #sql= 'exec master..xp_cmdshell ''type '+#data_file+' >> "'+#file_name+'"'''
exec(#sql)
--Delete dummy file
set #sql= 'exec master..xp_cmdshell ''del '+#data_file+''''
exec(#sql)
After creating the procedure, execute it by supplying database name, table name and file path
EXEC proc_generate_excel_with_columns 'your dbname', 'your table name','your file path'
As an alternative, you could use "External data" from Excel. It can use ODBC connection to fetch data from external source: Data/Get External Data/New Database Query. That way, even if the data in the database changes, you can easily refresh.

Export the content of a SQL Server table to a CSV without using xp_cmdshell

I need to export the content of a Table into a file CSV.
I tried to use the execution of a xp_cmdshell from a stored procedure but it doesn't work because this component is turned off as part of the security configuration for this server.
Do you know others way to write a file from a stored procedure?
Here are a couple of methods you can try :
1. Using BCP
syntax :
bcp "SELECT * FROM Database.dbo.input" queryout C:\output.csv -c -t',' -T -S .\SQLEXPRESS
microsoft document : BCP Utility
2. Second Method(for excel, but should work for csv too) :
Insert into OPENROWSET
('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=D:\testing.xls;',
'SELECT * FROM [SheetName$]') select * from SQLServerTable
For this, you need to enable adhoc distributed queries by following command :
EXEC sp_configure 'show advanced options', 1
RECONFIGURE
GO
EXEC sp_configure 'ad hoc distributed queries', 1
RECONFIGURE
GO
you might encounter a link server error. So you should refer : this stack overflow solution

How to save SQL query result to XML file on disk

I would like to export table from SQL Server 2012 to XML file. I have found nice answer and here how to make XML result from SQL Server database query, but still I am missing how to save this result physically into file.
SQL query is:
SELECT [Created], [Text]
FROM [db304].[dbo].[SearchHistory]
FOR XML PATH('Record'), ROOT('SearchHistory')
I use Microsoft SQL Server Management Studio to execute this result. I see the XML in a result window, but I cannot save it.
There is "Save Result As.." in context menu, but with 98900 rows I run out of my 8GB memory with this option.
Is there a way how to save this query directly to the XML file on disk?
You can also your SQL Server's extended stored procedures to export it to an xml file.
But you would need to configure the sql server before you can use it.
EXEC master.dbo.sp_configure 'show advanced options', 1
RECONFIGURE
EXEC master.dbo.sp_configure 'xp_cmdshell', 1
RECONFIGURE
Once xp_cmdshel is enabled in the SQL Server. You can use the following command to export the data to an xml file.
EXEC xp_cmdshell 'bcp "SELECT [Created], [Text] FROM [db304].[dbo].[SearchHistory] FOR XML PATH(''Record''), ROOT(''SearchHistory'')" queryout "C:\bcptest.xml" -T -c -t,'
You can always use the "Results to File" option in SSMS:
That should output the results of the query execution directly into a file on disk
To this job in SQL Server 2012 is a pain in ass. Finally I end up to update it to SQL Server 2014 as there is already support for SQL UTF-8 files in sqlcmd.
Create an SQL query and save it to the file.
run following:
sqlcmd -S -U sa -P sapassword -i inputquery_file_name -C65001 -o outputfile_name
This example works for me for result sets up to 2GB in size.
EXEC master.dbo.sp_configure 'show advanced options', 1
RECONFIGURE
EXEC master.dbo.sp_configure 'xp_cmdshell', 1
RECONFIGURE
DROP TABLE IF EXISTS ##AuditLogTempTable
SELECT A.MyXML
INTO ##AuditLogTempTable
FROM
(SELECT CONVERT(nvarchar(max),
(
SELECT
A.*
FROM
[dbo].[AuditLog] A
JOIN ImportProviderProcesses IPP ON IPP.ImportType = 'Z'
AND A.OperatorID = IPP.OperatorID
AND A.AuditTypeID in ( '400','424','425' )
WHERE
A.[PostTime] >= IPP.StartTime
AND A.[PostTime] <= dateadd(second, 90, IPP.StartTime)
FOR XML PATH('Record'), ROOT('AuditLog')
)
, 0
) AS MyXML
) A
EXEC xp_cmdshell 'bcp "SELECT MyXML FROM ##AuditLogTempTable" queryout "D:\bcptest1.xml" -T -c -t,'

Using query export SQL Server 2008 table to Excel Sheet

How to export sql server table data to excel sheet.
insert into OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0;Database=D:\Book1.xlsx;',
'SELECT * FROM [SheetName$]') select TOP 5 CustomerID from Customers
I used the above query but it shows following error
Msg 7308, Level 16, State 1, Line 1 OLE DB provider
'Microsoft.ACE.OLEDB.12.0' cannot be used for distributed queries
because the provider is configured to run in single-threaded apartment
mode.
I have found the solution
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ad Hoc Distributed Queries', 1;
GO
RECONFIGURE;
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'AllowInProcess', 1
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'DynamicParameters', 1
GO
insert into OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0;Database=D:\Book1.xls;',
'SELECT * FROM [Sheet1$]') select TOP 5 CustomerID from Customers
Its working fine

Resources