I have an SSIS package that contains a SQL task to create aN excel table once its been dropped, this works fine, my question is is it possible to either have a sql task to create a worksheet and a range. or a SQL task to insert data into a particular range within a previously created worksheet?
Related
I want to populate an excel spreadsheet using SSIS. each column that needs to be populated is generated by a different query (4 queries). I'm trying to automate that task instead of manually running the query every day, it should do it as a scheduled task
I have created an SSIS package for my database running on SQL Server 2014.
This SSIS package extracts data from an Excel workbook (residing in a shared folder on my Windows Server) and feeds a table in my database. I have tested the package (through a SQL Job) and it works fine.
The logic behind the creation of this package is to pull data on a daily basis from that Excel workbook (the latter being updated on a daily basis). It does have a date column; so, data updated are tagged with the relevant date.
Since this SQL job will be executed on a daily basis, the table will be filled with duplicates (that is, data from Day 1 will be repeated on Day 2 and so on).
How can I deal this issue? I was thinking on adding a T-SQL flow to my SSIS package to delete the contents of the Table before it pulls the data from the Excel Workbook. I guess this would work but I am on the look out for a more elegant solution.
Any help would be appreciated.
We have around 5000 tables in Oracle and the same 5000 tables exist in SQL server. Each table's columns vary frequently but at any point in time source and destination columns will always be the same. Creating 5000 Data flow tasks is a big pain. Further there's a need to map every time a table definition changes, such as when a column is added or removed.
Tried the SSMA (SQL Server Migration Assistance for Oracle ) but it is very slow for transferring huge amount of data then moved to SSIS
I have followed the below approach in SSIS:
I have created a staging table where it will have a table name, source
query (oracle), Target Query (SQL server) used that table in Execute
SQL task and stored the result set as the full result set
created for each loop container off that execute SQL task result set
and with the object and 3 variables table name, source query and
destination query
In the data flow task source I have chosen OLE DB source for oracle
connection and choose data access mode as an SQL command from a
variable (passed source query from loop mapping variable)
In the data flow task destination I have chosen OLE DB source for SQL
connection and choose data access mode as an SQL command from a
variable (passed Target query from loop mapping variable)
And looping it for all the 5000 tables..it is not working can you please guide us how I need to create it for 5000 tables dynamically from oracle to SQL server using SSIS. any sample code/help would be greatly appreciated. Thanks in advance
Using SSIS, when thinking about dynamic source or destination you have to take into consideration that the only case you can do that is when metadata is well defined at run-time. In your case:
Each table columns vary frequently but at any point of time source destination columns will always same.
You have to think about build packages programatically rather than looping over tables.
Yes, you can use loops in case you can classify tables into groups based on their metadata (columns names, data types ...). Then you can create a package for each group.
If you are familiar with C# you can dynamically import tables without the need of SSIS. You can refer to the following project to learn more about reading from oracle and import to SQL using C#:
Github - SchemaMapper
I will provide some links that you can refer to for more information about creating packages programatically and dynamic columns mapping:
How to manage SSIS script component output columns and its properties programmatically
How to Map Input and Output Columns dynamically in SSIS?
Implementing Foreach Looping Logic in SSIS
I am creating a SSIS package that imporr data from a SQL Server Source to an Excel Destination.
How can one truncate spreadsheet before run?
I tried the following way (using Execute SQL Task with no success.
Jet provider does not support neither truncate or delete command. You have 3 workarounds:
Have an empty excel template that you clone before the running the dataflow, or
Use execute sql task to create a new workbook/tab before running the dataflow
Drop the worksheet using Drop Table TableCall_Log and create a new one. You can referer to this Link for more details.
Useful Links
Deleting Records in an Excel Sheet using SSIS
Import Header-Line tables into Dynamic Excel destinations II
SSIS: Dynamically Generate Excel Table/Sheet
Truncation is not supported. You can recreate the whole excel file using two tasks:
The first task will be File task which deletes destination xls file.
The second task will be Execute SQL Task, which creates "table" (excel sheet). Use EXCEL connection type with excel connection manager and CREATE TABLE statement.
If you do not know the exact form of CREATE TABLE statement, then try to first prepare excel destination in a data flow task and by creating new excel sheet (by pushing New button on Connection Manager tab in Excel Destination editor) SSIS designer will show you CREATE TABLE statement which you need.
Connect the first task to the second task using Completion constraint if you are not sure, that the excel file exists every time you run the package.
You may also need to set DelayedValidation property to True on tasks following these first two tasks.
I've created an SSIS package with the purpose of deleting and then recreating an Excel sheet, and then inserting data from a SQL database into that Excel sheet.
What I currenty have is:
1.) An Execute SQL task to delete 'Sheet1' from a local Excel file.
2.) An Execute SQL task to recreate 'Sheet1' in the same Excel file.
3.) A Data Flow task that uses an OLE DB Source to a 2005 SQL Server database.
4.) An Excel Destinaton (using an Excel Connection Manager) which points at / maps the SQL Server columns to the local Excel spreadsheet.
So essentially I'm deleting the Excel Sheet to remove old data, recreating it, and then trying to import the SQL data into it.
This worked well when I ran it the first time. When I try to rerun it however, all the tasks run successfully, the correct number of rows are shown to move to the Excel destination on the data flow tab, but the Excel spreadsheets are blank.
Can anyone suggest why this might be happening?
Thanks, Gavin
Is this the only sheet in your Excel file? If so, it might be easier to use the File System Task component to delete the entire file before creating your sheet (and therefore file) in your Execute SQL statement.