Objective: automate creating a table along with all its columns and data types given a SSIS source
My guess is:
1) Pointing Sources to a Destination to a SQL command
2) Using Select * into ... Problem is I don't know what the from equivalent of a source is
Alternative) Store results in Recordset and pass on to Execute SQL task. Problem then is how to access that result from execute sql task
I think you should use a Recordset Destination to store data into an System.Object Variable, Then use a Script Task (starts after that Data Flow Task is executed) in which you will select the System.Object Variable as ReadOnly Variable. and you will write your own code to insert the Recordset to SQL using System.Data.SqlClient.SQLCommand Object
You can refer to one of these links
Issues with SSIS Script Task and SQL Bulk Insert - C#
Insert DataTable into SQL Table in C#
If you need just the structure of table use this trick
select top 0 * into NewTable from YourTable
Related
I have SSIS package with data flow task and execute SQL task components in For each loop container. Package flow is, Date flow task(flat file--> Conditional split to insert data into SQL server tables)-->Execute SQL task(perform some SQL operations on inserted data and insert the calculated values in one final analysis table) . File name is like name1_name2_yyyymmdd_1234.txt. I want to fetch the date from file name and insert that date value in table in SQL Server as FileDate. I am trying to do it using derived column but unable to figure our where will I save it so that it will be available in Insert statement in Execute SQL Task which is after Data flow task.
This should be done outside the dataflow but within the ForEach loop.
Pass in two parameters (package scope) to a script task. One with the #filename (read only) from the forloop and to store the #fileDate (read/write).
Split will create a 0-based array in which you only care about the third piece.
Dts.Variables["fileDate"].Value = DateTime.ParseExact(Dts.Variables["fileName"].Value.Split('_')[2]
,"yyyyMMdd", System.Globalization.CultureInfo.InvariantCulture);
Now you can use #fileDate anywhere you would like.
I need to extract data from a DB2 database to a SQL Server. I need to create my query based on a Excel file I have 176 records, which I need to create repeating queries & put in SQL server
So for example;
I have an Excel with a Number, From date, To date, and a Country
So the query should use these information from the records
SELECT *
FROM dbo.Test
WHERE Number = excel.Number1 AND Date BETWEEN excel.fromDate1 AND excel.toDate1 AND Country = excel.country1
And then another query with
SELECT *
FROM dbo.Test
WHERE Number = excel.Number2 AND Date BETWEEN excel.fromDate2 AND excel.toDate2 AND Country = excel.country2
Etc...
How should I do something like this in SSIS?
If needed I can put the DB2 and Excel data in MS SQL
You can proceed with the following approach:
Extract data rows from Excel and put it into SSIS Object Variable
Proceed with a Foreach loop to get each row from the Object Variable, parsing Object Variable to separate variables
Inject variable values into SQL Select command with Expressions
Perform Data Flow task based on SQL command, transform and put it into the target
Overall, your task seems to be feasible, but requires some knowledge on parsing Object Variable in Foreach Loop, and writing Variable Expressions.
I have to consolidate data from from 1000+ databases having the same structure/tables in one unique DB.
DBs may be added and removed potentially on a daily basis so I need to retrieve the list of DBs dynamically and run the dynamically generated SQL query to extract data on each of them.
I designed the Data Flow with a query from a variable that is working fine if executed with a static value:
With a SQL task I get the list of instances, I loop over the them and with a nested Foreach Loop/SQL task I retrieve the database names and create the dynamic SQL with the following statement (DB name is anonymized):
SELECT 'select ''' + name + ''' as DatabaseName, ID from ' + name + '.[dbo].[Orders] as querytext FROM sys.databases WHERE name LIKE ( 'XXX%_%' );
This part is also working fine:
How can I use the result of the SQL task "Execute SQL Task - Get query text" as query to be executed in the Source "OLE DB Source 1" (part of "Data Flow Task 3")?
I tried mapping an Object variable "User::SqlCommandFromSQLTask" in the result set of the SQL task, then set it up as ADO object source variable and with a Script task convert it to string and pass the value to the variable SqlStringFromSQLTask3 (used as source in "OLE DB Source 1") but I get the error Violation of PRIMARY KEY constraint, like if the data flow is always running with a static value I set up as default:
While, if I remove the value from the variable panel, I get the error "Command text was not set for the command object.", even changing the property DelayValidation of the Data Flow to false.
Any help is much appreciated.
When I have used SSIS to connect to multiple SQL Server boxes, I have stored those SQL Server connection strings in a table in a central database. Then I use a query of that table as the input to the foreach loop data flow task. If we ever have to change a sql server connection string, which does happen, we just update that table with the newest value.
I have an SSIS package with two Execute SQL Tasks, the first one has a simple select statement to select a single column from a table, the result set is an ID column which I have to use as input to the second Execute SQL Task to get records from another table which matching IDs. Basically I have to implement following SQL statement in SSIS:
SELECT * FROM TableB WHERE ID IN
(
SELECT ID FROM TableA
)
How can I configure the second Execute SQL task to use the result set of first Execute SQL task as input to execute an SQL statement like mentioned above?
One solution would be to populate an object variable with the result of the first Execute SQL Task.
Then have a Script Task which loops through the object variable and dynamically builds the SQL String for the second Execute SQL Task, and puts that into a string variable.
Then finally, the second Execute SQL Task uses the string variable for its SQL query.
I want to create a copy of a table, say TestTable, with a new name, say TestTableNew, in the same database with the use of an SSIS package. I've created a "Transfer SQL Server Objects Task" for this with the source database specified as both the SourceDatabase and the DestinationDatabase. When I run this task, the original table TestTable is overwritten with a new -empty- TestTable.
This might well be something really obvious that I've overlooked, but can I somehow specify another name for the destination table somewhere in this transfer task? Or should I solve this in another way?
You can't use the "Transfer SQL Server Objects Task" to copy a table to the same database because there isn't an option to specify the new table name. You would be copying table "TestTable" to table "TestTable", which will fail because they both have the same name.
You can set the "DropObjectsFirst" property to true, but that will make you lose your original table and its data, which I think you did on your test, otherwise you would have received a failure message.
The best option here is to use an "Execute SQL Task" to create the structure of your TestTableNew based on your TestTable and then do a simple OleDBSource -> OleDBDestination transformation to load all the data from one table to another.
My knowledge of SSIS is very limited but I assume you can run sql commands passing in
parameters and therefore generating something like the following dynamically
select *
insert into TestTableNew
from TestTable