I am using a GUID for a batch identifier in SSIS. My final output goes to SQL Server.
I know how I can generate one using Select NewId() MyUniqueIdentifier in Sql Server - I can generate one using a query and an Execute SQL task.
I am however looking to do this within a SSIS package if possible without SQL Server available.
Can I generate a GUID within SSIS?
I had a similar problem. To fix it, I created an SSIS "Composant Script" in which I created a "guid" output. The script VS C# code was the following :
Row.guid = Guid.NewGuid();
Finally, I routed the output as a derived column into my database "OLE DB Destination" to generate a guid for every new entry.
Simply do it in an Execute SQL Task.
Open the task
Under General -> SQL Statement, enter your query Select NewID() MyID in the "SQLStatement" field
Under General -> Result Set, choose "Single row"
Under Parameter Mapping, Enter your User::myID in Variable Name, "Input" as direction, 0 as Parameter Name, and -1 as Parameter Size
Under Result Set, enter "MyID" for your Result Name and type the variable in Variable Name
-Click OK
Done. Note that "MyID" is a value you can choose. EDIT: "User::myID" corresponds to the SSIS variable that you create.
Related
I have to consolidate data from from 1000+ databases having the same structure/tables in one unique DB.
DBs may be added and removed potentially on a daily basis so I need to retrieve the list of DBs dynamically and run the dynamically generated SQL query to extract data on each of them.
I designed the Data Flow with a query from a variable that is working fine if executed with a static value:
With a SQL task I get the list of instances, I loop over the them and with a nested Foreach Loop/SQL task I retrieve the database names and create the dynamic SQL with the following statement (DB name is anonymized):
SELECT 'select ''' + name + ''' as DatabaseName, ID from ' + name + '.[dbo].[Orders] as querytext FROM sys.databases WHERE name LIKE ( 'XXX%_%' );
This part is also working fine:
How can I use the result of the SQL task "Execute SQL Task - Get query text" as query to be executed in the Source "OLE DB Source 1" (part of "Data Flow Task 3")?
I tried mapping an Object variable "User::SqlCommandFromSQLTask" in the result set of the SQL task, then set it up as ADO object source variable and with a Script task convert it to string and pass the value to the variable SqlStringFromSQLTask3 (used as source in "OLE DB Source 1") but I get the error Violation of PRIMARY KEY constraint, like if the data flow is always running with a static value I set up as default:
While, if I remove the value from the variable panel, I get the error "Command text was not set for the command object.", even changing the property DelayValidation of the Data Flow to false.
Any help is much appreciated.
When I have used SSIS to connect to multiple SQL Server boxes, I have stored those SQL Server connection strings in a table in a central database. Then I use a query of that table as the input to the foreach loop data flow task. If we ever have to change a sql server connection string, which does happen, we just update that table with the newest value.
I am facing issue in loading-data-from-multiple-db-to-another-server-using-ssis
I have referred the below link
Loading data from multiple db to another server using SSIS
SSIS Package FLow :
Unfortunately i am getting the error in "Execute SQL Task" as below:
[Execute SQL Task] Error: There is an invalid number of result
bindings returned for the ResultSetType: "ResultSetType_Rowset".
Appreciate if you could help me with the solutions.
Thanks
Based on the comments, to solve your issue
1. Evaluate your SQL Statement. For instance :
2. Once your variable query is evaluating, move to Execute SQL Task. It should look like this:
3. Next the resultSet should look like this (object_variable is of object type)
Why are we not using anything in parameter mapping ?
Answer: If we had a SQL query like Select col1, col2 from table1 where col3 = ?, Then we would be replacing ? with either a parameter or a variable.
In your case, delete everything inside parameter mapping.
Updated : Also, since you're query is Select * into tbl2 from tbl1, ResultSet property should be None instead of any other thing.
You have 2 Execute SQL Tasks
1, First Execute SQL Task, get list of tables and schema, it expects a Full Result set and map it to a object type variable.
2, Foreach loop container, ADO Enumerator, ADO source is the object type variable. Variable Mappings to 2 string type variables, 1 is for table name and 1 is for schema name.
Second Execute SQL Task, it has no Full Result set, no parameter mapping. Because table name/schema name changes are taken care of by Foreach loop container.
I have an execute sql task ( Task 1 ) which runs sql to returns a column called Note from Table A and stores it as a String SSIS variable type. In Table A, Note is defined as varchar(2000).
I then have an execute sql task ( Task 2) to run a stored procedure. The input parameter is Note varchar(max).
I run these 2 task in SSIS and get the following error:
DECLARE #..." failed with the following error: "The text, ntext, and image data types are invalid for local variables.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I have tried several solutions with no success. How can you get round this error and get SSIS to store the variable and feed it to the SP ?
On sql server 2012 SSIS and hitting an old 2008 database where the SP resides.
You are somehow mapping a text type column in your tsql code. Before you return from your tsql proc convert the text column to a varchar column with select cast (textval as varchar(max)) and make sure your output variables are defined as varchars.
You need to make sure the Task 1 should only return one row.
example capture
Otherwise you need to use an Object variable. And use a For each loop container to loop thru that object.
My best attempt at visualizing this:
ForEach row in dbo.runThese
**** Start Loop
(grab select statements from sql table)
dbo.runThese
Output:
ID db2_script sql_script
---------------------------------------------------------------------------
1 'select count(\*) from db2_cstmr' 'select count(*) from sql_cstmr'
(Run each script on an individual connection to the DB2 and SQL Server database)
(create a combined string with each result)
149, 149
(Insert the combined results into a SQL Server table)
INSERT INTO dbo.storeResults
VALUES (149,149)
**** End Loop
I see three different ways to do this, but I'll provide the one I see is the most elegant. I will split the tasks based on location within the package:
1. Variables
New variable "Statements" of Object data type, which will hold a list of db2 and sql server statements
db2_script: String
sql_script: String
id: int32
2. Control Flow
Execute SQL Task: Get all the records (sql statements) into the object variable using something like: SELECT id, db2_script, sql_script FROM dbo.StatementsToExecute. You need to set ResultSet property of the component to "Full result set" and configure the object variable in the Result Set pane
For Each Loop: using the object variable as enumerator (Foreach From Variable Enumerator) in the Collection pane, and assigning into db2_script, sql_script and id variables in the Variable Mapping pane
Data Flow component (see next)
3. Data Flow
OLEDB Source for DB2 database: specify variable db2_script for source statement (Data access mode: SQL command from variable)
OLEDB Source for SQL Server database: specify variable sql_script for source statement (Data access mode: SQL command from variable)
Edit both sources with Advanced Editor, got to "Input and Output Properties" tab, click on "OLE DB Source Output", set IsSorted=True, click on "OLE DB Source Output"->"Output Columns"->db2_count/sql_count
MERGE: merge both sources into one single pipeline and two different output columns
OLEDB Target: map db2_count and sql_count to the target columns
Note: you would need to provide aliases for the Counts in each Select statement (e.g. SELECT COUNT(*) AS db2_count FROM ...) because they will give names to the columns in the Data Flow's pipeline. Another way is to edit in advanced mode both sources and give ad-hoc names
I want to create a copy of a table, say TestTable, with a new name, say TestTableNew, in the same database with the use of an SSIS package. I've created a "Transfer SQL Server Objects Task" for this with the source database specified as both the SourceDatabase and the DestinationDatabase. When I run this task, the original table TestTable is overwritten with a new -empty- TestTable.
This might well be something really obvious that I've overlooked, but can I somehow specify another name for the destination table somewhere in this transfer task? Or should I solve this in another way?
You can't use the "Transfer SQL Server Objects Task" to copy a table to the same database because there isn't an option to specify the new table name. You would be copying table "TestTable" to table "TestTable", which will fail because they both have the same name.
You can set the "DropObjectsFirst" property to true, but that will make you lose your original table and its data, which I think you did on your test, otherwise you would have received a failure message.
The best option here is to use an "Execute SQL Task" to create the structure of your TestTableNew based on your TestTable and then do a simple OleDBSource -> OleDBDestination transformation to load all the data from one table to another.
My knowledge of SSIS is very limited but I assume you can run sql commands passing in
parameters and therefore generating something like the following dynamically
select *
insert into TestTableNew
from TestTable