Good Afternoon,
I have created an SSIS Project with a single package in it. The SSIS Project and Package work as I expect when I manually executed from the server. I thought that if I set an environment variable on the server and mapped it to the project. That the project would then use those variables every time the package was executed unless otherwise told. The following code is my VB code
'VB.Net code
Imports System.Data.SqlClient
Imports Microsoft.SqlServer.Management.IntegrationServices
Imports System.Collections.ObjectModel
Public Class Form1
Private Sub StartPackageButton_Click(sender As System.Object, e As System.EventArgs) Handles StartPackageButton.Click
Try
' Connection to the database server where the packages are located
Dim ssisConnection As New SqlConnection("Data Source=" + txtServerName.Text + ";Integrated Security=SSPI;")
' SSIS server object with connection
Dim ssisServer As New IntegrationServices(ssisConnection)
' The reference to the package which you want to execute
Dim ssisPackage As PackageInfo = ssisServer.Catalogs("SSISDB").Folders("SSIS_PROJECTS").Projects("AgressoExport").Packages("File56Export.dtsx")
' Add a parameter collection for 'system' parameters (ObjectType = 50), package parameters (ObjectType = 30) and project parameters (ObjectType = 20)
Dim executionParameters As New Collection(Of PackageInfo.ExecutionValueParameterSet)
' Add execution parameter to override the default asynchronized execution. If you leave this out the package is executed asynchronized
Dim executionParameter1 As New PackageInfo.ExecutionValueParameterSet
executionParameter1.ObjectType = 50
executionParameter1.ParameterName = "SYNCHRONIZED"
executionParameter1.ParameterValue = 1
executionParameters.Add(executionParameter1)
' Add execution parameter (value) to override the default logging level (0=None, 1=Basic, 2=Performance, 3=Verbose)
Dim executionParameter2 As New PackageInfo.ExecutionValueParameterSet
executionParameter2.ObjectType = 50
executionParameter2.ParameterName = "LOGGING_LEVEL"
executionParameter2.ParameterValue = 3
executionParameters.Add(executionParameter2)
' Add execution parameter (value) to override the default logging level (0=None, 1=Basic, 2=Performance, 3=Verbose)
Dim executionParameter3 As New PackageInfo.ExecutionValueParameterSet
If (Trim(txtPreviousID.Text) <> "") Then
executionParameter3.ObjectType = 20
executionParameter3.ParameterName = "PreviousBatchID"
executionParameter3.ParameterValue = txtPreviousID.Text
executionParameters.Add(executionParameter3)
End If
' Get the identifier of the execution to get the log
Dim executionIdentifier As Long = ssisPackage.Execute(False, Nothing, executionParameters)
' Loop through the log and add the messages to the listbox
For Each message As OperationMessage In ssisServer.Catalogs("SSISDB").Executions(executionIdentifier).Messages
SSISMessagesListBox.Items.Add(message.MessageType.ToString() + ": " + message.Message)
Next
Catch ex As Exception
If ex.InnerException IsNot Nothing Then
SSISMessagesListBox.Items.Add(ex.Message.ToString() + " : " + ex.InnerException.Message.ToString())
Else
SSISMessagesListBox.Items.Add(ex.Message.ToString())
End If
End Try
End Sub
End Class
I'm starting to think that I did not understand environments correctly when it comes to SSIS. My environment was setup as TEST and PROD on their respective servers with the same variable names mapped to the same parameters but with different values. I am starting to believe I should have had the same Environment name on both the TEST and PROD server, which I would then refer too using my VB code. I have not been able to find out how to refer to the Environment using VB yet either though.
I would appreciate any help on the matter.
Cheers,
Johnathan
For the purpose ssis have given facility of configuration and you have to just create separate cofig file or db entry for your different environment .
https://msdn.microsoft.com/en-us/library/ms141682.aspx
You need to set the reference property of the package to be the ID of the environment.
To get the environment ID try:
DECLARE #environment_id AS BIGINT
SELECT #environment_id = reference_id FROM SSISDB.internal.environment_references where environment_name = 'your environment name'
To be able to use Environments inside a Visual Basic .NET program. You need to declare the Environment Reference and pass the reference to the Execute Method of the SSIS Package.
The following block of code is how it is done:
Dim re As EnvironmentReference
re = ssisServer.Catalogs("SSISDB").Folders("SSIS_PROJECTS").Projects("AgressoExport").References("AgressoExport", ".")
Dim executionIdentifier As Long = ssisPackage.Execute(False, re, executionParameters)
SSISDB is the catalog name, SSIS_PROJECTS is the folder name under the catalog that I am using. AgressoExport is my Project Name. ("AgressoExport", ".") refers to the Environment named AgressoExport under my project in the root of the Environment Folder.
Cheers
Related
I am using SSIS Script Task but whenever I am running it the the SSIS package fails and it gives the following error: Exception has been thrown by the target of an invocation.
Is it possible that it is giving this issue because I was using this script in Visual Studio 2008 and I am trying to implement the package in Visual Studio 2010.
here is my code:
enter code here ' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic 2008.
' The ScriptMain is the entry point class of the script.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports System.IO
<Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute> _
<System.CLSCompliantAttribute(False)> _
Partial Public Class ScriptMain
Inherits
Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
Enum ScriptResults
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
End Enum
' The execution engine calls this method when the task executes.
' To access the object model, use the Dts property. Connections, variables, events,
' and logging features are available as members of the Dts property as shown in the following examples.
'
' To reference a variable, call Dts.Variables("MyCaseSensitiveVariableName").Value
' To post a log entry, call Dts.Log("This is my log text", 999, Nothing)
' To fire an event, call Dts.Events.FireInformation(99, "test", "hit the help message", "", 0, True)
'
' To use the connections collection use something like the following:
' ConnectionManager cm = Dts.Connections.Add("OLEDB")
' cm.ConnectionString = "Data Source=localhost;Initial Catalog=AdventureWorks;Provider=SQLNCLI10;Integrated Security=SSPI;Auto Translate=False;"
'
' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'
' To open Help, press F1.
Public Sub Main()
Dim file_stream As New FileStream(CType(ReadVariable("filepath"), String) + "AIR_FEE1.TRN", FileMode.Append)
Using w As New StreamWriter(file_stream, Text.Encoding.UTF8)
w.WriteLine("T|" + CType(ReadVariable("Count"), String))
End Using
Dim FName As String
Dim LFName As String
FName = CType(ReadVariable("filename"), String)
LFName = CType(ReadVariable("logfile"), String)
WriteVariable("StaticLogFileName", LFName)
WriteVariable("StaticFileName", FName)
Dim file_stream1 As New FileStream("StaticFileName", FileMode.Create)
file_stream.Close()
Dts.TaskResult = ScriptResults.Success
End Sub
Private Function ReadVariable(ByVal varName As String) As Object
Dim result As Object
Try
Dim vars As Variables
Dts.VariableDispenser.LockForRead(varName)
Dts.VariableDispenser.GetVariables(vars)
Try
result = vars(varName).Value
Catch ex As Exception
Throw ex
Finally
vars.Unlock()
End Try
Catch ex As Exception
Throw ex
End Try
Return result
End Function
Private Sub WriteVariable(ByVal varName As String, ByVal varValue As Object)
Try
Dim vars As Variables
Dts.VariableDispenser.LockForWrite(varName)
Dts.VariableDispenser.GetVariables(vars)
Try
vars(varName).Value = varValue
Catch ex As Exception
Throw ex
Finally
vars.Unlock()
End Try
Catch ex As Exception
Throw ex
End Try
End Sub
End Class
First of all, "Exception has been thrown by the target of an invocation" is a generic message that is thrown when an error occurred during script task execution, try to debug your code to find a more precise error message.
I think you can write the same script without defining functions to manipulate your variables:
Public Sub Main()
Dim file_stream As New FileStream(Dts.Variables("filepath").Value + "AIR_FEE1.TRN", FileMode.Append)
Using w As New StreamWriter(file_stream, Text.Encoding.UTF8)
w.WriteLine("T|" + Dts.Variables("Count").Value)
End Using
Dim FName As String
Dim LFName As String
FName = Dts.Variables("filename").Value
LFName = Dts.Variables("logfile").Value
Dts.Variables("StaticLogFileName").Value = LFName
Dts.Variables("StaticFileName").Value = FName
Dim file_stream1 As New FileStream("StaticFileName", FileMode.Create)
file_stream.Close()
Dts.TaskResult = ScriptResults.Success
End Sub
Make sure that you have selected your ReadOnly Variables and ReadWrite Variables properly from the Script task properties form.
Helpful links
Using Variables in the Script Task
3 Ways -SSIS Read Write Variables – Script Task C# / VB.net
Unable to fetch "ReadWrite" Variable Value in Script Component of SSIS
I am trying to put columns names from a table inside a Microsoft Access database inside a list variable. I have done this so far but the line where I am trying to add it to the topic variable does not work and is coming up with the error
predefined type ‘valuetuple(of,,,)’ is not defined or imported.
The code is:
Dim topic = topic()
Dim filtervalues = {Nothing, Nothing, "Results", Nothing}
Dim counter As Integer = 0
Using con = _
New OleDb.OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=Database.mdb")
Dim columns = con.GetSchema("columns", filtervalues)
For Each row As DataRow In columns.Rows
topic(counter) = ("{0,-20}{1}", row("column_name"), row("data_type"))
counter = +1
Next
End Using
According to the documentation on Value Tupels you must get the NuGet package System.ValueTuple, if you are working with Framework version prior to 4.7:
Important
Tuple support requires the ValueTuple type. If the .NET Framework 4.7 is not installed, you must add the NuGet package System.ValueTuple, which is available on the NuGet Gallery. Without this package, you may get a compilation error similar to, "Predefined type 'ValueTuple(Of,,,)' is not defined or imported."
In Visual Studio 2017 right click on your solution and select "Manage NuGet Packages for Solution...". In the search-box enter "valuetuple". Select "System.ValueTuple" and on the right click the check boxes of the projects where you want to install the package and click Install.
See: NuGet Package Manager UI
Also, you must declare the list variable as
Dim topic = New List(Of (String, String, String))
and add new elements with
topic.Add(("{0,-20}{1}", row("column_name"), row("data_type")))
The counter is not needed anymore.
Alternatively, you could use a list of strings and format the string with string interpolation
Dim topic = New List(Of String)
Dim filtervalues = {Nothing, Nothing, "Results", Nothing}
Using con =
New OleDb.OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=Database.mdb")
Dim columns = con.GetSchema("columns", filtervalues)
For Each row As DataRow In columns.Rows
topic.Add($"{row("column_name"),-20}{row("data_type")}")
Next
End Using
Judging from the "{0,-20}{1}", you forgot the String.Format in
topic(counter) = String.Format("{0,-20}{1}", row("column_name"), row("data_type"))
and did not intend to use a tuple.
I have a client/server desktop application that I am having some database connection issues with on some of my clients pc's. When I wrote the app, I didn't know any better so I created and opened 1 database connection on application startup, and used that same connection all throughout the app. I know realize this is a bad idea since shaky network connections and it seems antivirus programs are causing these connection to be dropped at times, leading to some errors. I have hundreds of places in code where I need to go back and create/open/close the connection at the time they are being used.
The question is, is there any way to create a public function in which I can do just that, and then do a global find and replace to replace the connection name with the new function name?
something like:
Dim qry As NpgsqlCommand
sqlUpdateItem = "update table set field = value where id = 1"
qry = New NpgsqlCommand(sqlUpdateItem, con)
qry.ExecuteNonQuery()
to
Dim qry As NpgsqlCommand
sqlUpdateItem = "update table set field = value where id = 1"
qry = New NpgsqlCommand(sqlUpdateItem, newCon())
qry.ExecuteNonQuery()
public function newCon()
Dim con As New NpgsqlConnection(connectionString)
con.Open()
Return tcon
End Function
I tried this but no luck. I'm just looking for any possible solutions that don't involve me updating several lines of code in hundreds of places throughout my app. The nice thing is I would only need to do this for all commands, since I can pass a brand new connection into a data adapter and it will handle the opening/closing.
Here's an example of how I'd recommend you attempt it.
Enable option strict in your project. It's better to have your errors at compile time than at runtime.
Use a using statement to safely dispose of the database classes even if you get an exception.
Private _connectionString As String = "blah"
Public Function GetDbConnection() As NpgsqlConnection
Dim con As New NpgsqlConnection(_connectionString)
con.Open()
Return con
End Function
Public Sub DoMyQuery()
Using conn = GetDbConnection()
Using qry = New NpgsqlCommand("update table set field = value where id = 1", conn)
qry.ExecuteNonQuery()
End Using
End Using
End Sub
I have create a SSIS package which works perfectly from my BIDS. But when executing from the server it fails.
The package does the following:
Insert data (read from the SQL database) into an Excel file
Via a VB-script it opens the Excel file and removes line number 2
I can see that the package can insert the data correct in the Excel file. It fails when executing the VB-Script in a step called "Remove 2nd line":
Remove 2ndline: Error: Exception has been thrown by the target of an invocation.
The script is:
#Region "Help: Introduction to the script task"
'The Script Task allows you to perform virtually any operation that can be accomplished in
'a .Net application within the context of an Integration Services control flow.
'Expand the other regions which have "Help" prefixes for examples of specific ways to use
'Integration Services features within this script task.
#End Region
Option Strict Off
#Region "Imports"
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports Microsoft.Office.Interop.Excel
#End Region
'ScriptMain is the entry point class of the script. Do not change the name, attributes,
'or parent of this class.
<Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute()> _
<System.CLSCompliantAttribute(False)> _
Partial Public Class ScriptMain
Inherits Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
Private _app As Microsoft.Office.Interop.Excel.Application
Private _books As Microsoft.Office.Interop.Excel.Workbooks
Private _book As Microsoft.Office.Interop.Excel.Workbook
Protected _sheets As Microsoft.Office.Interop.Excel.Sheets
Protected _sheet1 As Microsoft.Office.Interop.Excel.Worksheet
#Region "Help: Using Integration Services variables and parameters in a script"
'To use a variable in this script, first ensure that the variable has been added to
'either the list contained in the ReadOnlyVariables property or the list contained in
'the ReadWriteVariables property of this script task, according to whether or not your
'code needs to write to the variable. To add the variable, save this script, close this instance of
'Visual Studio, and update the ReadOnlyVariables and
'ReadWriteVariables properties in the Script Transformation Editor window.
'To use a parameter in this script, follow the same steps. Parameters are always read-only.
'Example of reading from a variable:
' startTime = Dts.Variables("System::StartTime").Value
'Example of writing to a variable:
' Dts.Variables("User::myStringVariable").Value = "new value"
'Example of reading from a package parameter:
' batchId = Dts.Variables("$Package::batchId").Value
'Example of reading from a project parameter:
' batchId = Dts.Variables("$Project::batchId").Value
'Example of reading from a sensitive project parameter:
' batchId = Dts.Variables("$Project::batchId").GetSensitiveValue()
#End Region
#Region "Help: Firing Integration Services events from a script"
'This script task can fire events for logging purposes.
'Example of firing an error event:
' Dts.Events.FireError(18, "Process Values", "Bad value", "", 0)
'Example of firing an information event:
' Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, fireAgain)
'Example of firing a warning event:
' Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0)
#End Region
#Region "Help: Using Integration Services connection managers in a script"
'Some types of connection managers can be used in this script task. See the topic
'"Working with Connection Managers Programatically" for details.
'Example of using an ADO.Net connection manager:
' Dim rawConnection As Object = Dts.Connections("Sales DB").AcquireConnection(Dts.Transaction)
' Dim myADONETConnection As SqlConnection = CType(rawConnection, SqlConnection)
' <Use the connection in some code here, then release the connection>
' Dts.Connections("Sales DB").ReleaseConnection(rawConnection)
'Example of using a File connection manager
' Dim rawConnection As Object = Dts.Connections("Prices.zip").AcquireConnection(Dts.Transaction)
' Dim filePath As String = CType(rawConnection, String)
' <Use the connection in some code here, then release the connection>
' Dts.Connections("Prices.zip").ReleaseConnection(rawConnection)
#End Region
'This method is called when this script task executes in the control flow.
'Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'To open Help, press F1.
Public Sub Main()
Try
Dim FileName As String = ""
If Dts.Variables.Contains("DestinationPath") = True Then
FileName = CType(Dts.Variables("DestinationPath").Value, String)
End If
OpenExcelWorkbook(FileName)
_sheet1 = CType(_sheets(1), Microsoft.Office.Interop.Excel.Worksheet)
_sheet1.Activate()
Dim range As Microsoft.Office.Interop.Excel.Range = _sheet1.Rows(2)
range.Delete(Microsoft.Office.Interop.Excel.XlDeleteShiftDirection.xlShiftUp)
DoRelease(range)
DoRelease(_sheet1)
CloseExcelWorkbook()
DoRelease(_book)
_app.Quit()
DoRelease(_app)
Dts.TaskResult = ScriptResults.Success
Catch ex As Exception
MsgBox(ex.ToString())
End Try
End Sub
Protected Sub OpenExcelWorkbook(ByVal fileName As String)
'try
'{
_app = New Microsoft.Office.Interop.Excel.Application()
If _book Is Nothing Then
_books = _app.Workbooks
_book = _books.Open(fileName, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing)
_sheets = _book.Worksheets
End If
'}
'catch(Exception ex)
'{
' Console.WriteLine(ex.ToString());
'}
End Sub
Protected Sub CloseExcelWorkbook()
_book.Save()
_book.Close(False, Type.Missing, Type.Missing)
End Sub
Protected Sub DoRelease(ByVal o As Object)
Try
If Not o Is Nothing Then
System.Runtime.InteropServices.Marshal.ReleaseComObject(o)
End If
Finally
o = Nothing
End Try
End Sub
#Region "ScriptResults declaration"
'This enum provides a convenient shorthand within the scope of this class for setting the
'result of the script.
'This code was generated automatically.
Enum ScriptResults
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
End Enum
#End Region
End Class
The Excel file is reached by the variable "DestinationPath". This path works when outside the script (ie it can fill data in the file in a previous step). It also works when in BIDS in the script. I have also made sure to make it available to the script as a ReadOnlyVariable. So I am pretty confident that it is not a problem with the variable.
My server is a Windows 2012 R2 (64-bit)
The package is being executed on an SQL Server 2012 - 11.0.
I have installed Excel 2013 on the server.
I have created the following folder as this could also cause the problem (I have read): C:\Windows\SysWOW64\config\systemprofile\Desktop
The caller of my script is my administrator userid - and this userid is capable of using Excel on the server.
Is there any reason we couldn't use the regular Excel Connection manager to insert data, and instead of deleting line 2, just never send it? Use a Conditional Split to siphon it off. Here is a simple example. Here is an advanced example of using SSIS default Excel Connection manager. I'd also like to grant a pearl of wisdom against using Interop.Excel inside of script tasks. It can certainly lead to these types of "But, it works on my Machine" scenarios. Read KB257757 to see why Microsoft discourages this type of thing. However, some things only Interop.Excel can do, such as saving to PDF or running macros. You're not doing either of those things, so why not give the regular Excel connector a try?
I am trying to execute a SSIS package located in a database programatically.
I am using this API:
Imports Microsoft.SqlServer.Dts.Runtime
I have an image describing the path (in database) to package but I cannot figure out how to set the packagePath property properly in the LoadFromSqlServer method.
Here is the image describing my package path in database:
You will need to add a reference to Microsoft.SqlServer.Management.IntegrationServices. For me, it does not show up in the SQL Server folders and I could only find it in the GAC.
C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.IntegrationServices\11.0.0.0__89845dcd8080cc91\Microsoft.SqlServer.Management.IntegrationServices.dll
There's also a dependency from that assembly to
C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.Sdk.Sfc\11.0.0.0__89845dcd8080cc91\Microsoft.SqlServer.Management.Sdk.Sfc.dll
Sub Main()
'
' Do not fault me for my poor VB skills nor my lack of error handling
' This is bare bones code adapted from
' http://blogs.msdn.com/b/mattm/archive/2011/11/17/ssis-and-powershell-in-sql-server-2012.aspx
Dim folderName As String
Dim projectName As String
Dim serverName As String
Dim packageName As String
Dim connectionString As String
Dim use32BitRuntime As Boolean
Dim executionId As Integer
Dim integrationServices As Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices
Dim catalog As Microsoft.SqlServer.Management.IntegrationServices.Catalog
Dim catalogFolder As Microsoft.SqlServer.Management.IntegrationServices.CatalogFolder
Dim package As Microsoft.SqlServer.Management.IntegrationServices.PackageInfo
' Dimensions in your example
folderName = "SSISHackAndSlash"
' dimCalendar in your example
projectName = "SSISHackAndSlash2012"
serverName = "localhost\dev2012"
' dimCalendar in your example (no file extension)
packageName = "TokenTest.dtsx"
connectionString = String.Format("Data Source={0};Initial Catalog=msdb;Integrated Security=SSPI;", serverName)
integrationServices = New Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices(New System.Data.SqlClient.SqlConnection(connectionString))
' There is only one option for an SSIS catalog name as of this posting
catalog = integrationServices.Catalogs("SSISDB")
' Find the catalog folder. Dimensions in your example
catalogFolder = catalog.Folders(folderName)
' Find the package in the project folder
package = catalogFolder.Projects(projectName).Packages(packageName)
' Run the package. The second parameter is for environment variables
executionId = package.Execute(use32BitRuntime, Nothing)
End Sub
In addition to billinkc answer.
Here is the C# version of the code:
string folderName = "name";
string projectName = "name";
string serverName = "localhost";
string packageName = "name";
string connectionString = string.Format("Data Source={0};Initial Catalog=msdb;Integrated Security=SSPI;", serverName);
var integrationServices = new IntegrationServices(newSystem.Data.SqlClient.SqlConnection(connectionString));
var catalog = integrationServices.Catalogs["SSISDB"];
var catalogFolder = catalog.Folders[folderName];
var package = catalogFolder.Projects[projectName].Packages[packageName];
long execId = package.Execute(false, null);
In my case I had to add 4 dlls:
Microsoft.SqlServer.ConnectionInfo.dll
Microsoft.SqlServer.Management.IntegrationServices.dll
Microsoft.SqlServer.Management.Sdk.Sfc.dll
Microsoft.SqlServer.Smo.dll
All the dependencies can be found C:\Windows\assembly\GAC_MSIL\
If you want to find the package location deployed in SQL server.
Open SSMS.
Connect to Integration Services.
Go to View and Click "Object Explorer Details".
Now you select your package to know the package path in SQL server.
Take a look at the screenshot below.
Ignore the server name because it will be parameter for the LoadFromSqlServer method.
So package path should be : \Stored Package\MSDB\Data Collector\PerfCountersUpload.
Hope this helps.