So I have this SSIS package that works flawlessly when I start it within Visual Studio Integration 2012. However when I deploy this package via the SQL Catalog it neither works or breaks. It just runs with no output.
I have no idea why this is. My package is simple.
Query a DB table
Place data into a Recordset Destination using a local variable
Pass the local variable along with some project parameters into a script that is embedded into a sequence > foreach container. The script then updates the same db table with a new value based on the logic within the script and it also sends out an email using the data pulled from the query.
I know that my query in #1 is working. I can see it through my job execution reports... it writes in all rows from the DB table. Then it just sits in my script doing God knows what... So I am guessing since it is not my query it must be one of the project params/connections (that I set using an environment). Here is my syntax for these. Do you see anything wrong with them? Please ignore the code I left out as I am sure it is not breaking there. Also I am 100% positive I am setting the values correctly in my environment and that they are properly assigned in the package.
Here is my relevant code snippets (do not mind order):
Dim oleDA As New OleDbDataAdapter
oleDA.Fill(dt, Dts.Variables("User::CustomerRows").Value)
ScriptMain.Wrapper = New ET_AutoRenewal.WrapperClass(Dts.Variables("$Project::etUserName").Value.ToString(), Dts.Variables("$Project::etPassword").Value.ToString())
Dim cm As ConnectionManager = Dts.Connections("test")
Dim sqlConn As SqlClient.SqlConnection
Dim sqlCmdText As String
sqlConn = DirectCast(Dts.Connections("test").AcquireConnection(Dts.Transaction), SqlClient.SqlConnection)
sqlCmdText = "UPDATE [" + Dts.Variables("$Project::sDatabaseName").Value.ToString() + "].[dbo].[temp_AutoRenewal_listpull] SET [process_flag] = 1 WHERE [confirmation_num] = '" + xmlDict.Item("confirmation_num") + "'"
So if that all looks good, which to me it does. How could this happen, it works fine in VS but not when deployed. What is more frustrating is nothing fails, it simply runs and there is no output (ie no table updates or emails sent out).
Are there any techniques to debug a deployed package?
Related
I've created an application that uses the following logic to query data from SQL to my MS Access App.
Using an ODBC connection I execute a stored procedure
Using This is assigned as a Pass-Through Query to pull the data locally.
It looks something like this:
strSQL = "EXEC StoredProcedure " & Variable & "
Call ChangeQueryDef("qryPassThrough", strSQL)
Call SQLPassThrough(strQDFName:="qryPassThrough", _
strSQL:=strSQL, strConnect:=gODBCConn)
Me.frmDataSheet.Form.RecordSource = "qryPassThrough"
But, recently we have upgraded our SQL Server to 2016 using a high availability failover system - hence our connection string has changed to connect to a listener like so:
gODBCConn = "ODBC;Driver= {SQL Server Native Client 11.0};Trusted_Connection=Yes;Regional=Yes;Database=" & varDB & ";MultiSubnetFailover=Yes;IntegratedSecurity=SSPI;Server=tcp:SERVER_LISTENER,1433;"
However, it looks like using SQL Server Native Client in the connection string is not the same as what we originally had which was SQL Server. Certain data types have changed and do not work in Access.
Is there a better way for me to query data from SQL and persist/display this data in access using ADO or an alternative method?
EDIT Based on Comment:
The issue I'm having is that I have tables in SQL using the data type: Decimal(12,2). With some testing and experimenting this seems to fail when using an ODBC pass-through query. But changing the data type to Float seems to work fine. Then there are other datatypes which seem to error too which I've not managed to find yet. It just seems there are a few difference which I'm not aware of and I'm keen to find a better way to load data into my Access App.
EDIT 2
This is the error message I get relating to the data type issue.
Sounds like you're not really interested in making the underlying data structure compatible with Access, so:
How to load an ADODB recordset into a datasheet form
Create the form
First, create a datasheet form. For this example, we're going to name our form frmDynDS. Populate the form with 256 text boxes, named Text0 to Text255. To populate the form with the text boxes, you can use the following helper function while the form is in design view:
Public Sub DynDsPopulateControls()
Dim i As Long
Dim myCtl As Control
For i = 0 To 255
Set myCtl = Application.CreateControl("frmDynDS", acTextBox, acDetail)
myCtl.NAME = "Text" & i
Next i
End Sub
VBA to bind a recordset to the form
First, we're going to allow the form to persist, by allowing it to reference itself:
(all on in the code module for frmDynDS)
Public Myself As Object
Then, we're going to add VBA to make it load a recordset. I'm using Object instead of ADODB.Recordset to allow it to both take DAO and ADODB recordsets.
Public Sub LoadRS(myRS As Object)
Dim i As Long
Dim myTextbox As textbox
Dim fld As Object
i = 0
With myRS
For Each fld In myRS.Fields
Set myTextbox = Me.Controls("Text" & i)
myTextbox.Properties("DatasheetCaption").Value = fld.NAME
myTextbox.ControlSource = fld.NAME
myTextbox.ColumnHidden = False
i = i + 1
Next fld
End With
For i = i To 255
Set myTextbox = Me.Controls("Text" & i)
myTextbox.ColumnHidden = True
Next i
Set Me.Recordset = myRS
End Sub
Use the form
(all in the module of the form using frmDynDS)
As an independent datasheet form
Dim frmDS As New Form_frmDynDS
frmDS.Caption = "My ADO Recordset"
frmDS.LoadRS MyAdoRS 'Where MyAdoRS is an open ADODB recordset
Set frmDS.Myself = frmDS
frmDS.Visible = True
frmDS.SetFocus
Note that you're allowed to have multiple instances of this form open, each bound to different recordsets.
As a subform (leave the subform control unbound)
Me.MySubformControl.SourceObject = "frmDynDS"
Me.MySubformControl.Form.LoadRS MyAdoRS 'Where MyAdoRS is an open ADODB recordset
Warning: Access uses the command text when sorting and filtering the datasheet form. If it contains a syntax error for Access (because it's T-SQL), you will get an error when trying to sort/filter. However, when the syntax is valid, but the SQL can't be executed (for example, because you're using parameters, which are no longer available), then Access will hard crash, losing any unsaved changes and possibly corrupting your database. Even if you disable sorting/filtering, you can still trigger the hard crash when attempting to sort. You can use comments in your SQL to invalidate the syntax, avoiding these crashes.
You previously used the pretty ancient, original ODBC Driver for SQL Server simply named SQL Server. You made the right decision to use a newer driver to support your failover cluster. But I would not recommend to use SQL Server Native Client. Microsoft says, It is not recommended to use this driver for new development.
Instead I would use the Microsoft ODBC Driver 13.1 for SQL Server. This is the most recent and recommended (by Microsoft) ODBC Driver for SQL Server.
Your main issue seems to be a translation issue between Access and SQL Server via the ODBC layer. So, using the more modern driver might very well make this problem go away. - I do not know if it solves your problem, but this is the very first thing I would try.
I am fairly new to SSIS, Visual studio. Thought that might be good to mention in the beginning.
What I wanted to achieve was to download a certain xls file from http://www.ads-slo.org/statistika/ website and store it in a certain folder on my computer. I have achieved that, but the problem is that I know how to do it one file at a time. I did it by opening new connection, going to http connection and in the manager typing the server url: which in my case if lets say we start with January 2016 was this:http://www.ads-slo.org/media/xls/2016/Januar-2016.xls. After doing so I've constructed a script task or more or less copied it from a website that downloads the file given a certain url based on the connection manager.
My problem is that I would like to download all of the files on this site, so starting with January 2007 and ending with January 2016 with a single package and by not changing my connection manager server url settings 100 times.
Is there any way you might help me. I would be forever grateful.
Thank you in advance.
Kind regards, Domen
Here is one very simple example (it can be improved - see comments after code block) of changing a connection string dynamically by using a Script Task. You can also dynamically change connection strings using expressions and the Connection Manager's expressions property. However, since you are using a Script Task to handle the downloads, I have demonstrated it using one.
As you haven't tagged the Script language (VB or C#) you are using, I have written a rough draft in VB.
I have added comments, but stackoverflow syntax highlighting interprets it strangely; apologies.
Public Sub Main()
' Get the HTTP Connection
Dim connObj As Object = Dts.Connections("HTTP Connection Manager").AcquireConnection(Nothing)
Dim connection As New HttpClientConnection(connObj)
' Static list of month names for dynamic connection string (obviously add as many as needed)
Dim monthNames As String() = New String() {"Januar", "February", "March"}
' Nested loop - for each year and month, try to download the Excel file
For Y As Integer = 2007 To 2016 Step 1
For M As Integer = 0 To monthNames.Length - 1 Step 1
' Set the assumed name of the remote file
Dim remoteFileName As String = monthNames(M) + "-" + Y.ToString() + ".xls"
' Change the connection string a.k.a dynamic connection string
connection.ServerURL = "http://www.ads-slo.org/media/xls/" + Y.ToString() + "/" + remoteFileName
' Set where to download the file to
Dim localFileName As String = "C:\Temp\" + remoteFileName
Try
connection.DownloadFile(localFileName, True)
Dim buffer As Byte() = connection.DownloadData()
Dim data As String = Encoding.ASCII.GetString(buffer)
Catch E As Exception
' File may not exist on remote server, delete the blank copy it attempted to create
File.Delete(localFileName)
End Try
Next
Next
Dts.TaskResult = DTSExecResult.Success
End Sub
How can this be improved?
One potential improvement is to parse the remote server for the folders and directory contents (to save having static lists of month names, hardcoded start and end years and building file names) using a HttpWebRequest.
However, there might be an issue with the remote server permissions in allowing such requests to be made so you would have to investigate further and speak with the server administrator.
Testing the above code, it successfully downloaded the Januar-2015 and Januar-2016 Excel files from the website.
I crafted a macro in an Excel workbook to extract a subset of data from a SQL database based on user input.
The macro prompts the user for a parameter input and inserts that parameter into a ready-made stored procedure configured into a an Excel data connection - see below for my vba:
Sub RefreshDBQuery()
Dim Val As Integer
Application.ScreenUpdating = False
Worksheets("Adjustable CF").Select
Val = InputBox("Enter valid 4 digit number", , 1907)
Sheets("TestData").Visible = True
Worksheets("TestData").Select
Worksheets("TestData").Range("A1").Select
ActiveCell.Value = Val
With ActiveWorkbook.Connections("MacroExtraction 2Server").OLEDBConnection
.CommandText = "EXEC dbo.prV_FlowExtract '" & Range("A1").Value & "'"
End With
ActiveWorkbook.Connections("MacroExtraction 2Server").Refresh
Sheets("TestData").Visible = False
End Sub
When I run it - it works fine and additionally, since it's modifying an existing data connection ( the one I previously configured), I notice a odc file in a folder called "My Data Sources" under My Documents:
However, when I send this workbook over to a colleague to run the macro and to extract data - the macro is able to run up to a point, and she receives an error:
I ask her to open up the folder "My Data Sources" and I don't see an odc file:
My question is: what am I missing? Or rather what is my colleague missing in order to get her macro to work on her local machine?
I checked with the dB administrator who said that she has the permissions necessary to access the server, so that's why I am picking on the lack of the odc as a cause for my concern. Should I copy my odc file and send it to her to copy into her Data Sources folder? Should I rewrite the macro and re-setup the data connection on her local machine? Anyone with experience to comment would be much appreciated! Thanks!
The macro alone does not contain all the necessary info (server name, for example?). Try running "NewSQLServerConnection.odc" in you colleague's my data sources location, complete the necessary data, make sure the connection name is the same as in your macro, and then the macro should work.
Hope this helps!
I have some old VB Script code that I can modify but I cannot migrate the app to a new language...
This VB Script basically used to connect to a SQL 2000 Server and manipulate the connections of the package before running it, which would output a flatfile database locally.
Now I do not have a DTS package, I just have an SSIS package.
The code used to be this:
dim DTScon
dim DTSpkg
set DTSpkg = Server.CreateObject("DTS.Package")
DTSpkg.LoadFromSQLServer "mysqlserver","myuser","mypass",dts.DTSSQLStgFlag_Default,,,,"MyPackageName"
set DTScon = DTSpkg.Connections.Item("Conn1")
set DTScon.UserId = "conn_username"
set DTScon.Password = "conn_password"
set DTScnp = DTScon.ConnectionProperties.Item("Data Source");
DTScnp.Value = "c:\path\to\output\flatfile"
I am now trying to modify the code to
dim DTScon
dim DTSpkg
set DTSpkg = Server.CreateObject("DTS.Application")
DTSpkg.LoadFromSQLServer "mysqlserver","myuser","mypass",dts.DTSSQLStgFlag_Default,,,,"MyPackageName"
set DTScon = DTSpkg.Connections.Item("Conn1")
set DTScon.UserId = "conn_username"
set DTScon.Password = "conn_password"
set DTScnp = DTScon.ConnectionProperties.Item("Data Source");
DTScnp.Value = "c:\path\to\output\flatfile"
First error I got is:
Microsoft VBScript runtime error '800a01b6'
Object doesn't support this property or mother: 'DTSpkg.LoadFromSQLServer'
/main.asp.241
This I am guessing will be one of the many hurdles I will probably have to overcome.
However, I have struggled to find a solution to this by Google searching.
Has anyone got an idea of what I need to do to get IIS running this new code?
Or any problems I am likely to face trying to do this?
Your best option would be to change your SSIS packages to use Package Configurations for the Connection Managers. If you use SQL Server configurations, when the package is loaded and executed, it will change the Connection Managers based on what it finds in the SQL Server configuration table. So your steps would be to alter the data in that table, then start the packages - no direct manipulation of the packages is necessary.
My DBA has several SSIS packages that he would like the functionality of providing the end user with a way to input values for variables in the package. What would be the best solution for creating an application that would take the user input and pass the data through to the SSIS package variables?
I have looked at http://blogs.msdn.com/b/michen/archive/2007/03/22/running-ssis-package-programmatically.aspx , and I have even come pretty close with some of the information here - http://msdn.microsoft.com/en-us/library/ms403355(v=sql.100).aspx#Y1913 .
I can get this work locally using this code
Dim packageName As String
Dim myPackage As Package
Dim integrationServices As New Application
If integrationServices.ExistsOnSqlServer(packageName, ".", String.Empty, String.Empty) Then
myPackage = integrationServices.LoadFromSqlServer( _
packageName, "local", String.Empty, String.Empty, Nothing)
Else
Throw New ApplicationException( _
"Invalid package name or location: " & packagePath)
End If
myPackage.Variables.Item("Beg_Date").Value = startDate
myPackage.Variables.Item("End_Date").Value = endDate
myPackage.Execute()
Problem is this requires that the user have SSIS installed locally.
You can use variables as the configuration and pass them through at run time. We do this with parent packages that call child packages (which have the variables) but I believe it is possible to send them directly in the call to the package (which you could create dynamically) this way as well.
You could store the variables in a config table and have the user update the config table through t-sql and then call the package as well. This would only work if you have different parent packages for each user or there is no way that users would be running at the same time to avoid race conditions.