I have implemented a custom SSIS log provider where the core SSIS log properties are conveyed to a web api endpoint to log to an external system by overriding the Log method. Where I am having difficulty is in understanding how to access the DTS Runtime Parameters to do different things based on the build mode.
In particular I have a Project Parameter in the SSIS project for "FilePath" (ex. "C:\debug.txt") that gets transformed based on the build to "C\release.txt" when in release mode.
I am unable to access the DTS runtime similar to script tasks in the custom log provider that is installed in the GAC so am looking for suggestions.
You probably won't like that answer, however, it is not possible to access the package variables from the custom log provider similar to script tasks.
Referece: social.msdn.microsoft.com
Related
I have designed a SSIS project and deployed it to SQL server and also created the job to run on daily basis but its giving me this error when executing this as job (doesnt give any error within VS):
There is this CLSid in this error message but there is no application associated to it in
--> Component Services -> Computers -> My Computer -> DCOM Config
But this CLSid is registered inside registry editor
About this particular task on which this error is occurring: This is a script task which is modifying and deleting the un-wanted rows from the excel file in which I am trying to write SQL table data.
Script task code looks like this:
I have been working for hours now trying to fix this problem but no success. Kindly guide me how can I fix this issue. If any other information is required related to this project, please let me know....
Doing Excel automation in a SQL Server agent job is totally unsupported and probably won't work.
To have even a ghost of a chance of making this work you'll need to run a real desktop session on the server and automate Excel in that. Excel expects a real user to be logged in with a full profile. And Excel has failure conditions where it displays a popup window, which you'll need to be able to access via remote desktop.
You can read and write Excel files on a server with the OpenXML SDK, without actually having to run Excel. There's also a wrapper library called ClosedXML which you may find easier to use than using OpenXML directly.
tl;dr;
You need to install Office (Excel) on the server AND ensure that you install it in a manner that mirrors the SQL Agent's expected bit-edness. Default for Agent is going to be 64bit, default for Office is still 32 :(
Error guessing
You have a script task that uses the Office interop libraries to delete some rows (2 through 11?) out of a spreadsheet.
You have Office installed on your machine and therefore you have the libraries installed. Excel still has COM based "stuff" in it, thus the interop and errors shrieking about the CLSid, registry, etc but that's likely just secondary errors because there is no base "application is not installed" exception to be thrown.
If Office is installed, then ensure your agent execution model matches the version of Office. If 32 bit Excel is already installed, don't potentially break everyone else's stuff by uninstalling and reinstalling as 64 bit, just got the Advanced section of the SQL Agent Job Step and check the 32bit box.
Once all that's done, then if you're still getting errors but new ones, then the existing comments mentioning permissions may come into play - it depends on where the Excel document actually exists (on the computer where SQL agent can access vs on the computer where it cannot vs networked drive)
Good luck in not finding people on the sanctions lists.
We have several SSIS projects on Azure DevOps, everytime I pull the project and another user committed before it destroys my dtproj file. I tried to put the file in the gitignore but then the project do not show me the new files created by other users. I think the problem is that the credentials are stored encrypted inside the file and when the user uploads his credentials it destroys mine.
Any idea how to avoid this error?
Warning loading ETLXXX.dtproj: Warning: Failed to decrypt sensitive data in a project with a user key. You may not be the user who encrypted this project, or you are not using the same machine that was used to save the project. If the sensitive data is a parameter value, the value may be required to run the package on the Integration Services server.
ETLXXX.dtproj 0
This is caused by the package protection level configuration. You should not encrypt the package with the user key in production. This should only be used in the development phase or in some rare cases after being deployed.
I recommend reading more about the SSIS package access control and how it should be changed during the package lifecycle:
Access Control for Sensitive Data in Packages
Securing Your SSIS Packages Using Package Protection Level
I have created a SSIS package which is used to connect to the FTP server and retrieve all the xml files from that server. I have used mostly variables like sftp_server, username, etc., How can I make it a customisable package?
In other words, How can i plug in with other project and pass all the required fields and do the same thing there.
This is my package
Since your screen shot looks like it's from SSIS 2012+, you can make your variables into parameters. Then you can call the same package from multiple jobs, and in the job step where you call the package, you can go into the Configuration tab and set the values of the package parameters for that job.
So different jobs can all call the same package and all pass their own values for FTP Server, Credentials, Local Paths, etc.
This assumes you are using the Project Deployment Model that became available in 2012. Otherwise you can do the same thing using Config files instead.
I use EPPlus.dll library for generating Excel files on the fly for attachment and generate mailer in Script Task of SSIS package.
When there is new requirement for change in mailer comes, I do the change in Script Task on my local machine and send the built package file ( .dtsx ) to DBA team for deployment.
Now everytime I have to ask the DBA team to share production server screen with me where I:
Open the file in Visual Studio Data Tools solution
Browse to the dll location
Add Reference to the dll in Script Task.
Then they import the package file in MSDB from where scheduled job references and executes the package.
If I dont do the above step, the Script Task throws error of reference not found.
Error 1 The type or namespace name 'OfficeOpenXml' could not be found
(are you missing a using directive or an assembly reference?)
I overcame challenge of installing DLL inside GAC that is being referenced in Script Task by dynamically loading the assembly as below'
public void Main()
{
AppDomain.CurrentDomain.AssemblyResolve += new ResolveEventHandler(CurrentDomain_AssemblyResolve);
}
private System.Reflection.Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
{
return System.Reflection.Assembly.LoadFrom(System.IO.Path.Combine(strDLLPath, "EPPlus.dll"));
}
But I am, not able to find steps to avoid manually browsing and adding DLL reference. Please help as DBA team is reluctant/ avoids sharing screen.
Alternatively, what is the correct/best practice way to get package file deployed on server in which external dll is used if I dont have direct access to production server.
I don't think there is a direct way to do that from integration services packages, because the only workaround you can do it is - (what you have done using CurrentDomain_AssemblyResolve function) - loading an assembly from a location instead of installing it into GAC.
There are 3 ways for using custom dll's on deployment:
Assigning DLL's to the GAC
Using the AssemblyResolve Function
Copy all Dll's to the sql server DTS Assemblies folder (example for SQL Server 2008: C:\Program Files\Microsoft SQL Server\100\DTS\Binn) and to the .Net framework assemblies folder.
if the problem is to ask for screen sharing, you can create a small installation wizard that copy these dlls to the specific location and ask the dba team to execute it.
Workaround
When searching for this issue, i found an interesting workaround by using a Web Service instead of direct dll, so you can create a web service which contains the methods you are using and add a Web reference instead of a Local assembly
Side Note: i didn't try this approach before, but i am only trying to help
Useful Links & References
SSIS custom DLLs during deployment
How to load an Assembly in a SSIS script task that isn’t in the GAC
SQL Server SSIS Custom DLL Folders
Using Custom DLL's in an SSIS Script Task
SSIS Script Task cant find reference to assembly
Referencing Other Assemblies in Scripting Solutions
Access WebService via SSIS Script Component
Calling a secure webservice in SSIS through script task
Consume Webservice via SSIS Script Component
HOW TO: Write a Simple Web Service by Using Visual C# .NET
I'm trying to move my packages to production using a configuration file, but file is changed only partly and the results go still to DEV server.
Does anybody know what to do?
It is difficult to isolate the cause of your issues without access to your configuration files.
What I suggest you do is make use of package configurations that reference a database within your environment. The databases themselves can then be referenced using environment variables that are unique to each environment.
This a brilliant time saver and a good way to centrally manage the configuration of all your SSIS packages. Take a look at the following reference for details.
http://www.mssqltips.com/tip.asp?tip=1405
Once configured, you can deploy the same identical package between dev and production without needing to apply a single modification to the SSIS package or mess around with configuration files.
You could still have hard-coded connections in your package even though you are using a configuration file. You'll need to check every connection as well.
You can also go the long way around. Go into Integration Services and Export the stored package to its dtsx file. Then you can pull open the file in any good text editor, do a find/replace on your server name and then go back into Integration Services and Import the updated package. Alot of times it's just easier...
everybody and thanks for answering. I'd managed to solve this problem in an ugly way - editing packages on server, but I'd like very much more elegant solution - now I'm trying with environment variable,it seems great, but the wizard that I'm getting is different from that is given in link - and I don't know how to continue.(I'm using VStudio 2005) Besides, I tried configuration file as XML, but package run fails even on the source machine, so I'm stuck !
My personal technique has been to first have a single config file that points the package to a SQL Based Package Config (the connection string to the config DB). Subsequent entries in the package config use the SQL store to load their settings. I have a script that goes into the XML of the package and preps them for deployment to stage or prod. A config file holds the name of the Package Configuration's initial file config entry and where the stage and prod configuration db configruation file is located. The script produces two subdirectories for stage and prod. Each directory has a copy of the solution packages modified for their particular deployment.
Also! Don't forget to turn off encryption in the package files!