Simple question: how does the execution of a ssis package differ when using the SQL Server 2012 Integration Package Utility versus when using the 2012 SQL Server Agent to execute a package? (holding all else constant within deployment- ie using the local file system in both cases)
Thank you.
I am asking because I have an externally referenced dll in my ssis package script that successfully executes with the package utility but not when deployed to the server agent within SQL server 2012 mgmt studio.
If you are sure that the agent service is running on an account that has proper permissions, check to ensure that you are executing the same runtime version for both instances. Your driver may be 64 bit and incompatible with 32 bit runtime and vice-versa.
There is an option in "execution options" of the agent job step that will allow you to select the 32 bit runtime, if needed.
If you are unable to get the "SQL Server Integration Services Package" job step type to work, you can change to using the "Operating system (CmdExec)" job step type and simply build a command using the utility executable (dtexec) and copying the info from the "Command line" tab of the utility. Some prefer this method of executing SSIS packages with the agent as it provides cleaner maintenance of the jobs. Searching the google for this topic will bring up articles like this: http://bi-polar23.blogspot.com/2008/06/ssis-and-sql-server-agent-choosing.html
Hope this helps.
Related
I have an SSIS package that will be run in a another database and has some parameters that will depend on the environment in which will be running (database name, FilePath for an excel file, ServerName, etc)
and I want to set these in a way that when I send the package to the third party they are able to configure these parameters and run the package in an SQL job using SQL server agent (is the easiest way I thought for them to run a manual SSIS package).
In order to do this, I have "parametrized" the values I need in SSIS
these parameters are set at Package level.
and what I'm trying to do now is to pass these parameters from SQL Server Agent job configuration step set data tab.
I'm getting this error when running the job:
Executed as user: NT Service\SQLSERVERAGENT. Microsoft (R) SQL Server Execute Package Utility Version 14.0.2027.2 for 32-bit Copyright (C) 2017 Microsoft. All rights reserved.
Started: 9:47:32 PM
Error: 2020-04-05 21:47:32.91
Code: 0xC001F016
Source: Package
Description: Changing the Value of a variable failed because it is a parameter variable. Parameter variables are read-only. End Error
DTExec: Could not set \Package.Variables[Database_Name].Value value to dev.
Started: 9:47:32 PM
Finished: 9:47:32 PM
Elapsed: 0.125 seconds.
The package execution failed. The step failed.
These is the important line:
DTExec: Could not set \Package.Variables[Database_Name].Value value to dev.
I'm thinking that is having problems setting this variable, I don't know if it's something I setup wrongly in SSIS or the way I'm trying to assign it in SQL Sever Agent.
(the way I'm calling it I took it from recommendations online).
to summarize; I need to send parameters from sql server agent to the SSIS package.
Plus*:I'm also open to recommendations extra for a way to have SSIS packages executed easily for a third party without SQL knowledge.
P.S. I've been recommended to use catalog, but will this be possible to use when a third party has to use the package in his environment? i.e. will he be able without visual studio to "deploy" the package?
I was getting this error also:
DTExec: Could not set \Package.Variables[.....].Value value to ....
In my case, I was using VS2019. The solution project target server is 2019, which is different from our test SQLServer (2017). After changing the target server to 2017, the parameters got passed to the dtsx as expected.
I've been tasked with deploying a handful of SSIS packages on Microsoft SQL Server 2012 Enterprise 64-bit (11.0.6020.0) running on Windows Server 2012 R2.
The packages run perfectly when run from SQL Management Studio by right-clicking the package and clicking execute with the proper parameters.
However, when I schedule them to run under the SQL Server Agent, some of them fail. I have not been able to identify any pattern that distinguishes the failing packages from the working packages.
All of the packages are run using a, probably unconventional, approach which involves creating the job step in the job as a Operating system (CmdExec) step rather than a SQL Server Integration Services Package step. This is because the packages depend on access to a particular SMB share and must have credentials provided first to access this share. The CmdExec step calls a wrapper batch file and passes the package name and the environment reference for dtexec to run the package.
Here's the file bat file:
set ETL_PACKAGE=%1
set ETL_ENV_REF=%2
net use \\remote-share.some-domain.local\dropfolder\in /user:ssisagent mypassword
dtexec /Envreference %ETL_ENV_REF% /ISSERVER \SSISDB\%ETL_PACKAGE%
if "%errorlevel%"=="1" (
exit /b %errorlevel%
)
When the agent runs these jobs by rightclicking the job and selecting "Start Job at Step..." the job is started succesfully, but when you pull an execution report it fails with this error:
Message Source Name: Transact-SQL stored procedure
Operation is not valid due to the current state of the object.;
at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ISServerExecutionEvents.LoadPackage(PackageItem item)
at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.StartPackage()
at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.PerformOperation()
I would appreciate ANY idea about what could possibly cause this error or how I can troubleshoot it to narrow down the causes.
I have tried setting up a proxy account using my own admininstrator credentials to rule out possible permission issues, but it doesn't change a thing. I have checked permissions on the SSIS environment used, but that doesn't change anything.
The packages in question interact with DB2, MS SQL and various fileshares with CSV/XML files.
When setting up a SQL Agent job, there are options for choosing, among others, "SQL Server" or "SSIS Package Store". If I choose "SSIS Package Store" and browse to MSDB instead of File System, I can choose SSIS packages stored in sysssispackages.
However, the documentation for the /DTS switch, which SSMS automatically chooses when selecting SSIS Package Store, appears to be only for packages stored on the file system.
I've also noticed that if the user executing the job doesn't have file system rights, the job will fail when choosing SSIS Package Store --> MSDB --> Package with an "Access is denied" error or "~Object doesn't exist". Since the package doesn't exist on the file system though, why does execution work when choosing SQL Server and fail when using SSIS Package Store? Is dtexec actually looking for a dtsx file on the file system even when MSDB is chosen? I have, likely a false, understanding that SQL Server and SSIS Package Store --> MSDB are the same thing when referencing the Server on which the agent is running.
I've done some research and I can't find a clear distinction of the differences, if any, between using SQL Server and SSIS Package Store when selecting from the MSDB folder.
Can anyone shed some light on the differences?
There are 2 locations to store SSIS packages: The file system or SQL Server. The rest is semantics.
File System
You can use the SSIS Package Store which is nothing but a well known location in the installation location.
%Program Files%\Microsoft SQL Server\{Version}\DTS\Packages
Or you can pick anywhere on the file system you like. If you go this route, then you'll need to ensure the SQL Agent account, or the credentialed proxies or, if you running packages from xp_cmdshell the SQL Server Service Account has access to that location.
The only advantage, if you want to call it that, of using the Package Store (i.e. the folder I mentioned) is you can use the Integration Services management tool that exists in SSMS (by connecting to Integration Services instead of database engine).
However that has a lot of pitfalls such as not being able to handle multiple instances, packages only run in 64 bit mode, no access to proxy accounts, etc. You shouldn't run packages from SSMS anyway.
SQL Server
If memory serves correct
2005 - stored in msdb.dbo.sysdtspackages90
2008 - stored in msdb.dbo.sysssispackages (I seem to recall 2008 RTM using a different table, or reused the 90 table but that got patched out)
2008 R2 - stored in msdb.dbo.sysssispackages
2012 (package deployment model) - stored in msdb.dbo.sysssispackages
2012 (project deployment model) - stored in SSISDB.catalog.packages*
2014 (package deployment model) - stored in msdb.dbo.sysssispackages
2014 (project deployment model) - stored in SSISDB.catalog.packages*
*With the project deployment model, packages are "compiled" (zipped with a manifest) into a .ispac which is stored into the bowels of the SSISDB.internals.* tables.
Wrapup
Ultimately, where you store your packages does not affect your ability to run them. You can run packages using DTEXEC, SQL Agent or custom .NET code. The choice of storing packages is primarily dependent upon your management style.
References
Package Management
Our dev server has SQL Server 2008 R2 64-bit with SSIS. In order to be able to import XLSX files directly from T-SQL (via OPENROWSET), we installed on the dev server the Microsoft Access Database Engine Redistributable 64-bit. The server already has the 32-bit version installed, so we installed it with the /passive switch.
Since doing that, our existing packages are failing. One of them reports
The step did not generate any output. The return value was unknown.
The process exit code was -1066598274. The step failed.
Just wondering if anyone else has successfully configured their server to be able to use OPENROWSET to XLSX files and also have SSIS on the same server, Or does someone know if this configuration is not possible?
Edit: I should add that the failing packages are run from an SQL Agent job. When I connect to the server's Integration Services Manager via SQL Server Management Studio and run the package (by navigating to the package, right-clicking it and selecting Run Package), it executes successfully.
Personally I've given up on trying to support this scenario - I think the "/passive" install is a hack that doesnt actually work and usually breaks your other install of ADE. I would unintall ADE entirely and then only re-install 32-bit. I would then force all packages run via SQL Agent to execute as 32-bit (as described by Brian).
Following this process gives you some hope of debugging issues as you can open your package in Visual Studio and it will use the same 32-bit driver. You will also use a consistent driver when executing from SSMS (also 32-bit).
Yes there may be a performance hit but SSIS performance is typically limited to some extent by I/O or network speed, not just pure engine throughput.
Yes, it is possible. You have to specify to use the 32 bit runtime in the job step. Open every job step that runs one of your SSIS packages and click on the "Execution Options". Check the box at the bottom "Use 32 bit runtime".
I had "The step did not generate any output. The return value was unknown. The process exit code was -1066598274. The step failed" too.
I found a walk-around for myself: I switched from using xlsx to xls, and the error isn't raised anymore.
I am learning to work with the SSIS packages.
I have made an SSIS package to work with the delta data(changed data since last run).
But i am having troubles running the package outside the development enviroment
I have tried to runt it with DTEXECUI.EXE utilty but it gives me the following error.
I have also tried to run it from a stored procedure by enabling the xp_cmdshell but i still get some error like this.
Please help, i am also trying to Google my way through.
It's clear from the error messages To run a ssis package outside of sql server data tools you must...
SSIS 2012 components have to be installed on your server to resolve the issue.
As per this post:
What you need are the base SQL Server binaries which are installed
with the DB engine. So you need a SQL Server Instance of some flavour
to have dtexec running. Whether it's the DB Engine or the SSIS service
installed but disabled.
Running packages in BIDS is a special case. Run the package on the
same PC as BIDS via dtexec and it will fail unless you have installed
server components (= a SQL Server Instance)