I have deployed my SSIS solution to SQL Server 2014. The package is run manually from Management Studio. Randomly, the package shows status running in execution report, but when I connect to to Integration Service using Management Studio it doesn't show any package running. The package just hangs without any error, nor any step is started or completed.
How can I troubleshoot this? What could be causing this error?
Just for the records, I found that the problem rely in a database timeout caused by having too much logs in SSISDB tables.
The SSIS Execution Process could not write to the IS catalog: MAMSQLT02:SSISDB Error details: Unable to add new messages to the table of operation messages.:Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.; at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ISServerHelper.AppendEventMessage(SqlCommand cmd, Int64 operationId, Int32 message_type, DateTimeOffset message_time, Int16 message_source, String message_text, Nullable1 extendedInfoId, String packageName, String packageLocationType, String packagePathFull, String eventName, String messageSource, String messageSrcId, String subComponentName, String packagePath, String executionPath, UInt32 threadId, Nullable1 messageCode, Int64& eventMessageId) at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.BasicEventInfo.WriteInternal(SqlCommand sqlCommand) at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.Loggable.Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ILoggable.Write(SqlConnection conn)
Truncating the tables using SP [internal].[cleanup_server_retention_window_truncateall] solved the problem
Related
I started in a large company as an IT consultant. One of my tasks is to manage an application that has a SQL database.
I have very limited knowledge of SSIDB and SQL Server Management Studio - but I am willing to learn.
The SQL database is updated with external data. This can be done by users directly in the application, but it also happens through a scheduled job. The job runs from SQL manager. The job has only two steps, one of which is to execute a dtsx package.
The dtsx package is set up retrieves data from an external FTP server and merges the data into the database. The job was made by my predecessor and has run flawlessly for a very long time.
Now we are in the situation where the FTP, supplying the data, has been changed.
I have therefore been inside the Connection managers and changed to the new FTP server.
Running the jobs however we still get the following error message:
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”
I have checked the Connection managers and the ChunkSize is unchanged from when the job was working correctly. ChunkSize is set to 1000, both in the Connection manager, but also in the dtsx package itself.
When I have searched for the problem, it is mentioned that it may have something to do with the connection to the FTP server. So I have checked the connection to the FTP server from the server where the SQL database is located - and there is a connection. In addition to this, I have also made sure that there is a firewall rule that allows traffic between the two servers. This is ensured across protocols and port 20-22
When the job itself is run, however, no traffic leaves the server. So I believe the problem is with the job itself.
Edit: after having done a validation of the package i have gotten the following.
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”.
: at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties(Sting parameterName, object parameterValue)
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties()
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.ValidatePackageWithReference(Int64 validationId, Int64 infold, Int64 projectId, String packageName, Int64 versionId, Nullable'1 referenceId, Project isserverProject, Boolean OfflineMode)
I hope my description is comprehensive enough - otherwise please do write follow-up questions.
ps english is not my first language. sorry if something didn't turn out too well.
I have a SSIS package that pulls data from one DB to another, with some transformation of the data, that runs correctly when I run the package directly, but throws an error when run with the SQL Agent. The job used to run but is now failing with no identifiable changes in the package.
The error is.
An item with the same key has already been added.: at System.ThrowHelper.ThrowArgumentException(ExceptionResourse resource) at System.Collections.Generic.Dictionary'2.Insert(TKey key, TValue value, Boolean add) at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.StartPackage() at Microsoft.SQLServer.IntegrationServices.Server.ISServerExec.ProjectOperator.PerformOperation()
There is an Error when I want to import data from a snapshot(bacpac) file into SQL Server 2019.
It has occur twice on my two different PC.
Please help me if there is any solution about that.
Thank you very much!
Error :
Error Details :
Here are the error details:
===================================
Object reference not set to an instance of an object. (Microsoft.Data.Tools.Schema.Sql)
Program Location:
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.DeploymentScriptDomGenerator.GenerateFragment(Int32 operation, IModelElement element)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.DeploymentScriptDomGenerator.GenerateSteps(Int32 operation, IModelElement element)
at Microsoft.Data.Tools.Schema.Sql.Deployment.OrderedStepGenerator.Add(DeploymentScriptDomGenerator scriptDom, IModelElement element, Int32 operation)
at Microsoft.Data.Tools.Schema.Sql.Deployment.Analyzers.PlanMediator.BuildDependencyOrderedSteps(Int32 operation, List1 classOrder, List1 operationOrder, Dictionary2 changes, Boolean preserveGraphs, Dictionary2& relating, Dictionary2& related)
at Microsoft.Data.Tools.Schema.Sql.Deployment.Analyzers.PlanMediator.BuildOperations()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.OnGeneratePlan()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.GeneratePlan(List1 drops)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeployment.BuildPlan()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeployment.Controller.CreatePlan()
at Microsoft.SqlServer.Dac.DeployOperation.<>c__DisplayClass3.<>c__DisplayClass5.b__1()
at Microsoft.Data.Tools.Schema.Sql.Dac.OperationLogger.Capture(Action action)
at Microsoft.SqlServer.Dac.DeployOperation.<>c__DisplayClass3.b__0(Object operation, CancellationToken token)
at Microsoft.SqlServer.Dac.Operation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.ReportMessageOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.OperationExtension.CompositeOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.OperationExtension.CompositeOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.DeployOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.OperationExtension.Execute(IOperation operation, DacLoggingContext loggingContext, CancellationToken cancellationToken)
at Microsoft.SqlServer.Dac.DacServices.InternalDeploy(IPackageSource packageSource, Boolean isDacpac, String targetDatabaseName, DacDeployOptions options, CancellationToken cancellationToken, DacLoggingContext loggingContext, Action3 reportPlanOperation, Boolean executePlan)
at Microsoft.SqlServer.Dac.DacServices.ImportBacpac(BacPackage package, String targetDatabaseName, DacImportOptions importOptions, Nullable1 cancellationToken)
at Microsoft.SqlServer.Dac.DacServices.ImportBacpac(BacPackage package, String targetDatabaseName, DacAzureDatabaseSpecification creationDefaults, Nullable`1 cancellationToken)
at Microsoft.SqlServer.Management.Dac.DacWizard.ImportDatabase.DoWork()
at Microsoft.SqlServer.Management.TaskForms.SimpleWorkItem.Run()
I had the same issue and was able to resolve this with previous version of SSMS (SQL Server Management Studio) - not the SQL Server version. Previous SSMS versions can be downloaded here:
https://learn.microsoft.com/en-us/sql/ssms/release-notes-ssms?view=sql-server-ver15#1791
ATTOW, the current version is 18.5. It works for me at version 17.9.1 and 18.4.
This works for me on both SQL Server 2017 and 2019, it was the issue with SSMS, not the SQL Server version.
I also tried Azure Data Studio with SQL Dacpac extension (https://github.com/microsoft/azuredatastudio/tree/master/extensions/dacpac) and it failed with same error as latest SSMS version. There's an issue open for this: https://github.com/microsoft/azuredatastudio/issues/9522
Alternatively, instead of installing SSMS, install sqlpackage from https://learn.microsoft.com/en-us/sql/tools/release-notes-sqlpackage?view=sql-server-ver15#184-sqlpackage (version 18.4) and run the command manually. Something like:
sqlpackage.exe /a:Import /df:"import.log" /sf:"mybacpac-2020-04-26.bacpac" /tcs:"Server=.;Database=mybacpac;Integrated Security=true"
I ran into the exact same error message while attempting to import a BACPAC to my local SQL Server 2019. Exact same error on SQL Server 2017 through my testing. The import failed with the latest versions of SSMS and Azure Data Studio.
In my case the error was thrown due to the existence of an external contained user account in the database, principle type E. Type E is External user from Azure Active Directory.
You can check here for reference:
https://learn.microsoft.com/en-us/sql/relational-databases/system-catalog-views/sys-database-principals-transact-sql?view=sql-server-ver15
After removing the user account from the database, I was able to import the database successfully.
Same error as well. I updated it to v18.9.2, instead of downgrading it and it resolved the issue
I am using the SQL Server Import and Export wizard to pull data from Teradata into a SQL Server table:
In SSMS, I right-click my target database and select "Import Data"
In "Choose a Data Source", I select ".NET Framework Data Provider for Teradata"
I change the Command Timeout and Connection Timeout to a high number, say, 3000 (I've also tried 0 and -1).
I set the SQL Server data source
I specify the query to select the data (query takes 15 min+ to run in Toad)
I set the option to create the destination table in SQL Server
I execute the package.
The package fails in pre-execute after ~40 seconds (error below).
I can run this query fine from an SSIS package in Visual Studio, or if I add it to a job. Setting the command timeout there makes a difference. I want to be able to run one-offs directly from SSMS, without going to the trouble of creating an SSIS package every time.
I have no connection timeout limit on the SQL Server side (set to 0).
Error Text:
Pre-execute (Error)
Messages
Error 0xc0047062: Data Flow Task 1: Teradata.Client.Provider.TdException (0x80004005): [.NET Data Provider for Teradata] [100038] Command did not complete within the time specified (timeout).
[Teradata Database] [3110] The transaction was aborted by the user.
[Socket Transport] [115003] The receive operation timed out. ---> System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at Teradata.Client.Provider.WpTcpTransport.ReadLanHeader(Buffer buffer, Int32 timeout, Int32 readBytes)
at Teradata.Client.Provider.WpTcpTransport.ReadLanHeader(Buffer buffer, Int32 timeout, Int32 readBytes)
at Teradata.Client.Provider.WpTcpTransport.Receive(Buffer buffer, Int32 timeout)
at Teradata.Client.Provider.WpSession.Receive(Buffer buffer, Int32 timeout)
at Teradata.Client.Provider.WpMessageManager.Receive(Int32 timeout)
at Teradata.Client.Provider.WpStartRequestManager.ReceiveStartMessage()
at Teradata.Client.Provider.WpStartRequestManager.Action(ManagerActions step)
at Teradata.Client.Provider.WpStartRequestManager.RedriveAction(ManagerActions step)
at Teradata.Client.Provider.WpStartRequestManager.Action()
at Teradata.Client.Provider.Request.ExecuteStartRequest(String commandText, TeraTypeBase[][] parameters, ExecutionMode executionMode, Boolean asynchronous, Boolean isTrustedRequest)
at Teradata.Client.Provider.TdCommand.ExecuteRequest(CommandBehavior cmdBehavior, Boolean asynchronousCall)
at Teradata.Client.Provider.TdCommand.ExecuteReader(CommandBehavior behavior)
at Teradata.Client.Provider.TdCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.SqlServer.Dts.Pipeline.DataReaderSourceAdapter.PreExecute()
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPreExecute(IDTSManagedComponentWrapper100 wrapper) (SQL Server Import and Export Wizard)
Error 0xc004701a: Data Flow Task 1: Source - Query failed the pre-execute phase and returned error code 0x80004005.
(SQL Server Import and Export Wizard)
Information 0x4004300b: Data Flow Task 1: "Destination - MercTestTeradataPull" wrote 0 rows.
(SQL Server Import and Export Wizard)
I had exactly the same issue and just figured out and works for me. Open the SSIS package, find "command time out" which was by default hard coded by SQL in 30 secs. Change it to the time you want then save it. Then it all runs fine.
I created an SSIS package to pull data from OLAP Cube and push it into SQL Server using SSIS 2012. I deployed the same on the SQL Server SSIS DB and created a SQL Server Agent Job to run the package. I have an account configured to run the job (not by creating proxy in the job, but SQL Server Agent is running under that account), that has access to the OLAP Cube. The job is running sometimes and failing sometimes.
Why the job behaving weirdly? Any help on the issue will help me a lot.
I am using SQL Server 2012 SP1 enterprise edition (11.0.3000.0) (if it helps). The error message which pops up when it fails is :
OLE DB Source failed the pre-execute phase and returned error code 0xC0202009
SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E05.
An OLE DB record is available. Source: ""Microsoft OLE DB Provider for SQL Server 2012 Analysis Services."" Hresult: 0x00000001 Description: ""Error Code = 0x80040E05, External Code = 0x00000000:."".
"
I have had the same error on SQL2008 and now on SQL2012 SSIS, but have been able to eliminate / workaround.
We have a Loop Container in our Control flow that contains a data-flow task with an MDX source. The MDX query for the data-flow source is dynamically built (via an expression) on each iteration of the Loop container (however it always returns the "same shaped" results - only the filters in the WHERE clause are different).
We've found the error to be somewhat intermittent - sometimes the package will complete successfully, other times it will fail with the 0x80040E05 error at varying iterations thru the container loop.
To alleviate the problem we setup the SQL Agent job-step for this package to re-try on failure for up to 5 retries - not an ideal workaround, but it helped to improve the success rate of the Job.
We have no idea why this error is occurring or what is causing it, however it appears to be timing-related in some way and I have only seen the issue when using a SSAS OLE-DB data source with a dynamically generated MDX query. I have managed to virtually eliminate the error from occurring with a not ideal workaround in the SSIS package - no idea why this works/helps (hopefully Microsoft will be able to work it out and resolve the issue as it's been plaguing us since SQL2008 and is still here in SQL2012 SP1...
Workaround for MDX causing 0x80040E05 error:
Within our loop container we have added a Script task with OnSuccess precedent constraint to the data-flow task that contains the dynamically generated MDX source query. The script task simply introduces a WAIT in the processing immediately after the data-flow task completes of about 5 seconds, before allowing SSIS to continue with the next iteration (e.g. System.Threading.Thread.Sleep(5000)).
With this delay in place we have had much more stable SSIS package executions - dont know why, but that's what we have observed. Also note that when we migrated to SQL2012 SSIS packages the 0x80040E05 error returned, however we were able to eliminate it once more by increasing the WAIT time to 10 seconds on this script task.
Now waiting for 10 seconds is not an ideal solution / workaround to this problem - particularly when it is contained within a Loop Container (in our case it has added nearly 30 minutes of "WAIT time" to the package execution duration), however this workaround is better than having the package fail 80%+ of the time