Where I work, we are using SSIS to generate several reports. They've been running fine for over a year, possibly much longer, but the people to setup and configured the system are no longer here and I am a web developer, I have no experience with SSIS yet, just standard SQL.
Since the middle of July some of the data is no longer being added to the report. I investigated the server it runs on and found that the following error in the Event Viewer started occurring once a day since the day the data no longer appears in the reports.
Package "Populate fact_PageRequest" failed.
However I can't find any further information about what could be causing the problem.
I opened the .dtsx files in the repository for the reports and found "fact_PageRequest" referenced in one of the flowchart sections of the file, but these reports haven't changed since they were made so I'm guessing it's something more configuration or environment related.
Any suggestions or advice on where to look for more information about the error or possible causes would be really appreciated.
The server it's running on is Windows Server 2012 R2 and SQL Server 2014 if that helps.
There are a few ways to troubleshoot packages. The easiest way assumes that you are using project deployment mode and did not explicitly switch off default logging. Here are the steps:
Connect to your database engine with SSMS
Expand "Integration Services Catalogs" node
Expand SSISDB and locate your package
Right click the package -> Reports -> Standard Reports -> All executions
If the package is not there, it is either stored in the msdb database or in the filesystem and other logging way will have to be applied.
Related
As a company we have grown and we are now moving a couple of SQL Server 2016 databases over to a new server. We have SSIS packages that run off the databases that we are moving from server 1 to server 2.
Is there a way to easily identify using SSMS which SSIS packages use the current server and databases we are moving? Some of the old SSIS packages don't have documentation so we are trying to avoid physically opening up all the SSIS packages. We would prefer to identify the SSIS packages that are impacted.
Thank you!
Here are my solutions on the top of my head. I'm not an expert by any means so don't be surprised if someone comes up with something better.
In SSMS, you can view the data sources being utilized by a
package by... Object Explorer > SQL Server Agent > Jobs > (Your Job) > Steps > Edit... > Data Sources (Tab)
Here, you can view the data sources of your package. This is
slightly faster than opening all your SSIS packages; but, it isn't a
great solution either.
Conversely, recognize that .dtsx files are simply plain text
files. You can scan keywords within all of them using a number of
different scripts (PowerShell, Python, SSIS package w/ a Script
Task, etc.)
What you can use depends on the tech stack that your organization
supports but I imagining Googling for such a program/script would
not be difficult.
If you are utilizing SQL Server configurations in your packages
and you consistently do so for every package, you can query the
[SSIS_Configurations].[dbo].[SSIS Configurations]
NOTE: Solution (1) and (2) do not take configurations into account.
Hopefully, some of these solutions are helpful to you. I would be interested in an efficient means to do this without delving into scripts as well.
I have an issue setting up merge replication on a SQL 2012 instance.
The push of the snapshot to the subscriber is failing and the reason I discovered was because there is a SP that uses a Full Text Index that has not been generated on a table.
A lot of searching about revealed that there is a property that is false by default that defines if a full text index should be copied.
I thought I had found the solution, set this property to true and triggered a new snapshot, however, the same issue was still encountered and when I went back to check the property the copy full text index property was set to false again??
I have tried a few times in the hope it was just me forgetting to save or something, from what I can see, the property stays at true until the snapshot is rerun, after that the property is back at false again, I am wondering if I have come up against a bug in SQL server, however, a google does not appear to indicate this is true.
I have tried deleting and recreating the publication. I have also tried disabling the distributor and publisher in order to force a fresh distribution DB to be created, thinking that maybe there is a corruption somewhere.
Both servers are running SQL Server 2012 on Windows Server 2012R2
Does anyone have any fresh ideas?
I had similar problems with full text catalogs/search setup when using old SQL Server 2012 SSMS version. IDE has many full-text setup related issues(bugs were reported to MS). And only way to achieve proper functionality was to use t-sql commands only. After SSMS became separate from SQL Server product, these IDE bugs were fixed. No problems when working with full-text and SSMS IDE anymore. My SSMS version is 17.8.1, server version 2012 - 11.0.5343.0 (X64)(Build 7601: Service Pack 1).
Maybe your problems is related with IDE bugs too? Try to upgrade SSMS version (if you have older), ant then check situation again.
Update:
Here is related topic with full-text & merge replication problem. Hope this helps:
link1
OK, so I eventually ran out of ideas and logged a call with Microsoft. Turns out that this is a bug and it effects all SQL server versions from SQL 2012 right through to SQL server 2017 from what I am being told.
Microsoft have said they are working on a patch for this but in the meantime I am having to manually script Full Text resources until something comes through.
Hope this helps anyone that comes across this issue.
I would need to migrate a SQL database from Sybase to MS SQL Server. Before doing the actual migration on the production server I first created an SSIS-package with SQL Server Management Studio's Import/Export Wizard for testing with other databases. The test migration was successful and I would now like to deploy my SSIS-package to the real servers.
However, it seems I cannot simply run the package in Management Studio choosing different data sources for it - it only runs on the same databases for which it was created. Now, it can be edited in something called SQL Server Business Intelligence Development Studio (or BIDS for short)(I am using the SQL Server 2008 version), but going through every data flow task changing the destination source manually for each of the ~ 150 tables I am moving is ineffective and also introduces a possibility for error.
I there a way to quickly change what data source is to be used for ALL destination sources in ALL the flow tasks of an SSIS-package? If not, what simple method is there for testing migration with test databases first and simply changing the data sources when deploying?
I am using ODBC data sources, but for some the package shows OLE-sources in BIDS instead.
I hope I was clear enough. If you have additional questions, please ask! Thank you!
I would use a variable for the ConnectionString property of the connection manager. A package level configuration can be very useful for accomplishing this task. Several ways to do this. I prefer to use a table in SQL Server that holds all the configurations for all packages. This can be especially effective if you have multiple packages and need to dynamically change a set of connection managers across those multiple packages.
The basic steps are:
Opposite click on your SSIS design surface and select "Package Configurations..."
Create a package level configuration of Configuration Type "SQL Server"
Store your connection in a Configuration table in SQL Server
Alter your Connection Manager to use a variable for the ConnectionString Property
Populate that variable from the Configuration table via your package level configuration
When it comes time to switch from Test to Production, simply update the connection string in your configuration table
These screenshots can help...
This is part of a larger package management framework that I implemented using this book:
Microsoft SQL Server 2008 Integration Services: Problem, Design, Solution
I highly recommend it. Should take less than a day to set it up. Book has step by step instructions.
This question and its associated answers also helpful.
I recently had the unfortunate experience of having the size of a field changed in a table that was being used in SSIS packages. The developer who wrote the packages had since retired, and had grouped them all in one VS BI project. She developed them solely on her local PC and moved them to a shared drive when she left.
Anyone with any knowledge of Protection Levels in SSIS knows what happened next. She saved them with the default EncryptSensitiveWithUserKey option, so as a result, I couldn't modify the package due to the fact that I wasn't her and I wasn't on her machine. Her AD account has long since been deactivated, and her machine may be checking people out at a grocery store now for all I know.
I had to recreate the packages from scratch. Fortunately, the protection level prevents you from saving or building, not from looking at things in Design, so I was able to replicate what she did, but it was a long, tedious process that took the better part of a full day to complete.
My question is: We use SourceSafe to maintain our projects, so that's where the new SSIS project will be going. Given the following:
We each have our own PCs with working folders that sync to SourceSafe
We do not have the option of saving the project on the database server itself; we can only deploy the pacakges.
Software details: MS Visual Studio 2008, VSS 8.0, MS SQL Server 2008 R2
What would be the best security option to configure our project with? I immediately see that DontSaveSensitive would be the logical approach, but I don't know where the passwords would then be supplied from. I would think a config file, but I don't know how to set that up for an SSIS deployment.
Thanks!
Use VSS as the source control provider for Visual Studio and add the SSIS project just like any other source project. After installing VSS, go to Tools\Options\Source Control and set VSS as the provider.
I've been using this setup for several months. There's no difference, as far as I can tell, between any of my C# projects and my SSIS projects in relation to VSS.
In the 2008R2 version I was using SSIS logging to a sysssislog table in a defined database. 2012 brings now the concept of Integration Services Catalogs that have their own SSISDB db. Is it still necessary to use the logging to sysssisslog tables or is the information that ends there now somewhere in SSIS DB (what i would expect, since all the reporting for SSIS execution is based on this db as well).
The logging you are familiar with remains available to you with the 2012 release of SQL Server. That said, database logging no longer has to be explicitly defined in your package if you are using the Project Deployment model (SSISDB).
Out of the box, you'll get Basic logging level when you run a package. The other options are none, performance and verbsose. Read more about how to set these and other execution parameters via MVP Phil Brammer. Matt Masson of the actual SSIS team points out what events those levels correspond to in his post on What Events are Included in the SSIS Catalog Log Levels.
Finally, SSIS Reporting Pack is an open source project from MVP Jamie Thomson that provides different insight into the basic data being captured in the new integration services catalog.
My thoughts: necessary no. But if you already have a framework built out culling data from that log (we use it for an alerting system), you are supported to keep using it. If you run integration services packages from multiple servers, there is no functionality to combine the logging from all those disparate SSISDB catalogs to provide insight into your entire universe. You can get that if you all the packages log to a centralized server using the classic technique.