As a company we have grown and we are now moving a couple of SQL Server 2016 databases over to a new server. We have SSIS packages that run off the databases that we are moving from server 1 to server 2.
Is there a way to easily identify using SSMS which SSIS packages use the current server and databases we are moving? Some of the old SSIS packages don't have documentation so we are trying to avoid physically opening up all the SSIS packages. We would prefer to identify the SSIS packages that are impacted.
Thank you!
Here are my solutions on the top of my head. I'm not an expert by any means so don't be surprised if someone comes up with something better.
In SSMS, you can view the data sources being utilized by a
package by... Object Explorer > SQL Server Agent > Jobs > (Your Job) > Steps > Edit... > Data Sources (Tab)
Here, you can view the data sources of your package. This is
slightly faster than opening all your SSIS packages; but, it isn't a
great solution either.
Conversely, recognize that .dtsx files are simply plain text
files. You can scan keywords within all of them using a number of
different scripts (PowerShell, Python, SSIS package w/ a Script
Task, etc.)
What you can use depends on the tech stack that your organization
supports but I imagining Googling for such a program/script would
not be difficult.
If you are utilizing SQL Server configurations in your packages
and you consistently do so for every package, you can query the
[SSIS_Configurations].[dbo].[SSIS Configurations]
NOTE: Solution (1) and (2) do not take configurations into account.
Hopefully, some of these solutions are helpful to you. I would be interested in an efficient means to do this without delving into scripts as well.
Related
I want to find when an SSIS 2008 package was deployed under MSDB in an instance of SQL Server. In the table dbo.sysssispackages, I can see package creation date but where can I find the last modified/deployed date of a package?
The date an SSIS package was deployed to the MSDB is not tracked so you do not have the ability to know when a package was deployed, who performed this feat, etc.
With SQL Server 2012+ and the project deployment model, the SSISDB supports the ability to track when a project was deployed and by whom.
The best answer I have for you is much the same as Tab has just posted except I tied mine to VerBuild, which is a monotonically increasing number that VS updates whenever you save a package.
If it's absolutely crucial that you have this information, you could look at modifying msdb.dbo.sp_ssis_putpackage. That's definitely off the reservation so buyer beware, etc but depending on your appetite for risk, you could either extend dbo.sysssispackages by adding your custom columns there or create a new table dbo.sysssispackages_extended and there record who did what and when.
This information is not stored, and is not available for retrieval from SQL Server.
The best way to make this information available that I have found is to use the Version-related fields (VersionMajor, VersionMinor, VersionComments) in the SSIS package. Combined with use of source control, you can see which version of your package is currently live on your server, and find that version in source control to find which version of the code it is.
I have a huge number of SSIS 2012 packages from different SSIS projects, often the SSIS packages names in the different projects are identical.
I use SSIS logging from all the packages to one single table in a log database. I would like to keep it that way, to be sure only to have one database to search for all the SSIS logging.
When I am using the SSIS logging in the packages, is it possible to identify the project name also ? so I can identify what SSIS packages and project that are affected?
BR
Carsten
It sounds like you are using package deployment, in which case there is no ssis catalog and no project (per se), hence, no further information about your package. Logging that writes to sysssislog was designed before the concept of project deployment so that's why that piece of information is missing. As well, the use of MSDB also predated project deployment so it has no information either.
So there's no simple solution. I would guess that you could convert to project deployment and take advantage of all of the built in logging and reporting there (which you already said you don't want). Or you could modify all the packages to log there package id and project name into an additional table.
I would need to migrate a SQL database from Sybase to MS SQL Server. Before doing the actual migration on the production server I first created an SSIS-package with SQL Server Management Studio's Import/Export Wizard for testing with other databases. The test migration was successful and I would now like to deploy my SSIS-package to the real servers.
However, it seems I cannot simply run the package in Management Studio choosing different data sources for it - it only runs on the same databases for which it was created. Now, it can be edited in something called SQL Server Business Intelligence Development Studio (or BIDS for short)(I am using the SQL Server 2008 version), but going through every data flow task changing the destination source manually for each of the ~ 150 tables I am moving is ineffective and also introduces a possibility for error.
I there a way to quickly change what data source is to be used for ALL destination sources in ALL the flow tasks of an SSIS-package? If not, what simple method is there for testing migration with test databases first and simply changing the data sources when deploying?
I am using ODBC data sources, but for some the package shows OLE-sources in BIDS instead.
I hope I was clear enough. If you have additional questions, please ask! Thank you!
I would use a variable for the ConnectionString property of the connection manager. A package level configuration can be very useful for accomplishing this task. Several ways to do this. I prefer to use a table in SQL Server that holds all the configurations for all packages. This can be especially effective if you have multiple packages and need to dynamically change a set of connection managers across those multiple packages.
The basic steps are:
Opposite click on your SSIS design surface and select "Package Configurations..."
Create a package level configuration of Configuration Type "SQL Server"
Store your connection in a Configuration table in SQL Server
Alter your Connection Manager to use a variable for the ConnectionString Property
Populate that variable from the Configuration table via your package level configuration
When it comes time to switch from Test to Production, simply update the connection string in your configuration table
These screenshots can help...
This is part of a larger package management framework that I implemented using this book:
Microsoft SQL Server 2008 Integration Services: Problem, Design, Solution
I highly recommend it. Should take less than a day to set it up. Book has step by step instructions.
This question and its associated answers also helpful.
I'm looking for the best approach (or a couple of good ones to choose from) for extracting from a Progress database (v10.2b). The eventual target will be SQL Server (v2008). I say "eventual target", because I don't necessarily have to connect directly to Progress from within SQL Server, i.e. I'm not averse to extracting from Progress to a text file, and then importing that into SQL Server.
My research on approaches came up with scenarios that don't match mine;
Migrating an entire Progress DB to SQL Server
Exporting entire tables from Progress to SQL Server
Using Progress-specific tools, something to which I do not have access
I am able to connect to Progress using ODBC, and have written some queries from within Visual Studio (v2010). I've also done a bit of custom programming against the Progress database, building a simple web interface to prove out a few things.
So, my requirement is to use ODBC, and build a routine that runs a specific query on a daily basis daily. The results of this query will then be imported into a SQL Server database. Thanks in advance for your help.
Update
After some additional research, I did find that a Linked Server is what I'm looking for. Some notes for others working with SQL Server Express;
If it's SQL Server Express that you are working with, you may not see a program on your desktop or in the Start Menu for DTS. I found DTSWizard.exe nested in my SQL Server Program Files (for me, C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn), and was able to simply create a shortcut.
Also, because I'm using the SQL Express version of SQL Server, I wasn't able to save the Package I'd created. So, after creating the Package and running it once, I simply re-ran the package, and saved off my SQL for use in teh future.
Bit of a late answer, but in case anyone else was looking to do this...
You can use linked server, but you will find that the performance won't be as good as directly connecting via the ODBC drivers, also the translation of the data types may mean that you cannot access some tables. The linked server might be handy though for exploring the data.
If you use SSIS with the ODBC drivers (you will have to use ADO.NET data sources) then this will perform the most efficiently, and as well you should get more accurate data types (remember that the data types within progress can change dynamically).
If you have to extract a lot of tables, I would look at BIML to help you achieve this. BIML (Business Intelligence Markup Language) can help you create dynamically many SSIS packages on the fly which can be called from a master package. This master package can then be scheduled or run ad-hoc and so can any of the child packages as needed.
Can you connect to the Progress DB using OLE? If so, you could use SQL Server Linked Server to bypass the need for extracting to a file which would then be loaded into SQL Server. Alternately, you could extract to Excel and then import from Excel to SQL Server.
Our office uses SQL Server 2000, and by and large it has served our needs well. I'm looking into the possibility of upgrading to SQL 2005 or 2008 because of the new features not found in 2000 (when I saw Index usage statistics which told me which Indexes were the most used and which were never used, that blew me away).
I'm not too concerned with how to move the databases from 2000 to 2005 or 2008 - I can just take a full backup and then restore the backup to the new sql installation. What concerns me are the existing DTS packages. We have dozens of them that we rely on every day. How difficult/easy is it to convert the existing DTS packages to SSIS? What are some of the 'gotchas' that I have to look out for?
Doing a backup & restore to do a database server upgrade has many gotcha's. It's best to do an in-place upgrade (you might want to take a full backup of the server first) & let the MS installer do the work.
2005 supports 2000 DTS packages as Legacy so there's no need to learn SSIS & convert all your packages unless you have the time & there isn't too many.
As for the backup & restore upgrade, don't forget about:
logins, you'll need transfer these,
see KB246133 for more details
Any user and role permissions given
to system databases will need to be
re-created or transferred
linked server or remote servers will need to be re-setup
SQL Server Agent Jobs need to be exported-imported
Any Extended stored procedures need
to be re-created
Replication will need to be re-setup
Log shipping will need to be
re-setup & the standby server also
upgraded
There is a migration tool for DTS packages, but I wouldn't use it. SSIS is so much better than DTS that you should:
Learn about SSIS. Get comfortable with it. Rejoice at having actual control flow.
Take the time to understand and document your existing DTS packges while you've still got the environment they're written for.
Use that understanding to write equivalent SSIS packages.
This idea will make no sense to you if you don't know SSIS. For instance, I was just thinking about how I used to have to do loops in DTS - that whole business with finding and modifying the precedence constraint you wanted to loop back to.
SSIS actually has a For Loop container that can loop over a set of numbers, over the rows in a rowset, over files in a folder, etc. No hacks with precedence constraints and little hidden scripts.
DTS packages can still be run as usual in 2008 as long as Backwards Compatibility is installed. Here's the link to upgrading which uses a wizard.
I'd recommend a tool from Pragmatic Works called DTS xChange. It handles most of the migration and helps you get on with your job. You'll have other opportunities to learn SSIS on new stuff but you need to probably wrap up your project. It can be downloaded at http://www.pragmaticworks.com