Our office uses SQL Server 2000, and by and large it has served our needs well. I'm looking into the possibility of upgrading to SQL 2005 or 2008 because of the new features not found in 2000 (when I saw Index usage statistics which told me which Indexes were the most used and which were never used, that blew me away).
I'm not too concerned with how to move the databases from 2000 to 2005 or 2008 - I can just take a full backup and then restore the backup to the new sql installation. What concerns me are the existing DTS packages. We have dozens of them that we rely on every day. How difficult/easy is it to convert the existing DTS packages to SSIS? What are some of the 'gotchas' that I have to look out for?
Doing a backup & restore to do a database server upgrade has many gotcha's. It's best to do an in-place upgrade (you might want to take a full backup of the server first) & let the MS installer do the work.
2005 supports 2000 DTS packages as Legacy so there's no need to learn SSIS & convert all your packages unless you have the time & there isn't too many.
As for the backup & restore upgrade, don't forget about:
logins, you'll need transfer these,
see KB246133 for more details
Any user and role permissions given
to system databases will need to be
re-created or transferred
linked server or remote servers will need to be re-setup
SQL Server Agent Jobs need to be exported-imported
Any Extended stored procedures need
to be re-created
Replication will need to be re-setup
Log shipping will need to be
re-setup & the standby server also
upgraded
There is a migration tool for DTS packages, but I wouldn't use it. SSIS is so much better than DTS that you should:
Learn about SSIS. Get comfortable with it. Rejoice at having actual control flow.
Take the time to understand and document your existing DTS packges while you've still got the environment they're written for.
Use that understanding to write equivalent SSIS packages.
This idea will make no sense to you if you don't know SSIS. For instance, I was just thinking about how I used to have to do loops in DTS - that whole business with finding and modifying the precedence constraint you wanted to loop back to.
SSIS actually has a For Loop container that can loop over a set of numbers, over the rows in a rowset, over files in a folder, etc. No hacks with precedence constraints and little hidden scripts.
DTS packages can still be run as usual in 2008 as long as Backwards Compatibility is installed. Here's the link to upgrading which uses a wizard.
I'd recommend a tool from Pragmatic Works called DTS xChange. It handles most of the migration and helps you get on with your job. You'll have other opportunities to learn SSIS on new stuff but you need to probably wrap up your project. It can be downloaded at http://www.pragmaticworks.com
Related
As a company we have grown and we are now moving a couple of SQL Server 2016 databases over to a new server. We have SSIS packages that run off the databases that we are moving from server 1 to server 2.
Is there a way to easily identify using SSMS which SSIS packages use the current server and databases we are moving? Some of the old SSIS packages don't have documentation so we are trying to avoid physically opening up all the SSIS packages. We would prefer to identify the SSIS packages that are impacted.
Thank you!
Here are my solutions on the top of my head. I'm not an expert by any means so don't be surprised if someone comes up with something better.
In SSMS, you can view the data sources being utilized by a
package by... Object Explorer > SQL Server Agent > Jobs > (Your Job) > Steps > Edit... > Data Sources (Tab)
Here, you can view the data sources of your package. This is
slightly faster than opening all your SSIS packages; but, it isn't a
great solution either.
Conversely, recognize that .dtsx files are simply plain text
files. You can scan keywords within all of them using a number of
different scripts (PowerShell, Python, SSIS package w/ a Script
Task, etc.)
What you can use depends on the tech stack that your organization
supports but I imagining Googling for such a program/script would
not be difficult.
If you are utilizing SQL Server configurations in your packages
and you consistently do so for every package, you can query the
[SSIS_Configurations].[dbo].[SSIS Configurations]
NOTE: Solution (1) and (2) do not take configurations into account.
Hopefully, some of these solutions are helpful to you. I would be interested in an efficient means to do this without delving into scripts as well.
I have developed 25 SSIS packages with SQL Server backend. Now my requirements have changed so that the SSIS package stays the same but the database which I use should shift from SQL server to Oracle Database,
How I can change the DB in my SSIS package from SQL Server to Oracle?
Well, I do not think, there is a way to do that. SSIS stands for SQL Server Integration Services for a reason :)
Oracle does have a similar framework called Golden Gate:
http://www.oracle.com/us/products/middleware/data-integration/goldengate/overview/index.html
I guess, you would have to start from scratch. But since you already have got the concepts - assuming your SSIS-Packages are working - to create your Data Integration Services, you will "only" have to get familiar with Golden Gate.
Are you kidding? This can be solved. All you have to do is edit the packages to use the source and destination as your oracle databases. The rest remains the same. It also depends on how the data is processed internally in packages.
If your packages are using any transformations in Data Flow and if the data flow is using any sql commands then you need to think of that too. Also, if your control flow is using any SQL task then also you need to think of that.
You can edit a single package and try converting it to Oracle. SSIS can treat Oracle data as well, this is not a limitation of SSIS.
I would need to migrate a SQL database from Sybase to MS SQL Server. Before doing the actual migration on the production server I first created an SSIS-package with SQL Server Management Studio's Import/Export Wizard for testing with other databases. The test migration was successful and I would now like to deploy my SSIS-package to the real servers.
However, it seems I cannot simply run the package in Management Studio choosing different data sources for it - it only runs on the same databases for which it was created. Now, it can be edited in something called SQL Server Business Intelligence Development Studio (or BIDS for short)(I am using the SQL Server 2008 version), but going through every data flow task changing the destination source manually for each of the ~ 150 tables I am moving is ineffective and also introduces a possibility for error.
I there a way to quickly change what data source is to be used for ALL destination sources in ALL the flow tasks of an SSIS-package? If not, what simple method is there for testing migration with test databases first and simply changing the data sources when deploying?
I am using ODBC data sources, but for some the package shows OLE-sources in BIDS instead.
I hope I was clear enough. If you have additional questions, please ask! Thank you!
I would use a variable for the ConnectionString property of the connection manager. A package level configuration can be very useful for accomplishing this task. Several ways to do this. I prefer to use a table in SQL Server that holds all the configurations for all packages. This can be especially effective if you have multiple packages and need to dynamically change a set of connection managers across those multiple packages.
The basic steps are:
Opposite click on your SSIS design surface and select "Package Configurations..."
Create a package level configuration of Configuration Type "SQL Server"
Store your connection in a Configuration table in SQL Server
Alter your Connection Manager to use a variable for the ConnectionString Property
Populate that variable from the Configuration table via your package level configuration
When it comes time to switch from Test to Production, simply update the connection string in your configuration table
These screenshots can help...
This is part of a larger package management framework that I implemented using this book:
Microsoft SQL Server 2008 Integration Services: Problem, Design, Solution
I highly recommend it. Should take less than a day to set it up. Book has step by step instructions.
This question and its associated answers also helpful.
My question is similar to Upgrading SQL Server 2000 to 2005 or 2008 - DTS to SSIS but I have a few restrictions that I'd like the community to consider.
We have two databases that currently run on SQL Server 2000 and we are being pushed to move off of this in favor of SQL server 2005 (We do not have 2008, it isn't an option).
We have approximately 150 DTS packages that will need to be converted. According to the stack overflow post above, we shouldn't use the conversion tool available. We also anticipate that any "backward compatibility" patches that Microsoft may have released for SQL Server 2005 won't be installed on our server.
Can anyone help me understand the learning curve involved in going from DTS to SSIS? For 150 packages, we've come up with a 900 hour estimate. Myself and a more senior developer will be working on this but we've never worked with SSIS. The 900 hours will be divided between us. As with any project, quality is the utmost importance. Is this estimate outrageous or low?
We will have access to other developers who have worked with SSIS, so we won't be completely alone in this. Also, I've learned that DTS package conversion is not an option; it is not supported by our DBA staff.
Some things we do with DTS include the following. This is by no means an exhaustive list:
Load XML, XLS, MDB, CSV, etc files in to SQL Server
Export XML, XLS, MDB, CSV, etc files from SQL Server
Run Workflow-type processing
The built in conversion tool will not
migrate transaction settings
migrate ActiveX scripts associated
with steps
Analysis Services task
complex Data Transformation tasks
custom tasks
Data Driven Query tasks
Data Mining Prediction Query task
Dynamic Properties tasks
Execute Packages tasks
Parallel Data Pump tasks
tasks that obtain their connection
information from Microsoft Data Link
(.udl) files
ActiveX Script code that accesses
the DTS object model through the
Parent property of the
GlobalVariables collection
There is a tool called dtsXchange available from Pragmatic works which does most of these conversions, but still it does not re-engineer the package.
Typically, a DTS package would use staging tables and most of the transformation / data manipulation would happen using SQL or Stored procedures. Automatic conversion would continue to use the same work flow.
The SSIS engine is quite powerful and has a lot of features to help do these tasks. Rewriting the packages in SSIS would be the way to go.
Having said that, considering that you do not have any experience with SSIS, I feel that 900 hours for 150 packages is a bit conservative. I think you should plan for 8 hours per package, but aim to do it in 6 hours.
I hope you are not using 64 bit. Neither XLS nor MDB files have a 64 bit data provider and you will be forced to run in 32 bit mode.
I have been working on a strong argument for rewriting against conversion. I will gladly share all the documentation I have with you. Email me at b.rengarajan#gmail.com
Raj
I have to move an entire database from a SQL Server 2008 machine to a SQL Server 2000 machine.
I created a backup using Management Studio 2008, copied it to the hard drive of the 2000 box, and from withing Management Studio 2008, I choose Restore Database to the 2000 box.
I get an error message stating, "The media family on device ... is incorrectly formed. SQL Server cannot restore this media family".
If I use Enterprise Manager 2000 I get the same error.
Is there a way to move a whole database from the newer SQL server to the older?
The only thing I can think of is to recreate the whole structure and then copy data from a live database. So, create scripts that will create the tables, views, and sp's, and then create scripts to copy the data from the existing database.
As others already said there is no default way to do this. It’s just not supported. Here are more extensive details on how to do this properly and avoid any migration issues.
You need to generate scripts for structure and data and then execute these on SQL 2000 (like others already said) but there are couple things to take into account.
Generate scripts in SSMS
Make sure to check option for scripting data for SQL 2000 to avoid issues when trying to create something like geography type column on SQL 2000.
Make sure to review execution order of scripts to avoid dependency based errors
This is a great option for small to medium size databases and requires some knowledge of SQL Server (dependencies, differences between versions and such)
Third party tools
Idea is to use third party database comparison tools such as ApexSQL Diff or Data Diff
Good side is that these will take care of script execution and differences between versions
Not so good is the fact that you’ll need to pay for these after trial ends
I’ve used these two tools successfully but you can’t go wrong with any other tool on the market. Here is a list of other tools in this category.
you can't move backups from a newer version to an older, in that case you can script your database, execute it in the 2000 box, then you can use the standard data transfer to transfer any data you want
Provided you have a network connection between the machines use SSIS. Much easier and a lot less messing around.
You can use Script Generator for your database and then select in the properties form : General-> Script for server version : SQL Server 2000.
The script generator will show you things which not compatible with your server version.
I've heard you can only do it by generating the SQL statement dump from the DB administrator tool and re-running those queries on the target older database.
You can generate a script that will recreate all the objects and transfer all the data...as long as everything in the db is valid in SQL 2000. So no ROW_NUMBER(), no PARTITION, no CTEs, no datetime2, hierarchy or several other field types, no EXECUTE AS, and lots of other goodness. Basically, there's a pretty good chance it's not possible unless your db is pretty basic.
We got a similar situation. A very low-tech but handy solution is:
backup and truncate the tables in SQL 2000.
create a LINKED server in SQL 2008, pointing to SQL 2000
run a select query at sysobjects to generate a query script for insert into LINKED SERVER.table select * from table
execute query script.