I'm trying to desgin a program, that 100 people need to input infornamtion of 300 projects in Relational database(SQL SERVER), and then run SSIS pack to ETL and return calculation results from SSAS. Is it possible that I can pass varible of projet name individually to SSIS pack, and run SSIS pack on demand(not sql server agent scheduled way), so that I can get the SSAS results immediately for certain project? Is there any reference I can learn?
Thanks.
What SQL Server version do you have? If 2012 and newer you can use SSIS Project Deployment Model and Parameters and run packages using stored procedures = on demand
Here you have information about run packages using stored procedures
Here you have information about ssis parameters
Related
As a company we have grown and we are now moving a couple of SQL Server 2016 databases over to a new server. We have SSIS packages that run off the databases that we are moving from server 1 to server 2.
Is there a way to easily identify using SSMS which SSIS packages use the current server and databases we are moving? Some of the old SSIS packages don't have documentation so we are trying to avoid physically opening up all the SSIS packages. We would prefer to identify the SSIS packages that are impacted.
Thank you!
Here are my solutions on the top of my head. I'm not an expert by any means so don't be surprised if someone comes up with something better.
In SSMS, you can view the data sources being utilized by a
package by... Object Explorer > SQL Server Agent > Jobs > (Your Job) > Steps > Edit... > Data Sources (Tab)
Here, you can view the data sources of your package. This is
slightly faster than opening all your SSIS packages; but, it isn't a
great solution either.
Conversely, recognize that .dtsx files are simply plain text
files. You can scan keywords within all of them using a number of
different scripts (PowerShell, Python, SSIS package w/ a Script
Task, etc.)
What you can use depends on the tech stack that your organization
supports but I imagining Googling for such a program/script would
not be difficult.
If you are utilizing SQL Server configurations in your packages
and you consistently do so for every package, you can query the
[SSIS_Configurations].[dbo].[SSIS Configurations]
NOTE: Solution (1) and (2) do not take configurations into account.
Hopefully, some of these solutions are helpful to you. I would be interested in an efficient means to do this without delving into scripts as well.
Using Microsoft products, I have a collection of SQL scripts and a VS project that I use to refresh a DW on a weekly basis. I know the process can be automated, but the documentation for VS is so vast I don't know where to start. The HIGH LEVEL process is outlined below:
Open SSMS and MANUALLY run scripts to truncate tables and drop indexes
Open VS project and in dev mode MANUALLY press START to extract the data from application to the truncated tables in the DW
Open SSMS and MANUALLY run transformation scripts to create analysis cubes end users can access
I am trying to get to a point where I can just schedule this process to run every X period so I don't have to press any buttons.
From what you described it seems like SSIS is something that can cover the tasks you listed. SSDT will need to be installed to use this. Since you're using scripts I'm assuming you have SQL script files saved that you execute. These can definitely be run in SSIS using an Execute SQL Task, with a file connection as the SQL source. With a collection of scripts, I'd suggest looking into using a Foreach Loop to run these via an Execute SQL Task while iterating through the folder(s) that contain the scripts. As far as running the Visual Studio project in development mode, configurations in Visual Studio can used to accomplish this for SSIS. SSIS has both an Analysis Services Processing Task and Analysis Services Execute DDL Task, and sounds like you're looking for the latter. Both XMLA and TMSL commands can be executed from an SSAS Execute DDL Task. Below are some links to get you started. The Data Flow Task may help with what you're doing for your data extraction.
SSIS
SSDT
Execute SQL Task
Analysis Services Processing Task
Analysis Services Execute DDL Task
Configurations
Data Flow Task
Foreach Loop
DTEXEC
I think, you can use jobs. Descriptions is here
I have developed 25 SSIS packages with SQL Server backend. Now my requirements have changed so that the SSIS package stays the same but the database which I use should shift from SQL server to Oracle Database,
How I can change the DB in my SSIS package from SQL Server to Oracle?
Well, I do not think, there is a way to do that. SSIS stands for SQL Server Integration Services for a reason :)
Oracle does have a similar framework called Golden Gate:
http://www.oracle.com/us/products/middleware/data-integration/goldengate/overview/index.html
I guess, you would have to start from scratch. But since you already have got the concepts - assuming your SSIS-Packages are working - to create your Data Integration Services, you will "only" have to get familiar with Golden Gate.
Are you kidding? This can be solved. All you have to do is edit the packages to use the source and destination as your oracle databases. The rest remains the same. It also depends on how the data is processed internally in packages.
If your packages are using any transformations in Data Flow and if the data flow is using any sql commands then you need to think of that too. Also, if your control flow is using any SQL task then also you need to think of that.
You can edit a single package and try converting it to Oracle. SSIS can treat Oracle data as well, this is not a limitation of SSIS.
I deployed a Microsoft sql server 2008R2 with sample data.
Installed report builder 3.0, visual studio 2008.
These all installed on a windows server 2008 enterprise.
I successfully created Cube from ssas and used it for deploying report.
Now, my next task is make this process automatically.
I am thinking is that possible to tell the ssis to do the following
Creating a new cube base on the latest sql data and use current date for its name. This will do it daily.
Or
Copy the current cube and rename it. Use the same cube building information for a new cube.
Note that I also thinking is that possible to use linux bash script to rename the cube before ssis scheduled task kick-in. so this way can skip one step on above method.
The reason behind that is because of the high up want to have the cube created daily and so they can check them and keep a record for those data.
Any suggestion are welcome as I am a newbie on ssas cube thing and so my thought may not be on tge right track.
I think it can be done. But it is hard to meet your requirement with existing build-in component in SSIS, an alternative is to use script task to build .net code and create the XMLA and tsql script to process the logic you needs. Consider you're new to all that thing, I will not recommend you to do that. But it do works.
Hope it can help you!
Our office uses SQL Server 2000, and by and large it has served our needs well. I'm looking into the possibility of upgrading to SQL 2005 or 2008 because of the new features not found in 2000 (when I saw Index usage statistics which told me which Indexes were the most used and which were never used, that blew me away).
I'm not too concerned with how to move the databases from 2000 to 2005 or 2008 - I can just take a full backup and then restore the backup to the new sql installation. What concerns me are the existing DTS packages. We have dozens of them that we rely on every day. How difficult/easy is it to convert the existing DTS packages to SSIS? What are some of the 'gotchas' that I have to look out for?
Doing a backup & restore to do a database server upgrade has many gotcha's. It's best to do an in-place upgrade (you might want to take a full backup of the server first) & let the MS installer do the work.
2005 supports 2000 DTS packages as Legacy so there's no need to learn SSIS & convert all your packages unless you have the time & there isn't too many.
As for the backup & restore upgrade, don't forget about:
logins, you'll need transfer these,
see KB246133 for more details
Any user and role permissions given
to system databases will need to be
re-created or transferred
linked server or remote servers will need to be re-setup
SQL Server Agent Jobs need to be exported-imported
Any Extended stored procedures need
to be re-created
Replication will need to be re-setup
Log shipping will need to be
re-setup & the standby server also
upgraded
There is a migration tool for DTS packages, but I wouldn't use it. SSIS is so much better than DTS that you should:
Learn about SSIS. Get comfortable with it. Rejoice at having actual control flow.
Take the time to understand and document your existing DTS packges while you've still got the environment they're written for.
Use that understanding to write equivalent SSIS packages.
This idea will make no sense to you if you don't know SSIS. For instance, I was just thinking about how I used to have to do loops in DTS - that whole business with finding and modifying the precedence constraint you wanted to loop back to.
SSIS actually has a For Loop container that can loop over a set of numbers, over the rows in a rowset, over files in a folder, etc. No hacks with precedence constraints and little hidden scripts.
DTS packages can still be run as usual in 2008 as long as Backwards Compatibility is installed. Here's the link to upgrading which uses a wizard.
I'd recommend a tool from Pragmatic Works called DTS xChange. It handles most of the migration and helps you get on with your job. You'll have other opportunities to learn SSIS on new stuff but you need to probably wrap up your project. It can be downloaded at http://www.pragmaticworks.com