I have searched and have not found an answer for my following question:
I am running an SSIS package created from the Export data task for a database, in SSMS jobs. I want to add month and year to the file name when it is created but don't have access to Visual Studio. How would I do this in the Job Step Properties? Or would I need to do it on the SSIS package creation?
I could create a .bat file that could update the file name after the completion of the job but I would rather have it all done in one go.
Thanks.
SQL 2008 R2
--NO Visual Studio--
There is no way to do this using the Export Data Wizard. You would need to edit the SSIS package to do this.
Technically you could do this without Visual Studio, since an SSIS package is just a text file, but you would really have to know what you are doing. I wouldn't want to try it.
You're allowed to run Powershell scripts as job steps. Roughly the same as your .BAT file idea, but you could at least have it run in-line, manage it within the Job Agent, etc.
Related
Using Microsoft products, I have a collection of SQL scripts and a VS project that I use to refresh a DW on a weekly basis. I know the process can be automated, but the documentation for VS is so vast I don't know where to start. The HIGH LEVEL process is outlined below:
Open SSMS and MANUALLY run scripts to truncate tables and drop indexes
Open VS project and in dev mode MANUALLY press START to extract the data from application to the truncated tables in the DW
Open SSMS and MANUALLY run transformation scripts to create analysis cubes end users can access
I am trying to get to a point where I can just schedule this process to run every X period so I don't have to press any buttons.
From what you described it seems like SSIS is something that can cover the tasks you listed. SSDT will need to be installed to use this. Since you're using scripts I'm assuming you have SQL script files saved that you execute. These can definitely be run in SSIS using an Execute SQL Task, with a file connection as the SQL source. With a collection of scripts, I'd suggest looking into using a Foreach Loop to run these via an Execute SQL Task while iterating through the folder(s) that contain the scripts. As far as running the Visual Studio project in development mode, configurations in Visual Studio can used to accomplish this for SSIS. SSIS has both an Analysis Services Processing Task and Analysis Services Execute DDL Task, and sounds like you're looking for the latter. Both XMLA and TMSL commands can be executed from an SSAS Execute DDL Task. Below are some links to get you started. The Data Flow Task may help with what you're doing for your data extraction.
SSIS
SSDT
Execute SQL Task
Analysis Services Processing Task
Analysis Services Execute DDL Task
Configurations
Data Flow Task
Foreach Loop
DTEXEC
I think, you can use jobs. Descriptions is here
I'm trying to desgin a program, that 100 people need to input infornamtion of 300 projects in Relational database(SQL SERVER), and then run SSIS pack to ETL and return calculation results from SSAS. Is it possible that I can pass varible of projet name individually to SSIS pack, and run SSIS pack on demand(not sql server agent scheduled way), so that I can get the SSAS results immediately for certain project? Is there any reference I can learn?
Thanks.
What SQL Server version do you have? If 2012 and newer you can use SSIS Project Deployment Model and Parameters and run packages using stored procedures = on demand
Here you have information about run packages using stored procedures
Here you have information about ssis parameters
I deployed a Microsoft sql server 2008R2 with sample data.
Installed report builder 3.0, visual studio 2008.
These all installed on a windows server 2008 enterprise.
I successfully created Cube from ssas and used it for deploying report.
Now, my next task is make this process automatically.
I am thinking is that possible to tell the ssis to do the following
Creating a new cube base on the latest sql data and use current date for its name. This will do it daily.
Or
Copy the current cube and rename it. Use the same cube building information for a new cube.
Note that I also thinking is that possible to use linux bash script to rename the cube before ssis scheduled task kick-in. so this way can skip one step on above method.
The reason behind that is because of the high up want to have the cube created daily and so they can check them and keep a record for those data.
Any suggestion are welcome as I am a newbie on ssas cube thing and so my thought may not be on tge right track.
I think it can be done. But it is hard to meet your requirement with existing build-in component in SSIS, an alternative is to use script task to build .net code and create the XMLA and tsql script to process the logic you needs. Consider you're new to all that thing, I will not recommend you to do that. But it do works.
Hope it can help you!
I am trying to find a better way to test our SSIS application, which is used to load data from a file into SQL Server and validate the file data.
I have created a SQL script which can be run to insert 'bad' data into the database tables, and ensure that our validations are performing correctly.
The SQL script:
- loads 'bad' data
- executes the SSIS validations
- ensures the errors in the data were detected
- Outputs a PASS or FAIL
- Deletes the TEST data if passed
Is there anyway I can get this script to be run automatically somehow, for example after someone checks in some code? Should I add it as a stored proc?
I had a look at the Default template Build definition but I couldn't see how to run this SQL script.
The wrong way to do this is using a build server. There are lots of parts to a continuous integration and a build server is really designed for compilation and validation that does not require an instance of your environment. There are two good practices that you can adopt:
Create a test harness that allows you to load a singe package from your SSIS scripts and test the inputs and outputs. Essentially unit tests. I did this for a customer a few years ago and it worked well.
Use a release management tool to push out your packages, the data, and everything else that you need. Release Management for Visual Studio can do this easily.
A rule of thumb that I always go buy is if I do not need an instance of anything but in memory objects then all I need is a build server. However if I need an instance of my app then I want to add release management tools to my continuous integration strategy.
It doesn't appear that you can do it as part of a checkin based on my 2 minutes of searching.
How to modify the default Check-in Action in TFS?
http://msdn.microsoft.com/en-us/library/ms243849(VS.80).aspx
Can I modify the default Check-in Action in TFS 2012? (pretty picture)
http://www.codeproject.com/Tips/562994/Perform-a-custom-action-for-Check-in-event-in-Micr
Instead, you'll need to set up a build server to handle the action. Out of the box, MSBuild doesn't have the ability to run a SQL Script so you'll need to snag the MSBuild Extension Pack and leverage SqlExecute
http://msdn.microsoft.com/en-us/library/ms181712.aspx
http://msdn.microsoft.com/en-us/library/ms181710(v=vs.90).aspx
http://msdn.microsoft.com/en-us/library/cc668755(v=vs.90).aspx
http://www.msbuildextensionpack.com/help/4.0.5.0/html/0d864b98-649a-5454-76ea-bd3069fde8bd.htm
You can use SQL Server Agent to embed for example your SQL code and you can than run this job from your application.
I am trying to automate our build process. Until now I was able to create the code packages using CruiseControl.net, nAnt and MSBuild.
But I am completely stuck when it comes to database comparing. I want to compare the development database to our QA database and create a script with the changes.
I currently use the VS2010 database project and I could compare them manually. Is there any way I can make this comparison automatic?
If I deploy my project, I get an .sql file generated using the dev database, but it is the whole database. I would like to have the differences between dev and qa only.
Is there any way to perform this approach? Any tool that allow me to compare the schema and run it from msbuild or the command line.
Thanks!
I've had a lot of luck with RedGate's SQLCompare product, and it comes with an SDK.
http://www.red-gate.com/supportcenter/Content?c=knowledgebase%5CSQL_Comparison_SDK%5CKB200801000220.htm&p=SQL%20Comparison%20SDK
You can use xSQL Software's SQL Schema Compare command line utility to automate the process with very little effort. And, the tool is free for SQL Server Express.