I have automated UI tests with Selenium c#. It is a dotnet project.
SpecFlow to write the test cases and Nunit to run them.
We are using Azure Devops for storing the code.
Octopus is used for build and deployments.
I want to automate my tests with nightly builds. Which is a better options for the pipeline? Is it Azure DevOps or Octopus.
Please suggest which is the best way?, more steps and the process to start with it.
I want to automate my tests with nightly builds. Which is a better
options for the pipeline? Is it Azure DevOps or Octopus.
For this issue, you can configure scheduled trigger in azure devops pipeline.
Select the days and times when you want to run the build.
If your repository is Azure Repos Git, GitHub, or Other Git, then you can also specify branches to include and exclude.
Related
I have a database project with some important SQL-scripts. I want to deploy and run the scripts as part of the build pipeline, because they have a tendency to fail because of human error.
I've tried to find a way to add "deployment group job" to my build pipeline, but I'm not seeing how this can be done.
I'm using Azure Devops with Git. My servers are all on-premise.
I have a build and release pipeline which is working fine.
If you want to deploy and run the SQL scripts against all your servers. You need to create a deployment group and Register all your servers in it. Then add "deployment group job" in your release pipeline.
Select the deployment group you created in the configuration page. You can then add script tasks(eg. powershell task) to execute your SQL scripts (The SQL scripts should be included in the build artifacts).
When you run the release pipeline, and the tasks of deployment job will run the sql scripts on all your servers registered in the deployment group.
If you just want to run the sql scripts on a single database server in your build pipeline, you can simply create a self-hosted agent on the database server machine. And run your build pipeline on the self-hosted agent. The scripts in your pipeline will be able to access to your local database server, since they are on the same machine.
Hope above helps!
Deployment group job is not supported in build pipeline, you need to do it in release pipeline.
Database - SQL Server
Version control - GIT/Bit bucket
Automation - Jenkins pipeline.
Question/task - I need to build SQL database project & deploy thru jenkins pipeline.
Currently we manually build & publish the database but I have scripts which I can use to build(dacpac) & publish to the database but problem scripts only work in VS(visual studio) command prompt.
Build SQL database project thru Jenkins - Is it possible, if so how?
Publish SQL database project thru Jenkins - Is it possible, if so how?
Please help me understand the process involved.
I used the DacFx API provided by Microsoft and created an API service that I integrated with jerkins. In the middle of the pipeline, I passed the SqlProject location, my target database information, and dacpac destination. The API uses Publish Method to Deploy and generate the delta and deploy it to the target database.
See the link below.
https://www.nuget.org/packages/Microsoft.SqlServer.DACFx
One of our customers uses Visual Studio Online ( http://www.visualstudio.com/en-us/products/what-is-visual-studio-online-vs.aspx ) which is based on capabilities of Team Foundation Server (TFS)
We were researching how to do automated Builds and automated Unit Tests using the Visual Studio Online account.
Using Visual Studio 2012 IDE, I was able to setup a build in the Hosted Build Service. However, my Unit Tests need a Microsoft SQL Server Database to run properly, which I have installed on my development workstation.
In order to run my tests I need to have my database running in a SQL instance on the build agent.
Is it possible to install SQL server on the Hosted Build Agent, and if so, how?
It is not possible to add non-standard capabilities to a hosted build agent. You are assigned a clean pre-built VM with all of the standard pre-requisite for compiling binaries and running unit tests. As integration tests, what you are describing above, require an instance of your application you can't run them there.
Here are the options that you have in order of recommendation:
1 - Re-write your tests as Unit and not Integration
The tests that you are describing above do not meet the standard definition of "Unit Test". You can use mocking, stubs, and other testing techniques to decouple your tests from the database so that they are only testing a single unit of functionality rather than the integration between your code and the database. Having sets of tests for each will better help you isolate the route cause of the issue and more quickly fix it at less cost to your customer. It will also aid in the reduction of bugs and other issues that can be introduced over time.
Large complicated integration tests should be kept to a minimum as it increases the amount of time required to support them and maximises the cost to your customer.
One obviously needs integration tests at some point and option #2 is the best for running integrations.
2 - Use Release Management to deploy and test
You can setup a VM to host a working version of your application and use Release Management to deploy the instance then execute your Integration Tests there. If you use the VSO hosted Release Management server you are limited to Azure VM's as targets. If you deploy your own RM server then you need to manage that as well. I would and do use the hosted RM.
The bit we care about is the DevOps bit on the right. Any time you need an instance of your application it should be deployed correctly separately from your build server.
http://nakedalm.com/create-release-management-pipeline-professional-developers/
Once you have your application deployed it is fairly simple to get the integration tests you want executing against the instance.
http://blogs.msdn.com/b/visualstudioalm/archive/2014/11/11/deploying-and-testing-web-applications-using-release-management.aspx
If you download the example from the link above it has a powershell for executing the tests in the environment.
3 - Setup your own agent and controller
You can create your own build controller & agent attached to the VSO account and build the app yourself. Most folks create an Azure VM with all of the bits they need and run there. This is your server that you will need to manage and pay for. You can also use a local server.
Note: This is the wrong approach as an instance of your application should not be available on a build server. A build server is for compiling your binaries and running actual Unit Tests, not integration or UI tests.
Thanks for all your help.
I contacted #tamasf who is one of the developers for the Effort Testing Tool for Entity Framework, and he told me that Effort Testing Tool only needs to know about he EDMX file in order mock Entity Framework ( which means Database won't need to be up and running, therefore, allowing us to follow proper Unit testing practices ) For more information, please read the following post: How would I configure Effort Testing Tool to mock Entity Framework's DbContext withOut the actual SQL Server Database up and running?
We have recently migrated to using the Visual Studio database projects. What we want to do is for the database to deploy when the TFS build server builds.
This is relatively simple and we have this working for a single database, however, what we need is for it to deploy to multiple database as we have a SaaS product with multiple databases. So for example, when we do a QA build, all the different databases with various configurations on the QA DB server should be updated.
Is there a 'proper' way to do this?
Our current plan is to take the deployment .sql script that will be generated from the database configured for deployment, then create a custom build task which runs this script against the rest of the databases.
I don't think there is a standard way of doing this, so we created a custom build task that iterates over the databases we want to deploy to executing the deployment script generated by the standard database project's deploy against each DB.
In Visual Studio 2010 there is a nice feature of the database project that allows you to deploy to a database as well as set up various environments based on your configuration (Build deploy etc)
I Would like to integrate this into our automated build environment.
Firstly by having the deploy script run after a successful build on my local machine. (So I can go straight into running the unit tests)
Then by Having the deploy script run after the successful build on our build server. so that any schema changes required for the unit and integration tests will run.
How can I adjust the MSBuild or similar to run these in deploy mode.
Use the MSBuild task to call the "dbproj" file. Pass "DBDeploy" as the target and the build configuration as a property, e.g.:
<MSBuild Projects="MyDb.dbproj"
Targets="DBDeploy"
Properties="Configuration=$(Configuration)" />
On a build server, you might also need to supply properties like TargetConnectionString and TargetDatabase.