I'm sorry if this appears elsewhere, it may be that my Google-Fu is failing me.
I'm in the process of reading Roy Osherove's The Art of Unit Testing and am thrilled to be able to start doing some Test-First work on my current project.
Roy's book uses NUnit as the basis for it's training. I find myself in the position of needing to use MSTEst in VS 2008 Pro along with the Silverlight Unit Test Framework to do our testing.
Can anyone point me to a good tutorials and references entries for using both of these tools?
Cheers,
Steve
AFAIK, the test class attributes for MSTest are identical to that of NUnit. You should be able to use Osherove's book as a guide without too much impedance mismatch.
Related
I have some question regarding the NUnit test project and very new with NUnit.
For every wpf project there should be a test project or a single test project for a solution?
The test project will be class library project?
Thanks in advance. I am waiting for your quick response
Your first question is addressed here already: Which is better? Unit-test project per solution or per project? In a nutshell, there's no hard and fast rule, just be consistent and when possible try to mimic how you expect your libraries to be used.
For your second question, yes, it should be a library. It won't have anything other unit tests -- which are just classes with methods -- so there's no reason to make it an application or any other type of project.
I'm currently using the Silverlight Unit Test Framework, but I'd prefer to run tests directly in VS2010. I'm curious to know what approaches and tools everyone else uses.
I'm using Silverlight 4 with Prism and the MVVM pattern, and I'm specifically interested in integrated Silverlight unit test support in VS 2010 that I can use with my ViewModel unit tests. I'm using dependency injection with Unity, and I am writing unit tests by mocking the calls in my WCF layer using Moq for Silverlight. I am not even looking at integration tests at the moment, but even in a simple unit test which tests a single ViewModel command, the service request to my mocked service layer can take around 50 milliseconds. Therefore support for asynchronous tests is important to me.
The issue I'm raising here is not related to View testing, which I have handled with some success in the past using System.Windows.Automation.Peers, and - although I have not used it yet - could possibly now handle more easily with the support in VS 2010 Feature Pack 2 (which appears to be targeted at automation / playback of UI tests from what I gather).
I should mention that my findings from the products I've looked at and used so far are as follows:
Silverlight Unit Test Framework - I currently use this, and it's great as far as it goes, but its limitations are (a) it is not integrated with Visual Studio; and (b) if you don't want to run all tests, you are limited to the crude tag expression filter.
StatLight - very nice. I currently use this, and have used it since v0.9 when targeting Silverlight 3 on a previous project. Being a command-line tool, it can be integrated with a continuous integration server - which certainly handles another required scenario. But it is of no use directly in terms of Visual Studio integration during the development process.
Unit Test Result Viewer for Silverlight (Visual Studio extension on the Visual Studio gallery) - looks promising, but its limitations are (a) currently fails to find projects which are located in solution folders, rather than under the solution root; and (b) runs all tests in a given assembly (via StatLight), with no apparent ability to run a specific test, or a selection of tests.
Einar Ingebrigtsen's Silverlight Unit Test Runner for ReSharper, which later became Odin - ahead of the game (it first appeared in 2008), but the limitation is that it appears that this project is no longer maintained (most recent update is Apr 2009).
AgUnit ReSharper plugin ( http://agunit.codeplex.com/ ) - looks excellent initially. After downloading the source code for it and building the latest (bug-fixed) version to work against ReSharper 5.1, I was very encouraged. But unfortunately it does not handle asynchronous tests. This is a design limitation with the threading, so it does not matter whether you try to use the asynchronous support that is built into Silverlight Unit Test Framework (Microsoft.Silverliht.Testing.SilverlightTest base class), or whether you're using AutoResetEvent or anything else. This has been noted by the coordinator on the project's discussion forum on CodePlex. This is a massive limitation.
TestDriven.NET 3.0 - appears to have support for Silverlight 4.0 tests at first glance, but the limitation (I suspect) is Silverlight 4 "assembly portability" (i.e. the 5 dependent assemblies that are portable between SL4 and .NET 4). Certainly, when I tried using it with a simple POC, it crashed my instance of VS 2010.
Perhaps I've missed something here - I wonder if anyone in the community has any better ideas for Silverlight unit testing?
I use the Silverlight Unit test framework, AgUnit, and RX with a mock IScheduler provider to make my unit tests single threaded :)
UPDATE:
I'd already settled on using StatLight for my continuous integration server, but I was looking for a solution to allow me to run asynchronous Silverlight unit tests directly in VS2010 during development.
Inspired by Rob Fonseca-Ensor's suggestion to use Rx (see his separate answer on this page), I took another look at the issue. That led me to find a non-Rx solution to the problem of using AgUnit to run async Silverlight unit tests.
The solution I think I will use - at least for now - is the combination of:
Silverlight Unit Test Framework
ReSharper 5.1
AgUnit ReSharper plugin
[Mock async pattern]
My ViewModels have bespoke service classes injected into them which provide abstractions of my (auto-generated) WCF service reference classes. To help provide access to the WCF service methods, my bespoke service classes also rely on another common class which wraps the async pattern. For my unit tests, I was already mocking my WCF service reference classes and my bespoke service classes with Moq - but I hadn't looked at mocking my async pattern wrapper class.
So I decided to mock my async pattern wrapper class as well. The attraction of doing this was that I thought I might then be able to use Rx with a mock IScheduler (as Rob suggested in his answer) in all my mocks, while leaving my real classes free of any references to Rx (which is a requirement because for this project at the place I'm working, I need to keep Rx out of any code which gets deployed to a production environment). However, once I'd mocked the wrapper class, I realised that I didn't even need Rx, and that there was an even simpler solution - which I should have really looked at before. It's pretty trivial - all I really needed to do was mock the wrapper class and ensure that it called Invoke on the callback operations, rather than calling BeginInvoke. This ultimately prevents the callbacks which work fine when running under the Silverlight Unit Test Framework in a browser session from going into a black hole when the unit tests are run by AgUnit within VS2010. (Allowing the callbacks to go into black hole when using AgUnit would of course have prevented the individual tests from completing properly, and could have led to either timeouts or - worse still - false positives in the test results.)
In the absence of better advice from any subsequent answers to this question, this appears to be the easiest way for me to handle this scenario - allowing my async unit tests to run in AgUnit without requiring Rx in my production code.
For anyone else not facing company-imposed restrictions on deploying code that uses Rx (which is still a DevLabs project) to production, I think Rob's solution would definitely be worth looking at.
I'm the author of the AgUnit plugin.
AgUnit blocks asynchronous unit testing to speed up the test run. Under the covers it uses the same code as the Silverlight Unit Test Framework test runner, but this runner is very slow if you have to handle a large number of tests. I've seen differences between almost half an hour and a few minutes for a couple thousand tests.
That said, if you do want asynchronous testing, it's a very isolated part of AgUnit that does this. I'll try to create a build with it disabled. In the future this will be a configuration option or an attribute, I've not decided yet.
Feel free to contact me with any questions or requests.
I usually create a normal unit test project in VS.NET, add my Silverlight assembly in references and write unit test classes.
So everything works out of the box. What's a problem with this solution?
Have you read about the recently released Visual Studio 2010 Feature Pack 2. Currently it only accessible to MSDN subscribers, and it only works with certain editions of VS2010 (Premium, Ultimate and Test Pro).
Here is a quick overview: Link
I'm starting a new WPF/Silverlight custom control project and wanted to do unit testing on this one. However I'm a little confused about how to approach this.
This control would be based on the same codebase for both WPF and Silverlight with minor forking using #ifs and partial classes to tame the differences. I guess I could write unit tests for WPF part with NUnit, MSTest, xUnit, etc. and for the Silverlight part with Silverlight Unit Test Framework but this doesn't sound very elegant to me. I'd have to either ignore testing identical code on one of the platforms and test only differing parts (which is not very trustworthy) or rewrite tests for 2 frameworks (which is annoying). Is this the right way to go?
I'm wondering if there's some guidance, articles, tutorials out there on how to approach this task. Any pointers?
I am hardly an expert in WPF and Silverlight, but wouldn't it be possible to write the tests using the same techniques as the production code (#ifs and partial classes as you said)?
I tried to use xUnit first but it was kind of complicated to make same tests work in xUnit and SLUT (different attributes, syntax, etc.)
Then I did some basic experimenting with MSTest and from very simple test it looks like you can successfully use MSTest for the WPF part and same code with some #ifs, etc. and SLUT for the Silverlight part. So I'll try to go this route and see how it works in real-world situations.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
Looking for a tool to UI testing of a Windows (.NET WinForms) application. The idea is that the analysts will record the tests via some UI and it will be able to be played back over and over again.
Commercial tools are fine but bonus points for free ones.
Some others:
NUnitForms
Quail
I believe they're both free, and Quail looks really nice!
(I was going to just comment on an answer above, but ran out of room.)
We bought Ranorex, and I wouldn't do it again. Their licensing for 1.5 (what we're using) was unclear. Their written license was per user, their marketing said per machine. Still, enforcement was key-based, and not a big deal when a machine got wiped or a replacement tester came in.
For the 2.0 product, their licensing is now tied to the machine. Reimage a machine and you get to relicense it. I'm just not going to support that kind of hassle with my company's thousands of dollars, and we didn't upgrade.
For what they want for the product, a lot of the functionality could be easily written using the UI Automation Framework. Ranorex is a decent product, but I question its value for the money. We're moving a lot of our new tests to just coding to the UI Automation Framework, as we often end up modifying Ranorex-generated code anyway.
TestComplete.
You can definitely use capture-replay to capture and run the test script. But I would suggest that you must at least manually edit your scripts to make them
more readable
easier to maintain.
The good thing about TestComplete is that it is able to look into your form's properties, capture those properties so that you can refer to those properties by their name, not by just screen coordinates.
Like Tom E stated, do take caution while considering going down the record/playback path for test automation.
See Uncle Bob's article on Ruining your Test Automation Strategy.
The main problem is that the record/playback tools couple the tests to the GUI which makes them very fragile.
Uncle Bob's article does point out that some testing needs to occur on the GUI...but that he recommends stubbing out the business rule code.
Sorry I can't provide you with a specific UI test automation tool...but hopefully this caveat will help you make the best decision on how to employ the tool that you eventually use.
So far I have found:
Ranorex which looks really good.
Test Automation FX, also looks good and seems very well priced.
Microsoft's UI Automation Framework, which does not have the recorder but if I had to I could code one using this.
white which looks similar to the UI Automation Framework, but has an alpha quality recorder.
Comments please if you have used any of these.
At my company, we decided to go with http://www.sikuli.org/. We felt it was the perfect mix of cost (free), ease-of-use, functionality and extend-ability.
Sikuli is Python based (Jython, really) which is great and its open-source. There's a tiny bit of coding required, but it can be as easy as just calling functions. Their IDE makes it really easy to get started. It is not a record and playback tool. It functions based on computer vision algorithms - you give it screenshots of what to look for and it finds it on the screen and then performs the requested action (click, type, etc) on what it found. This is true independent testing since Sikuli knows nothing about the software its testing. It does not know about underlying APIs - it just does what a real human would do.
We have integrated Sikuli with Robot Framework, http://robotframework.org/, and have created lots of custom python code to build a robust testing platform. This may not be as easy as licensing a tool from a vendor but the time and talent investment in these two open source tools has been well worth it.
There are a bunch of similar questions on SO:
automate-interaction-with-a-gui-interface (Edit: no longer available)
automated-testing-of-windows-forms (Edit: no longer available)
automated-testing-of-gui
In my experience, there are a lot of good open source tools for the web, but not so much selection for open source thick client test automation tools. If you want good support with robust functionality, especially recording, you will need to look at the commercial tools (QTP, RFT, TestPartner, etc...)
You should have a look at http://opensourcetesting.org/functional.php A lot of tools are listed here and you should find something that meet your needs.
Visual Studio Team Test 2010 is coming with a tool for recording and playing back UI tests. You'll find some pointers at:
Introduction to Record and Playback Engine in VSTT 2010
Checkout Ranorex, commercial, quite expensive but powerful (not affiliated).
We were using Mercury TestDirector a few years ago and quite happy with it. (All the caveats as mentioned by others apply.)
Mercury was aquired by HP and the tools have been rebranded as HP QuickTest. Not sure how much has changed, but certainly worth a look.
I tried to include a link to the HP website, but the URL doesn't look too "stable". Not exactly a confidence builder ....
My proposal is QA Agent(http://qaagent.com/). It is free web based IDE for development of web automated tests. Basically you are using jQuery to develop your tests. You can run tests in your browser. It looks like a new projects, but I really like the idea to develop tests in the browser. You know how long it takes to set up your testing environment. With QA Agent it takes 10 seconds.
I want to some very basic IronRuby tests, such as checking the syntax on a regular expression, and if possible would like to be able to do it thorugh my browser. Does anyone know of a site that hosts an IronRuby in Silverlight demo?
1) Have you found/already tried this?
http://silverlight.net/samples/sl2/dlrconsole/index.html
Note, this is an older build, so you might wish to follow my 2nd suggestion if you want something a bit fresher:
2) If you have built IronRuby recently from the latest source from GitHub
You should find a Tutorial (which runs in WPF or Silverlight) .
You'd find it here in the source tree
Edit: Also, I know Jimmy has posted recently about some additional work going into this tutorial, so I believe this is being actively worked on.
Edit 2:
3) Perhaps will or will not suffice for your needs, but there is also Gestalt for running IronRuby/IronPython in the browser. This is a recent interesting development from mix online labs.
John Lam, the original iron Ruby guy, has a tutorial for doing just what you ask at:
http://www.iunknown.com/2008/03/dynamic-silverl.html