What are your opinions and experiences regarding using TDD when developing an user interface?
I have been pondering about this question for some time now and just can't reach a final decision. We are about to start a Silverlight project, and I checked out the Microsoft Silverlight Unit Test Framework with TDD in mind, but I am not sure how to apply the approach to UI-development in general - or to Silverlight in particular.
EDIT:
The question is about if it is practical thing to use TDD for UI-development, not about how to do separation of concerns.
Trying to test the exact placement of UI components is pointless. First because layout is subjective and should be "tested" by humans. Second, because as the UI changes you'll be constantly rewriting your tests.
Similarly, don't test the GUI components themselves, unless you're writing new components. Trust the framework to do its job.
Instead, you should be testing the behavior that underlies those components: the controllers and models that make up your application. Using TDD in this case drives you toward a separation of concerns, so that your model is truly a data management object and your controller is truly a behavior object, and neither of them are tightly coupled to the UI.
I look at TDD from a UI perspective more from the bare acceptance criteria for the UI to pass. In some circles, this is being labeled as ATDD or Acceptance Test Driven Development.
Biggest over-engineering I have found in using TDD for UIs is when I got all excited about using automated tests to test look and feel issues. My advice: don't! Focus on testing the behavior: this click produces these events, this data is available or displayed (but not how it's displayed). Look and Feel really is the domain of your independent testing team.
The key is to focus your energy on "High Value Add" activities. Automated style tests are more of a debt (keeping them up to date) than a value add.
If you separate you logic from the actual GUI code, you can easily use TDD to build the logic and it will be a lot easier to build another interface on top of your logic as well, if you ever need that.
I can't speak to Microsoft Silverlight, but I never use TDD for any kind of layout issues, just is not worth the time. What works well is using Unit Testing for checking any kind of wiring, validation and UI logic that you implemented. Most systems provide you with programmatic access to the actions the user takes, you can use these to assert that your expectations are correctly met. E.g. calling the click() method on a button should execute what ever code you intended. Selecting an item in a list view changes all the UI elements content to this items properties ...
Based on your edit, here's a little more detail about how we do it on my current team. I'm doing Java with GWT, so the application to Silverlight might be a bit off.
Requirement or bug comes in to the developer. If there is a UI change (L&F) we do a quick mock up of the UI change and send it over to the product owner for approval. While we are waiting on that, we start the TDD process.
We start with at least on of either a Web test (using Selenium to drive user clicks in a browser), or a "headless" functional test using Concordion, FiT or something like it. Once that's done and failing, we have a high level vision of where to attack the underlying services in order to make the system work right.
Next step is to dig down and write some failing unit and integration tests (I think of unit tests as stand-alone, no dependencies, no data, etc. Integration tests are fully wired tests that read/write to the database, etc.)
Then, I make it work from bottom up. Sounds like your TDD background will let you extrapolate the benefits here. Refactor on the way up as well....
I think this blog post by Ayende Rahien answers my question nicely using a pragmatic and sound approach. Here are a few quotes from the post:
Testing UI, for example, is a common
place where it is just not worth the
time and effort.
...
Code quality, flexibility and the
ability to change are other things
that are often attributed to tests.
They certainly help, but they are by
no mean the only (or even the best)
way to approach that.
Tests should only be used when they add value to the project, without becoming the primary focus. I am finally quite certain that using test-driven development for the UI can quickly become the source of much work that is simply not worth it.
Note that it seems the post is mainly about testing AFTER things have been built, not BEFORE (as in TDD) - but I think the following golden rule still applies: The most important things deserve the greatest effort, and less important things deserve less effort. Having a unit-tested UI is often not THAT important, and as Ayende writes, the benefit of using TDD as development model is probably not so great - especially when you think of that developing an UI is normally a top-down process.
GUIs by their very nature are difficult to test, so as Brian Rasmussen suggests keep the dialog box code separate from the the GUI code.
This is the Humble Dialog Box Pattern.
For example, you may have a dialog box where details (e.g. credit card number) are input and you need to verify them. For this case you would put the code that checks the credit card number with the Luhn algorithm into a separate object which you test. (The algorithm in question just tests if the number is plausible - it's designed for checking for transcription errors.)
At my workplace we use TDD and we actually unit test our UI code (for a web application) thanks to Apache Wicket's WicketTester but it's not testing that some descriptive label is before a text field or something like that, instead we test that hierarchy of the components is somewhat correct ("label is in the same subset as text field") and the components are what they're supposed to be ("this label really is a label").
Like others have already said, it's up to the real person to determine how those components are placed on the UI, not the programmer, especially since programmers have a tendency of doing all-in-one/obscure command line tools instead of UIs which are easy to use.
Test-Driven Development lends itself more to developing code than for developing user-interfaces. There are a few different ways in which TDD is performed, but the preferred way of true TDD is to write your tests first, then write the code to pass the tests. This is done iteratively throughout development.
Personally, I'm unsure of how you would go about performing TDD for UI's; however, the team that I am on performs automated simulation tests of our UIs. Basically, we have a suite of simulations that are run every hour on the most recent build of the application. These tests perform common actions and verify that certain elements, phrases, dialogs, etc, etc, properly occur based on, say, a set of use cases.
Of course, this does have its disadvantages. The downside to this is that the simulations are locked into the code representing the case. It leaves little room for variance and is basically saying that it expects the user to do exactly this behavior with respect to this feature.
Some testing is better than no testing, but it could be better.
Yes, you can use TDD with great effect for GUI testing of web apps.
When testing GUI's you typically use stub/fake data that lets you test all the different state changes in your gui. You must separate your business logic from your gui, because in this case you will want to mock out your business logic.
This is really neat for catching those things the testers always forget clicking at; they get test blindness too !
Related
I can't understand the full process of testing. I am using a testing library and I feel comfortable with testing just texts, labels and I feel I don't need to test any implementation in React or Redux, and this is what I read in the React testing library documentation, as Enzyme forced people always to test React implementations.
Now if all texts, labels and displayed values are right in my tests, this means that everything should be OK, but unfortunately when I am using Redux, I am always forced to test Redux implementations (mocking the store, mocking reducers, etc.) and async behaviors like fetching data and so on.
Why do I need and am forced to test Redux implementations as long as I can just test displayed values and passed tests will always indicate that Redux implementations work correctly in my project?
I can't understand the full process of testing
That's understandable. Testing and software quality assurance is a huge professional field (there's more to it than video game playtesting!)
I am using a testing library and I feel comfortable with testing just texts, labels and I feel I don't need to test any implementation in React or Redux
This is the wrong attitude to have. What you're describing is a very non-rigorous, slapdash, haphazard - and most importantly: shallow kind of testing.
Just because data on the user's computer screen appears correct, it doesn't mean it actually is. How do you know you aren't actually interacting with a mocked UI or that the backend database actually contains the new data? Or that there weren't any negative side effects (such as the system deleting everything else - and yes, that happened to me once...).
By analogy, you're saying you'd be happy to fly in an airplane that was tested only by ensuring that moving the control yoke resulted in the on-screen airplane icon moving in the right direction - even if the plane is still firmly on the ground. I would not want to fly on that plane.
Now if all texts, labels, displayed values all are true in my tests, this means that everything should be OK
No, it doesn't. See above.
but unfortunately when I am using Redux, I always forced to test Redux implementations (mocking the store, mocking reducers, etc.) and async behaviors like fetching data and so on.
Yes. You're being forced to the right thing. It's called the Pit of Success. Don't fight it. It's for the best. These libraries, platforms and frameworks are designed by people with more experience in software design than both of us, so if they tell us to do something we should do what they say - if we disagree we need to formalize our objections and duke it out in GitHub issues with academic rigour, not Stack Overflow posts arguing that something's unnecessary because you just don't feel like it. With apologies for being blunt, but I hope you never work in a safety-critical industry or sector until your attitude changes because I never want to see another Therac-25 incident - which was directly caused by people sharing your attitude towards software testing.
Why do I need and am forced to test Redux implementations as long as I can just test displayed values and passed tests will always indicate that Redux implementations work correctly in my project?
Because what you're describing does not provide for anywhere close to full code-coverage.
Here's a bunch of assorted things to consider:
Software testing (and systems testing in general, in any field) can generally be lumped into these categories:
Unit testing: testing a single "unit" of your code in isolation from everything else.
(Side-note: many people are currently abusing unit-testing frameworks like xUnit and MSTest to implement what are actually integration tests, so many people don't understand the real difference between integration and unit tests, which is depressing...). A "unit" would be something like a single class or function, not an entire GUI application.
Your current testing strategy is not a unit test because you aren't testing anything in isolation: you have to fire-up the entire application, including the React/Redux pipeline, web-server and an extremely complicated, multi-billion-dollar GUI web-browser program application.
Generally speaking: "if you need concrete dependencies (instead of fakes or mocks) to test something, it isn't a unit-test, it's an integration-test".
Integration testing: testing multiple components that interact with each other.
This is a rather abstract definition - but it can mean things like testing an application's business logic code when it's coupled to (a copy!) of your production database. This could also include testing a business-layer when there's a GUI attached to it, but GUI testing is not easily automated - so many, but not all, people would not consider what you're doing as a unit-test, especially as what you've described implies that your tests aren't testing for side-effects or verifying other changes of state elsewhere in the system (such as the database or a backend web-service).
There are other types of tests besides unit and integration - but those two are the main types of fully automated tests that every application should have - and every application should have good code-coverage from unit and integration tests especially. Do note that code-coverage does not imply exhaustiveness, and achieving 100% code-coverage is often a waste of time if that code includes trivial implementations like boilerplate code, or parameter validation code, or code generated by a tool that itself is very-well tested.
Generally speaking: if a piece of code "is complicated" or changes regularly, it should have "good" (75%+? 80%? 90%?) code-coverage.
Because testing software via GUIs is very difficult (and brittle: as GUIs are probably the part that experiences major changes the most in any user-facing software system) it's actually often not subject to automated testing anywhere near as much as it should - which is why it's important to ensure good coverage of non-GUI parts with automated testing which also reduces the amount of manual GUI testing that needs doing.
Finally, a big thing to consider with the Redux pattern in particular is that Redux is not specific to GUI applications. Theoretically you should be able to take a Redux application and copy and paste it into a server-side Node.js JavaScript application and hook it up to a virtual DOM and hey-presto, your application no longer requires client-side JavaScript to work! It also means you can get great code-coverage of your application just by using a special virtual DOM that's intended for testing rather than a real browser DOM - but your current approach will not work with this because you're talking about only verifying changes to a real browser DOM, not a virtual DOM.
The app which I am developing will have the same functionality for all users/clients/projects (call 'em what you will).
However, the HTML forms presented to the user and the AJAX used to send them to the server will vary for each project.
I was thinking of using Angular constants, with ng-show / ng-hide (maybe even ng-if) on the HTML and a switch in the controller, based on a constant for the AJAX send & receive.
Is this a good approach? I can see things getting complex with more than a handful of projects. Should I use a different view/controller for each project? I might lose out on some common code that way, but it's less likely t become spaghetti.
I would suggest taking a domain/(test) driven approach. Don't generalize too much code up front.
Building generalized code will create dependencies that are all a potential victim in need of future refactoring. Even in the case of simple changes. Nothing is more time consuming / frustrating than those small modifications that cause an avalanche. I've seen alot of projects run out of time because the code base was over engineered at the start.
My approach, especially for the more complex projects with no clear overview of overlapping functionalities, is to just start designing/building the functionalities separate of each other in small steps. Just like any agile workflow, deliver a complete working feature (a working form) and when you're working on feature and notice that there's a shared functionality build earlier on, make a (wise) decision to refactor/promote the existing code into generalized code. At this stage you'll be in a better position to make such a judgement. If you've taken the test driven approach (which I highly recommend) refactoring the existing code can be done without too much effort.
Working this way gives a greater guarantee to deliver and to end up with good readable optimised code.
TL;DR
It all comes down to common sense.
I've inherited a functional but messy WPF MVVM application. To my annoyance, there were virtually no unit tests which makes the adoption MVVM somewhat pointless. So I decided to add some.
After grabbing the low-hanging fruit, I'm starting to run into trouble. There's a lot of highly interdependent code, especially inside the properties and methods that are used in the Views. It's common, for instance, for one call to set off a chain of "property change" events which in turn set off other calls and so forth.
This makes it very hard to test, because you have to write enormous mocks and set up a large number of properties to test single functions. Of course you only have to do it once per class and then re-use your mock and ViewModel on each test. But it's still a pain, and it strikes me this is the wrong way to go about things. It would be better, surely, to try and break the code down and make it more modular.
I'm not sure how realistic that is in MVVM. And I'm in a vicious circle because without good tests, I'm worried about breaking the build by refactoring to write good tests. The fact it's WPF MVVM is a further concern because nothing tracks the interdependence between View and ViewModel - a careless name change could completely break something.
I'm working in C# VS2013 and grabbed the trial version of ReSharper to see if it would help. It's fun to use, but it hasn't so far. My experience of unit testing is not vast.
So - how can I approach this sensibly, methodically and safely? And how can I use my existing tools (and look at any other tools) to help?
Start at the heart -- business logic
Your application solves some real world problems and this is what business logic represents. Your view models wrap around those business logic components even if they (components) don't exist just yet (as separate classes).
As a result, you should assume that view model is lightweight, data preparation/rearrangement object. All the heavy lifting should be done within business logic objects. View model should be served with raw data, display ready.
Modularize
Having this important assumption in mind (VM = no BL) move business logic components to separate, possibly modular projects. Organizing BL in modular way will often result in projects structure similar to:
Company.Project.Feature.Contract - interfaces, entities, DTOs relating to feature
Company.Project.Feature - implementation of contract
Company.Project.Feature.Tests - tests for feature implementation
Your goal with #1 and #2 is to reach a state where abandoning WPF and replacing it with console-interface should only require writing console interface and wiring it with BL layer. Everything WPF related (Views, ViewModels) could be shift-deleted and such action should not break a thing.
IoC for testability
Get familiar with dependency injection and abstract where needed. Introduce IoC container. Don't make your code rely on implementation, make it rely on contract. Whenever view model takes dependency to BL-implementation, swap it with BL-abstraction. This will be one of the necessary steps to make view models testable.
This is how I would start. By no means this is exhausive list and I'm convinced you'll run into situations where sticking to it won't be enough. But that's what it is -- working with legacy code is not an easy task.
I was wondering, how difficult it is to run UI-Tests, if the code has been obfuscated (especially regarding WPF-Applications when using testing frameworks that access automation properties of the application and arent image-based like e.g. Ranorex, TestStudio, TestComplete, Squish,...).
I can only find few informations about this, which imply, that testing should always be done before the code is being obfuscated, but not precisely why.
One might argue however, that tests should be run on the version that´s actually being shipped to the customer. Also if we´re using 3rd-Party components as part of our SW, we might not have the luxury of using a not-obfuscated version.
As far as I understand UI-Automation, the goal is to expose relevant properties of the application, so that they can be used not just by testing-frameworks, but also screen-readers and the like.
Therefore I can´t quite understand why there might be problems once the code is being obfuscated. The obfuscation itself shouldn´t influence the number of exposed properties at all or does it?
I can't speak for the others, but Ranorex relies on UIA (UIAutomation), an automation and accessibility framework, to automate WPF apps.
UIA is almost never affected by obfuscation. Also keep in mind that most obfuscation tools avoid obfuscating public members of public classes, which is what most UI controls use.
The only exceptions are rare cases in which you explicitly configure the obfuscation tool to obfuscate strings that might affect UIA, such as the AutomationProperties attached properties.
Another rather rare exception might have to do with reflection. If you use reflection (usually a bad idea, but sometimes unavoidable) to activate less reachable areas of your app, then obfuscation might pose a problem. This problem is easily solved by adding a few exceptions to the obfuscation tool, or running the tests before obfuscation.
Theoretically, it shouldn't matter whether you test the app before or after it's been obfuscated because obfuscation should theoretically not have any effect on the app's logic. In practice, there are some differences and this might affect your tests, though very rarely. Obfuscated apps tend to be a bit faster, and sometimes obfuscation mangles unexpected or unplanned code elements. So you have to ask yourself whether you want to perform UI automation on the app the users will actually get and catch obfuscation issues that might only affect UI automation, or test the app before obfuscation to ensure the app's behavior is correct regardless of whatever additional build and deployment steps might be waiting in the pipeline. Obviously, you'll have to deal with the possible repercussions of whichever approach you choose.
Another consideration for running the tests before obfuscation is that if application errors are encountered while running the tests, developers will have an easier time debugging them. However, if the programmers know how to debug obfuscated symbols (with code maps or the like), then this consideration is mostly moot.
I am trying specflow for the first time for a WPF project with MVVM pattern. So, I have repository, controller, view and view model. Repository makes calls to webservice wich hits the database and gives back the data. View model has methods to validate the input from the user and make calls to repository methods.
In my specflow, should I be making a complete call including the repository methods or should I mock those methods using Moq? Does it make sense?
The short answer:
You could make you calls hit the database, but I would recommend mocking them out, unless you have logic in your DB
The long answer:
There's two competing ways of looking at this question. Firstly what kind of testing are you expecting to use SpecFlow for, and secondly how easy is it going to be to access your database.
If you are using SpecFlow to test your low level technical requirements, then you really are doing unit testing in a Specification by example style. So this really ought to dictate that you use mocking to isolate your units. (Personally I'd stick to using NUnit for these tests.)
If however you are using SpecFlow to test your business scenarios (i.e. Acceptance testing) then your scenarios are reliant on more than one unit to provide the functionality. This is more like Integration or System testing and so many components will need to be involved and in theory since we are testing the whole system, we should include a DB in that, particularly if you have even one stored procedure or view that you might want to regression test later.
However, the second way of looking at this is how much pain is required to have a database available for your testing.
If there is more than one person in your development team, what strategies would you need to avoid failing if you run your tests at the same time?
For example you could generate a new unique customer in each run and only access updates and queries against that customer, then you won't end up with the wrong number of orders.
How will you reset your database after your tests run?
E.g. If you use the right kind of database you can check an empty .mdb into your source control and just rollback/revert to that.
I've personally found that for a simple DB without stored procs and views, mocking is better, but as soon as you've added business logic into the DB, well, you need to test that logic somehow.