The AutomationProperties.AutomationId attached property seems to have two purposes:
Identifying UI controls during automated acceptance tests.
Accessibility.
Given that there are many cases when UI controls are generated at runtime (e.g. data-bound treeviews) which force the acceptance tests to search for controls using other means, when does it make sense to actually set a value for AutomationId, and what value? Is it an abitrary process to decide which controls should have them and which shouldn't? Or is there some general guideline that all developers can follow?
AutomationId is really for test purposes, so how you use it depends in how you want to test your app (or, for an app built by a team rather than an individual, how the test group wants to test the app). The key use for AutomationId is to allow the test code to reliably find an element without having to refer to something like Name, which can change on different localized builds (or which may change as the UI is tweaked during development). There's no requirement to use AutomationId at all; if test has some other means of identifying elements, they can use that instead and leave AutomationId unassigned.
Accessibility generally doesn't use it. It doesn't mean anything to an end-user, and isn't guaranteed to be present for any given control; so there's little useful that can be done with it.
Test and Accessibility do have a bunch of overlap; there are parts of UIAutomation that serve both, but also parts that serve only one or the other; AutomationId is one of the pieces that is specifically for testing.
It's arbitrary. The QA team should know about the controls that they'll need during automated UI tests, and you need to individually assign to them unique Id to simplify the work for testers. At least that's how I understand it. I haven't come across any guidelines regarding this.
Related
for the application I'm running automation on - I noticed that there was a problem that some of the elements cannot be found via view spy. These elements do not have names nor Automation ID. I suggested adding one of these to all elements, which would you recommend for my purpose and what are the differences between the two? I just want to be able to locate elements as quickly and as efficiently as possible. I am fairly new to WPF automation testing as I traditionally come from a web automation background.
Thanks
In my opinion it doesn't matter which attribute is used as long as it is unique.
In our main Silverlight business app, we have a lot of controls that are visible and/or enabled based on user roles (e.g. Admin menu link visible only if in Admin role). We control that visibility via converters.
I'm just starting out writing unit tests, and I'd like to be able to test the visibility of these controls based on different roles (it'd be really bad if we had a bug there). However, since we're using the MVVM pattern, and the controls are not named, I'm not sure how to go about that. Any thoughts?
This is exactly what ViewModel is for. You should be able to test your view models easily (in your case I suppose whether they expose correct roles to view). And since converter is just a regular class, they should be tested just as well.
In terms of unit testing, you really got just two tasks to do:
Test whether view models expose correct data
Test whether converters convert role to visibility correctly
Here's where unit testing ends. Those tests should guarantee your view is fed with appropriate data to render itself as you planned. Now whether it does is different thing, but that's out of scope of unit testing. The final testing belongs to acceptance testing, quality assurance and also automated/scripted UI tests and more often than not - human verification.
I don't think that naming controls breaks MVVM in any way. If you are fundamentally opposed to naming them you can probably use the VisualTreeHelper to walk the control tree and find the controls you are interested in.
You can use Moq to mock the viewmodels to simulate specific roles and Microsoft's Silverlight testing framework to run the tests.
I'm very much new to Test Automation kind of thing. Recently I've been assigned to a project where I have to write an application (or, a script may be, I'm not sure) that will automate the UI testing of a CAD-like WPF application which misses lots of AutomationIds.
After doing a little searching on MSDN and other sources I'm a bit confused about whether I should use the Microsoft UI Automation Library or the new Coded UI Test feature included in VS2010. I'm not getting the clear picture of which one of these two applies in which scenarios, what advantages one has over the other and which one suits my purpose.
Please shade some light if you have experience/knowledge on the matter. Thanks in advance.
Basically Microsoft UIA is the new accesibility library in .Net 4.0. WPF applications and controls have built-in support for UIA through the AutomationPeer class.
Coded-UI test is a Record & Play automation tool which uses the Microsoft UIA Library underneath. Since being a tool compared to writing code in C# it improves QA productivity for recording more test cases.
For applications with automation support planned into it, Coded-Ui should be sufficient. If the AutomationIDs are missing make sure the controls have some unique property like Name. Use UIVerify or Inspect to check for this.
If NO unique property is avialble, there are the other below mentioned techniques you can use in combination with Coded-UI.
From an Event
When your application receives a UI Automation event, the source object passed to your event handler is an AutomationElement. For example, if you have subscribed to focus-changed events, the source passed to your AutomationFocusChangedEventHandler is the element that received the focus. For more information, see Subscribe to UI Automation Events.
From a Point:
If you have screen coordinates (for example, a cursor position), you can retrieve an AutomationElement by using the static FromPoint method.
From a Window Handle:
To retrieve an AutomationElement from an HWND, use the static FromHandle method.
From the Focused Control:
You can retrieve an AutomationElement that represents the focused control from the static FocusedElement property.
If you can leverage and use the Coded UI Test then go that route. Make sure to verify that your given configuration is supported.
The UI Automation Library resolves everything in the code behind. This then forces you to use a tool like UISpy to gain access to the controls internals so that you can then build out your test.
A Coded UI Test on the other hand still has code behind however it allows for the recording of steps through the given application which you are testing which will greatly increase the number of tests you can create.
UI Automation library is a low-level library. Usually, you don't want to write tests against it directly as it requires a pretty decent amount of work.
I would recommend looking at more high-level libraries. You mentioned one of them - Coded UI; another good choice would be White from TestStack. They both suits different kinds of projects. Coded UI is good when you don't want to invest a lot of efforts into your test suite. At the same time, it doesn't scale much so if you are going to write a lot of tests, you are better of choosing White.
Here I compare the two frameworks in more detail: Coded UI vs White
To complement the above responses, please look at CUITE that helps quite a bit and may be an appropriate approach for you.
I began 'rolling-my-own' 'semi-framework' using the CodedUITest library and devised a paradigm for separating the details of automation from the (C#) code.
Basically, I am creating a driver that reads what needs to be done from spreadsheet(s) where each line in it is a test step (or a pointer to a scenario in a different worksheet).
At present, incomplete, but promising, I have it working against a WPF application with partial success.
One of the main problems is that the developers neglected to identify controls uniquely and consistently.
Bey
I'm just getting started in the exciting world of WPF development, having been a C++ developer for many years.
Testing applications with rich user interfaces has of course, always been hard. One of the problems compounding this has traditionally been that in most Windows apps, the UI, the UI logic and the App logic are all completely interdependent, and cannot be tested in isolation.
I'm very drawn to the MVVM approach that will allow me to separate the UI from the application, and run large amounts of automated tests on my view models, underneath which all my logic will be, with the view being a fairly dumb client of the view model.
That's all well and good, and neatly separates out testing of the application logic from the application UI. BUT, it provides no solution for actually testing the UI itself. Even though the view will typically contain very little logic, it will still have the potential to contain a huge amount of bugs of various kinds.
What's the current state of the art in testing the view itself?
Thanks
Tom
This is always a double edged sword. I see it as attempting to grab the low hanging fruit and build from there.
In theory the MVVM purist would state that absolutely no logic exists in the View's code behind. Making use of Prism for instance can help alleviate this as well as other varying frameworks out there. So coming at it from this angle it begins to get to the point of no logic existing in the View...fair enough, are we then going to begin testing the bindings? You could, however depending on the sizing of the app what is the return on that investment?
What I think this boils down to is where do you draw the line? For instance, even if you are testing the View, you will most likely be hooking into the code, you're white box testing at that point. You then can argue the black box angle, that only testing without internal hooks is valid. You can see that it becomes a circular nightmare.
In general I have focused on the big ticket items and went from there, testing what was possible with the time allotted.
Think of it this way...with a UI you can begin this fiasco of testing the coloring on every button, along with the placement, etc... That's silly to me. Automate the bulk of UI testing at the Model, ViewModel, layer and if you so desire test the bindings of your View. Other then that I would suggest the ad-hoc manual effort every UI developer should be doing at their workstation.
WPF and MVVM doesn't change the process of testing an application's UI. It just radically reduces the number of defects you'll find while doing so, because so much of the stuff you'd normally find and fix during UI testing is now found and fixed during view model testing.
WPF's view model oriented way of doing things makes it very tempting to just use business objects in the UI. Have you seen any issues with this? Why or why wouldn't you do this?
The guidance from Microsoft's product teams (e.g., that's what the Blend team is using) is the Model-View-ViewModel architecture, a variant of the popular MVC pattern. A good starting point is http://blogs.msdn.com/johngossman/archive/2005/10/08/478683.aspx. There are also good articles by Dr. WPF on this topic.
Essentially, they advocate to create a ViewModel layer which uses binding-friendly business objects, such as ObservableCollection and the like.
Also, if you might eventually move to Silverlight 2, you might want to keep the business objects out of the UI layer so you can swap out UI technology (until WPF and Silverlight become source-code compatible).
I guess I see it in a different light. I try to keep as much out of the UI as possible so I can use whichever UI presentation I need (ie. web, WPF, WinForms). The more business logic in the presentation layer, the more you may have to rewrite later if you migrate towards a different UI.
It's not a problem having business objects in the UI, as long as all you're doing is viewing them. In other words, if you want to change the properties of one, or delete one, or create a new one, you should be sending a message to the controller, presenter, or whatever to do that; and the results should then be updated in the view.
What you shouldn't do is use the ToString method of your objects (or any other methods or properties on your objects) to affect how they'll appear in the view. You should use DataTemplates to represent the view of your objects. If you need a more complex representation, you can use an IValueConverter to change the object into its visual representation.
Not being a WPF guru, I can't be sure, but the usual reason for separating your M, V and C is so you can test the controller independent of the view, and the other way around.
Nothing stopping you, of course, but it should be a lot more testable (ie, unit tests) if it's separate. The MVP pattern, which is usually the one that MS promotes, is more geared around the presenter (ie, your WPF form) having more control, and thats fine too....
Depending on your application architecture or the on the way you are planning to reuse your components and objects, you can choose a certain degree of independence from the user interface (in this case WPF).
Here is a sample of my experience:
I've worked with WPF just a little, on
a relatively small project, where the
business layer was already defined,
and we just needed to create a user
interface. Of course, the interface
had defined it's own rules and objects
that it was working with, and because
the application was defined just for
UX we have chosen to create our own
specific objects, mostly by extending
DependencyObject (mainly for Data
Binding purposes).
Some people may argue that it's not ok
to use dependency objects because they
not are serializable (actually they
are - to XAML), they bring a
dependency to WPF (the
System.Windows namespace), and some
other arguments. Also,
DependencyObjects support other
options, like attached properties
and dependency properties. Others
might like to use for example
INotifyPropertyChanged if it
makes sense, and others might say that
all of these patterns don't belong in
other layer than UI.
(If you want to learn more there are
some good WPF data binding
articles in the MSDN library,
including best practices for
perfomance and for user interface)
It's kind of bad that Microsoft has chosen to add some of the goodies to the System.Windows namespace, instead of, for example, to the System.ComponentModel where in my opinion they might have been more useful (by providing all of these important patterns not only to WPF but to the .NET Framework).
Of course this is just the beginning and many of us know that the thing will be evolving to the right direction in the end. (With the risk of going off-topic: Take silverlight 2.0 framework for example. It was a rushed release, with some of the objects in the WPF model missing and some not in their natural place.)
In the end, everything depends on you, your programming style, your architectural decisions and your knowledge of the technology.
If it seems more natural to do it in a way, than by the book, think why you should and why should you not before taking any decision!