for the application I'm running automation on - I noticed that there was a problem that some of the elements cannot be found via view spy. These elements do not have names nor Automation ID. I suggested adding one of these to all elements, which would you recommend for my purpose and what are the differences between the two? I just want to be able to locate elements as quickly and as efficiently as possible. I am fairly new to WPF automation testing as I traditionally come from a web automation background.
Thanks
In my opinion it doesn't matter which attribute is used as long as it is unique.
Related
For test automation purposes I would like to target specific UI controls in WinForms and WPF applications in a fast and resilient fashion. Currently I find myself using XPath to achieve this goal and it is brittle and slow. Ex:
/form[#controlname='frmPayloadDims']/container[#controlname='pnlAdajacent']/container[#controlname='grpThruster']/checkbox[#controlname='chkMegaJoulInclinationDeg'] ...
I'm looking for a better solution, perhaps similar to the HTML DOM's id attribute. There, model elements are available directly through their unique ids without having to build long paths.
Is there a global, unique and stable (across app restarts) identifier that I can set on controls when coding them so that I can use it to directly address them from Ranorex or similar UI automation tools?
My team is diving headfirst into a new project and we are taking the opportunity to bring ourselves up to date with Scrum methodology, new technology etc. Part of this involves trying to automate our QA process, using both Microsoft Test Manager and VS2010 Coded UI tests.
However, we are having problems getting the automated test playback to work with our currently-selected WPF control library, the DevExpress XPF suite. We've been using DevExpress for their WinForms and ASP.NET controls for years now, but this is our first experience with their WPF controls, and with automated UI tests, and the two don't seem to work well together.
In particular, trying to play back a recorded test that interacts with a grid control never works right, and even on simple controls the playback seems to get confused over which editor it should be interacting with. In the action recordings our test lead is generating from MTM I see a lot of this:
Type '5236' in 'PART_Editor' text box
Type '253' in 'PART_Editor' text box
And even worse, inside of the grids:
Click 'FooterPanel' pane
Last action on list item was not recorded because the control does not have any good identification property.
Click 'FooterPanel' pane
Click 'Item: CashEntry, Column ...' custom control
Move 'Item: CashEntry, Column ...' custom control from (141, 10) to 'Item: CashEntry, Column ...' custom control (5, 11)
Type '{NumPad8}{Down}{NumPad5}{NumPad0}{Up}{Down}{Down}' in 'CashEntry' window
Now, I have seen plenty of chatter on DevX's forums about their WinForms controls not supporting MSAA well enough for coded UI tests to work, but I thought that WPF used a different mechanism for automated UI testing that was supposed to work better. So, my questions are:
Is there a way to get the DevX XPF controls to work better with coded/automated UI tests?
If not, is there an alternative XPF control suite that is known to work well with coded/automated UI tests?
I'm specifically interested in a grid-like component with sorting, grouping, etc., a navbar-like component, and custom editors (dates, numbers, etc), all which support a unified system-wide theming.
(Also, if this type of thing is easy enough to do using just native WPF controls and styles, that would be helpful to know as well.)
EDIT: Since there appears to be no WPF control suite that actually works with MTM, we're being forced to use two separate testing tools at the sam time. Per several of the answers here, we're strongly leaning towards TestComplete. Hopefully the situation improves soon.
Don't have much idea about control suite supporting this but We had analyzed a test suite in past which supports objects based automated testing for WPF. Have a look at TestComplete (although it's not free)
Here is an excerpt from there site mentioning control support they provide -
WPF Control Support TestComplete offers rich support for WPF (XAML)
controls and robust automated UI testing of WPF applications.
TestComplete’s high-level test actions, such as item selection or cell
data input, are object-aware rather than coordinate-aware and thus are
more stable and resistant to the UI changes when performing WPF
testing. Flexible object mapping schemes allow testers to map custom
or derived WPF controls to standard types to simplify UI
manipulations. Support for custom WPF controls can also be added
using the TestComplete SDK.
http://smartbear.com/products/qa-tools/automated-testing/testing-wpf-apps/
I've heard very good things about WiPFlash.
In one of our projects we've used Infragistics WPF controls and TestComplete works with it.
Our QS Team also use TestComplete
I found a free utility that is independent of UI libraries.
PROJECT SIKULI
Sikuli is a visual technology to automate and test graphical user
interfaces (GUI) using images (screenshots). Sikuli includes Sikuli
Script, a visual scripting API for Jython, and Sikuli IDE, an
integrated development environment for writing visual scripts with
screenshots easily. Sikuli Script automates anything you see on the
screen without internal API's support. You can programmatically
control a web page, a Windows/Linux/Mac OS X desktop application, or
even an iphone or android application running in a simulator or via
VNC.
You can also use the VS internal UI test framework: Create a Coded UI Test.
just found this thread...
we had the same problem using DevExpress components,
which weren't findable on playback of CUIT.
But especially with Sikuli script the problem is solveable...at least.
Record test with MS Test Manager
Playback test, to find unreproduceble step
Create Sikuli sequence to 'step over'
Include created Sikuli script at appropriate position in CUIT and start again at Step 2
It really takes time to create tests this way, but at least its possible.
The AutomationProperties.AutomationId attached property seems to have two purposes:
Identifying UI controls during automated acceptance tests.
Accessibility.
Given that there are many cases when UI controls are generated at runtime (e.g. data-bound treeviews) which force the acceptance tests to search for controls using other means, when does it make sense to actually set a value for AutomationId, and what value? Is it an abitrary process to decide which controls should have them and which shouldn't? Or is there some general guideline that all developers can follow?
AutomationId is really for test purposes, so how you use it depends in how you want to test your app (or, for an app built by a team rather than an individual, how the test group wants to test the app). The key use for AutomationId is to allow the test code to reliably find an element without having to refer to something like Name, which can change on different localized builds (or which may change as the UI is tweaked during development). There's no requirement to use AutomationId at all; if test has some other means of identifying elements, they can use that instead and leave AutomationId unassigned.
Accessibility generally doesn't use it. It doesn't mean anything to an end-user, and isn't guaranteed to be present for any given control; so there's little useful that can be done with it.
Test and Accessibility do have a bunch of overlap; there are parts of UIAutomation that serve both, but also parts that serve only one or the other; AutomationId is one of the pieces that is specifically for testing.
It's arbitrary. The QA team should know about the controls that they'll need during automated UI tests, and you need to individually assign to them unique Id to simplify the work for testers. At least that's how I understand it. I haven't come across any guidelines regarding this.
I'm very much new to Test Automation kind of thing. Recently I've been assigned to a project where I have to write an application (or, a script may be, I'm not sure) that will automate the UI testing of a CAD-like WPF application which misses lots of AutomationIds.
After doing a little searching on MSDN and other sources I'm a bit confused about whether I should use the Microsoft UI Automation Library or the new Coded UI Test feature included in VS2010. I'm not getting the clear picture of which one of these two applies in which scenarios, what advantages one has over the other and which one suits my purpose.
Please shade some light if you have experience/knowledge on the matter. Thanks in advance.
Basically Microsoft UIA is the new accesibility library in .Net 4.0. WPF applications and controls have built-in support for UIA through the AutomationPeer class.
Coded-UI test is a Record & Play automation tool which uses the Microsoft UIA Library underneath. Since being a tool compared to writing code in C# it improves QA productivity for recording more test cases.
For applications with automation support planned into it, Coded-Ui should be sufficient. If the AutomationIDs are missing make sure the controls have some unique property like Name. Use UIVerify or Inspect to check for this.
If NO unique property is avialble, there are the other below mentioned techniques you can use in combination with Coded-UI.
From an Event
When your application receives a UI Automation event, the source object passed to your event handler is an AutomationElement. For example, if you have subscribed to focus-changed events, the source passed to your AutomationFocusChangedEventHandler is the element that received the focus. For more information, see Subscribe to UI Automation Events.
From a Point:
If you have screen coordinates (for example, a cursor position), you can retrieve an AutomationElement by using the static FromPoint method.
From a Window Handle:
To retrieve an AutomationElement from an HWND, use the static FromHandle method.
From the Focused Control:
You can retrieve an AutomationElement that represents the focused control from the static FocusedElement property.
If you can leverage and use the Coded UI Test then go that route. Make sure to verify that your given configuration is supported.
The UI Automation Library resolves everything in the code behind. This then forces you to use a tool like UISpy to gain access to the controls internals so that you can then build out your test.
A Coded UI Test on the other hand still has code behind however it allows for the recording of steps through the given application which you are testing which will greatly increase the number of tests you can create.
UI Automation library is a low-level library. Usually, you don't want to write tests against it directly as it requires a pretty decent amount of work.
I would recommend looking at more high-level libraries. You mentioned one of them - Coded UI; another good choice would be White from TestStack. They both suits different kinds of projects. Coded UI is good when you don't want to invest a lot of efforts into your test suite. At the same time, it doesn't scale much so if you are going to write a lot of tests, you are better of choosing White.
Here I compare the two frameworks in more detail: Coded UI vs White
To complement the above responses, please look at CUITE that helps quite a bit and may be an appropriate approach for you.
I began 'rolling-my-own' 'semi-framework' using the CodedUITest library and devised a paradigm for separating the details of automation from the (C#) code.
Basically, I am creating a driver that reads what needs to be done from spreadsheet(s) where each line in it is a test step (or a pointer to a scenario in a different worksheet).
At present, incomplete, but promising, I have it working against a WPF application with partial success.
One of the main problems is that the developers neglected to identify controls uniquely and consistently.
Bey
We are looking at creating a WPF UI that runs across multiple AppDomains. One of the app domains would run the application while the remaining AppDomains would host a series of user controls and logic. The idea, of course, is to sandbox these User Controls and the logic away from the main application.
Here is an example of doing this using MAF/System.AddIn. What are some of the experiences other have had with this? How does this solution handle RoutedEvents/Commands that might occur inside one user control and do these get properly serialized across AppDomains? What about WPF resources? Can they be accessed across AppDomains seamlessly?
Old question, but nonetheless: You will need to have multiple UI threads - one per AppDomain. Create them like this:
var thread = new Thread(() =>
{
var app = new Application();
app.Run();
});
thread.Name = AppDomain.CurrentDomain.FriendlyName;
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
The biggest challenge is then that you cannot send FrameworkElements between AppDomains (they are not MarshalByRefObject), but you can use the FrameworkElementAdapters.ViewToContractAdapter() and ContractAdapterToView() methods to work around this limitation. See the FrameworkElementAdapters MSDN page for more details.
Then, once you have this in place, the biggest problem IMHO is that you cannot lay anything on top of the FrameworkElement from the "remote" domain (the classical "airspace problem"). See this thread for more info about this.
I answered a simular question here and edited it for WPF also, you can use an intersting property of how the compisition engine operate's to tail-coat a dispatcher Pump, into one of the rendering contexts. It's a really light weight option.
Also, I'm guessing you know about the enterprise library and unity?
There is a WPF application block so using that pattern is not too painful ;) But don't they say, no pain no gain?
There's also CAB (Composite UI Application Block), ties into unity. The WPF SDK folks have crafted a Silverlight & WPF platform. (a.k.a Prism).
Oh right, also, you asked about Resources? I prefer to load reasources manually in the Application class. One thing I've realized, say you have a ResourceDictionary in a sub-folder and you are loading up MergedDictionaries in that ResourceDictionary. So, if in your Application class, you load "my-res-dir/MergedDictionaryLoader.xaml" (by code or xaml), ALL FUTURE LOADS OF MERGEDDICTIONARIES ARE LOADED FROM "my-res-dir".
Sort of insane if you ask me, I would think that as the process current directory has not changed, you should specify "my-res-dir/foo.xaml" for all your additional directories. However this is not the case (I do not believe this is documented anywhere at least very well and should be considered a bug imho).
So remember, WPF resource dictionary loading is going to be based off of the directory from which the current XAML is in. So you specify Source="foo.xaml" from within your "my-res-dir/MergedDictionaryLoader.xaml". I've even played with the URI pack / absolute syntax, however I've never found that too be much more intuative.