My team is diving headfirst into a new project and we are taking the opportunity to bring ourselves up to date with Scrum methodology, new technology etc. Part of this involves trying to automate our QA process, using both Microsoft Test Manager and VS2010 Coded UI tests.
However, we are having problems getting the automated test playback to work with our currently-selected WPF control library, the DevExpress XPF suite. We've been using DevExpress for their WinForms and ASP.NET controls for years now, but this is our first experience with their WPF controls, and with automated UI tests, and the two don't seem to work well together.
In particular, trying to play back a recorded test that interacts with a grid control never works right, and even on simple controls the playback seems to get confused over which editor it should be interacting with. In the action recordings our test lead is generating from MTM I see a lot of this:
Type '5236' in 'PART_Editor' text box
Type '253' in 'PART_Editor' text box
And even worse, inside of the grids:
Click 'FooterPanel' pane
Last action on list item was not recorded because the control does not have any good identification property.
Click 'FooterPanel' pane
Click 'Item: CashEntry, Column ...' custom control
Move 'Item: CashEntry, Column ...' custom control from (141, 10) to 'Item: CashEntry, Column ...' custom control (5, 11)
Type '{NumPad8}{Down}{NumPad5}{NumPad0}{Up}{Down}{Down}' in 'CashEntry' window
Now, I have seen plenty of chatter on DevX's forums about their WinForms controls not supporting MSAA well enough for coded UI tests to work, but I thought that WPF used a different mechanism for automated UI testing that was supposed to work better. So, my questions are:
Is there a way to get the DevX XPF controls to work better with coded/automated UI tests?
If not, is there an alternative XPF control suite that is known to work well with coded/automated UI tests?
I'm specifically interested in a grid-like component with sorting, grouping, etc., a navbar-like component, and custom editors (dates, numbers, etc), all which support a unified system-wide theming.
(Also, if this type of thing is easy enough to do using just native WPF controls and styles, that would be helpful to know as well.)
EDIT: Since there appears to be no WPF control suite that actually works with MTM, we're being forced to use two separate testing tools at the sam time. Per several of the answers here, we're strongly leaning towards TestComplete. Hopefully the situation improves soon.
Don't have much idea about control suite supporting this but We had analyzed a test suite in past which supports objects based automated testing for WPF. Have a look at TestComplete (although it's not free)
Here is an excerpt from there site mentioning control support they provide -
WPF Control Support TestComplete offers rich support for WPF (XAML)
controls and robust automated UI testing of WPF applications.
TestComplete’s high-level test actions, such as item selection or cell
data input, are object-aware rather than coordinate-aware and thus are
more stable and resistant to the UI changes when performing WPF
testing. Flexible object mapping schemes allow testers to map custom
or derived WPF controls to standard types to simplify UI
manipulations. Support for custom WPF controls can also be added
using the TestComplete SDK.
http://smartbear.com/products/qa-tools/automated-testing/testing-wpf-apps/
I've heard very good things about WiPFlash.
In one of our projects we've used Infragistics WPF controls and TestComplete works with it.
Our QS Team also use TestComplete
I found a free utility that is independent of UI libraries.
PROJECT SIKULI
Sikuli is a visual technology to automate and test graphical user
interfaces (GUI) using images (screenshots). Sikuli includes Sikuli
Script, a visual scripting API for Jython, and Sikuli IDE, an
integrated development environment for writing visual scripts with
screenshots easily. Sikuli Script automates anything you see on the
screen without internal API's support. You can programmatically
control a web page, a Windows/Linux/Mac OS X desktop application, or
even an iphone or android application running in a simulator or via
VNC.
You can also use the VS internal UI test framework: Create a Coded UI Test.
just found this thread...
we had the same problem using DevExpress components,
which weren't findable on playback of CUIT.
But especially with Sikuli script the problem is solveable...at least.
Record test with MS Test Manager
Playback test, to find unreproduceble step
Create Sikuli sequence to 'step over'
Include created Sikuli script at appropriate position in CUIT and start again at Step 2
It really takes time to create tests this way, but at least its possible.
Related
I am going to start work on new project in which i am using DevExpress controls with winform applicaiton.
I want to do Coded UI test for winforms application and DevExpress controls.
I have VS 2010 Ultimate and DevExpress Controls 12.2 version(latest version).
In one of my example, i have created winform with GridControl of DevExpress.XtraGrid. when i create Coded UI test for this Grid Control, It's not working.
I mean it does not support it.
Can anyone give me solution/example/tutorial to do Coded UI for DevExpress controls in winforms.
Thanks !!
Refer: DevExpress Releases DXperience 12.1 with Coded UI Support and this.
Here you will get few videos that help you to understand about grid control and others also.
You can also go through Coded UI Test Extension for 3rd party
controls - the basics explained article of MSDN and documentation Verifying Code by Using Coded User Interface Tests.
More References:
DevExpress 2012 GridControl Coded-ui-tests issue
Ok for starters, ditch VS2010 and upgrade to VS2013 (Including latest update). If you don't do this - stop reading.
Next thing is versioning
To get reliable record and playback you will need to have the exact same version of DevExpess installed on your VS2013 Premium development box that your application under test uses.
So for example, if your app under test uses DevExpress 14.1.8 - INSTALL THAT VERSION in your coded ui dev environment. If you don't get the versioning exactly right - then it simply won't work.
Once you have that setup, create a new coded ui test project, and you should automatically get a DevExpress coded ui extension dll added.
Now fire up your application under test. Inside Visual Studio you will most likely get a sample test method, after creating the project. Set focus somewhere inside the test method, and then start the coded ui recorder. Now you should be able to identify controls and interact with them using the recorder. Save your actions / control identifications off to the UI map using the Generate Code in the recorder.
This should give you a good place to start with. If you really want my help, then you can send me a PM and I can share a lot of information that is beyond the scope of this. I'm on skype too and have made a lot of progress in the time I've spent specifically testing DevExpress based WinForms.
I can also share with you some helpful wrapped controls that wrap stuff like DXGrid to make your life tons easier.
I am trying to create a multitouch application.
I have the hardware which will allow me to do this. On the software side I want to be able to have WPF textboxes, WPF web browsers, multiple focuses, multiple keyboards and multiple users at the same time.
From what I've seen, I can't be focused on two controls at the same time.
What is the Microsoft MultiTouch approach for this kind of job ?
The OS limitations are what they are (and don't appear to change in Win8): only one hWnd at a time can have focus.
Since you are using WPF though, everything within your application (with the exception of the WebBrowser control ActiveX widgets you may be using) is rendered within one big hWnd.
WPF 4 introduced native support for multitouch, including multi-touch capture. The APIs for this are many but pretty intuitive so I'll just say this... go to http://msdn.microsoft.com/en-us/library/ms590078.aspx and search within the page for all of the members with "Touch" in their name.
The catch however is that the controls shipping with WPF 4 don't work with the touch input events... you'll only be able to interact with one of those controls at a time. To take advantage of the multi-touch capture APIs, you'll have to create controls that are designed with it in mind. Fortunately, the Surface team at Microsoft has you covered on that... the "Surface 2.0 SDK" includes a suite of controls (usable on any Win7 machine, not just for Surface) that were built with this stuff in mind.
To create application with MultiTouch UI, use MultiTouch Framework in .Net
Go to http://multitouchvista.codeplex.com/
What is the difference between using System.Windows.Automation (commonly known as UIA) API and Microsoft.VisualStudio.TestTools.UITesting (one that is used in Coded UI Test behind the scene) for implementing UI Test automation of a WPF application programatically?
Is there any specific scenarios, restrictions or advantages/disadvantages?
Coded UI Tests are a framework which wraps many automation technologies. UIA is just one of them.
UIA is new in Vista/Windows 7 and it targets rich client apps, coded UI tests will use UIA when you write tests against WPF and Silverlight.
The advantage of Coded UI over using UIA directly is a lot of automatic retrys and knowledge of how UIA works is built into Coded UI tests.
In fact, the answer is in your question. UIA is used for automation, while UITesting is used for... testing.
An excerpt from MSDN:
UI Automation provides programmatic
access to most user interface (UI)
elements on the desktop, enabling
assistive technology products such as
screen readers to provide information
about the UI to end users and to
manipulate the UI by means other than
standard input. UI Automation also
allows automated test scripts to
interact with the UI.
Thus, UIA can be used for - but isn't limited to - UITesting.
I'm very much new to Test Automation kind of thing. Recently I've been assigned to a project where I have to write an application (or, a script may be, I'm not sure) that will automate the UI testing of a CAD-like WPF application which misses lots of AutomationIds.
After doing a little searching on MSDN and other sources I'm a bit confused about whether I should use the Microsoft UI Automation Library or the new Coded UI Test feature included in VS2010. I'm not getting the clear picture of which one of these two applies in which scenarios, what advantages one has over the other and which one suits my purpose.
Please shade some light if you have experience/knowledge on the matter. Thanks in advance.
Basically Microsoft UIA is the new accesibility library in .Net 4.0. WPF applications and controls have built-in support for UIA through the AutomationPeer class.
Coded-UI test is a Record & Play automation tool which uses the Microsoft UIA Library underneath. Since being a tool compared to writing code in C# it improves QA productivity for recording more test cases.
For applications with automation support planned into it, Coded-Ui should be sufficient. If the AutomationIDs are missing make sure the controls have some unique property like Name. Use UIVerify or Inspect to check for this.
If NO unique property is avialble, there are the other below mentioned techniques you can use in combination with Coded-UI.
From an Event
When your application receives a UI Automation event, the source object passed to your event handler is an AutomationElement. For example, if you have subscribed to focus-changed events, the source passed to your AutomationFocusChangedEventHandler is the element that received the focus. For more information, see Subscribe to UI Automation Events.
From a Point:
If you have screen coordinates (for example, a cursor position), you can retrieve an AutomationElement by using the static FromPoint method.
From a Window Handle:
To retrieve an AutomationElement from an HWND, use the static FromHandle method.
From the Focused Control:
You can retrieve an AutomationElement that represents the focused control from the static FocusedElement property.
If you can leverage and use the Coded UI Test then go that route. Make sure to verify that your given configuration is supported.
The UI Automation Library resolves everything in the code behind. This then forces you to use a tool like UISpy to gain access to the controls internals so that you can then build out your test.
A Coded UI Test on the other hand still has code behind however it allows for the recording of steps through the given application which you are testing which will greatly increase the number of tests you can create.
UI Automation library is a low-level library. Usually, you don't want to write tests against it directly as it requires a pretty decent amount of work.
I would recommend looking at more high-level libraries. You mentioned one of them - Coded UI; another good choice would be White from TestStack. They both suits different kinds of projects. Coded UI is good when you don't want to invest a lot of efforts into your test suite. At the same time, it doesn't scale much so if you are going to write a lot of tests, you are better of choosing White.
Here I compare the two frameworks in more detail: Coded UI vs White
To complement the above responses, please look at CUITE that helps quite a bit and may be an appropriate approach for you.
I began 'rolling-my-own' 'semi-framework' using the CodedUITest library and devised a paradigm for separating the details of automation from the (C#) code.
Basically, I am creating a driver that reads what needs to be done from spreadsheet(s) where each line in it is a test step (or a pointer to a scenario in a different worksheet).
At present, incomplete, but promising, I have it working against a WPF application with partial success.
One of the main problems is that the developers neglected to identify controls uniquely and consistently.
Bey
I just installed visio, and the installer almost seemed like it was built in flash.
The buttons kinda glowed when I hovered over them, and when I clicked on 'continue' the form phased out in a cool way.
I'm assuming it was built in WPF.
Anyhow, so are WPF more flash-like (visually speaking).
Do they have new properties where you can make forms phase out nicely/smoothly compared to winforms?
Disclaimer: I work for Microsoft. However, I don't work on Visio, WPF, CLR or Silverlight team. So, the following is my personal take on these technologies. If you want to quote me, don't do it implying it's the official Microsoft position. :-))
Update: Anything I say below about Flash/Flex/AIR might be wrong, as I have not worked with these technologies and what I know about them is based on what I read on the intertubes. :-) If you notice anything wrong, just shout in the comment and I'll correct it.
To the best of my knowledge, the Visio installer is not built with WPF. It's all unmanaged code; it's just people took a lot of care to make it really polished.
WPF is the new UI platform for building standalone applications for the Windows OS. It supports a declarative UI language - XAML, and related CLR types to program against. WPF is a different platform than WinForms, although it is possible to build applications that mix UI built with both. WPF supports a lot of things that WinForms does not, like bitmap effects, animations, control styling and so on and exposes them both in XAML or through code. Also, WPF relies heavily on vector graphics, as opposed to the pixel graphics in WinForms. In short, WPF is quite powerfull and allows building very snazzy UI. (Don't take my word for it, though, as I am biased; go check around for what people are saying about it or buiding with it. :-))
WPF and WinForms do not compete with Flash/Flex. WPF and WinForms are both UI frameworks for building standalone client applications. As far as I know, Flash/Flex are frameworks for building rich internet applications - RIA (though lately people started interpreting this abbreviation as rich interactive applications).
Adobe did come up with AIR about half a year (or maybe a year) ago, which allows building standalone client applications, so you could say that Adobe is trying to position Flash/Flex/AIR to compete with WPF. Of course, that's my take on it and I doubt Adobe's official positiong is anything like that.
If you want to compare particular MS technnologies with Flash/Flex, take a look at Silverlight - it's the MS RIA platform.
Silverlight is related to WPF in the sense that they share XAML and the corresponding CLR types. Silverlight supports only a subset of what WPF offers, though, as it is not targeting Windows OS only and thus is limited by the fact that it has to be portable.
Quick update to reflect the changes in the year since I've written the answer :-)
With Silverlight 3 shipped, SL and WPF are getting even closer and sharing bigger set of supported features. In addition, most of the new XAML controls are built for platform at the same time. Thus, SL/WPF are getting to a point of singularity...
Also, SL 3 supports out-of-browser applications. In that sense, SL is not only starting to compete with Flash/Flex, but it is also encroaching on AIR's turf.
And no, I still don't work on the WPF or Silverlight team. :-)
WPF is being used as a replacement for WinForms, and as a competitor to Flash in the form of Silverlight. WPF consists of an entirely new object model that sits on top of DirectX (at least the desktop version). You can create WPF windows, controls, etc, entirely using C# or another .Net language just like you can render WinForms. However, Microsoft has also created a markup language called XAML (eXensible Application Markup Language). Nodes in an XAML document (XML) map to objects in a similar fashion to the way ASP.Net maps to web controls. XAML typically exists in a .Net project alongside a code-behind style C# file (or VB.Net or whatever). The C# file interacts with the objects generated by the XAML. This is fairly consistent with the "graphics via markup, logic via code" model that Microsoft and others are pushing.
One of the overlooked features when discussing WPF is the completely awesome data-binding that Microsoft wrote for WPF. The new data binding framework is a quantum leap beyond Windows Forms 2.0 data-binding. Microsoft added a couple of new interfaces that make it much easier to make an object or collection emit data-biding events properly. They also provided a very rich set of data-binding classes. You can bind anything to just about anything else. You can bind one-way data to control, control to data, two-way control to data and back, control to control, etc.
Back on the graphics side of the house, WPF makes it fairly easy to make an existing control look like anything. WP lets you compose your own template for what a class of buttons should look like, or one button, or all buttons. Or radio buttons. Or labels. You get my drift. Imagine if CSS included the ability to define what an input button would look like using other HTML controls.
They also provide a number of layout controls. You can continue to use exact positioning like in WinForms, or you can leverage of variety of techniques to make your window act more like a web page that grows and shrinks with resizing, etc.
The downsides: It is too easy to create spectacular effects that crawl on slower machines. Some of the graphics do not take advantage of hardware of graphics cards, though Microsoft has incrementally improved support for this. I believe when 3.0 first came out drop shadows were rendered purely using software. I think 3.5 or 3.5 SP1 changed it so that WPF would utilize graphics hardware for the task. Microsoft has said they will continue to enhance WPF in this fashion.
WPF is .Net 3.0 and above, which runs on XP SP2, Vista, and Servers 03 & 08. So don't plan on deploying WPF to a customer with Win2k desktops.
Summary: If you are doing desktop programming in .Net, you should be doing it in WPF unless you are targeting Win2k. You can avoid the downsides of WPF, and there are many upsides. Microsoft will probably throw away WinForms in some future release, or at very least you will stop seeing new features, etc.
As far as Silverlight goes, the betas for SL 2.0 look good. I think that Silverlight will require some wide-spread adoption. Microsoft has already tried to get this going. The NBC Olypmics site used Silverlight, and Major League Baseball uses it for its MLB.tv product. As soon as Silverlight gets a good install base I think you will see the Microsoft side of the development world starting swinging away from Flash and to Silverlight.
Edit after using Silverlight 3 and MVVM:
I have moved away from WPF and am doing a lot of Silverlight 3 development. But I think my comments here will still apply to the WPF developer.
I have been using the MVVM pattern in my app (think MVC with a twist). The Microsoft Patterns and Practices team has released a set of libraries known as Prism that supports various aspects of MVVM. There are WPF and Silverlight versions. Take a look at MVVM and Prism if you are going to be doing WPF or Silverlight development.
You can do a lot of flash w/ Winforms, or with custom components. But if you want out-of-the-box bang-whizz availability, WPF is the way to go.
Yeah, I think the intention is to be flash-like, it seems to me that MS has set its sights on taking down Adobe.
The way I see it: WPF is to Flash as WinForms is to Flex. WPF has more emphasis on vectors and states than on programming.