I'm very much new to Test Automation kind of thing. Recently I've been assigned to a project where I have to write an application (or, a script may be, I'm not sure) that will automate the UI testing of a CAD-like WPF application which misses lots of AutomationIds.
After doing a little searching on MSDN and other sources I'm a bit confused about whether I should use the Microsoft UI Automation Library or the new Coded UI Test feature included in VS2010. I'm not getting the clear picture of which one of these two applies in which scenarios, what advantages one has over the other and which one suits my purpose.
Please shade some light if you have experience/knowledge on the matter. Thanks in advance.
Basically Microsoft UIA is the new accesibility library in .Net 4.0. WPF applications and controls have built-in support for UIA through the AutomationPeer class.
Coded-UI test is a Record & Play automation tool which uses the Microsoft UIA Library underneath. Since being a tool compared to writing code in C# it improves QA productivity for recording more test cases.
For applications with automation support planned into it, Coded-Ui should be sufficient. If the AutomationIDs are missing make sure the controls have some unique property like Name. Use UIVerify or Inspect to check for this.
If NO unique property is avialble, there are the other below mentioned techniques you can use in combination with Coded-UI.
From an Event
When your application receives a UI Automation event, the source object passed to your event handler is an AutomationElement. For example, if you have subscribed to focus-changed events, the source passed to your AutomationFocusChangedEventHandler is the element that received the focus. For more information, see Subscribe to UI Automation Events.
From a Point:
If you have screen coordinates (for example, a cursor position), you can retrieve an AutomationElement by using the static FromPoint method.
From a Window Handle:
To retrieve an AutomationElement from an HWND, use the static FromHandle method.
From the Focused Control:
You can retrieve an AutomationElement that represents the focused control from the static FocusedElement property.
If you can leverage and use the Coded UI Test then go that route. Make sure to verify that your given configuration is supported.
The UI Automation Library resolves everything in the code behind. This then forces you to use a tool like UISpy to gain access to the controls internals so that you can then build out your test.
A Coded UI Test on the other hand still has code behind however it allows for the recording of steps through the given application which you are testing which will greatly increase the number of tests you can create.
UI Automation library is a low-level library. Usually, you don't want to write tests against it directly as it requires a pretty decent amount of work.
I would recommend looking at more high-level libraries. You mentioned one of them - Coded UI; another good choice would be White from TestStack. They both suits different kinds of projects. Coded UI is good when you don't want to invest a lot of efforts into your test suite. At the same time, it doesn't scale much so if you are going to write a lot of tests, you are better of choosing White.
Here I compare the two frameworks in more detail: Coded UI vs White
To complement the above responses, please look at CUITE that helps quite a bit and may be an appropriate approach for you.
I began 'rolling-my-own' 'semi-framework' using the CodedUITest library and devised a paradigm for separating the details of automation from the (C#) code.
Basically, I am creating a driver that reads what needs to be done from spreadsheet(s) where each line in it is a test step (or a pointer to a scenario in a different worksheet).
At present, incomplete, but promising, I have it working against a WPF application with partial success.
One of the main problems is that the developers neglected to identify controls uniquely and consistently.
Bey
Related
We were able to use automation tool and it was able to identify html objects on Winforms while on WPF it does not since it is rendered as an image.
My main question is what does Winform CEFSharp uses to render and why WPF not able to use a similar rendering mechanism?
Warning: it is a very generic answer. I briefly looked at CEF source (briefly - 3-5 minutes) and the rest are my guesses based on my own WPF/WinForms interop experiences. I've had quite a few. I also played a bit with early Chromium builds. However, all of that was a few years ago, so it may simply be out dated. Maybe Chromium has now first-class WPF support. I have not found any information about that, but if this really happened, I encourage you even stronger to follow the last paragraph.
--
I doubt that there is any reason behind this other than time-cost to implement -
either on CEF or Chromium project.
WinForms and WPF are totally different GUI frameworks, written in different eras, using different architectures, different rendering techniques, different platform features, etc. This is as different as it can be, down to the idea of a "Window" itself.
In WinForms, almost every control is a separate small window-like thing, has a system-wide handle, has a system-tracked region, etc. All controls render themselves almost directly by unmanaged win32 GDI+ functions.
In WPF they don't. In WPF there's only one handle per whole window, controls don't render themselves. Instead they have a definition of their "look" and the WPF renders them to the 'surface', which is then blitted/streamed (sorry, dont remember) to target device.
That's true that CEF uses different approaches. For WinForms they make heavy use of a 'browser component' taken directly from Chromium, for WPF they render to bitmap and show/update the bitmap periodically.
Why? My guess is that it's because Chromium already provided a COM/OCX/ActiveX/whatever component, and WinForms can use it almost directly, thanks to the everything-has-a-handle "feature" - if you can call it a feature - one of the goals and successes of WPF was to eliminate that.
However, I don't think that Chromium at that point of time provides any such component for WPF.
If it does not exist, then for WPF there are only two options - one could embed the WinForms component in WPF window through a special 'host' intermediate control, but that actually hits the performance and also has many problems when some advanced rendering features (like movie streaming) are used. Diagnosing and fixing them is complex, hard, and even unstable (crosshosted components behave very differently on different windows and .net versions, even on .net patches sometimes change them, it can work one one, and freeze on other, hang and render as black on next and cause a blue-screen on another)
Other option for WPF is to use the "offscreen" mode. Chromium can render to a bitmap, so why not. Render to bitmap, and display that. Simple. Quick. No problems.
So, I'd say, it all boils down to a famous quote from Eric Lippert:
The question is "why does [snip] not have this feature?" The answer to that question is always the same. Features are unimplemented by default; [snip] does not have that feature because no one designed, implemented and shipped the feature to customers.
It's great we can at least display Chromium in WPF apps. If you think it can be done better and that it's worth doing, it's open source, feel free to implement it - if not in CEF, then in Chromium itself.
My team is diving headfirst into a new project and we are taking the opportunity to bring ourselves up to date with Scrum methodology, new technology etc. Part of this involves trying to automate our QA process, using both Microsoft Test Manager and VS2010 Coded UI tests.
However, we are having problems getting the automated test playback to work with our currently-selected WPF control library, the DevExpress XPF suite. We've been using DevExpress for their WinForms and ASP.NET controls for years now, but this is our first experience with their WPF controls, and with automated UI tests, and the two don't seem to work well together.
In particular, trying to play back a recorded test that interacts with a grid control never works right, and even on simple controls the playback seems to get confused over which editor it should be interacting with. In the action recordings our test lead is generating from MTM I see a lot of this:
Type '5236' in 'PART_Editor' text box
Type '253' in 'PART_Editor' text box
And even worse, inside of the grids:
Click 'FooterPanel' pane
Last action on list item was not recorded because the control does not have any good identification property.
Click 'FooterPanel' pane
Click 'Item: CashEntry, Column ...' custom control
Move 'Item: CashEntry, Column ...' custom control from (141, 10) to 'Item: CashEntry, Column ...' custom control (5, 11)
Type '{NumPad8}{Down}{NumPad5}{NumPad0}{Up}{Down}{Down}' in 'CashEntry' window
Now, I have seen plenty of chatter on DevX's forums about their WinForms controls not supporting MSAA well enough for coded UI tests to work, but I thought that WPF used a different mechanism for automated UI testing that was supposed to work better. So, my questions are:
Is there a way to get the DevX XPF controls to work better with coded/automated UI tests?
If not, is there an alternative XPF control suite that is known to work well with coded/automated UI tests?
I'm specifically interested in a grid-like component with sorting, grouping, etc., a navbar-like component, and custom editors (dates, numbers, etc), all which support a unified system-wide theming.
(Also, if this type of thing is easy enough to do using just native WPF controls and styles, that would be helpful to know as well.)
EDIT: Since there appears to be no WPF control suite that actually works with MTM, we're being forced to use two separate testing tools at the sam time. Per several of the answers here, we're strongly leaning towards TestComplete. Hopefully the situation improves soon.
Don't have much idea about control suite supporting this but We had analyzed a test suite in past which supports objects based automated testing for WPF. Have a look at TestComplete (although it's not free)
Here is an excerpt from there site mentioning control support they provide -
WPF Control Support TestComplete offers rich support for WPF (XAML)
controls and robust automated UI testing of WPF applications.
TestComplete’s high-level test actions, such as item selection or cell
data input, are object-aware rather than coordinate-aware and thus are
more stable and resistant to the UI changes when performing WPF
testing. Flexible object mapping schemes allow testers to map custom
or derived WPF controls to standard types to simplify UI
manipulations. Support for custom WPF controls can also be added
using the TestComplete SDK.
http://smartbear.com/products/qa-tools/automated-testing/testing-wpf-apps/
I've heard very good things about WiPFlash.
In one of our projects we've used Infragistics WPF controls and TestComplete works with it.
Our QS Team also use TestComplete
I found a free utility that is independent of UI libraries.
PROJECT SIKULI
Sikuli is a visual technology to automate and test graphical user
interfaces (GUI) using images (screenshots). Sikuli includes Sikuli
Script, a visual scripting API for Jython, and Sikuli IDE, an
integrated development environment for writing visual scripts with
screenshots easily. Sikuli Script automates anything you see on the
screen without internal API's support. You can programmatically
control a web page, a Windows/Linux/Mac OS X desktop application, or
even an iphone or android application running in a simulator or via
VNC.
You can also use the VS internal UI test framework: Create a Coded UI Test.
just found this thread...
we had the same problem using DevExpress components,
which weren't findable on playback of CUIT.
But especially with Sikuli script the problem is solveable...at least.
Record test with MS Test Manager
Playback test, to find unreproduceble step
Create Sikuli sequence to 'step over'
Include created Sikuli script at appropriate position in CUIT and start again at Step 2
It really takes time to create tests this way, but at least its possible.
When I first learned about Microsoft's then-new framework for developing desktop applications, WPF, I thought that one of the biggest benefits of it would be its efficiency. After all, I already know what the GUI controls and elements were that I wanted to use--I just have to slap them all on the page and then wire up all my bindings right, and I'll be done. After that, I learned about the MVVM pattern, which promised clean separation of concern within my WPF app.
I thought this was going to be great! I got into creating several different admin and data entry WPF apps with my team at work, and thus I began to crank out working software with robust but simple GUIs, right?
Not so fast, there, cowboy coder. My experience is that writing WPF is S-L-O-W.
Well, at least the way I do it. You see, after I have a picture of my GUI on a whiteboard, I code up the XAML with the controls that I want. This part, as expected, is fast--laying out the whole of a window is pretty quick. After that, its all the little stuff you want these elements to do takes awhile. I always seem to want to make this text bold in some cases; show a red error box in these other cases.
Then things unravel: this binding isn't working right over here--I have to write a converter and adjust the layout for the right values. Whoops, I forgot that extra NotifyPropertyChanged there. Oh, and I want to use a different template in this state vs. that, so I have to figure out what I can use to swap the templates in certain situation. This data is coming in asynchronously, so I need to make sure the right thing is happening on the right thread and that Property gets NotifyChanged as well. Crap, when I maximize my window, it doesn't stretch like I thought it would--must be because its container height isn't defined, but I don't want to define that, so I have to make it stretch in its parent. Oh, now I really want to make a user control out of this stuff over here, so I better implement some dependency properties...
On and on I go, spending hours and days on stuff that just feels so small and minor. I soon resort to cutting usability features and look-and-feel enhancements because this is taking just too darn long.
Does anyone have any tips or principles I need to try in order to write WPF efficiently?
A couple of things that have saved a lot of time for me:
Use DockPanel as your default panel for layout unless you have a good reason not to.
Keep a folder full of useful classes: a ViewModelBase class that implements INotifyPropertyChanged, a RelayCommand class, etc. There's no need to get fancy and try to make this a separate assembly that you build into your project; just write reasonably good implementations and copy/paste them into your new projects.
Get Resharper and use it. Use templates for creating dependency properties, properties that do change notification, and commands.
Find or build a good library for asynchronous task management.
I find that even for very simple applications I get more done faster with WPF than I did with Windows Forms. For applications that aren't very simple, there's absolutely no comparison.
For the most part, WPF applications are a lot of work to develop because it's harder to make the case for cutting out UI features. You can't just say, "Oh, that's not possible," because it probably is possible (whatever "it" is).
Write your own library, or find an existing one.
WPF is great, but out of the box it is missing some things that would make coding faster. I got tired of writing the same things repeatedly, so I ended up creating my own library full of things like converters, visual tree helpers, attached properties, custom controls, etc., and since then, my development time has sped up considerably.
In addition to my own library, I've also started using Microsoft's Prism and Galasoft's MVVM Light Toolkit. Both contain useful objects that I use all the time and are better than what I could code on my own. (Most used are NotificationObject from Prism, RelayCommand from MVVM Light Toolkit, and EventAggregator or Messenger from either one depending on the project, etc.)
Another thing I've done to speed up coding time is to take advantage of Visual Studio's macros. For example, to create a property with Change notification, I write the private property, hit Ctrl+E, Ctrl+R which generates the public version, then run a macro which automatically sets up the PropertyChanged notification in the setter method.
I almost never change the setter methods from the default macro'd one, and instead use the PropertyChanged event to handle any changes that should occur on the setter. This not only makes it easier to track application flow, but also greatly reduces the time I used to waste browsing through my public properties to alter a setter method.
I believe the right answer isn't for WPF at all, but it can fit what you're looking for.
Most of the times, when you want to leverage a new technology there's a time while you're not efficient, productive and your solutions aren't that impressive, innovative or just doesn't look like others.
What will give you more efficiency is working with WPF itself.
It's more about project management topics than programming. After finishing some project, your team and you should go to some room and discuss:
Success stories.
Problems during development.
Pros and cons.
Fails in the application architecture.
Communication problems within the team and customer.
... and so on.
If everyone shares their knowledge, project manager or team leader does a good job documenting each project story, finally everyone will have a "know-how".
In addition, it's important that you won't need to reinvent the wheel for every new project: if some pattern worked fine, do the same way next time, even if it's not the best way of doing it. And try to enhance it, if possible.
Design patterns, technologies, paradigms, languages, companies, colleagues and nothing are a silver bullet: Microsoft said WPF is a step-forward in Windows client developments, and it is that: a more modern approach to provide shinny user interfaces and a programming paradigm that fits nowadays' desired approaches, easing the relation between coders and designers, as WPF has XAML, which allows not only separation of concerns, but separation of professionals by area (designers, UI programmers, business programmers, ...).
Finally, as I said above, WPF won't be your silver bullet: learn from your own success and read a lot, see sample applications, download open source solutions, listen your colleagues, drink a coffee and, after all, after some headaches, some day in the near future, you'll leverage these technologies (and many others).
EDIT
I'd like to add that a good way of using the know-how is creating a Visual Studio guidance pack, so you can automate a lot of tasks like creating managers, views, models and other things just in the way your team would do by hand.
For example, you can create a guidance pack for a WPF CRM-like application and you can automate module creation. When you want to add a new module, guidance pack starts a process which adds all the necessary classes to start development this new module, and it can create a sample form already associated with a navigation manager, controller or whatever (it's just an example).
Guidance pack and T4 would be both good tools for automating tedious or repetitive tasks in everyday's tasks:
http://msdn.microsoft.com/en-us/library/ff631854.aspx
http://msdn.microsoft.com/en-us/library/bb126445.aspx
I have been using WPF since 2008 and can honestly say to do it right and clean does take more time than the same thing in WinForms would take to develop. I have written a lot more WPF than Winforms. That being said - if I need a simple internal utility - it is ALWAYS Winforms. If I have something forward facing to a client - it is always WPF. Why? Winforms are cheap and dirty and you get a lot for free. What you don't get is the fit and polish that WPF can provide. The flexibility with WPF does come at a cost - but in the longer run it's worth it for public facing software.
Yes WPF is a hurdle but it also has rewards. You are on the right track with a design pattern such as MVVM. Sounds like you have not even gotten to the "rewards" of dependency properties or event bubbling. But the control over the UI is great. Almost everything is a content control. In forms I was always writing custom controls to get the UI I wanted. In WFP I have never had to write a custom control for UI and doubt I ever will. The syntax is new but it is compact - I rewrote a Form app in WPF and the WPF has 1/3 the lines and more features. Read a whole book on WPF just to get grounded - I like PRO WPF in C# 2010. You could also say LINQ is complex but man does it do a lot in just a few key strokes. WPF is not something you just pick up on the fly as you next application.
I want to create an automated UI test that will test my syncfusion grid. My problem is that the recorder can't recognize this control (or any syncfusion control). I've searched a lot in the internet but I couldn't find any extension so the recorder will recognize my controls (I'm using WinForms, not WPF!), or at least a way to extend the recorder abilities so syncfusion's controls will be recognized somehow.
Is there any easy way to extend the recorder? Or is there any extension available?
Or maybe can I get the grid object from the WinClient that the recorder generates?
Thanks!
Start your program. Run the Spy++ utility. Type Ctrl+F to start the finder tool and drag the bulls-eye onto your form. Ok, Synchronize and have a look-see at the windows that are visible in the tree. If you see regular Windows Forms controls, like a Button or a Label, but not any of the SyncFusion controls then you've probably found the source of the problem.
Component vendors that try to improve .NET controls typically do so by creating 'window-less' controls. They are not really controls, they don't derive from the Control class and don't have a Handle property. They use the surface of the parent to draw themselves, making them look just like controls. The .NET ToolStripItem classes do this. And this is also the approach WPF uses.
The big advantage is that they render quickly and support all kinds of effects that regular controls can't support, like transparency, rotation and anti-aliased window edges. The big disadvantage is that the kind of tool that you are using suddenly gets noddy and can't find the control back. Because they work by finding the Windows window back on your form, there is no window for them.
This is a hard problem to solve, the 'control' exists only in memory and there's no good way for a tool to find it back. Using Accessibility is about the only other way for such a tool to find a control that I can think of. Which would have to be implemented by the control vendor first, a somewhat obscure feature that gets easily overlooked. You really do need the help of the vendor to find a workaround for this. Shouldn't be a problem, that's why you paid them the big money.
This is Rajadurai from Syncfusion. Thank you for your interest in Syncfusion Products. To make UI Test Automation recognize Syncfusion grids(WinForms), some internal support need to be provided in grid whose implementation is in progress and about to be completed. Please submit an incident through Direct-Trac for any further related inquiries in the following link.
http://www.syncfusion.com/Account/Logon?ReturnUrl=%2fsupport%2fdirecttrac
You can also contact us through support#syncfusion.com. We are happy to assist you.
Regards,
Rajadurai
We were building out the next version of an in-house thick-client application using WPF/Prism (Composite Application Library). As we were nearly done with the client our team was put under new management and shortly thereafter:
We were then directed to drop the Prism framework to keep things simple. This includes not using any type of Inversion of Control.
We were directed to build out the WPF application without using MVVM or similar; and more along the lines of a traditional WinForm application. The idea is that if a developer sees a control in Visual Studio’s designer view, then (s)he should be able to click on the control and see exactly what it's doing without having to traverse through a view-model (or similar).
We have now been tasked with building out the WPF application using one primary Window, use a Frame Control to contain the content, and use a Ribbon outside of the frame for the menu items. Reason we were provided to use Frame Control:
a. We will show a view in the Frame with a Page (not a user control) and then load the page in the Frame.
b. When a new view is to be shown in the Frame, the current view (Page) will be closed/disposed and the new view (Page) will take its place in the Frame.
c. When a developer looks at the Page in design view, (s)he will be able to click on any control and see exactly what is being done.
Given the restrictions of 1 and 2 above, we’d like to present another method of building out the application that:
Can be presented as an alternative to using the “Frame Methodology” (item 3 above) but still provides the same type of functionality.
Does not use MVVM (see #1 and #2 above).
Provided the direction we’ve been given, any suggestions as to an alternative we can present? I’d request that the responses be kept on the professional level and thank you in advance.
I'd personally try to argue to use Martin Fowler's Presentation Model. (That's a joke, btw...)
Basically, you're being given a restriction that says "Use WPF, but don't use any of the features that make WPF usable." It really sounds like your requirements are such that you would be much better off explaining, reasonably, the advantages of patterns like MVVM.
It sounds like the weird requirements are really boiling down to this:
The idea is that if a developer sees a control in Visual Studio’s designer view, then (s)he should be able to click on the control and see exactly what it's doing
If that's the main issue, and the reason you're avoiding MVVM and other similar patterns, I would seriously take the time to educate the management. Looking at a Command, by name, instead of an event, by name (which is what you see in the designer) is really no more difficult.
However, in a large scale application, the separation of concerns is key. Even a properly designed Windows Forms application requires a clean separation of concerns - but with event based programming, this becomes much more difficult, especially from the designer. If you try to develop a large scale, clean, application using an event approach, you'll have event handlers, but those event handlers will all eventually need to delegate their work to a separate component.
This is actually adding an extra level of effort, from an understandability and maintenance point of view, on top of what you get with MVVM. With MVVM, you only look to the ViewModel, which is very discoverable.
BTW - The "rationale" for using a Page instead of a UserControl doesn't make any sense. You can do exactly the same thing you're describing with UserControls... The only reason to use a Frame and Page is if you want to take advantage of navigation, in which case, you can't dispose the old pages directly (or they get regenerated constantly). Also, the navigation tools probably wouldn't be used with a ribbon - the two conceptual models are quite different.
There are criticisms of MVVM which may be applicable to your project; however having unreasonable dictates of programming methodology is always a recipe for disaster.
One of the reasons that we have frameworks and spending time building layers and separation is to avoid the coding mess that always results when you can "simply click on the button in visual studio to see the code that is being executed".
There may not be a way of achieving what you've been asked to do without something similar to MVVM, because anything that has an architecture may well be labelled as being too similar.
However I have been using a system for many years that provides simple inter-object plumbing currently called Emesary you may want to read my C# .NET Emesary walkthrough.
But basically it allows my buttons to be implemented thus:
private void addButton_Click(object sender, RoutedEventArgs e)
{
GlobalTransmitter.NotifyAll(new Notification(NotificationType.CreateRecipe));
}
This may be an answer to your problem. It's under hyped, small and so simple but it just works well.
I've achieved a solution to the second question by using a Window, a user control for the ribbon bar (the user control contains the listview), and another user control for the Frame part. This second user control obvious is built using other user controls using a very simple view class. All views and controls are connected using Emesary.
As a school project I had to develop a WPF client which allowed for multiple people to make use of it at the same time. And I used Pages. My verdict: Save yourself a huge amount of effort, and use UserControls instead.
Sometimes the Page Navigator (which you'll use to scroll through) tends to bug out and cause you a lot of problems. Maybe it was my crappy coding, but who knows?
Though I must say, the control being called "Pages" is somewhat misleading... I went "Eureka!" when I found them, and swore at them thereafter.
I totally agree with #2 (MS bigs take note!). It would be cool if you could double-click a Control and it would take you straight to its command (or event if its command is lacking). However till then, make sure that you organize your Views and ViewModels in separate folders.
Having a dual screen (or a very wide one) will allow you to have two instances of VS open on the project, one focused around the View and the other around the ViewModel (my personal choice was having Expression Blend on the View).
Although not a very big application, I managed to convert my project to proper MVVM (ie. ViewModel for every UI element, RelayCommands and Mediator) in a matter of days, so once you understand it it's not too complicated to implement. Plus, there are tools out there (such as Josh Smith's RelayCommand and Marlon Grech's Mediator - totally free, by the way) which make MVVM half as difficult, and twice as powerful.
Using WPF without MVVM is like trying to eat rice without a fork. You'd be better off using WinForms if you're not going to take advantage of what WPF has to offer. My 2 cents.
I wish I could say your management is totally wrong.. but I cannot say that as it will not be the most accurate truth. I guess that the main reason to the changes you described is either because the new manager is not comfortable with the concept of MVVM being the new messiah of UI development or/and another reason could be the cost of educated sophisticated developers vs. cheap developers which can be instructed to get the things done as fast as possible, a concept which is widely known as lean development.
So, putting all I wrote so far under "not what you asked for", here is what I suggest:
you can still use object oriented pure approach, meaning you can have a model object which already have method to show UI information. so every object will be a window derived object, that way you will loose on SOC but you still going to be OOP/OOD.
But LOL, The next phase will bring you to seperation of model from view in order to not repeat the same code in many derrived windows which relay on the same data... so your management will endorse MVC/MVP as good solution .. and the distance from it to MVVM is kinda of short if they want WPF.
Conclusion: you will have to teach your manager why it is better to go for MVVM, unless the project is very short.