I have a WPF application using standard MVVM pattern. I want to test its performance and load . I see in Visual studio 2013 there is an option to check for Web Performance and Load test . I tried the Load test part of it, but Load test accepts only a test type i.e Unit Test or Coded UI test.
My question is : Cant i directly test the Load of my application without any test cases. Or what is the correct process to test my WPF application.
Never run load test on WPF or any kind of UI application. Load testing is generally done on the tier supporting the load : asp.net, Web Api, WCF ...and sometimes DB.
On UI-tier, it's better to do performance test, I mean simulating a huge amount of data coming from data-tier or with a high rate of transactions/sec. Visual Studio Profiler & Profiling tools will help you to determine the Hot Path and to find application bottlenecks.
Related
Is it possible to test AngularJS SPA application developed using ASP.NET MVC with Visual Studio or any other alternative tool for the same purpose?
I solved the above issue by using a headless browser (PhantomJS). I wrote a script to simulate user logging into the angular application and loading the inbox. But this way I was able to test only one user.
We cannot span multiple instances of PhantomJS, so I created 100 copies of the PhantomJS and then wrote a script to launch all these 100 copies to span 1 process of PhantomJS per file, the solution worked like a charm for load testing a SPA website.
If you have a Visual Studio version which includes load testing framework the answer is yes, see Get started with performance testing for details.
If you don't have a matching Visual Studio edition you can consider any other load testing tool, for example see http://www.opensourcetesting.org/category/testing-tools-overview/performance/?menu-page=overview
Any tool which supports HTTP/HTTPS protocols will play for you.
The most outstanding free and open source load testing tools are listed and compared in the Open Source Load Testing Tools: Which One Should You Use? article.
I have been looking into using MS Coded-UI as part of a new testing initative. We have exisitng UI automation using the HP stack QC/QTP but have found this to be very heavy weight and error prone, and as a result are looking to coded-ui in a bit to move our automation stack closer to the dev stack.
I am testing a wpf application running on windows 7 desktop, primarily on .net 4.5.
I have 2 questions.
Is it possible to auto generate the UIMap? Could i use a top level UITestControl object, find this and iterate over to generate a map of all subsequent children?
As part of continuous integration, is it possible to look coded-ui tests into a TFS build cycle? I am using visual studio 2012 ultimate but only 2010 instance of TFS. I assume these really all need to be in sync in order to pull this off correctly?
Regarding your first question, you can create Coded UI tests manually by recording your actions through Visual Studio or by using an existing action recording from Microsoft Test Manager.
Since a Coded UI tests is just plain code (C# or VB) you an easily edit it and add extra control logic.
Regarding your second question, it's absolutely possible to run your Coded UI tests as a part of your continuous integration build.
You have a couple of options:
Run your Coded UI tests on your Build Server. If you configure your Build Agent for Interactive use, you can let it run Coded UI tests. This is not my favorite option.
Use Lab Management with virtual environments (requires Hyper-V and System Center Virtual Machine Manager) or with Standard Environments. By using Lab Management you can automatically deploy to test machines and execute Coded UI tests on them.
Use Release Management in combination with Microsoft Test Management. This allows you to setup continuous deployment and run Coded UI tests on your target servers.
Lab Management with Standard Environments is the easiest to get started with, especially since you have an Ultimate license which includes Microsoft Test Manager.
Release Management is new and added to TFS 2013 for deploying to machines from test to production. It's a very nice solution but it currently doesn't integrate with Lab Management.
First question: there is not functionality to discover all children of some top-level parent control. But you can start Coded UI Builder, star recording and click or interact in another way with all controls you are interested in(it takes like a minute even for a window with a lot of children controls). That click Generate, name method like Dummy or whatever. All controls will be added to UI Map, and you can change search properties, names etc.
Second question:
It is possible to set up continuous integration flow including Coded UI tests - build, deploy, test.
Create new Build definition, use LabDefaultTemplate.xaml template for it.
Our team is using Silverlight 5 for development as it has features we require going forward. I have discovered that NUnit doesn't work as standard as it isn't Silverlight and that the tools available for Silverlight testing appear to primarily be aimed at running within browsers and are a pain to automate it seems.
However a lot of the code I want to test is the View Models which aren't specifically Silverlight, so I was hoping there is a way of using NUnit or similar to just test this and integrating into a build server (which is currently being decided upon)
So, is it possible to test Silverlight in this way?
I haven't tried it but looks interesting:
http://statlight.codeplex.com/
and what about third parties (maybe Telerik offers something on their testing suite).
I know it's no ideal, but if it's just VM and non dependant on UI you can link the files to a WPF project and execute the unit tests (in our case we had to share two versions the SL and the WPF one that's why we chose to test our VM in the WPF version :-)).
The unit testing from Silverlight (toolkit) I haven't found a way to link it to an automated build process
The best solution I have found is AgUnit:
http://agunit.codeplex.com/
AgUnit is an open source plugin for ReSharper. This solution of course assumes that you're using ReSharper. AgUnit allows you to run Silverlight unit tests in the ReSharper test running. Unfortunately, I just checked the AgUnit site and it looks like they only support Silverlight 5. You may want to head over there and find out about their plans for supporting Silverlight 5 though as it is a great plugin.
Prior to using AgUnit, we were using Statlight and found it to be pretty good as well, just not quite as convenient as AgUnit.
We have a WinForms application that we're planning to port to Silverlight. Obviously the UI will need to be totally rewritten, but we have a lot of business logic that's well-tested with NUnit tests. We use ReSharper to run the tests inside the IDE, and nunit-console to run the tests on the continuous integration machines.
As we start moving our business-logic classes into Silverlight assemblies, how do we get these unit tests to run against those Silverlight assemblies? My understanding is that NUnit can't run Silverlight tests without significant tweaking, and said tweaking (in the form of a project template) appears to be VS2008- and Silverlight 3-specific (we'll most likely be using VS2010 and Silverlight 4 for our port).
We can move our tests to another testing framework if that's the best option, but there doesn't seem to be much else besides NUnit. The Silverlight Unit Test Framework looks like it runs inside the browser, which is a non-starter for continuous integration.
I'm aware that Silverlight 4.0 assemblies can be loaded into a .NET 4.0 executable, but I'm unsure of what this means for unit testing. Would it work to compile our business-logic assembly as a Silverlight 4.0 DLL, and write a full-framework .NET 4.0 NUnit test assembly that references the Silverlight DLL? It seems like this might work, but has anyone done this successfully with unit tests?
Bottom line, what should we do for our unit tests? We need a solution that
still lets us run tests in the ReSharper test runner;
can also be run from the command line;
works in VS2010.
We don't instantiate any UI objects in our tests, so those pesky DependencyObject threading issues aren't a concern. We just need to test our Silverlight business logic.
Not sure if NUnit is supported or how difficult it would be to add, but you might check out http://AGunit.codeplex.com for ReSharper Silverlight support.
I wrote a little tool to C.I. silverlight tests and attempt to help in normal dev/tdd scenarios - you can go check out http://StatLight.codeplex.com.
You should be able to find an NUnit build for silverlight out there (there's a few). Not sure of any runners that work with NUnit in the browser. However StatLight can run some basic NUnit functionality for you.
Hope this gives you a little help.
I am in the process of recommending an UI automation tool for a windows based WPF application in a company
Which targets wide range of customers . The Application lives on a layer of WCF services for getting
The data. The company is already having a small VBScript framework written specifically for Test Complete
and is really not robust, Hence I am lookout for a right tool, where we could reuse the scripts or
Improve the already existing tool. Any ideas are welcome.
I have looked at Microsoft UI Automation, Project White, UIA Verify and Ranorex. Each one of them
Is having a feature lack.
I was able to go through the process of Microsoft UI Automation. I am still in the process of understanding how it works
with WPF, since it is totally dependent on Automation IDS.
Our application is huge and is not written with this in mind. All the code either does not implement
Automation ID property or Name property which is very much essential for object identification in the visual tree
in tools like UISpy and VisualUIAVerify.
What I am looking in specific are the following. Please give ratings to each of these
1) Recording : Recording , play back and automatic execution of test scripts and generate a report.
2) Dependencies : Minimal Dependencies(.NET frameworks, API modules , SDK Versions etc..)
3) )Code Generation : Code Generation of test scripts from Templates, Macros and Recording.
4)Object Name Mapping: Avoid explicit usage of Automation IDS and screen corrdinates, if they use, does it support any tools which would take XAML Files and insert Automation IDS into it and also avoid Problems With duplicate Automation IDS.
5) Events: Any Events and call backs that the user(tester) may receive if the GUI has undergone any changes
and how the scripts would change in this Case.
6) Support : Support for different scripting languages and easy Code conversions(C#, VBScript, Python.. etc)
7) Fast and Reliable: Easy and fast accessibility of GUI elements for manipulation( Like support to xml dom, Json.. etc), navigation, code duplication replacement tasks for changes in UI and easy configuration(xml, ini files)
8)Code Converters: the tester writes code in vbscript and developer could simply convert it into c# for embedding them into any stand alone framework if need be).
9)Custom controls: Support for custom controls and object mapping. If you have a custom object that behaves like one of standard controls. are you able to map (tell the test tool that the custom control behaves like the standard) control? Does it support all the standard controls methods? Can you add the custom control to it’s own class of control?
10)Reports: Should be able to execute multiple tests at once and generate a report similar to VisualUIAVerify.
Visual Studio 2010 Coded UI Tests
You can use Visual Studio 2010 Ultimate or Visual Studio 2010 Premium to create automated tests of the user interface known as coded UI tests. These tests provide functional testing of the user interface and validation of user interface controls.
I am using Coded UI Tests to automate tests of a WPF application built on the MVVM pattern. The record/playback features of the tool are impressive, generating really nice models. For large suites, the advice is to tear apart the generated code, structuring and extending it for your own purposes.
I recommend Visual UIA Verify to assist viewing the automation structure of your application while developing and debugging.
Have you tried AutomatedQA testcomplete
http://www.automatedqa.com/products/testcomplete/