Hi I am evaluating ICUTest for use on a project. My initial view is that it looks like a promising Visual testing library. The scenario I have for using ICUTest is to start an application with a specific configuration and I expect the main application window to display based on the configuration settings. Each unit test should start the application and then after completing it should shutdown the application gracefully.
At the moment I can get individual tests to run, but when I run multiple tests I start running into all types of threading issues. Has anyone had any experience with this?
There are two ways to test your application.
1) The easiest (and most reusable) way is to just test your main app window like any other window. Do your initialization after a window event (like Window.Loaded) or through the constructor (e.g. new MainWindow("myapp.config") ).
2) If initialization must be done before the window is up then you can start the app thread with code similar to the one here.
Note: in WPF, you can only start an Application once, so method (1) is preferable.
Also, make sure you wrap all your GUI related calls in an ICU.Invoke(...) block.
Related
I would like to start automating the testing of my app written in CodenameOne, but I find it difficult to visualize how to use the TestRecorder (section "Unit Testing") for "industrial" testing.
If anyone here is already using it, could you share a few tips about how you use it?
E.g. how do you use the different "Asserts" buttons, how do you structure your tests into suites and how do you chain them together (e.g. so each test case will start in the right context like where in the navigation structure it is supposed to run), do you need to manually edit the tests, ... And is there anything to be aware of before creating lots of tests interactively, e.g. to avoid that your tests are invalidated by some irrelevant change to your UI?
I read in the blog post from May 2017 that the TestRecorder "wasn’t picked up by many developers and as such it stagnated". I tried TestRecorder and immediately came across a seemingly basis error in it (missing test for null) when recording a test case using the Toolbar, which gave the impression it is still the case. So, if anyone here is using another approach that is working well for you, I'd love to hear about that.
See the test classes we use to test Codename One itself here: https://github.com/codenameone/CodenameOne/tree/master/tests/core
You can use the test recorder to generate a skeleton but you can do this manually just like any test. The test API lets you invoke the app or just pieces of it and perform assertions on the behaviors within.
We are currently running smoke tests using Selenium Webdriver & JUnit against a B2C product. Since we are using Selenium, the scripts are totally dependent on the UI. Given that the product is out of a tech startup, the UI & workflows keep changing/evolving # an extremely high frequency.
The Consequence: Smoke tests which are supposed to validate the sanctity of the application keeps failing. The team spends more time fixing the scripts rather than validating the build.
I am pretty sure most of the Automation folks out there would have faced similar issues esp. with rapid dev cycles. Looking forward to see some approaches undertaken by others in the industry who have faced similar problems.
Note: The frontend is developed in PHP
Webdriver works roughly like this: there is a start point, webdriver interacts with it (by simulating a button press for example) and then finds the next item to interact with. The next item might be on the next page or the same page. It might be found in various ways, by id or the 3rd div that is class="foo" etc.
The tests are things like does the page load with 200 OK, does the string "login" appear in a particular place and so on
The problem with a changing UI is that all the elements "move about". The ids change and the 3rd div class foo disappears. This means that the webdriver interactions fail and the tests if they are looking for particular elements will fail too
One solution is to develop and test against a set of ids. These ids will refer to fixed UI elements. All searching in webdriver should use the ids. The development team writing the PHP will put the ids in the correct places.
The set of ids can also be used as the basis for a sort of specification and can be used to explain UI flow in different ways to different stake holders.
I do not know of any specific product that handles this process of managing ids in both tests and development code but maintaining a "lexicon" like this to describe the UI items should not be a major task
The more versatile the System under Test is the more important it is to have a framework on top of Selenium that reduces the maintenance effort for a change.
For the most common changes in a System under Test there are several known patterns that can help you to reduce the maintenance efforts:
By using UIMaps to model the UI of the application it is extremely easy to handle changed IDs, CSS classes or similar changes
PageObjects reduce the effort for larger UI changes (e.g. when an input field is changed from a TextBox to a Dropdown field)
Use Keyword Driven Testing to model test cases without any knowledge of the underlying technological representation. i.e. a keyword encapsulates an action from the users point of view – a example for a keyword can be: “loginWithValidUser()”
Don’t just utilize the UI for smoke testing if the UI / Application / Workflows change drastically and very often. Most of the time it is also helpful to test certain functionalities by calling WebServices without any Web-UI
I'm running a Windows Forms application that need to update the GUI, and therefore have to use the scheduler that come from the current synchronization context. Code: TaskScheduler.FromCurrentSynchronizationContext()
When writing unit tests - this scheduler need to be switched because we want to use our own scheduler that run stuff concurrently and synchronously in a test. There are alternatives (injection, ManualResetEvent) but that's ugly and we don't want it.
It is possible to modify TaskScheduler.Default by using reflection to overwrite a private variable and that's great, but there is no obvious way of doing the same with TaskScheduler.FromCurrentSynchronizationContext().
So, how do you do this?
One thing which might work when in a unit testing scenario, is to create a TestSynchronizationContext that inherits from SynchronizationContext, and assign it with SynchronizationContext.SetSynchronizationContext(). Your TestSynchronizationContext class would override Post method to instead redirect to the Send method, causing it to run synchronously.
You can find the sources here for reference.
I am putting together a Silverlight 5 application that will run out-of-browser and has a bit of everything on start-up. Specifically, here are the steps I need to follow:
Check that the app is running out of browser (if not, display a screen instructing the user to install it locally).
Display a "splash screen" (it would be nice if this would play animations while the remaining steps execute).
Configure MEF
Pre-load context information and 'static' data from the server (for example, settings). This data is required before any of the application logic can run.
Dynamically load additional XAP files including an external theme library.
Replace the "splash screen" with the shell which contains a navigation frame.
Navigate to the application's start page.
I also need to support Application Extension Services (IApplicationService, IApplicationLifetimeAware) so any process I implement must respect these services. Most of these services will require MEF to be configured, so they should not execute before MEF has been configured and imports satisfied.
Another consideration is that some imports may be satisfied only after the dynamic XAP files have been pulled in and MEF recomposed.
One of the hurdles I'm running into is the fact that I cannot do step 5 until the previous steps are complete. Loading the XAP files or calling the server for data asynchronously allows the code to proceed. I need a way to "stall" the UI until all of the composition is complete and all required context data has been loaded.
So, I'm looking for recommended approaches that satisfy all of these requirements and am happy to provide more details if that helps get to a working solution.
UPDATE
The best explanation I can give for my difficulty is that I must 'release' the UI thread to display a 'splash screen' but also suspend the normal life cycle of the application while each step executes. I can't (and don't want to) do everything in the Application.Start event handler because application services will have already started.
Plus, releasing the UI thread means I do work in the background and let the original method (Application.Start, for instance) return and the runtime will move forward in the startup process. For example, starting a background process in the Starting method of an application service then returning allows the runtime to raise the Start event on the Application object. But if I need the background process to complete before I can do the next thing, I have to suspend the current thread which blocks the UI.
So I'm not sure how to divide up the work or where to put it (App, application service, bootstrapper, workflow, etc.).
You can use ManualResetEvent class to force the service calls to be Sync (and guarantee the order they complete). You could also use this to synchronize the background thread to the main UI thread.
http://mohundro.com/blog/2006/06/27/a-little-bit-about-manualresetevent/
We want to test a connection to an application as a feature of a program we are developing, but to go further with this, we want to actually do a sort of diagnostic test to ensure that the app is working and not just take the service status as gospel (the main windows service running does not mean the app is working fully). However, this app has no api exposed by it, and the forms may be designed in C++ as the app is a mix of many languages (C# is just one of them).
One way to do this is by UI automation and then programatically perform the necessary UI actions to test the app works by performing a fundamental action which uses all the prerequisites like a domain-joined account, etc. However, is there a way to do this non interactively so the forms of the app don't actually show up? If not, is there another way to solve this problem?
Thanks
With no exposed API, you are stuck with automation.
Take a look at autoit. It excels at doing these types of tasks. If it's vbesque script isn't for you it has a DLL interface for use in your favorite language. It is free.
Check it out.
Here are some ideas
Headless UI
You should investigate if this application can be run in a "headless" mode, i.e. without a visible UI. Alot of applications have this option even though it might not always be obivous.
UI Automation
Some tools for UI Automation:
* Microsoft UI Automationbr
* HP QuickTest Proffesional
* AutoIt v3
Analyis Log, If there is one
You could investigate if the application you are connection to writes a log.
* 14:14 Status:OK Activity:Routed 24 messages (or whatever it does) Uptime:2h12m
* 14:15 Status:OK Activity:No Activity Uptime:2h13m
* 14:16 Status:OK Activity:Routed 12 messages, 2 failed see error.log for details Uptime:2h14m
If it does, then you can write a diagnostics script that reads the log, analyses Status, Activity, Uptime and raise flags for any strange behaviour.
Hope this helps!