I'm trying to use MS UI Automation to test a WPF application, and am using the Inspect Object tool (inspect.exe) included with the Windows SDK to look up the AutomationId property on certain elements.
Inspect is behaving very strangely for me:
If I close all applications and start the WPF application and Inspect, inspect is able to see the AutomationId property for various UI elements. Elements which do not have an AutomationId simply show two quotation marks denoting an empty string ("").
After I perform a few actions in the WPF application, inspect.exe hangs and I have to kill it and restart it. Even though the machine's CPU and RAM utilization are around 50% or less, I've tried waiting several minutes--possibly close to 20 or 30 mins on a couple occasions--to no avail.
After restarting, inspect.exe can no longer find an AutomationId for any UI element, even those which did have them previously. What's more, the property is completely missing when hovering the mouse over the WPF application--it is no longer listed at all, not even with an empty string value.
If I move the mouse to another screen (specifically, to another computer, using Mouse Without Borders), the AutomationId property reappears with a value of "FormDot"
If I restart only inspect additional times while the WPF application is still running, inspect still behaves the same as after the first restart.
If I restart only the WPF app while inspect is still running, inspect still behaves the same as after its first restart.
If I close both inspect and the WPF app, then start inspect, then start the WPF app, everything works correctly for a while and inspect finds the AutomationId on a few elements in the WPF app...up until the point at which inspect hangs again.
I've tried running inspect both normally and as an administrator as suggested in https://stackoverflow.com/a/7833728/44737, and it behaves the same either way.
What, if anything, am I doing wrong? Am I just too impatient and do I need to wait a really long time instead of assuming inspect is hung? And why does inspect's behavior regarding AutomationId vary?
There are more than one version of Inspect.exe. The latest to my knowledge is the one dated from 2012 that says version 7.2.0.0 in the help/about dialog box.
The old one doesn't have a tree view on the left with all detected automation elements displayed in a tree, so it's easy to check you're using the right one.
The latest one works quite correctly, however, IMHO, the best tool so far to work with UI Automation is Visual UI Automation Verify. It's a .NET program, and he source is available here:
UI Automation Verify (UIA Verify) Test Automation Framework.
Note that although it's a .NET program, it doesn't use the standard .NET automation dlls (more on that here: What's the difference of UISpy.exe and Inspect.exe? (From Microsoft Windows SDK)).
About the AutomationId property, to clarify my initial comment to the question, I meant its usefulness depends on the program that you're trying to automate.
If you own it as a developer, it's clearly interesting. For example, if you're working with WPF, you can use the x:Uid property, it's clearly meant for UI automation. In the Winforms space, it's also quite useful because UI Automation will use the control's AccessibleName by default and revert to the Name as a fallback, for the AutomationId value.
But there are many apps that don't rely on .NET (browsers, native apps, etc.) Usually, for these apps, it's easier to use other properties.
I have been using inspect.exe for a while on a Microsoft Surface Studio PC (running Windows 10), and my experience is that inspect.exe will hang much more frequently (sometimes always) when Windows Updates are pending. When the updates are out of the way, inspect.exe is still somewhat slow, but much more stable.
Related
Returning user here. I have a requirement to migrate a system of OpenMotif 2.3 apps currently running under RHEL5, Gnome Desktop 2.x, to Gnome 3 under RHEL7. The problem is that the apps require modifying the window menu, adding items on a per-window basis and accelerators. Currently we are required by our customer not to migrate to GTK or any other widget set. Customer security requirements require using gnome-shell, gdm, et al, so xfce or anything that uses an external process for screen locking is unusable.
I've looked at the mess that is gnome-shell, and while I've found the JavaScript that assembles the window menu, it requires access to the window property holding the customized menu. I see no way to have JavaScript call back into C code to do this. (If all else fails, I could use xprop to dump the property and go from there.)
I'm thinking client-side decoration, with all of its horrors of having to manage windows yourself, might be the solution here. I can't seem to get CSD to kick in, though, without using GTK. I've tried applying the _GTK_FRAME_EXTENTS property, but gnome-shell ignores it. I've tried _NET_FRAME_EXTENTS with equal lack of success. The Extended Window Manager Hints documents say nothing about this. Google, of course, is ripe with comments about using GtkHeaderBar and discussion of avoiding CSD, neither of which apply here. Wayland-based items are equally unworkable.
I am quite familiar with modifying & extending the Motif widget set, having done that for a good chunk of my career here. I need to know how to tell the Window Manager to lay off without making the window override-redirect.
We were able to use automation tool and it was able to identify html objects on Winforms while on WPF it does not since it is rendered as an image.
My main question is what does Winform CEFSharp uses to render and why WPF not able to use a similar rendering mechanism?
Warning: it is a very generic answer. I briefly looked at CEF source (briefly - 3-5 minutes) and the rest are my guesses based on my own WPF/WinForms interop experiences. I've had quite a few. I also played a bit with early Chromium builds. However, all of that was a few years ago, so it may simply be out dated. Maybe Chromium has now first-class WPF support. I have not found any information about that, but if this really happened, I encourage you even stronger to follow the last paragraph.
--
I doubt that there is any reason behind this other than time-cost to implement -
either on CEF or Chromium project.
WinForms and WPF are totally different GUI frameworks, written in different eras, using different architectures, different rendering techniques, different platform features, etc. This is as different as it can be, down to the idea of a "Window" itself.
In WinForms, almost every control is a separate small window-like thing, has a system-wide handle, has a system-tracked region, etc. All controls render themselves almost directly by unmanaged win32 GDI+ functions.
In WPF they don't. In WPF there's only one handle per whole window, controls don't render themselves. Instead they have a definition of their "look" and the WPF renders them to the 'surface', which is then blitted/streamed (sorry, dont remember) to target device.
That's true that CEF uses different approaches. For WinForms they make heavy use of a 'browser component' taken directly from Chromium, for WPF they render to bitmap and show/update the bitmap periodically.
Why? My guess is that it's because Chromium already provided a COM/OCX/ActiveX/whatever component, and WinForms can use it almost directly, thanks to the everything-has-a-handle "feature" - if you can call it a feature - one of the goals and successes of WPF was to eliminate that.
However, I don't think that Chromium at that point of time provides any such component for WPF.
If it does not exist, then for WPF there are only two options - one could embed the WinForms component in WPF window through a special 'host' intermediate control, but that actually hits the performance and also has many problems when some advanced rendering features (like movie streaming) are used. Diagnosing and fixing them is complex, hard, and even unstable (crosshosted components behave very differently on different windows and .net versions, even on .net patches sometimes change them, it can work one one, and freeze on other, hang and render as black on next and cause a blue-screen on another)
Other option for WPF is to use the "offscreen" mode. Chromium can render to a bitmap, so why not. Render to bitmap, and display that. Simple. Quick. No problems.
So, I'd say, it all boils down to a famous quote from Eric Lippert:
The question is "why does [snip] not have this feature?" The answer to that question is always the same. Features are unimplemented by default; [snip] does not have that feature because no one designed, implemented and shipped the feature to customers.
It's great we can at least display Chromium in WPF apps. If you think it can be done better and that it's worth doing, it's open source, feel free to implement it - if not in CEF, then in Chromium itself.
I am trying programming an automated tester application using the new native Microsoft UI Automation interface 3.0 (in VC++ 2010, Win7). The Application Under Test (AUT) is a WPF application.
Almost everything works fine... I can install event handlers, navigate through the tree, search elements using various conditions and control the found elements using their patterns.
But yesterday I found a behaviour that leaves me despaired: The UIA tree of my AUT simply is not updated after switching its GUI main panel by clicking one of its main menu buttons.
After clicking the main menu button, I can see the new widgets in the AUT's GUI but the UIA tree still contains the controls that have been there before clicking the main menu button. The (outdated) UIA tree can be still completely read using search functions or using walker but of course it can not be written since the widgets do not exist anymore.
This exactly looks like there would be an outdated cache... however I do NOT use any caching UIA functions at all. None. Never. Nowhere.
I was not able to make the UIA tree update programatically... neither by calling any UIA functions nor by restarting the tester application nor by switching the AUT's GUI back and forth.
This does not happen every time. Sometimes after clicking the main button the tree seems properly up-to-date and everything works fine. However most of the runs it fails.
There is only one (mysterious) way that updates the UIA tree reliably: Using inspect.exe. When using inspect.exe tool for having a brief look into the AUT's UIA sub tree, the problem is suddenly gone and my tester application can access the actual, updated tree immediately! Of course the problem re-appears after restarting the AUT.
What does inspect.exe do to make the UIA tree (of another application!!!) update? How is it possible at all to access gone elements without using any caching? What did I miss?
I really need help.
Ok, some more findings:
UISpy.exe is able to refresh the UIA tree the same mysterious way inspect.exe does (this is especially strange because inspect.exe uses the same native interface as I do but UISpy.exe uses the .NET interface AFAIK). This means this is a kind of system wide and persistent UIA problem and not a pure native-UIA-problem.
The problem does not happend if I do not access the tree before switching the view. I.e. if my tester application does not access the AUT'S view before switching the view by clicking the main menu button, it sees the new widgets without problems. This strongly indicates some caching problem of the native UIA API - even if I have no idea how this could happen because I do not caching at all. Does someone know if there is some internal caching taking place?
I think this could be a API bug. However considering my current experiences with Microsoft Connect I am kind of lost with that showstopper :-(
Someone any idea?
I also tested the Snoop tool. Using Snoop does NOT heal the problem temporarily like Inspect and UISpy do. Regarding Inspect.exe, there is another detail... it is enough to collapse and expands the AUT's sub-tree to temporarily heal the problem.
Ok, update on this. UIA simply seems horribly buggy. I found that reading an element's content array and the length of that array updates the hidden caches. I also had a case where a subtree of a tab was not update but could be updated by switching to another tab and back (reading the content by pattern did NOT help in that case). Both cases were reproducable but I could not find any way to predict or prevent them. Also al lot of third party WPF components seem to be buggy. We finally gave up to use that API.
I know the question is quite old but I figured out what Inspect.exe does to refresh the UIA tree for other applications: look at the Options menu; there's an item labeled SPI_SCREENREADER flag which is checked by default.
Do the following in your code and you will probably get a fresh UIA tree:
SystemParametersInfo( SPI_SETSCREENREADER, TRUE, NULL, SPIF_UPDATEINIFILE | SPIF_SENDCHANGE);
PostMessage( HWND_BROADCAST, WM_WININICHANGE, SPI_SETSCREENREADER, 0);
I am currently working on WPF touch-screen application. I am developing it on Windows XP machine. I have tested it on this machine and it works perfectly fine. But when I deploy it to Windows Embedded machine I start to get strange behaviour: all pop up messages, drop down list, context menus appear behind the main window.
I am also setting the focus on my main window, when application loads, to enable context menu on the main screen.
Also my main window's AllowTransparency is set to true, (I have seen people had similar issues when having set to AllowTransparency). And also this didn't happen in the previous release.
Edit: The issues has gone after several compilations, I was unable to reproduce it, but I am still trying.
I also think it has to be something related to graphic driver, as it happens on one windows XP machine, but not on another (hardware is different, one run XP embedded 2nd XP professional).
Any idea why this is happening?
Have you tried forcing the ZIndex of the elements to that they are higher than the main window?
This MSDN blog post describes it's use - but the important part might be:
The first set of Rectangle objects uses the default z-ordering rendering of objects, which is based on the position of the child object in the Canvas collection
So if the order the objects are created has changed (for whatever reason) you might see this behaviour.
Explicitly setting the ZIndex will confirm or deny this.
I am trying to use WPFPerf to profile a WPF 4.0 application (I have the latest WPFPerf that should work on WPF 4.0 aps). I start the tool Visual Profiler from WPFPerf, I start my aplication, but after that nothing happens and the element tree from the Visual Profiler is empty. No other error message is shown. Can anyone tell me what am I not doint right?
As an additional information, when I try to analize my .exe assembly or any other assembly from my application, I get a BadFormatException saying that the assembly was build with a newer version of .NET. From the download page http://go.microsoft.com/fwlink/?LinkID=191420
I see that this version of WPFPerf should be ok for my app
TimeZone patch to WPF Performance Profiling Tools for WPF 4 is now available
http://blogs.msdn.com/b/jgoldb/archive/2010/08/24/timezone-patch-to-wpf-performance-profiling-tools-for-wpf-4-is-now-available.aspx
First check to see if its working even with a small blank WPF application. If it is not working with a blank WPF app, then it likely is the patch already mentioned: TimeZone patch to WPF Performance Profiling Tools for WPF 4 is now available
However, in my case that wasn't the issue. Visual Profiler has a bug that causes it not to display the Element tree if it doesn't like some of the characters in its main window title bar.
Through hours of trial and error, I was able to trace the root cause of why it would not work on our application even though it would on a blank WPF application. The reason is that our application has a special character in the title bar text (trademark). If we eliminate the special character from the main window title, the element tree shows fine when the visual profiler is attached. I was able to reproduce this issue in the simple blank WPF application too simply by putting the following property on the main window:
<Window ..... Title="Test Application™" >
Looks like a bug in Visual Profiler that somehow is triggered by special characters. I didn't test for other special characters, but likely it isn't just this one.
So the workaround is to remove the special character while you are profiling until they fix this.