Snoop, the spy utility, uses some powerful technique (probably some sort of reflection) to inspect a running WPF application. Most interesting is the fact, that Snnop is able to readout the entire object structure.
A few days ago I downloaded the Snoop source code and spent some time on studying the internal behavior. Unfortunately, I couldn't find out yet how Snoop is doing these things, so I hope that anybody can help me out.
At work I am currently writing a Coded UI Testing-Framework and it would be fantastic if I had access to the application's object structures because this would allow me to not only assert the UI state.
UPDATE:
This is the code needed:
string filePath = "WpfApp.exe";
AppDomain appDomain = AppDomain.CurrentDomain;
byte[] bytes = System.IO.File.ReadAllBytes(filePath);
Assembly ass = appDomain.Load(bytes);
ass.EntryPoint.Invoke(null, new object[] { });
IntPtr handle = Process.GetCurrentProcess().MainWindowHandle;
Window w = System.Windows.Interop.HwndSource.FromHwnd(handle).RootVisual as Window;
This is already a big help for me, but it is also interesting to find out, how Snoop injects itself into another process.
You can accomplish what Snoop does by using the WPF VisualTreeHelper and/or the LogicalTreeHelper. Once you get a hold of any visual element, you can pretty much traverse its entire visual tree to see all the elements it contains. Visual tree helper here
So in your UI test, grab the main window and traverse its visual tree to find any element you want and then perform any validations or operations you want on that element.
Furthermore, you may be able to use System.Diagnostics.Process.MainWindowHandle to get the windows handle from an existing process and then use the window's handle to create a wpf window. Its been a while so I dont remember the specifics without doing more research. The code below may help:
Window window = (Window)System.Windows.Interop.HwndSource.FromHwnd(process.MainWindowHandle).RootVisual;
UPDATE:
Okay, I found the basic code location, that is used by Snoop to provide the injection ability. To my astonishment that code is written C++/CLI. Probably there is a reason for.
And that is the code (I hope that it is okay to post it here):
//-----------------------------------------------------------------------------
//Spying Process functions follow
//-----------------------------------------------------------------------------
void Injector::Launch(System::IntPtr windowHandle, System::String^ assembly, System::String^ className, System::String^ methodName)
{
System::String^ assemblyClassAndMethod = assembly + "$" + className + "$" + methodName;
pin_ptr<const wchar_t> acmLocal = PtrToStringChars(assemblyClassAndMethod);
HINSTANCE hinstDLL;
if (::GetModuleHandleEx(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS, (LPCTSTR)&MessageHookProc, &hinstDLL))
{
LogMessage("GetModuleHandleEx successful", true);
DWORD processID = 0;
DWORD threadID = ::GetWindowThreadProcessId((HWND)windowHandle.ToPointer(), &processID);
if (processID)
{
LogMessage("Got process id", true);
HANDLE hProcess = ::OpenProcess(PROCESS_ALL_ACCESS, FALSE, processID);
if (hProcess)
{
LogMessage("Got process handle", true);
int buffLen = (assemblyClassAndMethod->Length + 1) * sizeof(wchar_t);
void* acmRemote = ::VirtualAllocEx(hProcess, NULL, buffLen, MEM_COMMIT, PAGE_READWRITE);
if (acmRemote)
{
LogMessage("VirtualAllocEx successful", true);
::WriteProcessMemory(hProcess, acmRemote, acmLocal, buffLen, NULL);
_messageHookHandle = ::SetWindowsHookEx(WH_CALLWNDPROC, &MessageHookProc, hinstDLL, threadID);
if (_messageHookHandle)
{
LogMessage("SetWindowsHookEx successful", true);
::SendMessage((HWND)windowHandle.ToPointer(), WM_GOBABYGO, (WPARAM)acmRemote, 0);
::UnhookWindowsHookEx(_messageHookHandle);
}
::VirtualFreeEx(hProcess, acmRemote, 0, MEM_RELEASE);
}
::CloseHandle(hProcess);
}
}
::FreeLibrary(hinstDLL);
}
}
Snoop doesn't inspect a WPF from the outside. It injects itself into the application and actually adds the magnify or snoop window to it. Thats also why when you exit snoop the inspection windows actually stay open.
So the 'inspection' code simply inspects the window it wants and it can use all avaible WPF functions to do so. Like the VisualTreeHelper and LogicalTreeHelper as mentioned here earlier.
For a small test framework i build i injected code to add a small proxy object so i can control the application easily (press buttons, change values, execute functions on viewmodels etc).
The answer above doesn't work for me. It seems a bit vague. I expanded on accepted answer a little with this code:
var allProcesses = Process.GetProcesses();
var filteredProcess = allProcesses.Where(p => p.ProcessName.Contains(ProcessSearchText)).First();
var windowHandle = filteredProcess.MainWindowHandle;
var hwndSource = HwndSource.FromHwnd(windowHandle);
This answer seems more complete and will work for others if the accepted answer works for anyone. However, this the last line of this code returns null for me.
Related
On Windows 10, you can press Win+Tab to get a "Task View" view of all your windows. I'm trying to check if this is active at any given time. I have tried using a Low Level Keyboard Hook with WH_KEYBOARD_LL but this only allows me to detect the keypress, not if the switcher is active. I've looked at the Windows DWM API and haven't found anything else either.
I have also tried using EnumWindows() and EnumChildWindows(GetDesktopWindow(), ...) and did not find any difference in the output between having the task view shown and hidden.
Is there any accurate method to detect if this is being shown?
Here's a solution that works very consistently with my version of Windows (1709 build 16299.125) and doesn't require the processor-heavy approach of a call to EnumChildWindows:
bool isTaskView() {
//Get foreground window's name
HWND fgWindow = GetForegroundWindow();
TCHAR windowName[MAX_PATH] = L"";
GetWindowText(fgWindow, windowName, MAX_PATH);
//Compare with magic string name of Task View's window
std::wstring nameStr(windowName);
return nameStr == L"Task View";
}
I'm building a custom control for Windows Phone 7+ that can do augmented reality image processing. The control works wonderfully in practice (when I run the app), but because I have the image processing running on a separate thread, it breaks when I try to open the page in Blend or the Visual Studio designer.
Here's an example of the thread I'm trying to run (basically taken from http://msdn.microsoft.com/en-us/library/hh202982(v=vs.92).aspx) :
public override void OnApplyTemplate()
{
// assigning template stuff, initializing my camera
_myManualResetEvent = new ManualResetEvent(true);
_myCameraProcessingThread = new System.Threading.Thread(ProcessingMethod);
_myCameraProcessingThread.Start();
}
void ProcessingMethod()
{
int[] myBuffer = new int[640 * 480];
while(_someCondition)
{
_myManualResetEvent.WaitOne();
_myCamera.GetPreviewBufferArgb32(myBuffer);
// do my processing stuff
_myManualResetEvent.Set();
}
}
This breaks the ever-loving heck out of Blend. Would love to know why.
It looks like you are doing a lot of run-time stuff in the OnApplyTemplate method.
This will get called when Blend or Visual Studio instantiates the design view of your control.
You should either check to see if you are in design mode using the DesignMode:
if (!DesignMode)
{
_myManualResetEvent = new ManualResetEvent(true);
_myCameraProcessingThread = new System.Threading.Thread(ProcessingMethod);
_myCameraProcessingThread.Start();
}
or move this code into a method/event handler that only gets called when the application actually runs.
We are using the WPF FormattedText object to determine text size in a service that grabs the latest news headlines from an RSS feed. The text retrieved needs to be in a specified canvas size. The service runs the code every 10 seconds and uses up to 2 threads if one takes longer than that. I'm using TaskFactory (which I've overridden the LimitedConcurrencyLevelTaskScheduler to limit to the amount of threads I specified).
This works great, except after several days (the length is variable), we start to get the following exceptions. The same code was working fine before we started using TPL to make it mult-threaded.
I need help figuring out what this is caused by. A few thoughts I'm looking into are: thread collisions holding on to a TTF file, memory issue, the dispatcher (see the stack trace) isn't playing nicely with the TaskFactory, other??
We don't have good profiling setup, but we've looked at the TaskManager when the exception is occurring and memory usage looks normal.
My next attempt is to use the TextBlock object and see if the exception is avoided.
Error Message: The system cannot find the file specified
Error Source: WindowsBase
Error Target Site: UInt16 RegisterClassEx(WNDCLASSEX_D)
Exception Stack Trace:
at MS.Win32.UnsafeNativeMethods.RegisterClassEx(WNDCLASSEX_D wc_d)
at MS.Win32.HwndWrapper..ctor(Int32 classStyle, Int32 style, Int32 exStyle, Int32 x, Int32 y, Int32 width, Int32 height, String name, IntPtr parent, HwndWrapperHook[] hooks)
at System.Windows.Threading.Dispatcher..ctor()
at System.Windows.Threading.Dispatcher.get_CurrentDispatcher()
at System.Windows.Media.TextFormatting.TextFormatter.FromCurrentDispatcher(TextFormattingMode textFormattingMode)
at System.Windows.Media.FormattedText.LineEnumerator..ctor(FormattedText text)
at System.Windows.Media.FormattedText.DrawAndCalculateMetrics(DrawingContext dc, Point drawingOffset, Boolean getBlackBoxMetrics)
at System.Windows.Media.FormattedText.get_Metrics()
at
(my method using the FormattedText, which is in a loop)
private static Size GetTextSize(string txt, Typeface tf, int size)
{
FormattedText ft = new FormattedText(txt, new CultureInfo("en-us"), System.Windows.FlowDirection.LeftToRight, tf, (double)size, System.Windows.Media.Brushes.Black, null, TextFormattingMode.Display);
return new Size { Width = ft.WidthIncludingTrailingWhitespace, Height = ft.Height };
}
EDIT: so far I've tried placing a lock around the code that calls this function, and calling it inside the CurrentDispatcher.Invoke method like so:
return (Size)Dispatcher.CurrentDispatcher.Invoke(new Func<Size>(() =>
{
FormattedText ft = new FormattedText(txt, new CultureInfo("en-us"), System.Windows.FlowDirection.LeftToRight, tf, (double)size, System.Windows.Media.Brushes.Black, null, TextFormattingMode.Display);
return new Size { Width = ft.WidthIncludingTrailingWhitespace, Height = ft.Height };
}));
EDIT: I've found links to others having similar, but not the exact problem.
http://www.eggheadcafe.com/software/aspnet/31783898/problem-creating-an-bitmapsource-from-an-hbitmap-in-threaded-code.aspx ~having a similar problem, but no answers
System.Windows.Media.DrawingVisual.RenderOpen() erroring after a time ~having a similar problem, but no answers
http://connect.microsoft.com/VisualStudio/feedback/details/361469/net-3-5-sp1-breaks-use-of-wpf-under-iis# ~ similar exception, but we're not using 3.5SP1 or IIS 7.
I've also submitted this through the Microsoft Connect site (please vote for it if you are having a similar problem).
https://connect.microsoft.com/WPF/feedback/details/654208/wpf-formattedtext-the-system-cannot-find-the-file-specified-exception-in-a-service
EDIT: Response from Microsoft:
"WPF objects need to be created on Dispatcher threads, not thread-pool threads. We usually recommend dedicating a thread to run the dispatcher loop to service requests to create objects and return
them frozen. Thanks, WPF Team" ~ How would I implement this?
EDIT: final solution thanks to NightDweller
if(Application.Current == null) new Application();
(Size)Application.Current.Dispatcher.CurrentDispatcher.Invoke(new Func<Size>(() =>
{
...});
EDIT: When I deployed the change (new Application();), I got an error logged " Cannot create more than one System.Windows.Application instance in the same AppDomain."
Error Source: PresentationFramework
Error Target Site: Void .ctor()
A shot in the dark:
The stack trace seems to show that WPF does not find a Dispatcher in the thread executing GetTextSize, so it has to create a new one, which involves creating a handle to a window.
Calling this every 10 seconds means 8'640 threads, thus windows per day. According to Mark Russinovich, there is a limit of 32 K windows per session, which may explain the error in RegisterClassEx.
An idea to overcome this is to read the current dispatcher from your main thread and set it in your tasks.
Edit:
I had another look and it looks like one cannot set the Dispatcher of a thread (it's created automatically).
I'm sorry, I am unable to understand what is going on here.
In order to compute the text size, WPF needs a FormattedText instance, which is stored as a member of the Dispatcher class. The existing Dispatchers are stored in a list of weak references. Each one is associated with a specific thread.
Here, it looks like new Dispatcher instances are created many, many times.
So, either the calling thread is new or memory is quite low and the weak references have been discarded.
The first case (new thread) is unlikely as the task scheduler uses the thread pool, which has about 25 threads per core (if I remember correctly), which is not enough to deplete the pool of ATOMs or windows.
In the second case, the depletion of resource is unlikely as the HwndWrapper is IDisposable and the Dispose method takes care of freeing the registered class.
As you already know from the info you provided, All UI elements (FormattedText is one) have to be created on the UI thread.
The code you are looking for is:
return (Size)Application.Current.Dispatcher.CurrentDispatcher.Invoke(new Func<Size>(() =>
{
FormattedText ft = new FormattedText(txt, new CultureInfo("en-us"), System.Windows.FlowDirection.LeftToRight, tf, (double)size, System.Windows.Media.Brushes.Black, null, TextFormattingMode.Display);
return new Size { Width = ft.WidthIncludingTrailingWhitespace, Height = ft.Height };
}));
Notice the Application.Current - you want the "Application" dispatcher which is the dispatcher for the UI thread in WPF applications.
Your current code actually creates a dispatcher for the current thread so you didn't really change the executing thread (see here regarding the dispatcher)
Have you renamed anything? If yes, check that link: WPF Prism: Problem with creating a Shell
I'm trying to use the following code to press a button on my other application:
HWND ButtonHandle;
if( (wnd = FindWindow(0, "Do you want to save?")) )
{
ButtonHandle = FindWindowEx(wnd, 0, "SaveButton", "&Save");
SendMessage(wnd, WM_COMMAND, MAKEWORD(GetDlgCtrlID(ButtonHandle), BN_CLICKED ), (LPARAM)ButtonHandle);
}
It doesn't work. I tried passing different handles to MAKEWORD and to change the WPARM and LPARAM but nothing.
Any ideas on how to click a button on another application's window?
Code is appreciated.
Thanks.
EDIT:
The reason it doesn't seem to work permissions. I sent a PostMessage() and the result was an error with GetLastError() = 5 (or Access Denied).
Any ideas?
EDIT2 I don't mean to be rude but please please please, I already searched all the API's including getting and setting the regions for the button and then sending a button down and button up, getting the control ID, getting the class ID and a zillion more.
The reason I asked the question here in the first place is because I already exhausted my search on the internet.
If you know the answer PLEASE POST CODE, do not suggest an API and that's it, show me how does that API solves the problem. It's not hard.
thank you.
EDIT 3: The question's answer was selected automatically when the bounty finished. The question still remains without an answer.
Are you sure that "SaveButton" class name is valid? Do you get the button
handle?
Try to send messages to ButtonHandle window (directly to the button).
Update: I believe this should work,
SendMessage(ButtonHandle, BM_CLICK, 0, 0);
See the following solution, also you can use
SendMessage(ButtonHandle, WM_LBUTTONDOWN, 0, 0);
SendMessage(ButtonHandle, WM_LBUTTONUP, 0, 0);
Or
SendMessage(ButtonHandle, BM_CLICK, 0, 0);
HWND buttonHandle = 0;
BOOL CALLBACK GetButtonHandle(HWND handle, LPARAM)
{
char label[100];
int size = GetWindowTextA(handle,label,sizeof(label));
if(strcmp(label,"&Save") == 0)
{
buttonHandle = handle;
return false;
}
return true;
}
void main()
{
HWND windowHandle = FindWindowA(NULL,"Do you want to Save?");
if(windowHandle != 0)
{
BOOL ret = EnumChildWindows(windowHandle,GetButtonHandle,0);
if(buttonHandle != 0)
{
LRESULT res = SendMessage(buttonHandle,BM_CLICK,0,0);
//SendMessage(buttonHandle,WM_LBUTTONDOWN,0,0);
//SendMessage(buttonHandle,WM_LBUTTONUP,0,0);
}
}
}
Note:
Getting sure from the window text,button text (check if there is space at the end of the window title)
SendMessage(btnHandle, WM_LBUTTONDOWN, 0, 0);
SendMessage(btnHandle, WM_LBUTTONUP, 0, 0);
SendMessage(btnHandle, WM_LBUTTONDOWN, 0, 0);
SendMessage(btnHandle, WM_LBUTTONUP, 0, 0);
You have to send a button click twice. Not sure why (maybe the first click only activates the window of the button), but I'm using this code for a long time and it always worked for me.
maybe this can help:
http://www.cplusplus.com/forum/beginner/8806/
Access Denied errors on SendMessage or PostMessage make no sense unless the process sending the message is running at a lower integrity level than the target process.
This should not be happening unless the process that owns the target window is being run "asAdministrator" or is a service. And its damn hard for services to create windows on the interactive desktop with Windows 6 and up.
You can do some reading about Integrity Levels Here if they apply even remotely to this situation. Internet Explorer is about the only other application that 'opts in' to the integrity security model by purposely lowering the integrity level of itself in order to sandbox itself more effectively.
If you can raise the window containing the button you can send raw mouse event to a position within the boundaries of button.
There are two function to simulate mouse event SendInput and mouse_event. I recommend using mouse_event function. To raise a window you can use ShowWindow. I don't know how to get the handle of a button, but if you have its hWnd its easy to find its absolute position using GetWindowRect function. Try using these, if you run into any problems I will be glad to help.
Or define a custom WM within your application window to handle save request. WM_CUSTOM or WM_USER (cant remember which) marks the start of user defined window messages.
When I have to do these kind of things I use SendKeys. It is VB-ish and C# provides a nice interface to use but for C/C++ you'll have to do it <this way>. What is nice with it is that you could write scripts and run them instead of hard coding it in your code.
Microsoft is now pushing Active Accessibility (MSAA) for UI Automation,
(It has been renamed a number of times over the years)
see
UI Automation and Microsoft Active Accessibility
Using UI Automation for Automated Testing
UI Automation Clients for Managed Code
Sorry I don’t have any simple code to get you started. As “SendMessage()” does not seem to be working for you, I don’t know of another option apart from “UI Automation”
I am assuming you have check with Spy++ (installed with MsDev) that you message are being send to the correct button etc – and that the button is a standard windows buttons. My first instant would say use “SendMessage()" or "PostMessage()” but given the numbers of answers about “SendMessage()” and the fact it is not working for you. I expect someone is going on…
//Send digit 4 to the already opened calc.exe
HWND windowHandle;
windowHandle = FindWindowA(NULL,"Calculator");
if(windowHandle != 0)
ret = EnumChildWindows(windowHandle,GetButtonHandle,0);
BOOL CALLBACK GetButtonHandle(HWND handle, LPARAM)
{
char label[100];
int size = GetWindowTextA(handle,label,sizeof(label));
if(strcmp(label,"4") == 0)
{
PostMessage(handle ,WM_LBUTTONDOWN,(WPARAM)0x0001,0);
PostMessage(handle ,WM_LBUTTONUP,(WPARAM)0x0001,0);
PostMessage(handle ,WM_LBUTTONDOWN,(WPARAM)0x0001,0);
PostMessage(handle ,WM_LBUTTONUP,(WPARAM)0x0001,0);
return false;
}
return true;
}
You can use sendkeys (as tr3 said) to send mouse clicks, which is different than using SendMessage. It is also less direct and more hack-ish, but is useful for automation (in VBS).
Also, just a guess but the problem could be that your message handling is broken somewhere by not calling the base class member. Example:
void CMyClass::OnMessageY(CWnd *cwnd)
{
CBaseClass::OnMessageY(cwnd);
//... my code
}
if you sure ButtonHandle are valid handle you can use pair WM_LBUTTONDOWN and WM_LBUTTONUP message instead of BN_CLICKED
HWND ButtonHandle;
if( (wnd = FindWindow(0, "Do you want to save?")) )
{
SendMessage(ButtonHandle, WM_LBUTTONDOWN, MK_LBUTTON, 0);
SendMessage(ButtonHandle, WM_LBUTTONUP, MK_LBUTTON, 0);
}
A non-C approach: Use Java and the java.awt.Robot class to move the mouse around perform real clicks (I guess there is something in the Windows World for this, too).
Problem: You have to know where your button is :D
My app. will be running on the system try monitoring for a hotkey; when the user selects some text in any window and presses a hotkey, how do I obtain the selected text, when I get the WM_HOTKEY message?
To capture the text on to the clipboard, I tried sending Ctrl + C using keybd_event() and SendInput() to the active window (GetActiveWindow()) and forground window (GetForegroundWindow()); tried combinations amongst these; all in vain. Can I get the selected text of the focused window in Windows with plain Win32 system APIs?
TL;DR: Yes, there is a way to do this using plain win32 system APIs, but it's difficult to implement correctly.
WM_COPY and WM_GETTEXT may work, but not in all cases. They depend on the receiving window handling the request correctly - and in many cases it will not. Let me run through one possible way of doing this. It may not be as simple as you were hoping, but what is in the adventure filled world of win32 programming? Ready? Ok. Let's go.
First we need to get the HWND id of the target window. There are many ways of doing this. One such approach is the one you mentioned above: get the foreground window and then the window with focus, etc. However, there is one huge gotcha that many people forget. After you get the foreground window you must AttachThreadInput to get the window with focus. Otherwise GetFocus() will simply return NULL.
There is a much easier way. Simply (miss)use the GUITREADINFO functions. It's much safer, as it avoids all the hidden dangers associated with attaching your input thread with another program.
LPGUITHREADINFO lpgui = NULL;
HWND target_window = NULL;
if( GetGUIThreadInfo( NULL, lpgui ) )
target_window = lpgui->hwndFocus;
else
{
// You can get more information on why the function failed by calling
// the win32 function, GetLastError().
}
Sending the keystrokes to copy the text is a bit more involved...
We're going to use SendInput instead of keybd_event because it's faster, and, most importantly, cannot be messed up by concurrent user input, or other programs simulating keystrokes.
This does mean that the program will be required to run on Windows XP or later, though, so, sorry if your running 98!
// We're sending two keys CONTROL and 'V'. Since keydown and keyup are two
// seperate messages, we multiply that number by two.
int key_count = 4;
INPUT* input = new INPUT[key_count];
for( int i = 0; i < key_count; i++ )
{
input[i].dwFlags = 0;
input[i].type = INPUT_KEYBOARD;
}
input[0].wVK = VK_CONTROL;
input[0].wScan = MapVirtualKey( VK_CONTROL, MAPVK_VK_TO_VSC );
input[1].wVK = 0x56 // Virtual key code for 'v'
input[1].wScan = MapVirtualKey( 0x56, MAPVK_VK_TO_VSC );
input[2].dwFlags = KEYEVENTF_KEYUP;
input[2].wVK = input[0].wVK;
input[2].wScan = input[0].wScan;
input[3].dwFlags = KEYEVENTF_KEYUP;
input[3].wVK = input[1].wVK;
input[3].wScan = input[1].wScan;
if( !SendInput( key_count, (LPINPUT)input, sizeof(INPUT) ) )
{
// You can get more information on why this function failed by calling
// the win32 function, GetLastError().
}
There. That wasn't so bad, was it?
Now we just have to take a peek at what's in the clipboard. This isn't as simple as you would first think. The "clipboard" can actually hold multiple representations of the same thing. The application that is active when you copy to the clipboard has control over what exactly to place in the clipboard.
When you copy text from Microsoft Office, for example, it places RTF data into the clipboard, alongside a plain-text representation of the same text. That way you can paste it into wordpad and notepad. Wordpad would use the rich-text format, while notepad would use the plain-text format.
For this simple example, though, let's assume we're only interested in plaintext.
if( OpenClipboard(NULL) )
{
// Optionally you may want to change CF_TEXT below to CF_UNICODE.
// Play around with it, and check out all the standard formats at:
// http://msdn.microsoft.com/en-us/library/ms649013(VS.85).aspx
HGLOBAL hglb = GetClipboardData( CF_TEXT );
LPSTR lpstr = GlobalLock(hglb);
// Copy lpstr, then do whatever you want with the copy.
GlobalUnlock(hglb);
CloseClipboard();
}
else
{
// You know the drill by now. Check GetLastError() to find out what
// went wrong. :)
}
And there you have it! Just make sure you copy lpstr to some variable you want to use, don't use lpstr directly, since we have to cede control of the contents of the clipboard before we close it.
Win32 programming can be quite daunting at first, but after a while... it's still daunting.
Cheers!
Try adding a Sleep() after each SendInput(). Some apps just aren't that fast in catching keyboard input.
Try SendMessage(WM_COPY, etc. ).