I'm learning Chinese at the moment and I have gotten my hand on a Chinese dictionary definition.
Now I would like to make an interface.
All I really want the application to do is when I point my mouse pointer over any text on the screen (in any window), it would identify the text I am pointing at and then display a small form over it, which would the chinese transaction.
Is that possible to do? Can WinForms apps interact with windows outside of it's own application?
In C# you can get text under mouse cursor by P/Invoking
GetCursorPos
GetClassName
SendMessage
WM_GETTEXT
WM_GETTEXTLENGTH
WindowFromPoint
Like mentioned here
here is another example in C++
A WinForms application can interact with the Windows of other applications. Window handles exist in a global namespace so if you can get the handle of another application's window you can send it messages. You will have to use pinvoke to do some of this, have a look at WindowFromPoint
However, there is no standardized way to display text in a window; there are dozens of APIs for displaying text. So when you point at text with a mouse, you can only get the pixels, but not necessarily the text.
Some window classes will allow you to send class-specific messages query for the text at a specific location, but many will not. Your best bet is probably to use the same methods that screen readers for the blind use http://en.wikipedia.org/wiki/Screen_reader
Related
From what I understand, a GUI will have its windows, window classes, and use these for the main windows and all the buttons and tabs etc.
These would all have handles and be rendered either with the Windows GDI or another backend such as OpenGL. When a user interacts, say by clicking on a widget, there will be a callback function/event handler and it'll do its job. But what is happening when the user clicks on a button that switches the (I'm not sure what to call this so I'll call it a "form" - by this I mean the visible set of all menus and widgets and things - like on Google Chrome I have this tab open right now and I could move to another one that displays a different website and GUI) form.
How does the GUI framework change all the windows on the screen? I can understand it could change what's being rendered with the API of choice, like OpenGL, but how does it get rid of all the old windows and load the new ones? Does it disable all the child windows through their handles, and just leave them there on the screen, but unseen and not accepting input? Does it delete everything and create new windows? How does it actually perform this change (efficiently too)? I may be making a mountain out of a molehill here - if I'm overthinking this please let me know!
I once made a very bad game, using c Win32, the GDI and Direct2D, and when you pressed "play" it'd go to the game, but I just had to hide the buttons in a very glitchy fashion - I had no clue how to perform the "switch."
I have never ever used a "proper" GUI framework like Qt nor have I ever built one myself so apologies for any errata in the question, please correct me. I ask because I want to make my own GUI framework as a long term project (nothing special just something I can say that I've achieved) and I am at a loss as to how I can implement this from a low-level perspective, or rather how industry standards such as Qt will implement this at the lowest possible level.
Any answers would preferably not refer to managed code or any scripting languages, or external libraries - I want to know how to do this in c Win32 + any arbitrary graphics API. Thanks in advance.
This is accomplished by altering the z-order (the idea being that the windows form a stack from closest to the user to furthest away) of children at the appropriate level. The direct children of every window are in some z-order even if they are arranged such that they don't actually overlap.
For example, in the case of a tab control there will likely be a single child associated with each tab, that child representing the view for that tab. When a button is clicked the child for that tab is moved in the z-order so that is above all of its siblings (the forms for the other tabs). Those windows for the tab children will all be the same size (the empty area of the tab's client window) so bringing the child to the top of its parent's z-order will cover all other views.
In the case of the window's API you alter z-order placement via SetWindowPos, if you are going to roll your own (as WPF does) then you will need to re-implement this idea in some manner.
I am developing a CAD like application. This application is cross platform in the sense that I have a main window which is native to the platform it is running on (Linux, Windows and Apple). Within this window I have an OpenGL context and there I do all my rendering. The application in question does not really rely on common controls (radio buttons, check-boxes, labels etc), however there is the need for the user to be able to enter/edit some text. This text could be a few lines long but not more than one hundred.
How would I go about implementing such a control, I don't want to go as deep as developing my own text editor but it would be nice if it had some basic editor controls (cursor movement, delete, insert etc). I also don't want to use the native systems common controls as my own none cross platform code at the moment is the main window.
Any ideas?
Edit: This is informative
OPENGL User Interface Programming
Thank you
I think you are taking a longer approach here.
Rather than providing an interface through OpenGL and writing my own controls, I would go for a cross-platform GUI toolkit such as wxWindows, and use the GLCanvas provided to do my rendering. You'll have all the might and flexibility of common controls, you'll still be able to OpenGLize whatever you want, and the look of your application will be more standard, thus friendlier.
I have a binary application on windows (train timetable software) which contains a lot of interresting data which I need for my project (nothing illegal, just some weird optimization algorithm). But the application has no api and the data files have undocumented binary form.
So my idea is to control the application from my own code. I would like to send keystrokes to it to fill a form, run query and save result to a file (there are buttons and menu items for this in the app). And repeat many times.
Is there a library for this? Or an example? I have a general idea how to do it, but I am lazy and I do not want to invent the wheel.
Also, the same data is available on the web. Is there some solution for the same task with ASP (Win forms) web applications? I could probably handle parsing the results but I do not know how to fill the values of webforms controls.
Thanks in advance.
You can use simple Win32 APIs to do this.
FindWindowEx and then once you have the window handle you can send any message (such as WM_KEYDOWN) to it by using SendMessage.
A good tool which helps with this process is Spy++ because it allows you to see the window hierarchy more easily and also which messages are being used internally for the application you are monitoring.
As for web form controls, you will probably have to do more work than this because typically the web will be one canvas control that things are drawn custom onto depending on the browser. Perhaps doing this via some kind of proxy is a better approach where you actually filter the HTML pages.
I am working on a demo application for a library me and two colleagues are writing to allow GNOME applications that run audio events though libCanberra to allow users to select visual events to replace them. This is an accessibility-minded effort to help both visually and aurally impaired users gain the benefits of audio alerts and such.
For our first demo we're simply trying to make the entire screen flash with a color when a button is pressed in our simple GTK sample app. I've been looking at the GTK documentation and all drawing that I've seen has had to do with drawing directly to a window or other widget. I want to control the entire screen's hue. Would this be a GDK thing? Am I completely off base?
Any links/help will be much appreciated! Thanks.
PS: This is being written in C, though functions should be the same between languages with proper bindings, I assume.
You cannot. Your application has access only to its own window, and does not (and should not) know anything about other windows, or the screen. The "screen" is managed by whatever back-end GTK uses (X? Win32? DirectFB?).
That said, you could try to create a "full-screen" window that covers the entire screen area. That is the way full-screen apps are implemented in most windowing systems.
GTK doesn't have such option AFAIK, you probably want to use the backend: Xlib (or Xcb) for that.
When I try to use Morae Manager 3.0 to run a Screen Text search on a recording of our prototype (coded in WPF, .NET version 3.5 SP1), the only things that come up are the window titles. We are using Windows XP.
Even when I search for something that is editable, like text typed into a text box, it does not come up.
Screen text for things outside the prototype (e.g. desktop icons) still comes up perfectly.
I contacted TechSmith support on two separate occassions and both times the reply I got was it must be an issue with our technology, since the screen text search does work for other things, and that the support people are not developers and thus do not know what might be causing this.
Does anyone know:
what precisely might be causing this -- e.g. does WPF's rendering engine bypass some sort of Windows layer where Morae looks for text (please forgive me for any errors in terminology)
if there is anything I can tweak in the prototype to fix it
how I can get through to someone at TechSmith who knows the answers to 1. and 2.
P.S. Morae is a wonderful product and we've usually had great support from TechSmith. We are only having problems with this one little thing, and one can hardly blame Morae for not being compatible with something as new as WPF.
I have no idea what Morae is or how it works, but one big difference between WPF and for example WinForms is the following, copied from here:
When you create a WPF Window, WPF
creates a top-level HWND, and uses an
HwndSource to put the Window and its
WPF content inside the HWND. The rest
of your WPF content in the application
shares that singular HWND.
If Morae depends on HWNDs to find texts on the screen, this could explain why it can not find the text box. In contrast, i believe in WinForms every control (button, textbox) has its own HWND.