I have been struggling with NPAPI for "good" examples and "specific" documentations (on Unix). I have my libflashplayer.so, on which I can have access to certain information which is:
nm -D libflashplayer.so | more
00000000005cc4e0 T FlashPlayer_11_2_202_235_FlashPlayer
00000000005cc4a0 T Flash_DisableLocalSecurity
00000000005cc4c0 T Flash_EnforceLocalSecurity
00000000005ca870 T NP_GetMIMEDescription
00000000005ca850 T NP_GetValue
00000000005ca4d0 T NP_Initialize
00000000005ca4c0 T NP_Shutdown
I easily checked the MIMEDescription by doing this. Which return the flash MIME.
static char* (*mNP_GetMIMEDescription)(void);
//MAIN
lib = dlopen("./libflashplayer.so", RTLD_LAZY | RTLD_LOCAL);
*(void **) &mNP_GetMIMEDescription = dlsym(lib, "NP_GetMIMEDescription");
fprintf(stderr, "NP_GetMIMEDescription=%p\n", mNP_GetMIMEDescription);
fprintf(stderr, "NP_GetMIMEDescription=%s\n", mNP_GetMIMEDescription());
//output
NP_GetMIMEDescription=0x7f2d2239f870
NP_GetMIMEDescription=application/x-shockwave-flash:swf:Shockwave Flash;application/futuresplash:spl:FutureSplash Player
Now, I'm really lost on how to render (or test, open a swf file) content on a Xlib Window.
First, I do NP_INITIALIZE which I believe (from what I understand) provides Global Initialization (especially for NPNetscapeFuncs and NPPluginFuncs "instance"). Then the fun part begins, My guess (with what I read and understand, correct me if I am wrong) was to create an instance of my plugin (with NPP_New) and after, set my window (with NPP_SetWindow) to link my plugin instance and my window (from Xlib for example...)
This is exacly what I want to achieve.
http://blog.henryfan.net/post/2012/01/09/gtk%E6%92%AD%E6%94%BEflash.aspx
thanks to whoever wrote that down.
Related
Do graphics cards typically write their output to some place in memory which I can access? Do I have to use the driver? If so, can I use OpenGL?
I want to know if it's possible to "capture" the output of a VM on Linux which has direct access to a GPU, and is running Windows. Ideally, I could access the output from memory directly, without touching the GPU, since this code would be able to run on the Linux host.
The other option is to maybe write a Windows driver which reads the output of the GPU and writes it to some place in memory. Then, on the Linux side, a program can read this memory. This seems somewhat impossible, since I'm not really sure how to get a process on the host to share memory with a process on the guest.
Is it possible to do option 1 and simply read the output from memory?
I do not code under Linux but in Windows (you are running it in emulator anyway) you can use WinAPI to directly access canvas of any window or even desktop from 3th party App. Some GPU overlays could be problematic to catch (especially DirectX based) but I had no problems with mine GL/GLSL for now.
If you got access to App source you can use glReadPixels for image extraction from GL directly (but that works only for current GL based rendering).
Using glReadPixels
As mentioned this must be implemented directly in the targeted app so you need to have it source code or inject your code in the right place/time. I use for screenshoting this code:
void OpenGLscreen::screenshot(Graphics::TBitmap *bmp)
{
if (bmp==NULL) return;
int *dat=new int[xs*ys],x,y,a,*p;
if (dat==NULL) return;
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf32bit;
if ((bmp->Width!=xs)||(bmp->Height!=ys)) bmp->SetSize(xs,ys);
if ((bmp->Width==xs)&&(bmp->Height==ys))
{
glReadPixels(0,0,xs,ys,GL_BGRA,GL_UNSIGNED_BYTE,dat);
glFinish();
for (a=0,y=ys-1;y>=0;y--)
for (p=(int*)bmp->ScanLine[y],x=0;x<xs;x++,a++)
p[x]=dat[a];
}
delete[] dat;
}
where xs,ys is OpenGL window resolution, you can ignore the whole bmp stuff (it is VCL bitmap I use to store screenshot) and also can ignore the for it just copy the image from buffer to bitmap. So the important stuff is just this:
int *dat=new int[xs*ys]; // each pixel is 32bit int
glReadPixels(0,0,xs,ys,GL_BGRA,GL_UNSIGNED_BYTE,dat);
glFinish();
You need to execute this code after the rendering is done otherwise you will obtain unfinished or empty buffer. I use it after redraw/repaint events. As mentioned before this will obtain only the GL rendered stuff so if your App combines GDI+OpenGL it is better to use the next approach.
WinAPI approach
To obtain Canvas image of any window I wrote this class:
//---------------------------------------------------------------------------
//--- screen capture ver: 1.00 ----------------------------------------------
//---------------------------------------------------------------------------
class scrcap
{
public:
HWND hnd,hnda;
TCanvas *scr;
Graphics::TBitmap *bmp;
int x0,y0,xs,ys;
scrcap()
{
hnd=NULL;
hnda=NULL;
scr=new TCanvas();
bmp=new Graphics::TBitmap;
#ifdef _mmap_h
mmap_new('scrc',scr,sizeof(TCanvas() ));
mmap_new('scrc',bmp,sizeof(Graphics::TBitmap));
#endif
if (bmp)
{
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf32bit;
}
x0=0; y0=0; xs=1; ys=1;
hnd=GetDesktopWindow();
}
~scrcap()
{
#ifdef _mmap_h
mmap_del('scrc',scr);
mmap_del('scrc',bmp);
#endif
if (scr) delete scr; scr=NULL;
if (bmp) delete bmp; bmp=NULL;
}
void init(HWND _hnd=NULL)
{
RECT r;
if (scr==NULL) return;
if (bmp==NULL) return;
bmp->SetSize(1,1);
if (!IsWindow(_hnd)) _hnd=hnd;
scr->Handle=GetDC(_hnd);
hnda=_hnd;
resize();
}
void resize()
{
if (!IsWindow(hnda)) return;
RECT r;
// GetWindowRect(hnda,&r);
GetClientRect(hnda,&r);
x0=r.left; xs=r.right-x0;
y0=r.top; ys=r.bottom-y0;
bmp->SetSize(xs,ys);
xs=bmp->Width;
ys=bmp->Height;
}
void capture()
{
if (scr==NULL) return;
if (bmp==NULL) return;
bmp->Canvas->CopyRect(Rect(0,0,xs,ys),scr,TRect(x0,y0,x0+xs,y0+ys));
}
};
//---------------------------------------------------------------------------
//---------------------------------------------------------------------------
//---------------------------------------------------------------------------
Again it uses VCL so rewrite bitmap bmp and canvas scr to your programing environment style. Also igneore the _mmap_h chunks of code they are just for debugging/tracing of memory pointers related to some nasty compiler bug I was facing at that time I wrote this.
The usage is simple:
// globals
scrcap cap;
// init
cap.init();
// on screenshot
cap.capture();
// here use cap.bmp
If you call cap.init() it will lock on the whole windows desktop. If you call cap.init(window_handle) it will lock on specific visual window/component instead. To obtain window handle from 3th app side see:
is ther a way an app can display a message without the use of messagebox API?
Sorry it is on SE/RE instead of here on SE/SO but My answer here covering this topic was deleted. I use this for video capture ... all of the animated GIFs in my answers where created by this code. Another example could be seen on the bottom of this answer of mine:
Image to ASCII art conversion
As you can see it works also for DirectX overlay with media player classic (even Windows PrintScreen function cant do it right). As I wrote I got no problems with this yet.
Beware visual stuff WinAPI calls MUST BE CALLED FROM APP MAIN THREAD (WNDPROC) otherwise serious problems could occur leading to random unrelated WinAPI calls exceptions anywhere in the App.
I am learning to use glade 3 to create a GUI.
However, the *.glade file is an xml file. I am not sure how to go forward from here. Google search is not really helping. There is a question already asked for same thing here Tool to convert .Glade (or xml) file to C source . However I am not really able to understand the answer given in that.
Can someone tell the basic flow of the development cycle using glade 3?
Design the UI in glade.
Generate the *.glade xml file.
AND THEN WHAT ????
How can the xml file be converted to an executable ?
A. Should I convert this xml file to a language (C) and compile the C code ?
B. Or is there a way for xml code to be directly converted to an ELF executable ?
I am trying to make the GUI for my own use. I use linux and want an ELF executable (like how I would get if I wrote the C code using gtk library and compiled it using gcc).
If we look at the Wikipedia page for Glade it has an entire section about how to use Glade in a program: with GtkBuilder. Now all that remains is to read the docs and you can begin using Glade. No offense, but I've never used Glade before and this is fairly clear in all docs. For example, here's Glade's homepage.
I would do something like this:
DerivedWindow::DerivedWindow()
{
mainBox = Gtk::manage(new Gtk::Box(Gtk::ORIENTATION_VERTICAL, 7));
builder = Gtk::Builder::create();
try {
builder->add_from_file("filename.glade");
} catch (Glib::Error& ex) {
errMsg("Window Builder Failed: " + ex.what());
}
Gtk::Box* box;
builder->get_widget("name of box inside main window", box);
if (!box) { this->destroy_(); return; }
box->unparent();
mainBox->pack_start(*box, Gtk::PACK_SHRINK);
//optional - if you want full access to particular widgets
builder->get_widget("name of widged id", widgetname);
//connect signals here...
add(*mainBox);
show_all();
}
Note this is Gtkmm 3+.
It is important you unparent the box you got from the glade file, so you can attach it to your derived window.
I want to write an application that will automatically detect and fill the text field in the window shown below:
(assuming the data to be entered is in a file).
The question is how does my application find this text field?
I can do this job if I am able to find the location of the text field on the desktop through program.
Can someone help me understand possible ways for finding this text field?
I am using Windows Form application in C++.
Update:
I played with spy++.
I used spy++, to find the window handle. I did it by putting finder on the window I am interested in. Its giving handle in hex values: 00080086 (actually just for testing purpose I put the finder tool on Visual Studio new project page ). How do I interpret this Hex value into meaningful window name ?
See the below figure.
What is the next step to get to the text field " Enter name" under "name" field.
****Any sample code will be highly appreciated.**
I am open to any solution not necessarily how I am doing this.
One solution is to use the Microsoft UI Automation technology. It's shipped out-of-the-box with Windows since Vista. It's usable from .NET but also from C++ using COM.
Here is a short C++ console application example that displays the class name of the UI Automation Element currently at the middle of the desktop window, each second (you can have it run and see what it displays):
int _tmain(int argc, _TCHAR* argv[])
{
CoInitialize(NULL);
IUIAutomation *pAutomation; // requires Uiautomation.h
HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (LPVOID *)&pAutomation);
if (SUCCEEDED(hr))
{
RECT rc;
GetWindowRect(GetDesktopWindow(), &rc);
POINT center;
center.x = (rc.right - rc.left) / 2;
center.y = (rc.bottom - rc.top) / 2;
printf("center x:%i y:%i'\n", center.x, center.y);
do
{
IUIAutomationElement *pElement;
hr = pAutomation->ElementFromPoint(center, &pElement);
if (SUCCEEDED(hr))
{
BSTR str;
hr = pElement->get_CurrentClassName(&str);
if (SUCCEEDED(hr))
{
printf("element name:'%S'\n", str);
::SysFreeString(str);
}
pElement->Release();
}
Sleep(1000);
}
while(TRUE);
pAutomation->Release();
}
CoUninitialize();
return 0;
}
From this sample, what you can do is launch the application you want to automate and see if the sample detects it (it should).
You could also use the UISpy tool to display the full tree of what can be automated in your target app. You should see the windows and other elements (text field) of this target app and you should see the element displayed by the console application example.
From the pElement discovered in the sample, you can call FindFirst with the proper condition (class name, name, control type, etc...) to get to the text field. From this text field, you would use one of the UI Automation Patterns that should be available (probably TextPattern or ValuePattern) to get or set the text itself.
The cool thing is you can use the UISpy tool to check all this is possible before actually coding it.
You could enumerate windows and then find it.
For exploring application on your screenshot you could you Spy++ (spyxx.exe) that is distributed with visual studio. In you code, you clould use EnumWindows and EnumChildWindows to enumerates all window or all child windows to find one you need.
Although the answer given by Simon is accepted and is the best one, but still for future visitors I am providing this link which has more description for UI automation of windows applications. .
Also for automating a web application one may want to go to this link
Can GTK#'s FileChooserDialog be used as a unified file/URI dialog? I'd like it to accept http/https/ftp URIs without "rewriting" them (prepending local directory).
Even if I set LocalOnly=false and paste a http://.... uri into the text box inside the filechooser, I cannot get the original entry. Local directory is always prepended to the text.
I've done some research, and I don't think it's possible. At least not with the direct native C GTK+ API, which is what I tested.
In my testing, I always either got the local directory's path prepended to the http:// URI I had entered in the dialog, or I got back (null). I did call the get_uri() method, not just get_filename().
I also took a quick look, as a reference, at the GIMP application's File menu. As you probably know, GIMP provides the G in GTK+, so it can sometimes be used as a reference for ideas on how to use the toolkit. GIMP does not try to support URIs entered in the file chooser dialog, instead it has a dedicated Open Location command, that opens a simple dialog with just a GtkEntry.
I think you need to set local-only to FALSE and then use the GIO get_file ()/get_files () calls which return a GFile* accessible through the GIO File API and therefore through gvfs.
I found a solution / hack after all (in C#):
private string _extractUri(Widget wi) {
if (wi is Entry)
return ((wi as Entry).Text);
else if (wi is Container) {
foreach (Widget w in (wi as Container).Children) {
string x = _extractUri(w);
if (x!=null)
return x;
}
}
return null;
}
I'm not sure if that's always safe, but it worked for the standard FileChooserDialog. It will return the original string from the input field - even if standard Uri / File results are mangled.
Our application has a file format similar to the OpenDocument file format (see http://en.wikipedia.org/wiki/OpenDocument) - i.e. zipped with a manifest file, a thumbnail image, etc.
I notice that OpenOffice files have a preview image of the Open Office file as their icons, both in Windows and in Linux. Is there some way to accomplish this for our files: i.e. I want a dynamic icon based on the internal thumbnail.png?
Edit 1 Wow, thanks for all the quick answers. Thumbnailer looks great for the GNOME world. Windows I'll be looking into those links, thanks. As for the comment question: programmatically OR via our installer.
Edit 2 Oh, forgot Mac. How about on the Mac? (Sorry Mac lovers!) Also are there any links or info for how OpenOffice does their IconHandler stuff - since ours would be very similar?
Windows
What you need is an Icon Handler, also known as a Thumbnail Handler. Here is an example written as an active x control.
Another resource is to look up Property Handlers, which should also point to you to the latest and greatest way of having dynamic meta data handled correctly in windows.
These are dynamic solutions - they aren't needed if you just want an icon associated with all your files - they are only used when you want windows explorer to display an icon based on what's in the file, not just the extension, and when the file changes the icon is updated to reflect the changes. It doesn't have to be an image of the file itself, either, the thumbnail handler can generate any image based on the file contents.
The property handler updates other metadata, such as song or video length, so you can use all the metadata Windows Explorer supports.
Regarding MAC support, this page says, "The Mac and Windows operating systems have different methods of enabling this type of thumbnail, and in the case of the Mac OS, this support has been inconsistent from version to version so it hasn't been pursued [for Adobe InDesign]."
OS X
Icons for Mac OSX are determined by the Launch Services Database. However, it refers to a static icon file for all files handled by a registered application (it's not based on extension - each file has meta data attached that determines the application to which it belongs, although extensions give hints when the meta data doesn't exist, such as getting the file from a different OS or file system)
It appears that the dynamic icon functionality in OSX is provided by Finder, but searches aren't bringing up any easy pointers in this direction. Since Finder keeps changing over time, I can see why this target is hard to hit...
Gnome
For Gnome you use a thumbnailer. (thanks Dorward)
This is an extraordinarily simple program you write, which has 3 command line arguments:
input file name, the file you are describing with the thumbnail (or URI if you accept those instead)
output file name, where you need to write the PNG
size, a number, in pixels, that describes the maximum square image size you should produce (128 --> 128x128 or smaller)
I wish all systems were this simple. On the other hand this doesn't support animation and a few other features that are provided by more difficult to implement plugins on other systems.
KDE
I'm a bit uncertain, but there are a few pointers that should get you started. First is that Konqueror is the file manager and displays the icons - it supports dynamic icons for some inbuilt types, but I don't know if these are hardcoded, or plugins you can write. Check out the Embedded Components Tutorial for a starting point.
There's a new (ish?) feature (or planned feature...) called Plasma which has a great deal to do with icons and icon functionality. Check out this announcment and this initial implementation.
You may need to dig into the source of Konqueror and check out how they did this for text files and others already implemented.
-Adam
Mac OSX since version 10.5 …
… has two approaches:
Your document is in the standard OSX bundle format and has a static image
This can be done by creating a subfolder QuickLook and placing the Thumbnail/Preview.png/tiff/jpg inside.
Everything else needs a QuickLook generator plugin which can be stored in either /Library/QuickLook ~/Library/QuickLook or inside the YourApp.app/Contents/Library/QuickLook Folders.
This generator is being used to create Thumbnails and QuickLook previews on the fly. XCode offers a template for this. The template generates the needed ANSI C files which have to be implemented. If you want to write Object-C code you have to rename the GenerateThumbnailForURL.c and GeneratePreviewForURL.c to GenerateThumbnailForURL.m and GeneratePreviewForURL.m (and read the Apple Devel Docs carefully ;) )
Simple zip container based demo:
You will have to add the Cocoa.framework and Foundation.framework to your project
In your GenerateThumbnailForURL.c (this is partly out of my head - so no guarantee that it works out of the box ;) ):
#include <Cocoa/Cocoa.h>
#include <Foundation/Foundation.h>
OSStatus GenerateThumbnailForURL(void *thisInterface, QLThumbnailRequestRef thumbnail, CFURLRef url, CFStringRef contentTypeUTI, CFDictionaryRef options, CGSize maxSize)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
/* unzip the thumbnail and put it into an NSData object */
// Create temporary path and writing handle for extraction
NSString *tmpPath = [NSTemporaryDirectory() stringByAppendingFormat: [NSString stringWithFormat: #"%.0f.%#" , [NSDate timeIntervalSinceReferenceDate] * 1000.0, #"png"]];
[[NSFileManager defaultManager] createFileAtPath: tmpPath contents: [NSData alloc] attributes:nil];
NSFileHandle *writingHandle = [NSFileHandle fileHandleForWritingAtPath: tmpPath];
// Use task to unzip - create command: /usr/bin/unzip -p <pathToFile> <fileToExtract>
NSTask *unzipTask = [[NSTask alloc] init];
[unzipTask setLaunchPath: #"/usr/bin/unzip"];
// -p -> output to StandardOut, added File to extract, nil to terminate Array
[unzipTask setArguments: [NSArray arrayWithObjects: #"-p", [(NSURL *) url path], #"Thumbnails/thumbnail.png", nil]];
// redirect standardOut to writingHandle
[unzipTask setStandardOutput: writingHandle];
// Unzip - run task
[unzipTask launch];
[unzipTask waitUntilExit];
// Read Image Data and remove File
NSData *thumbnailData = [NSData dataWithContentsOfFile: tmpPath];
[[NSFileManager defaultManager] removeFileAtPath: tmpPath handler:nil];
if ( thumbnailData == nil || [thumbnailData length] == 0 ) {
// Nothing Found. Don't care.
[pool release];
return noErr;
}
// That is the Size our image should have - create a dictionary too
CGSize size = CGSizeMake(256, 256);
NSDictionary *properties = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:size.width],kQLPreviewPropertyWidthKey,
[NSNumber numberWithInt:size.height],kQLPreviewPropertyHeightKey,
nil];
// Get CGContext for Thumbnail
CGContextRef CGContext = QLThumbnailRequestCreateContext(thumbnail, size, TRUE, (CFDictionaryRef)properties);
if(CGContext) {
NSGraphicsContext* context = [NSGraphicsContext graphicsContextWithGraphicsPort:(void *)CGContext flipped:size.width > size.height];
if(context) {
//These two lines of code are just good safe programming…
[NSGraphicsContext saveGraphicsState];
[NSGraphicsContext setCurrentContext:context];
NSBitmapImageRep *thumbnailBitmap = [NSBitmapImageRep imageRepWithData:thumbnailData];
[thumbnailBitmap draw];
//This line sets the context back to what it was when we're done
[NSGraphicsContext restoreGraphicsState];
}
// When we are done with our drawing code QLThumbnailRequestFlushContext() is called to flush the context
QLThumbnailRequestFlushContext(thumbnail, CGContext);
// Release the CGContext
CFRelease(CGContext);
}
[pool release];
return noErr;
}
Info.plist
You will have to modify your info.plist file too - when you open it up it has a lot of fields pre-set. Most of them are self-explaning (or will not have to be changed) but I had to add the following structure (copy paste should do - copy the text, go into the plist editor and just paste.):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
<dict>
<key>UTTypeConformsTo</key>
<array>
<string>com.pkware.zip-archive</string>
</array>
<key>UTTypeDescription</key>
<string>i-net Crystal-Clear Report File</string>
<key>UTTypeIconName</key>
<string>generic</string>
<key>UTTypeIdentifier</key>
<string>com.company.product</string>
<key>UTTypeReferenceURL</key>
<string>http://your-url.com</string>
<key>UTTypeTagSpecification</key>
<dict>
<key>public.filename-extension</key>
<array>
<string>$fileEXT$</string>
</array>
</dict>
</dict>
</array>
</plist>
This will register your filetype $fileExt$ and tell the system that your filetype is a zipy format type. A nice refference, that I used here is the QuickLook IPA Plugin from googlecode
In Windows, what you need is to implement an Icon Handler. I did this many moons ago and it is not difficult as long as you know the basics of COM.
See: http://msdn.microsoft.com/en-us/library/bb776857(VS.85).aspx
For Gnome you use a thumbnailer.
for WINDOWS try this:
http://www.easydesksoftware.com/news/news12.htm
Executables have the icon inside the file (potentially multiple) as a "resource".
Data files pick up an icon based on file association.
If you want a custom icon per file that is much harder. you either need too fool the OS into thinking it is an executable and embed the icon as a resource in the file, or deep link into the OS to override the default icon selection routine.
I think, "custom own" icon can have only PE files in windows. Every other icons for file extensions are stored in windows registry.
For specification of PE file, you can look at An In-Depth Look into the Win32 Portable Executable File Format and Peering Inside the PE: A Tour of the Win32 Portable Executable File Format.
How it works in other OS, I don't know :/.
I don't know about Linux, but for Windows you can start here:
http://msdn.microsoft.com/en-us/library/bb774614.aspx
Edit: I think this interface is for the thumbnails shown in thumbnail view, not icons. Sorry for wasting your time.