GTK+3 replacing/swapping pixbuf seems to break - c

I have a simple GTK3 grid layout using images as buttons and I want to replace the images when various things happen.
What's weird is that this works fine if I replace/remove the image from the same piece of code each time but not when I alternate between the two different sources despite (as far as I can see) everything being almost identical.
Anyway, here's my code...
Initial setup - create pixbuf, make it an image, attach it to the button - this works fine:
/*
* Initial setup
*
* dbp is a pointer to a struct that contains the button, the pixbuf, etc.
*/
// Create new blank (&transprent) image
dbp->pixbuf = gdk_pixbuf_new(GDK_COLORSPACE_RGB, TRUE, 8, innerButton.width, innerButton.height);
gdk_pixbuf_fill(dbp->pixbuf, 0x00000000);
// Image holds pixbuf
dbp->image = (GtkImage*)gtk_image_new_from_pixbuf(dbp->pixbuf);
// Attach image to button
gtk_button_set_image(GTK_BUTTON(dbp->btn), GTK_WIDGET(dbp->image));
// Attach button to grid
gtk_grid_attach(GTK_GRID(_grid), dbp->btn, c, r, 1, 1);
Clear image code - blanks the image (transparent):
gdk_pixbuf_fill(dbp->pixbuf, 0x00000000);
Button refresh callback (updates image from pixbuf):
if(dbp->image_modified != 0)
{
gtk_image_set_from_pixbuf(GTK_IMAGE(dbp->image), dbp->pixbuf);
dbp->image_modified = 0;
}
1st image replacement code - takes data input in our own format & creates new pixbuf:
GdkPixbuf *newpixbuf = gdk_pixbuf_new(GDK_COLORSPACE_RGB, TRUE, 8, BITMAP_WIDTH, BITMAP_HEIGHT);
g_object_unref(dbp->pixbuf);
// <Data unpacked to pixels here, code removed for clarity>
dbp->pixbuf = gdk_pixbuf_copy(newpixbuf);
gdk_pixbuf_copy_options(newpixbuf , dbp->pixbuf);
dbp->image_modified = 1; // Trigger refresh
2nd image replacement - loads image from file and creates new pixbuf:
GdkPixbuf *newpixbuf = gdk_pixbuf_new_from_file_at_scale (file_path, BITMAP_WIDTH, BITMAP_HEIGHT, FALSE, &err);
// [Error check code removed for clarity]
g_object_unref(dbp->pixbuf);
dbp->pixbuf = gdk_pixbuf_copy(newpixbuf );
gdk_pixbuf_copy_options(newpixbuf , dbp->pixbuf);
dbp->image_modified = 1; // Trigger refresh
Now, if I do repeated calls to these routines I get odd interactions;
If I do clear, image_replace_1, clear, image_replace_1, clear, image_replace_1 etc... it works absolutely perfectly.
If I do clear, image_replace_2, clear, image_replace_2, clear, image_replace_2 etc... it works absolutely perfectly.
However, when I do:
If I do clear, image_replace_1, clear, image_replace_2, clear, image_replace_1... it falls over with complaints that GDK_IS_PIXBUF(pixbuf) failed and I can't for the life of me work out how that happens?
I have also tried this alternative code for the button refresh callback;
dbp->image = (GtkImage*)gtk_image_new_from_pixbuf(dbp->pixbuf);
gtk_button_set_image(GTK_BUTTON(dbp->btn), GTK_WIDGET(dbp->image));
But the result is the same.

Related

Simple GTK/GDK app very slow to update/refresh window & buttons

Apologies in advance, I'm not a GTK/GDK master and have been feeling my way round some code written by someone else who's no longer around.
Edited to add TL;DR - Full question below with some detail.
The TL;DR is that gtk_button_set_image seems to take ~1ms, multiplied by 50 buttons that causes a bottleneck when changing the image on every button in our window.
EDIT again to add timings for the various calls:
Call Time (ms) approximate
gtk_button_set_image 1ms
gtk_button_set_label 0.5
gdk_pixbuf_scale 0.5
recolour_pixbuf (re-written) 0.5
gdk_pixbuf_new_subpixbuf 0.4
gtk_css_provider_load_from_data 0.3
gtk_image_new_from_pixbuf 0.15
BuildButtonCSS 0.01
And yes, recolour_pixbuf() takes a long time but I can find no other way of colour-swapping pixels in a pixbuf other than going through the whole thing pixel-by-pixel.
This adds up to GTK/GDK calls taking ~2.35ms to update each button each time. I have refactored my code to check what's changed and ONLY execute necessary changes - but even then, the whole window is fairly regularly updated with new images for every button plus new colours etc. so it's not an edge case and it is noticeable.
Basically we have a pretty simple GTK C app, just a window with a grid of buttons in it. A TCP socketed connection sends messages to the app to (for example) change the label or colour of a button, and we send messages back when a button is pushed.
However, with 100ms polling on the main loop for refreshing/re-drawing the buttons it seems to be taking a very long time to refresh the window.
I'll try to keep this sane + readable - I can't really post a minimal working example (it would be huge) but I'll try and break down the basics of the code so you can see what's done.
Each button is a widget that can contain a straight text label or instead be an image created from a pixbuf.
Each button is attached to a grid, the grid is inside a window.
Hopefully this is sensible and obvious so far.
In our main application we have a check that happens every 100ms which will run through all the button data, and for any that have changed (EG new label or new pixbuf) it will update the button accordingly.
g_timeout_add(100, (GSourceFunc)check_refresh, _context->refresh);
The code then happening for each button (including the timestamps I've added to get debug info) is:
static void refresh_button(int buttonId)
{
char name[12];
snprintf(name, 10, "BTN_%02d", buttonId);
int bid = buttonId-1;
char tstr[VERY_LONG_STR];
struct dev_button *dbp;
dbp = &_context->buttons[bid];
// For debug timestamps:
struct timespec start, stop;
double result;
clock_gettime(CLOCK_MONOTONIC, &start);
if(dbp->css_modified != 0)
{
BuildButtonCSS(dbp, tstr, NULL);
gtk_css_provider_load_from_data(dbp->bp, tstr, -1, NULL);
if(dbp->text[0] != '\0')
{
gtk_button_set_label(GTK_BUTTON(dbp->btn), dbp->text);
}
else
{
gtk_button_set_label(GTK_BUTTON(dbp->btn), NULL);
}
}
clock_gettime(CLOCK_MONOTONIC, &stop);
result = ((stop.tv_sec - start.tv_sec) * 1e3) + ((stop.tv_nsec - start.tv_nsec) / 1e6); // in milliseconds
g_message("[BRF] %s took %.3fms to here", name, result);
/*
* CSS changes affect button image drawing (cropping etc.)
*/
if(dbp->image_modified != 0 || dbp->css_modified != 0)
{
uint8_t b = dbp->bpx; // Border in pixels
GdkPixbuf* tmp = gdk_pixbuf_new_subpixbuf (dbp->pixbuf, b, b, _context->innerButton.width - (b * 2), _context->innerButton.height - (b * 2));
dbp->image = (GtkImage*)gtk_image_new_from_pixbuf(tmp);
gtk_button_set_image(GTK_BUTTON(dbp->btn), GTK_WIDGET(dbp->image));
}
btn_timediff(buttonId);
clock_gettime(CLOCK_MONOTONIC, &stop);
result = ((stop.tv_sec - start.tv_sec) * 1e3) + ((stop.tv_nsec - start.tv_nsec) / 1e6); // in milliseconds
g_message("[BRF] %s took %.3fms for update", name, result);
dbp->css_modified = 0;
}
So I'm timing the milliseconds taken to update the button from CSS, then the time to have updated the image from pixbuf - running on a Raspberry Pi CM4 I'm getting results like this:
** Message: 10:25:22.956: [BRF] BTN_03 took 1.443ms to here
** Message: 10:25:22.959: [BRF] BTN_03 took 5.061ms for update
So around ~1.5ms to update a button from simple CSS, and ~3.5ms to update a button image from a pixbuf.
And before you say the Raspberry Pi is slow - even on a full fat Linux desktop machine I'm seeing similar timings - a little faster on average but sometimes the total can be beyond 10ms for a single button.
This feels very slow to me - almost like there's something blocking on screen refresh after each change to each button. I wonder if we're going about this wrong, perhaps we should be somehow inhibiting re-draws of the window until we get to the last button and then let the whole thing re-draw once?
As I said - I'm not experienced with GTK and am a bit in at the deep end on this project so may well be doing this all wrong or totally missing some obvious method or call or something.

Make grid of buttons fixed width & height, and wrap text

We have a GTK3 user interface (coded in C) in an embedded system that must stay fixed width/height in order to fit the screen.
Basically it's a fixed grid of buttons on a touchscreen, where the buttons can contain a text label or a graphical image - and the grid MUST stay fixed, all buttons the same width & height, and fitting the screen 100%.
However, on updating button text/labels with longer strings the button expands to fit which is not what we want at all.
I can't post all the code but I believe this is the relevant button creation code that would affect what we're trying to do;
// Number buttons 1..n
buttonId = n;
bid = buttonId - 1; // Because zero-indexing, natch
dbp = &_context->buttons[bid]; // Data about button
// Create new button, attach to global buttons
dbp->btn = gtk_button_new();
// Button name != Button CSS#ID
snprintf(value, SHORT_STR, "BTN_%02d", buttonId );
gtk_widget_set_name(dbp->btn, value);
gtk_button_set_relief(GTK_BUTTON(dbp->btn), GTK_RELIEF_NORMAL/*GTK_RELIEF_NONE*/);
// Set text label
strncpy(dbp->text, value, MAX_BUTTON_TEXT);
gtk_button_set_label (GTK_BUTTON(dbp->btn), dbp->text);
// Attempt to make labels wrap - does nothing
label = gtk_bin_get_child(GTK_BIN(dbp->btn));
gtk_label_set_line_wrap(GTK_LABEL(label), TRUE);
//gtk_label_set_max_width_chars(GTK_LABEL(label), 0);
// Create new blank (&transprent) image
dbp->pixbuf = gdk_pixbuf_new(GDK_COLORSPACE_RGB, TRUE, 8, _context->innerButton.width, _conte$
gdk_pixbuf_fill(dbp->pixbuf, 0x00000000);
// Image holds pixbuf
dbp->image = (GtkImage*)gtk_image_new_from_pixbuf(dbp->pixbuf);
dbp->image_modified = 0;
// Attach image to button
gtk_button_set_image(GTK_BUTTON(dbp->btn), GTK_WIDGET(dbp->image));
// Adds an individual CSS provider per button to allow style changes
GtkStyleContext *context = gtk_widget_get_style_context(dbp->btn);
dbp->bp = gtk_css_provider_new();
// Set default colour from table of default colours
dbp->bg = get_default_colour(bid);
dbp->fg = get_label_colour(dbp->bg);
BuildButtonCSS("btnid_", buttonId, dbp->bg, dbp->fg, tstr, NULL);
// Convert CSS to provider
gtk_css_provider_load_from_data(dbp->bp, tstr, -1, NULL);
// Add provider to button
gtk_style_context_add_provider (context, GTK_STYLE_PROVIDER(dbp->bp), GTK_STYLE_PROVIDER_PRIO$
// Give the button the CSS style/class corresponding to the one we just created for it
snprintf(value, SHORT_STR, "btnid_%02d", buttonId);
gtk_style_context_add_class(context, value);
g_object_unref(dbp->bp);
// Discourage expansion (again, doesn't appear to work when label gets too long)
gtk_widget_set_hexpand(dbp->btn, FALSE);
gtk_widget_set_vexpand(dbp->btn, FALSE);
// Attach button to grid
gtk_grid_attach(GTK_GRID(_grid), dbp->btn, c, r, 1, 1);
g_signal_connect(dbp->btn, "button_press_event", G_CALLBACK(button_press_callback), GINT_TO_P$
g_signal_connect(dbp->btn, "button_release_event", G_CALLBACK(button_release_callback), GINT_$
dbp->initialised = 1;
However when we update text within a button, it renders as a single (non-wrapped) line of text and causes the button to expand and totally bu&&er up the layout.
There's two problems here;
The text should wrap, the buttons have plenty of height but the text remains a single line
The buttons should be absoltuely FIXED width & height no matter what
This feels like it should be simple to do, but I'm not very familiar with GTK and all the examples out there seem to assume that you'd want everything to flex and resize itself all the time.
Edit: Tried Alexander Dmitriev's suggestion to use ellipsize - didn't seem to work although threw no errors;
// Experiment; ellipsize
label = gtk_bin_get_child(GTK_BIN(dbp->btn));
gtk_label_set_ellipsize(GTK_LABEL(label),PANGO_ELLIPSIZE_END);
gtk_label_set_max_width_chars(GTK_LABEL(label), 14);
I also tried this exmaple by creating a GtkWidget label, applying the properties to it and then adding it to the button - but it seems gtk_button_set_label will only accept a string not a label.

How to take a screenshot of a scrollable in Selenium using Java?

I am a beginner in Selenium.
I have very long scroll-able form which will automation script will fill, is there a way where I can take screenshot when the page is scrolled down every time (web-driver scrolls to fill the form, I haven't scripted to scroll down as web-driver takes care of it), so that the entire form can be captured in multiple screenshots.
Here is how I would do it. On every swipe You can take screenshot, like as step screenshot.
Here is part of code for screenshot take:
public static String takeScreenshot(WebDriver driver) {
driver.get("http://www.google.com/");
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
// Now you can do whatever you need to do with it, for example copy somewhere
File dest = new File("c:\\tmp\\screenshot.png");
FileUtils.copyFile(scrFile, dest);
return dest.getAbsolutePath();
}
And here is the code for vertical swipe:
public static void swipeVertical(WebDriver driver) {
Dimension size = driver.manage().window().getSize();
System.out.println(size);
int starty = (int) (size.height * 0.70);
int endy = (int) (size.height * 0.30);
int startx = size.width / 2;
new TouchAction(driver)
.press(startx, starty)
.waitAction(Duration.ofMillis(1000))
.moveTo(startx, endy)
.release()
.perform();
}
so my pseudo code would look like this:
WebDriver driver = initDriver();
load page and elements
inputText(text);
takScreenShot(driver);
swipe(driver);
...
repeat until end...
You can capture as many screenshots as you want. After fill in each column, after fill in each third column, whatever.
Or you can let the webdriver fill in the whole form and scroll up/down using press pgup/pgdn simulation:
import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;
Robot rob = new Robot();
rob.keyPress(KeyEvent.VK_PAGE_DOWN);
rob.keyRelease(KeyEvent.VK_PAGE_DOWN);

Saving kinect v2 frames in an array

I am trying to develop a software based on Kinect v2 and I need to keep the capturedframes in an array. I have a problem and I dont have any idea about it as follow.
The captured frames are processed by my processing class and the processed writable bitmap will be called as the source of the image box in my ui window which works perfectly and I have a realtime frames in my ui.
for example:
/// Color
_ProcessingInstance.ProcessColor(colorFrame);
ImageBoxRGB.Source = _ProcessingInstance.colorBitmap;
but when I want to assign this to an element of an array, all of the elements in array will be identical as the first frame!! I should mention that, this action is in the reading event which above action is there.
the code:
ColorFrames_Array[CapturingFrameCounter] = _ProcessingInstance.colorBitmap;
the equal check in intermediate window:
ColorFrames_Array[0].Equals(ColorFrames_Array[1])
true
ColorFrames_Array[0].Equals(ColorFrames_Array[2])
true
Please give me some hints about this problem. Any idea?
Thanks Yar
You are right and when I create a new instance, frames are saved correctly.
But my code was based on the Microsoft example and problem is that creating new instances makes the memory leakage because writablebitmap is not disposable.
similar problem is discussed in the following link which the frames are frizzed to the first frame and this is from the intrinsic properties of writeablebitmap:
http://www.wintellect.com/devcenter/jprosise/silverlight-s-big-image-problem-and-what-you-can-do-about-it
Therefore i use a strategy similar to the above solution and try to get a copy instead of the original bitmap frame. In this scenario, I have create a new writeblebitmap for each element of ColorFrames_Array[] at initialization step.
ColorFrames_Array = new riteableBitmap[MaximumFramesNumbers_Capturing];
for (int i=0; i < MaximumFramesNumbers_Capturing; ++i)
{
ColorFrames_Array[i] = new WriteableBitmap(color_width, color_height, 96.0, 96.0, PixelFormats.Bgr32, null);
}
and finally, use clone method to copy the bitmap frames to array elements.
ColorFrames_ArrayBuffer[CapturingFrameCounter] = _ProcessingInstance.colorBitmap.Clone();
While above solution works, but it has a huge memory leakage!!.
Therefore I use Array and .copypixel methods (of writeablebitmap) to copy the pixels of the frame to array and hold it (while the corresponding writeablebitmap will be disposed correctly without leakage).
public Array[] ColorPixels_Array;
for (int i=0; i< MaximumFramesNumbers_Capturing; ++i)
{
ColorPixels_Array[i]=new int[color_Width * color_Height];
}
colorBitmap.CopyPixels(ColorPixels_Array[Counter_CapturingFrame], color_Width * 4, 0);
Finally, when we want to save the arrays of pixels, we need to convert them new writeablebitmap instances and write them on hard.
wb = new WriteableBitmap(color_Width, color_Height, 96.0, 96.0, PixelFormats.Bgr32, null);
wb.WritePixels(new Int32Rect(0, 0, color_Width, color_Height)
, Ar_Px,
color_Width * 4, 0);

Rendering on a WPF Control with DirectX 11

I am trying to create a map editor based on WPF. Currently I'm using a hack to render DirectX contents. I created a WinFormsHost and rendered on a WinForms-Panel.
This all because DirectX (I´m using DirectX 11 with Featurelevel 10) wants a Handle (alias IntPtr) where to render. I don´t know how I can initialize and use the DX Device without a handle.
But a WPF control has no handle. So I just found out, there is an interop class called "D3DImage". But I don't understand how to use it.
My current system works like this:
The inner loop goes through a list of "IGameloopElement"s. For each, it renders its content calling "Draw()". After that, it calls "Present()" of the swap chain to show the changes. Then it resets the device to switch the handle to the next element (mostly there is only one element).
Now, because D3DImage doesn't have a handle, how do I render onto it? I just know I have to use "Lock()" then "SetBackBuffer()", "AddDirtyRect()" and then "Unlock()".
But how do I render onto a DirectX11.Texture2D object without specifying a handle for the device?
I´m really lost... I just found the "DirectX 4 WPF" sample on codeplex, but this implements all versions of DirectX, manages the device itself and has such a huge overhead.
I want to stay at my current system. I´m managing the device by myself. I don´t want the WPF control to handle it.
The loop just should call "Render()" and then passes the backbuffer texture to the WPF control.
Could anyone tell me how to do this? I´m totally stuck ...
Thanks a lot :)
R
WPF's D3DImage only supports Direct3D9/Direct3D9Ex, it does not support Direct3D 11. You need to use DXGI Surface Sharing to make it work.
Another answer wrote, "D3DImage only supports Direct3D9/Direct3D9Ex"... which is perhaps not entirely true for the last few years anyway. As I summarized in a comment here, the key appears to be that Direct3D11 with DXGI has a very specific interop compatibility mode (D3D11_SHARED_WITHOUT_MUTEX flag) which makes the ID3D11Texture2D1 directly usable as a D3DResourceType.IDirect3DSurface9, without copying any bits, which just so happens to be exactly (and only) what WPF D3DImage is willing to accept.
This is a rough sketch of what worked for me, to create a D3D11 SampleAllocator that produces ID3D11Texture2D1 that are directly compatible with WPF's Direct3D9. Because all the .NET interop shown here is of my own design, this will not be totally ready-to-run code to drop in your project, but the method, intent, and procedures should be clear for easy adaptation.
1. preliminary helper
static D3D_FEATURE_LEVEL[] levels =
{
D3D_FEATURE_LEVEL._11_1,
D3D_FEATURE_LEVEL._11_0,
};
static IMFAttributes GetSampleAllocatorAttribs()
{
MF.CreateAttributes(out IMFAttributes attr, 6);
attr.SetUINT32(in MF_SA_D3D11_AWARE, 1U);
attr.SetUINT32(in MF_SA_D3D11_BINDFLAGS, (uint)D3D11_BIND.RENDER_TARGET);
attr.SetUINT32(in MF_SA_D3D11_USAGE, (uint)D3D11_USAGE.DEFAULT);
attr.SetUINT32(in MF_SA_D3D11_SHARED_WITHOUT_MUTEX, (uint)BOOL.TRUE);
attr.SetUINT32(in MF_SA_BUFFERS_PER_SAMPLE, 1U);
return attr;
}
static IMFMediaType GetMediaType()
{
MF.CreateMediaType(out IMFMediaType mt);
mt.SetUINT64(in MF_MT_FRAME_SIZE, new SIZEU(1920, 1080).ToSwap64());
mt.SetGUID(in MF_MT_MAJOR_TYPE, in WMMEDIATYPE.Video);
mt.SetUINT32(in MF_MT_INTERLACE_MODE, (uint)MFVideoInterlaceMode.Progressive);
mt.SetGUID(in MF_MT_SUBTYPE, in MF_VideoFormat.RGB32);
return mt;
}
2. the D3D11 device and context instances go somewhere
ID3D11Device4 m_d3D11_device;
ID3D11DeviceContext2 m_d3D11_context;
3. initialization code is next
void InitialSetup()
{
D3D11.CreateDevice(
null,
D3D_DRIVER_TYPE.HARDWARE,
IntPtr.Zero,
D3D11_CREATE_DEVICE.BGRA_SUPPORT,
levels,
levels.Length,
D3D11.SDK_VERSION,
out m_d3D11_device,
out D3D_FEATURE_LEVEL _,
out m_d3D11_context);
MF.CreateDXGIDeviceManager(out uint tok, out IMFDXGIDeviceManager m_dxgi);
m_dxgi.ResetDevice(m_d3D11_device, tok);
MF.CreateVideoSampleAllocatorEx(
ref REFGUID<IMFVideoSampleAllocatorEx>.GUID,
out IMFVideoSampleAllocatorEx sa);
sa.SetDirectXManager(m_dxgi);
sa.InitializeSampleAllocatorEx(
PrerollSampleSink.QueueMax,
PrerollSampleSink.QueueMax * 2,
GetSampleAllocatorAttribs(),
GetMediaType());
}
4. use sample allocator to repeatedly generate textures, as needed
ID3D11Texture2D1 CreateTexture2D(SIZEU sz)
{
var vp = new D3D11_VIEWPORT
{
TopLeftX = 0f,
TopLeftY = 0f,
Width = sz.Width,
Height = sz.Height,
MinDepth = 0f,
MaxDepth = 1f,
};
m_d3D11_context.RSSetViewports(1, ref vp);
var desc = new D3D11_TEXTURE2D_DESC1
{
SIZEU = sz,
MipLevels = 1,
ArraySize = 1,
Format = DXGI_FORMAT.B8G8R8X8_UNORM,
SampleDesc = new DXGI_SAMPLE_DESC { Count = 1, Quality = 0 },
Usage = D3D11_USAGE.DEFAULT,
BindFlags = D3D11_BIND.RENDER_TARGET | D3D11_BIND.SHADER_RESOURCE,
CPUAccessFlags = D3D11_CPU_ACCESS.NOT_REQUESTED,
MiscFlags = D3D11_RESOURCE_MISC.SHARED,
TextureLayout = D3D11_TEXTURE_LAYOUT.UNDEFINED,
};
m_d3D11_device.CreateTexture2D1(ref desc, IntPtr.Zero, out ID3D11Texture2D1 tex2D);
return tex2D;
}

Resources