The input image is imageA. I want to copy the middle 1/3 data(imageB) into the opencl buffer.
I use the clEnqueueWriteBuffer function.(I use buffer but NOT the Image)
clEnqueueWriteBuffer(queue,
cl_buffer_input, // opencl buffer
1,
0, // NOW offset is 0
WIDTH_IMAGE*(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL*sizeof(cl_uchar),//1/3 image height
(void*)(image_input.data), // input data
0, 0, NULL);
After that ,the buffer I copied is image C's data.
So I want to use the offset to copy image B.
The code I used is
clEnqueueWriteBuffer(queue,
cl_buffer_input, // opencl buffer
1,
(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL*sizeof(cl_uchar), // NOW offset is the offset of data
WIDTH_IMAGE*(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL*sizeof(cl_uchar),//1/3 image height
(void*)(image_input.data), // input data
0, 0, NULL);
But the result can't be updated! Even I change the offset into 1. The result is also still.(New frame data in the video can't be upload, the result is only the first frame and the position is like image C).
So I changed the start pointer of image data, and let the offset be 0
The new code is like this:
clEnqueueWriteBuffer(queue,
cl_buffer_input, // opencl buffer
1,
0, // NOW offset is the offset of data
WIDTH_IMAGE*(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL*sizeof(cl_uchar),//1/3 image height
(void*)(image_input.data+(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL), // input data's pointer changed
0, 0, NULL);
And... THe new result is like imageD!
It only has the offset of X line.
So...what can I do to copy just the middle 1/3 data of the image into a opencl buffer?
Thank you~
You are not getting anything weird, the results for the code you run are ok.
However, if you want to copy the part B, you need this piece of code:
clEnqueueWriteBuffer(queue,
cl_buffer_input, // opencl buffer
CL_TRUE, //Blocking?
0, // No offset inside the buffer (the image will start at 0 inside the cl_buffer)
WIDTH_IMAGE*(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL*sizeof(cl_uchar),//Copy only 1/3 of image size
(void*)(image_input.data+WIDTH_IMAGE*(HEIGHT_IMAGE/3))*COLOR_IMAGE_CHANNEL), // Offset the input data by 1/3 as well (the first data to copy is at 1/3 inside the array)
0, 0, NULL);
In details: What you need is to copy 1/3 of the image, so the size is 1/3. The buffer offset is 0, because you don't want to write the image at the end of the buffer, but at the beggining. And the ptr where to take the data to copy has to have an offset of 1/3 of the image. So that you copy the portion of the pointer 1/3 to 2/3, into the buffer. (The buffer will have 1/3 of the original image size)
Related
I have a spritesheet that contains a simple sprite animation. I can successfully extract each animation frame sprite and display it as a texture. However, I managed to do that by calling glBufferSubData to change the texture coordinates (before the game loop, at initialization). I want to play a simple sprite animation using these extracted textures, and I guess I will do it by changing the texture coordinates each frame (except if animation is triggered by user input). Anyways, this results in calling glBufferSubData almost every frame (to change texture data), and my question is that is this approach efficient? If not, how can I solve the issue? (In Reddit, I saw a comment saying that the goal must be to minimize the traffic between CPU and GPU memory in modern OpenGL, and I guess my approach violates this goal.) Thanks in advance.
For anyone interested, here is my approach:
void set_sprite_texture_from_spritesheet(Sprite* sprite, const char* path, int x_offset, int y_offset, int sprite_width, int sprite_height)
{
float* uv_coords = get_uv_coords_from_spritesheet(path, x_offset, y_offset, sprite_width, sprite_height);
for (int i = 0; i < 8; i++)
{
/* 8 means that I am changing a total of 8 texture coordinates (2 for each 4 vertices) */
edit_vertex_data_by_index(sprite, &uv_coords[i], (i / 2) * 5 + 3 + (i % 2 != 0));
/*
the last argument in this function gives the index of the desired texture coordinate
(5 is for stride, 3 for offset of texture coordinates in each row)
*/
}
free(uv_coords);
sprite->texture = load_texture(path); /* loads the texture -
since the UV coordinate is
adjusted based on the spritesheet
I am loading the entire spritesheet as
a texture.
*/
}
void edit_vertex_data_by_index(Sprite *sprite, float *data, unsigned int start_index)
{
glBindBuffer(GL_ARRAY_BUFFER, sprite->vbo);
glBufferSubData(GL_ARRAY_BUFFER, start_index * sizeof(float), sizeof(data), data);
glBindBuffer(GL_ARRAY_BUFFER, 0);
/*
My concern is that if I call this almost every frame, it could be not efficient, but I am not sure.
*/
}
Editing buffers is fine. Literally every game has buffers that change every frame. Buffers are how you get the data to the GPU so it can render it! (And uniforms. Your driver is likely to secretly put uniforms in buffers though!)
Yes, you should minimize the amount of buffer updates. You should minimize everything, really. The less stuff the computer does, the faster it can do it! That doesn't mean you should avoid doing stuff entirely. It means you should only do as much stuff as you need to, instead of doing wasteful stuff that you don't need.
Every time you call an OpenGL function, the driver takes some time to check how to process your request, which buffer is bound, that it's big enough, that the GPU isn't using it at the same time, etc. You want to do as few calls as possible, because that way, the driver has to check all this stuff less often.
You are doing 8 separate glBufferSubData calls in this function. If you put the UV coordinates all next to each other in the buffer, you could update them all at once with 1 call. And if you have lots of animated sprites, you should try to put all of their UV coordinates in one big array, and update the whole array in one call - all the sprites at once.
And loading textures from paths is really slow. Maybe your program can load 100 textures per second but that still means you blew half your frame time budget on texture loading. The texture hasn't changed anyway so why would you load it again?
I have a three-dimensional array of binary numbers, which I use as a dictionary and pass through an LED array. The dictionary covers 27 letters, and each letter covers 30x30 pixels (where each pixel is a 0 or a 1).
I was using the Intel Edison - and the code worked well - but I ditched the Edison after having trouble connecting it to my PC (despite replacing it once). I switched to the Arduino Uno, but am now receiving an error that the array is too large.
Right now I have the array set as boolean. Is there anyway to reduce the memory demands of the array by setting it instead as bits? The array consists of just zeros and ones.
Here's a snip of the code:
boolean PHDict[27][30][30] = {
/* A */ {{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, /* this is one column of thirty, that show "A" as a letter */
You could write it as
#include <stdint.h>
//...
uint32_t PHdict[27][30] = {
{ 0x00004000, ... },
....
};
.. Where each entry contains 30 bits packed into a 32-bit number.
The size is under 4k bytes.
You would need a bit of code to unpack the bits when reading the array, and a way to generate the packed values (I.e a program which runs on your "host" computer, and generates the initialized array for the source code)
For the AVR processor, there's also a way to tell the compiler you want the array stored in PM (Flash memory) instead of DM - I think if you have it in DM, the compiler will need to put the initialization data in PM anyway, and copy it over before the program starts, so it's a good idea to explicitly store it in PM. See https://gcc.gnu.org/onlinedocs/gcc/AVR-Variable-Attributes.html#AVR-Variable-Attributes
In fact, depending on the amount of flash memory in the processor, changing it to PM may be sufficient to solve the problem, without needing to pack the bits.
I am filling a Frame with a BGR image for encoding, and I am getting a memory leak. I think I got to the source of the problem but it appears to be a library issue instead. Since FFmpeg is such a mature library, I think I am misusing it and I would like to be instructed on how to do it correctly.
I am allocating a Frame using:
AVFrame *bgrFrame = av_frame_alloc();
And later I allocate the image in the Frame using:
av_image_alloc(bgrFrame->data, bgrFrame->linesize, bgrFrame->width, bgrFrame->height, AV_PIX_FMT_BGR24, 32);
Then I fill the image allocated using:
av_image_fill_pointers(bgrFrame->data, AV_PIX_FMT_BGR24, bgrFrame->height, originalBGRImage.data, bgrFrame->linesize);
Where originalBGRImage is an OpenCV Mat.
And this has a memory leak, apparently, av_image_alloc() allocates memory, and av_image_fill_pointers() also allocates memory, on the same pointers (I can see bgrFrame->data[0] changing between calls).
If I call
av_freep(&bgrFrame->data[0]);
After av_image_alloc(), it's fine, but if I call it after av_image_fill_pointers(), the program crashes, even though bgrFrame->data[0] is not NULL, which I find very curious.
Looking FFmpeg's av_image_alloc() source code, I see it calls av_image_fill_pointers() twice inside it, once allocating a buffer buff....and later in av_image_fill_pointers() source code, data[0] is substituted by the image pointer, which is (I think) the source of the memory leak, since data[0] was holding buf from the previous av_image_alloc() call.
So this brings the final question: What's the correct way of filling a frame with an image?.
You should allocate your frame once.
AVFrame* alloc_picture(enum PixelFormat pix_fmt, int width, int height)
{
AVFrame *f = avcodec_alloc_frame();
if (!f)
return NULL;
int size = avpicture_get_size(pix_fmt, width, height);
uint8_t *buffer = (uint8_t *) av_malloc(size);
if (!buffer) {
av_free(f);
return NULL;
}
avpicture_fill((AVPicture *)f, buffer, pix_fmt, width, height);
return f;
}
Yes, the cast (AVPicture*) is allowed https://stackoverflow.com/a/20498359/2079934 .
In subsequent frames, you can write into the this frame. Since your OpenCV raw data is BGR and you need RGB or YUV, you can use sws_scale. In my application, I mirror vertically:
struct SwsContext* convertCtx = sws_getContext(width, height, PIX_FMT_RGB24, c->width, c->height, c->pix_fmt, SWS_FAST_BILINEAR, NULL, NULL, NULL);
avpicture_fill(&pic_raw, (uint8_t*)pixelBuffer, PIX_FMT_RGB24, width, height);
// flip
pic_raw.data[0] += (height - 1) * pic_raw.linesize[0];
pic_raw.linesize[0] *= -1;
sws_scale(convertCtx, pic_raw.data, pic_raw.linesize, 0, height, f->data, f->linesize);
out_size = avcodec_encode_video(c, outputBuffer, outputBufferSize, f);
(You can adapt PIX_FMT_RGB24 to your needs and read from cv::Mat without copying data.)
av_fill_arrays() does the job. It will fill the frame's data[] and linesizes[] but not reallocating any memory.
Too late for answer. But after take many hours, i want to share.
In document
/**
* AVBuffer references backing the data for this frame. All the pointers in
* data and extended_data must point inside one of the buffers in buf or
* extended_buf. This array must be filled contiguously -- if buf[i] is
* non-NULL then buf[j] must also be non-NULL for all j < i.
*
* There may be at most one AVBuffer per data plane, so for video this array
* always contains all the references. For planar audio with more than
* AV_NUM_DATA_POINTERS channels, there may be more buffers than can fit in
* this array. Then the extra AVBufferRef pointers are stored in the
* extended_buf array.
*/
AVBufferRef *buf[AV_NUM_DATA_POINTERS];
Then buf is "smart pointer" for data (extended_buf for extended_data)
for example, i using image one linesize only
int size = av_image_get_buffer_size(AVPixelFormat::AV_PIX_FMT_BGRA, width, height, 1);
AVBufferRef* dataref = av_buffer_alloc(size);//that for av_frame_unref
memcpy(dataref->data, your_buffer, size );
AVFrame* frame = av_frame_alloc();
av_image_fill_arrays(frame->data, frame->linesize, dataref->data, AVPixelFormat::AV_PIX_FMT_BGRA, source->width, source->height, 1
frame->buf[0] = dataref;
av_frame_unref will unref frame->buf and free pointer if ref count to zero
Suppose there is a frame with some image.I want to display only those parts which have pixel intensity above 120 or 130. How can I do that with OpenCv ? Is there any commands to do so?
Then i need to set those parts to some intensity of 190.
As mentioned by astay13, you can use the threshold function like this:
Mat image = imread("someimage.jpg", 0); // flag == 0 means read as grayscale
Mat mask;
// this tells you where locations >= 120 pixel intensity are
threshold(image, mask, 120.0, 255.0, THRESH_BINARY);
// this sets those locations to 190 based on the mask you just created
image.setTo(Scalar(190, 0, 0), mask);
imshow("image", image);
Hope that is helpful!
You could try the cvThreshold function. For the second part, cvFloodFill might be what you need.
I have two png images First one with Width1 2247 Height1 190 and second one with Width2 155 Height2 36. I wan't the second image(src) to be placed in the center of first image(dest). I created pixel buf of both and used gdk_pixbuf_composite as follows.
gdk_pixbuf_composite( srcpixbuf, dstpixbuf, 1000, 100, width2, height2, 0, 0, 1, 1, GDK_INTERP_BILINEAR, 255);
I get a hazy window of width2 and height2 on the first image.
If I replace width2 and height2 with 1.0 then I don't get the srcimage on the dstimage. Where am I going wrong?
gdk_pixbuf_composite( srcpixbuf, dstpixbuf, 1000, 100, width2, height2, 1000, 100, 1, 1, GDK_INTERP_BILINEAR, 255);
This solved. Wrongly understood the offset parameter. Basically an intermediate scaled image is created and only the part represented by the dest wid, height is composited. So in my case we need to move the entire unscaled image to the destination offset which is done by the offset parameter.