OpenGL ios 6 screenshot - ios6

I am trying to get screenshot of NinevehGL object which is 3D library build on top of OpenGL. So i think OpenGL commands would work here as well.
Problem is screenshot works fine in iOS 5 or earlier but not with iOS6. It works fine in iOS6 simulator as well but not on iOS 6 device. It gives me black background. I have read many posts most of them suggest to put
CAEAGLLayer *eaglLayer = (CAEAGLLayer *) [self theNGLView].layer;
[eaglLayer setDrawableProperties:#{
kEAGLDrawablePropertyRetainedBacking: [NSNumber numberWithBool:YES],
kEAGLDrawablePropertyColorFormat: kEAGLColorFormatRGBA8
}];
in init method. which i did but with no luck.
I am using below method to get screenshot.
-(UIImage *)getImage{
GLint width;
GLint height;
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &width);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &height);
NSLog(#"%d %d",width,height);
NSInteger myDataLength = width * height * 4;
// allocate array and read pixels into it.
GLubyte *buffer = (GLubyte *) malloc(myDataLength);
glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
// gl renders "upside down" so swap top to bottom into new array.
// there's gotta be a better way, but this works.
GLubyte *buffer2 = (GLubyte *) malloc(myDataLength);
for(int y = 0; y < height; y++)
{
for(int x = 0; x < width * 4; x++)
{
buffer2[((height - 1) - y) * width * 4 + x] = buffer[y * 4 * width + x];
}
}
// make data provider with data.
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, buffer2, myDataLength, NULL);
// prep the ingredients
int bitsPerComponent = 8;
int bitsPerPixel = 32;
int bytesPerRow = 4 * width;
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;
// make the cgimage
CGImageRef imageRef = CGImageCreate(width, height, bitsPerComponent, bitsPerPixel, bytesPerRow, colorSpaceRef, bitmapInfo, provider, NULL, NO, renderingIntent);
// then make the uiimage from that
UIImage *myImage = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);
CGDataProviderRelease(provider);
CGColorSpaceRelease(colorSpaceRef);
free(buffer);
free(buffer2);
return myImage;
}

Related

RGB to YUV conversion with libav (ffmpeg) triplicates image

I'm building a small program to capture the screen (using X11 MIT-SHM extension) on video. It works well if I create individual PNG files of the captured frames, but now I'm trying to integrate libav (ffmpeg) to create the video and I'm getting... funny results.
The furthest I've been able to reach is this. The expected result (which is a PNG created directly from the RGB data of the XImage file) is this:
However, the result I'm getting is this:
As you can see the colors are funky and the image appears cropped three times. I have a loop where I capture the screen, and first I generate the individual PNG files (currently commented in the code below) and then I try to use libswscale to convert from RGB24 to YUV420:
while (gRunning) {
printf("Processing frame framecnt=%i \n", framecnt);
if (!XShmGetImage(display, RootWindow(display, DefaultScreen(display)), img, 0, 0, AllPlanes)) {
printf("\n Ooops.. Something is wrong.");
break;
}
// PNG generation
// snprintf(imageName, sizeof(imageName), "salida_%i.png", framecnt);
// writePngForImage(img, width, height, imageName);
unsigned long red_mask = img->red_mask;
unsigned long green_mask = img->green_mask;
unsigned long blue_mask = img->blue_mask;
// Write image data
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
unsigned long pixel = XGetPixel(img, x, y);
unsigned char blue = pixel & blue_mask;
unsigned char green = (pixel & green_mask) >> 8;
unsigned char red = (pixel & red_mask) >> 16;
pixel_rgb_data[y * width + x * 3] = red;
pixel_rgb_data[y * width + x * 3 + 1] = green;
pixel_rgb_data[y * width + x * 3 + 2] = blue;
}
}
uint8_t* inData[1] = { pixel_rgb_data };
int inLinesize[1] = { in_w };
printf("Scaling frame... \n");
int sliceHeight = sws_scale(sws_context, inData, inLinesize, 0, height, pFrame->data, pFrame->linesize);
printf("Obtained slice height: %i \n", sliceHeight);
pFrame->pts = framecnt * (pVideoStream->time_base.den) / ((pVideoStream->time_base.num) * 25);
printf("Frame pts: %li \n", pFrame->pts);
int got_picture = 0;
printf("Encoding frame... \n");
int ret = avcodec_encode_video2(pCodecCtx, &pkt, pFrame, &got_picture);
// int ret = avcodec_send_frame(pCodecCtx, pFrame);
if (ret != 0) {
printf("Failed to encode! Error: %i\n", ret);
return -1;
}
printf("Succeed to encode frame: %5d - size: %5d\n", framecnt, pkt.size);
framecnt++;
pkt.stream_index = pVideoStream->index;
ret = av_write_frame(pFormatCtx, &pkt);
if (ret != 0) {
printf("Error writing frame! Error: %framecnt \n", ret);
return -1;
}
av_packet_unref(&pkt);
}
I've placed the entire code at this gist. This question right here looks pretty similar to mine, but not quite, and the solution did not work for me, although I think this has something to do with the way the line stride is calculated.
Don't use av_image_alloc use av_frame_get_buffer.
(unrelated to your question, But using avcodec_encode_video2 is considered bad practice now and should be replaced with avcodec_send_frame and avcodec_receive_packet)
In the end, the error was not in the usage of libav but on the code that fills the pixel data from XImage to the rgb vector. Instead of using:
pixel_rgb_data[y * width + x * 3 ] = red;
pixel_rgb_data[y * width + x * 3 + 1] = green;
pixel_rgb_data[y * width + x * 3 + 2] = blue;
I should have used this:
pixel_rgb_data[3 * (y * width + x) ] = red;
pixel_rgb_data[3 * (y * width + x) + 1] = green;
pixel_rgb_data[3 * (y * width + x) + 2] = blue;
Somehow I was multiplying only the the horizontal displacement within the matrix, not the vertical displacement. The moment I changed it, it worked perfectly.

libjpeg reads a scanline 1/8th the original width

I am trying to just read every pixel in a jpeg image. When I read a scanline, it appears that the image has been squashed to one eighth the size of the original image. The scanline is the correct width but the remaining 7/8'ths of the scanline are not filled (0, 0, 0).
I cant simply use each pixel 8 times since a large amount of information was lost.
I don't use any decompression parameters and am perfectly happy with the defaults. The Libjpeg version I am using is "libjpeg/9d" from the https://conan.io package center.
How can I get scanlines in the correct aspect ratio?
FILE* file_p = openfile(fileaddress);
struct jpeg_decompress_struct cinfo;
struct jpeg_error_mgr err; //the error handler
cinfo.err = jpeg_std_error( &err );
jpeg_create_decompress(&cinfo);
jpeg_stdio_src(&cinfo, file_p);
int result = jpeg_read_header(&cinfo, TRUE);
bool startedfine = jpeg_start_decompress(&cinfo);
int row_stride = cinfo.output_width * cinfo.output_components;
image output = create_image(cinfo.output_width, cinfo.output_height);
JSAMPARRAY buffer = calloc(1, sizeof(JSAMPROW*));
buffer[0] = calloc(1, sizeof(JSAMPROW) * row_stride);
while (cinfo.output_scanline < cinfo.output_height) {
int current_y = cinfo.output_scanline;
jpeg_read_scanlines(&cinfo, buffer, 1);
JSAMPROW row = buffer[0];
for(int row_i = 0; row_i < row_stride; row_i += 3) {
int r = (int)row[row_i];
int g = (int)row[row_i + 1];
int b = (int)row[row_i + 2];
int actual_x = row_i / 3;
output.pixels_array_2d[actual_x][current_y].r = r;
output.pixels_array_2d[actual_x][current_y].g = b;
output.pixels_array_2d[actual_x][current_y].b = g;
}
}
free(buffer[0]);
free(buffer);
jpeg_finish_decompress(&cinfo);
jpeg_destroy_decompress(&cinfo);
fclose(file_p);
As a side note, you may notice I assign the blue value to the green value when copying the pixels. This is because without it, my test image is the wrong colour and I don't know why.

Issue displaying IDirect3DTexture8 after backporting from IDirect3DTexture9

I'm trying to backport someones Direct3d9 port of Quake 1 by ID software to Direct3d8 so I can port it to the original Xbox (only uses the D3D8 API).
After making the changes to use Direct3d8 it displays some mashed up pixels on the screen that appear to be in little squares :/ (see pictures).
Does anyone know whats gone wrong here? It works flawlessly with D3D9, is there some extra arguments required that I'm missing require for D3D8, rect pitch maybe?
The data been passed in is a Quake 1 .lmp 2d image file. "It consists of two integers (width and height) followed by a string of width x height bytes, each of which is an index into the Quake palette"
Its been passed to the D3D_ResampleTexture() function.
Any help would be much appreciated.
Image output using D3D8
Image output using D3D9
The code:
void D3D_ResampleTexture (image_t *src, image_t *dst)
{
int y, x , srcpos, srcbase, dstpos;
unsigned int *dstdata, *srcdata;
// take an unsigned pointer to the dest data that we'll actually fill
dstdata = (unsigned int *) dst->data;
// easier access to src data for 32 bit resampling
srcdata = (unsigned int *) src->data;
// nearest neighbour for now
for (y = 0, dstpos = 0; y < dst->height; y++)
{
srcbase = (y * src->height / dst->height) * src->width;
for (x = 0; x < dst->width; x++, dstpos++)
{
srcpos = srcbase + (x * src->width / dst->width);
if (src->flags & IMAGE_32BIT)
dstdata[dstpos] = srcdata[srcpos];
else if (src->palette)
dstdata[dstpos] = src->palette[src->data[srcpos]];
else Sys_Error ("D3D_ResampleTexture: !(flags & IMAGE_32BIT) without palette set");
}
}
}
void D3D_LoadTextureStage3 (LPDIRECT3DTEXTURE8/*9*/ *tex, image_t *image)
{
int i;
image_t scaled;
D3DLOCKED_RECT LockRect;
memset (&LockRect, 0, sizeof(D3DLOCKED_RECT));
// check scaling here first
for (scaled.width = 1; scaled.width < image->width; scaled.width *= 2);
for (scaled.height = 1; scaled.height < image->height; scaled.height *= 2);
// clamp to max texture size
if (scaled.width > /*d3d_DeviceCaps.MaxTextureWidth*/640) scaled.width = /*d3d_DeviceCaps.MaxTextureWidth*/640;
if (scaled.height > /*d3d_DeviceCaps.MaxTextureHeight*/480) scaled.height = /*d3d_DeviceCaps.MaxTextureHeight*/480;
IDirect3DDevice8/*9*/_CreateTexture(d3d_Device, scaled.width, scaled.height,
(image->flags & IMAGE_MIPMAP) ? 0 : 1,
/*(image->flags & IMAGE_MIPMAP) ? D3DUSAGE_AUTOGENMIPMAP :*/ 0,
(image->flags & IMAGE_ALPHA) ? D3DFMT_A8R8G8B8 : D3DFMT_X8R8G8B8,
D3DPOOL_MANAGED,
tex
);
// lock the texture rectangle
//(*tex)->LockRect (0, &LockRect, NULL, 0);
IDirect3DTexture8/*9*/_LockRect(*tex, 0, &LockRect, NULL, 0);
// fill it in - how we do it depends on the scaling
if (scaled.width == image->width && scaled.height == image->height)
{
// no scaling
for (i = 0; i < (scaled.width * scaled.height); i++)
{
unsigned int p;
// retrieve the correct texel - this will either be direct or a palette lookup
if (image->flags & IMAGE_32BIT)
p = ((unsigned *) image->data)[i];
else if (image->palette)
p = image->palette[image->data[i]];
else Sys_Error ("D3D_LoadTexture: !(flags & IMAGE_32BIT) without palette set");
// store it back
((unsigned *) LockRect.pBits)[i] = p;
}
}
else
{
// save out lockbits in scaled data pointer
scaled.data = (byte *) LockRect.pBits;
// resample data into the texture
D3D_ResampleTexture (image, &scaled);
}
// unlock it
//(*tex)->UnlockRect (0);
IDirect3DTexture8/*9*/_UnlockRect(*tex, 0);
// tell Direct 3D that we're going to be needing to use this managed resource shortly
//FIXME
//(*tex)->PreLoad ();
}
LPDIRECT3DTEXTURE8/*9*/ D3D_LoadTextureStage2 (image_t *image)
{
d3d_texture_t *tex;
// look for a match
// create a new one
tex = (d3d_texture_t *) malloc (sizeof (d3d_texture_t));
// link it in
tex->next = d3d_Textures;
d3d_Textures = tex;
// fill in the struct
tex->LastUsage = 0;
tex->d3d_Texture = NULL;
// copy the image
memcpy (&tex->TexImage, image, sizeof (image_t));
// upload through direct 3d
D3D_LoadTextureStage3 (&tex->d3d_Texture, image);
// return the texture we got
return tex->d3d_Texture;
}
LPDIRECT3DTEXTURE8/*9*/ D3D_LoadTexture (char *identifier, int width, int height, byte *data, /*bool*/qboolean mipmap, /*bool*/qboolean alpha)
{
image_t image;
image.data = data;
image.flags = 0;
image.height = height;
image.width = width;
image.palette = d_8to24table;
strcpy (image.identifier, identifier);
if (mipmap) image.flags |= IMAGE_MIPMAP;
if (alpha) image.flags |= IMAGE_ALPHA;
return D3D_LoadTextureStage2 (&image);
}
When you lock the texture, you have to observe the returned Pitch member of the D3DLOCKED_RECT structure. Your code is assuming that all the data is contiguous, but the Pitch can be larger than the width of a scanline in order to allow for locking a subregion and other layouts of the buffer that don't have contiguous pixels at the end of one scanline to the beginning of the next.
Look at Chapter 4 of my book "The Direct3D Graphics Pipeline" to see an example of accessing a surface and using the Pitch properly.
For anyone else that comes across this issue, it was due to the way the image was been loaded into the Xbox's memory, it needed to be swizzled.

ARToolkit using a still image

I'm trying to use ARtoolkit, but with a static image instead of a video stream. I need to be able to load an image, identify markers, and locate them. I'm using SDL for loading the image. I'm able to obtain the RGB values for each pixel from the loaded image, but I'm unsure how to format the data for ARToolkit to work with it.
ARToolkit stores its images as type ARUint8* (an unsigned char*). I'm confused as to how this format works. Right now I have this code inside the main loop that runs continuously as the program is executing. This code (should) print out the RGB values for each pixel in the frame.
ARUint8* dataPtr;
dataPtr = arVideoGetImage(); // Get a new frame from the webcam
int width, height;
if (arVideoInqSize(&width, &height) == 0) // if width and height could be obtained
{
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
printf("pixel %i, %i: %i, %i, %i\n", x, y, dataPtr[(y * 320) + x], dataPtr[(y * 320) + x + 1], dataPtr[(y * 320) + x + 2]);
}
}
}
Typical output:
pixel 5, 100: 0, 0, 0
pixel 6, 100: 178, 3, 0
pixel 7, 100: 0, 0, 177
etc...
It seems to be accessing the RGB values correctly, but I'm unsure how to copy over the image data (from SDL's format) into this new format.
Figured it out. Posting answer in case anyone else ever needs it.
On Windows, ARToolkit defaults to BGRA for the dataPtr array. The following function will load an image (using SDL) and return a pointer to a ARUint8 (that contains the image data).
ARUint8* loadImage(char* filename, int* w, int* h)
{
SDL_Surface* img = IMG_Load(filename);
if (!img)
{
printf("Image '%s' failed to load. Error: %s\n", filename, IMG_GetError());
return NULL;
}
*w = img->w; // Assign width and height to the given pointers
*h = img->h;
ARUint8* dataPtr = (ARUint8*)calloc(img->w * img->h * 4, sizeof(ARUint8)); // Allocate space for image data
// Write image data to the dataPtr variable
for (int y = 0; y < img->h; y++)
{
for (int x = 0; x < img->w; x++)
{
Uint8 r, g, b;
SDL_GetRGB(getpixel(img, x, y), img->format, &r, &g, &b); // Get the RGB values
int i = ((y * img->w) + x) * 4; // Figure out index in array
dataPtr[i] = b;
dataPtr[i + 1] = g;
dataPtr[i + 2] = r;
dataPtr[i + 3] = 0; // Alpha
}
}
SDL_FreeSurface(img);
return dataPtr;
}
The getpixel function is borrowed from here: http://sdl.beuc.net/sdl.wiki/Pixel_Access
This function allowed me to use a photograph instead of a video feed from a webcam.

How would one draw to the sub display of a ds as if it was a framebuffer?

I need to draw raw pixel data to the Nintendo DS's "sub" screen, such as if I was drawing to the main screen in "framebuffer" mode or "Extended Rotation" mode. How can I do this with the current version of libnds (which seems to place restrictions on the use of VRAM_C)?
#include <nds.h>
int main(void)
{
int x, y;
//set the mode to allow for an extended rotation background
videoSetMode(MODE_5_2D);
videoSetModeSub(MODE_5_2D);
//allocate a vram bank for each display
vramSetBankA(VRAM_A_MAIN_BG);
vramSetBankC(VRAM_C_SUB_BG);
//create a background on each display
int bgMain = bgInit(3, BgType_Bmp16, BgSize_B16_256x256, 0,0);
int bgSub = bgInitSub(3, BgType_Bmp16, BgSize_B16_256x256, 0,0);
u16* videoMemoryMain = bgGetGfxPtr(bgMain);
u16* videoMemorySub = bgGetGfxPtr(bgSub);
//initialize it with a color
for(x = 0; x < 256; x++)
for(y = 0; y < 256; y++)
{
videoMemoryMain[x + y * 256] = ARGB16(1, 31, 0, 0);
videoMemorySub[x + y * 256] = ARGB16(1, 0, 0, 31);
}
while(1)
{
swiWaitForVBlank();
}
}
Here is a simple example which creates a 16 bit frame buffer on the main and sub screens and fills each with either red or blue.

Resources