CoreGraphics: Encode RGBA data to PNG - c

I am trying to use the C interface of CoreGraphics & CoreFoundation to save a buffer of 32-bit RGBA data (as a void*) to a PNG file. When I try to finialize the CGImageDestinationRef, the following error message is printed to the console:
libpng error: No IDATs written into file
As far as I can tell, the CGImageRef I'm adding to the CGImageDestinationRef is valid.
Relavent Code:
void saveImage(const char* szImage, void* data, size_t dataSize, size_t width, size_t height)
{
CFStringRef name = CFStringCreateWithCString(NULL, szImage, kCFStringEncodingASCII);
CFURLRef texture_url = CFURLCreateWithFileSystemPath(
NULL,
name,
kCFURLPOSIXPathStyle,
false);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGDataProviderRef dataProvider = CGDataProviderCreateWithData(NULL, data, dataSize, NULL);
CGImageRef image = CGImageCreate(width, height, 8, 32, 32 * width, colorSpace,
kCGImageAlphaLast | kCGBitmapByteOrderDefault, dataProvider,
NULL, FALSE, kCGRenderingIntentDefault);
// From Image I/O Programming Guide, "Working with Image Destinations"
float compression = 1.0; // Lossless compression if available.
int orientation = 4; // Origin is at bottom, left.
CFStringRef myKeys[3];
CFTypeRef myValues[3];
CFDictionaryRef myOptions = NULL;
myKeys[0] = kCGImagePropertyOrientation;
myValues[0] = CFNumberCreate(NULL, kCFNumberIntType, &orientation);
myKeys[1] = kCGImagePropertyHasAlpha;
myValues[1] = kCFBooleanTrue;
myKeys[2] = kCGImageDestinationLossyCompressionQuality;
myValues[2] = CFNumberCreate(NULL, kCFNumberFloatType, &compression);
myOptions = CFDictionaryCreate( NULL, (const void **)myKeys, (const void **)myValues, 3,
&kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
CFStringRef type = CFStringCreateWithCString(NULL, "public.png", kCFStringEncodingASCII);
CGImageDestinationRef dest = CGImageDestinationCreateWithURL(texture_url, type, 1, myOptions);
CGImageDestinationAddImage(dest, image, NULL);
if (!CGImageDestinationFinalize(dest))
{
// ERROR!
}
CFRelease(image);
CFRelease(colorSpace);
CFRelease(dataProvider);
CFRelease(dest);
CFRelease(texture_url);
}
This post is similar, except I'm not using the Objective C interface: Saving a 32 bit RGBA buffer into a .png file (Cocoa OSX)

Answering my own questions:
In addition to the issues pointed out by NSGod, the IDAT issue was an invalid parameter to CGImageCreate(): parameter 5 is bytesPerRow, not bitsPerRow. So 32 * width was incorrect; 4 * width is correct.
Despite what this page of the official documentation lists, UTCoreTypes.h is located in the CoreServices.framework for MacOSX, not MobileCoreServices.framework.

There are numerous issues with your code.
Here it is rewritten how I would do it:
void saveImage(const char* szImage, void* data, size_t dataSize, size_t width, size_t height)
{
CFStringRef name = CFStringCreateWithCString(NULL, szImage, kCFStringEncodingUTF8);
CFURLRef texture_url = CFURLCreateWithFileSystemPath(
NULL,
name,
kCFURLPOSIXPathStyle,
false);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGDataProviderRef dataProvider = CGDataProviderCreateWithData(NULL, data,
dataSize, NULL);
CGImageRef image = CGImageCreate(width, height, 8, 32, 32 * width, colorSpace,
kCGImageAlphaLast | kCGBitmapByteOrderDefault,
dataProvider, NULL, FALSE, kCGRenderingIntentDefault);
// From Image I/O Programming Guide, "Working with Image Destinations"
float compression = 1.0; // Lossless compression if available.
int orientation = 4; // Origin is at bottom, left.
CFStringRef myKeys[3];
CFTypeRef myValues[3];
CFDictionaryRef myOptions = NULL;
myKeys[0] = kCGImagePropertyOrientation;
myValues[0] = CFNumberCreate(NULL, kCFNumberIntType, &orientation);
myKeys[1] = kCGImagePropertyHasAlpha;
myValues[1] = kCFBooleanTrue;
myKeys[2] = kCGImageDestinationLossyCompressionQuality;
myValues[2] = CFNumberCreate(NULL, kCFNumberFloatType, &compression);
myOptions = CFDictionaryCreate(NULL, (const void **)myKeys,
(const void **)myValues, 3, &kCFTypeDictionaryKeyCallBacks,
&kCFTypeDictionaryValueCallBacks);
CGImageDestinationRef dest =
CGImageDestinationCreateWithURL(texture_url, kUTTypePNG, 1, NULL);
CGImageDestinationAddImage(dest, image, NULL);
CGImageDestinationSetProperties(dest, myOptions);
if (!CGImageDestinationFinalize(dest))
{
// ERROR!
}
}
First, never use ASCII when dealing with file system paths, use UTF8. Second, you were constructing a dictionary to be used to set the properties of the image, but you were using it with the wrong function. The documentation for CGImageDestinationCreateWithURL() says the following:
CGImageDestinationCreateWithURL
Creates an image destination that writes to a location specified by a
URL.
CGImageDestinationRef CGImageDestinationCreateWithURL (
CFURLRef url,
CFStringRef type,
size_t count,
CFDictionaryRef options
);
Parameters
options - Reserved for future use. Pass NULL.
You were trying to pass a dictionary of properties when you were supposed to pass NULL. (Also, you can simply use the kUTTypePNG Uniform Type Identifier string constant instead of re-creating it). First call CGImageDestinationCreateWithURL(), then call CGImageDestinationAddImage() to add the image, then call CGImageDestinationSetProperties() and pass in the dictionary of properties you created.
[UPDATE]: If after these changes you're still having libpng error: No IDATs written into file issues, try the following: First, make sure that dataProvider is non-NULL-- in other words, make sure the CGDataProviderCreateWithData() function succeeded. Second, if dataProvider is valid, perhaps try changing the options from kCGImageAlphaLast | kCGBitmapByteOrderDefault to simply kCGImageAlphaPremultipliedLast and see if it succeeds.

Related

How to set pts, dts and duration in ffmpeg library?

I want to pack some compressed video packets(h.264) to ".mp4" container.
One word, Muxing, no decoding and no encoding.
And I have no idea how to set pts, dts and duration.
I get the packets with "pcap" library.
I removed headers before compressed video data show up. e.g. Ethernet, VLAN.
I collected data until one frame and decoded it for getting information of data. e.g. width, height. (I am not sure that it is necessary)
I initialized output context, stream and codec context.
I started to receive packets with "pcap" library again. (now for muxing)
I made one frame and put that data in AVPacket structure.
I try to set PTS, DTS and duration. (I think here is wrong part, not sure though)
*7-1. At the first frame, I saved time(msec) with packet header structure.
*7-2. whenever I made one frame, I set parameters like this : PTS(current time - start time), DTS(same PTS value), duration(current PTS - before PTS)
I think it has some error because :
I don't know how far is suitable long for dts from pts.
At least, I think duration means how long time show this frame from now to next frame, so It should have value(next PTS - current PTS), but I can not know the value next PTS at that time.
It has I-frame only.
// make input context for decoding
AVFormatContext *&ic = gInputContext;
ic = avformat_alloc_context();
AVCodec *cd = avcodec_find_decoder(AV_CODEC_ID_H264);
AVStream *st = avformat_new_stream(ic, cd);
AVCodecContext *cc = st->codec;
avcodec_open2(cc, cd, NULL);
// make packet and decode it after collect packets is be one frame
gPacket.stream_index = 0;
gPacket.size = gPacketLength[0];
gPacket.data = gPacketData[0];
gPacket.pts = AV_NOPTS_VALUE;
gPacket.dts = AV_NOPTS_VALUE;
gPacket.flags = AV_PKT_FLAG_KEY;
avcodec_decode_video2(cc, gFrame, &got_picture, &gPacket);
// I checked automatically it initialized after "avcodec_decode_video2"
// put some info that I know that not initialized
cc->time_base.den = 90000;
cc->time_base.num = 1;
cc->bit_rate = 2500000;
cc->gop_size = 1;
// make output context with input context
AVFormatContext *&oc = gOutputContext;
avformat_alloc_output_context2(&oc, NULL, NULL, filename);
AVFormatContext *&ic = gInputContext;
AVStream *ist = ic->streams[0];
AVCodecContext *&icc = ist->codec;
AVStream *ost = avformat_new_stream(oc, icc->codec);
AVCodecContext *occ = ost->codec;
avcodec_copy_context(occ, icc);
occ->flags |= CODEC_FLAG_GLOBAL_HEADER;
avio_open(&(oc->pb), filename, AVIO_FLAG_WRITE);
// repeated part for muxing
AVRational Millisecond = { 1, 1000 };
gPacket.stream_index = 0;
gPacket.data = gPacketData[0];
gPacket.size = gPacketLength[0];
gPacket.pts = av_rescale_rnd(pkthdr->ts.tv_sec * 1000 /
+ pkthdr->ts.tv_usec / 1000 /
- gStartTime, Millisecond.den, ost->time_base.den, /
(AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
gPacket.dts = gPacket.pts;
gPacket.duration = gPacket.pts - gPrev;
gPacket.flags = AV_PKT_FLAG_KEY;
gPrev = gPacket.pts;
av_interleaved_write_frame(gOutputContext, &gPacket);
Expected and actual results is a .mp4 video file that can play.

Get 32 bit RGBA image from Windows clipboard

I want my app (which works with RGBA8888 images) to be able to paste images from the Windows clipboard. So it should be able to read images off the clipboard that come from any common raster image apps like Gimp, Photoshop, MSPaint, etc.
From reading up on the clipboard functions, it seems I should be able to call GetClipboardData(CF_DIBV5) to get access to pretty much any bitmap type that's on the Clipboard since Windows automatically converts between that and CF_BITMAP and CF_DIB. But from reading up on the DIB format, I see that there is an immense number of possible combinations of bit depth, RGB order, optional compression, etc. It seems like what I'm doing would be a common task, but I don't see any conversion functions in the Windows API (unless I'm poor at searching), and this seems like something that would take a week to write to support all possible formats. So I'm wondering if I've overlooked something obvious. Or if there is some kind of assumption I can make to simplify this...like if all the popular image apps happen to copy images to the clipboard in uncompressed/unindexed formats.
UPDATE: Here's what I have so far:
HGLOBAL clipboard = GetClipboardData(CF_DIBV5);
exists = clipboard != NULL;
int dataLength = GlobalSize(clipboard);
exists = dataLength != 0;
if (exists) {
LPTSTR lockedClipboard = GlobalLock(clipboard);
exists = lockedClipboard != NULL;
if (exists) {
BITMAPV5HEADER *header = (BITMAPV5HEADER*)lockedClipboard;
LONG width = header->bV5Width;
LONG height = header->bV5Height;
BYTE *bits = header + sizeof(header) + header->bV5ClrUsed * sizeof(RGBQUAD);
//Now what? Need function to convert the bits to something uncompressed.
GlobalUnlock(clipboard);
}
}
UPDATE 2:
To clarify, I need literally uncompressed 32 bit image data (RRGGBBAA) which I can manipulate however I like in a cross-platform app. I have no need to use Windows APIs to draw this image to screen.
I am aware of a 3rd party library called stdb_image.h that can load .bmps, .jpgs, and .pngs into the type of data I need. So if there's a way I can turn the clipboard data into bitmap or png file data without losing alpha, then I'll be in good shape.
The basic strategy I've found is to check if there's a raw PNG on the clipboard and use that first if available. That's the easiest. Some apps, such as GIMP, copy images as PNG to the clipboard.
Then check for CF_DIBV5. The location of the actual bits depends on whether the "compression" is BI_BITFIELDS:
int offset = bitmapV5Header->bV5Size + bitmapV5Header->bV5ClrUsed * (bitmapV5Header->bV5BitCount > 24 ? sizeof(RGBQUAD) : sizeof(RGBTRIPLE));
if (compression == BI_BITFIELDS)
offset += 12; //bit masks follow the header
BYTE *bits = (BYTE*)bitmapV5Header + offset;
If the header says compression is BI_BITFIELDS, then the data is already as I needed it.
If the header says compression is BI_RGB and the bit count is 24 or 32, then I can unpack the bytes. 24 bytes means row size might not land on a DWORD boundary, so you have to watch for that.
Finally, lower bit counts than 24 likely mean indexed color, which I don't have working yet.
Here is example of usage for CF_DIBV5 and CF_DIB. It's best to use CF_DIB as backup option. Note, this code won't work for palette based images (if it is not guaranteed 32bit then see the method further down)
You can use SetDIBitsToDevice to draw directly on HDC, or use SetDIBits
GDI functions don't support alpha transparency (except for a couple of functions like TransparentBlt), in general you have to use libraries such as GDI+ for that.
void foo(HDC hdc)
{
if (!OpenClipboard(NULL))
return;
HANDLE handle = GetClipboardData(CF_DIBV5);
if (handle)
{
BITMAPV5HEADER* header = (BITMAPV5HEADER*)GlobalLock(handle);
if (header)
{
BITMAPINFO bmpinfo;
memcpy(&bmpinfo.bmiHeader, header, sizeof(BITMAPINFOHEADER));
bmpinfo.bmiHeader.biSize = sizeof(BITMAPINFO);
//(use `header` to access other BITMAPV5HEADER information)
int w = bmpinfo.bmiHeader.biWidth;
int h = bmpinfo.bmiHeader.biHeight;
const char* bits = (char*)(header) + header->bV5Size;
//draw using SetDIBitsToDevice
SetDIBitsToDevice(hdc,0,0,w,h,0,0,0,h,bits,&bmpinfo,DIB_RGB_COLORS);
}
}
else
{
handle = GetClipboardData(CF_DIB);
if (handle)
{
BITMAPINFO* bmpinfo = (BITMAPINFO*)GlobalLock(handle);
if (bmpinfo)
{
int w = bmpinfo->bmiHeader.biWidth;
int h = bmpinfo->bmiHeader.biHeight;
const char* bits = (char*)(bmpinfo)+bmpinfo->bmiHeader.biSize;
SetDIBitsToDevice(hdc, 0, 0, w, h, 0, 0, 0, h, bits, bmpinfo, 0);
}
}
}
CloseClipboard();
}
If the original image is palette based, you would have to convert to 32bit. Alternatively you could add BITMAPFILEHEADER to the data (assuming the source is bitmap) then pass to the other library.
This is an example using CreateDIBitmap and GetDIBits to make sure the pixels are in 32bit:
HANDLE handle = GetClipboardData(CF_DIB);
if (handle)
{
BITMAPINFO* bmpinfo = (BITMAPINFO*)GlobalLock(handle);
if (bmpinfo)
{
int offset = (bmpinfo->bmiHeader.biBitCount > 8) ?
0 : sizeof(RGBQUAD) * (1 << bmpinfo->bmiHeader.biBitCount);
const char* bits = (const char*)(bmpinfo)+bmpinfo->bmiHeader.biSize + offset;
HBITMAP hbitmap = CreateDIBitmap(hdc, &bmpinfo->bmiHeader, CBM_INIT, bits, bmpinfo, DIB_RGB_COLORS);
//convert to 32 bits format (if it's not already 32bit)
BITMAP bm;
GetObject(hbitmap, sizeof(bm), &bm);
int w = bm.bmWidth;
int h = bm.bmHeight;
char *bits32 = new char[w*h*4];
BITMAPINFOHEADER bmpInfoHeader = { sizeof(BITMAPINFOHEADER), w, h, 1, 32 };
HDC hdc = GetDC(0);
GetDIBits(hdc, hbitmap, 0, h, bits32, (BITMAPINFO*)&bmpInfoHeader, DIB_RGB_COLORS);
ReleaseDC(0, hdc);
//use bits32 for whatever purpose...
//cleanup
delete[]bits32;
}
}

Get bitmap of a CF_DIBV5 from clipboard

I'm trying to get bitmap data from the clipboard. I can successfully get the header information for the CF_DIBV5 object:
BOOLEAN exists = IsClipboardFormatAvailable(CF_DIBV5) &&
OpenClipboard(session->windowHandle);
if (exists) {
HGLOBAL clipboard = GetClipboardData(CF_DIBV5);
exists = clipboard != NULL;
if (exists) {
LPTSTR lptstr = GlobalLock(clipboard);
exists = lptstr != NULL;
if (exists) {
BITMAPV5HEADER * header = clipboard;
//now need the HBITMAP!
}
}
}
//...
I can successfully log info from the header. Now I want the actual HBITMAP so I can pass it into GetDIBits. The docs say CF_DIBV5 is a BITMAPV5HEADER "followed by the bitmap color space information and the bitmap bits".
That last part confuses me ironically because it's in plain English. I assume to get to the bitmap bits, I need to add the size of the header and the "color space information" to the header pointer. So
HBITMAP bitmap = header + sizeof(BITMAPV5HEADER) + /* ???? */;
I think...
How can I know the size of this mysterious color space information? And are the "bitmap bits" literally an HBITMAP such that the above expression would be true?
I may be overlooking the obvious since I am a C newbie.
Update: I now realize from experimenting and rereading some documentation that an HBITMAP is a DDB, whereas I have a DIB. So GetDIBits is not the right function for me. What function can be used to convert any DIB to a format with no compression?
Here's how to get the appropriate pointer to the bitmap bits. The arrangement of the contents depends on the compression type and bit count described in the header.
HGLOBAL clipboard = GetClipboardData(CF_DIBV5);
BITMAPV5HEADER* bitmapV5Header = (BITMAPV5HEADER*)GlobalLock(clipboard);
int offset = bitmapV5Header->bV5Size + bitmapV5Header->bV5ClrUsed * sizeof(RGBQUAD);
if (bitmapV5Header->bV5Compression == BI_BITFIELDS)
offset += 12; //bit masks follow the header
BYTE *bits = (BYTE*)bitmapV5Header + offset;

Display RGBA32-BMP Images on Linux

today I got some code to review.
Since the code is going to work on an headless pc the code saves every frame as a seperate RGBa image.
On my Ubuntu install I cannot view theses images, GIMP complains about a broken header. Imagemagick options convert or display also did not show any images.
Here's the code fragment that generates the image:
if (act.doScreenshot || (act.doVideo && buddhabrot_animate.animating))
{
uchar4* tmpBuffer = new uchar4[env.static_env.save.imageW
* env.static_env.save.imageH];
for (int i = 0; i < env.static_env.save.imageW * env.static_env.save.imageH; i++)
{
const unsigned char tmp = tmpBuffer[i].x;
tmpBuffer[i].x = tmpBuffer[i].z;
tmpBuffer[i].z = tmp;
}
char filename[128];
FILE* fp = fopen(filename, "w+b");
BITMAPFILEHEADER bmpFH;
BITMAPINFOHEADER bmpIH;
memset(&bmpFH, 0, sizeof(bmpFH));
memset(&bmpIH, 0, sizeof(bmpIH));
bmpFH.bfType = 19778; //"BM"
bmpFH.bfSize = sizeof(bmpFH) + sizeof(bmpIH) + env.static_env.save.imageW * env.static_env.save.imageH;
bmpFH.bfOffBits = sizeof(bmpFH) + sizeof(bmpIH);
bmpIH.biSize = sizeof(bmpIH);
bmpIH.biWidth = env.static_env.save.imageW;
bmpIH.biHeight = env.static_env.save.imageH;
bmpIH.biPlanes = 1;
bmpIH.biBitCount = 32;
fwrite(&bmpFH, 1, sizeof(bmpFH), fp);
fwrite(&bmpIH, 1, sizeof(bmpIH), fp);
fwrite
(tmpBuffer,
env.static_env.save.imageW * env.static_env.save.imageH,
sizeof(uchar4),
fp);
fclose(fp);
delete[] tmpBuffer;
Is there any way to look at the image?
Or maybe another way to save the images as JPGs?
You don't calculate bmpFH.bfSize correctly, you need to multiply the number of pixels in the image by the size of the pixels (4). For example:
bmpFH.bfSize = sizeof(bmpFH) + sizeof(bmpIH) + env.static_env.save.imageW * env.static_env.save.imageH * sizeof(uchar4);
You should also initialize bmpIH.biCompression to BI_RGB. It'll work anyway because its value happens to be zero, but it's good to be explicit. You also might want to negate the value you're assigning to bmpIH.biHeight as positive height values indicate a bottom up image.

QT Movie Metadata Tagging with QTKit

I'm trying to do some metadata tagging to some video files using QTKit. I've got things down for tagging atom that take a string as their value, but having a hard time setting atoms that take an 8-bit integer as their argument. Here is what I got right now from Apple's Documentation and other various sources on the internet:
-(void) setMediaKind: (NSString *) value
{
QTMetaDataRef metaDataRef;
Movie theMovie;
OSStatus status;
theMovie = [movie quickTimeMovie];
status = QTCopyMovieMetaData (theMovie, &metaDataRef );
NSAssert(status == noErr,#"QTCopyMovieMetaData failed!");
if (status == noErr)
{
int intValue = NSSwapHostIntToBig([(NSNumber *)value intValue]);
UInt8 *dataValuePtr = (UInt8*)(&intValue);
ByteCount dataSize = sizeof(int);
if (dataValuePtr)
{
OSType key = 'stik';
QTMetaDataItem outItem;
status = QTMetaDataAddItem(metaDataRef,
kQTMetaDataStorageFormatiTunes,
kQTMetaDataKeyFormatiTunesShortForm,
(const UInt8 *)&key,
sizeof(key),
dataValuePtr,
dataSize,
kQTMetaDataTypeSignedIntegerBE,
&outItem);
NSAssert(status == noErr,#"QTMetaDataAddItem failed!");
char langCodeStr[] = "en";
status = QTMetaDataSetItemProperty(
metaDataRef,
outItem,
kPropertyClass_MetaDataItem,
kQTMetaDataItemPropertyID_Locale,
strlen(langCodeStr) + 1,
langCodeStr);
}
}
}
So the atom 'stik' sets the video's kind in iTunes. If I want to specify the video as a TV Show i'd need to assign it a value of 10. If I send #"10" to this method I don't get any errors but the video file isn't properly tagged either.
I'm sure part of my problem is I skipped learning C and went straight to Objective C so when I have to dive into C like this I have problems.

Resources