I'm trying to make a simple image viewer. I basically load an image into a surface and then create a texture from it.
At the end, I do the usual SDL_RenderClear(), SDL_RenderCopy() and SDL_RenderPresent() as per the migration guide.
This works fine, except that if I call SDL_UpdateTexture() before the 3 render calls above, I get a messed up image:
I am calling SDL_UpdateTexture() like this:
SDL_UpdateTexture(texture, NULL, image->pixels, image->pitch)
Where image is the surface I loaded for the image and texture is the texture I created from that. Attempts to vary the pitch result in differently messed up images. I also tried using a rect for the second parameter, but results are the same if the rect has the same dimensions as the image. If the dimensions are larger (e.g. same as the window), the update doesn't happen, but there are no errors.
The full code is available.
I would like to manipulate pixels of the surface directly via image->pixels and then call SDL_UpdateTexture(), but just calling SDL_UpdateTexture() without any tampering is enough to mess things up.
I think there is something wrong with the pitch or the SDL_Rect parameters,
but there is another SDL function which might help:
SDL_Texture* SDL_CreateTextureFromSurface(SDL_Renderer* renderer,
SDL_Surface* surface)
Could you maybe try the following. It should replace any pink (r=255,g=0,b=255) pixels to be transparent. You would simply change the pixel32 manipulation to accommodate your needs.
SDL_Surface* image = IMG_Load(filename);
SDL_Surface* imageFomatted = SDL_ConvertSurfaceFormat(image,
SDL_PIXELFORMAT_RGBA8888,
NULL);
texture = SDL_CreateTexture(renderer,
SDL_PIXELFORMAT_RGBA8888,
SDL_TEXTUREACCESS_STREAMING,
imageFomatted->w, imageFomatted->h);
void* pixels = NULL;
int pitch = 0;
SDL_LockTexture(texture, &imageFomatted->clip_rect, &pixels, &pitch);
memcpy(pixels, imageFomatted->pixels, (imageFomatted->pitch * imageFomatted->h));
int width = imageFomatted->w;
int height = imageFomatted->h;
Uint32* pixels32 = (Uint32*)pixels;
int pixelCount = (pitch / 4) * height;
Uint32 colorKey = SDL_MapRGB(imageFomatted->format, 0xFF, 0x00, 0xFF);
Uint32 transparent = SDL_MapRGBA(imageFomatted->format, 0xFF, 0x00, 0xFF, 0x00);
for (int i = 0; i < pixelCount; i++) {
if (pixels32[i] == colorKey) {
pixels32[i] = transparent;
}
}
SDL_UnlockTexture(texture);
SDL_FreeSurface(imageFormatted);
SDL_FreeSurface(image);
pixels = NULL;
pitch = 0;
width = 0;
height = 0;
I am encountering an issue when reading and writing a BitmapSource to disk using a JpegEncoder/Decoder. The following code sample illustrates the problem:
//initialize with some dummy test data
int outputHeight = 100;
int outputWidth = 100;
int outputStride = 100 * 3; //24 bpp
byte[] outputBytes = new byte[100 * outputStride];
double dpiX = 20.5;
double dpiY = 20.5;
//generate image
BitmapSource testOutput = BitmapImage.Create(outputWidth, outputHeight,
dpiX, dpiY, PixelFormats.Bgr24, null,
outputBytes, outputStride);
Trace.Assert(testOutput.DpiX == dpiX); //succeeds
Trace.Assert(testOutput.DpiY == dpiY); //succeeds
//write to disk
JpegBitmapEncoder encoder = new JpegBitmapEncoder();
using (FileStream fileStream = new FileStream(#"F:\Users\Caleb\Desktop\test.jpg", FileMode.Create))
{
encoder.Frames.Add(BitmapFrame.Create(testOutput));
encoder.QualityLevel = 100;
Trace.Assert(encoder.Frames[0].DpiX == dpiX); //succeeds
Trace.Assert(encoder.Frames[0].DpiY == dpiY); //succeeds
encoder.Save(fileStream);
}
//read back
using (Stream imageStreamSource = new FileStream(#"F:\Users\Caleb\Desktop\test.jpg", FileMode.Open, FileAccess.Read, FileShare.Read))
{
JpegBitmapDecoder decoder = new JpegBitmapDecoder(imageStreamSource, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.Default);
BitmapSource reread = decoder.Frames[0];
Trace.Assert(reread.DpiX == dpiX); //fails; reread.DpiX is 21.0
Trace.Assert(reread.DpiY == dpiY); //fails; reread.DpiY is 21.0
}
As indicated in the comments, the DPI that is read back is not equal to the input value. It seems that either during the encoding or decoding process, the DPI attributes are being rounded to the nearest pixel.
Is there a way to retain the number to the right of the decimal point in the DPI attributes of the image that is read back from disk?
Even though the API allows you to pass in a double (and hence a value like 20.5), DPI values are integers. Therefore the API is rounding (up in your case) to the nearest integer. Do you expect any difference between a DPI of 20.5 and 21.0?
Well, not sure what's going on with the JpegBitmapEncoder/Decoder chain, but using the BmpBitmapEncoder/Decoder instead yields results that are accurate to the nearest 10th of a pixel. Not ideal, but it will probably work for the my application.
This should be easy. I have a video stream coming in from my webcam. I'm just playing with image transformation etc. I'd like to be able to view the original images (video input) in one window and the transformed video in another. Problem is, as soon as I start capturing video instead of just single images, the original video window displays transformed video. I don't understand why.
cvNamedWindow("in", CV_WINDOW_AUTOSIZE);
cvNamedWindow("out", CV_WINDOW_AUTOSIZE);
CvCapture *fc = cvCaptureFromCAM(0);
IplImage* frame = cvQueryFrame(fc);
if (!frame) {
return 0;
}
IplImage* greyscale = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1);
IplImage* output = cvCreateImage(cvGetSize(frame),IPL_DEPTH_32F , 1);
while(1){
frame= cvQueryFrame(fc);
cvShowImage("in", frame);
// manually convert to greyscale
for (int y = 0; y < frame->height; y++) {
uchar* p = (uchar*) frame->imageData + y* frame->widthStep; // pointer to row
uchar* gp = (uchar*) greyscale->imageData + y*greyscale->widthStep;
for(int x = 0; x < frame->width; x++){
gp[x] = (p[3*x] + p[3*x+1] + p[3*x+2])/3; // average RGB values
}
}
cvShowImage("out", greyscale);
char c = cvWaitKey(33);
if (c == 27) {
return 0;
}
}
In this simple example, both video streams end up appearing greyscale... The pointer values and imagedata for frame and greyscale are totally different. If I stop showing greyscale in the "out" window, then frame will appear in color.
Also, if I continue and apply a Sobel operation on the greyscale image and display the result in "out", both "in" and "out" windows will show the Sobel image!
Any ideas?
Hmm This was weird, but it seems using CV_WINDOW_AUTOSIZE was the problem? Perhaps it's not supported in OpenCV 2.1 (which I'm pretty sure is what I'm running). Anyways, using 0 instead of CV_WINDOW_AUTOSIZE when creating the windows works fine.
I have tried your code with openCV 2.0 under mandriva 2010 and it is working fine either with CV_WINDOW_AUTOSIZE or 0.
You may try to convert to grayscale with cvCvtColor(frame,grayscale,CV_RGB2GRAY) and see if the problem persist.
I have a WPF BitmapImage which I loaded from a .JPG file, as follows:
this.m_image1.Source = new BitmapImage(new Uri(path));
I want to query as to what the colour is at specific points. For example, what is the RGB value at pixel (65,32)?
How do I go about this? I was taking this approach:
ImageSource ims = m_image1.Source;
BitmapImage bitmapImage = (BitmapImage)ims;
int height = bitmapImage.PixelHeight;
int width = bitmapImage.PixelWidth;
int nStride = (bitmapImage.PixelWidth * bitmapImage.Format.BitsPerPixel + 7) / 8;
byte[] pixelByteArray = new byte[bitmapImage.PixelHeight * nStride];
bitmapImage.CopyPixels(pixelByteArray, nStride, 0);
Though I will confess there's a bit of monkey-see, monkey do going on with this code.
Anyway, is there a straightforward way to process this array of bytes to convert to RGB values?
Here is how I would manipulate pixels in C# using multidimensional arrays:
[StructLayout(LayoutKind.Sequential)]
public struct PixelColor
{
public byte Blue;
public byte Green;
public byte Red;
public byte Alpha;
}
public PixelColor[,] GetPixels(BitmapSource source)
{
if(source.Format!=PixelFormats.Bgra32)
source = new FormatConvertedBitmap(source, PixelFormats.Bgra32, null, 0);
int width = source.PixelWidth;
int height = source.PixelHeight;
PixelColor[,] result = new PixelColor[width, height];
source.CopyPixels(result, width * 4, 0);
return result;
}
usage:
var pixels = GetPixels(image);
if(pixels[7, 3].Red > 4)
{
...
}
If you want to update pixels, very similar code works except you will create a WriteableBitmap, and use this:
public void PutPixels(WriteableBitmap bitmap, PixelColor[,] pixels, int x, int y)
{
int width = pixels.GetLength(0);
int height = pixels.GetLength(1);
bitmap.WritePixels(new Int32Rect(0, 0, width, height), pixels, width*4, x, y);
}
thusly:
var pixels = new PixelColor[4, 3];
pixels[2,2] = new PixelColor { Red=128, Blue=0, Green=255, Alpha=255 };
PutPixels(bitmap, pixels, 7, 7);
Note that this code converts bitmaps to Bgra32 if they arrive in a different format. This is generally fast, but in some cases may be a performance bottleneck, in which case this technique would be modified to match the underlying input format more closely.
Update
Since BitmapSource.CopyPixels doesn't accept a two-dimensional array it is necessary to convert the array between one-dimensional and two-dimensional. The following extension method should do the trick:
public static class BitmapSourceHelper
{
#if UNSAFE
public unsafe static void CopyPixels(this BitmapSource source, PixelColor[,] pixels, int stride, int offset)
{
fixed(PixelColor* buffer = &pixels[0, 0])
source.CopyPixels(
new Int32Rect(0, 0, source.PixelWidth, source.PixelHeight),
(IntPtr)(buffer + offset),
pixels.GetLength(0) * pixels.GetLength(1) * sizeof(PixelColor),
stride);
}
#else
public static void CopyPixels(this BitmapSource source, PixelColor[,] pixels, int stride, int offset)
{
var height = source.PixelHeight;
var width = source.PixelWidth;
var pixelBytes = new byte[height * width * 4];
source.CopyPixels(pixelBytes, stride, 0);
int y0 = offset / width;
int x0 = offset - width * y0;
for(int y=0; y<height; y++)
for(int x=0; x<width; x++)
pixels[x+x0, y+y0] = new PixelColor
{
Blue = pixelBytes[(y*width + x) * 4 + 0],
Green = pixelBytes[(y*width + x) * 4 + 1],
Red = pixelBytes[(y*width + x) * 4 + 2],
Alpha = pixelBytes[(y*width + x) * 4 + 3],
};
}
#endif
}
There are two implementations here: The first one is fast but uses unsafe code to get an IntPtr to an array (must compile with /unsafe option). The second one is slower but does not require unsafe code. I use the unsafe version in my code.
WritePixels accepts two-dimensional arrays, so no extension method is required.
Edit: As Jerry pointed out in the comments, because of the memory layout, the two-dimensional array has the vertical coordinate first, in other words it must be dimensioned as Pixels[Height,Width] not Pixels[Width,Height] and addressed as Pixels[y,x].
I'd like to add to Ray´s answer that you can also declare PixelColor struct as a union:
[StructLayout(LayoutKind.Explicit)]
public struct PixelColor
{
// 32 bit BGRA
[FieldOffset(0)] public UInt32 ColorBGRA;
// 8 bit components
[FieldOffset(0)] public byte Blue;
[FieldOffset(1)] public byte Green;
[FieldOffset(2)] public byte Red;
[FieldOffset(3)] public byte Alpha;
}
And that way you'll also have access to the UInit32 BGRA (for fast pixel access or copy), besides the individual byte components.
The interpretation of the resulting byte array is dependent upon the pixel format of the source bitmap, but in the simplest case of a 32 bit, ARGB image, each pixel will be composed of four bytes in the byte array. The first pixel would be interpreted thusly:
alpha = pixelByteArray[0];
red = pixelByteArray[1];
green = pixelByteArray[2];
blue = pixelByteArray[3];
To process each pixel in the image, you would probably want to create nested loops to walk the rows and the columns, incrementing an index variable by the number of bytes in each pixel.
Some bitmap types combine multiple pixels into a single byte. For instance, a monochrome image packs eight pixels into each byte. If you need to deal with images other than 24/32 bit per pixels (the simple ones), then I would suggest finding a good book that covers the underlying binary structure of bitmaps.
I'd like to improve upon Ray's answer - not enough rep to comment. >:( This version has the best of both safe/managed, and the efficiency of the unsafe version. Also, I've done away with passing in the stride as the .Net documentation for CopyPixels says it's the stride of the bitmap, not of the buffer. It's misleading, and can be computed inside the function anyway. Since the PixelColor array must be the same stride as the bitmap (to be able to do it as a single copy call), it makes sense to just make a new array in the function as well. Easy as pie.
public static PixelColor[,] CopyPixels(this BitmapSource source)
{
if (source.Format != PixelFormats.Bgra32)
source = new FormatConvertedBitmap(source, PixelFormats.Bgra32, null, 0);
PixelColor[,] pixels = new PixelColor[source.PixelWidth, source.PixelHeight];
int stride = source.PixelWidth * ((source.Format.BitsPerPixel + 7) / 8);
GCHandle pinnedPixels = GCHandle.Alloc(pixels, GCHandleType.Pinned);
source.CopyPixels(
new Int32Rect(0, 0, source.PixelWidth, source.PixelHeight),
pinnedPixels.AddrOfPinnedObject(),
pixels.GetLength(0) * pixels.GetLength(1) * 4,
stride);
pinnedPixels.Free();
return pixels;
}
I took all examples and created a slightly better one - tested it too
(the only flaw was that magic 96 as DPI which really bugged me)
I also compared this WPF tactic versus:
GDI by using Graphics (system.drawing)
Interop by directly invoking GetPixel from GDI32.Dll
To my supprise,
This works x10 faster than GDI, and around x15 times faster then Interop.
So if you're using WPF - much better to work with this to get your pixel color.
public static class GraphicsHelpers
{
public static readonly float DpiX;
public static readonly float DpiY;
static GraphicsHelpers()
{
using (var g = Graphics.FromHwnd(IntPtr.Zero))
{
DpiX = g.DpiX;
DpiY = g.DpiY;
}
}
public static Color WpfGetPixel(double x, double y, FrameworkElement AssociatedObject)
{
var renderTargetBitmap = new RenderTargetBitmap(
(int)AssociatedObject.ActualWidth,
(int)AssociatedObject.ActualHeight,
DpiX, DpiY, PixelFormats.Default);
renderTargetBitmap.Render(AssociatedObject);
if (x <= renderTargetBitmap.PixelWidth && y <= renderTargetBitmap.PixelHeight)
{
var croppedBitmap = new CroppedBitmap(
renderTargetBitmap, new Int32Rect((int)x, (int)y, 1, 1));
var pixels = new byte[4];
croppedBitmap.CopyPixels(pixels, 4, 0);
return Color.FromArgb(pixels[3], pixels[2], pixels[1], pixels[0]);
}
return Colors.Transparent;
}
}
A little remark:
If you are trying to use this code (Edit: provided by Ray Burns), but get the error about the array's rank, try to edit the extension methods as follows:
public static void CopyPixels(this BitmapSource source, PixelColor[,] pixels, int stride, int offset, bool dummy)
and then call the CopyPixels method like this:
source.CopyPixels(result, width * 4, 0, false);
The problem is, that when the extension method doesn't differ from the original, the original one is called. I guess this is because PixelColor[,] matches Array as well.
I hope this helps you if you got the same problem.
If you want just one Pixel color:
using System.Windows.Media;
using System.Windows.Media.Imaging;
...
public static Color GetPixelColor(BitmapSource source, int x, int y)
{
Color c = Colors.White;
if (source != null)
{
try
{
CroppedBitmap cb = new CroppedBitmap(source, new Int32Rect(x, y, 1, 1));
var pixels = new byte[4];
cb.CopyPixels(pixels, 4, 0);
c = Color.FromRgb(pixels[2], pixels[1], pixels[0]);
}
catch (Exception) { }
}
return c;
}
Much simpler. There's no need to copy the data around, you can get it directly. But this comes at a price: pointers and unsafe. In a specific situation, decide whether it's worth the speed and ease for you (but you can simply put the image manipulation into its own separate unsafe class and the rest of the program won't be affected).
var bitmap = new WriteableBitmap(image);
data = (Pixel*)bitmap.BackBuffer;
stride = bitmap.BackBufferStride / 4;
bitmap.Lock();
// getting a pixel value
Pixel pixel = (*(data + y * stride + x));
bitmap.Unlock();
where
[StructLayout(LayoutKind.Explicit)]
protected struct Pixel {
[FieldOffset(0)]
public byte B;
[FieldOffset(1)]
public byte G;
[FieldOffset(2)]
public byte R;
[FieldOffset(3)]
public byte A;
}
The error checking (whether the format is indeed BGRA and handling the case if not) will be left to the reader.
You can get color components in a byte array. First copy the pixels in 32 bit to an array and convert that to 8-bit array with 4 times larger size
int[] pixelArray = new int[stride * source.PixelHeight];
source.CopyPixels(pixelArray, stride, 0);
// byte[] colorArray = new byte[pixelArray.Length];
// EDIT:
byte[] colorArray = new byte[pixelArray.Length * 4];
for (int i = 0; i < colorArray.Length; i += 4)
{
int pixel = pixelArray[i / 4];
colorArray[i] = (byte)(pixel >> 24); // alpha
colorArray[i + 1] = (byte)(pixel >> 16); // red
colorArray[i + 2] = (byte)(pixel >> 8); // green
colorArray[i + 3] = (byte)(pixel); // blue
}
// colorArray is an array of length 4 times more than the actual number of pixels
// in the order of [(ALPHA, RED, GREEN, BLUE), (ALPHA, RED...]
I am trying to find out how to use FJCore to encode a WriteableBitmap to a jpeg. I understand that WriteableBitmap provides the raw pixels but I am not sure how to convert it to the format that FJCore expects for its JpegEncoder method. JpegEncoder has two overloads, one takes a FluxJpeg.Core.Image and the other takes in a DecodedJpeg.
I was trying to create a FluxJpeg.Core.Image but it expects a byte[][,] for the image data. byte[n][x,y] where x is width and y is height but I don't know what n should be.
I thought that n should be 4 since that would correspond to the argb info encoded in each pixel but when I tried that FJCore throws an argument out of range exception. Here is what I tried. Raster is my byte[4][x,y] array.
raster[0][x, y] = (byte)((pixel >> 24) & 0xFF);
raster[1][x, y] = (byte)((pixel >> 16) & 0xFF);
raster[2][x, y] = (byte)((pixel >> 8) & 0xFF);
raster[3][x, y] = (byte)(pixel & 0xFF);
Figured it out! I downloaded FJCore from code.google.com and went through the image class. It only expects the RGB bytes. Here is the function that I wrote. I need the base64 version of the image so that's what my function returns.
private static string GetBase64Jpg(WriteableBitmap bitmap)
{
int width = bitmap.PixelWidth;
int height = bitmap.PixelHeight;
int bands = 3;
byte[][,] raster = new byte[bands][,];
for (int i = 0; i < bands; i++)
{
raster[i] = new byte[width, height];
}
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
int pixel = bitmap.Pixels[width * row + column];
raster[0][column, row] = (byte)(pixel >> 16);
raster[1][column, row] = (byte)(pixel >> 8);
raster[2][column, row] = (byte)pixel;
}
}
ColorModel model = new ColorModel { colorspace = ColorSpace.RGB };
FluxJpeg.Core.Image img = new FluxJpeg.Core.Image(model, raster);
MemoryStream stream = new MemoryStream();
JpegEncoder encoder = new JpegEncoder(img, 90, stream);
encoder.Encode();
stream.Seek(0, SeekOrigin.Begin);
byte[] binaryData = new Byte[stream.Length];
long bytesRead = stream.Read(binaryData, 0, (int)stream.Length);
string base64String =
System.Convert.ToBase64String(binaryData,
0,
binaryData.Length);
return base64String;
}
This code is fine and it should work. I am using same code to send image stream to server via web service and than regenerate image using these bytes...you can save these bytes to Db also
[WebMethod]
public string SaveImage(string data, string fileName)
{
byte[] imageBytes = System.Convert.FromBase64String(data);
MemoryStream mem = new MemoryStream();
mem.Write(imageBytes, 0, imageBytes.Length);
System.Drawing.Image img = System.Drawing.Image.FromStream(mem);
img.Save("D:\\FinalTest.jpg");
return "Saved !";
}
Sounds like [n] should be the byte-array of the image, I have been looking into encoding WriteableBitmap into a JPEG and found the same library but have not looked into it in detail, but assume this would be the case, will add more later to this answer to see if this works, as I have not have the chance to try it yet. There will be some method to get the bytes of a WritableBitmap in Silverlight I guess as it is possible to save to other types.