Effect of adjusting Y component in YUV? - c

I've been reverse engineering a program and recently came across a function that is intended to create a sort of translucent-looking color to be used for text selections. It does this by converting RGB to YUV, alters the Y (luma?) component, then converts back to RGB.
uint32_t CalcSelectionColor(uint32_t bgr)
{
double r,g,b;
double y,u,v;
r = (bgr >> 0) & 0xFF;
g = (bgr >> 8) & 0xFF;
b = (bgr >> 16) & 0xFF;
/* RGB to YUV */
y = 0.299*r + 0.587*g + 0.114*b;
u = (b-y) * 0.565 * 0.5;
v = (r-y) * 0.713 * 0.5;
/* lower brightness? */
y = 255.0 - y;
/* YUV to RGB */
r = y + 1.403*v;
g = y - 0.344*u - 0.714*v;
b = y + 1.77*u;
return ((uint8_t)(b) << 16) | ((uint8_t)(g) << 8) | ((uint8_t)(r));
}
As someone with very limited knowledge of computer graphics, I'd just like a bit more detail of what it does between the conversions, and the actually intended effect in a broader sense. Is this a common approach of adjusting brightness of a color or something? If I pass in 0x00FF00, the result I get is 0x1E9D1E

The formulas used in this code are similar to Julien transformation from RGB to YUV and back:
Transformation from RGB to YUV:
Y = 0.299R + 0.587G + 0.114B
U'= (B-Y)*0.565
V'= (R-Y)*0.713
Transformation from YUV to RGB:
R = Y + 1.403V'
G = Y - 0.344U' - 0.714V'
B = Y + 1.770U'
However, the formulas in your code are a bit different. While the back transformation is the same, the forward transform has an additional multiplier 0.5 for both U and V components. There is also a trivial manipulation with the brightness component
y = 255.0 - y
which simply inverses the brightness. So, what happens here?
If you use normal Julien RGB->YUV transform, you get a representation for your color as a combination of brightness Y and two color tone components U and V, which define the color as shown on this picture:
However, in your code you also multiply both U and V components by 0.5. This means, that on this UV plane you move from any given color two times closer to the point of origin (0, 0). For example, if the initial color was A with UV coordinates (-0.4, 0.3), then you'll get a new color B with UV coordinates (-0.2, 0.15). Similarly, the color C (0.2, -0.3) becomes color D (0.1, -0.15):
After that you inverse the brightness of the color, making dark colors bright and bright colors dark. This is the effect of your code.

It's not terribly common, but it's a very good approach. Commonly used models like HSL/HSV don't represent intensity correctly and have some weird piecewise-linear stuff with hue/color going on. YUV is a really good colorspace, representing intensity along one axis and chroma (hue/color) in a perpendicular plane.
Normally modifying Y without also adjusting (at least clamping) U and V is somewhat dubious, because near the extremes (Y=0 black, Y=full white) U and V have limited range (no range at all at the endpoints). Otherwise applying them will take you outside of the RGB cube and result in bogus clipped results when you go back to RGB. But here the trick is very clever. The code is inverting Y while keeping chroma fixed, so the incoming range limits on U and V near black will automatically ensure they're roughly correct in the output, and vice versa.
As Alex noted, the code here is also halving the chroma values, reducing color saturation. This was probably to avoid the above mentioned clipping issue, but it's not needed. But maybe it's part of the intended visual effect too.
So, TL;DR: the effect is inverting intensity/luma and halving saturation.

Related

Resizing single 1 pixel wide bitmap strip - faster than this example? (for Raycaster algorithm)

I am attaching the picture example and my current code.
My question is: Can I make resizing/streching/interpolating single vertical bitmap strip faster
that using another for-loop.
The current Code looks very optimal:
for current strip size in the screen, iterate from start height to end height. Get corresponding
pixel from texture and add to output buffer. Add step to get another pixel.
here is an essential part of my code:
inline void RC_Raycast_Walls()
{
// casting ray for every width pixel
for (u_int16 rx = 0; rx < RC_render_width_i; ++rx)
{
// ..
// traversing thru map of grid
// finding intersecting point
// calculating height of strip in screen
// ..
// step size for nex pixel in texutr
float32 tex_step_y = RC_texture_size_f / (float32)pp_wall_height;
// starting texture coordinate
float32 tex_y = (float32)(pp_wall_start - RC_player_pitch - player_z_div_wall_distance - RC_render_height_d2_i + pp_wall_height_d2) * tex_step_y;
// drawing walls into buffer <- ENTERING ANOTHER LOOP only for SINGLE STRIP
for (int16 ry = pp_wall_start; ry < pp_wall_end; ++ry)
{
// cast the texture coordinate to integer, and mask with (texHeight - 1) in case of overflow
u_int16 tex_y_safe = (u_int16)tex_y & RC_texture_size_m1_i;
tex_y += tex_step_y;
u_int32 texture_current_pixel = texture_pixels[RC_texture_size_i * tex_y_safe + tex_x];
u_int32 output_pixel_index = rx + ry * RC_render_width_i;
output_buffer[output_pixel_index] =
(((texture_current_pixel >> 16 & 0x0ff) * intensity_value) >> 8) << 16 |
(((texture_current_pixel >> 8 & 0x0ff) * intensity_value) >> 8) << 8 |
(((texture_current_pixel & 0x0ff) * intensity_value) >> 8);
}
}
}
Maybe some bigger stepping like 2 instead of 1, got then every second line empty,
but adding another line of code that could fil that empty space results the same performance..
I would not like to have doubled pixels and interpolating between two of them I think would take even
longer. ??
Thank You in Advance!
ps.
Its based on Lodev Raycaster algorithm:
https://lodev.org/cgtutor/raycasting.html
You do not need floats at all
You can use DDA on integers without multiplication and division. These days floating is not that slow as it used to but your conversion between float and int might be ... See these QAs (both use this kind of DDA:
DDA line with subpixel
DDA based rendering routines
use LUT for applying Intensity
Looks like each color channel c is 8 bit and intensity i is fixed point in range <0,1> so you can precompute every combination into something like this:
u_int8 LUT[256][256]
for (int c=0;c<256;c++)
for (int i=0;i<256;i++)
LUT[c][i]=((c*i)>>8)
use pointers or union to access RGB channels instead of bit operations
My favorite is union:
union color
{
u_int32 dd; // 1x 32bit RGBA
u_int16 dw[2]; // 2x 16bit
u_int8 db[4]; // 4x 8bit (individual channels)
};
texture coordinates
Again looks like you are doing too many operations. for example [RC_texture_size_i * tex_y_safe + tex_x] if your texture size is 128 you can bitshift lef by 7 bits instead of multiplication. Yes on modern CPUs is this not an issue however the whole thing can be replaced by simple LUT. You can remember pointer to each horizontal ScanLine of texture and rewrite to [tex_y_safe][tex_x]
So based on #2,#3 rewrite your color computation to this:
color c;
c.dd=texture_current_pixel;
c.db[0]=LUT[c.db[0]][intensity_value];
c.db[1]=LUT[c.db[1]][intensity_value];
c.db[2]=LUT[c.db[2]][intensity_value];
output_buffer[output_pixel_index]=c.dd;
As you can see its just bunch of memory transfers instead of multiple bit-shifts,bit-masks and bit-or operations. You can also use pointer of color instead of texture_current_pixel and output_buffer[output_pixel_index] to speed up little more.
And finally see this:
Ray Casting with different height size
Which is my version of the raycast using VCL.
Now before changing anything measure the performance you got now by measuring the time it needs to render. Then after each change in the code measure if it actually improve performance or not. In case it didn't use old version of code as predicting what is fast on nowadays platforms is sometimes hard.
Also for resize much better visual results are obtained by using mipmaps ... that usually eliminates the weird noise while moving

Angle to Quaternion - Making an object facing another object

i have two Objects in a 3D World and want to make the one object facing the other object. I already calculated all the angles and stuff (pitch angle and yaw angle).
The problem is i have no functions to set the yaw or pitch individually which means that i have to do it by a quaternion. As the only function i have is: SetEnetyQuaternion(float x, float y, float z, float w). This is my pseudocode i have yet:
float px, py, pz;
float tx, ty, tz;
float distance;
GetEnetyCoordinates(ObjectMe, &px, &py, &pz);
GetEnetyCoordinates(TargetObject, &tx, &ty, &tz);
float yaw, pitch;
float deltaX, deltaY, deltaZ;
deltaX = tx - px;
deltaY = ty - py;
deltaZ = tz - pz;
float hyp = SQRT((deltaX*deltaX) + (deltaY*deltaY) + (deltaZ*deltaZ));
yaw = (ATAN2(deltaY, deltaX));
if(yaw < 0) { yaw += 360; }
pitch = ATAN2(-deltaZ, hyp);
if (pitch < 0) { pitch += 360; }
//here is the part where i need to do a calculation to convert the angles
SetEnetyQuaternion(ObjectMe, pitch, 0, yaw, 0);
What i tried yet was calculating the sinus from those angles devided with 2 but this didnt work - i think this is for euler angles or something like that but didnt help me. The roll(y axis) and the w argument can be left out i think as i dont want my object to have a roll. Thats why i put 0 in.
If anyone has any idea i would really appreciate help.
Thank you in advance :)
Let's suppose that the quaternion you want describes the attitude of the player relative to some reference attitude. It is then essential to know what the reference attitude is.
Moreover, you need to understand that an object's attitude comprises more than just its facing -- it also comprises the object's orientation around that facing. For example, imagine the player facing directly in the positive x direction of the position coordinate system. This affords many different attitudes, from the one where the player is standing straight up to ones where he is horizontal on either his left or right side, to one where he is standing on his head, and all those in between.
Let's suppose that the appropriate reference attitude is the one facing parallel to the positive x direction, and with "up" parallel to the positive z direction (we'll call this "vertical"). Let's also suppose that among the attitudes in which the player is facing the target, you want the one having "up" most nearly vertical. We can imagine the wanted attitude change being performed in two steps: a rotation about the coordinate y axis followed by a rotation about the coordinate z axis. We can write a unit quaternion for each of these, and the desired quaternion for the overall rotation is the Hamilton product of these quaternions.
The quaternion for a rotation of angle θ around the unit vector described by coordinates (x, y, z) is (cos θ/2, x sin θ/2, y sin θ/2, z sin θ/2). Consider then, the first quaternion you want, corresponding to the pitch. You have
double semiRadius = sqrt(deltaX * deltaX + deltaY * deltaY);
double cosPitch = semiRadius / hyp;
double sinPitch = deltaZ / hyp; // but note that we don't actually need this
. But you need the sine and cosine of half that angle. The half-angle formulae come in handy here:
double sinHalfPitch = sqrt((1 - cosPitch) / 2) * ((deltaZ < 0) ? -1 : 1);
double cosHalfPitch = sqrt((1 + cosPitch) / 2);
The cosine will always be nonnegative because the pitch angle must be in the first or fourth quadrant; the sine will be positive if the object is above the player, or negative if it is below. With all that being done, the first quaternion is
(cosHalfPitch, 0, sinHalfPitch, 0)
Similar analysis applies to the second quaternion. The cosine and sine of the full rotation angle are
double cosYaw = deltaX / semiRadius;
double sinYaw = deltaY / semiRadius; // again, we don't actually need this
We can again apply the half-angle formulae, but now we need to account for the full angle to be in any quadrant. The half angle, however, can be only in quadrant 1 or 2, so its sine is necessarily non-negative:
double sinHalfYaw = sqrt((1 - cosYaw) / 2);
double cosHalfYaw = sqrt((1 + cosYaw) / 2) * ((deltaY < 0) ? -1 : 1);
That gives us an overall second quaternion of
(cosHalfYaw, 0, 0, sinHalfYaw)
The quaternion you want is the Hamilton product of these two, and you must take care to compute it with the correct operand order (qYaw * qPitch), because the Hamilton product is not commutative. All the zeroes in the two factors make the overall expression much simpler than it otherwise would be, however:
(cosHalfYaw * cosHalfPitch,
-sinHalfYaw * sinHalfPitch,
cosHalfYaw * sinHalfPitch,
sinHalfYaw * cosHalfPitch)
At this point I remind you that we started with an assumption about the reference attitude for the quaternion system, and the this result depends on that choice. I also remind you that I made an assumption about the wanted attitude, and that also affects this result.
Finally, I observe that this approach breaks down where the target object is very nearly directly above or directly below the player (corresponding to semiRadius taking a value very near zero) and where the player is very nearly on top of the target (corresponding to hyp taking a value very near zero). There is a non-zero chance of causing a division by zero if you use these formulae exactly as given, so you'll want to think about how to deal with that.)

Encode rgb to yuv420p using libav

I'm trying to convert an vector of RGB image data (derived from a .png image) to YUV420p format using libav.
In the libav sample code the following is used to create a dummy image:
/* prepare a dummy image */
static void fill_yuv_image(AVFrame *pict, int frame_index, int width, int height)
{
int x, y, i;
i = frame_index;
/* Y */
for(y=0;y<height;y++) {
for(x=0;x<width;x++) {
pict->data[0][y * pict->linesize[0] + x] = x + y + i * 3;
}
}
/* Cb and Cr */
for(y=0;y<height/2;y++) {
for(x=0;x<width/2;x++) {
pict->data[1][y * pict->linesize[1] + x] = 128 + y + i * 2;
pict->data[2][y * pict->linesize[2] + x] = 64 + x + i * 5;
}
}
}
I'm not clear about a few things here:
Firstly, do I need to rearrange the RGB data in the input vector so that it's suitable for encoding as YUV420p?
Secondly, I understand that there's a Y value for every pixel and that the Cb and Cr values are used for four (2x2) pixels. What I don't understand is how the RGB data gets "reduced" to the Cb and Cr values - is there an example of how to do this anywhere?
I'm not entirely sure what you're trying to achieve exactly, so I'll just directly answer your questions as best I can (feel free to follow up with clarifying comments):
1) You will be transforming the RGB data to YUV which will involve some rearrangement. The packed RGB data is fine where it is. You don't really need to adjust it. Actually, it would probably be better to leave it packed the way it is for cache locality reasons.
2) As you already understand, YUV 4:2:0 encodes a Y sample for each pixel but each 2x2 block shares a Cb and a Cr value. However, there is also YUV 4:4:4 data. This is where each pixel gets its own Y, Cb, and Cr sample. A simple strategy for converting RGB -> YUV 4:2:0 is to convert RGB -> YUV 4:4:4 and then average (arithmetic mean) each block of 2x2 Cb samples. There are other algorithms (like filters that involve more of the surrounding samples), but this should work if you're just experimenting with how this stuff works.
Another strategy for experimentation (and speed) is to only compute the Y plane and hold the Cb and Cr planes constant at 128. That will result in a grayscale image.
For real work, you would probably want to leverage the built-in conversion facilities that libav has to offer.

Computing "average" of two colors

This is only marginally programming related - has much more to do w/ colors and their representation.
I am working on a very low level app. I have an array of bytes in memory. Those are characters. They were rendered with anti-aliasing: they have values from 0 to 255, 0 being fully transparent and 255 totally opaque (alpha, if you wish).
I am having trouble conceiving an algorithm for the rendering of this font. I'm doing the following for each pixel:
// intensity is the weight I talked about: 0 to 255
intensity = glyphs[text[i]][x + GLYPH_WIDTH*y];
if (intensity == 255)
continue; // Don't draw it, fully transparent
else if (intensity == 0)
setPixel(x + xi, y + yi, color, base); // Fully opaque, can draw original color
else { // Here's the tricky part
// Get the pixel in the destination for averaging purposes
pixel = getPixel(x + xi, y + yi, base);
// transfer is an int for calculations
transfer = (int) ((float)((float) (255.0 - (float) intensity/255.0) * (float) color.red + (float) pixel.red)/2); // This is my attempt at averaging
newPixel.red = (Byte) transfer;
transfer = (int) ((float)((float) (255.0 - (float) intensity/255.0) * (float) color.green + (float) pixel.green)/2);
newPixel.green = (Byte) transfer;
// transfer = (int) ((float) ((float) 255.0 - (float) intensity)/255.0 * (((float) color.blue) + (float) pixel.blue)/2);
transfer = (int) ((float)((float) (255.0 - (float) intensity/255.0) * (float) color.blue + (float) pixel.blue)/2);
newPixel.blue = (Byte) transfer;
// Set the newpixel in the desired mem. position
setPixel(x+xi, y+yi, newPixel, base);
}
The results, as you can see, are less than desirable. That is a very zoomed in image, at 1:1 scale it looks like the text has a green "aura".
Any idea for how to properly compute this would be greatly appreciated.
Thanks for your time!
You need to blend the background and foreground colours. A-la:
pixelColour = newColour * intensity + backgroundColour * (1 - intensity)
By the way, this is a really slow way of rendering and blending fonts. You should instead render all the characters of the font to an off-screen surface with all the properties you need, and then use that as a texture to render to other surfaces when you need text.
Edit:
This doesn't look right:
(255.0 - (float) intensity/255.0)
It should instead be:
(255.0 - (float) intensity)/255.0
I believe that "aura" is caused by anti aliasing. The technique averages pixels with their neighbors.
I realize you don't seem to be using OpenGL but this chapter might help explain some of the theory. Wish I had a better answer, but hopefully this points you in the right direction. My first attempt would be to disable Antialiasing since it seems to do more harm than good. There is probably a better solution than that though.
It may be too much complicated to doing alpha blending pixel by pixel because current pixel value modifies next pixel value.
I would redesign the algorithm with the thinking of box wise blending.
With many getPixel calling for a single glyph, you can't produce proper target image.

Getting RGB values for each pixel from a raw image in C

I want to read the RGB values for each pixel from a raw image. Can someone tell me how to achieve this? Thanks for help!
the format of my raw image is .CR2 which come from camera.
Assuming the image is w * h pixels, and stored in true "packed" RGB format with no alpha component, each pixel will require three bytes.
In memory, the first line of the image might be represented in awesome ASCII graphics like this:
R0 G0 B0 R1 G1 B1 R2 G2 B2 ... R(w-1) G(w-1) B(w-1)
Here, each Rn Gn and Bn represents a single byte, giving the red, green or blue component of pixel n of that scanline. Note that the order of the bytes might be different for different "raw" formats; there's no agreed-upon world standard. Different environments (graphics cards, cameras, ...) do it differently for whatever reason, you simply have to know the layout.
Reading out a pixel can then be done by this function:
typedef unsigned char byte;
void get_pixel(const byte *image, unsigned int w,
unsigned int x,
unsigned int y,
byte *red, byte *green, byte *blue)
{
/* Compute pointer to first (red) byte of the desired pixel. */
const byte * pixel = image + w * y * 3 + 3 * x;
/* Copy R, G and B to outputs. */
*red = pixel[0];
*green = pixel[1];
*blue = pixel[2];
}
Notice how the height of the image is not needed for this to work, and how the function is free from bounds-checking. A production-quality function might be more armor-plated.
Update If you're worried this approach will be too slow, you can of course just loop over the pixels, instead:
unsigned int x, y;
const byte *pixel = /* ... assumed to be pointing at the data as per above */
for(y = 0; y < h; ++y)
{
for(x = 0; x < w; ++x, pixel += 3)
{
const byte red = pixel[0], green = pixel[1], blue = pixel[2];
/* Do something with the current pixel. */
}
}
None of the methods posted so far are likely to work with a camera "raw" file. The file formats for raw files are proprietary to each manufacturer, and may contain exposure data, calibration constants, and white balance information, in addition to the pixel data, which will likely be in a packed format where each pixel can takes up more than one byte, but less than two.
I'm sure there are open-source raw file converter programs out there that you could consult to find out the algorithms to use, but I don't know of any off the top of my head.
Just thought of an additional complication. The raw file does not store RGB values for each pixel. Each pixel records only one color. The other two colors have to be interpolated from heighboring pixels. You'll definitely be better off finding a program or library that works with your camera.
A RAW image is an uncompressed format, so you just have to point where your pixel is (skipping any possible header, and then adding the size of the pixel times the number columns times the number of row plus the number of the colum), and then read whatever binary data is giving a meaningful format to the layout of the data (with masks and shifts, you know).
That's the general procedure, for your current format you'll have to check the details.

Resources