I would expect that there's some sort of library that I can use in this fashion:
int* buffer[720*480]; // read in from file/memory/network stream
raw_params params;
params.depth = 16;
params.width = 720;
params.height = 480;
params.map = "rgb";
params.interleave = JPEG_RAW_INTERLEAVE_OFF;
jpeg_encode(buffer, params)
But I can't seem to find it.
Clarification
I'm looking for a simple example of how to use such a library as well. Code as robust as the source of netpbm and magick are too complex for my level of understanding.
Yes, there is a library for this (luckily!): http://freshmeat.net/projects/libjpeg/ equal (?) to http://sourceforge.net/projects/libjpeg/
The libjpeg library from the Independent JPEG Group is the reference implementation of the JPEG standard, written by folks who were and are involved in the creation of the standard. It is robust, stable, and highly portable.
It comes as a source kit, which includes extensive documentation and samples.
Its preferred representation for a source image is as an array pointers to of arrays of pixels, not as a single monolithic array as you show. You can easily create the array of pointers to rows to address your image buffer, however.
It is highly configurable and extensible. Specifically, it allows both the data source and the destination to be configured through call backs. This adds some complexity to a first use, but it isn't all that hard to deal with.
There are certainly commercial libraries available as well, and I know that there is commercially available IP for implementation of JPEG in hardware. A commercial library is likely to be faster, but will never be cheaper.
Here's a fairly simple example that's copied in a number of places on the web:
http://oaf.mutualads.org/svn/code/renderer/tr_image_jpg.c
http://www.torquepowered.com/community/forums/viewthread/45455
And this particular page of the documentation is relevant to understanding the require structs:
http://apodeline.free.fr/DOC/libjpeg/libjpeg-2.html
Related
I want to read a large binary file (1MB in size) into memory using Lua. The target device is mobile so I very much want to minimise the memory footprint.
From a quick look online it seems that Lua tables will use 16B for each sequential integer index (key) plus the space to store the value, which as I am storing binary data will hopefully only use 2 bits but lets just say 1 byte.
For 1e6 records that will be 1e6*17 =~ 17MB - which is huge!
From my brief reading it seems that I can use userdata to implement anything I want in C. I have not used C before but it seems that it would use
1b * 1e6 = 125kB
Shall i do this or have I got something very wrong / is there an easier way to do this.
Any advice or even name-calling for crappy calculations very much welcome :)
EDIT: Some interesting answers below about storing the data in a String (thanks!) and using bitwise ops. I just came accross an example in the PIL book (3rd edition pg293) that compares storing arrays of booleans in C so they use 3% of the mem. While this is cool and useful it may be overkill for me as the solutions below suggest I can fit in 1MB which is fine for me.
EDIT: Came across this C blob impl
EDIT: Solution - I read the file contents into a String as suggested and as Im using 5.1 had to use a 3rd party bit op lib - I went with a pure Lua implementation LuaBit. Thanks everyone!!
You can store a big blob in Lua string, it will work with any binary data. Now the question is what you want to do with the data. In any way, you can use string.byte to extract any individual byte, and use Lua's bit32 library to get down to bits. (For Lua 5.1 and older, you'll either have to write your own C routines, or use third party package.)
You can store the data in a string and manipulate it with the string library and Lua BitOp
Lua5.2's built-in bit32 library is preferred if available.
If you want to read 1 MB into memory, you won't end up with 250 kB...
If you read the file into a Lua string, you end up with 1 MB, as Lua strings are just 8-bit clean bytes.
After that, you can process the data according its structure, using the perhaps the struct library.
I have used several dynamically typed languages and I have been avoiding C but enough is enough, it's the right tool for the job sometimes and I need to get over it.
The things I miss working with C are associative arrays and large string libraries. Is there a library that gives more options then string.h? Any general advice when it comes to make the transition with strings?
Thanks for reading-Patrick
You can take a look at the Better String Library. The description from the site:
The Better String Library is an
abstraction of a string data type
which is superior to the C library
char buffer string type, or C++'s
std::string. Among the features
achieved are:
Substantial mitigation
of buffer overflow/overrun problems
and other failures that result from
erroneous usage of the common C string
library functions
Significantly
simplified string manipulation
High
performance interoperability with
other source/libraries which expect
'\0' terminated char buffers
Improved
overall performance of common string
operations
Functional equivalency with
other more modern languages
The
library is totally stand alone,
portable (known to work with gcc/g++,
MSVC++, Intel C++, WATCOM C/C++, Turbo
C, Borland C++, IBM's native CC
compiler on Windows, Linux and Mac OS
X), high performance, easy to use and
is not part of some other collection
of data structures. Even the file I/O
functions are totally abstracted (so
that other stream-like mechanisms,
like sockets, can be used.)
Nevertheless, it is adequate as a
complete replacement of the C string
library for string manipulation in any
C program.
POSIX gives you <string.h>, <strings.h> and <regex.h>.
If you really need more of a string library than this, C is probably not the right tool for that particular job.
As for a hash table, you can't get a type-safe hash table in C without a lot of nasty macros.
If you're OK with just storing void-pointers, or with doing some manual work for each type of map, then you shouldn't be lacking for options. Coding your own hash table is a hoot and a half - just search Stackoverflow for help with the hash function. If you don't want to roll your own, strmap [LGPL] looks decent.
GLib provides many pre-made data structures and string handling functions, but it's a set of functions and types completely separated from the "usual" ones, and it's not a very lightweight dependency.
If instead C++ is a viable alternative for your task, it bundles a string class and several generic containers ready-made into the standard library (and much other related stuff can be found in Boost).
What specifically are you looking for in your extended c-string library?
One way to get better at C, is to create your own c-string library. Then make it open source, and let others help refine it.
I don't usually advocate creating your own string libaries, but w.r.t. C, it's a great way to learn C.
Much of the power of C consists of the ability to have direct control over the memory as a sequence of bytes. It is a bit against the philosophy of the language to treat strings as something higher-level than that.
I would recommend rolling your own very basic one. It will be an enlightening experience especially to learn pointer arithmetics and loops.
For example, learn about "Schlemiel the Painter's algorithm" regarding strcat and design your library to solve this problem.
I've not used it myself, but you should at least review the SEI/CERT library Specifications for Managed Strings, 2nd Edition. The code can be found at CERT.
An associative array associating string keys and struct values in C consists of:
A hash function for strings
An array with a prime number of elements, inside each of which is a linked-list head.
Linked-list elements containing char * pointers to the stored keys and (optionally) a struct * pointer to the corresponding value for each key.
To store a string key in your associative array:
Hash it modulo that prime array size.
In that array bin, add it to the linked-list.
Assign the value pointer to the value you are adding.
I'm writing a very small, project-specific OpenGLES engine for iphone and I really need to use a good, solid, and proven dynamic array library/macro in c99 dialect. (No C++, Obj-C, stl whatsoever)
It's strongly necessary for render batch and polygon mesh, so it should be able to handle various types of data, and additionally causes minimal overhead when array size changes and new data is inserted.
I've been searching around and found two candidates for my need.
the first one is from ccCArray from Cocos2d.
and another one is utarray written by Troy D. Hanson.
ccCArray IS rock solid, thoroughly proven by community. utarray looks fine but I cannot find anyone actually uses it.
Any more suggestion?
A library ?! A C++ template would be more than suitable for this need. I'd say about AT MOST 15 functions (excluding alternative constructors and const getters), and you're done. Also able to use it for ANY type, ANY size and ANY size type (byte, int etc.) And it's just one file: a .h or, better said, a .hpp
Any reason you're rejecting it ? Seems like you want to make life harder for yourself :)
I'm looking for a very portable, minimalistic/small XML/configuration language library in ANSI C with no external dependencies (or very few), compiling down to less than 100K. I need it for a moderately complex configuration file, and it must support Unicode.
Some more requirements:
OK to use/embed/statically link into proprietary code. Credit will always will be given where credit is due.
Not necessarily XML.
Really, clean code/no weird or inconsistent string handling.
UTF-8.
Thank you fellas.
This is somehow similar to this question: Is there a good tiny XML parser for an embedded C project?
I was able to tweak the compilation flags of the following XML parser libraries for C, and cut down more than 50% of their size on my Ubuntu machine. Mini-XML is the only one close to what you requested:
Mini-XML (36K)
Expat (124K)
RXP (184K)
IMO Protocol Buffers are a much, much better solution for this kind of use case than XML. With protocol buffers you get real types and a schema without having to layer them on top of base XML. Also the syntax is nicer.
// The schema file: can serve as documentation for what
// configuration values are available.
message MyAppConfig {
// Set to control the port the app listens on.
optional int32 port = 1 [default=1234];
// Set to control the local hostname.
optional string hostname = 2 [default="localhost"];
}
Then the user's actual config would look like this:
# I want to listen on a very high port.
port: 50000
The main protocol buffer library does not fit your criteria because it is in C++ and is very large. I am working on a much smaller implementation of the same called upb (ie. "micro" protobufs). It is written in ~5k lines of ANSI C and compiles to <50k.
Protocol buffers have both a binary and a text format, which are equivalent. My library does not (yet) support reading the text format, but you could have your users use the Google standard tool for converting the text version of their config to binary format ahead of time. Then your app itself would only need to read the binary format, and could just use upb.
upb is just now getting to the point where adventurous users could try it out, but it's a bit rough around the edges still and the APIs are still changing somewhat. If you're ok with this, you might try diving in now. If you prefer something more stable, at least keep upb on your radar.
I am wondering how I can both import and export bitmaps to and from C. I'm basically lost on where to begin.
A bitmap in memory looks similar to this:
struct Image {
int width;
int height;
char *data; // 1 byte per channel & only 1 channel == grayscale
}
struct Image theImage;
theImage.width = 100;
theImage.height = 100;
theImage.data = malloc(sizeof(char) * theImage.width * theImage.height);
As to importing and exporting, there are some really simple file formats out there, take a look at BMP. For more complex formats you best use an already available library.
Most frameworks already have load/save methods for the most common fileformats. You could take a look at SDL if you're looking for a lightweight library.
Have you taken a look at ImageMagick's C API wrapper, MagickWand? Here's the documentation if you want to peruse.
I like using SDL with the dummy driver. You can draw onto an in-memory buffer just like you would onto a screen, then save it out to a PNG or whatever with SDL_image or similar.
Another popular library for this is GD.
The simple answer is to use an appropriate library. What library is appropriate will depend on what platform you are using. On a GUI platform the GUI API/Framework will include these facilities.
I like the netpbm/pbmplus tools, although I usually like the command line; the API is efficient but not much fun to use.
This semester I wrote a significant amount of software for beginning students to use to manipulate images; you might want to check out the homework assignments and supporting software for the Tufts course Machine Structure and Assembly-Language Programming.
The term "bitmap" is somewhat generic, unless you specifically mean a Windows BMP.
I'd recommend using an image processing library on your platform (like gd). A good graphics library has routines to do input and output with images in various formats.
FreeImage is excellent. I've used this for my own game development work, and it supports tons of formats. Here's the list of features and formats supported - http://freeimage.sourceforge.net/features.html