gcc (GCC) 4.7.2
PJ SIP 2.1
Hello,
I am developing an application that will use the PJSIP API.
Just looking at the API documentation and I see some functions that seem to be just wrappers for the standard C library. i.e. pj_memset, pj_strncpy, pj_strlen, etc.
I can see some alternatives that might be worth considering pj_strncpy_with_null() which will always NULL terminate a string. A another advantage could be is that the pjsip uses a pj_str_t structure to store the string and the size. Which could be better than using a normal C string.
And is there any point using pj_size_t over size_t which is portable anyway?
The link for quick reference is here:
http://www.pjsip.org/pjlib/docs/html/group__PJ__PSTR.htm
It there any real advantage using PJSIP over the standard C library?
Many thanks for any suggestions,
Short answer: Use the PJSIP API (all of it).
Long answer: It depends.
If you were programming an application for standard Desktops, that is, x86/x64 Windows/Mac/Linux, then no, it wouldn't really matter too much if you used the standard C library or wrappers like the PJSIP functions. Practically, of course, there might be functions that take (as you pointed out) the pj_str_t struct instead of a char *; it would be easier then to use the PJSIP API just to simplify and remove the need for conversions.
The reason for wrappers, I'm assuming, is to make it easier to develop on embedded devices. I don't mean just ARM or other non-x86 processors—though it could apply there as well; I mean custom embedded devices: things that have a very specific purpose and change infrequently. These embedded devices have very limited capabilities and sometimes even lack an OS. Without an OS, these processors might not have a malloc function or the like. Frequently, the libraries associated with the devices, since they are customized so much, are not entirely "standard" and differ in some small way. By having wrappers for everything, PJSIP can avoid most issues and even provide implementations across the board for things such as strcpy or malloc such that all devices run the "same" code.
Wrappers also provide the means for "hooks." Hooks enable better error messaging (and possibly handling). It's unclear whether PJSIP is doing this (I have never used PJSIP—I am talking from experience using other frameworks), but I am pointing it out just to show why a framework might bother wrapping everything.
In the end, it boils down to your purpose: if you chose to use PJSIP in the first place, then I would go all out and use all of its API. If you are only using it in a few places (for whatever reason) then it probably doesn't matter. Again, it appears that PJSIP is targeting embedded devices (it lists Nokia and even RTOS systems), where it is fairly common to provide wrappers for even "standard" functions. If this is the case, and you are using it in this way, definitely use the entire API.
Will you be sticking with pjsip?
PJSIP source code ("The Software") is licensed under both General
Public License (GPL) version 2 or later and a proprietary license that
can be arranged...
If you think the GPL may be too restrictive for future expansion (such as Android's no-GPL-in-userspace policy) and their proprietary license is not acceptable, you may benefit from using your own portable code/wrappers that you could use with a less restrictive BSD stlye library like Baresip
There are plenty of other methods to provide needed functionality where the standard C library does not support it, many of which will be better tested (I hate to mention autotools, but... it does support most platforms - some would say too many) Or you could include implementations/adaptations from musl-libc
Another thing to consider is the C api is based on standards and fairly set in stone while the wrappers in a given project are much more free to break API compatibility from version to version (just ask a glib/gtk programmer)
Related
I'm trying to write a language runtime (and a language itself) that is similar to .NET or to the JVM. It's got a form of bytecode that is custom.
What I want is a way to translate said bytecode to actual, runnable machine code. So, because I'm not wanting to write such a translator myself (this is more of a toy project/personal side project) I want to find a good JIT library to use.
Here's what I want the library to do:
The library should be as easy to use as possible (toy project and I don't really have much experience here)
The library should support at least x86_64 (development machine), though preferably it should cover other architectures as well
The library should preferably do some low level optimizations (register tracking and allocation, reducing memory accesses etc); those optimizations shouldn't be very expensive to do though (I will myself do other optimizations to e.g. remove virtual calls and convert them to direct ones, for example). I can accept a library with no optimization if it's easiest to use though.
The library must have an interface that is usable from C (preferred) or C++ (acceptable).
I will use Boehm GC for garbage collection, if it matters (probably doesn't, but just in case). Maybe a compacting GC would be nice, but I guess I shouldn't combine the questions...
I would suggest llvm. There are some great tutorials on how to implement your own language with it and basic stuff is not too complicated. You also get the option to do a lot of more advanced stuff later on. As a bonus not only can you use JIT but you can also statically compile and optimize your binaries. LLVM also does have a C interface and can target all common CPU architectures and even a lot of more obscure ones.
I was looking at the NtDll export table on my Windows 10 computer, and I found that it exports standard C runtime functions, like memcpy, sprintf, strlen, etc.
Does that mean that I can call them dynamically at runtime through LoadLibrary and GetProcAddress? Is this guaranteed to be the case for every Windows version?
If so, it is possible to drop the C runtime library altogether (by just using the CRT functions from NtDll), therefore making my program smaller?
There is absolutely no reason to call these undocumented functions exported by NtDll. Windows exports all of the essential C runtime functions as documented wrappers from the standard system libraries, namely Kernel32. If you absolutely cannot link to the C Runtime Library*, then you should be calling these functions. For memory, you have the basic HeapAlloc and HeapFree (or perhaps VirtualAlloc and VirtualFree), ZeroMemory, FillMemory, MoveMemory, CopyMemory, etc. For string manipulation, the important CRT functions are all there, prefixed with an l: lstrlen, lstrcat, lstrcpy, lstrcmp, etc. The odd man out is wsprintf (and its brother wvsprintf), which not only has a different prefix but also doesn't support floating-point values (Windows itself had no floating-point code in the early days when these functions were first exported and documented.) There are a variety of other helper functions, too, that replicate functionality in the CRT, like IsCharLower, CharLower, CharLowerBuff, etc.
Here is an old knowledge base article that documents some of the Win32 Equivalents for C Run-Time Functions. There are likely other relevant Win32 functions that you would probably need if you were re-implementing the functionality of the CRT, but these are the direct, drop-in replacements.
Some of these are absolutely required by the infrastructure of the operating system, and would be called internally by any CRT implementation. This category includes things like HeapAlloc and HeapFree, which are the responsibility of the operating system. A runtime library only wraps those, providing a nice standard-C interface and some other niceties on top of the nitty-gritty OS-level details. Others, like the string manipulation functions, are just exported wrappers around an internal Windows version of the CRT (except that it's a really old version of the CRT, fixed back at some time in history, save for possibly major security holes that have gotten patched over the years). Still others are almost completely superfluous, or seem so, like ZeroMemory and MoveMemory, but are actually exported so that they can be used from environments where there is no C Runtime Library, like classic Visual Basic (VB 6).
It is also interesting to point out that many of the "simple" C Runtime Library functions are implemented by Microsoft's (and other vendors') compiler as intrinsic functions, with special handling. This means that they can be highly optimized. Basically, the relevant object code is emitted inline, directly in your application's binary, avoiding the need for a potentially expensive function call. Allowing the compiler to generate inlined code for something like strlen, that gets called all the time, will almost undoubtedly lead to better performance than having to pay the cost of a function call to one of the exported Windows APIs. There is no way for the compiler to "inline" lstrlen; it gets called just like any other function. This gets you back to the classic tradeoff between speed and size. Sometimes a smaller binary is faster, but sometimes it's not. Not having to link the CRT will produce a smaller binary, since it uses function calls rather than inline implementations, but probably won't produce faster code in the general case.
* However, you really should be linking to the C Runtime Library bundled with your compiler, for a variety of reasons, not the least of which is security updates that can be distributed to all versions of the operating system via updated versions of the runtime libraries. You have to have a really good reason not to use the CRT, such as if you are trying to build the world's smallest executable. And not having these functions available will only be the first of your hurdles. The CRT handles a lot of stuff for you that you don't normally even have to think about, like getting the process up and running, setting up a standard C or C++ environment, parsing the command line arguments, running static initializers, implementing constructors and destructors (if you're writing C++), supporting structured exception handling (SEH, which is used for C++ exceptions, too) and so on. I have gotten a simple C app to compile without a dependency on the CRT, but it took quite a bit of fiddling, and I certainly wouldn't recommend it for anything remotely serious. Matthew Wilson wrote an article a long time ago about Avoiding the Visual C++ Runtime Library. It is largely out of date, because it focuses on the Visual C++ 6 development environment, but a lot of the big picture stuff is still relevant. Matt Pietrek wrote an article about this in the Microsoft Journal a long while ago, too. The title was "Under the Hood: Reduce EXE and DLL Size with LIBCTINY.LIB". A copy can still be found on MSDN and, in case that becomes inaccessible during one of Microsoft's reorganizations, on the Wayback Machine. (Hat tip to IInspectable and Gertjan Brouwer for digging up the links!)
If your concern is just the need to distribute the C Runtime Library DLL(s) alongside your application, you can consider statically linking to the CRT. This embeds the code into your executable, and eliminates the requirement for the separate DLLs. Again, this bloats your executable, but does make it simpler to deploy without the need for an installer or even a ZIP file. The big caveat of this, naturally, is that you cannot benefit to incremental security updates to the CRT DLLs; you have to recompile and redistribute the application to get those fixes. For toy apps with no other dependencies, I often choose to statically link; otherwise, dynamically linking is still the recommended scenario.
There are some C runtime functions in NtDll. According to Windows Internals these are limited to string manipulation functions. There are other equivalents such as using HeapAlloc instead of malloc, so you may get away with it depending on your requirements.
Although these functions are acknowledged by Microsoft publications and have been used for many years by the kernel programmers, they are not part of the official Windows API and you should not use of them for anything other than toy or demo programs as their presence and function may change.
You may want to read a discussion of the option for doing this for the Rust language here.
Does that mean that I can call them dynamically at runtime through
LoadLibrary and GetProcAddress?
yes. even more - why not use ntdll.lib (or ntdllp.lib) for static binding to ntdll ? and after this you can direct call this functions without any GetProcAddress
Is this guaranteed to be the case for every Windows version?
from nt4 to win10 exist many C runtime functions in ntdll, but it set is different. usual it grow from version to version. but some of then less functional compare msvcrt.dll . say for example printf from ntdll not support floating point format, but in general functional is same
it is possible to drop the C runtime library altogether (by just using
the CRT functions from NtDll), therefore making my program smaller?
yes, this is 100% possible.
Is there a C library available for operations such as file operations, getting system information and the like which is generic, which can be used when compiled in different platforms and which behaves in a similar way?
Edit: Something like Java or .NET platform abstracting the hardware.
Have you tried the standard library? It should be implemented on any system that has an ISO compliant C runtime.
Yes; the ISO Standard C library. It may not cover all the functionality you want, but that is exactly because it is generic, and as such is also lowest common denominator. It only supports features that can reasonably be expected to exist on most hardware, including embedded systems.
The way to approach this is perhaps to specify the range of target platforms you need to support, and then the application domains (e.g. GUI, networking, multi-threading, image processing, file handling etc.), and then select the individual cross-platform libraries that suit your needs. There is probably no one library to fulfil all your needs, and in some cases no common library at all.
That said, you will always be better served in this respect by embracing C++ where you can use any C library as well as C++ libraries. Not only is the C++ standard library larger, but libraries such as Boost, wxWidgets, ACE cover a broader domain spectrum too. Another approach is to use a cross-platform language such as Java, which solves the problem by abstracting the hardware to a virtual machine. Similarly .NET/Mono and C# may provide a solution for suitably limited set of target platforms.
Added following comment:
Hardware abstraction in a real-machine targeted language (as opposed to a VM language such as Java or CLR based languages) is provided by the operating system, so what you perhaps need is a common operating system API. POSIX is probably the closest you will get to that, being supported on Linux, Unix, OSX (which is Unix), QNX, VxWorks, BeOS and many others; but not importantly Windows. One way of using POSIX on Windows is to use Cygwin. Another is to use a VM to host a POSIX OS such as Linux.
For anything not found in the standard library, GLib is a good first place to look along with other libraries built to interact with it. It offers for example threads, mutexes and IPC that you won't be able to write portably using plain standard libraries, and you can use many more GNU libraries that follow the same conventions up to a full GUI in GTK+. GLib supports the usual popular operating systems.
If the C standard library is not sufficient for your needs, a minimal hyperportable subset of POSIX might be the target you want to code to. For example, the main non-POSIX operating system Windows still has a number of functions with the same names as POSIX functions which behave reasonably closely - open, read, write, etc.
Documenting what exactly a "hyperportable subset of POSIX" includes, and which non-POSIX operating systems would conform to such a subset, is a moderately difficult task, which would be quite useful in and of itself for the sake of avoiding the plague of "MyCompanyName portable runtime" products which appear again and again every few years and unnecessarily bloat popular software like Firefox and Apache.
If you need facilities beyond what the Standard C library provides, take a look at the Apache Portable Runtime (APR). You should also review whether POSIX provides the functionality you are after, though that comes with its own bag of worms.
If you want to get into graphics and the like, then you are into a different world - GTK, Glib and Qt spring to mind, though I've not used any of them.
Are there any widely used I/O stream abstraction layers for plain C?
By an I/O stream abstraction layer I mean any layer that at least allows the creation of custom read/write functions. For C++, there's the standard iostream and boost::iostreams. For glibc users, there's a possibility to use custom streams. These won't do any good if the requirement is to write portable C code.
It is better to use a library that is either widely used or easy to embed to the source code, or both.
SDL_RWops is an undocumented but widely used feature of SDL, and zziplib can use it. However, it does not make much sense to add a dependency to SDL just to get this feature.
GLib contains a GIOChannel abstraction, but the library is again quite large and they say that "support for Windows is only partially complete".
However, the above are not quite satisfactory for small libraries, such as decoders, file format readers and signal processors: they contain lots of unnecessary stuff and the LGPL licensing prevents embedding the relevant parts to non-GPL code.
The BIO abstraction in OpenSSL sounds like it fits the bill.
I think you answered your own question.
No, there are no widely used steam abstraction libraries. Those that exist are usually a small part of libraries that most people don't want to depend on, or are very specialized.
What are your requirements? What is it you are looking for out of an abstraction library? I have to admit that every time I've looked at one, or started writing one, I usually end up back at the standard POSIX interfaces... what more abstraction could one want?
There is libslack (GNU GPL) which may provide some of the functionality you are after and this MIT License-ed input stream wrapper:
http://attractivechaos.wordpress.com/2008/10/11/a-generic-buffered-stream-wrapper/
The lexical analyzer generator Quex comes along with an input stream abtraction for POSIX, C++ stream, etc. It consists of a collection of classes derived from a class called ByteLoader.
It is easily adaptable to any infrastructure-simply by deriving from ByteLoader and implemting the interface. See: Link to Code.
A 'real' ByteLoader is only instantiated upon generation of a lexer, though.
I've been tasked with adding streams support (C89/C90) to the libraries for my company's legacy embedded C compiler. Our target hardware typically has 1MB or less of code space and does not have an operating system.
We have a lot of stream-like implementations throughout the codebase that I can use as a starting point. For example, a console that works over a TCP sockets or serial port, a web server that reads from FAT on SD card or in-memory file, and even a firmware updater that reads from many sources.
Before I go and re-invent the wheel, I'm wondering if there are existing implementations that I could either port or use as a starting point for my work. Even though we provide full source code to our customers, GPL-licensed code isn't an option since our customers don't want to release source code to their products.
Can anyone recommend a book (annotated Unix source, CompSci text) or public domain/BSD-licensed source? I'd prefer to look at an older OS targeted to a single device, as current operating systems contain a tangle of macros and layers of typedefs that make following even a simple struct definition difficult.
Take a look at P.J. Plauger's book The Standard C Library, which describes in detail one possible implementation of the complete C89 standard library.
You should be able to pull most of what you need from the source code for the GNU C standard library. It is licensed with the Lesser GPL, which means you can link to the library without affecting the license of your software (or forcing your customers to release their code). Porting this to your platform (thus keeping the LGPL-ed code in its own library) may be easier than implementing your own from scratch.
Several different projects have taken GNU GLIBC and optimized it for embedded systems. You may want to look at:
Embedded GLIBC (LGPL)
uLIBC (LGPL)
Newlib (multiple free licenses)
In particular, EGLIBC and uLIBC were designed to run properly on embedded systems that lack a MMU.
You can also have a look at BSD's implementation of libc
Alternatively there is STLSoft, who provides several libraries (including the C standard lib) under a BSD license. I can't attest to their quality since I haven't used their code myself, but it might be worth looking at if you can't work LGPL-ed code into your project.
Wouldn't *BSD (Net|Open|Free)'s libc be suitable? At least as a starting point.
Try looking at http://www.minix3.org/
Check your development tools. Some development tools come with their on source for their software libraries.
I took the source for the Compiler's printf and adapted for a debug port on an embedded system. There is less work when you have a foundation to build from.