How to implement a message queue in standard C - c

I have a project (micro controller STM32 using c code) where I need to receive messages from serial port (for example strings) and I need to put the messages in a queue where I will read the string later.
Can someone tell me where can I find some example on how to create a message queue (like FIFO) of strings (or byte array) using standard C and how to manage the queue? Thanks for any kind of support.

"example on how to create e message queue (like FIFO) of strings (or byte array) using standard C and how to manage the queue"
"in a micro controller with a standard C you should manage the buffers, create the queue, enqueue and dequeue the elements"
The example given below should meet the requirements.
If necessary, the library functions used can easily be replaced with platform-specific versions or standard C array operations.
The memory allocation for the queue can also be done as static variable instead of as stack variable. If desired, even malloc could be used.
The message type can easily be extended. The queue and data sizes are defined as constants.
#leonardo gave a good hint on how to structure the processing, i.e. enqueuing messages in an interrupt routine and dequeuing them on main. I guess that some kind of semaphore needs to be used so that the execution of functions which manipulate the queue don't get mixed up. Some thoughts on this are discussed in semaphore like synchronization in ISR (Interrupt service routine)
/*
Portable array-based cyclic FIFO queue.
*/
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#define MESSAGE_SIZE 64
#define QUEUE_SIZE 3
typedef struct {
char data[MESSAGE_SIZE];
} MESSAGE;
typedef struct {
MESSAGE messages[QUEUE_SIZE];
int begin;
int end;
int current_load;
} QUEUE;
void init_queue(QUEUE *queue) {
queue->begin = 0;
queue->end = 0;
queue->current_load = 0;
memset(&queue->messages[0], 0, QUEUE_SIZE * sizeof(MESSAGE_SIZE));
}
bool enque(QUEUE *queue, MESSAGE *message) {
if (queue->current_load < QUEUE_SIZE) {
if (queue->end == QUEUE_SIZE) {
queue->end = 0;
}
queue->messages[queue->end] = *message;
queue->end++;
queue->current_load++;
return true;
} else {
return false;
}
}
bool deque(QUEUE *queue, MESSAGE *message) {
if (queue->current_load > 0) {
*message = queue->messages[queue->begin];
memset(&queue->messages[queue->begin], 0, sizeof(MESSAGE));
queue->begin = (queue->begin + 1) % QUEUE_SIZE;
queue->current_load--;
return true;
} else {
return false;
}
}
int main(int argc, char** argv) {
QUEUE queue;
init_queue(&queue);
MESSAGE message1 = {"This is"};
MESSAGE message2 = {"a simple"};
MESSAGE message3 = {"queue!"};
enque(&queue, &message1);
enque(&queue, &message2);
enque(&queue, &message3);
MESSAGE rec;
while (deque(&queue, &rec)) {
printf("%s\n", &rec.data[0]);
}
}
Compiling and running:
$ gcc -Wall queue.c
$ ./a.out
This is
a simple
queue!
$

The C language does not have a queue build in (it's a battery excluded language), you need to build your own. If you just need a FIFO to push things on your interrupt routine and then pop them out on your main loop (which is good design BTW), Check A Simple Message Queue for C if this works for you.

Related

Callbacks in AIO asynchronous I/O

I have found discussion on using callbacks in AIO asynchronous I/O on the internet. However, what I have found has left me confused. An example code is listed below from a site on Linux AIO. In this code, AIO is being used to read in the contents of a file. My problem is that it seems to me that a code that actually processes the contents of that file must have some point where some kind of block is made to the execution until the read is completed. This code here has no block like that at all. I was expecting to see some kind of call analogous to pthread_mutex_lock in pthread programming. I suppose I could put in a dummy loop after the aio_read() call that would block execution until the read is completed. But that puts me right back to the simplest way of blocking the execution, and then I don't see what is gained by all the coding overhead that goes into establishing a callback. I am obviously missing something. Could someone tell me what it is?
Here is the code. (BTW, the original is in C++; I have adapted it to C.)
#include <stdio.h>
#include <stdlib.h>
#include <strings.h>
#include <aio.h>
//#include <bits/stdc++.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <signal.h>
const int BUFSIZE = 1024;
void aio_completion_handler(sigval_t sigval)
{
struct aiocb *req;
req = (struct aiocb *)sigval.sival_ptr; //Pay attention here.
/*Check again if the asynchrony is complete?*/
if (aio_error(req) == 0)
{
int ret = aio_return(req);
printf("ret == %d\n", ret);
printf("%s\n", (char *)req->aio_buf);
}
close(req->aio_fildes);
free((void *)req->aio_buf);
while (1)
{
printf("The callback function is being executed...\n");
sleep(1);
}
}
int main(void)
{
struct aiocb my_aiocb;
int fd = open("file.txt", O_RDONLY);
if (fd < 0)
perror("open");
bzero((char *)&my_aiocb, sizeof(my_aiocb));
my_aiocb.aio_buf = malloc(BUFSIZE);
if (!my_aiocb.aio_buf)
perror("my_aiocb.aio_buf");
my_aiocb.aio_fildes = fd;
my_aiocb.aio_nbytes = BUFSIZE;
my_aiocb.aio_offset = 0;
//Fill in callback information
/*
Using SIGEV_THREAD to request a thread callback function as a notification method
*/
my_aiocb.aio_sigevent.sigev_notify = SIGEV_THREAD;
my_aiocb.aio_sigevent.sigev_notify_function = aio_completion_handler;
my_aiocb.aio_sigevent.sigev_notify_attributes = NULL;
/*
The context to be transmitted is loaded into the handler (in this case, a reference to the aiocb request itself).
In this handler, we simply refer to the arrived sigval pointer and use the AIO function to verify that the request has been completed.
*/
my_aiocb.aio_sigevent.sigev_value.sival_ptr = &my_aiocb;
int ret = aio_read(&my_aiocb);
if (ret < 0)
perror("aio_read");
/* <---- A real code would process the data read from the file.
* So execution needs to be blocked until it is clear that the
* read is complete. Right here I could put in:
* while (aio_error(%my_aiocb) == EINPROGRESS) {}
* But is there some other way involving a callback?
* If not, what has creating a callback done for me?
*/
//The calling process continues to execute
while (1)
{
printf("The main thread continues to execute...\n");
sleep(1);
}
return 0;
}

libuv simple send udp

I'm doing a multiplatform shared library in C, which sends UDP messages using libuv, however I don't know much about libuv and I don't know if my implementation is good, or if there is another solution besides libuv.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <uv.h>
#define IP "0.0.0.0"
#define PORT 8090
#define STR_BUFFER 256
void on_send(uv_udp_send_t *req, int status) {
if (status) {
fprintf(stderr, "Send error %s\n", uv_strerror(status));
return;
}
}
int send_udp(char *msg){
uv_loop_t *loop = malloc(sizeof(uv_loop_t));
uv_loop_init(loop);
uv_udp_t send_socket;
uv_udp_init(loop, &send_socket);
struct sockaddr_in send_addr;
uv_ip4_addr(IP, PORT, &send_addr);
uv_udp_bind(&send_socket, (const struct sockaddr*)&send_addr, 0);
char buff[STR_BUFFER];
memset(buff,0,STR_BUFFER);
strcpy(buff,msg);
uv_buf_t buffer = uv_buf_init(buff,STR_BUFFER);
uv_udp_send_t send_req;
uv_udp_send(&send_req, &send_socket, &buffer, 1, (const struct sockaddr*)&send_addr, on_send);
uv_run(loop, UV_RUN_ONCE);
uv_loop_close(loop);
free(loop);
return 0;
}
int main() {
send_udp("test 123\n");
return 0;
}
Your implementation has multiple issues to date:
I'm not sure a single loop iteration is enough to send an UDP message on every platform. This is something you can check easily with the value returned by uv_run, see the documentation for uv_run when using the UV_RUN_ONCE mode:
UV_RUN_ONCE: Poll for i/o once. Note that this function blocks if there are no pending callbacks. Returns zero when done (no active handles or requests left), or non-zero if more callbacks are expected (meaning you should run the event loop again sometime in the future).
If you would keep your code as-is, I would suggest to do at least this:
int done;
do {
done = uv_run(loop, UV_RUN_ONCE);
} while (done != 0);
But keep on reading, you can do even better ! :)
It's quite costly in terms of performance, uv_loops are supposed to be long lasting, not to be created for each message sent.
Incomplete error handling: uv_udp_bind, uv_udp_send, ... they can fail !
How to improve
I would suggest you to change your code for one of the two following solutions:
Your library is used in a libuv context (a.k.a, you don't try to hide the libuv implementation detail but require all people who wish to use your library to use libuv explicitly.
You could then change your function signature to something like int send_udp(uv_loop_t *loop, char *msg) and let the library users manage the event loop and run it.
Your library uses libuv as an implementation detail: you don't want to bother your library users with libuv, therefore its your reponsibility to provide robust and performant code. This is how I would do it:
mylib_init: starts a thread and run an uv_loop on it
send_udp: push the message on a queue (beware of thread-safety), notify your loop it has a message to send (you can use uv_async for this), then you can send the message with approximately the same code you are already using.
mylib_shutdown: stop the loop and the thread (again, you can use an uv_async to call uv_stop from the right thread)
It would look like this (I don't have a compiler to test, but you'll have most of the work done):
static uv_thread_t thread; // our network thread
static uv_loop_t loop; // the loop running on the thread
static uv_async_t notify_send; // to notify the thread it has messages to send
static uv_async_t notify_shutdown; // to notify the thread it must shutdown
static queue_t buffer_queue; // a queue of messages to send
static uv_mutex_t buffer_queue_mutex; // to sync access to the queue from the various threads
static void thread_entry(void *arg);
static void on_send_messages(uv_async_t *handle);
static void on_shutdown(uv_async_t *handle);
int mylib_init() {
// will call thread_entry on a new thread, our network thread
return uv_thread_create(&thread, thread_entry, NULL);
}
int send_udp(char *msg) {
uv_mutex_lock(&buffer_queue_mutex);
queue_enqueue(&buffer_queue, strdup(msg)); // don't forget to free() after sending the message
uv_async_send(&notify_send);
uv_mutex_unlock(&buffer_queue_mutex);
}
int mylib_shutdown() {
// will call on_shutdown on the loop thread
uv_async_send(&notify_shutdown);
// wait for the thread to stop
return uv_thread_join(&thread);
}
static void thread_entry(void *arg) {
uv_loop_init(&loop);
uv_mutex_init_recursive(&buffer_queue_mutex);
uv_async_init(&loop, &notify_send, on_send_messages);
uv_async_init(&loop, &notify_shutdown, on_shutdown);
uv_run(&loop, UV_RUN_DEFAULT); // this code will not return until uv_stop is called
uv_mutex_destroy(&buffer_queue_mutex);
uv_loop_close(&loop);
}
static void on_send_messages(uv_async_t *handle) {
uv_mutex_lock(&buffer_queue_mutex);
char *msg = NULL;
// for each member of the queue ...
while (queue_dequeue(&buffer_queue, &msg) == 0) {
// create a uv_udp_t, send the message
}
uv_mutex_unlock(&buffer_queue_mutex);
}
static void on_shutdown(uv_async_t *handle) {
uv_stop(&loop);
}
It's up to you to develop or find a queue implementation ;)
Usage
int main() {
mylib_init();
send_udp("my super message");
mylib_shutdown();
}

Why does TCP socket slow down if done in multiple system calls?

Why is the following code slow? And by slow I mean 100x-1000x slow. It just repeatedly performs read/write directly on a TCP socket. The curious part is that it remains slow only if I use two function calls for both read AND write as shown below. If I change either the server or the client code to use a single function call (as in the comments), it becomes super fast.
Code snippet:
int main(...) {
int sock = ...; // open TCP socket
int i;
char buf[100000];
for(i=0;i<2000;++i)
{ if(amServer)
{ write(sock,buf,10);
// read(sock,buf,20);
read(sock,buf,10);
read(sock,buf,10);
}else
{ read(sock,buf,10);
// write(sock,buf,20);
write(sock,buf,10);
write(sock,buf,10);
}
}
close(sock);
}
We stumbled on this in a larger program, that was actually using stdio buffering. It mysteriously became sluggish the moment payload size exceeded the buffer size by a small margin. Then I did some digging around with strace, and finally boiled the problem down to this. I can solve this by fooling around with buffering strategy, but I'd really like to know what on earth is going on here. On my machine, it goes from 0.030 s to over a minute on my machine (tested both locally and over remote machines) when I change the two read calls to a single call.
These tests were done on various Linux distros, and various kernel versions. Same result.
Fully runnable code with networking boilerplate:
#include <netdb.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <netinet/ip.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
static int getsockaddr(const char* name,const char* port, struct sockaddr* res)
{
struct addrinfo* list;
if(getaddrinfo(name,port,NULL,&list) < 0) return -1;
for(;list!=NULL && list->ai_family!=AF_INET;list=list->ai_next);
if(!list) return -1;
memcpy(res,list->ai_addr,list->ai_addrlen);
freeaddrinfo(list);
return 0;
}
// used as sock=tcpConnect(...); ...; close(sock);
static int tcpConnect(struct sockaddr_in* sa)
{
int outsock;
if((outsock=socket(AF_INET,SOCK_STREAM,0))<0) return -1;
if(connect(outsock,(struct sockaddr*)sa,sizeof(*sa))<0) return -1;
return outsock;
}
int tcpConnectTo(const char* server, const char* port)
{
struct sockaddr_in sa;
if(getsockaddr(server,port,(struct sockaddr*)&sa)<0) return -1;
int sock=tcpConnect(&sa); if(sock<0) return -1;
return sock;
}
int tcpListenAny(const char* portn)
{
in_port_t port;
int outsock;
if(sscanf(portn,"%hu",&port)<1) return -1;
if((outsock=socket(AF_INET,SOCK_STREAM,0))<0) return -1;
int reuse = 1;
if(setsockopt(outsock,SOL_SOCKET,SO_REUSEADDR,
(const char*)&reuse,sizeof(reuse))<0) return fprintf(stderr,"setsockopt() failed\n"),-1;
struct sockaddr_in sa = { .sin_family=AF_INET, .sin_port=htons(port)
, .sin_addr={INADDR_ANY} };
if(bind(outsock,(struct sockaddr*)&sa,sizeof(sa))<0) return fprintf(stderr,"Bind failed\n"),-1;
if(listen(outsock,SOMAXCONN)<0) return fprintf(stderr,"Listen failed\n"),-1;
return outsock;
}
int tcpAccept(const char* port)
{
int listenSock, sock;
listenSock = tcpListenAny(port);
if((sock=accept(listenSock,0,0))<0) return fprintf(stderr,"Accept failed\n"),-1;
close(listenSock);
return sock;
}
void writeLoop(int fd,const char* buf,size_t n)
{
// Don't even bother incrementing buffer pointer
while(n) n-=write(fd,buf,n);
}
void readLoop(int fd,char* buf,size_t n)
{
while(n) n-=read(fd,buf,n);
}
int main(int argc,char* argv[])
{
if(argc<3)
{ fprintf(stderr,"Usage: round {server_addr|--} port\n");
return -1;
}
bool amServer = (strcmp("--",argv[1])==0);
int sock;
if(amServer) sock=tcpAccept(argv[2]);
else sock=tcpConnectTo(argv[1],argv[2]);
if(sock<0) { fprintf(stderr,"Connection failed\n"); return -1; }
int i;
char buf[100000] = { 0 };
for(i=0;i<4000;++i)
{
if(amServer)
{ writeLoop(sock,buf,10);
readLoop(sock,buf,20);
//readLoop(sock,buf,10);
//readLoop(sock,buf,10);
}else
{ readLoop(sock,buf,10);
writeLoop(sock,buf,20);
//writeLoop(sock,buf,10);
//writeLoop(sock,buf,10);
}
}
close(sock);
return 0;
}
EDIT: This version is slightly different from the other snippet in that it reads/writes in a loop. So in this version, two separate writes automatically causes two separate read() calls, even if readLoop is called only once. But otherwise the problem still remains.
Interesting. You are being a victim of the Nagle's algorithm together with TCP delayed acknowledgements.
The Nagle's algorithm is a mechanism used in TCP to defer transmission of small segments until enough data has been accumulated that makes it worth building and sending a segment over the network. From the wikipedia article:
Nagle's algorithm works by combining a number of small outgoing
messages, and sending them all at once. Specifically, as long as there
is a sent packet for which the sender has received no acknowledgment,
the sender should keep buffering its output until it has a full
packet's worth of output, so that output can be sent all at once.
However, TCP typically employs something known as TCP delayed acknowledgements, which is a technique that consists of accumulating together a batch of ACK replies (because TCP uses cumulative ACKS), to reduce network traffic.
That wikipedia article further mentions this:
With both algorithms enabled, applications that do two successive
writes to a TCP connection, followed by a read that will not be
fulfilled until after the data from the second write has reached the
destination, experience a constant delay of up to 500 milliseconds,
the "ACK delay".
(Emphasis mine)
In your specific case, since the server doesn't send more data before reading the reply, the client is causing the delay: if the client writes twice, the second write will be delayed.
If Nagle's algorithm is being used by the sending party, data will be
queued by the sender until an ACK is received. If the sender does not
send enough data to fill the maximum segment size (for example, if it
performs two small writes followed by a blocking read) then the
transfer will pause up to the ACK delay timeout.
So, when the client makes 2 write calls, this is what happens:
Client issues the first write.
The server receives some data. It doesn't acknowledge it in the hope that more data will arrive (so it can batch up a bunch of ACKs in one single ACK).
Client issues the second write. The previous write has not been acknowledged, so Nagle's algorithm defers transmission until more data arrives (until enough data has been collected to make a segment) or the previous write is ACKed.
Server is tired of waiting and after 500 ms acknowledges the segment.
Client finally completes the 2nd write.
With 1 write, this is what happens:
Client issues the first write.
The server receives some data. It doesn't acknowledge it in the hope that more data will arrive (so it can batch up a bunch of ACKs in one single ACK).
The server writes to the socket. An ACK is part of the TCP header, so if you're writing, you might as well acknowledge the previous segment at no extra cost. Do it.
Meanwhile, the client wrote once, so it was already waiting on the next read - there was no 2nd write waiting for the server's ACK.
If you want to keep writing twice on the client side, you need to disable the Nagle's algorithm. This is the solution proposed by the algorithm author himself:
The user-level solution is to avoid write-write-read sequences on
sockets. write-read-write-read is fine. write-write-write is fine. But
write-write-read is a killer. So, if you can, buffer up your little
writes to TCP and send them all at once. Using the standard UNIX I/O
package and flushing write before each read usually works.
(See the citation on Wikipedia)
As mentioned by David Schwartz in the comments, this may not be the greatest idea for various reasons, but it illustrates the point and shows that this is indeed causing the delay.
To disable it, you need to set the TCP_NODELAY option on the sockets with setsockopt(2).
This can be done in tcpConnectTo() for the client:
int tcpConnectTo(const char* server, const char* port)
{
struct sockaddr_in sa;
if(getsockaddr(server,port,(struct sockaddr*)&sa)<0) return -1;
int sock=tcpConnect(&sa); if(sock<0) return -1;
int val = 1;
if (setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &val, sizeof(val)) < 0)
perror("setsockopt(2) error");
return sock;
}
And in tcpAccept() for the server:
int tcpAccept(const char* port)
{
int listenSock, sock;
listenSock = tcpListenAny(port);
if((sock=accept(listenSock,0,0))<0) return fprintf(stderr,"Accept failed\n"),-1;
close(listenSock);
int val = 1;
if (setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &val, sizeof(val)) < 0)
perror("setsockopt(2) error");
return sock;
}
It's interesting to see the huge difference this makes.
If you'd rather not mess with the socket options, it's enough to ensure that the client writes once - and only once - before the next read. You can still have the server read twice:
for(i=0;i<4000;++i)
{
if(amServer)
{ writeLoop(sock,buf,10);
//readLoop(sock,buf,20);
readLoop(sock,buf,10);
readLoop(sock,buf,10);
}else
{ readLoop(sock,buf,10);
writeLoop(sock,buf,20);
//writeLoop(sock,buf,10);
//writeLoop(sock,buf,10);
}
}

Fail to allocate semaphore/netconn Tiva C TM4C1294

I am using tm4c1294+lwip1.4.1+FreeRTOS.
As netconn_alloc() is called for socket communication, it allocates an unused semaphore. the number of semaphore is defined as SYS_SEM_MAX, so it can not be over SYS_SEM_MAX.
However, as semaphores are allocated continuously it reaches SYS_SEM_MAX and stop working since I guess sys_sem_free() does not deallocate it properly
Here is function that creates a semaphore implemented in sys_arch.c
err_t
sys_sem_new(sys_sem_t *sem, u8_t count)
{
void *temp;
u32_t i;
/* Find a semaphore that is not in use. */
for(i = 0; i < SYS_SEM_MAX; i++) {
if(sems[i].queue == 0) {
break;
}
}
if(i == SYS_SEM_MAX) {
#if SYS_STATS
STATS_INC(sys.sem.err);
#endif /* SYS_STATS */
return ERR_MEM;
}
/* Create a single-entry queue to act as a semaphore. */
#if RTOS_FREERTOS
sem->queue = xQueueCreate(1, sizeof(void *));
if(sem->queue == NULL) {
#endif /* RTOS_FREERTOS */
#if SYS_STATS
STATS_INC(sys.sem.err);
#endif /* SYS_STATS */
return ERR_MEM;
}
/* Acquired the semaphore if necessary. */
if(count == 0) {
temp = 0;
xQueueSend(sem->queue, &temp, 0);
}
/* Update the semaphore statistics. */
#if SYS_STATS
STATS_INC(sys.sem.used);
#if LWIP_STATS
if(lwip_stats.sys.sem.max < lwip_stats.sys.sem.used) {
lwip_stats.sys.sem.max = lwip_stats.sys.sem.used;
}
#endif
#endif /* SYS_STATS */
/* Save the queue handle. */
sems[i].queue = sem->queue;
/* Return this semaphore. */
return (ERR_OK);
}
Here is another function that frees semaphore implemented in sys_arch.c
void
sys_sem_free(sys_sem_t *sem)
{
/* Delete Sem , By Jin */
vQueueDelete(sem->queue);
/* Clear the queue handle. */
sem->queue = 0;
/* Update the semaphore statistics. */
#if SYS_STATS
STATS_DEC(sys.sem.used);
#endif /* SYS_STATS */
}
Whenever netconn_free() is called sys_sem_free() deallocates the semaphore, but it does not free the semaphore assigned in sem[] array.
I added vQueueDelete(sem->queue); that was suggested by someone, but still all same.
Not only functions creates/frees semaphore but also functions handling mbox are same as functions above, so functions handling mbox could be wrong as well.
Someone already reported this issue to TI, but it seems they have not solved the problems yet.
Therefore, I may need to implement my own functions handling semaphore/mbox in sys_arch.c, but I don't have any clues so far.
Can anyone give me any ideas? or anything?
Thanks,
Jin
The sys_arch.txt file in /doc is somewhat helpful. Apparently, looking at that document, and at what lwip 1.3.2 used to do, it looks like the ports/tiva-tm4c129/sys_arch.c is incorrect and incomplete.
sys_sem_free() should indeed be doing the vQueueDelete() as you discovered. It should not be doing "sem->queue = 0". If you look at netconn_free() over in src/api/api_msg.c, you can see it calls sys_sem_free() and then sys_sem_set_invalid(). The queue handle will be needed in the second function, and should not be clobbered in the first.
sys_sem_set_invalid() should make a sweep over the sems[] array and if it finds a match to the sem->queue, it should zero out that copy in sems[]. Once that is done, it should set sem->queue to 0.
This, I think, best matches what's in Dunkel's sys_arch.txt document, and fixed the resource leak on my system.
I concur that the mailboxes are in the same shape. Fwiw, I went ahead and modified those in similar fashion to that just described for the sems.
btw, the lwip files I was working with were from the TivaWare_C_Series 2.1.0.12573 third_party folder.

Using system calls from C, how do I get the utilization of the CPU(s)?

In C on FreeBSD, how does one access the CPU utilization?
I am writing some code to handle HTTP redirects. If the CPU load goes above a threshold on a FReeBSD system, I want to redirect client requests. Looking over the man pages, kvm_getpcpu() seems to be the right answer, but the man pages (that I read) don't document the usage.
Any tips or pointers would be welcome - thanks!
After reading the answers here, I was able to come up with the below. Due to the poor documentation, I'm not 100% sure it is correct, but top seems to agree. Thanks to everyone who answered.
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/sysctl.h>
#include <unistd.h>
#define CP_USER 0
#define CP_NICE 1
#define CP_SYS 2
#define CP_INTR 3
#define CP_IDLE 4
#define CPUSTATES 5
int main()
{
long cur[CPUSTATES], last[CPUSTATES];
size_t cur_sz = sizeof cur;
int state, i;
long sum;
double util;
memset(last, 0, sizeof last);
for (i=0; i<6; i++)
{
if (sysctlbyname("kern.cp_time", &cur, &cur_sz, NULL, 0) < 0)
{
printf ("Error reading kern.cp_times sysctl\n");
return -1;
}
sum = 0;
for (state = 0; state<CPUSTATES; state++)
{
long tmp = cur[state];
cur[state] -= last[state];
last[state] = tmp;
sum += cur[state];
}
util = 100.0L - (100.0L * cur[CP_IDLE] / (sum ? (double) sum : 1.0L));
printf("cpu utilization: %7.3f\n", util);
sleep(1);
}
return 0;
}
From the MAN pages
NAME
kvm_getmaxcpu, kvm_getpcpu -- access per-CPU data
LIBRARY
Kernel Data Access Library (libkvm, -lkvm)
SYNOPSIS
#include <sys/param.h>
#include <sys/pcpu.h>
#include <sys/sysctl.h>
#include <kvm.h>
int
kvm_getmaxcpu(kvm_t *kd);
void *
kvm_getpcpu(kvm_t *kd, int cpu);
DESCRIPTION
The kvm_getmaxcpu() and kvm_getpcpu() functions are used to access the
per-CPU data of active processors in the kernel indicated by kd. The
kvm_getmaxcpu() function returns the maximum number of CPUs supported by
the kernel. The kvm_getpcpu() function returns a buffer holding the per-
CPU data for a single CPU. This buffer is described by the struct pcpu
type. The caller is responsible for releasing the buffer via a call to
free(3) when it is no longer needed. If cpu is not active, then NULL is
returned instead.
CACHING
These functions cache the nlist values for various kernel variables which
are reused in successive calls. You may call either function with kd set
to NULL to clear this cache.
RETURN VALUES
On success, the kvm_getmaxcpu() function returns the maximum number of
CPUs supported by the kernel. If an error occurs, it returns -1 instead.
On success, the kvm_getpcpu() function returns a pointer to an allocated
buffer or NULL. If an error occurs, it returns -1 instead.
If either function encounters an error, then an error message may be
retrieved via kvm_geterr(3.)
EDIT
Here's the kvm_t struct:
struct __kvm {
/*
* a string to be prepended to error messages
* provided for compatibility with sun's interface
* if this value is null, errors are saved in errbuf[]
*/
const char *program;
char *errp; /* XXX this can probably go away */
char errbuf[_POSIX2_LINE_MAX];
#define ISALIVE(kd) ((kd)->vmfd >= 0)
int pmfd; /* physical memory file (or crashdump) */
int vmfd; /* virtual memory file (-1 if crashdump) */
int unused; /* was: swap file (e.g., /dev/drum) */
int nlfd; /* namelist file (e.g., /kernel) */
struct kinfo_proc *procbase;
char *argspc; /* (dynamic) storage for argv strings */
int arglen; /* length of the above */
char **argv; /* (dynamic) storage for argv pointers */
int argc; /* length of above (not actual # present) */
char *argbuf; /* (dynamic) temporary storage */
/*
* Kernel virtual address translation state. This only gets filled
* in for dead kernels; otherwise, the running kernel (i.e. kmem)
* will do the translations for us. It could be big, so we
* only allocate it if necessary.
*/
struct vmstate *vmst;
};
I believe you want to look into 'man sysctl'.
I don't know the exact library, command, or system call; however, if you really get stuck, download the source code to top. It displays per-cpu stats when you use the "-P" flag, and it has to get that information from somewhere.

Resources