While creating different benchmarks I often aim for the best latency. As you all know, memory in the heap is much more expensive to access than on the stack. We are now in 2021 and when I try ulimit -s I see that I have 8192kbs available. Couldn't it set higher?
I was wondering about how to set it using c code and found setrlimit and getrlimit:
The getrlimit() and setrlimit() system calls get and set resource limits respectively. Each resource has an associated soft and hard limit, as defined by the rlimit structure [...]
I found this answer and this one, and, while they are both very interesting, there are still little things I don't get:
#define _GNU_SOURCE
#define _FILE_OFFSET_BITS 64
#include <stdio.h>
#include <time.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/resource.h>
#include <errno.h>
#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)
int main(int argc, char *argv[])
{
struct rlimit old, new;
struct rlimit *newp;
pid_t pid;
if (!(argc == 2 || argc == 4)) {
fprintf(stderr, "Usage: %s <pid> [<new-soft-limit> "
"<new-hard-limit>]\n", argv[0]);
exit(EXIT_FAILURE);
}
pid = atoi(argv[1]); /* PID of target process */
newp = NULL;
if (argc == 4) {
new.rlim_cur = atoi(argv[2]);
new.rlim_max = atoi(argv[3]);
// newp = &new; // I comment this line to avoid messing up with the processes too much;)
}
/* Set CPU time limit of target process; retrieve and display previous limit */
if (prlimit(pid, RLIMIT_CPU, newp, &old) == -1)
errExit("prlimit-1");
;
printf("Previous limits: soft=%lld; hard=%lld\n", (long long) old.rlim_cur, (long long) old.rlim_max);
/* Retrieve and display new CPU time limit */
if (prlimit(pid, RLIMIT_CPU, NULL, &old) == -1)
errExit("prlimit-2");
printf("New limits: soft=%lld; hard=%lld\n", (long long) old.rlim_cur, (long long) old.rlim_max);
//perror("error:");
exit(EXIT_FAILURE);
}
If I try on any process, I will see, at least on Ubuntu 18.04 and compiling with gcc, that when it tries to retrieve the current and max limits it will return -1. What is this -1 return value exactly? Does it mean "set to default system value" or "failed to retrieve" and if it is "failed to retrieve", what was the reason for that? Why is if (prlimit(pid, RLIMIT_CPU, newp, &old) == -1) not triggered?
Previous limits: soft=-1; hard=-1
However if I uncomment newp = &new; (commented on purpose), then I can set a new value for the process, both current and max, there is no problem with that and it is quite amazing. What would be the max value that I could theoretically set? I tried with sudo gcc raise_stack.c && ./a.out 9194 8192 16384 but I am too afraid to burn my cpu to try it myself ;)
New limits: soft=8192; hard=16384
Many thanks in advance for helping me to better understand getrlimit.
NB: avoid "if you have to raise your stack memory then you are doing something wrong". This is not an opinion question.
edit: cant figure out where to set errno to get the right result
error:: Success
error:: Success
error:: Success
error:: Success
error:: Success
error:: Success
Previous limits: soft=-1; hard=-1
error:: Success
New limits: soft=-1; hard=-1
error:: Success
I set it after
long long a = prlimit(pid, RLIMIT_CPU, newp, &old);
perror("error:");
but still not the ERRNO that I am looking for.
The -1 value is RLIM_INFINITY, meaning that there is no imposed limit.
Related
For a UNIX/C project, I'm supposed to allocate two shared memory segments (which child processes will eventually access with read-only and write permissions, respectively) of two integers each. But any time I try to call shmat(3), it ends up returning -1, setting errno to EACCES, apparently indicating insufficient permissions. I've produced what seems to be the minimum required code fro the error below, with possibly a couple extra includes:
#define _SVID_SOURCE
#include <errno.h>
#include <stdio.h>
#include <sys/shm.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
int main(int argc, char *argv[]){
int i, j, shmid, tshmid;
int * clock;
int * shmMsg;
tshmid = shmget(IPC_PRIVATE,sizeof(int)*2, IPC_CREAT | IPC_EXCL | 0777); //0777 permissions are more liberal than I need, but I've tried other various literals and numbers as well as just CREAT/EXCL.
if (tshmid < 1){
printf("Error: In parent process (%d), shmid came up %d \n",getpid(),shmid);
exit(-1);
}
clock = (int *) shmat(shmid,NULL,0); //I've also tried this with the second argument as (void *) 0, and with the third argument as "(SHM_R | SHM_W)" and "0777"
if (clock == (void *) -1){
printf("Error: First shmat couldn't shmattach to shmid #%d. ERRNO %d\n",shmid,errno);
shmdt(clock);
exit(-1);
} //it never even gets this far
shmdt(clock);
}
Each time, this produces an error message like:
Error: First shmat couldn't shmattach to shmid #1033469981. ERRNO 13
The longer version of my program initially returned an identical error, but errno was set to 43 (EIDRM: Segment identified by shm_id was removed). I've recursively chmodded the whole directory to full access, so that's not the issue, and every time my program crashes I have to manually deallocate the shared memory using ipcrm, so the shmids apply to actual segments. Why won't it attach?
I apologize if my code is extensively long, but I'm attempting to make a local server that handles multiple local clients. I even imported ideas from http://www.binarytides.com/multiple-socket-connections-fdset-select-linux/ to try to get it to work with no success.
I run it using 82 for a parameter, and see as expected:
Socket made and ready
Accepting 10 users
I then use CURL to connect to 127.0.0.1:82 and curl stalls. In my program I see as expected:
CLIENT CONNECTION MADE on socket# 0!
But the problem is data isn't being sent from the server to the client.
I tried forcing a break in CURL (via ctrl+c) to see if anything happened on the server and nothing did. I even tried using a web browser to connect and received similar results (a hang-up).
If I force a break on the server (via ctrl+c), then I got what I expect, a disconnection message (like "Empty Reply from server" from CURL).
What I expected to see in my browser is:
Error
This is a hack-ed-server
What could I be doing wrong here? I'm sort-of new to the select() calls so I'm not sure if I configured them correctly.
Here's the code:
#include <fcntl.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/errno.h>
#include <signal.h>
#include <time.h>
extern errno;
long asock=-1,nsock=-1;
void end_app(int s){
struct sigaction si;
si.sa_handler=SIG_DFL;si.sa_flags=0;
sigaction(SIGCHLD,&si,NULL);
sigaction(SIGTSTP,&si,NULL);
sigaction(SIGTTOU,&si,NULL);
sigaction(SIGTTIN,&si,NULL);
sigaction(SIGSEGV,&si,NULL);
sigaction(SIGTERM,&si,NULL);
sigaction(SIGHUP,&si,NULL);
char v[5000];
sprintf(v,"Abrupt exit detected sig# %d. Closing sockets.\n",s);
write(1,v,strlen(v));
if (asock > -1){close(asock);}
if (nsock > -1){close(nsock);}
}
const long trapsig(){
struct sigaction s,si;
si.sa_handler=SIG_IGN;si.sa_flags=0;
s.sa_handler=end_app;s.sa_flags=0;
sigaction(SIGCHLD,&si,NULL);sigaction(SIGTSTP,&si,NULL);sigaction(SIGTTOU,&si,NULL);sigaction(SIGTTIN,&si,NULL);
if (sigaction(SIGSEGV,&s,NULL)==-1){printf("Cant trap signal!\n");return 1;}
if (sigaction(SIGTERM,&s,NULL)==-1){printf("Cant trap signal!\n");return 1;}
if (sigaction(SIGHUP,&s,NULL)==-1){printf("Cant trap signal!\n");return 1;}
}
//getreq params in: req=external buffer for data
// reqsz=size of external buffer. I set 10000
// nsock=valid socket pointer from accept()
//
//getreq params out: reqsz=actual size of data returned
//
void getreq(char* req,unsigned long *reqsz,long nsock){
//bufsize=how many bytes to read at once. High values like 5000 cause a stall.
//buffer=buffer of data from recv call
const unsigned long ibs=*reqsz,bufsize=5000;
char buffer[ibs],*rp=req;
//spacect=# of spaces in data read
//szct=iterator variable
//mysz=total length of returned data
//bufct=buffer counter to prevent segfault
//recvsz=data size returned from recv or
// forced -2 if buffer hits capacity
// or 2nd space in returned data is found
unsigned long spacect=0,szct=0,mysz=0,bufct=0;
long recvsz=1;char *p=buffer;
//
//Expected data: GET /whatever HTTP/x.x but we
// want /whatever
//
//loop until 2nd space is found or
//ibs bytes of data have been processed
while (recvsz > 0 && bufct < ibs){
recvsz=recv(nsock, p, bufsize, 0);
if (recvsz < 1){break;}
for (szct=1;szct<=recvsz;szct++){
if (*p==' '){spacect++;if (spacect > 2){spacect=2;recvsz=-2;break;}}
if (spacect==1 && *p != ' '){mysz++;if (mysz <= *reqsz){*rp++=*p;}}
p++;bufct++;if (bufct > ibs){recvsz=-2;break;}
}
}
// Process rest of data to try to avoid client errors
while (recvsz == -2){
recvsz=recv(nsock, buffer, bufsize, 0);
}
*reqsz=mysz;
}
int main(int argc,char* argv[]){
if (trapsig() < 0){return 1;}
//set maximum users to 10 and allocate space for each
long maxusers=10;long csock[11];memset(csock,0,11);
//do sanity checks and bind local socket
if (!argv[1]){printf("Port # required\n");return 1;}
if ((asock=socket(AF_INET, SOCK_STREAM, 0)) < 1){printf("Can't make socket! %s\n",strerror(errno));return 1;}
struct sockaddr_in a;
memset(&a,0,sizeof(a));
a.sin_family=AF_INET;
a.sin_addr.s_addr=inet_addr("127.0.0.1");
a.sin_port=htons(strtol(argv[1],NULL,10));
if (bind(asock,(struct sockaddr*)&a, sizeof(a))==-1){printf("Can't bind socket! %s\n",strerror(errno));return 1;}
if (listen(asock,10) < 0){printf("Can't listen! %s\n",strerror(errno));return 1;}
printf("Socket made and ready\nAccepting %d users\n",maxusers);
while(1){
usleep(10); //sleep incase processor is overloaded
fd_set SR;long SMAX=asock,n,canadd=0;
FD_ZERO(&SR);FD_SET(asock,&SR);
for (n=0;n<maxusers;n++){
if (csock[n] > 0){FD_SET(csock[n],&SR);}else{canadd=1;}
if (csock[n] > SMAX){SMAX=csock[n];}
}
long act=select(SMAX+1,&SR,0,0,0);
if (act != EINTR && act < 0){printf("Select error\n");}
if (canadd==1 && FD_ISSET(asock,&SR)){
//incoming connection detected
socklen_t alen=sizeof(a);
if (nsock=accept(asock, (struct sockaddr*)&a, &alen)< 0){printf("Can't accept! %s\n",strerror(errno));close(asock);return -1;}
for (n=0;n<maxusers;n++){if (csock[n]==0){csock[n]=nsock;break;}}
printf("CLIENT CONNECTION MADE on socket# %d!\n",n);
fcntl(nsock, F_SETFD, O_NONBLOCK);
//program reaches here when client first connects
}
for (n=0;n<maxusers;n++){
if (csock[n] > 0 && FD_ISSET(csock[n],&SR)){
//this section never seems to execute
unsigned long reqsz=10000;
char req[reqsz];
printf("Checking incoming data...\n",n);
getreq(req,&reqsz,csock[n]);
if (reqsz > 0){
printf("Received %d bytes\nData: %s\n",reqsz,req);
const char buf[10000]={"HTTP/1.0 200 OK\nConnection: close\nContent-type: text/html\n\n<html><head><title>hacked</title></head><body><H1>Error</h1><p>This is a hack-ed-server</p></body></html>\n\n"};
send(csock[n],buf,strlen(buf),0);
}else{
printf("Received no data\n");
}
printf("Closing.\n");
close(csock[n]);
csock[n]=0;
}
}
}
printf("Closing sockets\n");
close(asock);
return 0;
}
Pay attention to warnings. This
if(nsock=accept(asock, (struct sockaddr*)&a, &alen)< 0)
is not parsed as you seem to expect. In fact it assigns nsock a result of comparison of accept return value vs 0 (which is false aka 0). You want some extra parenthesis:
if ((nsock = accept(asock, (struct sockaddr*)&a, &alen)) < 0)
This
long csock[11];memset(csock,0,11);
does not initialize csock completely -- just the first 11 bytes. You want memset(csock, 0, sizeof(csock));
This code when compiled allows me to connect to a server using a correct IP address as the program parameter. So far, the connection is fine and data is sent to the server, but when I attempt to use the recv() function to try to collect data in non-blocking mode, I receive a segmentation fault. It always appears after the "receiving data..." line.
As you can see, I reserved 5000 bytes of stack memory for the operation and I'm testing with trying to read only one byte without success. Does recv() not work with stack memory?
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
extern errno;
struct sockaddr_in a;
struct timeval tv;
fd_set wready;
long conntoserver(const char* ip,const char* data){
memset((char*)&a,0,sizeof(a));
if (inet_aton(ip,a.sin_addr.s_addr) == 0){printf("Can't set address\n");return -1;}
long s=socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);if (s < 0){printf("Can't make socket\n");return -1;}
tv.tv_sec=1;tv.tv_usec=0;FD_ZERO(&wready);FD_SET(s,&wready);
//start non-blocking mode
if (fcntl(s,F_SETFL,O_NONBLOCK) < 0){printf("Can't make socket non-blocking\n");close(s);return -1;}
a.sin_family=AF_INET;a.sin_port=htons(80);long ret=connect(s,(struct sockaddr*)&a,sizeof(a));
if (ret < 0 && errno != EINPROGRESS){printf("Can't connect to IP. %s\n",strerror(errno));close(s);return -1;}
if (errno == EINPROGRESS){
printf("connection in progress...\n");
ret=select(s+1,NULL,&wready,NULL,&tv);
if (ret < 0){printf("select() error. %s\n",strerror(errno));close(s);return -1;}
if (ret==0){printf("select() timeout. %s\n",strerror(errno));close(s);return -1;}
}
unsigned long sz=strlen(data);
ssize_t ns=send(s,data,sz,0);
if (ns==sz){return s;}else{return -1;}
}
int main(int argc,char* argv[]){
if (argc < 2){printf("Format: %s < IP to connect to >\n",argv[0]);return -1;}
long s=conntoserver(argv[1],"GET / HTTP/1.1\nHost: example.com\n\n");
if (s < 0){printf("Error making server request. %s\n",strerror(errno));close(s);return -1;}
printf("receiving data...\n");
char b[5001];
ssize_t br=recv(s,b,1,0);
printf("br=%s\n",br);
close(s);
return 0;
}
printf("br=%s\n",br);
Should be:
printf("br=%ld\n",br);
BTW -- I had to fix a number of problems to make this code compile. You should always pay attention to compiler warnings.
In addition to the error pointed out by #keithmo, you have an error here:
if (inet_aton(ip,a.sin_addr.s_addr) == 0){printf("Can't set address\n");return -1;}
The second argument to inet_aton() is expected to be a pointer to a struct in_addr but you are passing a value of type uint32_t. In fact, that value is initialized to zero, so the result is likely (but not certain) to be equivalent to passing a NULL pointer. Either way, this produces undefined behavior, and I'd surprised if that didn't manifest as a segfault. Even if the program runs past that, the call as you wrote it cannot produce the (apparently) desired effect of initializing the address in variable a.
What you want instead is
if (inet_aton(ip, &a.sin_addr) == 0) {
printf("Can't set address\n");
return -1;
}
And for goodness's sake, do make free with the spaces and newlines. They greatly improve code legibility. If you're short, I can lend you some.
Need to call a function every X (let's say 5) seconds and the below code does it.
But it is blocking the execution of code. As I want it to work like setitimer(), where I can (for example) call a function every 5 sec and do something else.
#include <sys/timerfd.h>
#include <time.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h> /* Definition of uint64_t */
#define handle_error(msg) \
do { perror(msg); exit(EXIT_FAILURE); } while (0)
int
main(int argc, char *argv[])
{
struct itimerspec new_value;
int max_exp, fd;
struct timespec now;
uint64_t exp, tot_exp;
ssize_t s;
if (clock_gettime(CLOCK_REALTIME, &now) == -1)
handle_error("clock_gettime");
/* Create a CLOCK_REALTIME absolute timer with initial
expiration and interval as specified in command line */
new_value.it_value.tv_sec = now.tv_sec + 1;
new_value.it_value.tv_nsec = now.tv_nsec;
new_value.it_interval.tv_sec = 5;
new_value.it_interval.tv_nsec = 0;
max_exp = 5; //say 5 times
fd = timerfd_create(CLOCK_REALTIME, 0);
if (fd == -1)
handle_error("timerfd_create");
if (timerfd_settime(fd, TFD_TIMER_ABSTIME, &new_value, NULL) == -1)
handle_error("timerfd_settime");
printf("timer started\n");
for (tot_exp = 0; tot_exp < max_exp;) {
s = read(fd, &exp, sizeof(uint64_t));
if (s != sizeof(uint64_t))
handle_error("read");
tot_exp += exp;
printf("read: %llu; total=%llu\n",
(unsigned long long) exp,
(unsigned long long) tot_exp);
}
//Do something else ?
//while(1);
exit(EXIT_SUCCESS);
}
EDIT
I have one more question.
On changing these lines in above code from
new_value.it_interval.tv_sec = 5;
new_value.it_interval.tv_nsec = 0;
to
new_value.it_interval.tv_sec = 0;
new_value.it_interval.tv_nsec = 5000000000;
I see that there is no 5 seconds delay. Whats happening here?
You need to understand how to use multiplexing syscalls like poll(2) (or the older select(2) which tends to become obsolete) and use them to test the readability of the file descriptor obtained by timerfd_create(2) before read(2)-ing it.
However, be aware that timerfd_create works only when that read call succeeded. So only when the poll says you that the fd is not readable can you do something else. That something else should be quick (last less than 5 seconds).
You might want to investigate event loop libraries, like e.g. libevent (wrapping poll). If you are coding a graphical application (using Qt or Gtk) it does already have its own event loop. If clever enough, you could do your 5-second period without any timerfd_create, just thru your event loop (by carefully setting the timeout given to poll, etc.).
Addenda:
the tv_nsec field should always be non-negative and less than 1000000000 (the number of nanoseconds in a second).
Any reason you have to use timerfd? Just schedule an alarm and make a handler for SIGALRM to call your function.
If you don't want to use signals, just create an extra thread to block on your timer fd and proceed as normal in the main thread.
If you don't like either of those and you want to do work while you're waiting, you have to poll. You can do it as basile suggests, or you could just store the current time and check whenever you would poll to see if the desired period has elapsed.
The following code is supposed to make 100,000 threads:
/* compile with: gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>
#define MAX_THREADS 100000
int i;
void run(void) {
sleep(60 * 60);
}
int main(int argc, char *argv[]) {
int rc = 0;
pthread_t thread[MAX_THREADS];
printf("Creating threads ...\n");
for (i = 0; i < MAX_THREADS && rc == 0; i++) {
rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
if (rc == 0) {
pthread_detach(thread[i]);
if ((i + 1) % 100 == 0)
printf("%i threads so far ...\n", i + 1);
}
else
{
printf("Failed with return code %i creating thread %i (%s).\n",
rc, i + 1, strerror(rc));
// can we allocate memory?
char *block = NULL;
block = malloc(65545);
if(block == NULL)
printf("Malloc failed too :( \n");
else
printf("Malloc worked, hmmm\n");
}
}
sleep(60*60); // ctrl+c to exit; makes it easier to see mem use
exit(0);
}
This is running on a 64bit machine with 32GB of RAM; Debian 5.0 installed, all stock.
ulimit -s 512 to keep the stack size down
/proc/sys/kernel/pid_max set to 1,000,000 (by default, it caps out at 32k pids).
ulimit -u 1000000 to increase max processes (don't think this matters at all)
/proc/sys/kernel/threads-max set to 1,000,000 (by default, it wasn't set at all)
Running this spits out the following:
65500 threads so far ...
Failed with return code 12 creating thread 65529 (Cannot allocate memory).
Malloc worked, hmmm
I'm certainly not running out of ram; I can even launch several more of these programs all running at the same time and they all start their 65k threads.
(Please refrain from suggesting I not try to launch 100,000+ threads. This is simple testing of something which should work. My current epoll-based server has roughly 200k+ connections at all times and various papers would suggest that threads just might be a better option. - Thanks :) )
pilcrow's mention of /proc/sys/vm/max_map_count is right on track; raising this value allows more threads to be opened; not sure of the exact formula involved, but a 1mil+ value allows for some 300k+ threads.
(For anyone else experimenting with 100k+ threads, do look at pthread_create's mmap issues... making new threads gets really slow really fast when lower memory is used up.)
One possible issue is the local variable thread in the main program. I think that pthread_t would be 8 bytes on your 64-bit machine (assuming 64-bit build). That would be 800,000 bytes on the stack. Your stack limit of 512K would be a problem I think. 512K / 8 = 65536, which is suspiciously near the number of threads you are creating. You might try dynamically allocating that array instead of putting it on the stack.
This might help set the stack size in the program to the smallest it can go (if that's not enough you pick):
/* compile with: gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>
#define MAX_THREADS 100000
int i;
void run(void) {
sleep(60 * 60);
}
int main(int argc, char *argv[]) {
int rc = 0;
pthread_t thread[MAX_THREADS];
pthread_attr_t thread_attr;
pthread_attr_init(&thread_attr);
pthread_attr_setstacksize(&thread_attr, PTHREAD_STACK_MIN);
printf("Creating threads ...\n");
for (i = 0; i < MAX_THREADS && rc == 0; i++) {
rc = pthread_create(&(thread[i]), &thread_attr, (void *) &run, NULL);
if (rc == 0) {
pthread_detach(thread[i]);
if ((i + 1) % 100 == 0)
printf("%i threads so far ...\n", i + 1);
}
else
{
printf("Failed with return code %i creating thread %i (%s).\n",
rc, i + 1, strerror(rc));
// can we allocate memory?
char *block = NULL;
block = malloc(65545);
if(block == NULL)
printf("Malloc failed too :( \n");
else
printf("Malloc worked, hmmm\n");
}
}
sleep(60*60); // ctrl+c to exit; makes it easier to see mem use
exit(0);
}
additionally you could add a call like this: pthread_attr_setguardsize(&thread_attr, 0); just after the call to pthread_attr_setstacksize() but then you'd loose stack overrun detection entirely, and it'd only save you 4k of address space and zero actual memory.
Are you trying to search for a formula to calculate max threads possible per process?
Linux implements max number of threads per process indirectly!!
number of threads = total virtual memory / (stack size*1024*1024)
Thus, the number of threads per process can be increased by increasing total virtual memory or by decreasing stack size. But, decreasing stack size too much can lead to code failure due to stack overflow while max virtual memory is equals to the swap memory.
Check you machine:
Total Virtual Memory: ulimit -v (default is unlimited, thus you need to increase swap memory to increase this)
Total Stack Size: ulimit -s (default is 8Mb)
Command to increase these values:
ulimit -s newvalue
ulimit -v newvalue
*Replace new value with the value you want to put as limit.
References:
http://dustycodes.wordpress.com/2012/02/09/increasing-number-of-threads-per-process/