Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have this single monolithic daemon which does multiple operations like interfacing with north bound APIs, interfacing with south bound APIs, executing state machines, building internal databases.
Currently I ended up with scalability issues and I want to redesign the daemon so that all the multiple actions inside the daemon are made concurrent. But using threads will complicate the logic since I end up having to:
Add locks for synchronization.
Take proper care for future extensions.
Debug the timing issues.
So my question is please suggest a design approach where I can still make the actions concurrent and remove the complexity of threads.
My application is currently in C. Any sample open source (FOSS) project as example would also help me to understand the design approach.
Your only remaining options are:
Multi-process approach (ie: spawn(), fork(), exec(), etc. You still need to synchronize data, setup shared memory, etc. Threads would likely be easier).
Bite the bullet and live with no concurrency.
Become proficient in "lock free / lockless programming" approaches, which will still likely require atomic operations at the least.
Software sync, protection, and future-proofing/scalability are common problems in any production code that does non-trivial operations. Trying to avoid them outright usually indicates you have bigger concerns than avoiding threaded models.
This sounds like a perfect case for go which provides a concurrency model based on Hoare's Communicating Sequential Processes (CSP)*. Fortunately you don't have to use go to get CSP. Martin Sustrik of ZeroMQ fame has given us libmill, which provides the go concurrency primitives in C. Still, you might consider go for its other features.
* Rather than try to describe CSP directly, I'd suggest you watch some of Rob Pike's excellent videos, like this one: Go Concurrency Patterns.
One way you can achieve asynchronous execution without running multiple threads is using command pattern and command queue. You can implement it in any programming language. Of course things will not be really executing in parallel but this is the way to do asynchronous programming in environments where resources are very limited. Robect C Martin describles this really well in his video.
Example scenario:
You add a initial command to the queue (for the sake of example it's just single simple command).
You start infinite loop which does only one thing:
Take next command from the queue
Execute taken command on the current thread
Our command (lets call it CheckButtonPressed) can do some simple check (for example if button was clicked or some web service responded with some value)
if condition check is negative command will add itself back to the queue (queue is never empty and we are checking all the time if button was pressed)
if condition check is positive we add to the queue the HandleButtonClick command that contains whatever code we want to run in respond to this event.
When HandleButtonClick command will be processed it will execute whatever code is required and at the end it will add CheckButtonPressed again to the queue so the button can be pressed again and queue is never empty.
As you see except the initial commands (the ones that are added to the queue before starting queue processing loop) all other commands are added to the queue by other commands. Commands can be statefull but there is no need for threads synchronization because there is only one thread.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I understand that implementing a state machine is the perfect way to program the computer. Since state machines are typically programmed using non-blocking calls, I wonder why blocking calls similar to the Berkeley sockets APIs were invented? Don't they encourage bad programming practice?
Thanks in advance.
Edit: The idea behind this question is to establish the fact that a multi-context event driven state machine based on non-blocking IO is indeed the perfect way to program the computer. Everything else is amateur. People who think otherwise should allow for a debate.
Your question makes some pretty substantial assertions / assumptions:
the underlying nature of computers is a state machine?
Well, surely you can model computers as state machines, but that does not in itself mean that such a model represents some fundamental "underlying nature".
I understand that implementing a state machine is the perfect way to program the computer.
Then by all means, write all your programs as state machines. Good luck.
In real life, some tasks can be conveniently and effectively written as state machines, but there are many for which a state-machine approach would be cumbersome to write and difficult to understand or maintain.
There is no "perfect" way to program a computer. Indeed, it would be pretty pretentious to claim perfection even for a single program.
Since state machines are typically programmed using non-blocking calls,
You don't say? I think you would need to be a lot more specific about what you mean by this. I have written state-machine based software at times in the past, and I would not characterize any of it as having been implemented using non-blocking calls, nor as exposing a non-blocking external API.
I wonder why blocking calls similar to the Berkeley sockets APIs were invented? Don't they encourage bad programming practice?
Before we could even consider this question, you would have to define what you mean by "bad programming practice". From what I can see, however, you are assuming the conclusion:
you assert that a state-machine approach to programming is ideal, with the implication that anything else is sub-par.
you claim, without support, that only non-blocking calls have state-machine nature
you conclude that anything that uses blocking calls must exhibit bad programming practice.
Your conclusion is not consistent with the prevailing opinion and practice of the programming community, to the extent that I can gauge it. Your argument is hollow and unconvincing.
Multiple processes (or later, threads) with synchronous (blocking) calls are easy to understand and program and easily composable - that is, you can take two tasks that are made up of synchronous calls and run them at the same time (via a scheduler) without having to modify either one in any way.
Programming as a state machine, on the other hand, requires either manually adding states (possibly in combinatorically growing numbers) when you add new code, or some kind of tightly-coupled framework of registering handlers and explicitly storing state for the next handler to use.
What? 'blocking call' implies preemptive multitasking. The kernel of such an OS is a state-machine with interrupts as input events and a set of running threads as output actions.
The OS kernel is a state machine, and blocking calls conveniently move the FSM functionality into the kernel so that you don't have to write the miserable state-machines in user apps.
I understand that implementing a state machine is the perfect way to program the computer
What? 'perfect'? What? Have you ever developed, debugged and delivered any non-trivial multithreaded app?
I came across this question and was very impressed by this answer.
I would really like to follow the advices from that answer, but I cannot imagine how to do that. How can I avoid multi threading?
There are often situations, that need to deal with different things concurrently (different hardware resources or networking for example) but at the same time they need to access shared data (like configurations, data to work on, and so on).
How could this be solved single-threaded without using any kinds of huge state-machines or event loops?
I know, that this is a huge topic, which cannot be answered as a whole on a platform like Stackoverflow. I think I really should go reading the advised book from the mentioned answer, but for now I would love to read some input here.
Maybe it is worth noting, that I am interested in solutions in C. Higher languages like Java, C++ and especially frameworks like Qt or similar ones simplify this a lot, but what about pure C?
Any input is very appreciated. Thank you all in advance
You already mentioned event loops, but I still think those provide an excellent alternative to multi-threading for many applications, and also serve as a good base when adding multi-threading later, if warranted.
Say you have an application that needs to handle user input, data received on a socket, timer events, and signals for example:
One multi-threaded design would be to spawn different threads to wait on the different event sources and have them synchronize their actions on some global state as events arrive. This often leads to messy synchronization and termination logic.
A single-threaded design would be to have a unified event loop that receives all types of events and handles them in the same thread as they arrive. On *nix systems, this can be accomplished using e.g. select(2), poll(2), or epoll(7) (the latter Linux-specific). Recent Linux versions also provide signalfd(2), timerfd (timerfd_create(2)), and eventfd(2) for cleanly fitting additional event types into this model, and on other unices you can use various tricks involving e.g. pipe(2)s to signal events. A nice library that abstracts much of this away is libevent, which also works on other platforms.
Besides not having to deal with multi-threading right away, the event loop approach also cleanly lends itself to adding multi-threading later if needed for performance or other reason: You simply let the event handler spawn threads for certain events. Having all event handling in a single location often greatly simplifies application design.
When you do need multiple threads (or processes), it helps to have narrow and well-tested interfaces between them, using e.g. synchronized queues. An alternative design for the event handler would be to have event-generating threads push events to an event queue from which the event handler then reads and dispatches them. This cleanly separates various parts of the program.
Read more about continuations and contination-passing style (and CPS transform).
CPS-transform could be a systematic way to "mimic" multi-threading.
You could have a look into CPC (Continuation Passing C, by Juliusz Chroboczek and Gabriel Kerneis), which is also a source to source C transformer. You could also read old Appel's book: Compiling with Continuations and Queinnec's book Lisp In Small Pieces
Read also more about event loops, callbacks, closures, call stacks, tail calls. These notions are related to your concerns.
See also the (nearly obsolete) setcontext(3) Linux function, and idle functions in event loops, see this.
You can implement concurrent tasks by using coroutines. You then have to explicitly pass control (the cpu) to another coroutine. It won't be done automatically by an interrupt after a small delay.
http://en.wikipedia.org/wiki/Coroutine
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I am trying to create a server which I expect to have high performance demands. This question deals with the server core. What programming ideas best support fast performance?
Do you split sockets into different threads and call blocking recv() on each?
Do you have one thread which sits in a select() loop and then notifies another thread to process the individual ports?
Do you have one thread which processes the select() and the response?
Do you do 2 or 3 but with clusters of ports instead of all of them?
Does using blocking vs nonblocking ports matter if you use select as specified above?
What setsockopt's improve performance: TCP_NODELAY, others?
I realize that some of these depend on the use case. For example, 6 with TCP_NODELAY off would have a negative impact if there are a lot of small packets. 3 sounds like it might be faster if the response is trivial. Any other questions that I havent thought of that affect performance would be appreciated as well.
I would start with a single-threaded approach: Use non-blocking I/O, and a fast polling mechanism like edge-triggered epoll on Linux. (Other platforms have similar technologies.) Centering everything around your polling loop simplifies the program design massively, so I would definitely throw signalfds, timerfds and eventfds in there, too. Then everything is handled by one central loop.
If and when you need to go multi-threaded, this may be as simple as running the main loop several times concurrently. If you set events to "one-shot", they'll be disabled from the poll until rearmed, and so the thread that processes the event can safely assume to be the only thread doing so (and re-arm the event at the end). You only need to synchronise the communication between different parts of your program, or shared data access, but a lot of synchronisation is already taken care by the poller.
The easiest thing to code, in my opinion, is one thread per connection using blocking I/O. It is also easy to write portably using your favorite threading model.
The problem with multiplexing non-blocking I/O is maintaining state for each connection. For example, I want to write 1024 bytes but write only consumed 900... So now have to remember the 124 bytes to write them some later time. And that is just state at the raw "send a buffer" level; consider the state of your entire protocol and it can become complex quickly. Nothing impossible, of course, but it is far simpler to just use blocking calls, assuming the connections do not need to interact with each other (much).
I have used this approach for a modest number (~dozens) of connections and moved data at over a gigabyte per second sustained on a pair of 10GbE links. The Linux kernel's scheduler is pretty good at handling thread counts in this range.
For a Web server type thing serving thousands or tens of thousands of clients... Well, I have not tried personally. I have read that multiplexing techniques (epoll etc.) are faster in that scenario. So as others have said, it depends on your application.
But if your application is like mine (modest number of connections, limited interaction among them), the "one thread per connection" approach wins hands down, IMO.
It depends.
This type of question is very hard to answer; that will be one of the roles of the project itself. You will need to measure the performance of your server under the work load that it's going to face and then see what options work best for your use case.
For example, set TCP_NODELAY will reduce the latency of requests, but that option is there for a reason; you will decrease throughput by setting TCP_NODELAY.
The following website has some information that you should look through: http://www.kegel.com/c10k.html. Some of it is a bit old now (by a few years), but it contains a list of the technologies that you should consider using: epoll, asynchronous I/O.
You should set about designing your system in a modular fashion so that your workers aren't tied to a specific implementation (select/poll/epoll). Things like setsockopt can be changed easily later and you shouldn't worry about them at all.
Make it work first - then make it "fast"; whatever you mean by "fast". If you want something that scales then be aware of big O of your algorithm (O(n), O(n^2) ... etc).
I have a process wherein a program running in an application server must access a table in an Oracle database server whenever at least one row exists in this table. Each row of data relates to a client requesting some number crunching performed by the program. The program can only perform this number crunching serially (that is, for one client at a time rather than multiple clients in parallel).
Thus, the program needs to be informed of when data is available in the database for it to process. I could either
have the program poll the database, or
have the database trigger the program.
QUESTION 1: Is there any conventional wisdom why one approach might be better than the other?
QUESTION 2: I wonder if programs have any issues "running" for months at a time (would any processes in the server stop or disrupt the program from running? -- if so I don't know how I'd learn there was a problem unless from angry customers). Anyone have experience running programs on a server for a long time without issues? Or, if the server does crash, is there a way to auto-start a (i.e. C language executable) program on it after the server re-boots, thus not requiring a human to start it specifically?
Any advice appreciated.
UPDATE 1: Client is waiting for results, but a couple seconds additional delay (from polling) isn't a deal breaker.
I would like to give a more generic answer...
There is no right answer that applies every time. Some times you need a trigger, and some times is better to poll.
But… 9 out of 10 times, polling is much more efficient, safe and fast than triggering.
It's really simple. A trigger needs to instantiate a single program, of whatever nature, for every shot. That is just not efficient most of the time. Some people will argue that that is required when response time is a factor, but even then, half of the times polling is better because:
1) Resources: With triggers, and say 100 messages, you will need resources for 100 threads, with 1 thread processing a packet of 100 messages you need resources for 1 program.
2) Monitoring: A thread processing packets can report time consumed constantly on a defined packet size, clearly indicating how it is performing and when and how is performance being affected. Try that with a billion triggers jumping around…
3) Speed: Instantiating threads and allocating their resources is very expensive. And don’t get me started if you are opening a transaction for each trigger. A simple program processing a say 100 meessage packet will always be much faster that initiating 100 triggers…
3) Reaction time: With polling you can not react to things on line. So, the only exception allowed to use polling is when a user is waiting for the message to be processed. But then you need to be very careful, because if you have lots of clients doing the same thing at the same time, triggering might respond LATER, than if you where doing fast polling.
My 2cts. This has been learned the hard way ..
1) have the program poll the database, since you don't want your database to be able to start host programs (because you'd have to make sure that only "your" program can be started this way).
The classic (and most convenient IMO) way for doing this in Oracle would be through the DBMS_ALERT package.
The first program would signal an alert with a certain name, passing an optional message. A second program which registered for the alert would wait and receive it immediatly after the first program commits. A rollback of the first program would cancel the alert.
Of cause you can have many sessions signaling and waiting for alerts. However, an alert is a serialization device, so if one program signaled an alert, other programs signaling the same alert name will be blocked until the first one commits or rolls back.
Table DBMS_ALERT_INFO contains all the sessions which have registered for an alert. You can use this to check if the alert-processing is alive.
2) autostarting or background execution depends on your host platform and OS. In Windows you can use SRVANY.EXE to run any executable as a service.
I recommend using a C program to poll the database and a utility such as monit to restart the C program if there are any problems. Your C program can touch a file once in a while to indicate that it is still functioning properly, and monit can monitor the file. Monit can also check the process directly and make sure it isn't using too much memory.
For more information you could see my answer of this other question:
When a new row in database is added, an external command line program must be invoked
Alternatively, if people aren't sitting around waiting for the computation to finish, you could use a cron job to run the C program on a regular basis (e.g. every minute). Then monit would be less needed because your C program will start and stop all the time.
You might want to look into Oracle's "Change Notification":
http://docs.oracle.com/cd/E11882_01/appdev.112/e25518/adfns_cqn.htm
I don't know how well this integrates with a "regular" C program though.
It's also available through .Net and Java/JDBC
http://docs.oracle.com/cd/E11882_01/win.112/e23174/featChange.htm
http://docs.oracle.com/cd/E11882_01/java.112/e16548/dbchgnf.htm
There are simple job managers like gearman that you can use to send a job message from the database to a worker. Gearman has among others a MySQL user defined function interface, so it is probably easy to build one for oracle as well.
Where can I find benchmarks on different networking architectures?
I am playing with sockets / threads / forks and I'd like to know what the best is. I was thinking there has got to be a place where someone has already spelled out all the pros and cons of different architectures for a socket service, listed benchmarks with code that runs.
Ultimately I'd like to run these various configurations with my own code and see which runs best in different circumstances.
Many people I talk to say that I should just use single threaded select. But I see an argument for threads when you're storing state information inside the thread to keep code simple. What is the trade off mark for writing my own state structure vs using a proven thread architecture.
I've also been told forking is bad... but when you need 12000 connections on a machine that cannot raise the open file per process limit, forking is an option! Forking is also a nice option for stability when you've got one process that needs restarting, it doesn't disturb the others.
Sorry, this is one of my longer questions... so many variables are left empty.
Thanks,
Chenz
edit: here's the link I was looking for, which is a whole paper answering your question. http://www.kegel.com/c10k.html
There are web servers designed along all three models (fork, thread, select). People like to benchmark web servers.
http://www.lighttpd.net/benchmark
Libevent has some benchmarks and links to stuff about how to choose a select() vs. threaded model, generally in favour of using the libevent model.
http://monkey.org/~provos/libevent/
It's very difficult to answer this question as so much depends on what your service is actually doing. Does it have to query a database? read files from the filesystem? perform complicated calculations? go off and talk to some other service? Also, how long-lived are client connections? Might connections have some semantic interaction with other connections, or are they all treated as independent of each other? Might you want to think about load-balancing your service across multiple servers later? (If so, you might usefully think about that now so that any necessary help can be designed in from the start.)
As you hint, the serving machine might have limits which interact with the various techniques, steering you towards one answer or another. You have a per-process file descriptor limit, but remember that you may also have a fixed size process table! How many concurrent clients are you expecting, anyway?
If your service keeps crashing and you need to keep restarting it or you think you want a multi-process model so that connections are isolated from each other, you're probably doing it wrong. Stability is extremely important in this sort of context, and that means good practice and memory hygiene, both in general and in the face of network-based attacks.
Remember the history... fork() is cheap in the Unix world, but spawning new processes relatively expensive on Windows. OTOH, Windows threads are lightweight, whereas threading has always been a bit alien to Unix and only relatively recently become widespread.