When trying to use a callback function for a DBus reply I get stuck on creating a good/well working main loop.
What I want to do is simple, make a DBus call and specify a function which should be called when the reply comes. This is because I do not want to block my thread for the whole time until a reply has been calculated and arrived.
I first use dbus_connection_send_with_reply(..) to get a DBusPendingCall, then i specify a callback function using dbus_pending_call_set_notify(..). Before doing this, when connecting to the bus, I have started another thread which should wait for a response and call the callback functions. I have found no examples at all and no very good documentation of how I should design a dbus main-loop like this. I have tried:
// Main dbus loop handling data transfer and callbacks..
void *dbus_main(void *args)
{
my_dbus dbus = (my_dbus)args;
while (MY_DBUS_STATUS_STOPPING != dbus->status
&& dbus_connection_read_write_dispatch(dbus->conn, -1))
;
return 0;
}
// Start the dbus main loop in a new thread
void dbus_main_start(my_dbus dbus) {
if (!pthread_create(&dbus->th, NULL, dbus_main, dbus)) {
// PRINT ERROR
}
}
My problem is two things:
I try to stop the app by setting the dbus->status flag to MY_DBUS_STATUS_STOPPING and waiting for the threads to join. This does not work if the thread is blocked in the dbus_connection_read_write_dispatch(..) function. If i want the app to stop fast then I need to specify a very short timeout. Can't I wake the blocked thread in some other way?
More seriously, with this code i don't get any callback from the method I call. If I add some fprintf(..) to write to stdout I might suddenly get my callback. It seems quite random, so maybe some kind of deadlock? I have tried having a dbus_connection_flush(..) between sending the message and adding the callback with _set_notify(..) function. Doesn't do any difference... But printing some letters to stdout in the same place fixes the problem. Printing to stdout in the dbus-main-loop insted of an empty ";" seems to do the trick sometimes...
So anyone who has an example of using the low-level dbus api together with async methods, ie not using _block(..)??
You can create a simple DBus application as follows...
To setup a server to handle incoming messages, call dbus_connection_register_object_path passing in a VTable containing function pointers to handle the messages. Such as:
{ .unregister_function = UnregisteredMessage, .message_function = ServiceMessage }
To send a new message, call dbus_connection_send_with_reply and then dbus_pending_call_set_notify to associate a callback function to handle the reply.
Next you will need to service DBus. This can be done in a separate thread or by calling periodically with non-blocking calls in the same thread, as shown below:
/* Non-blocking read of the next available message */
dbus_connection_read_write ( MyDBusConnection, 0 ) ;
while ( dbus_connection_get_dispatch_status ( MyDBusConnection ) == DBUS_DISPATCH_DATA_REMAINS )
{
dbus_connection_dispatch ( MyDBusConnection ) ;
}
There are some good example of using the DBUS C API here: http://www.matthew.ath.cx/misc/dbus
It is highly recommended that you use a D-Bus library other than libdbus, as libdbus is fiddly to use correctly, as you are finding. If possible, use GDBus or QtDBus instead, as they are much higher-level bindings which are easier to use. If you need a lower-level binding, sd-bus is more modern than libdbus.
If you use GDBus, you can use GMainLoop to implement a main loop. If you use sd-bus, you can use sd-event.
Related
I checked the docs and stackoverflow but didn't find exactly a suiting approach.
E.g. this post seems very close: Dispatch a blocking service in a Reactive REST GET endpoint with Quarkus/Mutiny
However, I don't want so much unneccessary boilerplate code in my service, at best, no service code change at all.
I generally just want to call a service method which uses entity manager and thus is a blocking action, however, want to return a string to the caller immidiately like "query started" or something. I don't need a callback object, it's just a fire and forget approach.
I tried something like this
#NonBlocking
#POST
#Produces(MediaType.TEXT_PLAIN)
#Path("/query")
public Uni<String> triggerQuery() {
return Uni.createFrom()
.item("query started")
.call(() -> service.startLongRunningQuery());
}
But it's not working -> Error message returned to the caller:
You have attempted to perform a blocking operation on a IO thread. This is not allowed, as blocking the IO thread will cause major performance issues with your application. If you want to perform blocking EntityManager operations make sure you are doing it from a worker thread.",
I actually expected quarkus takes care to distribute the tasks accordingly, that is, rest call to io thread and blocking entity manager operations to worker thread.
So I must using it wrong.
UPDATE:
Also tried an proposed workaround that I found in https://github.com/quarkusio/quarkus/issues/11535 changing the method body to
return Uni.createFrom()
.item("query started")
.emitOn(Infrastructure.getDefaultWorkerPool())
.invoke(()-> service.startLongRunningQuery());
Now I don't get an error, but service.startLongRunningQuery() is not invoked, thus no logs and no query is actually sent to db.
Same with (How to call long running blocking void returning method with Mutiny reactive programming?):
return Uni.createFrom()
.item(() ->service.startLongRunningQuery())
.runSubscriptionOn(Infrastructure.getDefaultWorkerPool())
Same with (How to run blocking codes on another thread and make http request return immediately):
ExecutorService executor = Executors.newFixedThreadPool(10, r -> new Thread(r, "CUSTOM_THREAD"));
return Uni.createFrom()
.item(() -> service.startLongRunningQuery())
.runSubscriptionOn(executor);
Any idea why service.startLongRunningQuery() is not called at all and how to achieve fire and forget behaviour, assuming rest call handled via IO thread and service call handled by worker thread?
It depends if you want to return immediately (before your startLongRunningQuery operation is effectively executed), or if you want to wait until the operation completes.
If the first case, use something like:
#Inject EventBus bus;
#NonBlocking
#POST
#Produces(MediaType.TEXT_PLAIN)
#Path("/query")
public void triggerQuery() {
bus.send("some-address", "my payload");
}
#Blocking // Will be called on a worker thread
#ConsumeEvent("some-address")
public void executeQuery(String payload) {
service.startLongRunningQuery();
}
In the second case, you need to execute the query on a worker thread.
#POST
#Produces(MediaType.TEXT_PLAIN)
#Path("/query")
public Uni<String> triggerQuery() {
return Uni.createFrom(() -> service.startLongRunningQuery())
.runSubscriptionOn(Infrastructure.getDefaultWorkerPool());
}
Note that you need RESTEasy Reactive for this to work (and not classic RESTEasy). If you use classic RESTEasy, you would need the quarkus-resteasy-mutiny extension (but I would recommend using RESTEasy Reactive, it will be way more efficient).
Use the EventBus for that https://quarkus.io/guides/reactive-event-bus
Send and forget is the way to go.
Now I'm learning Bidirectional streaming in asynchronous GRPC++.
Thanks for the master:https://github.com/Mityuha/grpc_async. I get much useful information to know the realization principle of this mode.But I have a question about it:
Not much to say,the code is following:
the server:
if(!ok || mcounter >= greeting.size())//ctx_.IsCancelled() doesn't work
{
std::cout << "[ProceedMM]: Trying finish" << std::endl;
status_ = FINISH;
responder_.Finish(Status(), (void*)this);
}
the client:
void AsyncCompleteRpc()
{
void* got_tag;
bool ok = false;
while(cq_.Next(&got_tag, &ok))
{
AbstractAsyncClientCall* call = static_cast<AbstractAsyncClientCall*>(got_tag);
call->Proceed(ok);
}
std::cout << "Completion queue is shutting down." << std::endl;
}
in this server,the end of ClientStream is judged by the bool value of OK which is send by client.It isn't similar to the way of synchronous GRPC,which is judged the steaming end by the return of bool Read(RequestType* request) in the class of ServerReaderWriter in many times.It's so strange to find the same way in the class of ServerAsyncReaderWriter which is void Read(R* msg, void* tag).Though I know it's because of the asynchronous way.But if I don't know how much times of asynchronous streaming without the judgement of "OK", how to find the way like synchronous streaming to judge the end of client streaming.Because I test the performance by java which is the same code between synchronous with asynchronous ways,which don't have the bool value of OK in asynchronous ways.
So can someone help me?Or tell me some ways to deal with it or find a way to test the performance testing of GRPC++ by Bazel of in my another question.
I'm not 100% sure that I get the question, but what ok tells you is (when false) that the operation you requested couldn't be completed and nothing else will ever complete successfully on that side of the stream. So if you issue a Read operation and the Next gives you a !ok value, then you can be sure that no more data will ever come back from the client. A more detailed explanation is given in the comments for the CompletionQueue class.
Thanks and good luck with gRPC.
In the case of receiving a stream in an asynchronous client of gRPC, you will use a ClientAsyncReader<> class to receive data. This class differs when both send and receive are stream, but logic is the same.
This class has a Finish() method which you need to call after finishing sending your rpc data to server. When answer stream from server is finished, a message to CompletionQueue will be added which corresponds to this method. This Finish method returns final status when its message is returned in CQ. You can find out that your stream is finished. Your code will be similar to this:
response_reader_ = stub->PrepareAsyncXYZ(ctx_, req, cq);
response_reader_->StartCall(&start_data_);
response_reader_->Finish(&status_, &finish_data_);
in this sample, message in CQ will have finish_data_ tag and you can use it to handle it properly. You will probably will need to manage messages for Finish() and Read() by reference counting, because you will probably get an additional failed read message too. when message with finish_data_ is received in CQ, status_ will have the valid value of status.
At least it is how I wrote it.
I'm trying to implement a very simple service connected to an AMQP broker with Alpakka. I just want it to consume messages from its queue as a stream at the moment they are pushed on a given exchange/topic.
Everything seemed to work fine in my tests, but when I tried to start my service, I realized that my stream was only consuming my messages once and then exited.
Basically I'm using the code from Alpakka documentation :
def consume()={
val amqpSource = AmqpSource.committableSource(
TemporaryQueueSourceSettings(connectionProvider, exchangeName)
.withDeclaration(exchangeDeclaration)
.withRoutingKey(topic),
bufferSize = prefetchCount
)
val amqpSink = AmqpSink.replyTo(AmqpReplyToSinkSettings(connectionProvider))
amqpSource.mapAsync(4)(msg => onMessage(msg)).runWith(amqpSink)
}
I tried to schedule the consume() execution every second, but I experienced OutOfMemoryException issues.
Is there any proper way to make this code run as an infinite loop ?
If you want to have a Source restarted when it fails or is cancelled, wrap it with RestartSource.withBackoff.
Iam looking to write a socket program based on libev. I noticed that several examples as stated in https://github.com/coolaj86/libev-examples/blob/master/src/unix-echo-server.c use the call backs based on init. For example,
main() {
......
ev_io_init(&client.io, client_cb, client.fd, EV_READ|EV_WRITE);
ev_io_start(EV_A_ &server.io);
}
static void client_cb (EV_P_ ev_io *w, int revents)
{
if (revents & EV_READ)
{
....
} else if (revents & EV_WRITE) {
......
}
}
My question comes from the expected behaviour, say for example, all that i read when in EV_READ is stored in a linked list. Lets say I keep getting free flow of packets to read, will i ever get a chance to get into EV_WRITE? I have to send out all that I recv through read to another socket. So Will it be once EV_READ and second time EV_WRITE? In other words when will EV_WRITE be unblocked? Or do I need to block EV_READ for EV_WRITE to be called. Can someone help me understand this?
I think you should keep write callback separated from read callback:
main() {
ev_io_init(&read.io, read_cb, client.fd, EV_READ);
ev_io_init(&write.io, writead_cb, client.fd, EV_WRITE);
ev_io_start(EV_A_ &read.io);
ev_io_start(EV_A_ &write.io);
}
This is my solution.
To answer shortly: If you allways check for one type of event first and then have an else
if for the other you risk starvation. In general I would check for both, unless the specified protocol made it impossible for both to be activated at the same time.
Here is a more iffy answer:
The link in your question does not contain a code structure such as your question. The client https://github.com/coolaj86/libev-examples/blob/master/src/unix-echo-client.c does have a similar callback. You will notice it disables write events, when it has written once.
// once the data is sent, stop notifications that
// data can be sent until there is actually more
// data to send
ev_io_stop(EV_A_ &send_w);
ev_io_set(&send_w, remote_fd, EV_READ);
ev_io_start(EV_A_ &send_w);
That looks like an attempt to avoid starvation of the pipe READ event branch. Even though Im not very familiar with libev, the github examples you linked to do not seem very robust. E.g static void stdin_cb (EV_P_ ev_io *w, int revents)does not use the return value of getline() to detect EOF. Also the send() and recv() socket operation return values are not inspected for how much was read or written (though on local named pipe streams the amounts will most likely match the amounts that were requested). If this was later changed to a TCP based connection, checking the amounts would be vital.
Here i have one function which is listen mode. this function listing something which i got form some device.
Here when my function is in listen mode that time i want to create timeout. if i will not get any response from particular device than i want o exit from this function and have to notify.
if during this timeout period if i will get response from device than i have to continue with work and stop this timeout and there is no limits to complete this work in any time duration.
So how can i implement this thing for a function.
Any body please can me help me to implement this thing with timeout functionality.
Depending on how you are waiting for a response from this device, the answer to your question will be different. The basic framework is:
int do_something_with_device()
{
if (!wait_for_response_from_device()) {
return TIMEOUT_ERROR;
}
// continue with processing
}
As for how you implement wait_for_response_from_device(), well, every device is different. If you're using sockets or pipes, use select(). If you're interfacing with something that requires a busy-wait loop, it might look like:
int wait_for_response_from_device()
{
time_t start = time(NULL);
while (time(NULL) - start < TIMEOUT) {
if (check_device_ready()) {
return 1;
}
}
return 0;
}
Naturally, the implementation of check_device_ready() would be up to you.
Take a look at man 2 alarm. You can set or disable signals which will be sent to your application after a certain time period elapses.