Is there a way to give the queue-name with async openfaas function invocation ?
We have a openfaas function deployed which will be invoked by multiple providers.
The execution time of the function depends on some parameters given by the provider. We want to queue function invocation via a separate queue-name but keep only a single function as the invocations are sparse but spiky.
What is the way to achieve this ?
You can specify the queue-name when deploying the function, but not when invoking it.
Reference :
https://docs.openfaas.com/reference/async/#multiple-queues
Related
Flink Stateful Functions 2.0 has the ability to make asychronous calls, for example to an external API:
[https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.0/sdk/java.html#completing-async-requests][1].
Function execution is then paused until the call completed with Success, Failure, or Unknown. Unknown is:
The stateful function was restarted, possibly on a different machine,
before the CompletableFuture was completed, therefore it is unknown
what is the status of the asynchronous operation.
What happens when there is a second call with the same ID to the paused/waiting function?
Does the callee then wait on the called function's processing of
its async result so that this second call executes with a clean,
non-shared post-async state?
Or does the second call execute on a
normal schedule, and thus on top of the state that was current as of
the async call, and then when the async call completes it continues
processing using the state that was updated while the async call was
pending?
Or maybe the call counts as a "restart" of the called
function - in which case what is the order of execution: the
"restart" runs and then the async returns with "restart" to execute
from the now updated state, or this order is reversed?
Or something else?
Function execution does not pause while an async request is completing. The instance for that id will continue to process messages until the request completes. This means the state can change while the future is running.
Think of your future as an ad-hoc function that you message and that then messages you back when it has a result. Functions can spawn multiple asynchronous requests without issue. Whichever future completes first will be processed first by the function instance, not necessarily the order in which they were spawned.
Newbie Gatling+Scala question: I’m using George Leung's gatling-grpc library (which is modeled after the http library) and trying to pass a value from the session (generated in a feeder), into a non-DSL, non-Gatling method call, specifically calls populating the gRPC payload object.
Before I start, let me add that it seems I can’t use the sessionFunction (Expression[T]) form of exec, which would resolve my issue:
.exec{ session => { … grpc(…).rpc(…)… }}
…because, AFAICT, the grpc call must be the last thing in the block, or else it’s never evaluated ... yet it can’t be the last thing in the block because there’s no way to coerce it to return a Session object (again, AFAICT).
Therefore, I have to use the ActionBuilder form of exec (grpc(...) returns a Call so this is as designed):
.exec( grpc(…).rpc(…)... )
and this works… until I have a gRPC payload (i.e., non-Gatling) method call to which I need to pass a non-constant value (from a feeder).
In this context, I have no access to a Session object, and the Gatling Expression Language is not applied because the library defining the gRPC types I need to use (to generate the payload) has no knowledge of Gatling.
So, in this fragment:
.header(transactionIdHeader)("${tid}.SAVE")
.payload(Student.newBuilder()
.setId(GlobalId.newBuilder().setValue("${authid}_${uniqId}").build()).build())
)
…the first call evaluates ${tid} because the param in the second parens is Expression[T], and hence is evaluated as Expression Language, but the second call fails to evaluate ${authid} or ${uniqId} because the external, generated library that defines the gRPC type GlobalId has no knowledge of Gatling.
So...
Is there a way to invoke the EL outside of Gatling's DSL?
Or a way to access a Session object via an ActionBuilder?
(I see that the Gatling code magically finds a Session object when I use the sessionFunction form, but I can't see whence it comes — even looking at the bytecode is not illuminating)
Or, turning back to the Expression[T] form of exec, is there a way to have an ActionBuilder return a Session object?
Or, still in the Expression[T] form, I could trivially pass back the existing Session object, if I had a way to ensure the grpc()... expression was evaluated (i.e., imperative programming).
Gatling 3.3.1, Scala 2.12.10
The gatling-grpc library is at phiSgr/gatling-grpc; I'm using version 0.7.0 (com.github.phisgr:gatling-grpc).
(The gRPC Java code is generated from .proto files, of course.)
You need the Gatling-JavaPB integration.
To see that in action, see here.
The .payload method takes an Expression[T], which is an alias for Session => Validation[T]. In plain English, that is a function that constructs the payload from the session with a possibility of failure.
Much of your frustration is not knowing how to get hold of a Session. I hope this clears up the confusion.
In the worst case one can write a lambda to create an expression. But for string interpolation or accessing one single object, Gatling provides an implicit conversation to turn an EL String into an Expression.
The problem is you want to construct well-typed payloads and Gatling's EL cannot help with that. The builders’ setters want a T, but you only have an Expression[T] (either from EL or the $ function). The library mentioned above is created to handle that plumbing.
After importing com.github.phisgr.gatling.javapb._, you should write the following.
...
.payload(
Student.getDefaultInstance
.update(_.getIdBuilder.setValue)("${authid}_${uniqId}")
)
For the sake of completeness, see the warning in Gatling's documentation for why defining actions in .exec(sessionFunction) is not going to work.
So I am having some problems with parameterized queries and combining them with callback functions in C.
So with sqlite3_exec(...) you can execute your SQL query and give a callback function as parameter with it. The callback function serves to print out the query output
But I want to use parameterized queries so I have to use sqlite3_prepare_v2(), one of the sqlite3_bind functions, sqlite3_step(), and sqlite3_finalize(), which don't support callback functions as parameter.
Is there another function that I can use to get best of both worlds?
(side note: the SQL statement that I try to execute is just a single line query)
Note the lead remark from the API docs for sqlite3_exec():
The sqlite3_exec() interface is a convenience wrapper around
sqlite3_prepare_v2(), sqlite3_step(), and sqlite3_finalize(), that
allows an application to run multiple statements of SQL without having
to use a lot of C code.
Things you should take away from that include:
the callback-based interface is not the main API.
it's written in terms of other accessible functions, so you could write your own, similar function that provides whatever callback interface you want.
If you really do need a callback interface that works with parameterized prepared statements, then you will indeed need to write an appropriate analogue of sqlite3_exec() yourself. If you want to stick as close as possible to sqlite3_exec() then you could use a similar signature, with a pointer to the (already prepared and filled) prepared statement in place of the first two parameters. The function would then need to execute the query via multiple calls to sqlite3_step(), each time extracting the needed data from the result row (sqlite3_column_sometype() * n), passing them to the callback function, and handling the callback's return value appropriately.
There's a lot of room for customization there, especially if you indeed have only one query to support. You might, for example, pass the parameters to the function instead of requiring them to be pre-bound. You might very well change the signature of the callback function to be better matched to the query results, and if there's only one query to be supported then it probably doesn't need to be told the column names.
If it turns out that you actually know the specific function that is supposed to be called per row, then it doesn't have to be a callback interface at all, in that you you receive a function pointer as a parameter. In that case, you could just call the function directly.
If we have several sources in our data flow/job, and some of them implement RichSourceFunction, can we assume that RichSourceFunction.open of these sources will be called and complete before any data will enter into this entire data flow (through any of the many sources) - that is even if the sources are distributed on different task managers?
Flink guarantees to call the open() method of a function instance before it passes the first record to that instance. The guarantee is scoped only to a function instance, i.e., it might happen that the open() method of a function instance was not called yet, while another function instance (of the same or another function) started processing records already.
Flink does not globally coordinate open() calls across function instances.
I'm trying to see if I can query all execution mitigation policies for a process in one call to GetProcessMitigationPolicy() function (or similarly set them via a call to SetProcessMitigationPolicy() function) instead of doing them individually, one-by-one.
The MSDN page suggests using ProcessMitigationOptionsMask parameter for that, but I'm not really sure how to use it, or what mask are they referring to?
Can someone show an example?