Starting with Windows Vista, every application automatically gets a thread pool (the "default thread pool").
My Question is: is there a way to configure the min and max number of threads for this default thread pool?
SetThreadpoolThreadMaximum seems to work only for a non-default thread pool (a pool created with CreateThreadpool):
SetThreadpoolThreadMaximum(NULL, 4); - throws 0xC000000D: An invalid parameter was passed to a service or function
According to Joe Duffy from Microsoft, this is not possible.
Here's a quote from his book, Concurrent Programming on Windows, chapter 7. Thread Pools, page 345:
Note that it is not possible to alter the default thread pool's
minimum and maximum count; instead, you must specify a pointer to a
custom TP_POOL object. Prior to Vista, you could change the
process-wide default pool's maximum (as we see later). The reason this
capability has been removed is because it depends on races: the last
component to call the API would win. This can cause conflicts between
components in the same process that are unaware of each other but want
different maximum or minimum values.
The "old way" of increasing the pool size the author is referring to is the macro WT_SET_MAX_THREADPOOL_THREADS, used in QueueUserWorkItem. That setting seems to be ignored in the new pool architecture.
So the default pool's minimum is fixed at 0 and the maximum is 500.
Related
I am having some trouble understanding the way windowing is implemented internally in Flink and could not find any article which explain this in depth. In my mind, there are two ways this can be done. Consider a simple window wordcount code as below
env.socketTextStream("localhost", 9999)
.flatMap(new Splitter())
.groupBy(0)
.window(Time.of(500, TimeUnit.SECONDS)).sum(1)
Method 1: Store all events for 500 seconds and at the end of the window, process all of them by applying the sum operation on the stored events.
Method 2: We use a counter to store a rolling sum for every window. As each event in a window comes, we do not store the individual events but keep adding 1 to previously stored counter and output the result at the end of the window.
Could someone kindly help to understand which of the above methods (or maybe a different approach) is used by Flink in reality. The reason is, there are pros and cons to both approach and is important to understand in order configure the resources for the cluster correctly.
eg: The Method 1 seems very close to batch processing and might potentially have issues related to spike in processing at every 500 sec interval while sitting idle otherwise etc while Method2 would need to maintain a common counter between all task managers.
sum is a reducing function as mentioned here(https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#reducefunction). Internally, Flink will apply reduce function to each input element, and simply save the reduced result in ReduceState.
For other windows functions, like windows.apply(WindowFunction). There is no aggregation so all input elements will be saved in the ListState.
This document(https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#window-functions) about windows stream mentions about how the internal elements are handled in Flink.
I am trying to shape a JMeter test involving a Concurrency Thread Group and a Throughput Shaping Timer as documented here and here. the timer is configured to run ten ramps and stages with RPS from 1 to 333.
I want to set up the Concurrency Thread Group to use the schedule feedback function and added the formula in the Target concurrency field (I have updated the example from tst-name to the actual timer name). ramp-up time and steps I have set to 1 as I assume the properties are not that important if the throughput is managed by the timer; the Hold Target Rate time is 8000, which is longer than the steps added in the timer (6200).
When I run the test, it ends without any exceptions within 3 seconds or so. The log file shows a few rows about starting and ending threads but nothing alarming.
The only thing I find suspicious is the Log entry "VirtualUserController: Test limit reached, thread is done plus thread name.
I am not getting enough clues from the documentation linked here to figure this out myself, do you have any hints?
According to the documentation rampup time and steps should be blank:
When using this approach, leave Concurrency Thread Group Ramp Up Time and Ramp-Up Steps Count fields blank"
So your assumption that setting them to 1 is OK, seems false...
I have my Gatling scenario set up and now I want to configure a simulation with fixed number of users for specific period of time - number of users should initially be increased gradually to specific value and then kept there by adding new as required as users finish.
I specifically don't want to use constantUsersPerSec (which injects users at a constant rate) but something like .throttle(reachUsers(100) in rampUpTime, holdFor(10 minute)) which should inject users when required.
If it's still relevant: Gatling supports a throttle method pretty much as you outlined it. You can use the following building blocks (taken from the docs):
reachRps(target) in (duration): target a throughput with a ramp over a given duration.
jumpToRps(target): jump immediately to a given targeted throughput.
holdFor(duration): hold the current throughput for a given duration.
So a modified example for your use case could look something like this:
setUp(scn.inject(constantUsersPerSec(100) during(10 minutes))).throttle(
reachRps(100) in (1 minute),
holdFor(9 minute)
)
I am asked to write a program,an abstraction of runtime scheduling algorithms in C.The one I'm having problem with is "priority scheduling".It takes runtimes from the user like the other algorithms,except it also takes priorities of these processes.What troubles me is that I can't relate priorities with runtimes. How am I supposed to make a relationship between the priority and the runtime?According to algorithm,it runs the one with highest priority first.I just need to know how to make this connection,thanks.
EDIT :-
Store the priorities in an array and sort out the array as per decreasing priorities and map each priority with it's timing! Then,simply display the sorted array from starting and it's mapping with the process-time.
If the process are added at run-time, the you need to adopt greedy-algorithm to solve that problem. Select that process with highest priority in increasing order of incoming time.
Check this to learn more Interval Scheduling Algorithm
BTW,for implementing the real-scheduling algorithm :-
If the first process is currently running and then there comes a new process with the same or lesser priority level, then you should add just the new process to the FIFO data-structure(queue) so that it gets executed at the next(for equal/lesser priority).
If the first process is currently running, and a new process with a higher priority arrives and requests memory, you must pass the current process' upcoming instruction to the stack and execute the higher priority process and then return interrupt to execute the upcoming instruction!
I hope you are getting confused with the DATA-STRUCTURES to implement in a peculiar manner. Also,there is significant use of priority(higher)!
I'm trying to write a web server in C under Linux using protocol HTTP1.1 .
I've used select for multiple requests and I'd like to implement persistent connections but it didn't work so far 'cause I can't set a timeout properly. How can I do it? I think about setsockopt function:
setsockopt(connsd, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv, sizeof(tv))
where tv is a struct timeval. This isn't working either.
Any suggestions?
SO_RCVTIMEO will only work when you are actually reading data. select() won't honor it. select() takes a timeout parameter in its last argument. If you have a timer data structure to organize which connections should timeout in what order, then you can pass the soonest to timeout time to select(). If the return value is 0, then a timeout has occurred, and you should expire all timed out connections. After processing live connections (and re-setting their idle timeout in your timer data structure), you should again check to see if any connections should be timed out before calling select() again.
There are various data structures you can use, but popular ones include the timing wheel and timer heap.
A timing wheel is basically an array organized as a circular buffer, where each buffer position represents a time unit. If the wheel units is in seconds, you could construct a 300 element array to represent 5 minutes of time. There is a sticky index which represents the last time any timers were expired, and the current position would be the current time modulo the size of the array. To add a timeout, calculate the absolute time it needs to be timed out, modulo that by the size of the array, and add it to the list at that array position. All buckets between the last index and the current position whose time out has been reached need to be expired. After expiring the entries, the last index is updated to the current position. To calculate the time until the next expiration, the buckets are scanned starting from the current position to find a bucket with an entry that will expire.
A timer heap is basically a priority queue, where entries that expire sooner have higher priority than entries that expire later. The top of a non-empty heap determines the time to next expiration.
If your application is inserting a lots and lots of timers all the time, and then cancelling them all the time, then a wheel may be more appropriate, as inserting into the wheel and removing from the wheel is more efficient than inserting and removing from a priority queue.
The simplest solution is probably to keep a last-time-request-received for each connection, then regularly check that time and if it's too long ago then close the connection.