Multi-paxos instance - distributed

In wikipediahttp://en.wikipedia.org/wiki/Paxos_(computer_science), the proposal number and the instance number is inconsistent.
Message flow: Multi-Paxos, steady-state
| X--------->|->|->| | | Accept!(N,I+1,W)
Message flow: Collapsed Multi-Paxos, steady state
| X->|->| Accept!(N+1,I,W)
one is increment on instance number, one is ballot number, why?

Nice catch, the following is what should happen:
at the Start phase of the Multi-Paxos protocol, a Promise message is sent: here you increase the ballot number N with 1;
at the Steady phase of the Multi-Paxos protocol, a Accept message is sent: here you increase the instance number I with 1.
Thus, the correct answer for would be Accept(N,I+1,W) for both Multi-Paxos and Collapsed Multi-Paxos steady state.

Related

Timing range from request to response in J1939

Generally a request is sent via 0xEB00 and response is capured by 0xEC00 (Response more than 8 bytes) in J1939,what is the range for the response from request?
Example:-
0.00 - 0xEB00 - EC FE 00
xx.xx - 0xEC00 - xx xx xx xx xx EC FE 00.
What can be the possible range of xx.xx be ?
Looked into many options but unable to find the exact range.
Somewhere its mentioned as 10 - 200 => Datapackets
and somewhere its mentioned as 0 - 1250
All devices, when required to provide a response, must do so within 0.20s (Tr). All devices expecting a response must wait at least 1.25s (T3) before giving up or retrying. These times assure that any latencies due to bus access or message forwarding across bridges do not cause unwanted timeouts. Different time values can be used for specific applications when required. For instance, for high-speed control messages, a 20 ms response may be expected. Reordering any buffered messages may be necessary to accomplish the faster response. There is no restriction on minimum response time.
Time between packets of a multipacket message directed to a specific destination is 0 to 200 ms. This means that backto-back messages can occur and they may contain the same identifier. The CTS mechanism can be used to assure a
given time spacing between packets. The required time interval between packets of a Multipacket Broadcast message is 50 to 200 ms. A minimum time of 50 ms assures the responder has time to pull the message from the CAN hardware.
The responder shall use a timeout of 250 ms (provides margin allowing for the maximum spacing of 200 ms).
a. Maximum forward delay time within a bridge is 50 ms Total number of bridges = 10 (i.e. 1 tractor + 5 trailers + 4 dollies = 10 bridges) Total network delay is 500 ms in one direction.
b. Number of request retries = 2 (3 requests total); this includes the situation where the CTS is used to request the retransmission of data packet(s). c. 50 ms margin for timeouts

Tuning receive ACK timeout on Linux for Webserver

I work with remote sensing equipment for a particular university's environmental sciences department and am having trouble with latency. We have sensors in the field that periodically send updates (as POSTs) back into a box running linux and a simple web server. The devices connect via a daisy chain of wireless repeaters (we're experimenting with shortwave as an alternative to commercial cellular networks).
Our problem comes from high latency in some of our links; approaching 20 seconds. Our sensors are unable to complete the threeway handshake to establish the connection to the server. We see the SYN get to the server and then the SYNACK get back to the remote, but by the time the ACK gets to the server from the remote the server has already sent a RST.
Is there a way to tune the linux TCP stack so that the timeout for receiving these ACK's can be lengthened?
Timeout is related to RTO.
See RFC6298 2.1. Until a round-trip time (RTT) measurement has been made for a segment sent between the sender and receiver, the sender SHOULD set RTO <- 1 second.
See RFC6298 5.5. The host MUST set RTO <- RTO * 2
My Ubuntu(14.04)'s "/proc/sys/net/ipv4/tcp_synack_retries" is set to 5.
In other words,
Server state.
1. SYN/ACK First Send ===> 1 second wait ...
2. SYN/ACK Retransmission ===> 2 second wait ...
3. SYN/ACK Retransmission ===> 4 second wait ...
4. SYN/ACK Retransmission ===> 8 second wait ...
5. SYN/ACK Retransmission ===> 16 second wait ...
6. SYN/ACK Retransmission ===> 32 second wait ...
Above total wait time is longer than 20 seconds.
Try "/proc/sys/net/ipv4/tcp_synack_retries" changing.

How to identify reason of OverQuotaError when sending emails?

I send emails with cron job and task queue usage. The job is executed every 15 minutes and the queue used has the following setup:
- name: send-emails
rate: 1/m
max_concurrent_requests: 1
retry_parameters:
task_retry_limit: 0
But quite often apiproxy_errors.OverQuotaError exception happens. I am checking Quota Details and see that I am still within the daily quota (Recipients Emailed, Attachment Data Sent etc.), and I believe I couldn't be over maximum per minute limit, since the the rate I use is just 1 task per minute (i.e. send no more than 1 mail per minute).
Where am I wrong and what should I check?
How many emails are you sending? You have not set a bucket-size, so it defaults to 5. Your rate sets how often the bucket is replenished. So, with your current configuration, you can send 5 emails every minute. That means if you are sending more than 75 emails to the queue every 15 minutes, the queue will fill up, and eventually go over quota.
I have not tried this myself, but when you catch the apiproxy_errors.OverQuotaError exception, does the message contain any detail as to why it is over quota/which quota has been exceeded?
try:
send_mail_here
except apiproxy_errors.OverQuotaError, message:
logging.error(message)

Client-server restaurant simulation

I am making a client-server with select() in C that simulates a fast-food or whatever.
I have clients that order a random "food" between 1-5. The server decides every 30 sec. what is the most preferred food from all the clients. He serves those clients and they close connection, and for the rest, sends them a message to wait. The clients that are not served try 2 more times, otherwise they leave.
My question is, how can I make the server check every 30 sec. what their orders are?
I tried making an array, but I can't figure out how to make the server 'check' continuously every 30 sec. and set the array to 0, then.
Here is the pseudocode:
**client**
patience=0;served=0;
do
{send random 1-5
receieve message. if 1->served=1; if 0, patience++;
}while patience !=3 and served!=1;
if served==1 send -1
else send -2
close connection
**Server**
while(1)
{
serves clients in a concurent manner
select
adds client in socket list
serves client
waits message
if 1-5 adds in vector
//here I don't know how to make it wait for 30 sec.
//If I put sleep(30), it's going to sleep for each client every time. I want it to permanently check every 30 sec and send messages to clients.
send respones(0 or 1, depending on if the max order is the same as the client's)
if -1, thanks for the food
if -2, going somewhere else
close client connection
}
You may want to try sleep() function in a loop, what it will do is pause your program for that much duration and then execute the statements that follow.
sleep(30);
Is what you want i suppose. Have a look here for more about sleep.
What your code should be like is:
for(i=0; i<=100; i++)//Checks 100 times
{
sleep(30);
checkserverforupdate();
}
Update
Code:
while(1){//runs infinite times
sleep(30);//pauses here for 30 seconds everytime before/after running the following code
*client**
patience=0;served=0;
do
{send random 1-5
receieve message. if 1->served=1; if 0, patience++;
}while patience !=3 and served!=1;
if served==1 send -1
else send -2
close connection
**Server**
while(1)
{
serves clients in a concurent manner
select
adds client in socket list
serves client
waits message
if 1-5 adds in vector
//here I don't know how to make it wait for 30 sec.
//If I put sleep(30), it's going to sleep for each client every time. I want it to permanently check every 30 sec and send messages to clients.
send respones(0 or 1, depending on if the max order is the same as the client's)
if -1, thanks for the food
if -2, going somewhere else
close client connection
}
}
See what happens with sleep(30) is that the program pauses for the time of 30 seconds and then executes the code written after it and if you put it in a while loop, it waits every time.
Update 2:
Code for how to get the current time in C:
here
/* localtime example */
#include <stdio.h>
#include <time.h>
int main ()
{
time_t rawtime;
struct tm * timeinfo;
time ( &rawtime );
timeinfo = localtime ( &rawtime );
printf ( "Current local time and date: %s", asctime (timeinfo) );
return 0;
}
Update 3:
So your code should be like this:
time_t rawtime;
struct tm * timeinfo;
time ( &rawtime );
timeinfo = localtime ( &rawtime );
printf ( "Current local time and date: %s", asctime (timeinfo) );
All of your code here
and then get the time again,
time_t rawtime1;
.....
and then you may calculate the difference between them and put a sleep statement as you wish. If the time you want it to pause is in x then,
sleep(x);
return 0;
You can use a thread to wait for 30 secs and when the thread returns do a check to find out the preferred food from all the clients. This link for threading will help.
As you are into pseudocode here goes
t = time now
start loop
... do stuff
t2 = time now
delta = t2 - t
sleep (30 - delta % 30)
end of loop
The select() function in C has a timeout parameter. The correct logic would be something like:
nextTime = now() + 30 seconds;
while(running) {
currentTime = now();
if( nextTime <= currentTime() ) {
makeFood();
nextTime += 30 seconds;
} else {
timeout = nextTime - currentTime;
if(select(..., timeout) > 0) {
// Check FD sets here
}
}
}
EDIT (added explanation below).
You have to have some sort of data structure to track that state (e.g. which foods have been requested by which clients). This data structure isn't shown above.
The // Check FD sets part modifies the current state. For example, when a new client orders some food you might add the details to a linked list of clients waiting for orders. When a client gets sick of waiting you'd remove their order from the list of clients waiting for orders.
The makeFood() function would look at the (previously stored) state and determine which type of food to make; then inform clients to either keep waiting or that their food was made (and remove clients that did receive their food).
EDIT 2 - The problem with using sleep().
If you sleep until it's the right time to send replies and then receive packets, and then send replies; then the time it takes to receive packets varies (depending on the number of packets received) which means that the time that you send replies varies (depending on the number of packets received).
A time-line might look like this:
12:00:59 - stop sleeping
12:00:59 - check for received packets and find there's none
12:00:59 - do 1 second of processing
12:01:00 - start sending reply packets
12:01:02 - finish sending reply packets
12:01:02 - sleep for the remaining 27 seconds
12:01:29 - stop sleeping
12:01:29 - check for received packets and find there's lots of them
12:01:29 - process received packets for 3 seconds
12:00:33 - do 1 second of processing (3 seconds too late)
12:01:34 - start sending reply packets (3 seconds too late)
12:01:02 - finish sending reply packets (3 seconds too late)
12:01:02 - sleep for the remaining 24 seconds
In this case, the replies are sent between 0 and 3 seconds too late because it takes between 0 and N seconds to handle received packets. This timing variation is called "jitter".
To fix the "jitter", you handle received packets while you're waiting so that the time spent handling those received packets doesn't influence the time that replies are sent.
In this case, the time-line looks like this:
12:00:59 - stop waiting and receiving packets
12:00:59 - do 1 second of processing
12:01:00 - start sending reply packets
12:01:02 - finish sending reply packets
12:01:02 - receive packets while waiting for the remaining 27 seconds
12:01:29 - stop waiting and receiving packets
12:00:29 - do 1 second of processing
12:01:00 - start sending reply packets
12:01:02 - finish sending reply packets
12:01:02 - receive packets while waiting for the remaining 27 seconds
In this case, the replies are always sent at the correct time.
EDIT 3 - The other problem with using sleep().
The first problem with sleep exists because it's impossible to predict the ideal amount of time to spent sleeping in advance, which causes you to wake up too late (which causes "jitter"). The second problem is the effect this has only the server's ability to handle periods of "almost maximum load".
Basically, you sleep too long, and as load increases you reach a point where you've slept too long and left yourself with insufficient time to process the load you should've received while you were waiting.
This causes oscillations when there's constant "almost maximum load". For example, if there's enough load that it takes 29 seconds to do everything (receive packets, process them and send reply packets); then for one 30 second period you sleep too long, fail to handle the load in time and finish too late; then you realise that you need to sleep for "negative seconds" (but you can't turn back time - the best you can do is skip the sleep()). This pushes the excess load onto the next 30 second period of time. After several (many) of these "sleep skipped but still starting too late" 30 seconds periods you eventually catch up; and as soon as that happens you sleep too long again and start the entire thing all over again.
Of course this assumes there's constant "almost maximum load". Real life is almost never like that. In practice you tend to get fluctuations - e.g. one 30 second period with "slightly higher than maximum average load", followed by a 30 second period with "slightly less than maximum average load". What we're really talking about here is the ability of the server to recover quickly from the occasional "slightly higher than maximum average load" periods.
For my method, the server recovers as quickly as possible. For the "sleep()" method it takes much longer to recover, resulting in prolonged "lag surges".

Looping through a changing event list

I have a list of machines that I ping depending on their response time.
Initially all machines are pinged each t milliseonds for 5 times. Depending on the response time for these 5 pings on each machine I adjust the time of the ping to stretch or contract it, till arriving at some configuration like this:
machine1: t1....t2....t3....t4....t5
machine2: t1......t2......t3......t4......t5
..
machineN: ..t1..t2..t3..t4..t5
^
|machine needs to be pinged at this tick
t1..tN represents the (millisecond) ticks of the clock.
Having a thread per machine to do the ping is evident, but not an optimal solution due the number of machines.
Rather, one thread that iteratites through the global order of the events is desirable, something like this:
while(true){
fetch_next_machine_to_be_pinged();
ping_it();
if(any_machine_pinged_5_times());
reorder_events(); //adjust the time of its next 5 ping
//continue
}
What is the best way to achieve this? (ps: language C).
I would use a priority queue.
At any time, the queue would contain one entry per machine. The "priority" of entry M would be the timestamp when machine M needs to be pinged next, and the payload would be some token identifying the machine (so that you know whom to ping).

Resources