I am a newbie trying to understand the working of Giraph 1.2.0. with hadoop 1.2.1.
Is there any way to figure out the number of vertices processed by each mapper?
The call method of org.apache.giraph.graph.ComputeCallable class is executed once per superstep. Inside this function, for each partition owned by this map task, the computePartition function is called. So, you can easily define an integer (counter) to this class. Then, in computePartition, if compute method of the vertex is called, increment the counter. Finally, at the end of call method print your counter. So, for each superstep of each mapper, it prints the number of vertices processed.
Related
I want to use Faster rcnn inception v2 to do object detection in tensorflow.js. But i can't find some method in tfjs like get_tensor_by_name and session run for prediction.
In tensorflow (python), the code as the following:
Define input and output node:
# Definite input Tensors for detection_graph
self.image_tensor = self.detection_graph.get_tensor_by_name('image_tensor:0')
# Definite output Tensors for detection_graph
self.detection_boxes = self.detection_graph.get_tensor_by_name('detection_boxes:0')
self.detection_scores = self.detection_graph.get_tensor_by_name('detection_scores:0')
self.detection_classes = self.detection_graph.get_tensor_by_name('detection_classes:0')
self.num_detections = self.detection_graph.get_tensor_by_name('num_detections:0')
Predict:
(boxes, scores, classes, num) = self.sess.run(
[self.detection_boxes, self.detection_scores, self.detection_classes, self.num_detections],
feed_dict={self.image_tensor: image_np_expanded})
Do anyone know how to implement those two part of code in tfjs?
Please help. Thank you!
You don't have a session.run function in tensorflow.Js as there is in Python. In python, you start defining a graph and in the run function, you execute the graph. Tensors and Variables are assigned values in the graph, but the graph defines only the flow of computation, it does not hold any values. The real computation occurs when you run the session. One can create many session where each session can assign different values to the variable, that is why the graph has a get_by_tensor_name which outputs the tensor whose name is given as parameter.
You don't have the same mechanism in Js. You can use the variable as soon as you define them. It means that whenever you define a new tensor or variable, you can print it in the following line whereas in python, you can only print the tensor or variable only during a session. The get_by_tensor_name does not really have a sense in Js.
As for the predict function, you do have one in Js as well. if you create a model using tf.model or tf.sequential, you can invoke predict to make a prediction.
How to use an step function for an array in Anylogic?
step function is applied to double values, but I want to applied on elements of an array at a specific time.
You can't... so this is a solution:
Instead of an array you should use a linkedHashMap where your key is the specific time and the element is the step value you want at that time. So you defined it as follow:
And you put the values like this:
stepsArray.put(3.0,2.3);
where 3.0 is the time in which the step will occur and 2.3 is the value the step will take. You have to put there all the values you need. You are the one who has to fill these values according to your needs.
Then you create an cyclic event that will evaluate if it's time to apply a step and you create a variable of type double that will be the one storing the value of the step.
So, the event:
double theTime=round(100*time())/100.0;//it's better to round up the time just in case
if(stepsArray.containsKey(theTime)){
variable=stepsArray.get(theTime);
}
note that I'm using a variable, not a dynamic variable.. they you can connect the variable to wherever your step is needed in the sd model.
This method is a bit complicated, but it's the most general for your completely ambiguous question.
Not sure Felipe's approach is the best one but maybe I misunderstand the question.
Have you tried using a "table function" object? Define it as below where the horizontal axis represents the time unit and the vertical your step function data:
Then, use a cyclic event that every relevant time unit (depends on your model) pulls the current required value from the table function:
I read somewhere in the past that angular.foreach is asynchronous unlike looping over arrays which is synchronous. For a long time I was taking into account this and doing the necessary to avoid executing the code which comes after the loop before it's finishes all its iterations (by wrapping the angular.foreach inside an anonymous JavaScript function which calls a callback which will be executed once the loop finishes all iterations).
(function(callback){
angular.foreach(..)
callback();
})(callback)
But I had a conversation with a collegue who didn't agree that angular.foreach is asynchronous and I also couldn't find that information again which makes me confused now.
no. Take a look at the docs
Furthermore your code wouldn't work if foreach would be asynchronous.
If foreach would be async, the callback would be called immediately after calling foreach and foreach would be put onto the eventqueue which would execute it some time in the future.
Javascripts concurrency model does not have threads but instead uses an eventloop. This means every async operation is pushed onto the eventqueue and executed later.
Have a look into the MDN
There may be a scenario where you want to make code behave asynchronously.
I had a scenario where I used local storage to store an ad-hoc user selected collection of jobs that I wanted to perform the same operation on.
I had a web service call to convert a list of job names into a returned a collection of job objects. I initially tried using a
foreach loop inside the subscribe pf the service layer, that operated on the results.
Then I tried calling another method within the foreach loop that as it performed the operations removed the job name from local storage when the operation posted to the web service correctly.
The problem was on the second iteration I read the collection of names from local storage again - before the set to remove had completed.
There was a lot of manipulation of the job and object properties to create the parameters passed on the function call, so I ended up refactoring the code, creating a value object interface and stored the information in a value object array for the whole job collection I had returned. I included the index of the job too in the value object.
I introduced a BehaviourSubject property to the class.
During the restructuring, I just added an entry to the value object array collection within the forEach loop instead. At the end of the loop. I sent next(0) to the BehaviourSubject to start the ball rolling.
Each time a job name was removed from local storage, I converted service to return a Promise.
Then in the code after the service was called I put this code in the then part, behaviour subject.next(index from value object +1)
In the initialisation I set the behaviour subject up with a -1 value..
Then in the subscription to the BehaviourSubject class I ignored -1,
And when the index +1 was > length of value object collection called completion routine - which bounce app back to prior page.
When the index was between 0 and 1 less than collection size, I just called the method that had originally been in the forEach loop with the value object entry with the value object match the index of the behaviour subject.
By doing this I had converted the behaviour of the forEach into something asynchronous.
I'm trying to evaluate Apache Flink for the use case we're currently running in production using custom code.
So let's say there's a stream of events each containing a specific attribute X which is a continuously increasing integer. That is a bunch of contiguous events have this attributes set to N, then the next batch has it set to N+1 etc.
I want to break the stream into windows of events with the same value of X and then do some computations on each separately.
So I define a GlobalWindow and a custom Trigger where in onElement method I check the attribute of any given element against the saved value of the current X (from state variable) and if they differ I conclude that we've accumulated all the events with X=CURRENT and it's time to do computation and increase the X value in the state.
The problem with this approach is that the element from the next logical batch (with X=CURRENT+1) has been already consumed but it's not a part of the previous batch.
Is there a way to put it back somehow into the stream so that it is properly accounted for the next batch?
Or maybe my approach is entirely wrong and there's an easier way to achieve what I need?
Thank you.
I think you are on a right track.
Trigger specifies when a window can be processed and results for a window can be emitted.
The WindowAssigner is the part which says to which window element will be assigned. So I would say you also need to provide a custom implementation of WindowAssigner that will assign same window to all elements with equal value of X.
A more idiomatic way to do this with Flink would be to use stream.keyBy(X).window(...). The keyBy(X) takes care of grouping elements by their particular value for X. You then apply any sort of window you like. In your case a SessionWindow may be a good choice. It will fire for each key after that key hasn't been seen for some configurable period of time.
This approach will be much more robust with regard to unordered data which you must always assume in a stream processing system.
How to do a basic loop through different properties for a fixed set of controllers? Loop controller runs a set group a certain number of times, does not use properties though.
I can do modules, and set the values to properties for multi thread group usage, but how to pass the next iteration of the property, and run the loop again?
property x
do module (points to controllers)
next property
Say I have a list of 44 characters, and I want to loop through those characters in a ${name} while I'm doing a test. I'd very much not like to build 44 sets of controllers for one character change.
Please Note I cannot add extra files to my computer. It has to work via the stock available controllers. I'm using Jmeter 2.4 r961953
Thanks
I will elaborate slightly more about the BeanShell method. My assumption is that you'd like to do it within one User Thread, if so my proposal would be:
Create a Loop Controller.
Logic Controller->Loop Controller
Inside Loop Controller add following entries:
Config Element -> Counter
Preprocessors -> BeanShell preprocessor
Sampler -> yourSampler
The Counter element will be used as an index that will be used to choose valid value from our array, hence we need to specify a Reference Name for the Counter - let's say that it will be loopCounter.
Now we have to switch to BeanShell preprocessor and define the array of values. A great thing is that we have vars variable available and it gives us CRUD access to variables used in the scenario:
String[] varArray = {"Value1", "Value2"};
idx = Integer.parseInt(vars.get("loopCounter"))-1;
vars.put("myVariable", varArray[idx]);
And for the final step, inside mySampler we can use a variable in a regular JMeter way : ${myVariable}
JMeter API can be very helpfull if you want a more sophisticated solution.
There are a handful of different ways to loop through different values without adding external files:
Use beanshell controller, and write javascript to set your variable
Use a counter to increment by one
Use User Parameters
You can set it up so each loop gets a different value.
Check out the various configuration controllers to find one that works best for you.
EDIT:
I meant user parameters, not user define variables.
User Parameter
You'd need one row per variable with 44 columns. Sorry for the confusion.
User Parameter Structure
test plan
- Thread Group looped 44 times
-- User parameter
-- Request
Beanshell Method
Alternately, you could do an array in javascript in connection with a counter. The Beanshell samplers have access to Jmeter variables and properties, allowing the beanshell sampler to read the counter value. This may be a faster, cleaner way then using User Parameters.
Beanshell structure
test plan
- Thread Group looped 44 times
-- Counter
-- Request
---- Beanshell pre-processor
Beanshell Pseudo code would be
def counter = value of Jmeter Counter
def array = array of values
declare the variable "sampler_value" to be used by sampler
def sampler_value = array # counter
Counter with CharAt function
If you only need to generate characters, you could use the javascript function to utilize the function charAt, using the value from the Counter. Basic structure would be:
test plan
- Thread Group looped 44 times
-- Counter
-- Request
with the request using something like ${__javaScript(charAt(${counter})) as the parameter value. You may have to use JEXL instead of javaScript or evalVar/V/eval inside the charAt function.