How we know the value of message.get() in SimpleShortestPathsComputation?
if we have Vertex<DoubleWritable, DoubleWritable, DoubleWritable> vertex instead of
Vertex<LongWritable, DoubleWritable, FloatWritable> vertex
How we know that Messages has the value of MinDist and not e.g VertextID or EdgeValue?
#Override public void compute(
Vertex<LongWritable, DoubleWritable, FloatWritable> vertex,
Iterable<DoubleWritable> messages) throws IOException {
if (getSuperstep() == 0) {
vertex.setValue(new DoubleWritable(Double.MAX_VALUE));
}
double minDist = isSource(vertex) ? 0d : Double.MAX_VALUE;
for (DoubleWritable message : messages) {
minDist = Math.min(minDist, message.get());
}
Thank you
Message will have the value you will put inside via the sendMessage method. Just because they have the same type, doesn't mean that things can get mixed up. That's not how the serialisation / deserialisation works in Giraph.
If you don't trust it, you can also have a look at the code here: https://github.com/apache/giraph
Besides that you mixed up a thing, Message doesn't contain the min distance, The min distance is saved inside the vertex' value. The message contains a distance to the source vertex, when it passes the current vertex and is actually the edge data (you called it edge value). The message data, or edge data, is actually of type FloatWritable in the original case - see the code here:
....
/**
* Class which holds vertex id, data and edges.
*
* #param <I> Vertex id
* #param <V> Vertex data
* #param <E> Edge data
*/
public interface Vertex<I extends WritableComparable,
V extends Writable, E extends Writable> extends
ImmutableClassesGiraphConfigurable<I, V, E> {
....
}
Related
How does the stop mechanism work in this code?
#interface GifAnaimator()
#property BOOL stopping;
#end
#implementation GifAnaimator
- (void)startWithURL:(CFURLRef)imageURL {
__weak GifAnaimator *weakSelf = self;
CGAnimateImageAtURLWithBlock(imageURL, nil, ^(size_t index, CGImageRef image, bool* stop) {
// Some image handling code...
*stop = weakSelf.stopping;
});
}
- (void)stop {
self.stopping = YES;
}
#end
What's confusing me about this code is that the dereferenced stop is assigned a plain, non-pointed to, BOOL, stopping. Afterwards, when stopping is mutated, stop somehow gets the same mutation.
I tried capturing stop in a block, and calling the block later on to mutate it like so:
weakSelf.stopAnimation = ^{
*stop = YES;
};
This code makes more sense to me but it doesn't work.
What exactly is happening here?
The documentation comment for CGAnimateImageAtURLWithBlock says:
/* Animate the sequence of images contained in the file at `url'. Currently supported image
* formats are GIF and APNG. The `options' dictionary may be used to request additional playback
* options; see the list of keys above for more information. The block is called on the main queue
* at time intervals specified by the `delay time' of the image. The animation can be stopped by
* setting the boolean parameter of the block to false.
*/
If self.stopping is mutated and *stop later gets the same mutation I assume that's because the block is called after the value of self.stopping is changed and the block sets *stop to that value.
Capturing *stop won't work because in all likelihood it doesn't exist outside of the block.
NSDictionary has a method with a similar signature:
- (void)enumerateKeysAndObjectsWithOptions:(NSEnumerationOptions)opts
usingBlock:(void (^)(KeyType key, ObjectType obj, BOOL *stop))block
In pseudo code this does something like:
for (key, object) in storage {
BOOL stop = NO;
block(key, object, &stop);
if(stop) {
break;
}
}
So stop does not exist outside of the closure.
there is my program：
static class Vprog extends AbstractFunction3< Object, OddRange, OddRange, OddRange> implements Serializable {
#Override
public OddRange apply(Object l, OddRange self, OddRange sumOdd) {
System.out.println(self.getS()+self.getI()+" ---> "+sumOdd.getS()+sumOdd.getI());
self.setS(sumOdd.getS() + self.getS());
self.setI(self.getI() + sumOdd.getI());
return new OddRange(self.getS(), self.getI());
}
}
the question is if I use return new OddRange like above in class Vprog，I can change the vertexRDD
But, if I use retuen self, like:
static class Vprog extends AbstractFunction3< Object, OddRange, OddRange, OddRange> implements Serializable {
#Override
public OddRange apply(Object l, OddRange self, OddRange sumOdd) {
System.out.println(self.getS()+self.getI()+" ---> "+sumOdd.getS()+sumOdd.getI());
self.setS(sumOdd.getS() + self.getS());
self.setI(self.getI() + sumOdd.getI());
return self;
}
}
The vertexRDD didn't change.
I know RDD is immutable, but how can I update the vectexRDD in spark.graphx.pregel correctly？Can you give me any advise？
I have found the same question:
Spark Pregel is not working with Java
But I use spark 2.3.0,maybe it have the same problem?
I think I have found the answer:
We must return a new one, if we wanna change the data which will be used in next sendMsg in Vprog.
that's because Vprog changes the vertexRDD, but sendMsg uses the tripletsRDD. And what's more, the verteies in the tripletsRDD are not equels to vertexRDD, it's just a copy of vertexRDD. So，the problem is when to update the verteies in tripletsRDD when vertexRDD is changed.
We can follow the source below to find out the reason：
first part：pregel(in Pregel.scala)->joinVertices（in GraphOps.scala）->outerJoinVertices(in GraphImpl.scala)->diff(in VertexRddImpl.scala)
And then:
second part：pregel(in Pregel.scala)->mapReduceTriplets(in GraphXUtils.scala)->aggregateMessagesWithActiveSet(in GraphImpl.scala).
In first part, I found that Vprog will compare the VertexRDD data before and after execution. SO, if it is modified on the source data, they will be the same. Then a data structure named replicatedVertexView will be generated to store different VertexRDD info. If they are same, nothing will be stored.
In second part, it will update the tripletsRDD with the infomations which stored in the relicatedVertexView. And then, use the tripletsRDD in sendMsg.
So, if we don't return new in Vprog, the tripletsRDD will not be changed with VertexRDD, and the results will be wrong.
I can't seem to put my finger on this and why the array is not being initialized.
Basically I am coding a 2d top down spaceship game and the ship is going to be fully customizable. The ship has several allocated slots for certain "Modules" (ie weapons, electronic systems) and these are stored in an array as follows:
protected Array<Weapon> weaponMount;
Upon creating the ship none of the module arrays are initialized, since some ships might have 1 weapon slot, while others have 4.
So when I code new ships, like this example:
public RookieShip(World world, Vector2 position) {
this.width = 35;
this.height = 15;
// Setup ships model
bodyDef.type = BodyType.DynamicBody;
bodyDef.position.set(position);
body = world.createBody(bodyDef);
chassis.setAsBox(width / GameScreen.WORLD_TO_BOX_WIDTH, height / GameScreen.WORLD_TO_BOX_HEIGHT);
fixtureDef.shape = chassis;
fixtureDef.friction = 0.225f;
fixtureDef.density = 0.85f;
fixture = body.createFixture(fixtureDef);
sprite = new Sprite(new Texture(Gdx.files.internal("img/TestShip.png")));
body.setUserData(sprite);
chassis.dispose();
// Ship module properties
setShipName("Rookie Ship");
setCpu(50);
setPower(25);
setFuel(500);
setWeaponMounts(2, world);
setDefenseSlots(1);
addModule(new BasicEngine(), this);
addModule(new BasicBlaster(), this);
// Add hp
setHullHP(50);
setArmorHP(125);
setShieldHP(125);
}
#Override
public void addModule(Module module, Ship currentShip) {
// TODO Auto-generated method stub
super.addModule(module, currentShip);
}
#Override
public void setWeaponMounts(int weaponMounts, World world) {
weaponMount = new Array<Weapon>(weaponMounts);
// super.setWeaponMounts(weaponMounts, world);
}
#Override
public String displayInfo() {
String info = "Everyones first ship, sturdy, reliable and only a little bit shit";
return info;
}
When I set the number of weapon mounts the following method is called:
public void setWeaponMounts(int weaponMounts, World world) {
weaponMount = new Array<Weapon>(weaponMounts);
}
This basically initializes the array with a size (weapon mounts available) to whatever the argument is. Now to me this seems fine but I have setup a hotkey to output the size of the Array, which reports zero. If I try to reference any objects in the array, it throws an outofbounds exception.
The addModule method adds to the array as follows:
public void addModule(Module module, Ship currentShip) {
currentShip.cpu -= module.getCpuUsage();
currentShip.power -= module.getPowerUsage();
if(module instanceof Engine){
engine = (Engine) module;
}else if(module instanceof Weapon){
if(maxWeaponMounts == weaponMount.size){
System.out.println("No more room for weapons!");
}else{
maxWeaponMounts += 1;
weaponMount.add((Weapon)module);
}
}
}
My coding ain't great but heh, better than what I was 2 month ago....
Any ideas?
First of all, You should avoid instanceof. It's not a really big deal performance-wise, but it always points to problems with your general architecture. Implement two different addModule methods. One that takes a Weapon, and one that takes an Engine.
Now back to topic:
else if(module instanceof Weapon){
if (maxWeaponMounts == weaponMount.size) {
System.out.println("No more room for weapons!");
} else{
maxWeaponMounts += 1;
weaponMount.add((Weapon)module);
}
}
It looks like you use maxWeaponMounts as a counter instead of a limit. That's why I assume that it will initially be 0. The same holds for Array.size. It is not the limit, but size also counts how many elements the Array currently holds. Thus you will always have (maxWeaponMounts == weaponMount.size) as 0 == 0 and you will not add the weapon to the array. It will always stay empty and trying to reference any index will end in an OutOfBoundsException.
What you should actually do is using maxWeaponMounts as a fixed limit and not the counter.
public void setWeaponMounts(int weaponMounts, World world) {
weaponMount = new Array<Weapon>(weaponMounts);
maxWeaponMounts = weaponMounts;
}
else if(module instanceof Weapon){
if (weaponMount.size >= maxWeaponMounts) {
System.out.println("No more room for weapons!");
} else{
weaponMount.add((Weapon)module);
}
}
I have trained a classifier for my instances, and now want to export it to an Android application, where the Weka library will be unavailable.
It is not suitable to simply add the Weka library in the Android application, because of it's size (6.5 Mb).
Is there any other way to use my classifier to evaluate and label other unlabeled instances? Are there any smaller, independent library specifically design for this?
Of course I could, eventually, write my own library to interpret the output model of Weka, but it would seem logical to me, that such a solution already exists. (although it escapes me, somehow)
There are no independent libraries that would do what you want. You could remove all the parts of Weka you don't need and package that into a library.
In your particular case, the easiest thing to do might be to take the decision tree that Weka learns and put it directly into the code in a series of if...else statements. You could even write a script that takes the (graphical) output of the decision tree and writes that code for you.
After paying more attention the output model of weka, I noticed that by using the option that generates the tree in a Java class form, I can use it separatly from the weka library.
You can remove the generated WekaWrapper and keep only the internal class, which is a basic implementation of the tree:
The class looks something like this:
public class WekaWrapper
extends Classifier {
/**
* Returns only the toString() method.
*
* #return a string describing the classifier
*/
public String globalInfo() {
return toString();
}
/**
* Returns the capabilities of this classifier.
*
* #return the capabilities
*/
public Capabilities getCapabilities() {
weka.core.Capabilities result = new weka.core.Capabilities(this);
result.enable(weka.core.Capabilities.Capability.NOMINAL_ATTRIBUTES);
result.enable(weka.core.Capabilities.Capability.NOMINAL_CLASS);
result.enable(weka.core.Capabilities.Capability.MISSING_CLASS_VALUES);
result.setMinimumNumberInstances(0);
return result;
}
/**
* only checks the data against its capabilities.
*
* #param i the training data
*/
public void buildClassifier(Instances i) throws Exception {
// can classifier handle the data?
getCapabilities().testWithFail(i);
}
/**
* Classifies the given instance.
*
* #param i the instance to classify
* #return the classification result
*/
public double classifyInstance(Instance i) throws Exception {
Object[] s = new Object[i.numAttributes()];
for (int j = 0; j < s.length; j++) {
if (!i.isMissing(j)) {
if (i.attribute(j).isNominal())
s[j] = new String(i.stringValue(j));
else if (i.attribute(j).isNumeric())
s[j] = new Double(i.value(j));
}
}
// set class value to missing
s[i.classIndex()] = null;
return WekaClassifier.classify(s);
}
/**
* Returns the revision string.
*
* #return the revision
*/
public String getRevision() {
return RevisionUtils.extract("1.0");
}
/**
* Returns only the classnames and what classifier it is based on.
*
* #return a short description
*/
public String toString() {
return "Auto-generated classifier wrapper, based on weka.classifiers.trees.Id3 (generated with Weka 3.6.9).\n" + this.getClass().getName() + "/WekaClassifier";
}
/**
* Runs the classfier from commandline.
*
* #param args the commandline arguments
*/
public static void main(String args[]) {
runClassifier(new WekaWrapper(), args);
}
}
class WekaClassifier {
private static void checkMissing(Object[] i, int index) {
if (i[index] == null)
throw new IllegalArgumentException("Null values are not allowed!");
}
public static double classify(Object[] i) {
return node0(i);
}
protected static double node0(Object[] i) {
return 0.0; // unacc
}
}
So, yeah, in fact you can do it really easy. Things to remember:
to classify an instance, call the classify(Object[]) method;
the return value will be a float value;
the return values are explained in comments, right next to the return command;
the parameters have no validation, so be careful in which order you are inputing them (this part was done by the weka dependent part);
the order is the one defined in the arff file.
If you want to run RandomForests, you can use a little script I wrote that turns the output of WEKA's -printTrees option of the RandomForest classifier into Java source code.
http://pielot.org/2015/06/exporting-randomforest-models-to-java-source-code/
The code you need to include into your Android app will consist of three classes only: the class with the generated model + two classes to make the classification work.
I have an array that stored some Object with its data, and I try to store it in my computer.
If I try to load for the data after I've save the data, I could get a correct data.
Exp: [Object Player]
But if I restart the flash, the data seems to be gone.
What is the problem?
private var sharedObject:SharedObject = SharedObject.getLocal("aquarium", "/");
public function save(n:String):void
{
/* player list will only handle the list of all the Players
* each player data will handle by Player class itself.
*/
registerClassAlias("Player", Player)
player = new Player()
player.newPlayer(n, LATEST_VERSION)
playerArray.push(player)
//saving as shared object
sharedObject.data.aquariumData = playerArray
sharedObject.flush()
load()
}
public function load():void
{
if (sharedObject.size > 0)
{
trace("loading player info")
playerArray = sharedObject.data.aquariumData
trace(playerArray)
}
else
{
trace("there's no record")
}
}
Can you please provide the code how you obtain the shared object ?
Do you use var sharedObject:SharedObject = SharedObject.getLocal("sharedObject"); or something like this ?
Apart from that when calling registerClassAlias("Player", Player) before serialization keep in mind that it must be called before extraction of the data also, so the de-serialization will work correctly and returns array of Player objects not array of Object objects.
And ofc closing the sharedObject is very nice practice after flushing :)
P.S. Your code works as far as i've tested it replacing your Player class with other custom class.