What's the difference between change input arguments and creating a new object in Vprog of spark graphx - spark-graphx

there is my program:
static class Vprog extends AbstractFunction3< Object, OddRange, OddRange, OddRange> implements Serializable {
#Override
public OddRange apply(Object l, OddRange self, OddRange sumOdd) {
System.out.println(self.getS()+self.getI()+" ---> "+sumOdd.getS()+sumOdd.getI());
self.setS(sumOdd.getS() + self.getS());
self.setI(self.getI() + sumOdd.getI());
return new OddRange(self.getS(), self.getI());
}
}
the question is if I use return new OddRange like above in class Vprog,I can change the vertexRDD
But, if I use retuen self, like:
static class Vprog extends AbstractFunction3< Object, OddRange, OddRange, OddRange> implements Serializable {
#Override
public OddRange apply(Object l, OddRange self, OddRange sumOdd) {
System.out.println(self.getS()+self.getI()+" ---> "+sumOdd.getS()+sumOdd.getI());
self.setS(sumOdd.getS() + self.getS());
self.setI(self.getI() + sumOdd.getI());
return self;
}
}
The vertexRDD didn't change.
I know RDD is immutable, but how can I update the vectexRDD in spark.graphx.pregel correctly?Can you give me any advise?
I have found the same question:
Spark Pregel is not working with Java
But I use spark 2.3.0,maybe it have the same problem?

I think I have found the answer:
We must return a new one, if we wanna change the data which will be used in next sendMsg in Vprog.
that's because Vprog changes the vertexRDD, but sendMsg uses the tripletsRDD. And what's more, the verteies in the tripletsRDD are not equels to vertexRDD, it's just a copy of vertexRDD. So,the problem is when to update the verteies in tripletsRDD when vertexRDD is changed.
We can follow the source below to find out the reason:
first part:pregel(in Pregel.scala)->joinVertices(in GraphOps.scala)->outerJoinVertices(in GraphImpl.scala)->diff(in VertexRddImpl.scala)
And then:
second part:pregel(in Pregel.scala)->mapReduceTriplets(in GraphXUtils.scala)->aggregateMessagesWithActiveSet(in GraphImpl.scala).
In first part, I found that Vprog will compare the VertexRDD data before and after execution. SO, if it is modified on the source data, they will be the same. Then a data structure named replicatedVertexView will be generated to store different VertexRDD info. If they are same, nothing will be stored.
In second part, it will update the tripletsRDD with the infomations which stored in the relicatedVertexView. And then, use the tripletsRDD in sendMsg.
So, if we don't return new in Vprog, the tripletsRDD will not be changed with VertexRDD, and the results will be wrong.

Related

Solr Determinative Random on Solr Cloud

We want randomized results that stay consistent with paging. We attempted to modify the solr.RandomSortFieldType by changing the randomSeed function, based upon this stack overflow question: Solr: Random sort order after index version change.
It modified this function, by removing "(int)top.getVersion()":
private static int getSeed(String fieldName, LeafReaderContext context) {
final DirectoryReader top = (DirectoryReader) ReaderUtil.getTopLevelContext(context).reader();
// calling getVersion() on a segment will currently give you a null pointer exception, so
// we use the top-level reader.
return fieldName.hashCode() + context.docBase + [Removed (int)top.getVersion()];
}
Since this did not yield the expected results, I am wondering if it would be possible to do something a little more determinative by creating a stored field that is a hash of the document title field called (title_hash). Then I could replace the getSeed function as follows:
private static int getSeed(String fieldName, LeafReaderContext context) {
final DirectoryReader top = (DirectoryReader) ReaderUtil.getTopLevelContext(context).reader();
// calling getVersion() on a segment will currently give you a null pointer exception, so
// we use the top-level reader.
return fieldName.hashCode() + getTitleHashValue();
}
Is it possible to reference a document field value from this function that inside a class that inherits from field type? If so, how would I do that, as I am not familiar with the internals of Solr?

Subclassing in Objective C Runtime

I am attempting to implement a solution from How to set canBecomeKeyWindow? Into my native C application using Objective-C runtime (The app is already written with objective C Runtime). Is there a way to create a subclass purely in Objective-C Runtime?
Right now I just create NSWindow object but need to be able to create my own so I can override the function specified in that question.
objc_msgSend((id)objc_getClass("NSWindow"), sel_registerName("alloc"));
The signature of can_become_key_window_true is slightly incorrect. According to the documentation (https://developer.apple.com/documentation/objectivec/objective-c_runtime/imp?language=objc) the function should have at least two arguments: "self" and "_cmd". So the signature should be like:
static bool can_become_key_window_true(__unused id _self, __unused SEL _cmd) {
return true;
}
You could also use #encode to construct the type encoding for the function.
char encoding[10]; // should be enough
snprintf(encoding, 10, "%s%s%s", #encode(BOOL), #encode(id), #encode(SEL));
... or you could get a method from UIWindow and get its type encoding like:
Method m = class_getInstanceMethod(objc_lookUpClass("UIWindow"), sel_getUid("canBecomeKeyWindow"));
const char *encoding = method_getTypeEncoding(m);
And as you might have noticed you could use sel_getUid() instead of sel_registerName as you expect this selector to be already registered by this time (because you are about to override an existing method).
To allocate a new instance you could use
window = class_createInstance(__UIWindow);
Figured it out after a lot of code searching:
// Subclass NSWindow with overridden function
Class __NSWindow =
objc_allocateClassPair(objc_getClass("NSWindow"), "__NSWindow", 0);
class_addMethod(__NSWindow,
sel_registerName("canBecomeKeyWindow"),
(IMP)can_become_key_window_true, "B#:");
objc_registerClassPair(__NSWindow);
// Allocate a new __NSWindow
window = objc_msgSend((id)__NSWindow, sel_registerName("alloc"));
And then can_become_key_window_true is defined as:
static bool can_become_key_window_true() {
return true;
}
I use objc_allocateClassPair to subclass the object and return a Class of that object. Then I use class_addMethod to override the method canBecomeKeyWindow. And finally use objc_registerClassPair to register my new class before using it as I would a normal NSWindow.

Static extension methods in Kotlin

How do you define a static extension method in Kotlin? Is this even possible? I currently have an extension method as shown below.
public fun Uber.doMagic(context: Context) {
// ...
}
The above extension can be invoked on an instance.
uberInstance.doMagic(context) // Instance method
but how do I make it static method like shown below.
Uber.doMagic(context) // Static or class method
To achieve Uber.doMagic(context), you can write an extension to the companion object of Uber (the companion object declaration is required):
class Uber {
companion object {}
}
fun Uber.Companion.doMagic(context: Context) { }
This is what the official documentation says:
Kotlin generates static methods for package-level functions. Kotlin
can also generate static methods for functions defined in named
objects or companion objects if you annotate those functions as
#JvmStatic. For example:
Kotlin static methods
class C {
companion object {
#JvmStatic fun foo() {}
fun bar() {}
}
}
Now, foo() is static in Java, while bar() is not:
C.foo(); // works fine
C.bar(); // error: not a static method
I actually had this exact question 30 minutes ago, so I started digging around and couldn't find any solution or workaround for this, BUT while searching I found this section on the Kotlinglang website that states that:
Note that extensions can be defined with a nullable receiver type. Such extensions can be called on an object variable even if its value is null.
So then I had the craziest idea ever, why not define an extension function with a nullable receiver (without actually using that receiver) and then call it on a null object!
So I tried that, and it worked pretty well, but it looked so ugly. It was like this:
(null as Type?).staticFunction(param1, param2)
So I went around that by creating a val in my extensions file of the receiver type that had a value of null and then use it in my other class.
So, as an example, here is how I implemented a "static" extension function for the Navigation class in Android:
In my NavigationExtensions.kt file:
val SNavigation: Navigation? = null
fun Navigation?.createNavigateOnClickListener(#IdRes resId: Int, args: Bundle? = null, navOptions: NavOptions? = null,
navigationExtras: Navigator.Extras? = null) : (View) -> Unit {
//This is just implementation details, don't worry too much about them, just focus on the Navigation? part in the method declaration
return { view: View -> view.navigate(resId, args, navOptions, navigationExtras) }
}
In the code that uses it:
SNavigation.createNavigateOnClickListener(R.id.action_gameWonFragment_to_gameFragment)
Obviously, this isn't a class name, it is just a variable of the class type that has a null value. This is obviously ugly on the extension maker side (because they have to create the variable) and on the developer side (because they have to use the SType format instead of the actual class name), but it is the closest that can be achieved right now compared to actual static functions. Hopefully, the Kotlin language makers will respond to the issue that was created and add that feature in the language.
Since I keep coming across this when searching, here's a different approach I haven't seen anyone mention that works in a static way and it works with generics!
Extension definitions:
// Extension function
fun <T> KClass<T>.doSomething() = /* do something */
// Extension Property
val <T> KClass<T>.someVal get() = /* something */
Usage:
MyType::class.doSomething()
MyType::class.someVal
As you can see, the trick is attaching the extension function to the KClass of the type instead since that can be referenced statically.
You can create a static method with using Companion object like:
class Foo {
// ...
companion object {
public fun bar() {
// do anything
}
}
}
and then you can call it like:
class Baz {
// ...
private fun callBar() {
Foo.bar()
}
}
Recomend you to look at this link. As you can see there, you just should declare method at the top-level of the package (file):
package strings
public fun joinToString(...): String { ... }
This is equal to
package strings;
public class JoinKt {
public static String joinToString(...) { ... }
}
With constans everything are the same. This declaration
val UNIX_LINE_SEPARATOR = "\n"
is equal to
public static final String UNIX_LINE_SEPARATOR = "\n";
I also required the ability to extend a Java object with a static method and found the best solution for me was to create a Kotlin object that extended the Java class and add my method there.
object Colour: Color(){
fun parseColor(r: Int?, g: Int?, b: Int?) = parseColor(String.format("#%02x%02x%02x", r, g, b))
}
invocation:
val colour = Colour.parseColor(62, 0, 100)
I'm also quite fond of having the possibility to add static extension methods in Kotlin. As a workaround for now I'm adding the exntension method to multiple classes instead of using one static extension method in all of them.
class Util
fun Util.isDeviceOnline(context: Context): Boolean {
val connMgr = context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
val networkInfo = connMgr.activeNetworkInfo
return networkInfo != null && networkInfo.isConnected
}
fun Activity.isDeviceOnline(context: Context) = { Util().isDeviceOnline(context) }
fun OkHttpClient.isDeviceOnline(context: Context) = { Util().isDeviceOnline(context) }
To create an extension method in kotlin you have to create a kotlin file(not a class) then declare your method in the file
Eg:
public fun String.toLowercase(){
// **this** is the string object
}
Import the function in the class or file you are working on and use it.

How do I get class of a Ruby object in C?

I have a function that handles two types: NVector and NMatrix; the former is derived from the latter. This function is basically a specialized copy constructor. I want it to return an object of the same type as that upon which it was called, so, NVector returns NVector, not NMatrix.
static VALUE nm_init_modifiedcopy(VALUE self) {
// ... some code ...
// formerly, I had cNMatrix where klass is. But it could also be cNVector!
return Data_Wrap_Struct(klass, mark_func, delete_func, unwrapped_self_copy);
}
How do I get the class property of an object to pass into Data_Wrap_Struct?
Like clockwork, as soon as I ask a question on Stackoverflow, I find the answer.
The macro is CLASS_OF.
static VALUE nm_init_modifiedcopy(VALUE self) {
// ... some code ...
return Data_Wrap_Struct(CLASS_OF(self), mark_func, delete_func, unwrapped_self_copy);
}

AS3: Only allow a certain number of a certain type of object into an Array

I want to find a way to only allow certain objects into an array that have a certain word in thier class name. Or at least find the optimal way of doing something like this. Heres the details. I have an Array that stores all the objects dropped into a cart.
function addProductToArray (e:MouseEvent):void{
currMC = (e.target as MovieClip);
myCart.itemsInCart.push(currMC);
trace(myCart.itemsInCart);}
If, for example, I drop an [object BreadGrain] and a [object PastaGrain].
trace(myCart.itemsInCart);// would trace [object BreadGrain],[object PastaGrain].
Easy, no problems there. But what do I do if I only want to allow 2 objects with "Grain" in their Classname into the array? I want to do this so that the user can only drop 2 of each type of food into the 'cart'. The types of food are Grain, Fruit, Vegetable, Meats etc and I've appended the type of food to the end of the Classname, hopefully so that I can use it to detect what type of food it is and stop it from being added over the limit as well as displaying an error. i.e "You already have 2 Grain products".
I hope that makes sense. Anyway, i've found that works well to a degree:
if (currMC is BreadGrain) {
myCart.itemsInCart.push(currMC);
} else {
// error message code here
}
BUT I have several products and I don't want to have to write a if/else or switch statement for them all. I was hoping to do this dynamically with something similar to:
//this is an example of the logic
if (currMC classNameContainsTheWord "Grain" AND myCart.itemsInCart DoesNotContainMoreThan 2 Grain Objects) {
myCart.itemsInCart.push(currMC);
} else {
// error message code here
}
I'm stumped. Even just a "Dude, you are doing this all wrong" would help. Thanks.
You can get the class name of any object with the getQualifiedClassName function. Then you could try to match strings agains a certain pattern, with a RegExp or you could also just check if the class name contains some substring.
That said, I think a better approach could be using either a common base class or an interface.
// assuming your objects extend MovieClip
public class Grain extends MovieClip{
public function Grain() {
super();
}
public function someCommonMethodToAllGrains():void {
}
}
or
// It's customary to prefix interfaces name with an "I" in AS;
// I'm not doing it here so the code works for both a base class and an interface
public interface Grain {
function someCommonMethodToAllGrains():void;
}
Then, if you went with the base class:
public class BreadGrain extends Grain {
public function BreadGrain() {
super();
}
override public function someCommonMethodToAllGrains():void {
// if this makes sense for your object...
super.someCommonMethodToAllGrains();
}
}
public class PastaGrain extends Grain {
public function PastaGrain() {
super();
}
override public function someCommonMethodToAllGrains():void {
// if this makes sense for your object...
super.someCommonMethodToAllGrains();
}
}
Or, with the interface
public class BreadGrain extends MovieClip implements Grain {
public function BreadGrain() {
super();
}
public function someCommonMethodToAllGrains():void {
}
}
public class PastaGrain extends MovieClip implements Grain {
public function PastaGrain() {
super();
}
public function someCommonMethodToAllGrains():void {
}
}
If these objects are MovieClips, perhaps it's less tedious to use a base class, because otherwise you'd have to cast your objects back to MovieClip (or DisplayObject) any time you want to add them to the display list (or remove them). By the way, that's because someone at Adobe forgot to include an IDisplayObject interface and have the display list API accept objects that implemented this interface instead of a half-assed abstract class that you can't derive directly anyway (a.k.a. DisplayObject); this would have make it easier to treat display objects as interfaces, but I digress).
Anyway, either with an interface or a common base class you could do your validation with the is operator, but you'd just have to check for one type: Grain.
if(theObject is Graing && theArray.lenght <= 2) {
theArray.push(theObject);
}
You could also take this further and use a Vector instead of an Array. A Vector works almost the same as an Array, but it's strictly typed, so you could do:
var items:Vector.<Grain> = new Vector.<Grain>();
items.push(grainObject);
You'll get a compile time error if you try to add an object that does not extend/implement Grain.
Vectors are available for Flash Player 10 and you'd need Flash CS4, though (if you're using the Flash IDE; otherwise, I think you'd need at least the 3.2 SDK to compile).
Hm. I think you're going to need something a bit more complex to make this work properly. You're actually asking a two-part question: how to keep track of stuff, and how to identify stuff. I'll start with the easy bit, keeping track.
DISCLAIMER: My AS3 is pretty rusty, but at least the theory should be sound, even if the implementation might be a bit off.
First, you'd want to define the limits for each type of food, thus:
var dctLimits = new Object(); // not really a Dictionary, but we'll use it like one
dctLimits[ "grain" ] = 3;
dctLimits[ "meat" ] = 5;
...
Then, you want to keep count of objects you're adding to your cart
var dctCount = new Object();
dctCount[ "grain" ] = 0;
dctCount[ "meat" ] = 0;
...
Then, when you add a new object, first check its type against the relevant count. If the counter is less than the limit, let it in and increment the counter:
var type:String = getFoodTypeForObject( currMc );
if( dctCount[ type ] < dctLimit[ type ] ){
items.push( currMc );
dctCount[ type ]++;
} else {
// throw error
}
You'll notice that I've created an imaginary function, getFoodTypeForObject(). This is the trickier bit: identification.
You could implement your example logic like so:
function getFoodTypeForObject( currMc ){
if( getQualifiedClassName( currMc ).indexOf( "Grain" ) > -1 ){
return( "grain" );
} else if( getQualifiedClassName( currMc ).indexOf( "Meat" ) > -1 ){
return( "meat" );
}
...
}
Where classNameContainsTheWord is achieved with a combination of getQualifiedClassName and indexOf, but better would be to use a common base class, as suggested by Juan Pablo Califano. I'd suggest a slightly different style though:
public class CartItem extends MovieClip{
public var isGrain:Boolean;
public var isMeat:Boolean;
public function CartItem() {
super();
}
}
use that as the base Class for your cart item MCs, then set those boolean properties on the instances of MCs on your stage. Then, you can detect the type of something like this:
function getFoodTypeForObject( object ){
if( object.isGrain ){
return( "grain" );
} else if( object.isMeat ){
return( "meat" );
}
...
}
Cleaner than all that classname business, and has the added benefit that you can set something's properties independent of its class name.
It's not perfect; for instance, you'd need something more advanced if you needed a lot of properties, but it should be enough for you to keep going.
Uni had me doing other stuff for a while but finally I can get back into my game project.
I've got it working. I used Juan Pablo Califano's method. I did initially use Henry Cooke's because I wanted to get away with making a .AS file for each food (i.e. apple.as, cereal.as, bread.as, oranges.as). With Henry Cooke's method I created a
`var foodTypeLimit:Object = new Object();
foodTypeLimit["grain"]=2;
foodTypeLimit["fruit"]=2;
And var foodTypeCount:Object = new Object();
etc etc
`
For each food type. Then used the:
var type:String = getFoodTypeForObject( currMc );
if( foodTypeCount[ type ] < foodTypeLimit[ type ] ){
items.push( currMc );
foodTypeCount[ type ]++;
} else {
// throw error
}
As suggested. The function returned the string and viola it worked fine. However because my foodTypeCount variables (for example foodTypeCount["grain"]=0;) was inside the function, every time the function called these were set to 0 so the increment never got above with each call. So I thought, ok, i'll put these foodTypeCount variables outside of the function along with the instantiation of the var foodTypeCount:Object = new Object(); BUT NO, I kept getting the:
Error #1120: Access of undefined property foodTypeObject.
Even though it was right under the freakin declaration. I get i'm just too noob to understand why this is so. Anyway, for this reason (the lack of incrementation, which was essential to this function) I bit the bullet and used Juan Pablo Califano's way.
First I wrote out the classes like so:
public class Bread extends MovieClip implements Grain {
public function Bread() {
super();
}
public function someCommonMethodToAllGrains():void {
}
}
And then added the interface
`public interface Grain {
function someCommonMethodToAllGrains():void;
}
`
And now my function looks something like this:
if(currMC is Grain){
if(grainCount<2){
addProductToBasket();
grainCount++;
} else {
notAllowed("Grain");
}
}
function addProductToBasket(){
removeChild(currMC);
basketArray.push(currMC);
//remove listeners set mc to null etc
}
function notAllowed(foodType:String){
error.text="Sorry but you already have 2"+foodType+"products";
}
I tried to put all this into a switch. For example:
switch(currMC){
case is Grain:
//do this
break;
}
The above implementation didn't work. Perhaps switch statements probably aren't meant to be used that way. idk. :S
Anyway, thanks for the really great answers guys, this is my favorite site to come to for answers to life the universe and everything.

Resources