We are using Zeppelin 0.8.0.
When running all paragraphs in a note, we expect Zeppelin to stop at the first paragraph where an error occurred. Unfortunately, we found that this is not the case with the %spark.r interpreter: i.e., errors are printed but effectively ignored.
Here is an example of this using the %spark interpreter where an error occurred in the execution of the 2nd paragraph (1/0); consequently, Zeppelin shows the status of this paragraph as error and does not execute the 3rd paragraph (1+1) at all.
Now, lookat this example using the %spark.r interpreter where an error occurred in the execution of the 2nd paragraph (1 + "foo"). The interpreter prints the error; however, this error is not reported at the level of the interpreter per se and the paragraph status shows "FINISHED" as if execution was error-free. Consequently, Zeppelin continues executing the 3rd paragraph.
How can we force any error that occurs in the execution of a %spark.r paragraph to be reported as an "ERROR" of the paragraph itself, thereby preventing the execution of subsequent paragraphs when we invoke on "Run All Paragraphs"?
Related
I created a main behavior for my NAOv6 (nao_main_behavior_V2) that is supposed to be executed on startup. It sets the robots language to german, activates some of NAOs LEDs and waits for me to touch his head sensors to activate a different behavior called Manual_Mode.
Every time I start NAO he executes the main behavior but stops as soon as I touch his head sensors to activate the other behavior. Using the Choreograph I can find this Error message:
[ERROR] behavior.box :onInput_onStart:16 _Behavior__nao_main_behavior_v28d046fMain2726901504:/Error! Manual_Mode_12: _Behavior__nao_main_behavior_v28d046fMain2726901504:/Manual Mode_29: ALBehaviorManager::runBehavior Box _Behavior__nao_main_behavior_v28d046fManualMode2716269632:/Speech Reco. Ger_1 has failed with error: Traceback (most recent call last): File "/opt/aldebaran/lib/python2.7/site-packages/albehavior.py", line 120, in _safeCallOfUserMethod func() File "", line 55, in onInput_onStart RuntimeError: ALSpeechRecognition::pushContexts AsrHybridNuance::xPushContexts You need to stop or pause the ASR engine to be able to make this call.
The Manual_Mode behavior accesses the speech recognition so I can say commands but I never access anything speech recognition related earlier.
Starting (or restarting) the behavior manually using Choreograph works just fine, everything works like it is supposed to and I don't get any Errors.
I tried using a third behavior during startup that starts the Main_Behavior but I get the same result.
The behavior was originally written for a NAOv5 and it worked without any problems.
I think this is related to some autonomous life running in the background (depending of the robot/version you use).
Try by stopping the autonomous life using api command when starting your "behavior_v2" or to change the nature of your "manual_mode" behavior (interactive/solitary) (or of the behavior).
About stopping the autonomous life: refer to api in ALAutonomousLife: setState or enableAnAbility methods (depending of your version).
About changing nature of behavior, it can be seen in this video for example: https://youtu.be/xPdNoiuaQag
I found a workaround. I created a launch trigger condition for my behavior:
(('Launchpad/LifeTime' ~30 ))
After NAO finishes its startup and 30 seconds have passed my behavior is being triggered and works without any problems.
I have met with a strange issue in running R notebooks in zeppelin(0.7.2).
Spark intrepreter is in per note Scoped mode and spark version is 1.6.2 and SPARK_HOME is set.
Please find the steps below to reproduce the issue:
Create a notebook (Note1) and run any r code in a paragraph. I ran the following code.
%r
rdf <- data.frame(c(1,2,3,4))
colnames(rdf) <- c("myCol")
sdf <- createDataFrame(sqlContext, rdf)
withColumn(sdf, "newCol", sdf$myCol * 2.0)
Create another notebook (Note2) and run any r code in a paragraph. I ran the same code as above.
Till now everything works fine.
Create third notebook (Note3) and run any r code in a paragraph. I ran the same code. This notebook fails with the error
org.apache.zeppelin.interpreter.InterpreterException: sparkr is not
responding
What I understood from the analysis is that the process created for sparkr interpreter is not getting killed properly and this makes every third model to throw an error while executing. The process will be killed on restarting the sparkr interpreter and another 2 models could be executed successfully. ie, For every third model run using the sparkr interpreter, the error is thrown.
Help me to fix the problem.
you need to set spark.r.numRBackendThreads larger than 2. By default it is 2 which means you can only have 2 threads for RBackend. Since you are using scoped mode per note, each note will consume one thread for RBackend, so you can only run 2 note.
We are analyzing sequencing data while filtering and trimming fastq files encountered following error. Is the following error due to unavailability of core for processing commands?
Error in colnames<-(*tmp*, value = c("cs103_R1_dada.fastq", "cs110_R1_dada.fastq", : attempt to set 'colnames' on an object with less than two dimensions In addition: Warning message: In mclapply(seq_len(n), do_one, mc.preschedule = mc.preschedule, : all scheduled cores encountered errors in user code >
As pengchy suggested there may be something wrong with function.
try the same call by using lapply and error message will be more informative.
To clarify on what #f2003596 and #HelloWorld said: This just means that a crash occurred within the function you called, i.e. while it was executing that function. But this does not necessarily mean that your function is incorrect. For example, you get the same error when a variable has not been found.
That would mean your R function has a crash.
Note: If you include an unexpected argument in mclapply you also can get this error message. I put mC.cores instead of mc.cores by mistake and I got it.
It appears that fork fails under Mac OS 10.6.8. The algorithm is coded in R and I have mainly be using the foreach package for parallel processing. The code works perfectly well when run sequentially (foreach and %do%) but not when run in parallel (foreach and %dopar%) even though the processes run in parallel do NOT communicate.
I used foreach on this same machine a month ago and it worked fine. An update of the OS has been performed in the meantime.
Error Messages
I received several kinds of error messages that seems to come almost stochastically. Also the errors differ depending on whether the code is run from the terminal (either by copy-pasting in the R environment or with R CMD BATCH) or from the R console.
When run on the Terminal, I get
Error in { : task 1 failed - "object 'dp' not found"
When run on the R console I get either
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
....
<repeated many times when run on the R console>
or
Error in { : task 1 failed - "object 'dp' not found"
with the exact same code! Note that although this second error message is the same than the one received on the Terminal, the number of things that are printed (through the print() function) on the screen vastly differ!
What I've tried
I updated the package foreach and I also restarting my computer but it did not quite help.
I tried to print pretty much anything I could but it ended up being quite hard to keep track of what this algorithm is doing. For example, it often through the error about the missing object dp without executing the print statement at the line that precedes the call of the object dp.
I tried to use %dopar% but registering only 1 CPU. The output did not change on the Terminal, but it changed on the Console. Now the console gives the exact same error, at the same time than the terminal.
I made sure that several CPUs were in used when I ask for several CPUs.
I tried to use mclapply instead of foreach and registerDoMC() instead of registerDoParallel() to register the number of cores.
Extra-Info
My version of R is 3.0.2 GUI 1.62 Snow Leopard build. My machine has 16 cores.
I have a suite of scalatest tests that output information to the console using println as they run.
When I run the suite using the Eclipse Scala plug-in (using Run As ... / 3 ScalaTest - File from the context menu) there is additional output to the console about which tests pass and which fail. I guess this output is from the runner.
The problem is that the lines from my code and the lines from the runner are not interleaved sensibly. It's as if they are being printed from two different threads that aren't synchronized.
For example here is the output from a run
>>>>>>>>>>>>>Starting The parser should warn when the interface name at the end does not match >>>>>>>>>>>>>>>>>>
(interface Fred
interface Bob)
-----------------------------
File: <unknown> line: 2 column: 11 Name does not match.
----The AST after parsing-------------
[ IntfDeclNd( (), (), () ) ]
---------------------------------------
<<<<<<<<<<<<<Finished The parser should warn when the interface name at the end does not match <<<<<<<<<<<<<<<<<
>>>>>>>>>>>>>Starting The parser should parse a class with generic args >>>>>>>>>>>>>>>>>>
(class Fred{type a, type b extends B}() class)
- should parse multiline comment at end of file *** FAILED ***
Expected 0, but got 1 (TestsBase.scala:103)
- should fail on incomplete multiline comment
- should parse single line comments
- should allow a class name to be repeated at the end
- should warn when the class name at the end does not match
- should allow an interface name to be repeated at the end
- should warn when the interface name at the end does not match
----The AST after parsing-------------
The lines starting with "- should" or "Expected" come from the runner and you can see that a bunch of them are plunked in the middle of the output from one of my tests. Other output from the runner appears elsewhere, this isn't all of it.
My questions: Why is this happening? Is there some way to get the runner's output to coordinate with my output?
Most likely, the suites are running in parallel.
http://www.scalatest.org/user_guide/using_the_runner#executingSuitesInParallel
With the proliferation of multi-core architectures, and the often
parallelizable nature of tests, it is useful to be able to run tests
in parallel. [...]
The -P option may optionally be appended with a number (e.g. "-P10" --
no intervening space) to specify the number of threads to be created
in the thread pool. If no number (or 0) is specified, the number of
threads will be decided based on the number of processors available.
So basically pass -P1 to the runner. For eclipse, the place would possibly be the Arguments tab.