With both DKPy-SITL and our APM2 board, the wait_ready method is causing our program to raise an API Exception due to the command list (waypoints) taking too long to download. In the past (with droneapi) this wasn't an issue for me. Some waypoints are being downloaded, but the process takes about 10 seconds for each one, which leads me to believe something weird is going on.
Are there any ways to speed up the download process? I've posted the relevant code below.
self.vehicle = connect(connection_string, baud=baud_rate,
status_printer=dronekit_printer, wait_ready=True)
and later in another asynchronous method
def commands(self):
commands = self.vehicle.commands
commands.download()
commands.wait_ready()
return commands
The error occurs on commands.wait_ready(). There has to be a faster way to download commands than sitting there for over 30 seconds on an i7 4790k processor, especially since I've run the same code off a slower computer in the past with droneapi. If need be, I can raise an issue on the dronekit github as well.
I had the same issue. First time download call always goes well (0 commands). Once you have uploaded some commands the second time you try to download it fails ('Timeout' exception).
What I did to solve this was calling clear without download after the first time.
Something like this:
cmds = vehicle.commands
if not cmds.count > 0:
# Download
cmds.download()
# Wait until download is finished
cmds.wait_ready()
cmds.clear()
# Add / Modify the commands here and then upload them
Related
Please excuse me for a long post.
I am a newbie in logging implementations and I'm trying to read a log file which gets overwritten every time the build is run. The log file contains details of some workflow execution steps.
Configurations for log4j2.xml file :-
<appenders>
<File name="InfoLog" fileName="build/var/debug.log" bufferedIO="true" immediateFlush="false" append="false">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS z}{UTC} ~ [%t] ~ %-5level ~ %logger{1} ~ %msg%n"/>
</File>
</appenders>
<loggers>
<Root level="Debug">
<AppenderRef ref="InfoLog"/>
</Root>
</loggers>
An example code snippet for my logger implementation :-
Note → I have api calls to other services and require those logs too
in my file, hence I used mu.KLogging library as it made it super
easy. I wasn’t able to get the logs from calls to other apis, using
java.util.logging or org.apache.log4j libraries.
import mu.KLogging
interface TestsLogger {
companion object {
val logger = KLogging().logger(" WORKFLOW STEPS ")
}
}
//In some class
logger.info { "Running ${(workflowMap.currentStep).toUpperCase()}" }
logger.error { workflowExecutionException }
Now, when I am trying to read the debug.log file (a few seconds after all steps have been executed), for some reason, not all logs are being read from the file.
TimeUnit.MILLISECONDS.sleep(5_000)
var lines: List<String> = Files.readAllLines(File("build/var/debug.log").toPath())
On diving deep into the cause of such behaviour, I found out that, when it tries to read the debug.log file, at that moment the log file does not have all the logs in it. For some reason, last few lines of the logs get appended to file at the moment, when my build gets complete. Although, the executions have already been happened for which I need information from the logs.
FYI -> I am using Gradle Build in Intellij
I am trying to understand why this strange behaviour? And how can I solve it such that logs are
generated correctly and my file reads all the logs generated till that moment ?
You have bufferedIo set to true and immediateFlush set to false. This is good for performance in long running apps that write logs frequently, but for an application that doesn't write much it means the log events will be sitting in a buffer waiting to be written. The buffers should be flushed at application shutdown, unless it is killed or has some other terminal error that doesn't allow it to terminate normally.
I'm running into a very peculiar and undocumented issue with a GAE Managed VM and Task Queues. I understand that the Managed VM service is in beta, so this question may not be relevant forever, but it's definitely causing me lots of headache now.
The main symptom of the issue is that, in certain (not completely known to me) circumstances, I'm seeing the following error/traceback:
File "/home/vmagent/my_app/some_file.py", line 265, in some_ndb_tasklet
res = yield some_task.add_async('some-task-queue-name')
File "/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py", line 472, in _on_rpc_completion
result = rpc.get_result()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/home/vmagent/python_vm_runtime/google/appengine/api/taskqueue/taskqueue.py", line 1948, in ResultHook
rpc.check_success()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
self.__rpc.CheckSuccess()
File "/home/vmagent/python_vm_runtime/google/appengine/ext/vmruntime/vmstub.py", line 312, in _WaitImpl
raise self._ErrorException(*_DEFAULT_EXCEPTION)
RPCFailedError: The remote RPC to the application server failed for call taskqueue.BulkAdd().
I've gone through my local App Engine SDK to trace this through, and I can get up to the last line of the trace, but google/appengine/ext/vmruntime/ doesn't exist on my machine at all, so I have no idea what's happening in vmstub.py. From looking at the local code, some_task.add_async('the-queue') is spinning up an RPC and waiting for it to finish, but this error is not what the except apiproxy_errors.ApplicationError, e: at line 1949 of taskqueue.py is expecting...
The code that's generating the error looks something like this:
#ndb.tasklet
def kickoff_tasks(batch_of_payloads):
for task_payload in batch_of_payloads:
# task_payload is a dict
task = taskqueue.Task(
url='/the/handler/url',
params=payload)
res = yield task.add_async('some-valid-task-queue-name')
Other things worth noting:
this code itself is running in a task handler kicked off by another task.
I first saw this error before implementing this sort of batching, and assumed the issue was because I had added too many tasks from within a task handler.
In some cases, I can run this successfully with a batch size of 100, but in others, it fails consistently (depending on the data in the payloads) at 100, and sometimes succeeds at batch sizes of 50.
The task payloads themselves include batches of items, and are tuned to be just small enough to fit in a task. App Engine advertises a maximum task size of 100KB, so I'm keeping the payloads to under 90,000 bytes right now. Lowering the size even more doesn't seem to help any.
I've also tried implementing an exponential backoff to retry the kickoff_tasks method when this error appears, but it seems that once the error is raised, I can't add any other tasks at all from within the same handler (i.e. I can't kickoff a "continue from where you left off" task, I just have to let this one fail and restart itself)
So, my question is, what is actually causing this error? How can I avoid it, or fix this so that I'm handling it correctly?
This is a known issue that is being worked on. There are actually two issues - the RPC failure itself and the lack of handling of the RPCFailedError exception by the SDK.
There is some public discussion of the issue here.
If you're using App Engine Flexible and the python-compat-multicore image, a new bug popped up related to App Engine using a newer version of the requests library that broke the communication between App Engine Flexible and the datastore. You can fix this error by monkey patching the library in your appengine_config.py file.
Add the following code to appengine_config.py:
try:
import appengine.ext.vmruntime.vmstub as vmstub
except ImportError:
pass
else:
if isinstance(vmstub.DEFAULT_TIMEOUT, (int, long)):
# Newer requests libraries do not accept integers as header values.
# Be sure to convert the header value before sending.
# See Support Case ID 11235929.
vmstub.DEFAULT_TIMEOUT = bytes(vmstub.DEFAULT_TIMEOUT)
Note that if you do not have an appengine_config.py file, you can just create it in your base project directory (wherever you put your app.yaml file). This file gets run during App Engine startup..
It appears that fork fails under Mac OS 10.6.8. The algorithm is coded in R and I have mainly be using the foreach package for parallel processing. The code works perfectly well when run sequentially (foreach and %do%) but not when run in parallel (foreach and %dopar%) even though the processes run in parallel do NOT communicate.
I used foreach on this same machine a month ago and it worked fine. An update of the OS has been performed in the meantime.
Error Messages
I received several kinds of error messages that seems to come almost stochastically. Also the errors differ depending on whether the code is run from the terminal (either by copy-pasting in the R environment or with R CMD BATCH) or from the R console.
When run on the Terminal, I get
Error in { : task 1 failed - "object 'dp' not found"
When run on the R console I get either
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
....
<repeated many times when run on the R console>
or
Error in { : task 1 failed - "object 'dp' not found"
with the exact same code! Note that although this second error message is the same than the one received on the Terminal, the number of things that are printed (through the print() function) on the screen vastly differ!
What I've tried
I updated the package foreach and I also restarting my computer but it did not quite help.
I tried to print pretty much anything I could but it ended up being quite hard to keep track of what this algorithm is doing. For example, it often through the error about the missing object dp without executing the print statement at the line that precedes the call of the object dp.
I tried to use %dopar% but registering only 1 CPU. The output did not change on the Terminal, but it changed on the Console. Now the console gives the exact same error, at the same time than the terminal.
I made sure that several CPUs were in used when I ask for several CPUs.
I tried to use mclapply instead of foreach and registerDoMC() instead of registerDoParallel() to register the number of cores.
Extra-Info
My version of R is 3.0.2 GUI 1.62 Snow Leopard build. My machine has 16 cores.
using this code example (dranger - ffmpeg):
https://github.com/arashafiei/dranger-ffmpeg-tuto/blob/master/tutorial03.c
and dranger tutorial for ffmpeg:
http://dranger.com/ffmpeg/tutorial02.html
The video runs as fast as possible but it makes sense because there is no timer and we just extract the frames as soon as we have them ready. But for some reason, the sound also runs as fast as possible even though he says that it shouldn't.
I'm using mac os x (Maybe that has something to do with it).
Any suggestions?
Try adding:
aCodecCtx = pFormatCtx->streams[audioStream]->codec;
> aCodecCtx->request_sample_fmt = AV_SAMPLE_FMT_S16;
i would like to measure time how a manually triggered update task takes and for that, I would like to access the system time. How can I do this?
Assuming that your target already has a working timer, and it gives resolution that suits you (millisecond ticks have become standard), it should be easy.
For a task that's manually triggered through u-boot COLI, there are commands that can be enabled which give time info. You wouldn't have to write any code. If these commands are not presently in your target's build, you would add to your board config file with #define.
From u-boot/README
CONFIG_CMD_TIME run command and report execution time
CONFIG_CMD_TIMER access to the system tick timer
See common/cmd_time.c and common/cmd_misc.c.
CONFIG_CMD_TIME example, if there is an update sequence defined in an environment variable:
MyTarget # time run myupdatesequence
...
time: 0.214 seconds, 214 ticks
MyTarget #
If the events of start/stop are only visible in running code, you would write code yourself to call get_timer() and output result to console. The referenced source code is pretty clear.