For my volttron agent that I used the agent creation wizard to develop, can I get a tip on an error related to this , 'Timezone offset does not match system offset: -18000 != 0. Please, check your config files.'
When testing my script with the from volttron.platform.scheduling import cron feature I noticed the timezone/computer time was way off on my edge device so I reset the time zone with this tutorial which I am thinking definitely screwed things up.
ERROR: volttron.platform.jsonrpc.RemoteError: builtins.ValueError('Timezone offset does not match system offset: -18000 != 0. Please, check your config files.')
Whether or not this makes a difference this edge device does use the fowarding agent to push the data to a central VOLTTRON instance.
2021-05-14 12:45:00,007 (actuatoragent-1.0 313466) volttron.platform.vip.agent.subsystems.rpc ERROR: unhandled exception in JSON-RPC method 'request_new_schedule':
Traceback (most recent call last):
File "/var/lib/volttron/volttron/platform/vip/agent/subsystems/rpc.py", line 158, in method
return method(*args, **kwargs)
File "/home/volttron/.volttron/agents/8f4ee1c0-74cb-4070-8a8c-57bf9bea8a71/actuatoragent-1.0/actuator/agent.py", line 1343, in request_new_schedule
return self._request_new_schedule(rpc_peer, task_id, priority, requests, publish_result=False)
File "/home/volttron/.volttron/agents/8f4ee1c0-74cb-4070-8a8c-57bf9bea8a71/actuatoragent-1.0/actuator/agent.py", line 1351, in _request_new_schedule
local_tz = get_localzone()
File "/var/lib/volttron/env/lib/python3.8/site-packages/tzlocal/unix.py", line 165, in get_localzone
_cache_tz = _get_localzone()
File "/var/lib/volttron/env/lib/python3.8/site-packages/tzlocal/unix.py", line 90, in _get_localzone
utils.assert_tz_offset(tz)
File "/var/lib/volttron/env/lib/python3.8/site-packages/tzlocal/utils.py", line 46, in assert_tz_offset
raise ValueError(msg)
ValueError: Timezone offset does not match system offset: -18000 != 0. Please, check your config files.
This is my raise_setpoints_up function below that is alot like the CSV driver agent code.
def raise_setpoints_up(self):
_log.info(f'*** [Setter Agent INFO] *** - STARTING raise_setpoints_up function!')
schedule_request = []
# create start and end timestamps
_now = get_aware_utc_now()
str_start = format_timestamp(_now)
_end = _now + td(seconds=10)
str_end = format_timestamp(_end)
# wrap the topic and timestamps up in a list and add it to the schedules list
for device in self.jci_device_map.values():
topic = '/'.join([self.building_topic, device])
schedule_request.append([topic, str_start, str_end])
# send the request to the actuator
result = self.vip.rpc.call('platform.actuator', 'request_new_schedule', self.core.identity, 'my_schedule', 'HIGH', schedule_request).get(timeout=4)
_log.info(f'*** [Setter Agent INFO] *** - actuator agent scheduled sucess!')
Thanks for any tips
I suspect the time configured by tzdata is different than the timezone configured by the system since you changed this manually. Give this a try:
sudo dpkg-reconfigure tzdata
use sudo dpkg-reconfigure tzdata to change back to UTC time
Related
I have a BACnet system for HVAC controls where I am using the VOLTTRON actuator agent to write # priority 10 in BACnet to a value of 2 which works good.
result = self.vip.rpc.call('platform.actuator', 'set_multiple_points', self.core.identity, set_multi_topic_values_master).get(timeout=20)
_log.debug(f'*** [Setter Agent INFO] *** - set_multiple_points ON ALL VAVs WRITE SUCCESS!')
Then the system sleeps for some time period for testing purposes:
_log.debug(f'*** [Setter Agent INFO] *** - SETTING UP GEVENT SLEEP!')
gevent.sleep(120)
_log.debug(f'*** [Setter Agent INFO] *** - GEVENT SLEEP DONE!')
Where after the gevent sleep I am running into some issues on the revert point not working. The code below executes just fine but using a BACnet scanning tool the priority 10 value of 2 are still present on the HVAC controls, like the revert point isn't doing anything.
for device in revert_topic_devices_jci:
response = self.vip.rpc.call('platform.actuator', 'revert_point', self.core.identity, topic_jci, self.jci_setpoint_topic).get(timeout=20)
_log.debug(f'*** [Setter Agent INFO] *** - REVERT POINTS ON {device} SUCCESS!')
_log.debug(f'*** [Setter Agent INFO] *** - REVERT POINTS JCI DONE DEAL SUCCESS!')
One thing I notice is the building automation writes occupancy/unoccupancy to the HVAC controls # BACnet priority 12. Its either ALWAYS a 1 for occupancy or a 2 for unoccupancy.
What I am trying to do with VOLTTRON is write in BACnet at priority 10 a value of 2, and then release to nothing on the revert. Could this by the revert isnt doing anything because there was nothing to revert too? I was hoping that VOLTTRON could write # BACnet priority 10 and then just release. On BACnet scan tool I can do the same thing write # priority 10 then release priority 10 with a priority 10 write null
Should I just be writing at priority 12 same as the building automation system so VOLTTRON can just revert back too whatever the building automation was doing?
I have a few observations:
In your revert loop, the third code-block above, you're not actually
changing the topic being passed to the RPC call. Each call will use
the device topic which is not in that code-block (but that we can
see is not being changed inside the block) and a device topic, which
similarly is not defined in the block but at least appears not to be
being changed. It is likely worth setting some breakpoints and/or
debug statements here to be sure that you're passing the correct
topics to revert on.
Your use of priority appears to be consistent
with BACnet protocol specification, and with the VOLTTRON BACnet
driver implementation. We would not recommend that you attempt to
write at the same priority as an existing building automation
system.
The BACnet driver code will send a NULL (None) value in a "writeProperty"
service request when the "revert_point" function is called by the
Platform Driver. This functionality I am frankly not terribly
familiar with, but given that your scan tool performs the expected
revert functionality when passed a NULL value, I suspect this is the expected
way of performing a "revert to previous value" type function in BACnet protocol.
I do not have reason to believe that the behavior you're experiencing is the
result of a bug in the driver code base.
Overall, I suggest debugging the topics being passed in the "revert_point" RPC call.
I am having a good luck to revert point using set_multiple_points to None
Something like this:
self.jci_device_map = {
'VMA-2-6': '27',
'VMA-2-4': '29',
'VMA-2-7': '30',
'VMA-1-8': '6',
}
revert_multi_topic_values_master = []
set_multi_topic_values_master = []
for device in self.jci_device_map.values():
topic_jci = '/'.join([self.building_topic, device])
final_topic_jci = '/'.join([topic_jci, self.jci_setpoint_topic])
# BACnet enum point for VAV occ
# 1 == occ, 2 == unnoc
# create a (topic, value) tuple and add it to our topic values
set_multi_topic_values_master.append((final_topic_jci, self.unnoccupied_value)) # TO SET UNNOCUPIED
revert_multi_topic_values_master.append((final_topic_jci, None)) # TO SET FOR REVERT
result = self.vip.rpc.call('platform.actuator', 'set_multiple_points', self.core.identity, revert_multi_topic_values_master).get(timeout=20)
I'm getting started with Kapacitor and have been trying to run the first guide in the Kapacitor documentation, but with data I already have. I managed to define a task, but I can neither enable it nor can I run a backfill. I came across this question, which is similar to my problem, but the answer there didn't help. In contrast to the error message there I get empty strings for database, retention policy, and/or measurement.
In Kapacitor config I set an InfluxDB connection to the local host instance with the name localhost (which has a database mydb and the measurements weather.current.clouds and weather.current.visibility with default retention policy autogen) and created the following weathertest.tick script:
dbrp "mydb"."autogen"
var clouds = batch
|query('select mean(value) / 100.0 as val from "mydb"."autogen"."weather.current.clouds"')
.period(1h)
.every(1h)
.groupBy(time(1m), *)
.fill(0)
var vis = batch
|query('select mean(value) / 10000.0 as val from "mydb"."autogen"."weather.current.visibility"')
.period(1h)
.every(1h)
.groupBy(time(1m), *)
.fill(0)
clouds
|join(vis)
.as('c', 'v')
|eval(lambda: 100 * (1 - "c.val") * "v.val")
.as('pcent')
|influxDBOut()
.cluster('localhost')
.database('mydb')
.retentionPolicy('autogen')
.measurement('testmetric')
.tag('host', 'myhost.local')
.tag('key', 'weather.current.lightidx')
This is what I came up with after hours of trial and (especially) error. As given in the title, when I try to enable my task with kapacitor enable weathertest, I get the error message enabling task weathertest: batch query is not allowed to request data from ""."". Same thing when I try to record as in the "Backfill" example. Also, in that example there is a start and a stop date for limiting the time frame. The time format given there is wrong and is not understood by Kapacitor. Instead of e. g. 2015-10-01 I have to put in 2015-10-01T00:00Z to make it at least pass the error message regarding time format error.
In the Kapacitor logs there is not a single line regarding these errors, only when I try to remove a record, I get something like remove /var/lib/kapacitor/replay/1f5...750.brpl: no such file or directory and this can be found in the logs. There are lots of info lines in the logs showing successful POSTs to/from InfluxDB for the _internal database with HTTP response result 204.
Has anyone an Idea what I may be doing wrong?
OK, after the weekend I tried again. Without any change it accepted my script now in the failing steps, however, now I was able to find error messages in the log. The node mentioned there was the eval node and pointed towards a type mismatch. When I changed the line
|eval(lambda: 100 * (1 - "c.val") * "v.val")
to
|eval(lambda: 100.0 * (1.0 - "c.val") * "v.val")
the error messages were gone and the command kapacitor show weathertest showed a rather sane content now.
Furthermore, I redefined, recorded, replayed and deleted the tasks and recordings during my tests over and over again and I may have forgotten to redefine tasks after making changes to the tick script (not really sure). After changing the above, redefining the task and replaying it I finally found the expected data in the InfluxDB instance.
Running migration in a docker kiwitcms/kiwi:latest (v7.2 digest d757b56bc10c) image based docker container.
During migration testcases.0010_remove_bug fails with some database constraint problem.
Is this a bug in the migration script or an issue with data not consistent?
Operations to perform:
Apply all migrations: admin, attachments, auth, bugs, contenttypes, core, django_comments, kiwi_auth, linkreference, management, sessions, sites, testcases, testplans, testruns
Running migrations:
Applying testcases.0010_remove_bug...Traceback (most recent call last):
File "/venv/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/venv/lib/python3.6/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1048, "Column 'execution_id' cannot be null")
This is a bug in the migration script. The problem lies in the fact that prior to these changes you could attach bugs directly to test cases, without an actual test execution. This resulted in the case_run_id field being None.
With the new changes bugs can only be attached to test executions and their execution_id field should always have a value.
Can you try this patch and let me know if it works for you:
https://github.com/kiwitcms/Kiwi/pull/1266
I'm using the blobstore to backup and recovery entities in csv format. The process is working well for all of my smaller models. However, once I start to work on models with more than 2K entities, I am exceeded the soft memory limit. I'm only fetching 50 entities at a time and then writing the results out to the blobstore, so I'm not clear why my memory usage would be building up. I can reliably make the method fail just by increasing the "limit" value passed in below which results in the method running just a little longer to export a few more entities.
Any recommendations on how to optimize this process to reduce memory consumption?
Also, the files produced will only <500KB in size. Why would the process use 140 MB of memory?
Simplified example:
file_name = files.blobstore.create(mime_type='application/octet-stream')
with files.open(file_name, 'a') as f:
writer = csv.DictWriter(f, fieldnames=properties)
for entity in models.Player.all():
row = backup.get_dict_for_entity(entity)
writer.writerow(row)
Produces the error:
Exceeded soft private memory limit with 150.957 MB after servicing 7 requests total
Simplified example 2:
The problem seems to be with using files and the with statement in python 2.5. Factoring out the csv stuff, I can reproduce almost the same error by simply trying to write a 4000 line text file to the blobstore.
from __future__ import with_statement
from google.appengine.api import files
from google.appengine.ext.blobstore import blobstore
file_name = files.blobstore.create(mime_type='application/octet-stream')
myBuffer = StringIO.StringIO()
#Put 4000 lines of text in myBuffer
with files.open(file_name, 'a') as f:
for line in myBuffer.getvalue().splitlies():
f.write(line)
files.finalize(file_name)
blob_key = files.blobstore.get_blob_key(file_name)
Produces the error:
Exceeded soft private memory limit with 154.977 MB after servicing 24 requests total
Original:
def backup_model_to_blobstore(model, limit=None, batch_size=None):
file_name = files.blobstore.create(mime_type='application/octet-stream')
# Open the file and write to it
with files.open(file_name, 'a') as f:
#Get the fieldnames for the csv file.
query = model.all().fetch(1)
entity = query[0]
properties = entity.__class__.properties()
#Add ID as a property
properties['ID'] = entity.key().id()
#For debugging rather than try and catch
if True:
writer = csv.DictWriter(f, fieldnames=properties)
#Write out a header row
headers = dict( (n,n) for n in properties )
writer.writerow(headers)
numBatches = int(limit/batch_size)
if numBatches == 0:
numBatches = 1
for x in range(numBatches):
logging.info("************** querying with offset %s and limit %s", x*batch_size, batch_size)
query = model.all().fetch(limit=batch_size, offset=x*batch_size)
for entity in query:
#This just returns a small dictionary with the key-value pairs
row = get_dict_for_entity(entity)
#write out a row for each entity.
writer.writerow(row)
# Finalize the file. Do this before attempting to read it.
files.finalize(file_name)
blob_key = files.blobstore.get_blob_key(file_name)
return blob_key
The error looks like this in the logs
......
2012-02-02 21:59:19.063
************** querying with offset 2050 and limit 50
I 2012-02-02 21:59:20.076
************** querying with offset 2100 and limit 50
I 2012-02-02 21:59:20.781
************** querying with offset 2150 and limit 50
I 2012-02-02 21:59:21.508
Exception for: Chris (202.161.57.167)
err:
Traceback (most recent call last):
.....
blob_key = backup_model_to_blobstore(model, limit=limit, batch_size=batch_size)
File "/base/data/home/apps/singpath/163.356548765202135434/singpath/backup.py", line 125, in backup_model_to_blobstore
writer.writerow(row)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 281, in __exit__
self.close()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 275, in close
self._make_rpc_call_with_retry('Close', request, response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 388, in _make_rpc_call_with_retry
_make_call(method, request, response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 236, in _make_call
_raise_app_error(e)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 179, in _raise_app_error
raise FileNotOpenedError()
FileNotOpenedError
C 2012-02-02 21:59:23.009
Exceeded soft private memory limit with 149.426 MB after servicing 14 requests total
You'd be better off not doing the batching yourself, but just iterating over the query. The iterator will pick a batch size (probably 20) that should be adequate:
q = model.all()
for entity in q:
row = get_dict_for_entity(entity)
writer.writerow(row)
This avoids re-running the query with ever-increasing offset, which is slow and causes quadratic behavior in the datastore.
An oft-overlooked fact about memory usage is that the in-memory representation of an entity can use 30-50 times the RAM compared to the serialized form of the entity; e.g. an entity that is 3KB on disk might use 100KB in RAM. (The exact blow-up factor depends on many factors; it's worse if you have lots of properties with long names and small values, even worse for repeated properties with long names.)
In What is the proper way to write to the Google App Engine blobstore as a file in Python 2.5 a similar problem was reported. In an answer there it is suggested that you should try inserting gc.collect() calls occasionally. Given what I know of the files API's implementation I think that is spot on. Give it a try!
I can't speak for the memory use in Python, but considering your error message, the error most likely stems from the fact that a blobstore backed file in GAE can't be open for more than around 30 seconds so you have to close and reopen it periodically if your processing takes longer.
It can possibly be a Time Exceed error, due to the limitation of the request to 30 secs. In my implementation in order to bypass it instead of having a webapp handler for the operation I am firing an event in the default queue. The cool thing about the queue is that it takes one line of code to invoke it, it has a 10 min time limit and if a task fails it retries before the time limit. I am not really sure if it will solve your problem but it worths giving a try.
from google.appengine.api import taskqueue
...
taskqueue.add("the url that invokes your method")
you can find more info about the queues here.
Or consider using a backend for serious computations and file operations.
This is a common exception I'm getting on my application's log daily, usually 5/6 times a day with a traffic of 1K visits/day:
db error trying to store stats
Traceback (most recent call last):
File "/base/data/home/apps/stackprinter/1b.347728306076327132/app/utility/worker.py", line 36, in deferred_store_print_statistics
dbcounter.increment()
File "/base/data/home/apps/stackprinter/1b.347728306076327132/app/db/counter.py", line 28, in increment
db.run_in_transaction(txn)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 1981, in RunInTransaction
DEFAULT_TRANSACTION_RETRIES, function, *args, **kwargs)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 2067, in RunInTransactionCustomRetries
ok, result = _DoOneTry(new_connection, function, args, kwargs)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 2105, in _DoOneTry
if new_connection.commit():
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1585, in commit
return rpc.get_result()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 530, in get_result
return self.__get_result_hook(self)
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1613, in __commit_hook
raise _ToDatastoreError(err)
Timeout: The datastore operation timed out, or the data was temporarily unavailable.
The function that is raising the exception above is the following one:
def store_printed_question(question_id, service, title):
def _store_TX():
entity = Question.get_by_key_name(key_names = '%s_%s' % \
(question_id, service ) )
if entity:
entity.counter = entity.counter + 1
entity.put()
else:
Question(key_name = '%s_%s' % (question_id, service ),\
question_id ,\
service,\
title,\
counter = 1).put()
db.run_in_transaction(_store_TX)
Basically, the store_printed_question function check if a given question was previously printed, incrementing in that case the related counter in a single transaction.
This function is added by a WebHandler to a deferred worker using the predefined default queue that, as you might know, has a throughput rate of five task invocations per second.
On a entity with six attributes (two indexes) I thought that using transactions regulated by a deferred task rate limit would allow me to avoid datastore timeouts but, looking at the log, this error is still showing up daily.
This counter I'm storing is not so much important, so I'm not worried about getting these timeouts; anyway I'm curious why Google App Engine can't handle this task properly even at a low rate like 5 tasks per second and if lowering the rate could be a possible solution.
A sharded counter on each question to avoid timeouts would be an overkill to me.
EDIT:
I have set the rate limit to 1 task per second on the default queue; I'm still getting the same error.
A query can only live for 30 seconds. See my answer to this question for some sample code to break a query up using cursors.
Generally speaking, a timeout like this is usually because of write contention. If you've got a transaction going and you're writing a bunch of stuff to the same entity group concurrently, you run into write contention issues (a side effect of optimistic concurrency). In most cases, if you make your entity group smaller, that will usually minimize this problem.
In your specific case, based on the code above, it's most probably because you should be using a sharded counter to avoid stacking of serialized writes.
Another far less likely possibility (mentioned here only for completeness) is that the tablet your data is on is being moved.