How can I create and execute a function/method while a script is running? - hy

A part of a script I'm making requires that new functions be defined and made available on during execution. The function defines OK, but I'm getting an error when I try to use it:
(import io)
(defn exec-code [str]
(eval (apply read [] {"from_file" (.StringIO io str)})))
(setv ectest "(print \"Code execution function works!\")")
(exec-code ectest)
(setv testfunc [])
(.append testfunc "(defn -testfunc- [] (print \"Self-execution works!\"))")
(.append testfunc "(-testfunc-)")
(for [line testfunc] (print line) (exec-code line))
Results in:
Code execution function works!
(defn -testfunc- [] (print "Self-execution works!"))
(-testfunc-)
Traceback (most recent call last):
File "/usr/bin/hy", line 9, in <module>
load_entry_point('hy==0.11.0', 'console_scripts', 'hy')()
File "/usr/lib/python3/dist-packages/hy/cmdline.py", line 347, in hy_main
sys.exit(cmdline_handler("hy", sys.argv))
File "/usr/lib/python3/dist-packages/hy/cmdline.py", line 335, in cmdline_handler
return run_file(options.args[0])
File "/usr/lib/python3/dist-packages/hy/cmdline.py", line 210, in run_file
import_file_to_module("__main__", filename)
File "/usr/lib/python3/dist-packages/hy/importer.py", line 78, in import_file_to_module
eval(ast_compile(_ast, fpath, "exec"), mod.__dict__)
File "code-trace.hy", line 41, in <module>
(for [line testfunc] (print line) (exec-code line))
File "code-trace.hy", line 33, in exec_code
(eval (apply read [] {"from_file" (.StringIO io str)})))
File "/usr/lib/python3/dist-packages/hy/importer.py", line 126, in hy_eval
return eval(ast_compile(expr, "<eval>", "eval"), namespace)
File "<eval>", line 1, in <module>
NameError: name '_testfunc_' is not defined

This looks like a namespace issue. eval works in an isolate namespace I think, so the second call knows nothing about the first one.
Try this:
(defn exec-code [str]
(eval (apply read [] {"from_file" (.StringIO io str)}) (globals)))
Which specifies the namespace to be globals. This outputs:
Code execution function works!
(defn -testfunc- [] (print "Self-execution works!"))
(-testfunc-)
Self-execution works!
for me.

Related

ODOO 12 server error regarding invoice sequencing

I am trying to change the sequence of my invoicing. Instead of resetting it each new year, I can keep the count going upwards continuously.
(for example)
inv/2021/0001 date 1/1/2023   (this one should be 2366)
inv/2021/2365    date 31/12/2022
researching on the subject I found out I need to go into technical -> sequences to get the invoice numbers I want.
but my problem is, once i click sequences I get the following server error:
Error:
Odoo Server Error
Traceback (most recent call last):
File "/odoo/odoo-server/odoo/api.py", line 1039, in get
value = self._data[key][field][record._ids[0]]
KeyError: 254
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/odoo/odoo-server/odoo/fields.py", line 981, in __get__
value = record.env.cache.get(record, self)
File "/odoo/odoo-server/odoo/api.py", line 1041, in get
raise CacheMiss(record, field)
odoo.exceptions.CacheMiss: ('ir.sequence(254,).number_next_actual', None)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/odoo/odoo-server/odoo/http.py", line 656, in _handle_exception
return super(JsonRequest, self)._handle_exception(exception)
File "/odoo/odoo-server/odoo/http.py", line 314, in _handle_exception
raise pycompat.reraise(type(exception), exception, sys.exc_info()[2])
File "/odoo/odoo-server/odoo/tools/pycompat.py", line 87, in reraise
raise value
File "/odoo/odoo-server/odoo/http.py", line 698, in dispatch
result = self._call_function(**self.params)
File "/odoo/odoo-server/odoo/http.py", line 346, in _call_function
return checked_call(self.db, *args, **kwargs)
File "/odoo/odoo-server/odoo/service/model.py", line 97, in wrapper
return f(dbname, *args, **kwargs)
File "/odoo/odoo-server/odoo/http.py", line 339, in checked_call
result = self.endpoint(*a, **kw)
File "/odoo/odoo-server/odoo/http.py", line 941, in __call__
return self.method(*args, **kw)
File "/odoo/odoo-server/odoo/http.py", line 519, in response_wrap
response = f(*args, **kw)
File "/odoo/odoo-server/addons/web/controllers/main.py", line 904, in search_read
return self.do_search_read(model, fields, offset, limit, domain, sort)
File "/odoo/odoo-server/addons/web/controllers/main.py", line 926, in do_search_read
offset=offset or 0, limit=limit or False, order=sort or False)
File "/odoo/odoo-server/odoo/models.py", line 4589, in search_read
result = records.read(fields)
File "/odoo/odoo-server/odoo/models.py", line 2791, in read
vals[name] = convert(record[name], record, use_name_get)
File "/odoo/odoo-server/odoo/models.py", line 5117, in __getitem__
return self._fields[key].__get__(self, type(self))
File "/odoo/odoo-server/odoo/fields.py", line 985, in __get__
self.determine_value(record)
File "/odoo/odoo-server/odoo/fields.py", line 1098, in determine_value
self.compute_value(recs)
File "/odoo/odoo-server/odoo/fields.py", line 1052, in compute_value
self._compute_value(records)
File "/odoo/odoo-server/odoo/fields.py", line 1043, in _compute_value
getattr(records, self.compute)()
File "/odoo/odoo-server/odoo/addons/base/models/ir_sequence.py", line 96, in _get_number_next_actual
seq.number_next_actual = _predict_nextval(self, seq_id)
File "/odoo/odoo-server/odoo/addons/base/models/ir_sequence.py", line 68, in _predict_nextval
self.env.cr.execute(query % {'seq_id': seq_id})
File "/odoo/odoo-server/odoo/sql_db.py", line 148, in wrapper
return f(self, *args, **kwargs)
File "/odoo/odoo-server/odoo/sql_db.py", line 225, in execute
res = self._obj.execute(query, params)
psycopg2.ProgrammingError: relation "ir_sequence_1000015" does not exist
LINE 6: FROM ir_sequence_1000015
I believe it could be a database error but I am not sure what this is about. Any idea?
Thanks!

mmrotate eval_map.py IndexError: tuple index out of range

When I trained the /mmrotate/configs/rotated_retinanet/rotated_retinanet_obb_r50_fpn_6x_hrsc_rr_le90.py with fine_grained, I got IndexError: tuple index out of range.
After trained this baseline without evaluation(--no-validate), it can execute successfully.
This is the link to the baseline from GitHub.(https://github.com/open-mmlab/mmrotate/blob/main/configs/rotated_retinanet/rotated_retinanet_obb_r50_fpn_6x_hrsc_rr_le90.py)
The following is the traceback. Could you tell me how to solve it? I just want to evaluate the result of baseline with fine_grained on HRSC2016 dataset. Thank you very much.
Traceback (most recent call last):
File "tools/train.py", line 192, in <module>
main()
File "tools/train.py", line 181, in main
train_detector(
File "/root/autodl-tmp/mmrotate/mmrotate/apis/train.py", line 141, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 136, in run
epoch_runner(data_loaders[i], **kwargs)
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 58, in train
self.call_hook('after_train_epoch')
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 317, in call_hook
getattr(hook, fn_name)(self)
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmcv/runner/hooks/evaluation.py", line 271, in after_train_epoch
self._do_evaluate(runner)
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmdet/core/evaluation/eval_hooks.py", line 63, in _do_evaluate
key_score = self.evaluate(runner, results)
File "/root/miniconda3/envs/mmrotate/lib/python3.8/site-packages/mmcv/runner/hooks/evaluation.py", line 367, in evaluate
eval_res = self.dataloader.dataset.evaluate(
File "/root/autodl-tmp/mmrotate/mmrotate/datasets/hrsc.py", line 251, in evaluate
mean_ap, _ = eval_rbbox_map(
File "/root/autodl-tmp/mmrotate/mmrotate/core/evaluation/eval_map.py", line 243, in eval_rbbox_map
print_map_summary(
File "/root/autodl-tmp/mmrotate/mmrotate/core/evaluation/eval_map.py", line 305, in print_map_summary
label_names[j], num_gts[i, j], results[j]['num_dets'],
IndexError: tuple index out of range
By the way, I checked the num_classes=33 and the classwise=True.
To evaluate the https://github.com/open-mmlab/mmrotate/blob/main/configs/rotated_retinanet/rotated_retinanet_obb_r50_fpn_6x_hrsc_rr_le90.py with fine_grained on HRSC2016 dataset.

How to create a Tensorflow Dataset without labels? Input 'filename' of 'ReadFile' Op has type float32 that does not match expected type of string

Using Tensorflow 2.3, I'm trying to create a tf.data.Dataset without labels.
I have my .png files in a folder './Folder/'. For creating the minimal working sample, I think the only relevant line is the one where I am calling tf.keras.preprocessing.image_dataset_from_directory. The class definition is here.
dataset = tf.keras.preprocessing.image_dataset_from_directory('./Folder/',label_mode=None,batch_size=100)
When the Python interpreter reaches the line above, it returns this error message:
Traceback (most recent call last):
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 465, in _apply_op_helper
values = ops.convert_to_tensor(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1473, in convert_to_tensor
raise ValueError(
ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: <tf.Tensor 'args_0:0' shape=() dtype=float32>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "04-vaeAnomalyScores.py", line 135, in <module>
historicKLD, encoder, decoder, vae = artVAE_Instance.run_autoencoder() # Train
File "/media/roi/9b168630-3b62-4215-bb7d-fed9ba179dc7/images/largePatches/artvae.py", line 386, in run_autoencoder
trainingDataSet = self.loadImages(self.trainingDir)
File "/media/roi/9b168630-3b62-4215-bb7d-fed9ba179dc7/images/largePatches/artvae.py", line 231, in loadImages
dataset = tf.keras.preprocessing.image_dataset_from_directory(dir[:-1]+'Downscaled/',label_mode=None,batch_size=self.BATCH_SIZE)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/keras/preprocessing/image_dataset.py", line 192, in image_dataset_from_directory
dataset = paths_and_labels_to_dataset(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/keras/preprocessing/image_dataset.py", line 219, in paths_and_labels_to_dataset
img_ds = path_ds.map(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1695, in map
return MapDataset(self, map_func, preserve_cardinality=True)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 4041, in __init__
self._map_func = StructuredFunctionWrapper(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3371, in __init__
self._function = wrapper_fn.get_concrete_function()
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2938, in get_concrete_function
graph_function = self._get_concrete_function_garbage_collected(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2906, in _get_concrete_function_garbage_collected
graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3213, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3065, in _create_graph_function
func_graph_module.func_graph_from_py_func(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 986, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3364, in wrapper_fn
ret = _wrapper_helper(*args)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3299, in _wrapper_helper
ret = autograph.tf_convert(func, ag_ctx)(*nested_args)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 255, in wrapper
return converted_call(f, args, kwargs, options=options)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 532, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 339, in _call_unconverted
return f(*args, **kwargs)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/keras/preprocessing/image_dataset.py", line 220, in <lambda>
lambda x: path_to_image(x, image_size, num_channels, interpolation))
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/keras/preprocessing/image_dataset.py", line 228, in path_to_image
img = io_ops.read_file(path)
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/ops/gen_io_ops.py", line 574, in read_file
_, _, _op, _outputs = _op_def_library._apply_op_helper(
File "/home/roi/.local/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 492, in _apply_op_helper
raise TypeError("%s expected type of %s." %
TypeError: Input 'filename' of 'ReadFile' Op has type float32 that does not match expected type of string.
Thank you so much for your help.
One way to fix this I found is to put all your images in another sub-directory inside the directory whose path you are feeding to the image_dataset_from_directory.
Taking your example, you would create a new folder, let's call it new_folder, inside of ./Folder/ where you would put all your images, such that now the path to all your images is ./Folder/new_folder/. Then you can call the image_dataset_from_directory method with the exact same arguments as you have done in your question:
tf.keras.preprocessing.image_dataset_from_directory(
'./Folder/',
label_mode=None,
batch_size=100
)
I found this to work for me so hopefully someone else will also find it helpful!

'numpy.ndarray' object is not callable?

Running this script:
import time
import picamera
import picamera.array
import numpy as np
with picamera.PiCamera() as camera:
with picamera.array.PiBayerArray(camera) as stream:
camera.capture(stream, 'jpeg', bayer=True)
# Demosaic data and write to output (just use stream.array if you
# want to skip the demosaic step)
output = (stream.array() >> 2).astype(np.uint8)
with open('image.jpg', 'wb') as f:
output.tofile(f)
Gives the following error:
Traceback (most recent call last):
File "numpy_simple.py", line 11, in <module>
output = (stream.array() >> 2).astype(np.uint8)
TypeError: 'numpy.ndarray' object is not callable
While running:
output = (stream.demosaic() >> 2).astype(np.uint8)
with open('image.data', 'wb') as f:
output.tofile(f)
Does not give any error.
I'm a bit confused.
array is an attribute, not a method. You don't need to call it.
Use stream.array, not stream.array().
Source: PiArrayOutput, which is the base class for PiBayerArray.
Conversely, .demosaic() is an instance method, which is why you need to call it to get its return value.

In Google App Engine, how to check input validity of Key created by urlsafe?

Suppose I create a key from user input websafe url
key = ndb.Key(urlsafe=some_user_input)
How can I check if the some_user_input is valid?
My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this, and point me some better way for user input validity checking instead of catching the exception?
Thanks a lot!
If you try to construct a Key with an invalid urlsafe parameter
key = ndb.Key(urlsafe='bogus123')
you will get an error like
Traceback (most recent call last):
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/tim/git/project/main.py", line 10, in <module>
from src.tim import handlers as handlers_
File "/home/tim/git/project/src/tim/handlers.py", line 42, in <module>
class ResetHandler(BaseHandler):
File "/home/tim/git/project/src/tim/handlers.py", line 47, in ResetHandler
key = ndb.Key(urlsafe='bogus123')
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 212, in __new__
self.__reference = _ConstructReference(cls, **kwargs)
File "/opt/google/google_appengine/google/appengine/ext/ndb/utils.py", line 142, in positional_wrapper
return wrapped(*args, **kwds)
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 642, in _ConstructReference
reference = _ReferenceFromSerialized(serialized)
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 773, in _ReferenceFromSerialized
return entity_pb.Reference(serialized)
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1710, in __init__
if contents is not None: self.MergeFromString(contents)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 152, in MergeFromString
self.MergePartialFromString(s)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 168, in MergePartialFromString
self.TryMerge(d)
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1839, in TryMerge
d.skipData(tt)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 677, in skipData
raise ProtocolBufferDecodeError, "corrupted"
ProtocolBufferDecodeError: corrupted
Interesting here are is
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 773, in _ReferenceFromSerialized
return entity_pb.Reference(serialized)
which is the last code executed in the key.py module:
def _ReferenceFromSerialized(serialized):
"""Construct a Reference from a serialized Reference."""
if not isinstance(serialized, basestring):
raise TypeError('serialized must be a string; received %r' % serialized)
elif isinstance(serialized, unicode):
serialized = serialized.encode('utf8')
return entity_pb.Reference(serialized)
serialized here being the decoded urlsafe string, you can read more about it in the link to the source code.
another interesting one is the last one:
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1839, in TryMerge
in the entity_pb.py module which looks like this
def TryMerge(self, d):
while d.avail() > 0:
tt = d.getVarInt32()
if tt == 106:
self.set_app(d.getPrefixedString())
continue
if tt == 114:
length = d.getVarInt32()
tmp = ProtocolBuffer.Decoder(d.buffer(), d.pos(), d.pos() + length)
d.skip(length)
self.mutable_path().TryMerge(tmp)
continue
if tt == 162:
self.set_name_space(d.getPrefixedString())
continue
if (tt == 0): raise ProtocolBuffer.ProtocolBufferDecodeError
d.skipData(tt)
which is where the actual attempt to 'merge the input to into a Key' is made.
You can see in the source code that during the process of constructing a Key from an urlsafe parameter not a whole lot can go wrong. First it checks if the input is a string and if it's not, a TypeError is raised, if it is but it's not 'valid', indeed a ProtocolBufferDecodeError is raised.
My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this
Sort of confirmed - we now know that also TypeError can be raised.
and point me some better way for user input validity checking instead of catching the exception?
This is an excellent way to check validity! Why do the checks yourself if the they are already done by appengine? A code snippet could look like this (not working code, just an example)
def get(self):
# first, fetch the user_input from somewhere
try:
key = ndb.Key(urlsafe=user_input)
except TypeError:
return 'Sorry, only string is allowed as urlsafe input'
except ProtocolBufferDecodeError:
return 'Sorry, the urlsafe string seems to be invalid'

Resources