Running this script:
import time
import picamera
import picamera.array
import numpy as np
with picamera.PiCamera() as camera:
with picamera.array.PiBayerArray(camera) as stream:
camera.capture(stream, 'jpeg', bayer=True)
# Demosaic data and write to output (just use stream.array if you
# want to skip the demosaic step)
output = (stream.array() >> 2).astype(np.uint8)
with open('image.jpg', 'wb') as f:
output.tofile(f)
Gives the following error:
Traceback (most recent call last):
File "numpy_simple.py", line 11, in <module>
output = (stream.array() >> 2).astype(np.uint8)
TypeError: 'numpy.ndarray' object is not callable
While running:
output = (stream.demosaic() >> 2).astype(np.uint8)
with open('image.data', 'wb') as f:
output.tofile(f)
Does not give any error.
I'm a bit confused.
array is an attribute, not a method. You don't need to call it.
Use stream.array, not stream.array().
Source: PiArrayOutput, which is the base class for PiBayerArray.
Conversely, .demosaic() is an instance method, which is why you need to call it to get its return value.
Related
Given the following iterable:
arcpy.da.SearchCursor (fc, ['SHAPE #', 'OID #'])
I am trying to get a numpy array using the fromiter method (I think it is faster):
dt = [('G',arcpy.Geometry),('ID','<i4')]
arr = np.fromiter(arcpy.da.SearchCursor(fc, ['SHAPE #', 'OID #']),dtype=dt)
However I get the following error:
Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: cannot create object arrays from iterator
I have tried without luck also with a generator:
dt = [('G',arcpy.Geometry),('ID','<i4')]
arr = np.fromiter(i for i in arcpy.da.SearchCursor(fc,['SHAPE#','OID#'])),dtype=dt)
But I get the same error:
Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: cannot create object arrays from iterator
And:
dt = [('G','O'),('ID','<i4')]
arr = np.fromiter(arcpy.da.SearchCursor(fc, ['SHAPE #', 'OID #']),dtype=dt)
The only thing that works for me is to create the list and pass it to the array constructor.
I would like to know why I am getting this error and how I could fix it so that I can use fromiter
Others have reported a similar error, but the solutions given do not solve my problem.
For example there is a good answer here. The answer in the link mentions how ndb changes from a first use to a later use and suggests there is a problem because a first run produces a None in the Datastore. I cannot reproduce or see that happening in the Datastore for my sdk, but that may be because I am running it here from the interactive console.
I am pretty sure I got an initial good run with the GAE interactive console, but every run since then has failed with the error in my Title to this question.
I have left the print statements in the following code because they show good results and assure me that the error is occuring in the put() at the very end.
from google.appengine.ext import ndb
class Account(ndb.Model):
week = ndb.IntegerProperty(repeated=True)
weeksNS = ndb.IntegerProperty(repeated=True)
weeksEW = ndb.IntegerProperty(repeated=True)
terry=Account(week=[],weeksNS=[],weeksEW=[])
terry_key=terry.put()
terry = terry_key.get()
print terry
for t in list(range(4)): #just dummy input, but like real input
terry.week.append(t)
print terry.week
region = 1 #same error message for region = 0
if region :
terry.weeksEW.append(terry.week)
else:
terry.weeksNS.append(terry.week)
print 'EW'+str(terry.weeksEW)
print 'NS'+str(terry.weeksNS)
terry.week = []
print 'week'+str(terry.week)
terry.put()
The idea of my code is to first build up the terry.week list values incrementally and then later store the whole list to the appropriate region, either NS or EW. So I'm looking for a workaround for this scheme.
The error message is likely of no value but I am reproducing it here.
Traceback (most recent call last):
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/request_handler.py", line 237, in handle_interactive_request
exec(compiled_code, self._command_globals)
File "<string>", line 55, in <module>
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3458, in _put
return self._put_async(**ctx_options).get_result()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
self.check_success()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/context.py", line 824, in put
key = yield self._put_batcher.add(entity, options)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 430, in _help_tasklet_along
value = gen.send(val)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1858, in async_put
pbs = [entity_to_pb(entity) for entity in entities]
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 697, in entity_to_pb
pb = ent._to_pb()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3167, in _to_pb
prop._serialize(self, pb, projection=self._projection)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1422, in _serialize
values = self._get_base_value_unwrapped_as_list(entity)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1192, in _get_base_value_unwrapped_as_list
wrapped = self._get_base_value(entity)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1180, in _get_base_value
return self._apply_to_values(entity, self._opt_call_to_base_type)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1352, in _apply_to_values
value[:] = map(function, value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1234, in _opt_call_to_base_type
value = _BaseValue(self._call_to_base_type(value))
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1255, in _call_to_base_type
return call(value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1331, in call
newvalue = method(self, value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1781, in _validate
(value,))
BadValueError: Expected integer, got [0, 1, 2, 3]
I believe the error comes from these lines:
terry.weeksEW.append(terry.week)
terry.weeksNS.append(terry.week)
You are not appending another integer; You are appending a list, when an integer is expected.
>>> aaa = [1,2,3]
>>> bbb = [4,5,6]
>>> aaa.append(bbb)
>>> aaa
[1, 2, 3, [4, 5, 6]]
>>>
This fails the ndb.IntegerProperty test.
Try:
terry.weeksEW += terry.week
terry.weeksNS += terry.week
EDIT: To save a list of lists, do not use the IntegerProperty(), but instead the JsonProperty(). Better still, the ndb datastore is deprecated, so... I recommend Firestore, which uses JSON objects by default. At least use Cloud Datastore, or Cloud NDB.
I'm adapting a script.py to achieve transfer learning. I find many script to retrain a model by TFRecord files, but none of them worked for me bacause of something about TF2.0 and contrib, so I'm trying to convert a script to adapt to TF2 and to my model.
This is my script at the moment:
from __future__ import absolute_import, division, print_function, unicode_literals
import os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
keras = tf.keras
EPOCHS = 1
# Data preprocessing
import pathlib
#data_dir = tf.keras.utils.get_file(origin="/home/pi/venv/raccoon_dataset/", fname="raccoons_dataset")
#data_dir = pathlib.Path(data_dir)
data_dir = "/home/pi/.keras/datasets/ssd_mobilenet_v1_coco_2018_01_28/saved_model/saved_model.pb"
######################
# Read the TFRecords #
######################
def imgs_input_fn(filenames, perform_shuffle=False, repeat_count=1, batch_size=1):
def _parse_function(serialized):
features = \
{
'image': tf.io.FixedLenFeature([], tf.string),
'label': tf.io.FixedLenFeature([], tf.int64)
}
# Parse the serialized data so we get a dict with our data.
parsed_example = tf.io.parse_single_example(serialized=serialized,
features=features)
print("\nParsed example:\n", parsed_example, "\nEnd of parsed example:\n")
# Get the image as raw bytes.
image_shape = tf.stack([300, 300, 3])
image_raw = parsed_example['image']
label = tf.cast(parsed_example['label'], tf.float32)
# Decode the raw bytes so it becomes a tensor with type.
image = tf.io.decode_raw(image_raw, tf.uint8)
image = tf.cast(image, tf.float32)
image = tf.reshape(image, image_shape)
#image = tf.subtract(image, 116.779) # Zero-center by mean pixel
#image = tf.reverse(image, axis=[2]) # 'RGB'->'BGR'
d = dict(zip(["image"], [image])), [label]
return d
dataset = tf.data.TFRecordDataset(filenames=filenames)
# Parse the serialized data in the TFRecords files.
# This returns TensorFlow tensors for the image and labels.
#print("\nDataset before parsing:\n",dataset,"\n")
dataset = dataset.map(_parse_function)
#print("\nDataset after parsing:\n",dataset,"\n")
if perform_shuffle:
# Randomizes input using a window of 256 elements (read into memory)
dataset = dataset.shuffle(buffer_size=256)
dataset = dataset.repeat(repeat_count) # Repeats dataset this # times
dataset = dataset.batch(batch_size) # Batch size to use
print("\nDataset batched:\n", dataset, "\nEnd dataset\n")
iterator = tf.compat.v1.data.make_one_shot_iterator(dataset)
print("\nIterator shape:\n", tf.compat.v1.data.get_output_shapes(iterator),"\nEnd\n")
#print("\nIterator:\n",iterator.get_next(),"\nEnd Iterator\n")
batch_features, batch_labels = iterator.get_next()
return batch_features, batch_labels
raw_train = tf.compat.v1.estimator.TrainSpec(input_fn=imgs_input_fn(
"/home/pi/venv/raccoon_dataset/data/train.record",
perform_shuffle=True,
repeat_count=5,
batch_size=20),
max_steps=1)
and this is the resulting screen:
Parsed example:
{'image': <tf.Tensor 'ParseSingleExample/ParseSingleExample:0' shape=() dtype=string>, 'label': <tf.Tensor 'ParseSingleExample/ParseSingleExample:1' shape=() dtype=int64>}
End of parsed example:
Dataset batched:
<BatchDataset shapes: ({image: (None, 300, 300, 3)}, (None, 1)), types: ({image: tf.float32}, tf.float32)>
End dataset
Iterator shape:
({'image': TensorShape([None, 300, 300, 3])}, TensorShape([None, 1]))
End
2019-11-20 14:01:14.493817: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Feature: image (data type: string) is required but could not be found.
2019-11-20 14:01:14.495019: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at iterator_ops.cc:929 : Invalid argument: {{function_node __inference_Dataset_map__parse_function_27}} Feature: image (data type: string) is required but could not be found.
[[{{node ParseSingleExample/ParseSingleExample}}]]
Traceback (most recent call last):
File "transfer_learning.py", line 127, in <module>
batch_size=20),
File "transfer_learning.py", line 107, in imgs_input_fn
batch_features, batch_labels = iterator.get_next()
File "/home/pi/venv/lib/python3.7/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 737, in get_next
return self._next_internal()
File "/home/pi/venv/lib/python3.7/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 651, in _next_internal
output_shapes=self._flat_output_shapes)
File "/home/pi/venv/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_dataset_ops.py", line 2673, in iterator_get_next_sync
_six.raise_from(_core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __inference_Dataset_map__parse_function_27}} Feature: image (data type: string) is required but could not be found.
[[{{node ParseSingleExample/ParseSingleExample}}]] [Op:IteratorGetNextSync]
I don't know what I'm doing wrong.
It seems like I can't do this:
user=csv_reader[0 + row_count]
This code is for a university project and I'm running it on repl.it. If you want to take a look, it's posted here: https://repl.it/#Lia_AlexaAlexa/ConsciousYummySeahorse
import csv
def comprobando_usuario(usuario_var):
csv_reader = open("ale.csv")
row_count = len(csv_reader.readlines())
while row_count >= 1:
user=csv_reader[0 + row_count]
useri=user[1]
while usuario_var in useri:
usuario_var=str(("Ingrese nuevo usuario o escribe no para
terminar."))
row_count=row_count - 1
if usuario_var in abc:
return(0)
return(100)
Error:
Traceback (most recent call last):
File "main.py", line 14, in <module>
respuesta_de_usuario= usuario.comprobando_usuario(usuario_var)
File "/home/runner/usuario.py", line 6, in comprobando_usuario
user=csv_reader[0 + row_count]
TypeError: '_io.TextIOWrapper' object is not subscriptable
Indeed, you can't do user=csv_reader[0 + row_count] as csv_reader is a file object that you created from csv_reader = open("ale.csv").
Also, you should get rid of this complex while loop and follow the doc to know how to read a file properly in Python: https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files
I'm trying to get an object from my neo4j database using neo4django
>>> # There is a single Person object in the database, so I get a value
>>> slug=Person.objects.get().name_slug
>>> print(slug)
doe-john
>>> # ok, it's there
>>> p=Person.objects.get(name_slug=slug)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/[...]/src/neo4django/neo4django/db/models/manager.py", line 37, in get
return self.get_query_set().get(*args, **kwargs)
File "/[...]/lib/python2.7/site-packages/django/db/models/query.py", line 366, in get
% self.model._meta.object_name)
DoesNotExist: Person matching query does not exist.
>>> p=Person.objects.get(name_slug__exact=slug)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/[...]/src/neo4django/neo4django/db/models/manager.py", line 37, in get
return self.get_query_set().get(*args, **kwargs)
File "/[...]/lib/python2.7/site-packages/django/db/models/query.py", line 366, in get
% self.model._meta.object_name)
DoesNotExist: Person matching query does not exist.
The error message is not sensible. I just received the queried string from the very field, so there must be a match. Any ideas? Or did I stumble upon a bug?
This is really strange, as it works with the other properties, but not with name_slug:
>>> Person.objects.get(surname='Doe')
<Person: Person object>
>>> Person.objects.get(given_name='John')
<Person: Person object>
>>> Person.objects.get(name_slug='john-doe')
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/[...]/src/neo4django/neo4django/db/models/manager.py", line 37, in get
return self.get_query_set().get(*args, **kwargs)
File "/[...]/lib/python2.7/site-packages/django/db/models/query.py", line 366, in get
% self.model._meta.object_name)
DoesNotExist: Person matching query does not exist.
>>> print( p.surname, p.given_name, p.name_slug )
(u'Doe', u'John', u'john-doe')
My model is defined as such:
class Person(models.NodeModel):
surname = models.StringProperty(required=True, indexed=True)
given_name = models.StringProperty(required=True, indexed=True)
name_slug = models.StringProperty(indexed=True)
So the only difference is that it's not required, but that should make no difference, in my understanding of the documentation.
I can't replicate this using neo4django master, on Neo4j 1.9.
I created a test_models.py
from neo4django.db import models
class Person(models.NodeModel):
class Meta:
# since test_models isn't in an app
app_label='test'
surname = models.StringProperty(required=True, indexed=True)
given_name = models.StringProperty(required=True, indexed=True)
name_slug = models.StringProperty(indexed=True)
and then ran
>>> from test_models import Person
>>> john = Person.objects.create(surname=u'Doe', given_name=u'John', name_slug=u'john-doe')
>>> Person.objects.get(name_slug='john-doe')
<Person: Person object>
>>> john == Person.objects.get(name_slug='john-doe')
True
>>> jane = Person.objects.create(surname=u'Doe', given_name=u'Jane', name_slug=u'jane-doe')
>>> jane == Person.objects.get(name_slug='jane-doe')
True
>>> jane == Person.objects.get(given_name='Jane', surname='Doe')
True
Thoughts?