It seems like I can't do this:
user=csv_reader[0 + row_count]
This code is for a university project and I'm running it on repl.it. If you want to take a look, it's posted here: https://repl.it/#Lia_AlexaAlexa/ConsciousYummySeahorse
import csv
def comprobando_usuario(usuario_var):
csv_reader = open("ale.csv")
row_count = len(csv_reader.readlines())
while row_count >= 1:
user=csv_reader[0 + row_count]
useri=user[1]
while usuario_var in useri:
usuario_var=str(("Ingrese nuevo usuario o escribe no para
terminar."))
row_count=row_count - 1
if usuario_var in abc:
return(0)
return(100)
Error:
Traceback (most recent call last):
File "main.py", line 14, in <module>
respuesta_de_usuario= usuario.comprobando_usuario(usuario_var)
File "/home/runner/usuario.py", line 6, in comprobando_usuario
user=csv_reader[0 + row_count]
TypeError: '_io.TextIOWrapper' object is not subscriptable
Indeed, you can't do user=csv_reader[0 + row_count] as csv_reader is a file object that you created from csv_reader = open("ale.csv").
Also, you should get rid of this complex while loop and follow the doc to know how to read a file properly in Python: https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files
Related
Given the following iterable:
arcpy.da.SearchCursor (fc, ['SHAPE #', 'OID #'])
I am trying to get a numpy array using the fromiter method (I think it is faster):
dt = [('G',arcpy.Geometry),('ID','<i4')]
arr = np.fromiter(arcpy.da.SearchCursor(fc, ['SHAPE #', 'OID #']),dtype=dt)
However I get the following error:
Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: cannot create object arrays from iterator
I have tried without luck also with a generator:
dt = [('G',arcpy.Geometry),('ID','<i4')]
arr = np.fromiter(i for i in arcpy.da.SearchCursor(fc,['SHAPE#','OID#'])),dtype=dt)
But I get the same error:
Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: cannot create object arrays from iterator
And:
dt = [('G','O'),('ID','<i4')]
arr = np.fromiter(arcpy.da.SearchCursor(fc, ['SHAPE #', 'OID #']),dtype=dt)
The only thing that works for me is to create the list and pass it to the array constructor.
I would like to know why I am getting this error and how I could fix it so that I can use fromiter
Others have reported a similar error, but the solutions given do not solve my problem.
For example there is a good answer here. The answer in the link mentions how ndb changes from a first use to a later use and suggests there is a problem because a first run produces a None in the Datastore. I cannot reproduce or see that happening in the Datastore for my sdk, but that may be because I am running it here from the interactive console.
I am pretty sure I got an initial good run with the GAE interactive console, but every run since then has failed with the error in my Title to this question.
I have left the print statements in the following code because they show good results and assure me that the error is occuring in the put() at the very end.
from google.appengine.ext import ndb
class Account(ndb.Model):
week = ndb.IntegerProperty(repeated=True)
weeksNS = ndb.IntegerProperty(repeated=True)
weeksEW = ndb.IntegerProperty(repeated=True)
terry=Account(week=[],weeksNS=[],weeksEW=[])
terry_key=terry.put()
terry = terry_key.get()
print terry
for t in list(range(4)): #just dummy input, but like real input
terry.week.append(t)
print terry.week
region = 1 #same error message for region = 0
if region :
terry.weeksEW.append(terry.week)
else:
terry.weeksNS.append(terry.week)
print 'EW'+str(terry.weeksEW)
print 'NS'+str(terry.weeksNS)
terry.week = []
print 'week'+str(terry.week)
terry.put()
The idea of my code is to first build up the terry.week list values incrementally and then later store the whole list to the appropriate region, either NS or EW. So I'm looking for a workaround for this scheme.
The error message is likely of no value but I am reproducing it here.
Traceback (most recent call last):
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/request_handler.py", line 237, in handle_interactive_request
exec(compiled_code, self._command_globals)
File "<string>", line 55, in <module>
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3458, in _put
return self._put_async(**ctx_options).get_result()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
self.check_success()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/context.py", line 824, in put
key = yield self._put_batcher.add(entity, options)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/tasklets.py", line 430, in _help_tasklet_along
value = gen.send(val)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1858, in async_put
pbs = [entity_to_pb(entity) for entity in entities]
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 697, in entity_to_pb
pb = ent._to_pb()
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3167, in _to_pb
prop._serialize(self, pb, projection=self._projection)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1422, in _serialize
values = self._get_base_value_unwrapped_as_list(entity)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1192, in _get_base_value_unwrapped_as_list
wrapped = self._get_base_value(entity)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1180, in _get_base_value
return self._apply_to_values(entity, self._opt_call_to_base_type)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1352, in _apply_to_values
value[:] = map(function, value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1234, in _opt_call_to_base_type
value = _BaseValue(self._call_to_base_type(value))
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1255, in _call_to_base_type
return call(value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1331, in call
newvalue = method(self, value)
File "/Users/brian/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 1781, in _validate
(value,))
BadValueError: Expected integer, got [0, 1, 2, 3]
I believe the error comes from these lines:
terry.weeksEW.append(terry.week)
terry.weeksNS.append(terry.week)
You are not appending another integer; You are appending a list, when an integer is expected.
>>> aaa = [1,2,3]
>>> bbb = [4,5,6]
>>> aaa.append(bbb)
>>> aaa
[1, 2, 3, [4, 5, 6]]
>>>
This fails the ndb.IntegerProperty test.
Try:
terry.weeksEW += terry.week
terry.weeksNS += terry.week
EDIT: To save a list of lists, do not use the IntegerProperty(), but instead the JsonProperty(). Better still, the ndb datastore is deprecated, so... I recommend Firestore, which uses JSON objects by default. At least use Cloud Datastore, or Cloud NDB.
Running this script:
import time
import picamera
import picamera.array
import numpy as np
with picamera.PiCamera() as camera:
with picamera.array.PiBayerArray(camera) as stream:
camera.capture(stream, 'jpeg', bayer=True)
# Demosaic data and write to output (just use stream.array if you
# want to skip the demosaic step)
output = (stream.array() >> 2).astype(np.uint8)
with open('image.jpg', 'wb') as f:
output.tofile(f)
Gives the following error:
Traceback (most recent call last):
File "numpy_simple.py", line 11, in <module>
output = (stream.array() >> 2).astype(np.uint8)
TypeError: 'numpy.ndarray' object is not callable
While running:
output = (stream.demosaic() >> 2).astype(np.uint8)
with open('image.data', 'wb') as f:
output.tofile(f)
Does not give any error.
I'm a bit confused.
array is an attribute, not a method. You don't need to call it.
Use stream.array, not stream.array().
Source: PiArrayOutput, which is the base class for PiBayerArray.
Conversely, .demosaic() is an instance method, which is why you need to call it to get its return value.
Migrated column type from HSTORE to JSONB and am using this snippet of code...
from sqlalchemy.dialects.postgresql import ARRAY, JSONB
if employment_type:
base = base.filter(Candidate.bio["employment_type"].cast(ARRAY).contains(employment_type))
and am getting this error...
127.0.0.1 - - [28/Mar/2016 12:25:13] "GET /candidate_filter/?employment_type_3=true HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Library/Python/2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/surajkapoor/Desktop/lhv-talenttracker/app/views.py", line 660, in investor_filter
base = base.filter(Candidate.bio["employment_type"].cast(ARRAY).contains(employment_type))
File "/Library/Python/2.7/site-packages/sqlalchemy/dialects/postgresql/json.py", line 93, in cast
return self.astext.cast(type_)
File "/Library/Python/2.7/site-packages/sqlalchemy/dialects/postgresql/json.py", line 95, in cast
return sql.cast(self, type_)
File "<string>", line 2, in cast
File "/Library/Python/2.7/site-packages/sqlalchemy/sql/elements.py", line 2314, in __init__
self.type = type_api.to_instance(type_)
File "/Library/Python/2.7/site-packages/sqlalchemy/sql/type_api.py", line 1142, in to_instance
return typeobj(*arg, **kw)
TypeError: __init__() takes at least 2 arguments (1 given)
Candidate.bio["employment_type"] is an array of integers and I'm simply trying to query all the rows that contain a specific integer in them.
Also, .cast() works perfectly on the same column when calling Integer...
if internship:
base = base.filter(Candidate.bio["internship"].cast(Integer) == 1)
SqlAlchemy is probably having difficulty constructing the where clause because it can't figure out what type bio->'employment_type' is.
If the contains method is called from a String object, it would generate a LIKE clause, but for JSONB or ARRAY it would need to generate the #> operator.
To give SqlAlchemy the necessary hints, use explicit casting everywhere, i.e. write your query like
from sqlalchemy import cast
if employment_type:
casted_field = Candidate.bio['employment_type'].cast(JSONB)
casted_values = cast(employment_type, JSONB)
stmt = base.filter(casted_field.contains(casted_values))
In my example, I have a JSONB column named bio with the following data:
{"employment_type": [1, 2, 3]}
Edit: Casting to JSONB works:
>>> from sqlalchemy.dialects.postgresql import JSONB
>>> employment_type = 2
>>> query = (
... session.query(Candidate)
... .filter(Candidate.bio['employment_type'].cast(JSONB).contains(employment_type)))
>>> query.one().bio
{"employment_type": [1, 2, 3]}
Original answer:
I couldn't get .contains to work on Candidate.bio['employment_type'], but we can do the equivalent of the following SQL:
SELECT * FROM candidate WHERE candidate.bio #> '{"employment_type": [2]}';
like this:
>>> employment_type = 2
>>> test = {'employment_type': [employment_type]}
>>> query = (
... session.query(Candidate)
... .filter(Candidate.bio.contains(test)))
>>> query.one().bio
{"employment_type": [1, 2, 3]}
Suppose I create a key from user input websafe url
key = ndb.Key(urlsafe=some_user_input)
How can I check if the some_user_input is valid?
My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this, and point me some better way for user input validity checking instead of catching the exception?
Thanks a lot!
If you try to construct a Key with an invalid urlsafe parameter
key = ndb.Key(urlsafe='bogus123')
you will get an error like
Traceback (most recent call last):
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/opt/google/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/tim/git/project/main.py", line 10, in <module>
from src.tim import handlers as handlers_
File "/home/tim/git/project/src/tim/handlers.py", line 42, in <module>
class ResetHandler(BaseHandler):
File "/home/tim/git/project/src/tim/handlers.py", line 47, in ResetHandler
key = ndb.Key(urlsafe='bogus123')
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 212, in __new__
self.__reference = _ConstructReference(cls, **kwargs)
File "/opt/google/google_appengine/google/appengine/ext/ndb/utils.py", line 142, in positional_wrapper
return wrapped(*args, **kwds)
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 642, in _ConstructReference
reference = _ReferenceFromSerialized(serialized)
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 773, in _ReferenceFromSerialized
return entity_pb.Reference(serialized)
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1710, in __init__
if contents is not None: self.MergeFromString(contents)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 152, in MergeFromString
self.MergePartialFromString(s)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 168, in MergePartialFromString
self.TryMerge(d)
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1839, in TryMerge
d.skipData(tt)
File "/opt/google/google_appengine/google/net/proto/ProtocolBuffer.py", line 677, in skipData
raise ProtocolBufferDecodeError, "corrupted"
ProtocolBufferDecodeError: corrupted
Interesting here are is
File "/opt/google/google_appengine/google/appengine/ext/ndb/key.py", line 773, in _ReferenceFromSerialized
return entity_pb.Reference(serialized)
which is the last code executed in the key.py module:
def _ReferenceFromSerialized(serialized):
"""Construct a Reference from a serialized Reference."""
if not isinstance(serialized, basestring):
raise TypeError('serialized must be a string; received %r' % serialized)
elif isinstance(serialized, unicode):
serialized = serialized.encode('utf8')
return entity_pb.Reference(serialized)
serialized here being the decoded urlsafe string, you can read more about it in the link to the source code.
another interesting one is the last one:
File "/opt/google/google_appengine/google/appengine/datastore/entity_pb.py", line 1839, in TryMerge
in the entity_pb.py module which looks like this
def TryMerge(self, d):
while d.avail() > 0:
tt = d.getVarInt32()
if tt == 106:
self.set_app(d.getPrefixedString())
continue
if tt == 114:
length = d.getVarInt32()
tmp = ProtocolBuffer.Decoder(d.buffer(), d.pos(), d.pos() + length)
d.skip(length)
self.mutable_path().TryMerge(tmp)
continue
if tt == 162:
self.set_name_space(d.getPrefixedString())
continue
if (tt == 0): raise ProtocolBuffer.ProtocolBufferDecodeError
d.skipData(tt)
which is where the actual attempt to 'merge the input to into a Key' is made.
You can see in the source code that during the process of constructing a Key from an urlsafe parameter not a whole lot can go wrong. First it checks if the input is a string and if it's not, a TypeError is raised, if it is but it's not 'valid', indeed a ProtocolBufferDecodeError is raised.
My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this
Sort of confirmed - we now know that also TypeError can be raised.
and point me some better way for user input validity checking instead of catching the exception?
This is an excellent way to check validity! Why do the checks yourself if the they are already done by appengine? A code snippet could look like this (not working code, just an example)
def get(self):
# first, fetch the user_input from somewhere
try:
key = ndb.Key(urlsafe=user_input)
except TypeError:
return 'Sorry, only string is allowed as urlsafe input'
except ProtocolBufferDecodeError:
return 'Sorry, the urlsafe string seems to be invalid'