I want to ensure that my Class' staticmethod is called with the correct arguments without actually calling it, therefore I am mocking it. E.g.:
import unittest
from unittest.mock import patch
class FooStatic:
#staticmethod
def bar_static(self, baz_static):
print(baz_static)
pass
class TestFooStatic(unittest.TestCase):
def test_foo_static(self):
with patch.object(FooStatic, 'bar_static', autospec=True):
FooStatic.bar_static()
def test_foo_static_instance(self):
with patch.object(FooStatic, 'bar_static', autospec=True):
foo_s = FooStatic()
foo_s.bar_static()
Both these tests should complain that FooStatic.bar_static cannot be called without argument 'baz_static'. Unfortunately they don't, the tests succeed.
Without the staticmethod decorator, patch behaves as I expect:
class Foo:
def bar(self, baz):
print(baz)
pass
class TestFoo(unittest.TestCase):
def test_foo(self):
with patch.object(Foo, 'bar', autospec=True):
foo = Foo()
foo.bar() # raises TypeError: missing a required argument: 'baz'
I have found a loosely related issue in python, that was fixed from python 3.7 onwards: merged PR.
I am on python 3.8.5 (default on ubunutu 20).
I am not opposed to investing some time to try and propose a fix myself. However, I first want to make sure I am not overlooking anything. Any thoughts?
Related
I have a php project that has composer dependencies which are inherently tested in the code path of my unit tests. Here's my sample code:
<?php
// where FooBar is a composer package but I'm purposely typing it incorrectly here
use \fooBaR
public function appendNameToWords(array $words, $name)
{
$start = microtime(true);
$newWords = array_map(function($word){
return $word . $name;
}, $words);
// logs the diff between start and end time
FooBar::logTimer($start);
return $newWords;
}
My test is simply testing the method but of course executes the line FooBar::logTimer in my source code. The problem is I'm expecting my test to fail if I mistype the class FooBar to be fooBaR. Unfortunately, the travis build is passing...but i'm unclear why.
.travis.yml file:
language: php
php:
- 5.6
install: script/install
script:
- script/test
Any ideas on what could be wrong?
PHP is not case sensitive when it comes to class names. If your code declares a class named Foo, and this definition is executed, you can also instantiate any other case style, like foo or fOO.
PHP will preserve the case of the occurrence that triggers the autoload (i.e. the first time PHP encounters a class name), and if that case style does not match a case sensitive file name, using the class will fail.
I consider writing the class in the correct case style a problem that should not be tested with a unit test. It's a problem that cannot be solved in your own code - and it is basically not existing if you use a powerful IDE that knows all classes that can be used.
Additionally: Your question does not provide code that demonstrates the problem. And it contains code that probably does not do what you think it does.
I have a google-cloud-endpoints, in the docs, I did'nt find how to write a PATCH method.
My request:
curl -XPATCH localhost:8080/_ah/api/hellogreeting/1 -d '{"message": "Hi"}'
My method handler looks like this:
from models import Greeting
from messages import GreetingMessage
#endpoints.method(ID_RESOURCE, Greeting,`
path='hellogreeting/{id}', http_method='PATCH',
name='greetings.patch')
def greetings_patch(self, request):
request.message, request.username
greeting = Greeting.get_by_id(request.id)
greeting.message = request.message # It's ok, cuz message exists in request
greeting.username = request.username # request.username is None. Writing the IF conditions in each string(checking on empty), I think it not beatifully.
greeting.put()
return GreetingMessage(message=greeting.message, username=greeting.username)
So, now in Greeting.username field will be None. And it's wrong.
Writing the IF conditions in each string(checking on empty), I think it not beatifully.
So, what is the best way for model updating partially?
I do not think there is one in Cloud Endpoints, but you can code yours easily like the example below.
You will need to decide how you want your patch to behave, in particular when it comes to attributes that are objects : should you also apply the patch on the object attribute (in which case use recursion) or should you just replace the original object attribute with the new one like in my example.
def apply_patch(origin, patch):
for name in dir( patch ):
if not name.startswith( '__' ):
setattr(origin,name,getattr(patch,name))
class Dummy(TestCase):
def setUp(self):
thing = Thing.objects.create(name="Thing")
def test_a(self):
self.assertTrue(Thing.objects.get(pk=1))
def test_b(self):
self.assertTrue(Thing.objects.get(pk=1))
In this example I expect for setUp to be run prior to every test case, but it is only run prior to the first and then the changes are rolled back. This causes test_a to pass, but the equivalent test_b to fail. Is this the expected behavior? What do I need to do to make sure that the database is in the same state prior to every test case?
Figured it out. setUp is being run each time, it's just that it's incrementing the private key in the database. Therefore the Thing with pk=1 no longer exists. This works just fine:
class Dummy_YepThatsMe(TestCase):
def setUp(self):
thing = Thing.objects.create(name="Thing")
def test_a(self):
self.assertTrue(Thing.objects.get(name="Thing"))
def test_b(self):
self.assertTrue(Thing.objects.get(name="Thing"))
Currently my application caches models in memcache like this:
memcache.set("somekey", aModel)
But Nicks' post at http://blog.notdot.net/2009/9/Efficient-model-memcaching suggests that first converting it to protobuffers is a lot more efficient. But after running some tests I found out it's indeed smaller in size, but actually slower (~10%).
Do others have the same experience or am I doing something wrong?
Test results: http://1.latest.sofatest.appspot.com/?times=1000
import pickle
import time
import uuid
from google.appengine.ext import webapp
from google.appengine.ext import db
from google.appengine.ext.webapp import util
from google.appengine.datastore import entity_pb
from google.appengine.api import memcache
class Person(db.Model):
name = db.StringProperty()
times = 10000
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
m = Person(name='Koen Bok')
t1 = time.time()
for i in xrange(int(self.request.get('times', 1))):
key = uuid.uuid4().hex
memcache.set(key, m)
r = memcache.get(key)
self.response.out.write('Pickle took: %.2f' % (time.time() - t1))
t1 = time.time()
for i in xrange(int(self.request.get('times', 1))):
key = uuid.uuid4().hex
memcache.set(key, db.model_to_protobuf(m).Encode())
r = db.model_from_protobuf(entity_pb.EntityProto(memcache.get(key)))
self.response.out.write('Proto took: %.2f' % (time.time() - t1))
def main():
application = webapp.WSGIApplication([('/', MainHandler)], debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
The Memcache call still pickles the object with or without using protobuf. Pickle is faster with a protobuf object since it has a very simple model
Plain pickle objects are larger than protobuf+pickle objects, hence they save time on Memcache, but there is more processor time in doing the protobuf conversion
Therefore in general either method works out about the same...but
The reason you should use protobuf is it can handle changes between versions of the models, whereas Pickle will error. This problem will bite you one day, so best to handle it sooner
Both pickle and protobufs are slow in App Engine since they're implemented in pure Python. I've found that writing my own, simple serialization code using methods like str.join tends to be faster since most of the work is done in C. But that only works for simple datatypes.
One way to do it more quickly is to turn your model into a dictionary and use the native eval / repr function as your (de)serializers -- with caution of course, as always with the evil eval, but it should be safe here given that there is no external step.
Below an example of a class Fake_entity implementing exactly that.
You first create your dictionary through fake = Fake_entity(entity) then you can simply store your data via memcache.set(key, fake.serialize()). The serialize() is a simple call to the native dictionary method of repr, with some additions if you need (e.g. add an identifier at the beginning of the string).
To fetch it back, simply use fake = Fake_entity(memcache.get(key)). The Fake_entity object is a simple dictionary whose keys are also accessible as attributes. You can access your entity properties normally, except referenceProperties give keys instead of fetching the object (which is actually quite useful). You can also get() the actual entity with fake.get(), or more interestigly, change it and then save with fake.put().
It does not work with lists (if you fetch multiple entities from a query), but could be easily be adjusted with join/split functions using an identifier like '### FAKE MODEL ENTITY ###' as the separator. Use with db.Model only, would need small adjustments for Expando.
class Fake_entity(dict):
def __init__(self, record):
# simple case: a string, we eval it to rebuild our fake entity
if isinstance(record, basestring):
import datetime # <----- put all relevant eval imports here
from google.appengine.api import datastore_types
self.update( eval(record) ) # careful with external sources, eval is evil
return None
# serious case: we build the instance from the actual entity
for prop_name, prop_ref in record.__class__.properties().items():
self[prop_name] = prop_ref.get_value_for_datastore(record) # to avoid fetching entities
self['_cls'] = record.__class__.__module__ + '.' + record.__class__.__name__
try:
self['key'] = str(record.key())
except Exception: # the key may not exist if the entity has not been stored
pass
def __getattr__(self, k):
return self[k]
def __setattr__(self, k, v):
self[k] = v
def key(self):
from google.appengine.ext import db
return db.Key(self['key'])
def get(self):
from google.appengine.ext import db
return db.get(self['key'])
def put(self):
_cls = self.pop('_cls') # gets and removes the class name form the passed arguments
# import xxxxxxx ---> put your model imports here if necessary
Cls = eval(_cls) # make sure that your models declarations are in the scope here
real_entity = Cls(**self) # creates the entity
real_entity.put() # self explanatory
self['_cls'] = _cls # puts back the class name afterwards
return real_entity
def serialize(self):
return '### FAKE MODEL ENTITY ###\n' + repr(self)
# or simply repr, but I use the initial identifier to test and eval directly when getting from memcache
I would welcome speed tests on this, I would assume this is quite faster than the other approaches. Plus, you do not have any risks if your models have changed somehow in the meantime.
Below an example of what the serialized fake entity looks like. Take a particular look at datetime (created) as well as reference properties (subdomain) :
### FAKE MODEL ENTITY ###
{'status': u'admin', 'session_expiry': None, 'first_name': u'Louis', 'last_name': u'Le Sieur', 'modified_by': None, 'password_hash': u'a9993e364706816aba3e25717000000000000000', 'language': u'fr', 'created': datetime.datetime(2010, 7, 18, 21, 50, 11, 750000), 'modified': None, 'created_by': None, 'email': u'chou#glou.bou', 'key': 'agdqZXJlZ2xlcgwLEgVMb2dpbhjmAQw', 'session_ref': None, '_cls': 'models.Login', 'groups': [], 'email___password_hash': u'chou#glou.bou+a9993e364706816aba3e25717000000000000000', 'subdomain': datastore_types.Key.from_path(u'Subdomain', 229L, _app=u'jeregle'), 'permitted': [], 'permissions': []}
Personally I also use static variables (faster than memcache) to cache my entities in the short term, and fetch the datastore when the server has changed or its memory has been flushed for some reason (which happens quite often in fact).
I want to be able to take a dynamically created string, say "Pigeon" and determine at runtime whether Google App Engine has a Model class defined in this project named "Pigeon". If "Pigeon" is the name of a existant model class, I would like to then get a reference to the Pigeon class so defined.
Also, I don't want to use eval at all, since the dynamic string "Pigeon" in this case, comes from outside.
You could try, although probably very, very bad practice:
def get_class_instance(nm) :
try :
return eval(nm+'()')
except :
return None
Also, to make that safer, you could give eval a locals hash: eval(nm+'()', {'Pigeon':pigeon})
I'm not sure if that would work, and it definitely has an issue: if there is a function called the value of nm, it would return that:
def Pigeon() :
return "Pigeon"
print(get_class_instance('Pigeon')) # >> 'Pigeon'
EDIT: Another way of doing it is possibly (untested), if you know the module:
(Sorry, I keep forgetting it's not obj.hasattr, its hasattr(obj)!)
import models as m
def get_class_instance(nm) :
if hasattr(m, nm) :
return getattr(m, nm)()
else : return None
EDIT 2: Yes, it does work! Woo!
Actually, looking through the source code and interweb, I found a undocumented method that seems to fit the bill.
from google.appengine.ext import db
key = "ModelObject" #This is a dynamically generated string
klass = db.class_for_kind(key)
This method will throw a descriptive exception if the class does not exist, so you should probably catch it if the key string comes from the outside.
There's two fairly easy ways to do this without relying on internal details:
Use the google.appengine.api.datastore API, like so:
from google.appengine.api import datastore
q = datastore.Query('EntityType')
if q.get(1):
print "EntityType exists!"
The other option is to use the db.Expando class:
def GetEntityClass(entity_type):
class Entity(db.Expando):
#classmethod
def kind(cls):
return entity_type
return Entity
cls = GetEntityClass('EntityType')
if cls.all().get():
print "EntityType exists!"
The latter has the advantage that you can use GetEntityClass to generate an Expando class for any entity type, and interact with it the same way you would a normal class.