Why is the entity I create in setUp() not accessible via nosetests? - google-app-engine

Question

I still can't seem to get Nosetest to run properly.
The dev_appserver runs fine when started from the command line, but when I attempt to start it from the functional_tests.py it fails.
I create a Client entity in the setUp(), but it's not accessable from the test.
How do I step into pdb debugger with dev_appserver starting up in the test?
I have tried to put pdb() breakpoints in the code. While the code stops execution, I can't step into the debugger. I'm not even sure how to see the output.
$ nosetests
INFO 2015-02-24 19:08:56,172 devappserver2.py:726] Skipping SDK update check.
INFO 2015-02-24 19:08:56,242 api_server.py:172] Starting API server at: http://localhost:62049
INFO 2015-02-24 19:08:56,247 dispatcher.py:186] Starting module "default" running at: http://localhost:8080
INFO 2015-02-24 19:08:56,249 admin_server.py:118] Starting admin server at: http://localhost:8000
ERROR 2015-02-24 19:09:00,307 webapp2.py:1552] 'NoneType' object has no attribute 'key'
Traceback (most recent call last):
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 18, in dispatch
webapp2.RequestHandler.dispatch(self)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 95, in get
self.session['client'] = client.key.urlsafe()
AttributeError: 'NoneType' object has no attribute 'key'
INFO 2015-02-24 19:09:00,314 module.py:737] default: "GET / HTTP/1.1" 500 2354
INFO 2015-02-24 19:09:00,377 module.py:737] default: "GET /favicon.ico HTTP/1.1" 200 8348
INFO 2015-02-24 19:09:00,381 module.py:737] default: "GET /favicon.ico HTTP/1.1" 304 -
EINFO 2015-02-24 19:09:08,482 shutdown.py:45] Shutting down.
INFO 2015-02-24 19:09:08,483 api_server.py:588] Applying all pending transactions and saving the datastore
======================================================================
ERROR: test_guest_can_submit_contact_info (dermalfillersecrets.functional_tests.NewVisitorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/functional_tests.py", line 88, in test_guest_can_submit_contact_info
self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 302, in find_element_by_name
return self.find_element(by=By.NAME, value=name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: Unable to locate element: {"method":"name","selector":"id_name"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpSjWZ6W/extensions/fxdriver#googlecode.com/components/driver-component.js:9641:26)
at fxdriver.Timer.prototype.setTimeout/<.notify (file:///var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpSjWZ6W/extensions/fxdriver#googlecode.com/components/driver-component.js:548:5)
Here is the code in functional_tests.py
import sys, os, subprocess, time, unittest, shlex
sys.path.append("/usr/local/google_appengine")
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")
sys.path.append("/usr/local/google_appengine/lib/django-1.5")
sys.path.append("/usr/local/google_appengine/lib/cherrypy")
sys.path.append("/usr/local/google_appengine/lib/concurrent")
sys.path.append("/usr/local/google_appengine/lib/docker")
sys.path.append("/usr/local/google_appengine/lib/requests")
sys.path.append("/usr/local/google_appengine/lib/websocket")
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")
sys.path.append("/usr/local/google_appengine/lib/antlr3")
from selenium import webdriver
from google.appengine.api import memcache, apiproxy_stub, apiproxy_stub_map
from google.appengine.ext import db
from google.appengine.ext import testbed
import dev_appserver
from google.appengine.tools.devappserver2 import devappserver2
class NewVisitorTest(unittest.TestCase):
def setUp(self):
# Start the dev server
cmd = "/usr/local/bin/dev_appserver.py /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/app.yaml --port 8080 --storage_path /tmp/datastore --clear_datastore --skip_sdk_update_check"
self.dev_appserver = subprocess.Popen(shlex.split(cmd),
stdout=subprocess.PIPE)
time.sleep(2) # Important, let dev_appserver start up
self.testbed = testbed.Testbed()
self.testbed.setup_env(app_id="dev~myapp")
self.testbed.activate()
#self.testbed.setup_env(app_id='dermalfillersecrets')
self.testbed.init_user_stub()
# Create a consistency policy with a probability of 1,
# the datastore should be available.
self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)
# Initialize the datastore stub with this policy.
self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)
self.testbed.init_memcache_stub()
self.datastore_stub = apiproxy_stub_map.apiproxy.GetStub('datastore_v3')
# setup the dev_appserver
APP_CONFIGS = ['app.yaml']
# setup client to make sure
from main import Client
if not ( Client.query( Client.name == "Bryan Wheelock").get()):
logging.info("create Admin")
client = Client(
email = "bryan#mail.com",
name = "Bryan Wheelock",
street1 = "555 Main St",
street2 = "unit 1",
city = "Atlanta",
zipcode = 99999,
phone = "[phone]"
).put()
# this sleep is to allow eventual consistency to propogate
time.sleep(2)
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
self.testbed.deactivate()
self.dev_appserver.terminate()
def test_guest_can_submit_contact_info(self):
from main import Client, Customer
client = Client.query( Client.name == "Bryan Wheelock").get()
orig_customer_count = Customer.query(ancestor=client.key).count()
self.browser.get('http://localhost:8080')
time.sleep(5)
self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")
self.browser.find_element_by_name('id_street').send_keys("123 main st")
self.browser.find_element_by_name('id_phone').send_keys('[phone]')
self.browser.find_element_by_name('id_zip').send_keys("30306")
self.browser.find_element_by_name('submit').submit()
# the time delay is to allow eventual consisenency to happen.
time.sleep(4)
assert(Customer.query(Customer.name == "Kallie Wheelock").get())
# this should return 1 more record
final_customer_count = Customer.query(ancestor=client.key).count()
self.assertNotEqual(orig_customer_count, final_customer_count)
# Delete the Customer record
Customer.query(Customer.name =="Kallie Wheelock").delete()
Here's the code in main.py:
import os
import urllib
import logging
from google.appengine.api import users
from google.appengine.ext import ndb
import jinja2
import webapp2
from webapp2_extras import sessions
class BaseHandler(webapp2.RequestHandler):
def dispatch(self):
self.session_store = sessions.get_store(request=self.request)
try:
# dispatch the request
webapp2.RequestHandler.dispatch(self)
finally:
# save all sessions
self.session_store.save_sessions(self.response)
#webapp2.cached_property
def session(self):
# Returns a session using the default cookie key.
return self.session_store.get_session()
JINJA_ENVIRONMENT = jinja2.Environment(
loader = jinja2.FileSystemLoader(os.path.dirname(__file__)),
extensions=['jinja2.ext.autoescape'],
autoescape=True)
DEFAULT_LEADBOOK_NAME = 'whatsmyname'
def leadbook_key(leadbook_name=DEFAULT_LEADBOOK_NAME):
"""Constructs a Datastore key for a LeadBook entity with leadbook_name."""
return ndb.Key('LeadBook', leadbook_name)
class Client(ndb.Model):
email = ndb.StringProperty()
name = ndb.StringProperty(indexed=True)
street1 = ndb.StringProperty()
street2 = ndb.StringProperty()
city = ndb.StringProperty()
zipcode = ndb.IntegerProperty()
phone = ndb.StringProperty()
signup = ndb.DateTimeProperty(auto_now_add=True)
# this just creates a Client to use
if not ( Client.query( Client.name == "Bryan Wheelock").get()):
client = Client(
email = "bryan#mail.com",
name = "Bryan Wheelock",
street1 = "555 Main St",
street2 = "unit 1",
city = "Atlanta",
zipcode = 99999,
phone = "[phone]"
).put()
class Customer(ndb.Model):
# I commented out client property because using Ancestor Query( limited to 1 write per second)
#client = ndb.KeyProperty(kind=Client)
#email = ndb.StringProperty(indexed=True)
name = ndb.StringProperty(indexed=True)
street1 = ndb.StringProperty()
street2 = ndb.StringProperty()
city = ndb.StringProperty()
zipcode = ndb.IntegerProperty()
phone = ndb.StringProperty()
signup = ndb.DateTimeProperty(auto_now_add=True)
class MainPage(BaseHandler):
def get(self):
leadbook_name = self.request.get('leadbook_name',
DEFAULT_LEADBOOK_NAME)
# This should be the Client record that shows the info of the owner of the local clinic
# the question is how do I get the site to show the correct Client?
client = Client.query( Client.name == "Bryan Wheelock").get()
self.session['client'] = client.key.urlsafe()
template_values = {
'client': client,
'leadbook_name': urllib.quote_plus(leadbook_name),
}
template = JINJA_ENVIRONMENT.get_template('index.html')
self.response.write(template.render(template_values))
class LeadBook(BaseHandler):
def post(self):
leadbook_name = self.request.get('leadbook_name',
DEFAULT_LEADBOOK_NAME)
client = ndb.Key(urlsafe=self.session['client']).get()
customer = Customer( parent = client.key)
customer.name = self.request.get('id_name')
customer.street1 = self.request.get('id_street')
customer.phone = self.request.get('id_phone')
customer.zipcode = int(self.request.get('id_zip'))
# show original number of customer to show the code works
starting_customer_count = Customer.query(ancestor=client.key).count()
#import pdb; pdb.set_trace()
customer.put()
# This should return the record
assert(Customer.query(Customer.name == "Kallie Wheelock").get())
final_customer_count = Customer.query(ancestor=client.key).count()
#import pdb; pdb.set_trace()
query_params = {'leadbook_name': leadbook_name}
self.redirect('/?' + urllib.urlencode(query_params))
config = {}
config['webapp2_extras.sessions'] = {
'secret_key': 'my-super-secret-key',
}
application = webapp2.WSGIApplication([
('/', MainPage),
('/sign', LeadBook),
], config = config,
debug=True)

Answer 1

The problem is hinted at by your comment:
# this sleep is to allow eventual consistency to propogate
That's really not how it works. Eventual consistency has nothing to do with time, and the way it is emulated in the local datastore has even less; in the tests, the datastore testbed implements a policy whereby the initial read almost always fails. The documentation explains how you can tweak the policy in your test; one shortcut - again, that only works in tests and not in production - is to do an explicit .get() after saving, which will always make the entity visible.