I'm trying to switch my app engine app from python 2.5 to 2.7 and urlfetch.fetch() just doesn't want to work (on the dev server at least; I haven't deployed it yet). If I do something as simple as:
file = urlfetch.fetch(url="http://www.google.com")
I get the following error:
File "C:\workspace\DjangoServer\src\mycode.py", line 167, in request
file = urlfetch.fetch(url="http://www.google.com")
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\urlfetch.py", line 264, in fetch
return rpc.get_result()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_stub_map.py", line 592, in get_result
return self.__get_result_hook(self)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\urlfetch.py", line 366, in _get_fetch_result
raise DownloadError(str(err))
DownloadError: ApplicationError: 2 [Errno 11003] getaddrinfo failed
Searching the internet has suggested it might be a firewall or proxy issue, but I turned off the firewall and I'm not behind a proxy.
This works fine in the python 2.5 dev server, but not the python 2.7 dev server. If I open a python 2.7 console and manually runsocket.getaddrinfo("www.google.com", 80) it works there as well.
Any ideas? Has anyone else encountered this?
Ok, I have been able to reproduce this issue and after a lot of going back and forth in git from a version that was working to a version that was not working I found that (at least in my case) the problem was in django-nonrel (because I updated all at once: django-nonrel and appengine).
So, please do the following: edit djangoappengine/settings_base.py and change
try:
from google.appengine.api import apiproxy_stub_map
except ImportError:
from .boot import setup_env
setup_env()
from djangoappengine.utils import on_production_server, have_appserver
to:
from djangoappengine.utils import on_production_server, have_appserver
if not on_production_server:
from .boot import setup_env
setup_env()
The real issue seems to be that in a previous version of django-nonrel, the import was actually written as: from google.appengine.api import api_proxy_stub_map (which was never found and the setup_env() was always called), so, when the import was fixed, the setup_env() was no longer called (which in my case made the error appear).
A 2nd note is that for some reason this was only happening when running inside of Eclipse/PyDev and it did work on the command line (although the environment variables seemed the same for me, so, I'm unsure why running from inside java actually made a difference).
Related
I am not sure what I have done but from one day to the next all requests I am trying to run in my local environment lead to the same error:
"/env/lib/python2.7/site-packages/requests_toolbelt/adapters/appengine.py", line 122, in __init__
validate_certificate=validate_certificate)
File "/env/lib/python2.7/site-packages/urllib3/contrib/appengine.py", line 103, in __init__
"URLFetch is not available in this environment.")
AppEnginePlatformError: URLFetch is not available in this environment.
I am running a local virtualenv having basically followed the steps from here:
https://cloud.google.com/appengine/docs/flexible/python/writing-and-responding-to-pub-sub-messages
I have tried searching for a solution but am at a bit of a loss.
I found my mistake: I was trying to test some FCM notifications using pyfcm and initialised with this line:
push_service = FCMNotification(api_key="<api-key>", proxy_dict=proxy_dict, env='app_engine')
It seems that the env='app_engine' part switches all requests to the URLFetch version used on google app engine.
I'm new to Eclipse and PyDev and have been stuck on this for while having had a look at quite a few answers to similar issues on here.
I'm trying to build a simple web app using PyDev, Eclipse, Python 2.7 and Flask (on Windows) and have followed this guide (https://cloud.google.com/appengine/docs/python/getting-started/python-standard-env) which all worked fine.
I made some small changes, but am currently stuck on the first step where I am trying to import pandas in my script (main.py)
I'm getting this error from the debugger when I try to import pandas from in my script
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\runtime\wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\runtime\wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\runtime\wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "C:\Users\LONTI\workspace\Flask-app\main.py", line 3, in <module>
import pandas as pd
ImportError: No module named pandas
I've checked in Preferences > Interpreters > Python Interpreter that C:\Python27\lib\site-packages has been added (where my pandas module sits). I've also tried removing and adding the interpreter again but to no avail.
Also, in the editor I can see that pandas isn't Unresolved, so it seems like it can 'see' it. And in the workspace, under the Python > System Libs > lib/site-packages that pandas is also there.
I'm a bit at a loss where else to check.
main.py looks like this, where I've just cut out as much as possible to try and make sure there wasn't anything else that affect the import:
import logging
from flask import Flask, render_template, request
import pandas as pd
app = Flask(__name__)
#app.route('/form')
def form():
return render_template('form.html')
#app.route('/submitted', methods=['POST'])
def submitted_form():
name = request.form['name']
pc1 = request.form['pc1']
pc2 = request.form['pc2']
pc3 = request.form['pc3']
return render_template(
'submitted_form.html',
name=name,
pc1=pc1,
pc2=pc2,
pc3=pc3)
#app.errorhandler(500)
def server_error(e):
# Log the error and stacktrace.
logging.exception('An error occurred during a request.')
return 'An internal error occurred.', 500
Thanks in advance for your help and let me know if I need to provide any more info .
I think the issue is that you're using google-app-engine, which limits what's allowed to run.
Can Pandas run on Google App Engine for Python? has information which may be useful.
The import error in particular is caused by improper installation of pandas in your application. See Using third-party libraries. From Installing a third-party library:
In order to use a third-party library, copy it into a folder in your
project's source directory. The library must be implemented as pure
Python code with no C extensions. The code is uploaded to App Engine
with your application code, and counts towards file quotas.
This quote also ties into the answer mentioned by Fabio, it's unlikely you'll get this working on the standard GAE environment.
It might work on the flex environment - less restrictions, but that's a significantly different beast.
As far as I can tell, I have set up my flask app right. lib, containing all dependencies is at my root, and contains what Requirements.txt has in it. my appengine_config.py contains the below
print 'running app config yaya!'
from google.appengine.ext import vendor
import os
import sys
print os.path
print os.path.realpath
print os.path.realpath(__file__)
print os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')
sys.path.insert(0,'./lib')
vendor.add('lib')
print 'I am the line after adding lib, it should have worked'
per all those print statements, nothing is erroring, but I am getting
No module named flask_sqlalchemy
after deploying and seeing a 500. What am I missing to get these suckers installed?
EDIT---------------
thanks and here --
running app config yaya!
18:36:29.499
['./lib', '/base/data/home/apps/s~nimble-poet-150223/20161221t183424.397920801519685819', '/base/data/home/runtimes/python27/python27_dist/lib/python27.zip', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7/plat-linux2', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-tk', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-old', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-dynload', '/base/data/home/runtimes/python27/python27_dist/lib/python2.7/site-packages', '/base/data/home/runtimes/python27/python27_lib/versions/1', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/jinja2-2.6', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/markupsafe-0.15', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/protorpc-1.0', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/webob-1.1.1', '/base/data/home/runtimes/python27/python27_lib/versions/third_party/yaml-3.10']
18:36:29.499
at print dir(vendor)
['PYTHON_VERSION', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'add', 'os', 'site', 'sys']
18:36:29.503
at final print statement
I am the line after adding lib, it should have worked
18:36:29.948
the error
(/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~nimble-poet-150223/20161221t183424.397920801519685819/main.py", line 6, in <module>
from nimble import *
File "/base/data/home/apps/s~nimble-poet-150223/20161221t183424.397920801519685819/nimble/__init__.py", line 10, in <module>
from flask_sqlalchemy import SQLAlchemy, SignallingSession
ImportError: No module named flask_sqlalchemy
18:36:30.191
This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.
Not sure if it is related, but I am also getting https://code.google.com/p/google-cloud-sdk/issues/detail?id=729
but that fix doesnt help
---more edits
I moved everything from /site_pakcages up a dir to /lib and I am alot closer, some of my pages are even working! but now I get
ImportError: No module named _sqlite3
but the code reads
except ImportError as e:
try:
from sqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
except ImportError as ee:
raise ee
return sqlite
why would there be an underscore in the import attempt here? help?
tldr: use appengine_config.py and copy your virtualenv to a folder called lib, then make SURE you are running the app via dev_appserver.py
(the below is via bash in ubuntu) SO after a long battle, I find that virtual env and gcloud dont play nice -
I copied everything from my virtual env dir
.../.virtualenvs/nimble/local/lib/python2.7/site-packages
into
[projectdir]/lib
and my appengine_config.py finally worked locally like it does in the cloud, but I absolutely HAVE to run
dev_appserver.py [my proj dir here]
or the google.appengine module wont load. did not know I should be using dev server. I feel very dumb.
for reference, heres the appengine_config.py
"""`appengine_config` gets loaded when starting a new application instance."""
print 'running app config yaya!'
from google.appengine.ext import vendor
vendor.add('lib')
print 'I am the line after adding lib, it should have worked'
import os
print os.getcwd()
I am new to Google App Engine and so, I just followed the procedure to test an application which just prints "hello world".
(followed according to Google App Engine Documentation)
The project folder name is "GoogleApp"
This is my Python file:
File Name : sayHello.py
#!/usr/bin/env python
def main():
print "hello"
pass
if __name__ == '__main__':
main()
This the yaml file
File Name : app.yaml
application: GoogleApp
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: sayHello.py
When ever I just run the project in Google App Engine Launcher, this is the log error I am getting
2012-04-19 10:52:23 Running command: "['C:\\Python27\\pythonw.exe', 'C:\\Program Files\\Google\\google_appengine\\dev_appserver.py', '--admin_console_server=', '--port=8080', 'D:\\Code\\Projects\\IRCmathBot\\GoogleApp']"
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\dev_appserver.py", line 125, in <module>
run_file(__file__, globals())
File "C:\Program Files\Google\google_appengine\dev_appserver.py", line 121, in run_file
execfile(script_path, globals_)
File "C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py", line 157, in <module>
from google.appengine.tools import appcfg
File "C:\Program Files\Google\google_appengine\google\appengine\tools\appcfg.py", line 41, in <module>
import mimetypes
File "C:\Python27\lib\mimetypes.py", line 29, in <module>
import urllib
File "C:\Python27\lib\urllib.py", line 26, in <module>
import socket
File "C:\Python27\lib\socket.py", line 47, in <module>
import _socket
ImportError: Module use of python25.dll conflicts with this version of Python.
2012-04-19 10:52:25 (Process exited with code 1)
Actually, I use Python 2.7. The above log says that python25.dll conflicts with version of python.
I don't know what exactly it means but I could interpret that
It should either support only python 2.5 or
It I should have used python 2.5 code.
as the 2nd isn't possible and Google supports 2.7, I don't know what is this error.
Try to figure out myself but couldn't succeed.
I had a very similar problem (while running the tutorial code), and solved it by checking my PythonPath system environment variable.
For me, OpenCV edited the PythonPath variable to point to its own directory structure instead of the base Python installation itself.
You can edit the environment variables by opening up the 'Advanced System Properties' window and clicking on the 'Environment Variables' button. The 'PythonPath' variable under 'System Variables'. At the minimum, it should include the path of your python executable, such as 'C:\Python27\'.
Have you tried in command-line? Forget the launcher.
I had same problem! I re-installed Python several times, changed versions. It was hilarious.
With version 2.5 launcher can start the server, but the application doesn't work, but I can upload it online through launcher.
With Python ver. 2.7, launcher displays same problems you're dealing with. I also can't get the server started locally. But command-line works well, both starting the server locally and uploading online.
So I'll stick with version 2.7. Hope it helps, let me know..
You have threadsafe enabled, so you should use WSGI handlers. That could be the problem. I would replace your code with the following:
File: helloworld.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, WebApp World!')
app = webapp2.WSGIApplication([('/', MainPage)])
File: app.yaml
application: GoogleApp
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: helloworld.app
If using Python2.7 on Google AppEngine, refer to this official documentation to create your first "Hello World" Application.
I fixed this after 3 hrs of effort. Reinstall cannot solve my problem, messing around the user/system environment variable also not work.
Follow this step work:
Open Google Drive Perference, turn off auto run after win start up
turn off Google Drive
or simply uninstall google drive (Google Drive seems using Python 2.5 _socket library
uninstall python and GAE
reboot
install python27 and GAE
Most importantly, specify a new path, do not use the default c:\python27 again (I changed it to c:\mypython27. same as GAE, use a new path.
Then it works again.
Something's seriously broken with your Python 2.7 installation - parts of it are referencing Python 2.5 libraries. You should uninstall all versions of Python on your machine, and reinstall the version(s) you need.
I guess you've added the GAE path into the environment variable PATH? Just remove it.
I had exactly the same problem with you, it seems to me that Python 2.7 will try to use an incorrect _socket.pyd lib under GAE path, removing GAE path from PATH will fix this.
It needs no extra coding, just a tiny configuration of your win OS.
Since my OS language is Chinese, names of icons/links below are translated from Chinese by myself, which i cannot guarantee to be correct :(
Right click on the "Computer" icon, choose 'Properties', you'll see the system panel showing some basic system information;
Click on the "Advanced System Settings" link, you'll see the system property panel containing several tabs;
Choose the "Advanced" tab, there should be a button named "Environment variables" on the bottom, click on it then you can see a dialog displaying all environment variables of your system;
Find and edit the variable named "PATH" or "Path" (case insensible): if its value contains a path to your GAE, then remove the GAE path.
For me it was that I did set the System Variable PYTHONPATH to '.'. Deleting the variable did the job!
I set PYTHON_PATH to
"C:\Program Files\Google\google_appengine;C:\Program
Files\Google\google_appengine\lib\antlr3;C:\Program
Files\Google\google_appengine\lib\django_0_96;C:\Program
Files\Google\google_appengine\lib\fancy_urllib;C:\Program
Files\Google\google_appengine\lib\graphy;C:\Program
Files\Google\google_appengine\lib\ipaddr;C:\Program
Files\Google\google_appengine\lib\simplejson;C:\Program
Files\Google\google_appengine\lib\webob;C:\Program
Files\Google\google_appengine\lib\yaml\lib"
but then I realize that django 0.96 require python 2.5, so use django 1.4, and it works fine.
"C:\Program Files\Google\google_appengine;C:\Program
Files\Google\google_appengine\lib\antlr3;C:\Program
Files\Google\google_appengine\lib\django-1.4;C:\Program
Files\Google\google_appengine\lib\fancy_urllib;C:\Program
Files\Google\google_appengine\lib\graphy;C:\Program
Files\Google\google_appengine\lib\ipaddr;C:\Program
Files\Google\google_appengine\lib\simplejson;C:\Program
Files\Google\google_appengine\lib\webob;C:\Program
Files\Google\google_appengine\lib\yaml\lib"
If you have same problem as above, just check PATH, PYTHON_PATH to see whether mismatch version between library and python. Hope this help.
Some programs may modify the PYTHONPATH environment variable. If you check it's value (System -> Environment variables) and ensure it is set to the correct python 2.7 path (default C:\Python27) then this should fix the issue. In my case something had set it to my google appengine path.
ImportError: Module use of python25.dll conflicts with this version of Python.
The problem is that python25.dll conflicts with your Python 2.7 installation. This python25.dll comes from the greatly outdated Google App Engine Launcher for Windows and is located in C:\Program Files\Google\google_appengine\launcher
It is not a problem of your Python 2.7 install - it is a problem of Launcher, who injects its own Python path into app environment. To test that, add the following lines at the top of google_appengine/dev_appserver.py after first imports:
from pprint import pprint
pprint(sys.path)
Then re-run you app to get fresh log and see for yourself:
...
'C:\\OpenCV2.3\\opencv\\build\\python\\2.7',
'C:\\Google\\google_appengine\\launcher',
'C:\\Windows\\system32\\python27.zip',
'C:\\Python27\\DLLs',
...
I got the launcher from Python .msi SDK 1.8.8, and it is of course suxx that Google still ships several versions of SDK with broken GUI tool. To fix that, add the following after first imports in dev_appserver.py:
# --- Repair sys.path after broken GAE Windows Launcher
# see http://stackoverflow.com/questions/10222342/
ROOT = os.path.abspath(os.path.dirname(__file__))
LAUNCHPATH = os.path.join(ROOT, 'launcher')
if LAUNCHPATH in sys.path:
sys.stderr.write('[BUG] GAE Windows Launcher detected. Fixing..\n')
sys.path.remove(LAUNCHPATH)
# /--
Unfortunately, you'll need to repeat the procedure when new version of GAE comes out. There is an issue #8568 on AppEngine tracker that you can star and comment to make it fixed someday.
Do you have a PYTHONPATH variable in Environment Variables? delete it and restart the Google App Engine Launcher,it will work!
I have created my GAE application in directory "my_application". Inside this directory I created a .py file and named it "my_scrypt".
The contents of "my_scrypt" in the beginning were as following:
print 'Content-Type: text/plain'
print ''
print 'This is my first application'
Then I ran it locally on my machine (Windows XP) in the installed browser (Mozilla FireFox) with "GAE Launcher" - everything was fine - I could see that sentence ("This is my first application") on the screen.
Then I deployed this application to GAE (again with the help of "GAE Launcher") - everything was fine again - I could see the same sentence on the screen.
Then I changed the contents of "my_scrypt" a bit:
from twill.commands import *
config('use_tidy', '0')
go ("http://us.yahoo.com/")
showlinks()
Downloaded "twill0.9" (from here), chose and copied "twill" folder from there, and pasted it in "my_application" directory.
When I ran this new application locally (with "GAE Launcher") everything was fine - I could see a list of yahoo.com links on the screen, but when I uploaded this application to GAE, I received only an error message.
Why is it so? I don't think it's because the version of mechanize being used by twill here is too old - the code in "my_script" is so simple, any version of mechanize must be able to handle it.
Does GAE accept twill (as an external module) at all?
You can view the stack trace of the error in the "Update 1" section right below (↓).
UPDATE 1:
(This update is my answer to Nick)
Hello, Nick. I checked the admin console, so here is the stack trace:
<type 'exceptions.ImportError'>: No module named fcntl
Traceback (most recent call last):
File "/base/data/home/apps/silkybutton/1.344911014283513184/bumper.py", line 1, in <module>
from twill.commands import *
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/__init__.py", line 52, in <module>
from shell import TwillCommandLoop
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/shell.py", line 9, in <module>
from twill import commands, parse, __version__
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/commands.py", line 70, in <module>
from browser import TwillBrowser
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/browser.py", line 17, in <module>
from _browser import PatchedMechanizeBrowser
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/_browser.py", line 9, in <module>
from utils import FixedHTTPBasicAuthHandler, FunctioningHTTPRefreshProcessor
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/utils.py", line 12, in <module>
import subprocess
File "/base/data/home/apps/silkybutton/1.344911014283513184/twill/other_packages/subprocess.py", line 378, in <module>
import fcntl
Twill is trying to import 'subprocess'. This is a Python module for spawning threads, and it's not available on App Engine. You'll either need to see if you can persuade Twill to work without spawning processes (probably by modifying the code), or you'll need to use mechanize or simply urlfetch directly.