Based on other questions, I expected to be able to run the following command to get a dump of datastore entities for my app:
appcfg.py download_data --filename data.db .
(Here's the app.yaml file in that directory.)
When I run that command, however, I get a long stack trace ending with a crypic error:
google.appengine.api.datastore_errors.BadRequestError: app s~old-nyc cannot access app old-nyc's data
I get this error even if I explicitly specify --application=old-nyc. If, as suggested in this question, I specify --application=s~old-nyc, then I get this error:
[ERROR ] Authentication Failed: Incorrect credentials or unsupported authentication type (e.g. OpenId).
What do I need to do to download my data?
Specifying --application s~old-nyc changed the default URL to http://s~old-nyc.appengine.com rather than http://old-nyc.appengine.com. To get the command to work, I also had to specify --url http://old-nyc.appspot.com/remote_api.
My full command was:
appcfg.py download_data --application s~old-nyc --url http://old-nyc.appspot.com/_ah/remote_api --filename data.db
Related
I am trying to install solr so that my data catalog could use it. To do so, I used these steps which are mentioned on the documentation of my data portal:
cd solr/solr-config
wget http://apache.crihan.fr/dist/lucene/solr/6.0.0/solr-6.0.0.tgz
tar xvfz solr-6.0.0.tgz
solr-6.0.0/bin/solr start -c -p 8984
solr-6.0.0/bin/solr create -p 8984 -c catalog_srv -d src/main/solr-cores/catalog
accourding to these instructions I made a directory /opt/solr/solr-config the downloaded and unziped solr, and started it on port 8984. Now I don't understand how does the last command works. what should be the second path src/main/solr-cores/catalog? I thought I should generatet the directories src, main solr-cores and catalog inside my solr-config directory and then run this command. but I got errors that solrconfig.xml could not be found. after adding solrconfig.xml to /opt/solr/solr-config/src/main/solr-cores/catalog, now I get an error:
ERROR: Failed to create collection 'catalog_srv' due to: {127.0.1.1:8984_solr=org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://127.0.1.1:8984/solr: Error CREATEing SolrCore 'catalog_srv_shard1_replica1': Unable to create core [catalog_srv_shard1_replica1] Caused by: Can't find resource 'schema.xml' in classpath or '/configs/catalog_srv', cwd=/opt/solr/solr-config/solr-6.6.0/server}
what is schema.xml? is it something about data of my data portal? could you please explain the issue and how should I determine the path src/main/solr-cores/catalog to avoid these errors? what does exactly a default collection of solr do?
More info: my data portal is an opensource software called geonetwork and the documentation about solr is here: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/installing-solr.html?highlight=solr
I needed to test some changes on my local dev server before pushing to production. Doing so required having the full dataset on my local machine.
A colleague directed me to:
https://developers.google.com/appengine/docs/python/tools/uploadingdata?csw=1
I downloaded the data using an administrator's username and password, but unfortunately, I was unable to upload the data to my localhost "dev" app engine server.
Ran this command from the commandline:
appcfg.py upload_data --filename=../data/data1.dat --url=http://localhost:9080/_ah/remote_api ./
Where:
9080 was my app port on my localhost copy of the app
I was running this command from my app directory
Had the downloaded data stored in relative directory
../data/data1.dat
Received this error:
raise _ToDatastoreError(err)
google.appengine.api.datastore_errors.BadRequestError: app "dev~appname" cannot access app "appname"'s data
UPDATE: It seems that the answer was as simple as adding the following to my upload_data call:
--application="dev~appname"
Thanks #DavidBennett.
ORIGINAL ANSWER: (which also works)
After a ton of searching on SO and code.google.com, the solution I found that worked was a comment on this question:
devappserver2, remote_api, and --default_partition
I used my original command as described in the question:
appcfg.py upload_data --filename=../data/data1.dat --url=http://localhost:9080/_ah/remote_api ./
The username and password I entered when prompted were my apps username (in my case, my email) and the corresponding password. (If that doesn't work you might want to try blank or test#example.com based on other comments I've read, but have not tested that theory.)
I also restarted my app engine with the following flag: (don’t forget to remove the flag the next time you restart the server) (You might want to try without using this flag, since I can’t confirm that it affects anything - I’m including it here, since it was a setting that I used.)
--clear_datastore=yes
The commenter recommends to delete “dev~” in your local server code on line 84 in this file:
google/appengine/tools/devappserver2/application_configuration.py, line 84
Where:
that base directory 'google' is located inside of:
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
assuming your GoogleAppEngineLauncher.app directory is in your Applications directory on your Mac
IMPORTANT: Restart your local app engine server for the changes to take effect.
Trying to run my web2py app from the development server using GoogleAppEngineLauncher
Not sure if the on-line tutorials are out of date or I'm just missing something, but when I follow the link to download the GoogleAppEngine pythonSDK for OSX I get a dmg for the GoogleAppEngineLauncher.
I download and use that, which installs the proper executables, however after I setup my app.yaml file and run "dev_appserver.py myApp" I get this error:
fancy_urllib.InvalidCertificateException: Host appengine.google.com returned an invalid certificate (_ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed):
I don't get the error if I try and launch the app from the launcher itself, however I can't launch the app because it says the directory already exists and I don't have write permissions. I even tried chmod 777 on the myApp directory.
Should I not be using the GoogleAppLauncher?
additionally I tried using the linux SDX and received the same "certificate" error. The error message directs me to a link mentioning that I need the "ssl" module, but that is included in python 2.7.2 which I am using.
The link also mentions: "appcfg uses SSL when connecting to the Admin Console by default, unless the --insecure flag is passed." But I cannot find that flag in the help menu.
Found this answer which solves the problem.
Basically:
rm google_appengine/lib/cacerts/cacerts.txt
From the SDK
I have GWT app, which is deployed on GAE (Java). I'm trying to download data from App Engine datastore using appcfg.py . I did all the setup according to http://ikaisays.com/2010/06/10/using-the-bulkloader-with-java-app-engine/ .
GAE Python SDK version is 1.4.3
Python version is 2.5.4
appcfg.py is on my PATH. When I run appcfg.py on the command-line, I get the "help" message. But the problem is that no matter which command I use, it always returns the help message. I have not been able to run any command using appcfg.py.
It doesn't give any specific error message no matter what arguments I give. My app is using Google Accounts authentication, but I don't think it even gets to the point of authentication.
I'm able to use the Java appcfg (for other actions like rollback) without any problem. But the Python version simply refuses to work for all commands.
I've tried different formats like:
appcfg.py create_bulkloader_config --url=http://myappid.appspot.com/remote_api --application=myappid --filename=config.yml
appcfg.py create_bulkloader_config --filename=bulkloader.yaml --url=http://myappid.appspot.com/remote_api
appcfg.py --filename=bulkloader.yaml --url=http://myappid.appspot.com/remote_api create_bulkloader_config
All give me the same help message:
Usage: appcfg.py [options]
Action must be one of:
create_bulkloader_config: Create a bulkloader.yaml from a running application.
cron_info: Display information about cron jobs.
download_app: Download a previously-uploaded app.
download_data: Download entities from datastore.
help: Print help for a specific action.
request_logs: Write request logs in Apache common log format.
rollback: Rollback an in-progress update.
set_default_version: Set the default (serving) version.
update: Create or update an app version.
update_cron: Update application cron definitions.
update_dos: Update application dos definitions.
update_indexes: Update application indexes.
update_queues: Update application task queue definitions.
upload_data: Upload data records to datastore.
vacuum_indexes: Delete unused indexes from application.
Use 'help <action>' for a detailed description.
Options:
-h, --help Show the help message and exit.
-q, --quiet Print errors only.
-v, --verbose Print info level logs.
--noisy Print all logs.
-s SERVER, --server=SERVER
...
...
...
Even when I try "appcfg.py help create_bulkloader_config" for a detailed description, it still shows me the same standard help.
I have also tried on the local development server using the url http://127.0.0.1:8888/remote_api but it still gives the same help message.
I'm totally clueless as to what the problem is. I'm new to GWT and GAE, and any help will be appreciated.
Thanks.
The following fix worked for me. It looks like appcfg.py doesn't like PYTHON27 and ALWAYS returns the help menu. I fixed it by executing it with PYTHON25 and hard coded all my file locations:
C:\Python25-archive\python "C:\Program Files (x86)\Google\google_appengine\appcfg.py" rollback C:\scripts\myapp
The right way is to change the environment variables on Windows 7:
Go to System Properties
Go to Advance System Settings
Click on Environment Variables
Append to Path variable the values C:\Python27\
Click Ok and restart your computer. (Yes, it is needed.)
Another way is to:
Open command Prompt
Locate your python.exe file. For example:
C:\Python27>_
Then, run a python command that looks like this.
python <appcfg_directory> download_app -A <your_app_id> -V <your_app_version> <output-dir>
Where <appcfg_directory> is equal to C:\Program Files\Google\google_appengine\appcfg.py. (Depending on your file location)
Don't forget to put quotes before and after <appcfg_directory>
I'm trying to get sendmail working on localhost. The problem is that the app engine has det wrong path to sendmail, since it throws the error: /bin/sh: sendmail: command not found
Using Mac OS X, the path to sendmail is: /usr/sbin/sendmail
Does anyone know how to change this?
..fredrik
I don't know how to change it but a symlink will do the job. I had the same problem and this solved it:
sudo ln -s /usr/sbin/sendmail /usr/bin/sendmail
Another option is to configure your local AppEngine runtime to use an external SMTP server.
According to the documentation,
The web server will use the sendmail
command to send email messages, with
your installation's default
configuration
If it uses NETINFO to determine the location of sendmail, the full path to sendmail.cf is stored as the value of the "sendmail.cf" property in the "/locations/sendmail" subdirectory of NETINFO, so try to set the value of this property to "/usr/sbin/sendmail.cf" (without the quotes)
Hope it helps