I'm trying to use GAE image service, but I have encountered a problem.
I have two projects. Project A has GAE, and Project B has GCS bucket.
This is because the GAE region can not be changed once the project is created.
When get_serving_url is called from the GAE to the object of the GCS bucket, the following exception occurs.
Traceback (most recent call last):
...
File "/base/data/home/apps/b~[PROJECT_NAME]/20161215t131056.397760967541775314/main.py", line 125, in _get_serving_url
return get_serving_url(blob_key)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/images/__init__.py", line 1868, in get_serving_url
return rpc.get_result()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/images/__init__.py", line 1972, in get_serving_url_hook
raise _ToImagesError(e, readable_blob_key)
TransformationError
When putting them in the same project this exception will not be thrown.
GAE's service account and project A have full access to buckets and objects.
Is there anything else I need to try?
Self-solving
I founds cause in the docs of Image API in GAE/Java.
Important: If you serve images from Google Cloud Storage, you cannot serve an image from two separate apps. Only the first app that calls getServingUrl on the image can get the URL to serve it because that app has obtained ownership of the image. Any other app that subsequently calls getServingUrl on the image will therefore be unsuccessful. If a second app needs to serve the image, the app needs to first copy the image and then invoke getServingUrl on the copy.
I used objects of the bucket from another project simultaneously.
Consequently, I have verified get_serving_url correctly.
You do not have to use a serving url. You can serve files grom GCS:
https://storage.googleapis.com/your_bucket/your_filename
or SDK http://localhost:8080/_ah/gcs/your_bucket/your_filename
Related
I have an json file in the public folder, to which after starting the application I make a request to it. When I load the application from a port on which I hosted it, everything is fine, but when I change the port with some url (test.project.bg) this json file does not build or somehow does not find the way to it.
I'm using Google Cloud Endpoints as back end of a mobile application.
Now i want to implement push notifications for the iOS client but can't load the .p12 certificate from an #ApiMethod, get this error message:
Invalid keystore reference. File does not exist:
/base/data/home/apps/s~my-ws/1.379168523188882449/MyCert.p12"
I added the certificate under /src directory but does not seem to recognize it.
I'm using IntelliJ IDEA, Appengine API 1.9.12, javapns (for Push Notifications) and Maven.
Edit
Maybe i made a step forward.
I put the .p12 file under /src/webapp/WEB-INF/ and added
<configuration>...<webResources><resource><includes><include>*.p12</include>
in my pom.xml.
Then i run mvn clean install && mvn appengine:endpoints_get_discovery_doc and inspected the generated myws-1.0-SNAPSHOT.war. Within the .war file there is my MyCert.p12certificate, but i get this error message now:
java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/WEB-INF/MyCert.p12\" \"read\")"
Could you check the code that is actually loading the cert? It may be that you need to remove a leading slash from the File constructor. This is not an App Engine thing but a java File thing.
Also, unrelated to your solution but helpful, is a thread here on best ways to store p12 files on App Engine.
There's a similar question that was recently responded to on Stackoverflow here: Google Cloud Storage Client not working on dev appserver
The solution was to either upgrade the SDK to 1.8.8 or use the previous revision of the GCS client library which didn't have the bug instead.
I'm currently using 1.8.8 and have tried downloading multiple revisions and /_ah/gcs doesn't load for me. After using up a significant number of my backend instances trying to understand how GCS and app engine work together, it'd be great if I could just test it on my local server instead!
When I visit localhost:port/_ah/gcs I get a 404 not found error.
Just a heads up, to install the library all I did was drag and drop the code into my app folder. I'm wondering if maybe I skipped a setup step? I wasn't able to find the answer in the documentation!
thanks!!
Note
To clarify this is my first week using GCS, so my first time trying to use the dev_server to host it.
I was able to find the google cloud storage files I wrote to a bucket locally at:
localhost:port/_ah/gcs/bucket_name/file_suffix
Where port is by default 8080, and the file was written to: /bucket_name/file_suffix
For those trying to understand the full process of setting up a simple python GAE app and testing local writes to google cloud storage:
1. Follow the google app engine "quickstart":
https://cloud.google.com/appengine/docs/standard/python/quickstart
2. Run a local dev server with:
dev_appserver.py app.yaml
3. If using python, follow "App Engine and Google Cloud Storage Sample":
https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/app-engine-cloud-storage-sample
If you run into "ImportError: No module named cloudstorage" you need to create a file named appengine_config.py
touch appengine_config.py
and add to it:
from google.appengine.ext import vendor
vendor.add('lib')
GAE runs this script automatically when starting your local dev server with dev_appserver.py app.yaml, and it is necessary to run this script for GAE to find the cloudstorage library in your lib/ folder
4. "Writing a file to cloud storage" from the same tutorial:
def create_file(self, filename):
"""Create a file."""
self.response.write('Creating file {}\n'.format(filename))
# The retry_params specified in the open call will override the default
# retry params for this particular file handle.
write_retry_params = cloudstorage.RetryParams(backoff_factor=1.1)
with cloudstorage.open(
filename, 'w', content_type='text/plain', options={
'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
retry_params=write_retry_params) as cloudstorage_file:
cloudstorage_file.write('abcde\n')
cloudstorage_file.write('f'*1024*4 + '\n')
self.tmp_filenames_to_clean_up.append(filename)
with cloudstorage.open(
filename, 'w', content_type='text/plain', options={
'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
retry_params=write_retry_params) as cloudstorage_file:
cloudstorage_file.write('abcde\n')
cloudstorage_file.write('f'*1024*4 + '\n')
Where filename is /bucket_name/file_suffix
4. After calling create_file via a route in your WSGI app, your file will be available at:
localhost:port/_ah/gcs/bucket_name/file_suffix
Where port is by default 8080, and the file was written to: /bucket_name/file_suffix
Postscript
Unfortunately, I did not find either 3) or 4) in their docs, so I hope this helps someone get set up more easily in the future.
To access gcs objects on dev_appserver, you must specify the bucket & object name, i.e. /_ah/gcs/[bucket]/[object].
The storage simulator for the local server is working in later versions of the SDK. For Java, one may choose to follow a dedicated tutorial: “App Engine and Google Cloud Storage Sample”.
I know it would be great if the file wouldn't link to local resource, but using phonegap/steroids framework, FILE_URI returns "file:///Users/" path which I can use for uploading to S3 or else, but Angular won't show it in the template.
Is there a possible solution? I tried adding config to my app
.config(function ($compileProvider){
$compileProvider.aHrefSanitizationWhitelist(/^\s*(https?|ftp|mailto|file|tel):/);
$compileProvider.imgSrcSanitizationWhitelist(/^\s*(https?|ftp|mailto|file|tel):/);
})
but it seems that doesn't have impact on the error.
I can base64 encode my images (then works), but I would like to avoid this if possible.
thanks
See the Camera example in the Steroids Kitchensink app, where the Cordova File API is used to move the picture from the tmp folder to the Steroids app's User Files folder. Since Steroids's localhost looks for assets both in the App folder and User Files folder, you can use an absolute path, e.g. src="/my_image.png". See also the App Structure on Device guide for more information on the App and User Files folder.
We have a gwt app deployed on gae for java. The app runs fine in google chrome but fails with below exception on ie and firefox
NetworkError: 404 Not Found - http://www.sakshum.org/adminmodule/67883654A8944A4C561CF25763FB1D79.cache.html
based on Setup a GWT Project correctly with SVN and Eclipse we have excluded the files in adminmodule directory to upload to app engine.
Please advise what is reason for it to fail and how to make it working.
The ignored patterns are:
.svn
*.bak
classes/
thumbs.db
*.class
.gwt*
gwt-unitCache/
deploy/
war/adminmodule/
war/sakshumwebgae/
sakshumweb/war/WEB-INF/deploy/adminmodule/
sakshumweb/war/sakshumwebgae/
.bin
*.orig
You will get 404. HORROR!!!!
Currently only /deploy/ folder can be ignored and not war/gwtmodule.
All the gwt generated scripts are in war/gwtmodule and you need to upload them per compilation into appengine.
These are generated every build in compilation phase and hence are not checked into svn.
They need to be in deployment folder for APP Engine.
I suggest you go through the GWT teams excellent document for App Engine with GWT https://developers.google.com/web-toolkit/doc/latest/tutorial/appengine
Edit -
<modulename>.nocache.js loads <longnumeric>.cache.html based on browser * language permutation. GWT compiles your java code to create the <modulename>.nocache.js and the relevant cache.html files. cache and nocache indicates whether browser is supposed to cache or not cache the file.
You will have the .nocache.js script reference in your html file for the gwtapp.
It's normal for the GWT compiler to build different javascript files for each permutation. A permutation is for a specific user-agent (browser, eg ie9 gecko (ff), webkit (chrome/safari)) and language (English, French). So you've correctly uploaded all the output files for the chrome browser, presumably in English. It would seem, as you say, that you're filtering out other files from uploading, and that some of those files are requested when using a different permutation for Firefox in English. You should try not to filter those files.