I have a script that uploads a photo to google compute engine to be processed, saves it in google cloud storage, and then responds with the path which i send to an app engine app to be read... testing with the gsutil cp command shows that the picture is saved correctly to GCS as the cp command always finds it.
However a lot of times app engine has problems find the photo when I send the path, returning a:
NotFoundError: Expect status [200] from Google Storage. But got status 404
Any thoughts?
Thanks to Voscausa, you can simply solve the issue by utilizing adequate parameters:
https://developers.google.com/appengine/docs/python/googlecloudstorageclient/retryparams_class
Related
I am running a Java Microengine on GAE.
I have my own html pages for data input and output. When there is a data error and the engine cannot complete its execution (crashes) - the microengine spits out the "Response" as Server not available, please try later.
In order to debug, I run the dataset in the dev environment - as a Java application to identify the error in the console output.
Is there a way to capture the "error" (the console output equivalent when run as a Java application) - as an output string and send it as a content of the servlet response from the deployed Application in GAE..
thanks,
assuming you are using App Engine Managed VM, and a logging framework, you should also forward the log entry into a file, e.g. /var/log/app_engine/custom_logs/app.log
https://cloud.google.com/appengine/docs/managed-vms/custom-runtimes#logging
Subsequently, you'll be able to read the output from Google Cloud Logging.
Im unable to see images by the URL returned from getPublicUrl(). The image is uploaded directly to Google Cloud Storage via HTTP POST and I can see them in Datastore/Blobstore Viewer on http://localhost:8000.
try{
echo CloudStorageTools::getPublicUrl("gs://qbucket_first/sample.jpeg", true);
}catch ( CloudStorageException $e ) {
echo 'There was an exception creating the Image Public URL, details ' . $e->getMessage();
}
What I get is this: http://localhost:8080/_ah/gcs/qbucket_first/sample.jpeg But I receive a 404 error when entering that URL. How can I fix this problem? I am using Chrome, Mac OS X, GAELauncher v1.8.9 PHP.
Edit 1
If I run this code:
file_put_contents("gs://qbucket_first/hello.txt", "Hello");
echo CloudStorageTools::getPublicUrl("gs://qbucket_first/hello.txt", true);
I get
http://localhost:8080/_ah/gcs/qbucket_first/hello.txt
And when entering this URL, I can download the file and read the content. So it works with text files which means that Datastore is working on local dev.
There's the same thing in Python version Google App Engine.
dev_appserver.py create a fake Cloud Storage for local development, when you write something into Google Cloud Storage in local environment, it actually writes it into a fake space, getPublicUrl() in local environment only gets the fake url in your local Google Cloud Storage.
The first time you upload the file to the real Google Cloud Storage, so in the local environment you get 404.
the second time you save the file in your local runtime, so it actually writes into the local fake GCS environment, so the second time it works.
you are using the dev server, which seems to be the problem. Run it from production.
Does the image filename have spaces?
If yes, remove the spaces or replace with _.
TransformationError
This error keeps coming up for a specific image.
There are no problems with other images and I'm wondering what the reason for this exception could be.
From Google:
"Error while attempting to transform the image."
Update:
Development server it works fine, only live it fails.
Thanks
Without more information I'd say it's either the image is corrupted, or it's in a format that cannot be used with get_serving_url (animate GIF for example).
I fought this error forever and incase anyone finds they get the dreaded TransformationError please note that you need to make sure that your app has owner permissions on the files you want to generate a url for
It'll look something like this in your IAM tab:
App Engine app default service account
your-project-name-here#appspot.gserviceaccount.com
In IAM on that member you want to scroll down to Storage and grant "Storage Object Admin" to that user. That is as long as you have your storage bucket under the same project... if not I'm not sure how...
This TransformationError exception seems to show up for permissions errors so it is a bit misleading.
I way getting this error because I had used the Bucket Policy Only permissions on a bucket in a different project.
However after changing this back to Object Level permissions and giving my App Engine app access (from a different project) I was able to perform the App Engine Standard Images operation (google.appengine.api.images.get_serving_url) that I was trying to implement.
Make sure that you set your permissions correctly either in the Console UI or via gsutil like so:
gsutil acl ch -u my-project-a#appspot.gserviceaccount.com:OWNER gs://my-project-b
I'm receiving an image file through a HTTP request and storing it in the Blobstore. My code is along the lines of that found here: [https://developers.google.com/appengine/docs/java/blobstore/overview#Writing_Files_to_the_Blobstore][1].
When calling the function ImagesService.getServingUrl() on the blobkey for the image I had stored I get the message below:
HTTP ERROR 500
Problem accessing /upload. Reason:
Failed to read image
Caused by:
java.lang.IllegalArgumentException: Failed to read image
at com.google.appengine.api.images.ImagesServiceImpl.getServingUrl(ImagesServiceImpl.java:282)
After hours of googling, trying to find a solution, I think that the problem is that the function closeFinally() is faulty. I think this because after closing the new blob image that I make, the FileStats object associated with the blob returns false for the hasFinalizedName(). Yet the blob returns true for isClosed() and isFinalized(). Also, its full path is something like this:
/blobstore/writable:xFSAF_493dsfAfw489vFa...
If it is closed it shouldn't be writable, I think.
So my questions are: Does anyone have a solution to the getServingUrl not working? Does anyone have a workaround getServingUrl? I want to have pictures to serve to users and I want these pictures to exist as static files. getServingUrl with blobstore seems to be the best choice for me. I don't want to use Google Cloud Storage because for every Blobstore vs Google Cloud Storage question I saw, Blobstore won it seemed.
I'm trying to use the "Copy to another app" feature of AppEngine and keep getting an error:
Fetch to http://datastore-admin.moo.appspot.com/_ah/remote_api failed with status 302
This is for a Java app but I followed the instructions on setting up a default Python runtime.
I'm 95% sure it's an authentication issue and the call to remote_api is redirecting to the Google login page. Both apps use Google Apps as the authentication mechanism. I've also tried copying to and from a third app we have which uses Google Accounts for authentication.
Notes:
The user account I log in with is an Owner on all three apps. It's a Google Apps account (if that wasn't obvious).
I have a gmail account this is an Owner on all three apps as well. When I log in to the admin console with it, I don't see the datastore admin console at all when I click it.
I'm able to use the remote_api just fine from the command-line after I enter my details
Tried with both the Python remote_api built-in and the Java one.
I've found similar questions/blog posts about this, one of which required logging in from a browser, then manually submitting the ACSID cookie you get after that's done. Can't do that here, obviously.
OK, I think I got this working.
I'll refer to the two appIDs as "source" and "dest".
To enable datastore admin (as you know) you need to upload a Python project with the app.yaml and appengine_config.py files as described in the docs.
Either I misread the docs or there is an error. The "appID" inthe .yaml should be the app ID you are uploading to to enable DS admin.
The other appID in the appengine_config file, specifically this line:
remoteapi_CUSTOM_ENVIRONMENT_AUTHENTICATION = (
'HTTP_X_APPENGINE_INBOUND_APPID', ['appID'])
Should be the appID of the "source", ID the app id of where the data is coming from in the DS copy operation.
I think this line is what allows the source appID to be authenticated as having permissions to write to the "dest" app ID.
So, I changed that .py, uploaded again to my "dest" app ID. To be sure I made this dummy python app as default and left it as that.
Then on the source app ID I tried the DS copy again, and all the copy jobs were kicked off OK - so it seems to have fixed it.