I am trying to make a Google App Engine that takes a file from Storage, and the converts arbitrary audio file to FLAC. App Engine, however, does not permit running executables.
My current code looks something like this:
cmd := exec.CommandContext(ctx, `./ffmpeg`,
`-i`, `pipe:0`, `pipe:1`, `-ac`, `1`, `-c:a`, `flac`, `-f`, `flac`)
cmd.Stdin = rc
cmd.Stdout = wc
var errOutput bytes.Buffer
cmd.Stderr = &errOutput
err = cmd.Run()
fmt.Printf("Running ffmpeg: %v... \nstderr: %s\n", err, errOutput.String())
Tried looking for go packages (e.g. https://github.com/xfrr/goffmpeg) that do this, but all that I found seem to use the same "run executable on inputs" paradigm as the code above.
How should I approach this? Is there a package that provides bindings to FFMPEG or similar?
You can use ffmpeg functionality in App Engine importing ffmpeg-python: Python bindings for FFmpeg or for example Libav.
Please note that there are two steps to use third-party library with App Engine:
Add library to requirements file, that will be used during app build: ffmpeg-python==0.1.17
Add it to the app code: import ffmpeg
Examples of video encoding apps for App Engine:
Scalable Video Transcoding With App Engine Flexible.
Distributed FFMPEG using Google App Engine.
Related
I am getting following error when I am running "goapp serve myapp/" from Myproject folder inside src.
go-app-builder: Failed parsing input: parser: bad import "unsafe" in
github.com/gorilla/websocket/client.go from GOPATH
my file structure is some something like this
$GOPATH/src
|-github.com/gorilla/websocket
|-MyProject
|-myapp
|-app.yaml
|-mainApp.go (which contain init function and part of app package)
Please let me know how to correct this.
on a related subject, I read that google app provide websocket support only in paid app. Is there a way for me to test my website before getting into payment mode? Or is there a better option that google app engine?
I am developing an API in golang directly on the "App Engine flexible environment" (formerly known as "Managed VMs").
So far, i have been using this kind of import in my .go files :
import (
"appengine"
"appengine/datastore"
...)
Recently I decided to use Google Cloud Storage to store images. It requires the import of "cloud.google.com/go/storage". My problem is that i'm unable to deploy the app with this import (not found), or any other short version ("go/storage") like I use for the appengine import.
After much research, I found this : https://github.com/golang/appengine#user-content-3-update-code-using-deprecated-removed-or-modified-apis
It specifies how to migrate an application using short imports (deprecated, like mine) to full imports (with repository explicit like "google.golang.org/appengine")
I followed the procedure and used the script they provide to update my code (aefix). They also say to add this line to my app.yaml file :
vm : true
If I do, I got this error message running 'gcloud app deploy' :
ERROR: (gcloud.app.deploy) Your application does not satisfy all of the requirements for a runtime of type [go]. Please correct the errors and try again.
If I don't, none of my imports are working and I get the following error :
can't find import: "google.golang.org/appengine/datastore"
Here is my app.yaml file :
runtime: go
api_version: go2
#vm : true
handlers:
- url: /.*
script: _go_app
Of course, all the imports are on the server under $GOPATH/src/ so they're not really missing, more badly referenced I guess.
I'm stuck on this problem since several days, any help of any kind would be appreciated !
Thanks
So sorry - we have some docs to go update. You cannot use the golang/appengine package with the App Engine flexible environment. The aefix tool won't work here either. Instead of the App Engine Go SDK, you want to use the Go client library here:
https://github.com/GoogleCloudPlatform/google-cloud-go
If you were previously using vm:true, you will need to upgrade to env:flex - the instructions (and the note on the go app engine library) are here:
https://cloud.google.com/appengine/docs/flexible/go/upgrading
Let me know if you have any questions!
There's a similar question that was recently responded to on Stackoverflow here: Google Cloud Storage Client not working on dev appserver
The solution was to either upgrade the SDK to 1.8.8 or use the previous revision of the GCS client library which didn't have the bug instead.
I'm currently using 1.8.8 and have tried downloading multiple revisions and /_ah/gcs doesn't load for me. After using up a significant number of my backend instances trying to understand how GCS and app engine work together, it'd be great if I could just test it on my local server instead!
When I visit localhost:port/_ah/gcs I get a 404 not found error.
Just a heads up, to install the library all I did was drag and drop the code into my app folder. I'm wondering if maybe I skipped a setup step? I wasn't able to find the answer in the documentation!
thanks!!
Note
To clarify this is my first week using GCS, so my first time trying to use the dev_server to host it.
I was able to find the google cloud storage files I wrote to a bucket locally at:
localhost:port/_ah/gcs/bucket_name/file_suffix
Where port is by default 8080, and the file was written to: /bucket_name/file_suffix
For those trying to understand the full process of setting up a simple python GAE app and testing local writes to google cloud storage:
1. Follow the google app engine "quickstart":
https://cloud.google.com/appengine/docs/standard/python/quickstart
2. Run a local dev server with:
dev_appserver.py app.yaml
3. If using python, follow "App Engine and Google Cloud Storage Sample":
https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/app-engine-cloud-storage-sample
If you run into "ImportError: No module named cloudstorage" you need to create a file named appengine_config.py
touch appengine_config.py
and add to it:
from google.appengine.ext import vendor
vendor.add('lib')
GAE runs this script automatically when starting your local dev server with dev_appserver.py app.yaml, and it is necessary to run this script for GAE to find the cloudstorage library in your lib/ folder
4. "Writing a file to cloud storage" from the same tutorial:
def create_file(self, filename):
"""Create a file."""
self.response.write('Creating file {}\n'.format(filename))
# The retry_params specified in the open call will override the default
# retry params for this particular file handle.
write_retry_params = cloudstorage.RetryParams(backoff_factor=1.1)
with cloudstorage.open(
filename, 'w', content_type='text/plain', options={
'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
retry_params=write_retry_params) as cloudstorage_file:
cloudstorage_file.write('abcde\n')
cloudstorage_file.write('f'*1024*4 + '\n')
self.tmp_filenames_to_clean_up.append(filename)
with cloudstorage.open(
filename, 'w', content_type='text/plain', options={
'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
retry_params=write_retry_params) as cloudstorage_file:
cloudstorage_file.write('abcde\n')
cloudstorage_file.write('f'*1024*4 + '\n')
Where filename is /bucket_name/file_suffix
4. After calling create_file via a route in your WSGI app, your file will be available at:
localhost:port/_ah/gcs/bucket_name/file_suffix
Where port is by default 8080, and the file was written to: /bucket_name/file_suffix
Postscript
Unfortunately, I did not find either 3) or 4) in their docs, so I hope this helps someone get set up more easily in the future.
To access gcs objects on dev_appserver, you must specify the bucket & object name, i.e. /_ah/gcs/[bucket]/[object].
The storage simulator for the local server is working in later versions of the SDK. For Java, one may choose to follow a dedicated tutorial: “App Engine and Google Cloud Storage Sample”.
I'm new to Google App Engine and Python. I've almost completed a project, but can't get the get_serving_url() function to work. I've stripped everything down to the most basic functionality, following the documentation. And yet I still get a 500 error from the server. Any thoughts? Here is the code:
from google.appengine.api import images
....
class Team(db.Model):
avatar = db.BlobProperty()
....
def to_dict(self):
....
image_url = images.get_serving_url(self.avatar.key())
The last line is the problem...commenting it out makes the app run fine. But it is copied almost directly from the documentation. I should note that I can download the avatar blob directly with:
class GetTeamAvatar(webapp2.RequestHandler):
def post(self):
team_id = self.request.get('team_id')
team = Team.get_by_id(long(team_id))
self.response.write(team.avatar)
So I know it is stored correctly. I do not have PIL on my machine...is that the issue? The datastore's image API says it has PIL locally so if I'm deploying my app it shouldn't matter, right? I have Python 3.3 and apparently PIL stopped at 2.6.
Python appengine run time is 2.7, (OK and 2.5) so don't even try to work with 3.x.
Secondly get_serving_URL is a method you call with a BlobStore entity key not a BlobProperty.
You are confusing two different things here.
I would concentrate on getting your code to run locally correctly under 2.7 first, and PIL is available for 2.7.
I'm very impressed if you're trying to deploy your app without even testing it locally.
One thing you'll need to do is make PIL available in your app.yaml via the libraries attribute.
On the tutorial for Apps Script/Appengine:
https://developers.google.com/apps-script/articles/appengine
When trying to run google_appengine/dev_appserver.py google-apps-script/ the response is:
WARNING 2012-09-06 14:56:33,570 rdbms_mysqldb.py:74] The rdbms API is not available because the MySQLdb library could not be loaded.
INFO 2012-09-06 14:56:33,840 appengine_rpc.py:163] Server: appengine.google.com
CRITICAL 2012-09-06 14:56:33,842 appcfg.py:561] The api_version specified in app.yaml (1) is not supported by this release of the SDK. The supported api_versions are ['3', 'go1'].
I have tried the following app.yaml, but it doesn't work.
application: google-apps-script-tutorial
version: 1
runtime: go
api_version: go1
handlers:
- url: /*
script: _go_app
Also with - url: /rpc and it doesn't work. Since the code is Python is it possible to get App script and Go linked up in app engine?
The code for that tutorial is in Python and Javascript. If you want to use the go runtime, you will have to rewrite the Python portion in Go.
That example demonstrates the use of a Google Apps Script frontend with a Google App Engine (GAE) backend written in Python. GAE currently runs apps written in Java, Python, Go and PHP.
That particular Python backend accepts and produces messages in JSON format. Therefore, to link Apps Script and Go similarly, using GAE or not, you would need to replicate the functionality of the Python backend using, probably, the net/http library and the encoding/json library.
For examples of using these libraries together, have a look at this, this and this.
For examples on using Go with GAE, have a look at this and this.
Hope that helps.