I have mp3 files storaged in Google App Engine Cloud Storage and I want to get their durations.
I made this code with help from one guy here but unfortunately the class AudioSystem doesn't work with the Google App Engine Cloud Storage.
Does someone know a way to do it?
ListResult lr = gcsService.list(mybucketname, ListOptions.DEFAULT);
while (lr.hasNext() && playlistLength > 0) {
ListItem li = lr.next();
String filename = li.getName();
GcsService gcsService =
GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsInputChannel readChannel = gcsService.openPrefetchingReadChannel(new GcsFilename(mybucketName, fileName), 0, 1024 * 1024);
AudioInputStream audioInputStream;
try (InputStream in = Channels.newInputStream(readChannel)) {
audioInputStream = AudioSystem.getAudioInputStream(in);
}
long frames = audioInputStream.getFrameLength();
double durationInSeconds = (frames+0.0) / format.getFrameRate();
playlistLength-=(int)(durationInSeconds)/60;
Here is the error returned :
Error for /hello java.lang.NoClassDefFoundError: javax.sound.sampled.AudioSystem is a restricted class.
Please see the Google App Engine developer's guide for more details.
at com.google.apphosting.runtime.security.shared.stub.javax.sound.sampled.AudioSystem.<clinit>(AudioSystem.java)
Looking into the issue and the doc here, there's a good chance your issue is not solvable with this library, as it is currently restricted because it's making some kind of system call that the platform won't let you do.
You have multiple solutions available to you. I would suggest, when you upload the file, uploading an entity to the datastore containing the metadata and retrieving that instead.
You can also parse the file yourself and read the duration from it (mp3 being a pretty simple format, as Igor Artamonov is pointing out).
Related
I have a Flask app where a user can upload an image and the image is saved on a static folder on the filesystem.
Currently, I'm using Google App Engine for hosting and found that it's not possible to save to the static folder on the standard environment. Here is the code
def save_picture(form_picture,name):
picture_fn = name + '.jpg'
picture_path = os.path.join(app.instance_path, 'static/image/'+ picture_fn)
output_size = (1000,1000)
i = Image.open(form_picture)
i.thumbnail(output_size)
i.save(picture_path)
return picture_path
#app.route('/image/add', methods=['GET', 'POST'])
def addimage():
form = Form()
if form.validate_on_submit():
name = 'randomname'
try:
picture_file = save_picture(form.image.data,name)
return redirect(url_for('addimage'))
except:
flash("unsuccess")
return redirect(url_for('addimage'))
My question is if I change from standard to flex environment would it be possible to save to a static folder? If not what are the other hosting options that I should consider? Do you have any suggestions?
Thanks in advance.
following your's advice I'm changing to use Cloud Storage. i'm wondering what should i use from upload_from_file(), upload_from_filename() or upload_from_string(). the source_file takes data from form.photo.data from flask-wtform. i'm not successfully saving on the cloud storage yet. this is my code:
def upload_blob(bucket_name, source_file, destination_blob_name):
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(destination_blob_name)
blob.upload_from_filename(source_file)
return destination_blob_name
#app.route('/image/add', methods=['GET', 'POST'])
def addimage():
form = Form()
if form.validate_on_submit():
name = 'randomname'
try:
filename = 'foldername/'+ name + '.jpg'
picture_file = upload_blob('mybucketname', form.photo.data, filename)
return redirect(url_for('addimage'))
except:
flash("unsuccess")
return redirect(url_for('addimage'))
I have successfully able to save file on google cloud storage by changing the save_picture function just in case anyone have trouble with this in the future:
app.config['BUCKET'] = 'yourbucket'
app.config['UPLOAD_FOLDER'] = '/tmp'
def save_picture(form_picture,name):
picture_fn = secure_filename(name + '.jpg')
picture_path = os.path.join(app.config['UPLOAD_FOLDER'], picture_fn)
output_size = (1000,1000)
i = Image.open(form_picture)
i.thumbnail(output_size)
i.save(picture_path)
storage_client = storage.Client()
bucket = storage_client.get_bucket(app.config['BUCKET'])
blob = bucket.blob('static/image/'+ picture_fn)
blob.upload_from_filename(picture_path)
return picture_path
The problem with storing it to some folder is that it would live on that one instance and other instances would not be able to access it. Furthermore, instances in GAE come and go, so you would lose the image eventually.
You should use Google Cloud Storage for this:
from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket('bucket-id-here')
blob = bucket.get_blob('remote/path/to/file.txt')
blob.upload_from_string('New contents!')
https://googleapis.dev/python/storage/latest/index.html
With Flask and Appengine, Python3.7, I save files to a bucket in the following way, because I want to loop it for many files:
for key, upload in request.files.items():
file_storage = upload
content_type = None
identity = str(uuid.uuid4()) # or uuid.uuid4().hex
try:
upload_blob("f00b4r42.appspot.com", request.files[key], identity, content_type=upload.content_type)
The helper function:
from google.cloud import storage
def upload_blob(bucket_name, source_file_name, destination_blob_name, content_type="application/octet-stream"):
"""Uploads a file to the bucket."""
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(destination_blob_name)
blob.upload_from_file(source_file_name, content_type=content_type)
blob.make_public()
print('File {} uploaded to {}.'.format(
source_file_name,
destination_blob_name))
Changing from Google App Engine Standard Environment to Google App Engine Flexible Environment will allow you to write to disk, as well as to choose a Compute Engine machine type with more memory for your specific application [1]. If you are interested on following this path find all the relevant documentation from migrating a Python app here.
Nonetheless, as it was explained by user #Alex on his provided answer as instances are created (the number of instances is scaled up) or deleted (the number of instances is scaled down) according to your load, the better option in your particular case would be to use Cloud Storage. Find an example for uploading objects to Cloud Storage with Python here.
I am following the EXACT steps indicated here
https://cloud.google.com/translate/docs/glossary#create-glossary
to create a online glossary.
I am getting the following error
madan#cloudshell:~ (focused-pipe-251317)$ ./rungcglossary
{
"error": {
"code": 400,
"message": "Empty resource name.; Resource type: glossary",
"status": "INVALID_ARGUMENT"
}
}
Here is the body of my request.json
{
"languageCodesSet": {
"languageCodes": ["en", "en-GB", "ru", "fr", "pt-BR", "pt-PT", "es"]
},
"inputConfig": {
"gcsSource": {
"inputUri": "gs://focused-pipe-251317-vcm/testgc.csv"
}
}
}
The inputUri path i copied from the google cloud bucket file URI box.
I am not able to understand what the issue is. All I know is something is wrong with the inputUri string.
Please help.
Thanks.
I am a Google Cloud Technical Support Representative and we know that, for the moment, there is an issue with the REST API which is on track. I tried to reproduce your situation and while trying to create the glossary using directly the API I got the same issue as you.
After that, I have tried to create the glossary programmatically using a HTTP Triggered Python Cloud Function and everything went just right. In this manner your API will be called with the Cloud Functions service account.
I will attach the code of my Python Cloud function:
from google.cloud import translate_v3beta1 as translate
def create_glossary(request):
request_json = request.get_json()
client = translate.TranslationServiceClient()
## Set your project name
project_id = 'your-project-id'
## Set your wished glossary-id
glossary_id = 'your-glossary-id'
## Set your location
location = 'your-location' # The location of the glossary
name = client.glossary_path(
project_id,
location,
glossary_id)
language_codes_set = translate.types.Glossary.LanguageCodesSet(
language_codes=['en', 'es'])
## SET YOUR BUCKET URI
gcs_source = translate.types.GcsSource(
input_uri='your-gcs-source-uri')
input_config = translate.types.GlossaryInputConfig(
gcs_source=gcs_source)
glossary = translate.types.Glossary(
name=name,
language_codes_set=language_codes_set,
input_config=input_config)
parent = client.location_path(project_id, location)
operation = client.create_glossary(parent=parent, glossary=glossary)
result = operation.result(timeout=90)
print('Created: {}'.format(result.name))
print('Input Uri: {}'.format(result.input_config.gcs_source.input_uri))
The requirements.txt should include the following dependencies:
google-cloud-translate==1.4.0
google-cloud-storage==1.14.0
Do not forget to modify the code with your parameters
Basically, I have just followed the same tutorial as you, but for Python and I used Cloud Functions. My guess is that you can use App Engine Standard, as well.This may be an issue regarding the service account that are used to call this API. In case this doesn´t work for you let me know and I will try to edit my comment.
When I try to save imported entities from local storage it thrown exception here.
var extraMetadata = aspect.extraMetadata;
var uri = extraMetadata.uri || extraMetadata.id;
if (core.stringStartsWith(uri, baseUri)) {
uri = routePrefix + uri.substring(baseUri.length);
}
request.requestUri = uri;
if (extraMetadata.etag) {
request.headers["If-Match"] = extraMetadata.etag;
}
But if I get data from OData service directly it is saving correctly. Anything I am missing when importing data from local storage. I tried this solution but it didn't help me.
This is a bug we are tracking (#2574). I was hoping we'd fix it for v.1.4.12 but it looks like it will have to wait a cycle.
There is no good workaround. You can try to remember the extraMetadata yourself (in some sideband storage) and re-attach it when you re-import. Not fun I know. Sorry.
I would like to implement a mv (copy-in-the-cloud) operation on google cloud storage that is similar to how gsutil does it (http://developers.google.com/storage/docs/gsutil/commands/mv).
I read somewhere earlier that this involves a read and write (download and reupload) of the data, but I cannot find the passages again.
Is this the correct way to move a file in cloud storage, or does one have to go a level down to the boto library to avoid copying the data over the network for renaming the file?
istream = cloudstorage.open(src, mode='r')
ostream = cloudstorage.open(dst, content_type=src_content, mode='w')
while True:
buf = istream.read(500000)
if not buf:
break
ostream.write(buf)
istream.close()
ostream.close()
Update: I found the rest api that supports copy and compose operations and much more. It seems that there is hope that we do not have to copy data across continents to rename something.
Useful Links I have found sofar ...
Boto based approach: https://developers.google.com/storage/docs/gspythonlibrary
GCS Clinet Lib: https://developers.google.com/appengine/docs/python/googlecloudstorageclient/
GCS Lib: https://code.google.com/p/appengine-gcs-client
RAW JSON API: https://developers.google.com/storage/docs/json_api
Use the JSON API, there is a copy method. Here is the official example for Python, using the Python Google Api Client lib :
# The destination object resource is entirely optional. If empty, we use
# the source object's metadata.
if reuse_metadata:
destination_object_resource = {}
else:
destination_object_resource = {
'contentLanguage': 'en',
'metadata': {'my-key': 'my-value'},
}
req = client.objects().copy(
sourceBucket=bucket_name,
sourceObject=old_object,
destinationBucket=bucket_name,
destinationObject=new_object,
body=destination_object_resource)
resp = req.execute()
print json.dumps(resp, indent=2)
Most of the solutions I've come across for Sharepoint doc library uploads use the HTTP "PUT" method, but I'm having trouble finding a way to do this in Silverlight because it has restrictions on the HTTP Methods. I visited this http://msdn.microsoft.com/en-us/library/dd920295(VS.95).aspx to see how to allow PUT in my code, but I can't find how that helps you use an HTTP "PUT".
I am using client web-services, so that limits some of the Sharepoint functions available.
That leaves me with these questions:
Can I do an http PUT in Silverlight?
If I can't or there is another better way to upload a file, what is it?
Thanks
Figured it out!! works like a charm
public void UploadFile(String fileName, byte[] file)
{
// format the destination URL
string[] destinationUrls = {"http://qa.sp.dca/sites/silverlight/Answers/"+fileName};
// fill out the metadata
// remark: don't set the Name field, because this is the name of the document
SharepointCopy.FieldInformation titleInformation = new SharepointCopy.FieldInformation
{DisplayName =fileName,
InternalName =fileName,
Type = SharepointCopy.FieldType.Text,
Value =fileName};
// to specify the content type
SharepointCopy.FieldInformation ctInformation = new SharepointCopy.FieldInformation
{DisplayName ="XML Answer Doc",
InternalName ="ContentType",
Type = SharepointCopy.
FieldType.Text,
Value ="xml"};
SharepointCopy.FieldInformation[] metadata = { titleInformation };
// initialize the web service
SharepointCopy.CopySoapClient copyws = new SharepointCopy.CopySoapClient();
// execute the CopyIntoItems method
copyws.CopyIntoItemsCompleted += copyws_CopyIntoItemsCompleted;
copyws.CopyIntoItemsAsync("http://null", destinationUrls, metadata, file);
}
Many Thanks to Karine Bosch for the solution here: http://social.msdn.microsoft.com/Forums/en/sharepointdevelopment/thread/f135aaa2-3345-483f-ade4-e4fd597d50d4
What type of SharePoint deployment and what version of silverlight? If say it is an intranet deployment you could use UNC paths to access your document library in sharepoint and the savefiledialog/openfiledialog available in Silverlight 3.
http://progproblems.blogspot.com/2009/11/saveread-file-from-silverlight-30-in.html
or
http://www.kirupa.com/blend_silverlight/saving_file_locally_pg1.htm
Silverlight has restrictions on what it can do with local files, though I've read that silverlight 4 has some changes.
http://www.wintellect.com/CS/blogs/jprosise/archive/2009/12/16/silverlight-4-s-new-local-file-system-support.aspx