I was wondering if anything had changed recently in relation to uploading blobs to appengine from external applications? What used to work perfectly only 3 months ago is now hanging when doing a http post to upload the blob.
The code (see below) which was working fine previously consists in fetching a pull queue from AppEngine (using the REST API), doing some stuff with this task received and then uploading back the result as a Blob on AppEngine. The url to upload the blob to is created by appengine using blobstoreService.createUploadUrl("/upload");
and is of the form:
http://myapp.appspot.com/_ah/upload/AMmfu6aAHnkuS4ngyRJDn7urFFZeBxb_-3P-r7RY9udMvRjLWkEZNJMgUX1DFczNVi-NhIxcFat2AEPXs2IRJ0AOmznSMgcrCKmL7mGAmS7nqtr-UyYFkglD88BwCfzIui9M2yez7DSQ/ALBNUaYAAAAAUGRlEwpeGEc5ozp8Z8sDO33qgCi2AiIE/
I had a look at the logs on AppEngine and it seems like the servlet in charge of /upload isn't being triggered.
I'm honestly out of ideas at this stage, any help would be greatly appreciated ! :-)
Cheers,
Killian
public boolean uploadAsBlob(String dataToWrite, String uploadURL) {
try {
BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter(tempFileLocation));
bufferedWriter.write(dataToWrite);
bufferedWriter.newLine();
bufferedWriter.close();
MultipartEntity entity = new MultipartEntity();
entity.addPart(blobFileName, new FileBody(new File(tempFileLocation)));
HttpPost method = new HttpPost(uploadURL);
method.setEntity(entity);
final HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 10000);
DefaultHttpClient httpclient = new DefaultHttpClient(httpParams);
//It hangs at the following line!
HttpResponse response = httpclient.execute(method);
if (response.getStatusLine().getStatusCode() == 200) {
logger.info("Uploaded blob to url: " + uploadURL);
return true;
} else {
logger.warning("Couldn't upload blob to url: " + uploadURL);
}
} catch (Exception e) {
logger.warning("Exception " + e.getMessage() + " occured while uploading blob to url:" + uploadURL);
logger.warning("Couldn't upload blob to url: " + uploadURL);
}
return false;
}
I have found that GAE has recently started to keep any GET parameters when invoking blobstoreService.createUploadUrl(). In my case:
http://www.myapp.com/BG?_=1354631578951
With this (unexpected) parameter, the created URL was:
http://www.myapp.com/_ah/upload/?_=1354631578951/AMmfu6YgVPoJzWXdbf70k6J0zdjEeRnnRJ2PYCb3Jgdwk3SqmKEnFyKgy_17CKwiqbC2HyO-FlPVX-C53W0LjHSywaq7YmLegD97uU-GrpWRdBdWbfKf0Dk/ALBNUaYAAAAAUL4L8iDS5E99f3Wky2p59wWpCD84AqoP/
Notice that the '_' parameter is still there. Removing the parameter (or maybe moving from GET to POST) fixed the problem.
Related
I have a Web API that I want to return a file and the file details.
HttpResponseMessage result = Request.CreateResponse(HttpStatusCode.OK);
var file = new FileStream(filePath, FileMode.Open, FileAccess.Read);
result.Content = new StreamContent(file);
result.Headers.Add("filename", "MyFile");
return result;
Within Angular I do the following:
$http.get(url).then(function(response) {
console.log(response.headers());
});
The response.headers() does not contain my header record.
What am I missing?
I've looked at other examples online and they are like this.
I needed to add the following line and change the name to x-filename
result.Headers.Add("Access-Control-Expose-Headers", "x-filename");
Currently I am using Ionic to develop a Facial Recognition app.
I am using Microsoft's Face API for the same.
My problem is I keep getting Error 400: Decoding error, image format unsupported.
After doing some research, I ran into this link
Quoting START,
When you're submitting a binary payload, you should not base64 encode it. Here's what you might do instead..
URI uri = builder.build();
HttpPost request = new HttpPost(uri);
request.setHeader("Ocp-Apim-Subscription-Key", "...");
File file = new File("...");
FileEntity reqEntity = new FileEntity(file, ContentType.APPLICATION_OCTET_STREAM);
request.setEntity(reqEntity);
Quoting END
With Ionic/AngularJS, the captured image using Device Camera or selected from Image Gallery is returned in the form of base64 string. So I looked for ways to decode the data into Binary using this gist but still failed.
Here's my controller code where I make the API call.
$scope.postToAPI = function () {
// make http call to MS cognitive API
var formData = new FormData();
formData.append('file', $scope.picture);
$http.post($scope.cognitiveServices.endpoint, formData, {
headers: { 'Content-Type': 'application/octet-stream', 'Ocp-Apim-Subscription-Key':$scope.cognitiveServices.apiKey }
}).success(function (data) {
console.log(JSON.stringify(data));
alert("Awesome");
}).error(function (err) {
alert("Fail ->" + err.code + " " + err.message);
console.log("Some error occured");
});
}
I have
tried setting transform:angular.identity and Content-Type: undefined
tried sending image without Form Data (just a random trial)
tried converting base64 string to Uint8 Array (by following the solution to this SO question)
All my attempts so far have failed. Can someone explain what I am doing wrong?
P.S: This is my first time with Ionic framework.
I'm implementing file download using AngularJS and WCF. My back-end is a .NET project hosted in IIS. The file is serialized as an array of bytes and then on the client side I utilize the File API to save the content.
To simplify the problem, back-end is like:
[WebInvoke(Method = "GET", UriTemplate = "FileService?path={path}")]
[OperationContract]
public byte[] DownloadFileBaseOnPath(string path)
{
using (var memoryStream = new MemoryStream())
{
var fileStream = File.OpenRead(path);
fileStream.CopyTo(memoryStream);
fileStream.Close();
WebOperationContext.Current.OutgoingResponse.Headers["Content-Disposition"] = "attachment; filename=\"Whatever\"";
WebOperationContext.Current.OutgoingResponse.ContentType = "application/octet-stream"; // treat all files as binary file
return memoryStream.ToArray();
}
}
And on client side, it just sends a GET request to get those bytes, converts in into a blob and save it.
function sendGetReq(url, config) {
return $http.get(url, config).then(function(response) {
return response.data;
});
}
Save the file then:
function SaveFile(url) {
var downloadRequest = sendGetReq(url);
downloadRequest.then(function(data){
var aLink = document.createElement('a');
var byteArray = new Uint8Array(data);
var blob = new Blob([byteArray], { type: 'application/octet-stream'});
var downloadUrl = URL.createObjectURL(blob);
aLink.setAttribute('href', downloadUrl);
aLink.setAttribute('download', fileNameDoesNotMatter);
if (document.createEvent) {
var event = document.createEvent('MouseEvents');
event.initEvent('click', false, false);
aLink.dispatchEvent(event);
}
else {
aLink.click();
}
setTimeout(function () {
URL.revokeObjectURL(downloadUrl);
}, 1000); // cleanup
});
}
This approach works fine with small files. I could successfully download files up to 64MB. But when I try to download a file larger than 64MB, the response.body is empty in Chrome. I also used Fiddler to capture the traffic. According to Fiddler, Back-end has successfully serialized the byte array and returned it. Please refer to the screenshot below.
In this example, I was trying to download a 70MB file:
And the response.data is empty:
Any idea why this is empty for file over 70MB? Though the response itself is more than 200MB, I do have enough memory for that.
Regarding to the WCF back-end, I know I should use Stream Mode when it comes to large files. But the typical use of my application is to download files less than 10MB. So I hope to figure this out first.
Thanks
Answer my own question.
Honestly I don't know what's going wrong. The issue still persists if I transfer it as a byte array. I eventually gave up this approach by returning a stream instead. Then on the client side, adding the following configuration
{responseType : blob}
and save it as a blob.
A server side application requires authorization on file download links. This means a normal <a ng-href="{{createLinkToFile()}}"> is no longer sufficient to get enough parameters passed to the server.
When trying to use a programmatic call to the file download, I get the response data back to Dart client application. Using a simple http GET:
var url = "http://example.com/file";
headers.putIfAbsent("Authorization", () => "bearer " + token;
_http.get(url: url, headers : headers);
The future returned by the GET will hold the data, but how do I instruct the browser to download it as a file, instead of just trying to keep it in memory?
Or is there a way to just do it in a normal link?
After downloading the data from the server like shown in Using Dart to Download a PNG File (Binary File) and displaying it not working you can create a download link like shown at http://blog.butlermatt.me/2014/03/dynamically-generating-download-files/
import 'dart:html';
void main() {
List body = [ 'Some test data ...\n'];
// Create a new blob from the data.
Blob blob = new Blob(body, 'text/plain', 'native');
// Create a data:url which points to that data.
String url = Url.createObjectUrlFromBlob(blob);
// Create a link to navigate to that data and download it.
AnchorElement link = new AnchorElement()
..href = url
..download = 'random_file.txt'
..text = 'Download Now!';
// Insert the link into the DOM.
var p = querySelector('#text');
p.append(link);
}
The code of Seth solves indeed part of the problem. To make it a bit more complete, I'm now using the following:
void doPdfFileRequest(String url) {
var request = new HttpRequest();
request.open('GET', url);
request.responseType = "blob";
request.withCredentials = false;
request.setRequestHeader("Accept", _httpAcceptHeader);
request.setRequestHeader("Authorization", "bearer " + token);
request.onReadyStateChange
.listen((r) => onData(request, "filename.pdf"));
request.send();
}
void onData(HttpRequest request, String filename) {
if (request.readyState == HttpRequest.DONE && request.status == 200) {
if (!isIE()) {
var contentType = request.getResponseHeader("content-type");
AnchorElement downloadLink = new AnchorElement(
href: Url.createObjectUrlFromBlob(request.response));
downloadLink.rel = contentType;
downloadLink.download = filename;
var event = new MouseEvent("click", view: window, cancelable: false);
downloadLink.dispatchEvent(event);
} else {
var href = Url.createObjectUrlFromBlob(request.response);
window.open(href, "_self");
}
}
}
A few things to notice. Instead of using the downloadLink.click(), a mouse event is constructed to ensure that it works on Firefox as well as on Safari and Chrome. Firefox seems not to handle the click() otherwise. Binding it to the DOM as is done in the code of Seth isn't necessary.
Internet Explorer doesn't understand the download attribute, so nothing will happen, therefore a window.open is used to at least have it work (though not ideal) on IE, it's redirecting to self to avoid being hit by the pop up blocker.
There are solutions that convert the result download result to Base64 first and put it in a data:mimetype href, using the blob this isn't necessary.
A nice way to set the filename on the file to download would be through the content disposition header, but this header is marked as unsafe, so cannot be used. The filename is now set in the code.
Another note, notice that a HttpRequest is used instead http.get(), The HttpRequest allows you to set the responseType, in this case blob, which can be transformed into a object url.
I try to call AppIdentityService#getDefaultGcsBucketName but it throws an exception: Response contained no data. According to the docs, the default bucket should be ready to use. Code:
//Field
private AppIdentityService service;
//in ctor
this.service = AppIdentityServiceFactory.getAppIdentityService();
//call inside a method
final String baseUrl = GCS_URL + "/" + service.getDefaultGcsBucketName() + "/" + fileName;
If you created your app prior to the 1.9.0 SDK release, you have to manually setup your default GCS bucket. Read this article for more information (I know its for PHP, but its applicable across all languages).