I have a Spring Boot Rest End Point defined in an interface to download an image
#GetMapping(value = "/{name}")
ResponseEntity<ByteArrayResource> getFileByName(#PathVariable("name") String name);
And I use Feign Builder to invoke this end point.
Feign.builder()
.client(new ApacheHttpClient())
.contract(new SpringMvcContract())
.decoder(new JacksonDecoder())
.encoder(new JacksonEncoder())
.target(clazz, url)
On invoking, I get below error
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('�' (code 65533 / 0xfffd)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (BufferedReader); line: 1, column: 2]
When I try to invoke the end point directly from Insomnia, it works fine. But fails through Feign Builder. The response content type is image/jpeg
Is there any specific decoder in feign to handle ByteArrayResource? I tried ResponseEntityDecoder, StreamDecoder and JacksonDecoder. None of it works.
On debugging, I see that Jackson ObjectMapper readValue fails. I tried changing the return type from ByteArraySource to byte[], didn't work either.
Any help?
I wrote my own little decoder and the problem was resolved. Below is the decoder
private Decoder byteArrayResourceDecoder() {
Decoder decoder = (response, type) -> {
if (type instanceof Class && ByteArrayResource.class.isAssignableFrom((Class) type)) {
return StreamUtils.copyToByteArray(response.body().asInputStream());
}
return new JacksonDecoder().decode(response, type);
};
return new ResponseEntityDecoder(decoder);
}
Hope this template helps others who has similar issues. Would have expected Feign to have decoder that supports all return types.
Thanks Maz - your solution helped me.
I modified your solution for my needs to read Spring StreamingResponseBody
1.) Create the decoder wrapper that either returns JacksonDecoder (Default) or reads the responsebody into a byte array.
Decoder decoder = (response, type) -> {
Map<String, Collection<String>> headers = response.headers();
Collection<String> contentType = null;
for (String x : headers.keySet()){
if ("content-type".equals(x.toLowerCase())){
contentType = headers.get(x);
}
}
if (contentType == null || contentType.stream().filter(x -> x.contains("application/json")).findFirst().isPresent()) {
return new JacksonDecoder(getMapper()).decode(response, type);
}
InputStream initialStream = response.body().asInputStream();
byte[] buffer = new byte[512];
byte[] result = null;
try(ByteArrayOutputStream out = new ByteArrayOutputStream()) {
try {
int length = 0;
while ((length = initialStream.read(buffer)) != -1) {
out.write(buffer, 0, length);
}
} finally {
out.flush();
}
result = out.toByteArray();
} finally {
initialStream.close();
}
return result;
};
2.) Use the custom decoder with the Feign.Builder
Feign.Builder builder = Feign.builder()
// --
.decoder(decoder)
// --
openfeignfeignspringstreamingresponsebody
Related
I am trying to create an app where user can upload a text file, and gets the altered text back.
I am using React as FE and ASP.NET Core for BE and Azure storage for the database storage.
This is how my HomeController looks like.
I created a separate "UploadToBlob" method, to post the data
public class HomeController : Controller
{
private readonly IConfiguration _configuration;
public HomeController(IConfiguration Configuration)
{
_configuration = Configuration;
}
public IActionResult Index()
{
return View();
}
[HttpPost("UploadFiles")]
//OPTION B: Uncomment to set a specified upload file limit
[RequestSizeLimit(40000000)]
public async Task<IActionResult> Post(List<IFormFile> files)
{
var uploadSuccess = false;
string uploadedUri = null;
foreach (var formFile in files)
{
if (formFile.Length <= 0)
{
continue;
}
// read directly from stream for blob upload
using (var stream = formFile.OpenReadStream())
{
// Open the file and upload its data
(uploadSuccess, uploadedUri) = await UploadToBlob(formFile.FileName, null, stream);
}
}
if (uploadSuccess)
{
//return the data to the view, which is react display text component.
return View("DisplayText");
}
else
{
//create an error component to show there was some error while uploading
return View("UploadError");
}
}
private async Task<(bool uploadSuccess, string uploadedUri)> UploadToBlob(string fileName, object p, Stream stream)
{
if (stream is null)
{
try
{
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");
// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
//Create a unique name for the container
string containerName = "textdata" + Guid.NewGuid().ToString();
// Create the container and return a container client object
BlobContainerClient containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);
string localPath = "./data/";
string textFileName = "textdata" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, textFileName);
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(textFileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
FileStream uploadFileStream = File.OpenRead(localFilePath);
await blobClient.UploadAsync(uploadFileStream, true);
uploadFileStream.Close();
}
catch (StorageException)
{
return (false, null);
}
finally
{
// Clean up resources, e.g. blob container
//if (blobClient != null)
//{
// await blobClient.DeleteIfExistsAsync();
//}
}
}
else
{
return (false, null);
}
}
}
but the console throws errors, saying "'ControllerBase.File(byte[], string)' is a method, which is not valid in the given context (CS0119)"
And because of this error, another error follows "'HomeController.UploadToBlob(string, object, Stream)': not all code paths return a value (CS0161)"
my questions are
Is it a better idea to create a separate method like I did?
how can I resolve the issue regarding the "File" being valid inside of the UploadToBlob method?
If I want to add the file type validation, where should it happen? t.ex. only text file is alid
If I want to read the text string from the uploaded text file, where should I call the
string contents = blob.DownloadTextAsync().Result;
return contents;
How can I pass down the "contents" to my react component? something like this?
useEffect(() => {
fetch('Home')
.then(response => response.json())
.then(data => {
setForcasts(data)
})
}, [])
Thanks for helping this super newbie with ASP.NET Core!
1) It is ok to put uploading into separate method, it could also be put into a separate class for handling blob operations
2) File is the name of one of the controllers methods, if you want to reference the File class from System.IO namespace, you need to fully qualify the name
FileStream uploadFileStream = System.IO.File.OpenRead(localFilePath);
To the other compile error, you need to return something from the UploadToBlob method, now it does not return anything from the try block
3) File type validation can be put into the controller action method
4) it depends on what you plan to do with the text and how are you going to use it. Would it be a new action of the controller (a new API endpoint)?
5) you could create a new API endpoint for downloading files
UPDATE:
For word replacement you could use a similar method:
private Stream FindMostFrequentWordAndReplaceIt(Stream inputStream)
{
using (var sr = new StreamReader(inputStream, Encoding.UTF8)) // what is the encoding of the text?
{
var allText = sr.ReadToEnd(); // read all text into memory
// TODO: Find most frequent word in allText
// replace the word allText.Replace(oldValue, newValue, stringComparison)
var resultText = allText.Replace(...);
var result = new MemoryStream();
using (var sw = new StreamWriter(result))
{
sw.Write(resultText);
}
result.Position = 0;
return result;
}
}
it would be used in your Post method this way:
using (var stream = formFile.OpenReadStream())
{
var streamWithReplacement = FindMostFrequentWordAndReplaceIt(stream);
// Upload the replaced text:
(uploadSuccess, uploadedUri) = await UploadToBlob(formFile.FileName, null, streamWithReplacement);
}
You probably have this method inside MVC controller in which File method exists. Add in your code System.IO.File instead of File
In the controller I have below function:
#RequestMapping(value = "administrator/listAuthor/{authorName}/{pageNo}", method = { RequestMethod.GET,
RequestMethod.POST }, produces = "application/json")
public List<Author> listAuthors(#PathVariable(value = "authorName") String authorName,
#PathVariable(value = "pageNo") Integer pageNo) {
try {
if (authorName == null) {
authorName = "";
}
if (pageNo == null) {
pageNo = 1;
}
return adminService.listAuthor(authorName, pageNo);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
This function fetches and returns data from mysql database based on "authorName" and "pageNo". For example, when "authorName = a" and "pageNo = 1" I have:
Data I get when "authorName = a" and "pageNo = 1"
Now I want to set "authorName" as ""(empty string), so that I can fetch all the data from mysql database (because the SQL statement "%+""+%" in backend will return all the data).
What can I do if I want to set authorName = empty string?
http://localhost:8080/spring/administrator/listAuthor/{empty string}/1
Thanks in advance!
I don't think that you can encode empty sting to url, what I suggest you to do is to declare some constant that will be your code to empty string - such as null.
Example:
administrator/listAuthor/null/90
Afterwards , on server side, check if authorName is null and set local parameter with empty stirng accordingly.
I use OpenSAML. I get a problem it cannot validate sign
I try to validate after I got I get Response, It' Ok. But when I convert to XML and unmarshall, it cannot validate it.
My Code
Element responseElement = null;
ResponseMarshaller marshaller = new ResponseMarshaller();
try {
responseElement = marshaller.marshall(response); //
/*
* SAML utility = new SAML(); Document doc =
* utility.asDOMDocument(response); // Element target =
* doc.getDocumentElement();
*/
Signer.signObjects(signatures);
} catch (Exception ex) {
ex.printStackTrace();
}
When I use ResponseElement to validate, it works.
MY Code Convert SAMLResponse to String
String samlXmlString = XMLHelper.prettyPrintXML(responseElement);
BasicParserPool parserPool = new BasicParserPool();
parserPool.setNamespaceAware(true);
Document document;
logger.debug("Parsing...");
document = parserPool.parse(new StringReader(samlXmlString));
Element responseElement = document.getDocumentElement();
logger.debug("responseElement is:\n{}", convertToString(responseElement));
Response response;
logger.debug("Unmarshalling...");
UnmarshallerFactory unmarshallerFactory = Configuration.getUnmarshallerFactory();
Unmarshaller unmarshaller = unmarshallerFactory.getUnmarshaller(responseElement);
response = (Response) unmarshaller.unmarshall(responseElement);
logger.debug("Successfully unmarshalled");
I use Response to validate. It doesn't work.
How can I validate it?
I am trying to upload an photo to Facebook from a Windows Phone Silverlight application using the Facebook Graph API but I am getting an error: (#324) Requires upload file. Can anyone see anything wrong in my code?
internal void PublishPhoto(System.IO.MemoryStream stream, string message, string accessToken)
{
var requestUriString = string.Format(
CultureInfo.InvariantCulture,
"https://graph.facebook.com/{0}/photos?access_token={1}&message={2}",
"me",
accessToken,
message);
var webRequest = WebRequest.CreateHttp(requestUriString);
webRequest.Method = "POST";
var boundary = "7db3d9202a1";
webRequest.ContentType = string.Format("multipart/form-data; boundary={0}", boundary);
webRequest.BeginGetRequestStream(new AsyncCallback(delegate (IAsyncResult result)
{
GetRequestStream(stream, boundary, result);
BeginGetResponse(webRequest);
}), webRequest);
}
private static void GetRequestStream(System.IO.MemoryStream imageStream, string boundary, IAsyncResult result)
{
var webRequest2 = result.AsyncState as HttpWebRequest;
using (var requestStream = webRequest2.EndGetRequestStream(result))
{
using (StreamWriter writer = new StreamWriter(requestStream))
{
writer.WriteLine("--{0}\r", boundary);
writer.WriteLine("Content-Disposition: form-data; filename=\"sketch.jpg\"\r");
writer.WriteLine("Content-Type: image/jpg\r");
byte[] buffer = imageStream.GetBuffer();
requestStream.Write(buffer, 0, buffer.Length);
writer.WriteLine("\r");
writer.WriteLine("--{0}--\r", boundary);
}
imageStream.Close();
}
}
private static void BeginGetResponse(HttpWebRequest webRequest)
{
webRequest.BeginGetResponse(new AsyncCallback(delegate(IAsyncResult result2)
{
var webRequest2 = result2.AsyncState as HttpWebRequest;
try
{
using (var response = webRequest2.EndGetResponse(result2))
{
using (var responseStream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
System.Diagnostics.Debug.WriteLine(reader.ReadToEnd());
}
}
}
}
catch (WebException we)
{
System.Diagnostics.Debug.WriteLine(we.Message);
using (var responseStream = we.Response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
var errorJson = reader.ReadToEnd();
var response = Newtonsoft.Json.JsonConvert.DeserializeObject<FacebookErrorResponse>(errorJson);
System.Diagnostics.Debug.WriteLine("Could not upload image to Facebook: {0}", response.Error.Message);
}
}
}
}), webRequest);
}
}
Try specifying a name of "source" as well as a filename in the Content-Disposition header, i.e.
writer.WriteLine("Content-Disposition: form-data; name=\"source\"; filename=\"sketch.jpg\"\r");
Ok, I was wrong the first time around, but now I have it.
The first problem, which we already took care of above, was that you were missing the "--" before each boundary and the "--" after the last boundary in the POST body.
The second problem is that you're not leaving a blank line after the MIME headers before writing the image content.
The third problem is that you're not flushing the writer before writing the image data to its underlying stream (unless silverlight on a phone is different from normal .NET in auto-flushing StreamWriters).
To sum up, this should work:
writer.WriteLine("--{0}\r", boundary);
writer.WriteLine("Content-Disposition: form-data; filename=\"sketch.jpg\"\r");
writer.WriteLine("Content-Type: image/jpg\r");
writer.WriteLine("\r");
writer.Flush();
byte[] buffer = imageStream.GetBuffer();
requestStream.Write(buffer, 0, buffer.Length);
writer.WriteLine("\r");
writer.WriteLine("--{0}--\r", boundary);
I wish to have a file upload form that in addition to the file selection input , also has other input fields like textarea, dropdown etc. The problem is that I cannot access any post parameters other than the file in my blobstore upload handler. I am using the following function call to get the parameter name but it always returns an empty screen.
par = self.request.get("par")
I found another question with a similar problem Uploading a video to google app engine blobstore. The answer to that question suggests a workaround solution to set the filename to the parameter you wish to read which is not sufficient for my needs. Is there a way to access other form parameters in the post method of blobstore upload handler?
Did you find the solution?
In my experience, when using form/multipart request doesn't include the other parameters and they have to be dug out manually.
This is how I dig out parameters out of request that is used to send a file.
import java.util.Map;
import java.util.HashMap;
import java.util.Iterator;
import javax.servlet.http.HttpServletRequest;
// for reading form data when posted with multipart/form-data
import java.io.*;
import javax.servlet.ServletException;
import org.apache.commons.fileupload.FileItemStream;
import org.apache.commons.fileupload.FileItemIterator;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import com.google.appengine.api.datastore.Blob;
// Fetch the attributes for a given model using rails conventions.
// We need to do this in Java because getParameterMap uses generics.
// We currently only support one lever: foo[bar] but not foo[bar][baz].
// We currently only pull the first value, so no support for checkboxes
public class ScopedParameterMap {
public static Map params(HttpServletRequest req, String model)
throws ServletException, IOException {
Map<String, Object> scoped = new HashMap<String, Object>();
if (req.getHeader("Content-Type").startsWith("multipart/form-data")) {
try {
ServletFileUpload upload = new ServletFileUpload();
FileItemIterator iterator = upload.getItemIterator(req); // this is used to get those params
while (iterator.hasNext()) {
FileItemStream item = iterator.next();
InputStream stream = item.openStream();
String attr = item.getFieldName();
if (attr.startsWith(model + "[") && attr.endsWith("]")) { // fetches all stuff like article[...], you can modify this to return only one value
int len = 0;
int offset = 0;
byte[] buffer = new byte[8192];
ByteArrayOutputStream file = new ByteArrayOutputStream();
while ((len = stream.read(buffer, 0, buffer.length)) != -1) {
offset += len;
file.write(buffer, 0, len);
}
String key = attr.split("\\[|\\]")[1];
if (item.isFormField()) {
scoped.put(key, file.toString());
} else {
if (file.size() > 0) {
scoped.put(key, file.toByteArray());
}
}
}
}
} catch (Exception ex) {
throw new ServletException(ex);
}
} else {
Map params = req.getParameterMap();
Iterator i = params.keySet().iterator();
while (i.hasNext()) {
String attr = (String) i.next();
if (attr.startsWith(model + "[") && attr.endsWith("]")) {
String key = attr.split("\\[|\\]")[1];
String val = ((String[]) params.get(attr))[0];
scoped.put(key, val);
// TODO: when multiple values, set a List instead
}
}
}
return scoped;
}
}
I hope this speedy answer helps, let me know if you have questions.