In my Spring Integration pipeline I am getting a XML payload and depending on the value of the attributes in the XML I have to generate a key and publish it to kafka.
return IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainer))
.wireTap(ACARS_WIRE_TAP_CHNL) // Log the raw message
.enrichHeaders(h ->h.headerFunction(KafkaHeaders.MESSAGE_KEY, m -> {
StringBuilder header = new StringBuilder();
Expression expression = new SpelExpressionParser().parseExpression("payload.Body.toString()");
//Expression expression = new SpelExpressionParser().parseExpression("m.payload.Body.ACIFlight.fltNbr.toString()");
String flightNbr = expression.getValue(String.class);
header.append(flightNbr);
return header.toString();
}))
.get();
XMl is
<?xml version="1.0" encoding="UTF-8"?>
<ns0:Envelope xmlns:ns0="http://www.exmaple.com/FlightLeg">
<ns0:Header>
<ns1:eventHeader xmlns:ns1="http://www.exmaple.com/header" eventID="659" eventName="FlightLegEvent" version="1.0.0">
<ns1:eventSubType>FlightLeg</ns1:eventSubType>
</ns1:eventHeader>
</ns0:Header>
<ns0:Body>
<ns1:ACIFlight xmlns:ns1="http://ual.com/cep/aero/ACIFlight">
<flightKey>1267:07042020:UA</flightKey>
<fltNbr>1267</fltNbr>
<fltLastLegDepDt>07042020</fltLastLegDepDt>
<carrCd>UA</carrCd>
</ns1:ACIFlight>
</ns0:Body>
</ns0:Envelope>
I am trying to get the fltNbr from this xml payload using spel. Please suggest
Updated
String flight = XPathUtils.evaluate(message.getPayload(), "/*[local-name() = 'fltNbr']",XPathUtils.STRING);
String DepDate = XPathUtils.evaluate(message.getPayload(), "/*[local-name() = 'fltLastLegDepDt']",XPathUtils.STRING);
return MessageBuilder.fromMessage(message).setHeader("key", flight+DepDate).build();
You can use the XPath Header Enricher.
XPath is also available as a Spel function, but you'd be better off using the enricher in this case.
public class XPathHeaderEnricher extends HeaderEnricher {
Here's a test case...
#Test
public void convertedEvaluation() {
Map<String, XPathExpressionEvaluatingHeaderValueMessageProcessor> expressionMap =
new HashMap<String, XPathExpressionEvaluatingHeaderValueMessageProcessor>();
XPathExpressionEvaluatingHeaderValueMessageProcessor processor = new XPathExpressionEvaluatingHeaderValueMessageProcessor(
"/root/elementOne");
processor.setHeaderType(TimeZone.class);
expressionMap.put("one", processor);
String docAsString = "<root><elementOne>America/New_York</elementOne></root>";
XPathHeaderEnricher enricher = new XPathHeaderEnricher(expressionMap);
Message<?> result = enricher.transform(MessageBuilder.withPayload(docAsString).build());
MessageHeaders headers = result.getHeaders();
assertThat(headers.get("one")).as("Wrong value for element one expression")
.isEqualTo(TimeZone.getTimeZone("America/New_York"));
}
Related
I am currently trying to make a system in which I can create a collection in solr based on a JSON schema I have. I want to do it all programmatically.
Plese refer the below code to create a new collection using solrj api.
final String[] solrUrl = { "http://localhost:8983/solr" };
final CloudSolrClient cloudSolrClient = new CloudSolrClient.Builder(Arrays.asList(solrUrl)).build();
final String collectionName = "collection2";
final int numShards = 2;
final int numReplicas = 2;
final int maxShardsPerNode = 2;
final String solrZKConfigName = "_default";
public String createCollection(){
final CollectionAdminRequest.Create adminRequest = CollectionAdminRequest.Create
.createCollection(collectionName, solrZKConfigName, numShards, numReplicas)
.setMaxShardsPerNode(maxShardsPerNode);
CollectionAdminResponse adminResponse = adminRequest.process(cloudSolrClient);
System.out.println(adminResponse);
}
cloudSolrClient.close();
Scenario:
User submit XML using WebApi and I want to store it in SQL database in XML Column and retrieve later using Ajax/WebApi
Question:
How should I store it in the database? With or without declaration/encoding? Or should I add the encoding when returning the XML?
public async IActionResult Post([FromBody]XDocument xml)
{
var entity = new MyDocument();
entity.Xml = xml.ToString(); //??
db.Documents.Add(entity);
db.SaveChanges();
return Created();
}
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
return Content(entity.Xml, "application/xml"); //missing xml declaration
}
My Observations:
XDocument.ToString() trims the XML declaration element:
var xml = XDocument.Load(#"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>");
xml.ToString(); //<Root><Child>Content</Child></Root>
It's easy to include it, but I tought that maybe it's for a reason.
Edge browser does not display the XML when the response does not include xml declaration:
public IActionResult Get()
{
return Content("<Root><Child>Content</Child></Root>", "application/xml")
}
When the response include xml declaration, but the encoding from the declaration does not match response encoding, it also fails with "Unable to switch encodings":
public IActionResult Get()
{
return Content(#"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>", "application/xml");
}
In order to make Edge browser to display the XML properly, I have to do folowing:
public IActionResult Get()
{
string xml = #"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>")
var xmlDoc = XDocument.Parse(xml);
return Content(xml, "application/xml", Encoding.GetEncoding(xmlDoc.Declaration.Encoding));
}
Since database also has some encoding, it's quite unclear to me, what is actually the right way.
I've identified these rules as best practices:
You should not store XML encoding in database
declaration is optional, but if present, it must not include encoding
Your WebApi should always return XML Declaration.
encoding is optional, but if present, it must match response encoding
Explanation:
It's clear that you don't want to store encoding in the database, since the database threats XML columns on it own. However, it makes sense to store XML declaration in the database, because it can contain other information (version, standalone).
Your REST service should always return XML declaration in the response. In case it is not stored in database, you can construct it at runtime.
public async IActionResult Post([FromBody]XDocument xml)
{
var entity = new MyDocument();
var declarationWithoutEncoding = new XDeclaration(xml.Declaration?.Version, null, xml.Declaration?.Standalone);
entity.Xml = $#"{declarationWithoutEncoding}\n\r{xml.ToString()}"
db.Documents.Add(entity);
db.SaveChanges();
return Created();
}
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
return Content(entity.Xml, "application/xml"); //in case XML Declaration is in database
}
alternativelly
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
XDocument xmlDoc = XDocument.Parse(entity.xml);
var declaration = new XDeclaration(xml.Declaration?.Version ?? "1.0", null, xml.Declaration?.Standalone ? "no");
//you may change enccoding information to match response encoding, or just skip it
//declaration.Encoding = ...
string xml = $#"{declaration}\n\r{xml.ToString()}"
return Content(xml, "application/xml");
}
Now I'm trying to convert Java List object to JSON array, and struggling to convert UTF-8 strings. I've tried all followings, but none of them works.
Settings.
response.setContentType("application/json");
PrintWriter out = response.getWriter();
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
final ObjectMapper mapper = new ObjectMapper();
Test#1.
// Using writeValueAsString
String json = ow.writeValueAsString(list2);
Test#2.
// Using Bytes
final byte[] data = mapper.writeValueAsBytes(list2);
String json = new String(data, "UTF-8");
Test#3.
// Using ByteArrayOutputStream with new String()
final OutputStream os = new ByteArrayOutputStream();
mapper.writeValue(os, list2);
final byte[] data = ((ByteArrayOutputStream) os).toByteArray();
String json = new String(data, "UTF-8");
Test#4.
// Using ByteArrayOutputStream
final OutputStream os = new ByteArrayOutputStream();
mapper.writeValue(os, list2);
String json = ((ByteArrayOutputStream) os).toString("UTF-8");
Test#5.
// Using writeValueAsString
String json = mapper.writeValueAsString(list2);
Test#6.
// Using writeValue
mapper.writeValue(out, list2);
Like I said, none of above works. All displays characters like "???". I appreciate your helps. I'm using Servlet to send JSON response to clients.
This problem only happens when I write java.util.List object. If I write single data object, e.g. customer object in below example, then there is no ??? characters, and UTF-8 is working with the following code.
PrintWriter out = response.getWriter();
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = ow.writeValueAsString(customer);
out.print(json);
The answer was very simple. You need to specify UTF-8 charset encoding in response.setContentType too.
response.setContentType("application/json;charset=UTF-8");
Then, many of above code will work correctly. I will leave my question as is, since it will show you several ways of writing JSON to clients.
On RequestMapping in Controller:
#RequestMapping(value = "/user/get/sth",
method = RequestMethod.GET,
produces = { "application/json;**charset=UTF-8**" })
I am trying to use resteasy. While I am able to do send a mixed multipart as a request to a webservice, I am unable to do get a mixed multipart in the response.
For eg: Requesting for a file (byte[] or stream) and the file name in a single Response.
Following is what I have tested:
Service code:
#Path("/myfiles")
public class MyMultiPartWebService {
#POST
#Path("/filedetail")
#Consumes("multipart/form-data")
#Produces("multipart/mixed")
public MultipartOutput fileDetail(MultipartFormDataInput input) throws IOException {
MultipartOutput multipartOutput = new MultipartOutput();
//some logic based on input to locate a file(s)
File myFile = new File("samplefile.pdf");
multipartOutput.addPart("fileName:"+ myFile.getName(), MediaType.TEXT_PLAIN_TYPE);
multipartOutput.addPart(file, MediaType.APPLICATION_OCTET_STREAM_TYPE);
return multipartOutput;
}
}
Client code:
public void getFileDetails(/*input params*/){
HttpClient client = new DefaultHttpClient();
HttpPost postRequest = new HttpPost("urlString");
MultipartEntity multiPartEntity = new MultipartEntity();
//prepare the request details
postRequest.setEntity(multiPartEntity);
HttpResponse response = client.execute(postRequest);
HttpEntity returnEntity = response.getEntity();
//extracting data from the response
Header header = returnEntity.getContentType();
InputStream is = returnEntity.getContent();
if (is != null) {
byte[] bytes = IOUtils.toByteArray(is);
//Can we see the 2 parts that were added?
//Able to get a single InputStream only, and hence unable to differentiate two objects in the response
//Trying to see the contents - printing as string
System.out.println("Output from Response :: " + new String(bytes));
}
}
The output is as follows - able to see 2 different objects with different content types, but unable to extract them separately.
Output from Response ::
--af481055-4e4f-4860-9c0b-bb636d86d639
Content-Type: text/plain
fileName: samplefile.pdf
--af481055-4e4f-4860-9c0b-bb636d86d639
Content-Length: 1928
Content-Type: application/octet-stream
%PDF-1.4
<<pdf content printed as junk chars>>
How can I extract the 2 objects from the response?
UPDATE:
Tried the following approach to extract the different parts - use the 'boundary' to break the MultipartStream; use the content type string to extract approp object.
private void getResponeObject(HttpResponse response) throws IllegalStateException, IOException {
HttpEntity returnEntity = response.getEntity();
Header header = returnEntity.getContentType();
String boundary = header.getValue();
boundary = boundary.substring("multipart/mixed; boundary=".length(), boundary.length());
System.out.println("Boundary" + boundary); // --af481055-4e4f-4860-9c0b-bb636d86d639
InputStream is = returnEntity.getContent();
splitter(is, boundary);
}
//extract subsets from the input stream based on content type
private void splitter(InputStream is, String boundary) throws IOException {
ByteArrayOutputStream boas = null;
FileOutputStream fos = null;
MultipartStream multipartStream = new MultipartStream(is, boundary.getBytes());
boolean nextPart = multipartStream.skipPreamble();
System.out.println("NEXT PART :: " + nextPart);
while (nextPart) {
String header = multipartStream.readHeaders();
if (header.contains("Content-Type: "+MediaType.APPLICATION_OCTET_STREAM_TYPE)) {
fos = new FileOutputStream(new File("myfilename.pdf"));
multipartStream.readBodyData(fos);
} else if (header.contains("Content-Type: "+MediaType.TEXT_PLAIN_TYPE)) {
boas = new ByteArrayOutputStream();
multipartStream.readBodyData(boas);
String newString = new String( boas.toByteArray());
} else if (header.contains("Content-Type: "+ MediaType.APPLICATION_JSON_TYPE)) {
//extract string and create JSONObject from it
} else if (header.contains("Content-Type: "+MediaType.APPLICATION_XML_TYPE)) {
//extract string and create XML object from it
}
nextPart = multipartStream.readBoundary();
}
}
Is this the right approach?
UPDATE 2:
The logic above seems to work. But got another block, when receiving the RESPONSE from the webservice. I could not find any references to handle such issues in the Response.
The logic assumes that there is ONE part for a part type. If there are, say, 2 JSON parts in the response, it would be difficult to identify which part is what. In other words, though we can add the part with a key name while creating the response, we are unable to extract the key names int he client side.
Any clues?
You can try the following approach...
At the server side...
Create a wrapper object that can encapsulate all types. For eg., it could have a Map for TEXT and another Map for Binary data.
Convert the TEXT content to bytes (octet stream).
Create a MetaData which contains references to the Key names and their type. Eg., STR_MYKEY1, BYTES_MYKEY2. This metadata can also be converted into octet stream.
Add the metadata and the wrapped entity as parts to the multipart response.
At the Client side...
Read the MetaData to get the key names.
Use the key name to interpret each part. Since the Keyname from the metadata tells if the original data is a TEXT or BINARY, you should be able to extract the actual content with appropriate logic.
The same approach can be used for upstream, from client to service.
On top of this, you can compress the TEXT data which will help in reducing the content size...
i have the following code but don't understand where am i getting wrong. There are five items in the xml file but this code shows none.
public static List<string> LoadLedgersString()
{
List<string> ListLedgerStringRecords = new List<string>();
// Execute the query using the LINQ to XML
XDocument doc = XDocument.Load(#"Ledgers.xml");
var records = from r in doc.Element("Ledgers").Elements("Ledger") select r;
foreach (var record in records)
{
string lLedgerString = record.Element("Name").Value;
ListLedgerStringRecords.Add(lLedgerString);
}
return ListLedgerStringRecords;
}
The code that is calling and utilizing its result is following
List<string> ledgerList = new List<string>();
ledgerList = DAL_Ledgers.LoadLedgersString();
ledgerListView.DataContext = ledgerList;
ICollectionView view =
CollectionViewSource.GetDefaultView(ledgerList);
new TextSearchFilter(view, this.searchTextBox);
This will probably be a problem with namespacing or with pathing.
Please post some example XML, then we can try to help.