I have a Solr Schema and I am trying to do Facet Pivoting on it by passing multiple fields in facet.pivot parameter while making a call through SolrJ. My Solr response looks something like below:
"facet_pivot": {
"boolean_value,int_value": [
{
"field": "boolean_value",
"value": false,
"count": 1,
"pivot": [
{
"field": "int_value",
"value": 364,
"count": 1
}
]
},
{
"field": "boolean_value",
"value": true,
"count": 2,
"pivot": [
{
"field": "int_value",
"value": 406,
"count": 1
},
{
"field": "int_value",
"value": 409,
"count": 1
}
]
}
]
}
How, can I parse the above response using SolrJ in form of Nested objects i.e. PivotField having relation of which int_value falls under which boolean_value.
SolrJ Version tried: 4.10.4
Update:
When you make a call through SolrJ check the SolrQuery that gets generated. In my above case the SolrQuery was:
facet.pivot=boolean_value&facet.pivot=int_value
Solr considers above pivots as two different ones and you won't get nested pivoting. For Nested Pivoting your SolrQuery should have
facet.pivot=boolean_value,int_value
I had the same problem and wrote a simple tester application. It queries Solr and writes the pivot values to StdOut as hierarchical Strings. If there are any suggestions to implement that more nicely, please let me know. Here's the code:
public class Tester {
public static final String HIERARCHICAL_FACET_SEPARATOR = "/";
public static void main(String[] args) throws SolrServerException, IOException {
CloudSolrClient solr = ... ;
SolrQuery query = new SolrQuery(...);
query.setFacet(true);
query.addFacetPivotField(new String[]{"field1,field2,field3"});
QueryResponse result = solr.query(query);
NamedList<List<PivotField>> facetPivot = result.getFacetPivot();
List<String> parsedPivotResult = parsePivotResult(facetPivot);
parsedPivotResult.forEach((s) -> {
System.out.println(s);
});
}
private static List<String> parsePivotResult(final NamedList<List<PivotField>> pivotEntryList) {
final Set<String> outputItems = new HashSet<>();
for (final Entry<String, List<PivotField>> pivotEntry : pivotEntryList) {
System.out.println("Key: " + pivotEntry.getKey());
pivotEntry.getValue().forEach((pivotField) -> {
renderOutput(new StringBuilder(), pivotField, outputItems);
});
}
final List<String> output = new ArrayList<>(outputItems);
Collections.sort(output);
return output;
}
private static void renderOutput(final StringBuilder sb, final PivotField field, final Set<String> outputItems) {
final String fieldValue = field.getValue() != null ? ((String) field.getValue()).trim() : null;
final StringBuilder outputBuilder = new StringBuilder(sb);
if (field.getPivot() != null) {
if (outputBuilder.length() > 0) {
outputBuilder.append(HIERARCHICAL_FACET_SEPARATOR);
}
outputBuilder.append(fieldValue);
outputItems.add(new StringBuilder(outputBuilder).append(" (").append(field.getCount()).append(")").toString());
field.getPivot().forEach((subField) -> {
renderOutput(outputBuilder, subField, outputItems);
});
} else {
if (outputBuilder.length() > 0) {
outputBuilder.append(HIERARCHICAL_FACET_SEPARATOR);
}
outputBuilder.append(fieldValue);
outputItems.add(outputBuilder.append(" (").append(field.getCount()).append(")").toString());
}
}
}
Related
can I get the sum of amount if I have an array like the one below?
[{"_id":"5e154cf38c52231ee19f8",
"refunds":[
{"_id":"5e38f10a754fcf3d48015",
"reason":"refund 1",
"amount":50000,
]},
{"_id":"5e1578b48c52231ee19f8",
"refunds":[
{"_id":"5e37e09ef9ea5e3784043",
"reason":"refund 1",
"amount":100000,
{"_id":"5e37e12a02c27c14580a1",
"reason":"refund 2",
"amount":100000,
{"_id":"5e38f02b754fcf3d48015",
"reason":"refund 3",
"amount":50000,
]},
{"_id":"5e1578b48c52231ee19f8",
"refunds":[]
}]
I hope to get res = 300000
This should be relatively straightforward, we create a flat array of all refund objects using flatMap, then we'll use reduce to sum the total amount.
I've added another function, sumRefundAmountsII to workaround JavaScript environments that do not support.flatMap (pre ES2019)
const data = [ { "_id": "5e154cf38c52231ee19f8", "refunds": [ { "_id": "5e38f10a754fcf3d48015", "reason": "refund 1", "amount": 50000 } ] }, { "_id": "5e1578b48c52231ee19f8", "refunds": [ { "_id": "5e37e09ef9ea5e3784043", "reason": "refund 1", "amount": 100000 }, { "_id": "5e37e12a02c27c14580a1", "reason": "refund 2", "amount": 100000 }, { "_id": "5e38f02b754fcf3d48015", "reason": "refund 3", "amount": 50000 } ] }, { "_id": "5e1578b48c52231ee19f8", "refunds": [] }];
function sumRefundAmounts(array) {
// Get an array of all refunds.
const refunds = array.flatMap(obj => obj.refunds);
// Sum amount for each refund.
return refunds.reduce((sum, refund) => {
return sum + refund.amount;
}, 0);
}
console.log("Total amount:", sumRefundAmounts(data));
// Use this function if your JavaScript environment complains about .flatMap
function sumRefundAmountsII(array) {
// Use a work around if we don't have flatMap...
const refunds = [].concat.apply([], array.map(obj => obj.refunds));
// Sum amount for each refund.
return refunds.reduce((sum, refund) => {
return sum + refund.amount;
}, 0);
}
console.log("Total amount (no flat map):", sumRefundAmountsII(data));
Before answering ... I assume 2 things -
1st - you are looking answer in the Java. 2nd - And you can do things much easier way in java List :
Let we make 2 classes Refund & Composite (Containing List)-
Refund Class -
package com.sabre.ticketing.dhs.service.create.domain;
public class Refund {
String _id;
String reason;
int amount;
public Refund(String _id, String reason, int amount) {
this._id = _id;
this.reason = reason;
this.amount = amount;
}
public int getAmount() {
return amount;
}
}
And here is Composite Class as -
package com.sabre.ticketing.dhs.service.create.domain;
import java.util.List;
public class Composite {
String _id;
List<Refund> refunds;
public Composite(String _id, List<Refund> refunds) {
this._id = _id;
this.refunds = refunds;
}
public List<Refund> getRefunds() {
return refunds;
}
}
The calculation technique is in SumCalc class -
package com.sabre.ticketing.dhs.service.create.domain;
import java.util.ArrayList;
import java.util.List;
public class SumCalc {
public static void main(String[] args){
List<Composite> composites = List.of(new Composite("5e154cf38c52231ee19f8", List.of(new Refund("5e38f10a754fcf3d48015", "refund 1", 50000))),
new Composite("5e154cf38c52231ee19f8",
List.of(new Refund("5e37e09ef9ea5e3784043", "refund 1", 100000),
new Refund("5e37e12a02c27c14580a1", "refund 2", 100000),
new Refund("5e38f02b754fcf3d48015", "refund 3", 50000))),
new Composite("5e154cf38c52231ee19f8", new ArrayList<>()));
// Basically get you Json converted into composites list.. for simplicity and dont want to jackson thing i have initialized list as new ..
Integer finalSum = composites.stream()
.map(Composite::getRefunds)
.flatMap(List::stream)
.map(Refund::getAmount)
.reduce((sum, val) -> sum + val)
.orElse(0);
System.out.println("final Sum is " + finalSum);
}
}
Run the SumClac ...
final Sum is 300000
Process finished with exit code 0
This is my second question in a few days about working with spring data rest and HATEOAS links.
tl;dr
How can my application use the link for 'manyToOne' attribute to populate the 'current value' for an html select drop down?
"Spring data rest" generated links behave differently than with "old school ids".
Context
Converting an app to spring JPA and spring data rest.
Simple 'product/category' table structure per below
I've added a projection name 'full' to 'join/get' descriptions
Tables
*Product*
---------
Product ID
Product Name
Category ID
*Category*
---------
Category ID
Category Name
Spring Data Rest Domain Objects
Product
#Entity
#Table(name="products")
public class Products
{
private static final long serialVersionUID = 5697367593400296932L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
public long id;
public String product_name;
#ManyToOne(optional = false,cascade= CascadeType.MERGE)
#JoinColumn(name = "category_id")
private ProductCategory productCategory;
public Products(){}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
public Products(String product_name) {
this.product_name = product_name;
}
public String getProduct_name() {
return product_name;
}
public void setProduct_name(String product_name) {
this.product_name = product_name;
}
public ProductCategory getProductCategory()
{
return productCategory;
}
public void setProductCategory(ProductCategory pProductCategory)
{
productCategory = pProductCategory;
}
#Projection(name = "full", types = { Products.class })
interface VirtualProjection
{
String getProductName() ;
#Value("#{target.productCategory.category}")
String getCategory();
}
}
Product Category
#Entity
#Table(name="productcat")
public class ProductCategory implements Serializable{
private static final long serialVersionUID = 890485159724195243L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
public long id;
public String category;
#OneToMany(mappedBy = "productCategory", cascade = CascadeType.ALL)
#JsonBackReference
Set<Products> products;
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
public String getCategory() {
return category;
}
public void setCategory(String category) {
this.category = category;
}
public Set<Products> getProducts() {
return products;
}
}
Problem: Impedance Mismatch between list of Categories, value in object
When the angular client retrieves a product, the product 'category link' for the product category looks like this:
href" : "http://localhost:8080/api/rest/products/8/productCategory"
When the client retrieves a list of 'product categories', the link for the category looks like this:
href": "http://localhost:8080/api/rest/productcat/4"
Here's the "select" code. The UI will only show select if "product.productCategory" maches one of the value for 'options'.
<select ng-model="product.productCategory >
<option value="http://localhost:8080/api/rest/productcat/1">Foo Product Type
</option>
<option value="http://localhost:8080/api/rest/productcat/2">Bar Product Type
</option>
<option value="http://localhost:8080/api/rest/productcat/3">Baz Product Type
</option>
</select>
Question
How do I code my server API to make "generated links" client
friendly? What's the Springiest approach?
Should I change the API to receive, for update, "values" rather than "IDs/Links", i.e. the Category Name,rather than the ID? (i.e. and look up the category by name and update it on the server side)
What javascript/angular tools and techniques do you recommend for working with links?
I coded around the problem by 'de-referencing' the "productCategory" value via javascript,
- retrieving the URL in the "productCategory" attribute, e.g.:
http://localhost:8080/api/rest/products/8/productCategory
replacing the "productCategory" attribute/url with the values from the HATEOAS links.
pItem[pPropName] = data._links.self.href;
This approach brings a lot of complexity for a simple drop down. It doesn't smell right.
De-reference code
function dereferenceProperty(pItem,pPropName )
{
var myRef = pItem._links[pPropName];
if (myRef && myRef.href)
{
var myUrl = myRef.href;
$http.get(myUrl).success(function (data, status)
{
pItem[pPropName] = data._links.self.href;
});
}
}
Usage
dereferenceProperty(pProduct,"productCategory");
After this 'de-refrence':
Angular/Browser populated the drop down correctly
an HTTP PATCH update the API (and tables) correctly
I don't completely understand what do you mean, but if you need to get Product list within ProductCategory or you need to get implicitly ProductCategory within Product you can use, for example, projections:
#Projection(name = "withProducts", types = ProductCategory.class)
public interface WithProducts {
String getName();
Set<Product> getProducts();
}
#Projection(name = "withCategory", types = Product.class)
public interface WithCategory {
String getName();
BigDecimal getPrice();
ProductCategory getCategory();
}
Then if you GET http://localhost:8080/api/productCategories/1?projection=withProducts you retrieve something like this:
{
"_embedded": {
"products": [
{
"name": "Product1",
"category": {
"name": "Category1"
},
"price": 1,
"_links": {
"self": {
"href": "http://localhost:8080/api/products/1"
},
"product": {
"href": "http://localhost:8080/api/products/1{?projection}",
"templated": true
},
"category": {
"href": "http://localhost:8080/api/products/1/category"
}
}
},
{
"name": "Product2",
"category": {
"name": "Category1"
},
"price": 2,
"_links": {
"self": {
"href": "http://localhost:8080/api/products/2"
},
"product": {
"href": "http://localhost:8080/api/products/2{?projection}",
"templated": true
},
"category": {
"href": "http://localhost:8080/api/products/2/category"
}
}
}
]
},
"_links": {
"self": {
"href": "http://localhost:8080/api/productCategories/1/products?projection=withCategory"
}
}
}
Or if you GET http://localhost:8080/api/products/1/category?projection=withProducts you retrieve something like this:
{
"name": "Category1",
"products": [
{
"name": "Product1",
"price": 1
},
{
"name": "Product2",
"price": 2
}
],
"_links": {
"self": {
"href": "http://localhost:8080/api/productCategories/1"
},
"productCategory": {
"href": "http://localhost:8080/api/productCategories/1{?projection}",
"templated": true
},
"products": {
"href": "http://localhost:8080/api/productCategories/1/products"
}
}
}
Working example.
I am migrating my application from Lucene to Solr. Solr handles highlighting a lot better, however if for instance I search the key word "city", I would expect a response like:
{
"id":"fdc3833a-0e4f-4314-ba8c",
"title": "Paris is a beautiful <b>city</b>",
"description": "The <b>city</b> is a great example of......",
}
while I am getting the following response instead:
{
"id":"fdc3833a-0e4f-4314-ba8c",
"title": "Paris is a beautiful city",
"description": "The city is a great example of......",
}
"highlighting": {
"fdc3833a-0e4f-4314-ba8c": {
"title": [
"Paris is a beautiful <b>city</b>"
],
"description": [
"The <b>city</b> is a great example of......"
]
}
}
So as you can see, instead of getting the highlighted term within the result, I am getting an extra section called highlighting and that means that my Java code has to change.
My question is: how will I get the highlight snippets in SolrJ?
In SolrJ is possible to get the highlighted snippets using the following code:
public String getHighlightedText(final QueryResponse queryResponse, final String fieldName, final String docId) {
String highlightedText = "";
Map<String, Map<String, List<String>>> highlights = queryResponse.getHighlighting();
if (highlights!=null && MapUtils.isNotEmpty(highlights.get(docId))) {
List<String> snippets = highlights.get(docId).get(fieldName);
if (CollectionUtils.isNotEmpty(snippets)) {
highlightedText = getFragments(snippets);
}
}
return highlightedText;
}
private static final String getFragments(List<String> snippets){
StringBuilder fragments = new StringBuilder();
for (int i = 0; i < snippets.size(); i++) {
if (i > 0) {
fragments.append("............");
}
fragments.append(snippets.get(i));
}
return fragments.toString();
}
Please notice that this code will get you the best snippets for single-value fields while you will need some variations for multi-value fields.
I have one json file in my local path sendmail.json it contains data like this
{
"from": {
"datatype": "String",
"required": false
},
"subject": {
"datatype": "String",
"required": false
},
"text": {
"datatype": "String",
"required": false
},
"to": {
"datatype": "String",
"required": false
}
}
i want to deserialize this data in mvc and i want to pass this data to .js angular controller please provide the way how to do it.
If you want to deserialize JSON and sent it to the client side, you can do it like this.
[HttpGet]
public ActionResult mailData()
{
List<test> myDeserializedObjList = (List<test>)Newtonsoft.Json.JsonConvert.DeserializeObject(Request["jsonString"], typeof(List<test>));
return Json(jsonString, JsonRequestBehavior.AllowGet);
}
Hope it works for you..!
There is no need to deserialize the JSON into an object and then serialize the object into JSON to send it to the browser.
Why not send directly the JSON file to the browser?
public class MailController : Controller
{
// GET: Mail
public ActionResult Schema()
{
return File(this.Server.MapPath("~/App_Data/sendmail.json"), "application/json");
}
}
EDIT:
Anyway here you have what you asked for:
public class MailController : Controller
{
// GET: Mail
public ActionResult Schema()
{
using (var reader = new StreamReader(this.Server.MapPath("~/App_Data/sendmail.json")))
{
var json = reader.ReadToEnd();
var data = JsonConvert.DeserializeObject<Dictionary<string, PropertyDefinition>>(json);
return this.Json(data, JsonRequestBehavior.AllowGet);
}
}
}
public class PropertyDefinition
{
public string datatype;
public bool required;
}
It needs Newtonsoft.JSON nuget package
I would like to create an array with JSON-Builder.
Expected format:
{
"Header": {
"SomeKey" : "SomeValue"
}
"Data": [
{
"SomeKey" : "SomeValue"
},
{
"SomeKey" : "SomeValue"
}
]
}
My Code:
def builder = new groovy.json.JsonBuilder()
def root = builder {
Header {
"Typ" "update"
"Code" "UTF-8"
"TransaktionsNr" item.transactionNumber
"DatumZeit" new Date().format("dd.MM.yyyy HH:mm")
}
customers.each ({ customer->
"Data" {
"Email" customer.code
"Newsletter" customer.newsletterSubscribed
}
})
However whatever I do I only get one element in the Data section. I tried using [] instead of {}, but I still only get one element, what am I doing wrong?
That's duplicate key for JSON structure. There should not be duplicate key in the same hierarchy or they will override each other:
class Customer { String code; boolean newsletterSubscribed }
customers = [
new Customer(code:"11111", newsletterSubscribed:true),
new Customer(code:"22222", newsletterSubscribed:false)
]
def builder = new groovy.json.JsonBuilder()
def root = builder {
Header {
"Typ" "update"
"Code" "UTF-8"
"TransaktionsNr" 987
"DatumZeit" new Date().format("dd.MM.yyyy HH:mm")
}
customers customers.collect { customer ->
["Email":customer.code,
"Newsletter":customer.newsletterSubscribed]
}
}
assert builder.toString() == {"Header":{"Typ":"update","Code":"UTF-8","TransaktionsNr":987,"DatumZeit":"21.12.2012 13:38"},"Data":{"Email":"22222","Newsletter":false},"customers":[{"Email":"11111","Newsletter":true},{"Email":"22222","Newsletter":false}]}