I am looking for pointers for libraries or methods that would be able to generate full text from the structured information returned by Wikidata - if possible in multiple languages.
To be clearer: from data like the one provided here (this is the JSON version) I would like to be able to generate text similar to the intro paragraph of the wikipedia page for the same item:
Orvieto Cathedral (Italian: Duomo di Orvieto; Cattedrale di Santa Maria Assunta) is a large 14th-century Roman Catholic cathedral dedicated to the Assumption of the Virgin Mary and situated in the town of Orvieto in Umbria, central Italy.
The reason is that the text is provided by Wikipedia for all those cases where a page exists, but I would like to have something also for the Wikidata items without a wikipedia page.
My problem #1 here is: I don't know what something like this is called, so I have no idea what to google for. Any pointers to start from are appreciated, including services or APIs.
This problem falls under the Data to text generation task. I do not know of any services that are currently offering solution. You can look at WEBNLG challenge which has the same objective and similar data. AFAIK mostly template based methods are used to automatically insert data from wikidata into wikipedia as text.
I have a Solr/Lucene set up where I have indexed a set of documents (MS Word files) and can happily search the content of these documents. However I would like to return a snippet from within the content of the document which shows where the matching line (+/- 5 words from the match term) is. I have tried to follow a range of Google hits but my indexing does not seem to have a direct access to the "content".
Can anyone give me some basic and simple pointers to where I might have made any errors on this - I have based all my work so far on the guidance and examples of the Solr Reference Guide - so I am not sure if the issue is in the search parameters or the original index.
I am doing this to create a clear set of user requirements for building an end solution rather than creating the end solution myself, so I am no expert on the tools and do not need to become one, just need to evidence what is possible with this tool set.
As MatsLindh noted above the issue was that the config was not drawing across the actual content of the Tika parse into a specific field, and so there was no full content of the text to display and highlight
To resolve this I followed the link (https://lucene.apache.org/solr/guide/7_1/uploading-data-with-solr-cell-using-apache-tika.html#configuring-the-solr-extractingrequesthandler) to the guidance documents and reviewed the part on fmap and used the example given for Last Modified Date as a guide on what to apply.
I then went to my solrconfig.xml file in the relevant core folder and added in the following line in the code beneath an already present fmap entry:
<str name="fmap.content">testcontent</str>
I had previously set up the testcontent field under the solr web interface in my core. I then re-ran my indexing line via a command prompt and that seemed to do the trick in terms of pulling out the basic content and rapping it with a basic emphasis.
All thanks for the input on this - still a lot more I want to test to help develop a clear requirement set but this really helps prove some of the basics are not complected.
I am building an Angular test preparation app (with Laravel 5.1 API). One of the requirements is to allow the user to print a certificate of achievement.
The client wants the person's name and credentials interpolated into the document (e.g., highlighted below). Here is a snapshot of the PDF template they sent:
The way I'm handling PDF viewing is simply by storing the file on S3 and giving them a link to that file.
Interpolating information into a PDF doc doesn't seem trivial and I haven't found much information on programmatically allowing this, but there are tools like DocHub, that allow you do edit while viewing the PDF.
I'm interested in learning:
is doing this programmatically trivial?
are there 3rd party tools I'm unaware of?
would I even be able to send this information along to the S3 link to interpolate in the first place?
Using PDF as a format for editing is usually a bad choice. If you have a form with fixed fields, then it's easy. Create a PDF template with an interactive form. In this form, based on AcroForm technology, you'll define fields with fixed coordinates, and a fixed size. You can then add content to these fields.
One major disadvantage with this approach is the lack of flexibility. Did you notice that I used the word "fixed" three times in the previous paragraph? If text doesn't fit the predefined field, you're out of luck. If the field is overdimensioned, you'll end up with plenty of white space. This approach is great if you can predict what the data will be like. A typical use case is a ticket or a voucher. For instance: the empty form is a really nice page, with only a couple of fields where an automated system can put a name, a date, a time, and a seat number.
This isn't the best approach for the example you show in your screen shot. The position of every line of text, every word, every character is known in advance. If you want to replace a short word with a long word (or vice-versa), then all those positions (of each line, of the complete page, possibly of the complete document) need to be recalculated. That's madness. Only people with very poor design skills come up with such an idea.
A better idea, is to store the template as HTML. See for instance chapter 5 of iText's pdfHTML tutorial, where we have this snippet of HTML:
<html>
<head>
<title>Invitation to SXSW 2018</title>
</head>
<body>
<u><b>Re: Invitation</b></u>
<br>
<p>Dear <name>SXSW visitor</name>,
we hope you had a great SXSW film festival experience last year.
And we would like to invite you to the next edition of SXSW Film
that takes place from March 9 until March 17, 2018.</p>
<p>Sincerely,<br>
The SXSW crew<br>
<date>August 4, 2017</date></p>
</body>
</html>
Actually, it's not really HTML, because the <name> tag and the <date> tag don't exist in HTML. All HTML processors (browsers as well as pdfHTML) ignore those tags and treat their content as if the tag was a <span>:
It doesn't make much sense to have such tags in the context of pure HTML, but it does make a lot of sense in the case of pdfHTML. With pdfHTMLL, you can configure custom tags, and have a result that looks like the PDFs shown below:
Look at the document for "John Doe" and compare it with the document for "Bruno Lowagie". The name "John Doe" is much shorter than my name, hence more words fit on that first line. The text flows nicely (we could also have chosen to justify the text on both sides). This "flow" is impossible to achieve with your approach, because you will never get a PDF template to reflow nicely.
OK, I get it, you probably say, but what about the practical aspects? You talk about a Java / .Net library, but I am working with Laravel and Angular.js. First, let me tell you that I don't think you'll find any good PDF tools for Laravel or Angular.js, because of the nature of PDF and those development environments (in my opinion, those technologies don't play well together). Regardless of my opinion, this shouldn't be much of a problem for you because you work in an Amazon environment. AWS supports Java, and the Java code needed to get pdfHTML working is minimal. Most of the code samples I wrote for the pdfHTML tutorial are shorter than 15 lines. So why not try Java and pdfHTML?
If you're already using Amazon services, why not use an amazon lambda function, in combination with iText7 (java), to generate the pdf on demand?
That way, you are guaranteed that the pdf is correct, and has nice layout every time.
Generating the pdf can either be done by:
converting HTML,
programmatically creating your entire document,
filling and flattening an XFA form.
I think for your use-case, either option 1 or 2 are the most sustainable.
I read different SO posts and blog posts about that question but none of them answered my questions.
Here's what I have:
Angular-translate is activated to find the preferred language, store it in localstorage/cookie and use that to translate the page.
Most people are trying to do this the other way around: someone enters a URL like domain.com/en/pagename.
I do not care about the locale in the url (although it can be there, i really don't care).
Here is what I'm trying to do:
But an important thing would be to translate the title in there for SEO and user-friendly purposes. Having:
domain.com/find-a-car/
domain.com/trouver-une-voiture/
domain.com/ein-auto-finden/
or
domain.com/en/find-a-car/
domain.com/fr/trouver-une-voiture/
domain.com/de/ein-auto-finden/
Any suggestions to achieve that ? Knowing that the page title should be dynamic, coming from the locale-??.json files !
My text is like this: The searched word WildCard shall be partially highlighted
I search using wildcard expression "wild*".
What I need is to have the highlight snippet to be [tag]Wild[/tag]Card. What I got was [tag]WildCard[/tag], and I spent lots of time researching it, but could not find an answer.
This behavior can be illustrated on linkedin.com, where you type other people's name at the top right corner.
Once this is figured out, I will have a follow-up questions.
Thanks,
I am not sure if you can achieve what you want directly in solr. The obvious solution is to parse the returned doc yourself searching for [tag]WildCard[/tag] and find out what part of the term you need to highlight.
This is not possible with Solr. What you need to do is change the characters Solr uses to highlight found words to something you can easily remove (maybe an html comment) and then build a highlight yourself. You are likely already storing your query in a variable, so just scan the search return docs and highlight all instances of your query.