I am looking for pointers for libraries or methods that would be able to generate full text from the structured information returned by Wikidata - if possible in multiple languages.
To be clearer: from data like the one provided here (this is the JSON version) I would like to be able to generate text similar to the intro paragraph of the wikipedia page for the same item:
Orvieto Cathedral (Italian: Duomo di Orvieto; Cattedrale di Santa Maria Assunta) is a large 14th-century Roman Catholic cathedral dedicated to the Assumption of the Virgin Mary and situated in the town of Orvieto in Umbria, central Italy.
The reason is that the text is provided by Wikipedia for all those cases where a page exists, but I would like to have something also for the Wikidata items without a wikipedia page.
My problem #1 here is: I don't know what something like this is called, so I have no idea what to google for. Any pointers to start from are appreciated, including services or APIs.
This problem falls under the Data to text generation task. I do not know of any services that are currently offering solution. You can look at WEBNLG challenge which has the same objective and similar data. AFAIK mostly template based methods are used to automatically insert data from wikidata into wikipedia as text.
Related
I am building an Angular test preparation app (with Laravel 5.1 API). One of the requirements is to allow the user to print a certificate of achievement.
The client wants the person's name and credentials interpolated into the document (e.g., highlighted below). Here is a snapshot of the PDF template they sent:
The way I'm handling PDF viewing is simply by storing the file on S3 and giving them a link to that file.
Interpolating information into a PDF doc doesn't seem trivial and I haven't found much information on programmatically allowing this, but there are tools like DocHub, that allow you do edit while viewing the PDF.
I'm interested in learning:
is doing this programmatically trivial?
are there 3rd party tools I'm unaware of?
would I even be able to send this information along to the S3 link to interpolate in the first place?
Using PDF as a format for editing is usually a bad choice. If you have a form with fixed fields, then it's easy. Create a PDF template with an interactive form. In this form, based on AcroForm technology, you'll define fields with fixed coordinates, and a fixed size. You can then add content to these fields.
One major disadvantage with this approach is the lack of flexibility. Did you notice that I used the word "fixed" three times in the previous paragraph? If text doesn't fit the predefined field, you're out of luck. If the field is overdimensioned, you'll end up with plenty of white space. This approach is great if you can predict what the data will be like. A typical use case is a ticket or a voucher. For instance: the empty form is a really nice page, with only a couple of fields where an automated system can put a name, a date, a time, and a seat number.
This isn't the best approach for the example you show in your screen shot. The position of every line of text, every word, every character is known in advance. If you want to replace a short word with a long word (or vice-versa), then all those positions (of each line, of the complete page, possibly of the complete document) need to be recalculated. That's madness. Only people with very poor design skills come up with such an idea.
A better idea, is to store the template as HTML. See for instance chapter 5 of iText's pdfHTML tutorial, where we have this snippet of HTML:
<html>
<head>
<title>Invitation to SXSW 2018</title>
</head>
<body>
<u><b>Re: Invitation</b></u>
<br>
<p>Dear <name>SXSW visitor</name>,
we hope you had a great SXSW film festival experience last year.
And we would like to invite you to the next edition of SXSW Film
that takes place from March 9 until March 17, 2018.</p>
<p>Sincerely,<br>
The SXSW crew<br>
<date>August 4, 2017</date></p>
</body>
</html>
Actually, it's not really HTML, because the <name> tag and the <date> tag don't exist in HTML. All HTML processors (browsers as well as pdfHTML) ignore those tags and treat their content as if the tag was a <span>:
It doesn't make much sense to have such tags in the context of pure HTML, but it does make a lot of sense in the case of pdfHTML. With pdfHTMLL, you can configure custom tags, and have a result that looks like the PDFs shown below:
Look at the document for "John Doe" and compare it with the document for "Bruno Lowagie". The name "John Doe" is much shorter than my name, hence more words fit on that first line. The text flows nicely (we could also have chosen to justify the text on both sides). This "flow" is impossible to achieve with your approach, because you will never get a PDF template to reflow nicely.
OK, I get it, you probably say, but what about the practical aspects? You talk about a Java / .Net library, but I am working with Laravel and Angular.js. First, let me tell you that I don't think you'll find any good PDF tools for Laravel or Angular.js, because of the nature of PDF and those development environments (in my opinion, those technologies don't play well together). Regardless of my opinion, this shouldn't be much of a problem for you because you work in an Amazon environment. AWS supports Java, and the Java code needed to get pdfHTML working is minimal. Most of the code samples I wrote for the pdfHTML tutorial are shorter than 15 lines. So why not try Java and pdfHTML?
If you're already using Amazon services, why not use an amazon lambda function, in combination with iText7 (java), to generate the pdf on demand?
That way, you are guaranteed that the pdf is correct, and has nice layout every time.
Generating the pdf can either be done by:
converting HTML,
programmatically creating your entire document,
filling and flattening an XFA form.
I think for your use-case, either option 1 or 2 are the most sustainable.
Is it possible to highlight a list of countries with a different colors?
I need to display some countries' statistics on the world map.
Now I use an image and fill a region with color (calculated for each country) by country's coordinates. It's a simple solution and it works well. But now I need to specify the countries' name too (and I think it's not the last customization).
There is a polygon solution, but it uses an array of coordinates. I don't think it's a suitable solution to highlight countries's territory.
I haven't found a solution yet. Any suggestions?
Thanks in advance.
Highlighting countries or regions to support statistics is known as Choropleth Mapping
, but unfortunately there usually isn't direct library support for Choropleth maps bundled into an online map API. This means you'll have to create your own framework, but fortunately it is possible to create one - I wrote an example using jQuery + HERE Maps to
answer the question here
Updated WKT solution now available
Access to KML shapes is no longer required, since the Geocoder API now offers an IncludeShapes attribute which returns the shape of a country in WKT format. A WKT parser can be found here.
A simple WKT choropleth example can be found here.
KML Base solution
For any framework you will need to have a file holding the boundaries of the countries or regions you need. The example uses a KML file, but you could also start with polygons if you had them. Country borders are a political minefield, which is the reason I guess most online mapping APIs steer clear of them. As a hint: try starting with something like http://geocommons.com/overlays/119819 and simplify it as much as possible to speed up the rendering- many small wiggles in the coast lines and small outlying islands are unnecessary.
Of course you could also try searching for "create choropleth map" from a search engine of your choice and use an tool to create a static image for your data (potentially at several zoom levels) and then use this as the basis of an map tile overlay. This requires a lot more work up front, but would push all the calculations server side and hence be faster to display.
Working example can be found on GitHub here
You could put the country's name into the image. It's not that difficult to place text into an image. The only tricky bit is if you are using tiles, you need to deal with names that cross tile boundaries by drawing the name once for each tile.
Is a wave limited to the sharing of textual information (HTML), or am I correct in assuming that a wave can contain arbitrary data (represented in XML), so long as it also contains the javascript necessary to render it in a meaningful way?
I ask because the collaborative document preparation demonstrated in the Google I/O video looks very powerful, but there are many other types of documents than simple rtf text. In my case I would be looking interactively to develop gantt charts.
There is a lot that can be done inside each Wave. They have not yet made all features available, but here is a link to some samples: http://wave-samples-gallery.appspot.com/ which includes my Slashdot Gadget:http://wave-samples-gallery.appspot.com/about_app?app_id=18006
The Slashdot Gadget actually takes the RSS feed for Slashdot and displays the latest headlines.
Here is the XML: http://www.m1cr0sux0r.com/slashdot.xml
alt text http://www.m1cr0sux0r.com/xml.jpg
I got access to Google Wave a few days ago, and here's what the raw data for their Sokoban game (which supports two players playing simultaneously on the same board) looks like, for example:
<blip>
<p _t="title">
</p>
<p>
<w:gadget author="blixt#wavesandbox.com" prefs="" state="" title="" url="http://sokoban-server.appspot.com/com.example.simplegadget.client.SokobanGadget.gadget.xml">
<w:pref name="playerAllocation" value="1 1,blixt">
</w:pref>
<w:pref name="totalMoves" value="8">
</w:pref>
<w:pref name="playerPositions" value="1 4,2">
</w:pref>
<w:pref name="rockPositions" value="6 2,2 3,2 14,2 15,2 16,2 4,3">
</w:pref>
</w:gadget>
</p>
</blip>
So yes, you can store any data you like in a single blip, with the possibility to go backwards in "time" to see older versions of the data etc.
By the way, if you're interested in seeing some code for a robot that sits in a wave and interacts with users, I made one for a game I'm developing: Google Code Project for multifarce (and the game in question, it's not really public yet and as such not particularly functional.) The bot source is here: multifarce Wave robot source
Basically, all you need to get a bot running are the 14 last lines in that code. I love it! =)
My goal is to display various shapes(polygons, points, linestring) on google maps by using data entered into a Postgis database dynamically(i mean by that we can see modifications in the map in real time).
I was looking for a way to do this that used the spatial structure already provided in postgis(already designating if shape is a linestring or polygon, etc) instead of parsing out the coordinates and then re-entering spatial structure in google maps. I saw that google maps api is now compatible with kml data formats. And then I read that i have to convert postgis data to kml format.
I've done some reading in the forums about the actual process of converting postgis data to kml via FWTools, but didn't see anything that would help me. I'm new to kml but am familiar with postgis and perl and PHP. Is there a tutorial for the process of converting postgis data to kml? Where can I get started? Thanks for any help
You can use PostGIS to convert to KML directly:
SELECT ST_AsKML(geometry) from MyTable;
ST_AsKML is one of several output formats, including WKT, GML, GeoJSON, etc.
To show dynamic data in Google Earth, a common pattern is to use KML with a NetworkLink element. Have the link's viewRefreshMode equal to onStop and Google Earth will make requests (to a URL served by PHP, presumably) with bounding box parameters attached. Use the bounding box to query features in the PostGIS database, and return results as kml. This is great if you have lots and lots of features, but only want to retrieve those in the region the user is looking at.
Depending on the complexity of your application, you may also want to look at GeoDjango. (Familiarity with PostGIS is a big head start!)
You can get a textual representation of the spatial data from a Postgres DB using a text conversion function, like
SELECT AsText(MyGemoetry) from MyTable
then you parse the string, create your objects using various API functions - depending on the PostGIS geometry type - and append these object to the main GE plugin object in a DOM like way.
If you are familiar with JavaScript and have a fundamental knowledge of XML, a good start is http://code.google.com/apis/earth/documentation/reference/
Don't forget to specify unique ID's to your objects so you can find them later to drop/modify.
Maybe you can get some inspirations here, display the linked "locator.js" file and look at function PaintSubField(Coord) ... this is another way, bit crude but effective, avoiding to mess around with too many individual parent/child objects and structures
You also may want to consult sample applications and use the code playground for "rapid prototyping"
re "realtime" you need at least an event that you can link your generation/redraw routines to.
Good luck
MikeD
A friend of mine is now building a web application with J2EE and Struts, and it's going to be prepared to display pages in several languages.
I was told that the best way to support a multi-language site is to use a properties file where you store all the strings of your pages, something like:
welcome.english = "Welcome!"
welcome.spanish = "¡Bienvenido!"
...
This solution is ok, but what happens if your site displays news or something like that (a blog)? I mean, content that is not static, that is updated often... The people that keep the site have to write every new entry in each supported language, and store each version of the entry in the database. The application loads only the entries in the user's chosen language.
How do you design the database to support this kind of implementation?
Thanks.
Warning: I'm not a java hacker, so YMMV but...
The problem with using a list of "properties" is that you need a lot of discipline. Every time you add a string that should be output to the user you will need to open your properties file, look to see if that string (or something roughly equivalent to it) is already in the file, and then go and add the new property if it isn't. On top of this, you'd have to hope the properties file was fairly human readable / editable if you wanted to give it to an external translation team to deal with.
The database based approach is useful for all your database based content. Ideally you want to make it easy to tie pieces of content together with their translations. It only really falls down for all the places you may want to output something that isn't out of a database (error messages etc.).
One fairly old technology which we find still works really well, is to use gettext. Gettext or some variant seems to be available for most languages and platforms. The basic premise is that you wrap your output in a special function call like so:
echo _("Please do not press this button again");
Then running the gettext tools over your source code will extract all the instances wrapped like that into a "po" file. This will contain entries such as:
#: myfolder/my.source:239
msgid "Please do not press this button again"
msgstr ""
And you can add your translation to the appropriate place:
#: myfolder/my.source:239
msgid "Please do not press this button again"
msgstr "s’il vous plaît ne pas appuyer sur le bouton ci-dessous à nouveau"
Subsequent runs of the gettext tools simply update your po files. You don't even need to extract the po file from your source. If you know you may want to translate your site down the line, then you can just use the format shown above (the underscored function) with all your output. If you don't provide a po file it will just return whatever you put in the quotes. gettext is designed to work with locales so the users locale is used to retrieve the appropriate po file. This makes it really easy to add new translations.
Gettext Pros
Doesn't get in your way while coding
Very easy to add translations
PO files can be compiled down for speed
There are libraries available for most languages / platforms
There are good cross platform tools for dealing with translations. It is actually possible to get your translation team set up with a tool such as poEdit to make it very easy for them to manage translation projects
Gettext Cons
Solves your site "furniture" needs, but you would usually still want a database based approach for your database driven content
For more info on gettext see this wikipedia page
They way I have designed the database before is to have an News-table containing basic info like NewsID (int), NewsPubDate (datetime), NewsAuthor (varchar/int) and then have a linked table NewsText that has these columns: NewsID(int), NewsText(text), NewsLanguageID(int). And at last you have a Language-table that has LanguageID(int) and LanguageName(varchar).
Then, when you want to show your users the news-page you do:
SELECT NewsText FROM News INNER JOIN NewsText ON News.NewsID = NewsText.NewsID
WHERE NewsText.NewsLanguageID = <<Session["UserLanguageID"]>>
That Session-bit is a local variable where you store the users language when they log in or enters the site for the first time.
Java web applications support internationalization using the java standard tag library.
You've really got 2 problems. Static content and dynamic content.
for static content you can use jstl. It uses java ResourceBundles to accomplish this. I managed to get a Databased backed bundle working with the help of this site.
The second problem is dynamic content.
To solve this problem you'll need to store the data so that you can retrieve different translations based on the user's Locale. (Locale includes Country and Language).
It's not trivial, but it is something you can do with a little planning up front.
#Auron
thats what we apply it to. Our apps are all PHP, but gettext has a long heritage.
Looks like there is a good Java implementation
Tag libraries are fine if you're using JSP, but you can also achieve I18N using a template-based technology such as FreeMarker.