I would like to validate queries against a schema before actually executing them.
Is there an official API which will give me access to a schema, or will I have to parse the Solr configuration XML myself?
The usual trick for finding these resources it to open the Admin interface with the developer network tool running in your browser, then navigating to the resource you're looking for while watching which requests your browser perform. Since the frontend is purely Javascript based and runs in your browser, it accesses everything through the API exposed by Solr.
You'll have to parse something, either in JSON or XML (probably) format. For my older, 4.10.2-installation, it is available as:
/solr/corename/admin/file?file=schema.xml&contentType=text/xml;charset=utf-8
Related
I am using the nutch REST API to run nutch searches on a seperate server. I would like to retrieve the crawled data back to my local machine. Is there a way I can use the nutch dump functionality to dump the data and retrieve it via the API, or am I better off indexing the data into Solr and retrieving it from Solr.
Thanks for your help.
Currently, the REST API doesn't provide such functionality. The main purpose of the REST API is to configure and lunch your crawl jobs. At its core, it will allow you to set the configuration of a new crawl job and manage it (to some extent).
The transfer of the crawled data is up to you. That being said I do have a couple of recommendations:
If you're sending the data into Solr/ES (or any other indexer) I would recommend getting the data directly from there. Both Solr&ES already provide a REST API, with the additional benefit that you might filter which data to "copy over".
If you're running Nutch in a distributed mode (i.e in a Hadoop cluster) try to use the Hadoop libraries to copy the data to the destination.
If none of this applies then perhaps relying on something else like rsync or similar might be worth considering.
Normally we do indexing in solr from a browser. Can we do it automatically by writing a batch job or java code?
Please provide me some idea, if it is possible.
You can use the DataImportHandler, which can import from lot of different sources such as databases or xml files: https://wiki.apache.org/solr/DataImportHandler
If you have specific requirements which are not satisfied by the DataImportHandler you may implement your own indexer by using a solr client api:
https://cwiki.apache.org/confluence/display/solr/Client+APIs
If you want to do stuff with Solr programmaticaly take a look at: Solrj which is an API that'll do what your asking for.
You can use a web debugging proxy such as Fiddler to view the HTTP request that is generated when you trigger the data import via a web browser. Then send the same request from your Java code.
Im building a simple angular application and there is a small administrator panel for updating the content (a .json document). I'm looking for a way to edit the json document from the administrator panel.
I can manipulate the memory-loaded json but I can't save it. Is there a way to put the json file in some kind of cloud database and connect to it without setting up a server or backend for my application?
I want my application to be easily deployable on any ftp so I can't setup a nodeserver or install something like couchdb.
Any ideas are appreciated.
You could use a provider like Parse. It's free (up to a limit of requests/month), has a nice JavaScript SDK that would get you up and running quickly. https://parse.com/
Also, check out this query builder to aid in retrieving your data from Parse. It's built as an Angular service for easy integration. https://github.com/dpollot/parse-query
EDIT
Parse also offers hosting, for free.
I'm looking at using Sailsjs for server side processing and validation and angular on the client side. Apparently BreezeJS supports Odata and WebApi for its interaction with the datasource. I know that Sails provides a REST Api out of the box.
How would I go about setting up Sails to provide either OData or WebApi to connectivity for Breeze?
Also, is there any other well known way of connecting Sails to Angular that provides the ability to generate queries on the client side that then get executed on the server? (My use case is filtering large datasets and manipulating the result on the client before batch updating the datasource)
Thanks!
What database and/or ORM are you using? That affects the answer, because the OData query must be turned into the appropriate query for your ORM/database. One of the following might work for you:
The breeze.server.node repo includes an OData parser that turns OData URLs into expression trees. Then there are adapters that turn these expression trees into MongoDB queries or Sequelize queries. If you are using MongoDB or Sequelize, this might be what you need. You can write your own adapter for Waterline if none exists yet.
The breeze.js.labs repo includes breeze.ajaxrestinterceptor.js which converts Breeze's OData URLs into REST-style resource URLs. It only supports a subset of the possible URLs, and will need to be adapted to your own API conventions.
An upcoming Breeze release will offer an alternative to OData: an all-JSON query language, inspired by the MongoDB and Sequelize query languages. This will make it easier to write your own backend, and to support Sails/Waterline in the future.
I have been attempting to teach myself quite a bit about silverlight, and how it all works, for the past few weeks, and I am to the point in my app development where I would like to connect up to my web server's MySQL database.
My web server is capable of running ASP.NET pages, but is Apache, and natively runs PHP (which is what I'm far more familiar with). It has a MySQL database engine, and I am very well-versed in your typical dynamic page creation with PHP and MySQL.
What I'm NOT familiar with are these "Web Services" that people keep mentioning every time I find an answer regarding the question of "how do you connect silverlight to a database?"...
So my basic question is really one of data FLOW, and where everything fits in the puzzle, and how to get it all working in this particular configuration. Most of the answers I have seen deal with IIS instead of Apache, ASP.NET instead of PHP, and MS SQL Server instead of MySQL.
Also, answers tend to start using abbreviations and acronyms without actually explaining what they stand for.
For example: What is WCF, and RIA services, and how do they fit in to the puzzle as a whole?
I suppose I'm just looking for a top-down overview of the structure of data flow on a MACRO level, not on the micro (code) level.
(Edited to add:)
Also: I have done vb.net apps in the past which have used MySQLConnector.NET to pull from my web server's database remotely, but I understand that the client machine would have to be whitelisted as a remote machine, meaning I'd have to open my MYSQL server up, and make the access mask basically %.%.%.% in order for any client to connect... and that is undesirable... so if I understand things right, the web service runs on the web server, and the client sends requests to it, and the web service acts as an intermediary, grabbing the data from the database (possibly with some sort of "stored procedure" look-alike?), and passes the data on to the client... which also means all database access credentials are on the server, and not inside the (potentially hackable) client...
Do I have it right?
Also, when answering, I need to know where the access to the web services is... in the silverlight APP project code, or the silverlight WEB project code...
I have found this wonderful tutorial that helps to explain it...
http://www.designersilverlight.com/2010/05/23/php-mysql-and-silverlight-the-complete-tutorial-part-1/
Here is how I understand it all working.
There are 3 "Layers" to the process: The application, the web server, and the database.
The application calls out to the web server to execute a script file (like a normal PHP script). There script file can have normal URL variables passed to it (like script.php?foo=bar, so $foo is defined as "bar" in the script)... so you can use those URL encoded variable/value pairs to tweak your script results as you would normally with a web page.
I imagine you would have one script per TYPE of database query, with var/value pairs to tweak your results. So on your web server you would end up with numerous PHP scripts, just like you would normally for a website with different pages, and you pass variables in to those scripts to customize the results.
For example, for users, you could have a get_users.php script that would return all users...
but get_users.php?loggedin=true would get all users that are logged in currently
get_users.php?loggedin=true&ingame=true would get all users that are logged in and in a game... You just script the logic and the resulting SQL query accordingly.
All of the results are encoded either with XML or JSON (Javascript Open Notation: see What is JSON and why would I use it? ) for transport to the app... the app, in effect, is reading the results of an echo of the JSON encoded stuff.
If you were to open these scripts in a web browser, the only thing you would see is a text printout of the JSON data... no web page... just data that is read by the app and then decoded in to objects.
So in effect, the silverlight app is reading a text output of a PHP script executed on your web server, and interpreting the output.
^^^^^ THIS IS THE SHORT ANSWER TO MY QUESTION. :)
To be blunt, the whole use of the terminology "web service" is misleading, and what was really leading me astray. I thought it was some sort of service or app you had to install on your web server just like PHPMyAdmin or something.