Using Alfresco Community - 5.0.0 version.
Tried below properties in alfresco-global.properties file:
queryConsistency=EVENTUAL then all queries are going to Solr.
queryConsistency=TRANSACTIONAL then all queries are going to DB.
And Default mode executes all queries from DB first then Solr in case of failure from DB.
Q1: Is there any mode by which I can query Solr first and if there are no results from Solr then query DB in one transaction?
Q2: Can we execute some queries from Solr and some from DB from JAVA side?
Alfresco calls this a Transactional Metadata Query, where the query can fall back to the database when Solr returns no results.
Details on configuring this behavior can be found here, but basically queryConsistency should be set to TRANSACTIONAL_IF_POSSIBLE.
It's important to note, especially if you're converting Lucene queries to CMIS, that certain query parameters (e.g., +PATH:) and wildcards (e.g., #docs:pageId:homepage) require Solr to function, and cannot refer to the database directly in the event of a Solr failure. You can test whether a given parameter requires Solr in Alfresco under Admin Tools / Node Browser. Run your query selecting different types; type: solr-fts-alfresco is Solr-only; type: db-afts to query Solr first and fallback to the database if Solr returns nothing.
And to answer your second question, the second link above mentions:
The solr.query.cmis.queryConsistency and solr.query.fts.queryConsistency properties can also be set per query on the SearchParameters and QueryOptions objects.
So it seems you can accomplish what you want within your Java code, but I have not tried this myself. I find it most desirable, as a general rule, to hit Solr first for its performance gains then fallback to the database only if necessary. I can't think of a good case for embedding the data source selection within your business logic.
Related
We have some Premium Azure SQL Databases, and we are attempting to use indexed views to materialize a subset of some larger tables, so the data can be used in some search / reportring queries. What I am noticing is that the query optimizer refuses to use the view.
My first attempt was to create the view and not modify the query. I thought the optimizer would substitute the view when it determined it was a better choice, but it didn't.
My second attempt I selected from the view directly, the execution plan didn't change at all.
Finally I added the (NOEXPAND) hint on the view and this time the plan did use the indexed view.
I've seen this behavior with 3 different indexed views in 3 different queries. In all 3 cases the queries performed at several orders of magnitude better. In my most recent example the duration of the query went from 90 seconds to 1 second.
I'm not a big fan of using hints to override the query optimizer, has anyone else experienced this?
Indexes views are fully supported on Azure SQL Database Premium. No functionality of indexed views is blocked or not supported on Azure SQL Database.
To be able to help you, you will have to share the database schema and a step-by-step procedure of how your reproduce the issue. I guess you won't agree to share the schema on this site, so maybe the best you can do is create a support ticket at Azure Support. Please send an email to this (azcommunity#microsoft.com) address explaining the issue, they will return instructions to have this handled by Azure Support. They will safely take a look at your schema.
I'm new to new Solr. I got the examples of how to index data directly from database for Solr 4.9 but still not able to find anything for Solr 6.5.
Does Solr 6.5 support database indexing? If yes, then how to achieve same.
DataImportHandler is usually the way to load data from a database into Solr. It was there in Solr 4.9 and is still there in Solr 6.5.
Specifically, Solr ships with a dih example (bin/solr start -e dih) that has a number of collection, one of them showing database indexing.
There are also third party products that can read from database and index into Solr (eg. Apache NiFi), but their levels of Solr support may vary.
I'm working on a SaaS which have a database for each account, with basically the same tables. What's the best way to index all databases separately? I was thinking about setting different solr instances(different ports) for each database in the same server, but it could be hard on the server. So, i'm in this crazy doubt on what to do next. I haven't found any useful idea in the solr documentation. Could you guys help out. Thanks in advance.
If you store all the data from all of your tenants on one collection, it will be easy in the beginning because probably you will do several changes on your schema and it is easier if you do them once for all your customers.
As a negative point in this scenario you will have lots of unrelated data grouped together and you always have to use a filter query for the tenant (client) id.
What if you create, for starters, a collection for each of the tenant on the same Solr server? This way you don't mix the data of your tenants and you achieve the functionality you basically need.
In this scenario, as it happens for your relational database instances, you have to keep the schema changes in sync.
For relational databases there are tools like flyway or liquibase that can be used to version the changes applied on each of the tenant database.
For Solr there aren't AFAIK such tools, but you can apply your schema changes programmatically through Solr Schema API. In case you have to do highly detailed changes that can't be done via the Schema API, you can replace the schema.xml file of each collection with an updated version of it and restart the solr server.
What you need to keep in mind is backward compatibility. Whenever you add some changes to any of the databases (relational DB or Solr) you need to take into account that the old code must still work with the latest updates that you perform on the relational database/ solr schema structure.
It seems easy to apply an Azure Search index to an SQL Azure database. I undertand that you query the search index using REST APIs and that the index then needs to be maintained/updated. Now, consider a web server running IIS, with an underlying SQL Server database.
What is considered best practice; querying and updating the index from the web server or from SQL Server, e.g. from within a CLR stored procedure? Are there specific design considerations here?
I work on Azure Search team and will try to help.
Querying and updating the index are two different use cases. Presumably, you want to query the index in response to user input in your Web app. (It is also possible that you have a SQL stored procedure with some complex logic that needs full test search, but that seems less likely).
Updating the index can be done in multiple ways. If you can tolerate updating your index at most every 5 minutes, use Azure Search SQL indexer automagically update the index for you - see http://azure.microsoft.com/en-us/documentation/articles/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers-2015-02-28/ for details on how to do it. That article describes creating indexers using REST API, but we now have support for that in .NET SDK as well.
OTOH, if you need hard real-time updates, you can update the search index at the same time you produce data to insert / update your SQL database.
Let me know if you have any follow up questions!
HTH,
Eugene
I am new user to solr,I want to access and search the MYSQL database tables in java applications via solr.i am able to index my table in solr admin interface. Can anyone tell me how to connect and access MYSQL tables in java application so that i can search data fastly ? I was not able to understand tutorials whatever i found.
Solr provides client libraries in java, ruby and other languagues to help you connect to Solr and query it.
Check for the Java library Solrj to connect and query Solr.
If you are using Frameworks with your projects you might want to check for Spring data as well which will help you seamlessly query and transform Solr response.
So you will need to set up an instance of a Solr Server, SOlr will store and indexes from your database using the DataImportHandler.
http://amac4.blogspot.co.uk/2013/08/configuring-solr-4-data-import-handler.html
Solr creates indexes using Lucene, so you have two options, you can use classes from the Lucene jar file or SolrJ to search your indexes.
OR
You can query Solr by sending http requests. I set-up a Java Web Service so you can snatch some of my code if you need to.
http://amac4.blogspot.co.uk/2013/07/restful-java-web-service-for-solr.html