Solr 7.4 index not updated after upgrading Solr to 8.11 - solr

I am upgrading my Java program from Solr 7.4 to Solr 8.11.1 (same upgrade for SolrJ). It does not use SolrCloud.
I did not change the code base because the upgrade guide reads :
If you are not using Solr in SolrCloud mode (you use Standalone Mode instead), we expect you can upgrade to Solr 8 from any 7.x version without major issues.
However the old index is never updated (its contents does not change). I know it is recommended to reindex when upgrading to a new major version, and if I do so on a 2 document index I get the expected contents. But as it took weeks to process and index the actual data some years ago I'd rather avoid reindexing if it is possible!
I also tried to apply the upgrader tool with and without -delete-prior-commits and although it ran smoothly the old index did not get updated afterwards when used in my program.
My test index has 2 documents in it. During the index update in my program I do a deleteById(coreName,listOfDocIdsToBeRemoved) on a SolrClient which returns {responseHeader={status=0,QTime=1}} and just afterwards I do a commit(coreName) on the same SolrClient which returns {responseHeader={status=0,QTime=23}}.
Following the commit if I run mySolrClient.query(coreName,new SolrQuery("*:*")).getResults().getNumFound();, I still get 2 documents whereas it should be 0 (the expected result I get when doing a full reindex). So it looks like the old index is read only (although file permissions are the same compared to the full reindexed version of the index).
If I log to the Solr Admin dashboard, the core overview reads (the version keeps increasing as expected as the last modified field gets updated) :
Last Modified:
3 minutes ago
Num Docs:
2
Max Doc:
2
Heap Memory Usage:
2452
Deleted Docs:
0
Version:
56
Segment Count:
1
Current: V
Finally my question is : Can I still upgrade Solr from 7.4 to 8.11 and use the old index without fully reindexing or is it just impossible without fully reindexing ?
Please note : when doing a full reindex I use the old solrconfig.xml file in which I only update the luceneMatchVersion tag and I use the same managed-schema file.

Related

After upgrading to Solr v8.11.1 from v7.7.2 search result is very different

We have upgraded solr version from 7.7.2. to 8.11.1, after that we are facing issue for search result. Less result are not matching for both version. Our document list are same as it is.
Also we have using same schema and fields.
Can someone please help us on it what can be possible reason or solution or or any algorithm change happened for v8 which can result into differ?
We recently upgraded from Solr 7.3 to Solr 8.9 running Solr Cloud. We had an issue with it showing a different number of documents even though we took a backup in Solr 7 and restored to Solr 8. Eventually we discovered that in Solr 8 there were some major changes to nested documents, we do not use nested documents in our environment, however in Solr 8 it was causing some of our documents to appear with the same id resulting in those documents appearing twice causing the discrepancy. We ended up taking another full backup and then deleting all our collections/configsets. After that we deleted the root field from the managed-schema and restored. After the restore our document counts matched up. Hope this helps.

How to get the version of lucene index in solr

My use case: I am currently on solr 5.5 and upgrading to solr 8.8
For this, I will need to do re-indexing on all machines where solr is installed. I need to do a check on the index version, if the index is made from the old version, then I will run the re-indexing logic, and if it already is the new version, I will skip the re-indexing.
Is there a way to detect the index version?
NOTE: the config files will already be updated to the new version so cannot use tag from solrconfig.xml
HTTP GET request to retrive the info :
yoursolrhost:8983/solr/admin/info/system
It would be something like below
http://yoursolrhost:8983/solr/admin/info/system?wt=json

Latest stable version of solr

solr
Now i am using solr4. I need to migrate solr to latest version.
Which solr version is latest and stable?
What are the conditions i need to check for migration?
How can i merge this version to latest version?
Which version can u prefer for me?
any migration tool is available for this migration?
I think about solr 6.3.0
Is it stable ?
What are the manger difference in solr 4 and 6 ??
When upgrading you really have to choices:
Reindex. This is the easiest (and you should have a way to reindex on request anyway, but some use cases makes reindexing hard, since Solr could be your primary datastore), most flexible and gives you the best end result.
Do a version-by-version upgrade, going from each version that supports the old index format to the most recent version that supports that format, then repeat until you're on your target version of Solr. Since old index versions are usually readable through each main version tree (I think), you'll have to go 4 -> 5 -> 6. You can use the IndexUpgrader tool to help with this, and as you can see at the bottom of that page, Jan made a tool that automates 4 -> 5 -> 6 for you.
The differences are far too many to really list, please refer to the Changelog for each release. In particular there's a world of difference regarding cloud / cluster support between 4.x and 6.x, and there's a lot of new features, such as streaming expressions.
Upgrading the schema is a matter of going through the datatypes you're using and checking if they've changed definitions (ints vs tries, for example). You'll probably be OK with just minor changes. The default schema factory has changed from the classic index schema to the managed schema, so if you want to keep editing the schema.xml as you're used to, you'll have to change to the ClassicIndexSchemaFactory in your configuration.

commit in solr 4.0 takes longer time compared to solr 3.6

I have recently migrated from solr 3.6 to solr 4.0. The documents in my core are getting constantly updated and so I fire a code commit after every 10 thousand docs . However moving from 3.6 to 4.0 I have noticed that for the same core size it takes about twice the time to commit in solr4.0 compared to solr 3.6.
Is there any workaround by which I can reduce this time. Any help would be highly appreciated
Solr 4 has transaction logging enabled by default. If you don't need that, you can disable this option. I would provide a link, but the Solr Wiki is currently down for maintenance.

Upgrade solr 1.4 index to solr 3.3?

I have an existing index build using apache solr 1.4.
I want to use this existing index in version 3.3. As you know the index format is changed after 3.x, so how is it possible to do this?
I have exported the existing index (that is in 1.4 version) using Luke to XML.
There's two ways to do this:
if your index is unoptimized, then simply optimize it - this will upgrade the file format along the way.
if your index is already optimized, you can't do this. Instead, use the command line tool supplied with solr (your path may differ from mine
java -cp work/Jetty_0_0_0_0_8983_solr.war__solr__k1kf17/webapp/WEB-INF/lib/lucene-core-3.3.0.jar org.apache.lucene.index.IndexUpgrader -verbose /path/to/index/directory
However, note that this only changes the file format - it won't stop deprecation warnings because unless you tell it otherwise, solrconfig.xml defaults to still assuming you're using an old index format. see http://www.mail-archive.com/dev#lucene.apache.org/msg23233.html
You may still get lots of lines like this in your logfile:
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
until you tell solrconfig.xml that you're ready to use all the features of the new index format. You do this by adding the following to solrconfig.xml (at the top level, just after the abortOnConfigurationError setting).
<!-- Controls what version of Lucene various components of Solr
adhere to. Generally, you want to use the latest version to
get all bug fixes and improvements. It is highly recommended
that you fully re-index after changing this setting as it can
affect both how text is indexed and queried.
-->
<luceneMatchVersion>LUCENE_33</luceneMatchVersion>
If you have the data: the best way is indexing all the data new in solr 3.3
You can use the data import handler to index your exported XML files.
If building up a new index is not an solution for you, you have got different possibilities:
As far as i know, Solr 3.3 can read old indexes.
So one idea could be using shards. One shard for the old data (read only) an the other shard for the new data. Unfortunately, in this solution you will be unable to modify old data.

Resources