I set up a replication master and a replication salve for SOLR and it doesn't do anything. My suspicion is the generation number. The master has 232 while the slave has 241. If somebody can confirm my suspicion, that would be great and I also would like to know how to resolve this issue if so?
I can't confirm it but I can deny it. :)
From this post, here is how it works:
It look at the index version AND the index generation. If both slave's
version and generation are the same as on master, nothing gets
replicated. if the master's generation is greater than on slave, the
slave fetches the delta files only (even if the partial merge was done
on the master) and put the new files from master to the same index
folder on slave (either index or index., see further
explanation). However, if the master's index generation is equals or
less than one on slave, the slave does the full replication by
fetching all files of the master's index and place them into a
separate folder on slave (index.). Then, if the fetch is
successfull, the slave updates (or creates) the index.properties file
and puts there the name of the "current" index folder. The "old"
index. folder(s) will be kept in 1.4.x - which was treated
as a bug - see SOLR-2156 (and this was fixed in 3.1). After this, the
slave does commit or reload core depending whether the config files
were replicated. There is another bug in 1.4.x that fails replication
if the slave need to do the full replication AND the config files were
changed - also fixed in 3.1 (see SOLR-1983).
plus there is another bug still opened about the index generation id:
http://lucene.472066.n3.nabble.com/replication-problems-with-solr4-1-td4039647.html
Now to answer your question in one line, the replication always occur (full pr partial) if the version (or replication) numbers are different from master to slave.
Related
We have some sites which use solr as an internal search. This is done with the extension ext:solr from DKD. Within the extension there is an install script which provides core for multiple languages.
This is working well on most systems.
Meanwhile we have some bigger sites and as there are some specialities we get problems:
We have sites which import data on a regulary base from outside of TYPO3. To get the solr index up to date we need to rebuild the complete index (at night). But as the site gets bigger the reindex takes longer and longer. And if an error occurs the index is broken the next day.
You could say: no problem just refresh all records, but that would leave information in the index for records which are deleted meanwhile (there is no 'delete' information in the import, except that a deleted record is no longer in the import. So a complete delete of all records before the import (or special marking and explicit deletion afterwards) is necessary.
Anyway, the reindex takes very long and can't be triggered any time. And an error leaves the index incomplete.
In theory there is the option to work with two indices: one which is build up anew and the other one is used for search requests. In this way you always have a complete index, so it might be not up to date. After the new index is build you can swap the indices and rebuild the older one.
That needs to be triggered from inside of TYPO3, but I have not found anything about such a configuration.
Another theoretic option might be a master-slave configuration, but as far as I think about it:
when the index of master is reset to rebuild it, this reset would be synchronized to slave which looses all the information it should provide until the rebuild is complete.
(I think the problem is independent of a specific TYPO3 or solr version, so no version tag)
you know about our read and write concept introduced in EXT:Solr 9 https://docs.typo3.org/p/apache-solr-for-typo3/solr/11.0/en-us/Releases/solr-release-9-0.html#support-to-differ-between-read-and-write-connections ?
Isn't it something for your case?
The only one thing what you need is to setup it in deployment properly.
If your fresh Index is finalized and fine and not broken, you just switch the read core to read from previous write one.
I remove document in CouchDB by setting the _deleted attribute to true (PUT method). The last revision of document is deleted but previous revision is still available.
And when I pull documents of specific type from database, this document is still available.
How I should delete document to not be available?
I use synchronization between CouchDB on the server and PouchDB instances on mobile applications (Ionic).
You need to compact your database. Compaction is a process of removing unused and old data from database or view index files, not unlike vacuum in RDBMS. It could be triggered by calling _compact end-point of a database, e.g. curl -X POST http://192.168.99.100:5984/koi/_compact -H'Content-Type: application/json'. After that the attempts to access the previous revisions of a deleted document should return error 404 with a reason missing.
Note that the document itself not going to completely disappear, something called "tombstone" will be left behind. The reason is that CouchDB needs to track deleted documents during replication to prevent accidental document recovery.
I modified the schema of mysql database(added a new table etc.), I cleared the tmp/cache(except directories).
Now the save in the new table happens only once(I have multiple save calls in a for loop and save happens for all of them) and fails from next time I enter the flow.
I am using cakephp 1.3.
What else should I check ?
Got it.
The cache issue was one part of the problem, which got fixed by clearing the files in tmp/cache directory.
Learning is -
If you make mysql schema changes(add new table / column etc.) in mysql, either clear the tmp/cache directory or set the debug level as 3 and refresh the page and set the debug level again to 0(if on production).
I was also getting a save error - mysql server has gone away, because in configs the wait_timeout value was 600 seconds. But my script was taking longer than that.
So model->save() was not working.
In my.cnf I updated the timeout to 4800 and restarted mysql and it fixed the problem.
I have moved to a new nagios installation (new server). I have transfered the data from the original server (/var/log/nagios2/archives) to my new server (/var/log/nagios3/archives) (I think they have compatible format). Now I would like to regenerate the pnp4nagios graphs using this historical data.
This is the command I have to process data on the fly:
/usr/lib/pnp4nagios/libexec/process_perfdata.pl --bulk=/var/lib/pnp4nagios/perfdata/host-perfdata
But this is just processing new data in /var/lib/pnp4nagios/perfdata/host-perfdata.
I have several questions:
Where does pnp4nagios store the processed data (graphs)?
How can I force pnp4nagios to regenerate all graphs?
pnp4nagios will call process_perfdata.pl which itsself invokes a call to rrdtool, storing the graph data within round robin databases (rrds). Within your pnp4nagios configuration, you should find the logical path for those on disk, letting you backup/move that data.
I'm just guessing that you already have pnp4nagios 0.6 (of not, look for the 0.4 docs):
http://docs.pnp4nagios.org/pnp-0.6/webfe_cfg (look for rrdbase)
But keep in mind - you cannot move rrds between different architectures. i.e. old server i386, new server amd64 - does not work at all by just copying stuff with rsync/scp. Look here for details: http://oss.oetiker.ch/rrdtool/doc/rrddump.en.html
Further, you should consider moving from the synchronous mode (which yours is) to "bulk mode with npcd and npcdmod" which will create asynchronous spooling, decreasing blocking the core when perfdata processing hangs (lower latency in case).
If you happen to have further questions on pnp4nagios itsself, you might like the idea of posting your questions to the monitoring-portal.org as well where the devs are reading too.
The nagios.log (and log archives) do not contain the perfdata information. What you need to do is move the .rrd & .xml files to the new server from the perfdata directory.
Nagios doesn't log or retain the perfdata, at runtime if processing of perfdata is enable it will process it and then cheal it's hands of it.
Events till now
We have a CC 7.1.2.2, multisite setup where we do deliveries between 2 sites. Now when resuming a delivery at the destination site, we get this error :
Unable to read change set entry for activity "<activity name>". Unable to
convert diffs to elements. Unexpected error in deliver. Unable to perform merge.
Unable to do integration.
Then running checkvob -ucm shows some broken hyperliniks which the SCM support team fixes for us. IBM tech note says this is a synch issue.
Now the actual problem:
This has started happening on a regular basis suddenly and we know its NOT a synch issue between VOB and PVOB as the packets are getting synched properly. What I am interested in finding out is whether this could occur due to a specific set of user actions like deleting checked out versions etc. The key point is its not a one off thing and impacts our deliveries everyday. We are not able to find any concrete triggering actions or root cause
Any ideas ?
This has been linked to a multi-site sync issue from a long time now (here an example in 2005), and was also associated with a bug in CC multi-site 7.0.
But if you are really sure multi-site sync is not the issue, then it could be linked to "lost+found" issue, where elements could have been:
deleted (rmelem by an admin -- I know regular users in your setup don't have rmver or rmelem rights -- in order to clean the lost+found directory automatically, maybe through a ClearCase scheduled job or some kind of trigger?)
not selected because the config spec of views involved by your deliver are setup to not select the lost+found directory
Found out the issue; it was hidden synch issues indeed. What really was happening was that multisite synch was timing out for packets larger than 250 meg. This would create problems for bid inetersite deliveries where PVOB would synch over and VOBs would not. This was hidden as otherwise sync was happening properly.
Thanks VonC for the other inputs; I know you'd have pointed me to synch issues as a first measure had I not confirmed it wasn't the issue.