In SymmetricDS we've been replicating successfully from master to children and children to master for years. We're now needing to replicate from master to master (mirror) where the latter master is a mirrored reporting database. Keep in mind that some tables between the master/child relationship are asynchronous. When we build out the SymmetricDS configurations for the master we see a successful initial load. We also see successful replication between master --> mirror. What we cannot resolve is the replication between child --> master --> mirror. Any changes occurring on a child, as usual, replicates to the master, but nothing moves to the mirror. If a change happens directly on the master, the mirror is updated. We need both.
Configure sync on incoming batch as explained in the chapter on bi-directional synchronization https://www.jumpmind.com/downloads/symmetricds/doc/3.10/html/user-guide.html#_bi_directional_synchronization
I believe I've found a work-around. In sym_data the node_id is recorded to prevent looping. This value represents a child node. The mirror master has no contact or relationship with a child node, so when the master trigger fires, it records the child node and so the master mirror ignores it. If I modify the triggers to hard code the master node in the result instead of the child node the master mirror then processes the result. Again, I believe there is a limitation in SymmetricDS with regards to the sym_node table as it should not require a node_group_id (a node should be allowed to join many node_groups).
Related
After configuring replication all the data from tables has replicated from publisher to subscriber. Now my requirement is whenever I insert new record in publisher only that record should replicate rather than reinitialize all the records to subscriber because my table is having millions of records.
Rather than drop and reinitialize snapshot cont we store earlier sync data of table in subscriber and only sync the new record.
Is there any way to do this?
If you're using transactional replication (as you say above in your comments), something is configured incorrectly. Transactional replication is supposed to work as you describe (i.e. a one row change gets pushed over and applied as a one row change). I'd guess that you have the immediate_sync property turned on at the publisher and are running the snapshot agent on a schedule. At the very least, turn off the schedule for the snapshot agent and I think you'll see your problem go away.
We have a very weird problem using EF 6 with MSSQL and MassTransit with Rabbit MQ.
The scenario is as follows:
Client application inserts a row in database (via EF code first - implicit transaction only in DbContext SaveChanges)
Client application publishes Id of the row via MassTransit
Windows Service with consumers processes the message
Row is not found initially, after a few retries, the row appears
I always thought that after commit the row is persisted and becomes accessible for other connections...
We have ALLOW_SNAPSHOT_ISOLATION on in the database.
What is the reason of this and is there any way to be assured that the row is accessible before publishing the Id to MQ?
If you are dependent upon another transaction being completed before your event handler can continue, you need to make you read serializable. Otherwise, transactions are isolated from each other and the results of the write transaction are not yet available. Your write may also need to be serializable, depending upon how the query is structured.
Yes, the consumers run that quickly.
Solr replicates from master to slave on either commit,optimize or startup.However i m setting up an empty core which acts as a slave core and I want it to replicate data from the master core without firing a commit operation on master core.
You can Force Replication on the slave from the Solr Admin dashboard.
You can also fire replication through url fetchindex
http://slave_host:port/solr/replication?command=fetchindex
Forces the specified slave to fetch a copy of the index from its
master.
If you like, you can pass an extra attribute such as masterUrl or
compression (or any other parameter which is specified in the tag) to do a one time replication from a master. This
obviates the need for hard-coding the master in the slave.
OK when working with table creation, is it always assumed that creating a table on one database (the master) mean that the DBA should create the table on the slave as well? Also, if using a master/slave configuration, shouldn't data always be getting replicated from the master to the slave to synch?
Right now the problem I am having is my database has a lot of stuff in the master, but the slave is missing parts that only exist in the master. Is something not configured correctly here?
Depends how the replication is configured. Real time replication should keep the master and slave in sync at all times. "Poors mans" replication is usually configured to sync upon some time interval expiring. This is whats probably happening in your case.
I prefer to rely on CREATE TABLE statements being replicated to set up the table on the slave, rather than creating the slave's table by hand. That, of course, relies on the DBMS supporting this.
If you have data on the master that isn't on the slave, that's some sort of failure of replication, either in setup or operationally.
Any table creation on master is replication on slave. Same goes with the inserting data.
Go through the replication settings in my.cnf file for mysql and check if any database / table is ignored from replicating.
I have a scenario where I'm using transactional replication to replicate multiple SQL Server 2005 databases (same instance) into a single remote database (different instance on a separate physical machine).
I am then performing some processing on the replicated data for reporting purposes. I'm using table level triggers to identify changes which actions my post processing code.
Up to this point everything is fine.
However, what I'd like to know is, where certain tables are created, updated or deleted in the same transaction, is it possible to identify some sort of transaction ID from replication (or anywhere) so then I don't perform the same post processing multiple times for a single transaction.
Basic Example: I have a TUser Table and TAddress table. If I was to create both in a single transaction, they would be replicated across in a single transaction too. However, there would be two triggers fired in the replicated database - which at present causes my post processing code to be run twice. What I'd really like to identify is that these two changes arrived in the replicated in the same transaction.
Is this possible in any way? Does an identifier as I've describe exist and is it accessible?
Short answer is no, there is nothing of the sort that you can rely on. Long answer in summary would be that yes it exists, but it would not be recommended in any way to be used for anything.
Given that replication is transactionally consistent, one approach you could consider would be pushing an identifier for the primary record (in this case TUser, since an TAddress is related to TUser) onto a queue (using something like Service Broker ideally or potentially a user-defined queue) and then perform the post-processing by popping data off the queue and processing separately.
Another possibility would be simply batch processing every 'x' amount of time by polling for new/updated records from the primary tables and post-processing in that manner - you'd need to track id's, rowversions, or timestamps of some sort that you've processed for each primary table as meta-data and pull anything that hasn't yet been processed during each batch run.
Just a few thoughts, hope that helps.