Hi I need your assistance.
I have a table that I must synchronize in near real time.
There is however a couple of challenges that I'm sitting with:
The table (Table A) to by synchronized do not reside on the same server, but on two different servers. Server A (Production) and Server C (Development).
The table (Table A) on server A and server C do not have a primary key, so merge and transactional replication cannot work.
Setting triggers up on server A - the production server - is out of the question, this is a operation critical server.
Adding a primary key is also out of the question.
There is +/- 90 records being added to the table per minute.
Please can anyone assist?
Many thanks
You have ruled out so many options for any HA technologies to work..Only solution i could think of is
1.Write a custom solution in C#,which compares tables and does the necessary merge based on some column like date
2.Write the last written records into some table for the exe to start from there again
3.Trigger this through SQLServer Agent
You can also use AlwaysOn which provides readable secondaries as well,but i am not sure you be failing over production and development ..
Related
I'm looking for the best (best practice) option for one way replication between two databases. I would like to keep this purely SQL but, can write something in C# or use an ETL tool if there are no other good options.
Current setup:
DB1 - There are three instances of this database. It is a large relational database, the schema is the same for each but, they are separate data pots (no replication). Two databases on a 2012 server and one on a 2014 server
DB2 - There are two instances of this database on seperate servers (Europe, Americas) and the data is merge replicated between the two. The publisher is the 2014 server.
The Goal:
DB2 is tied to some reports. It has one table and a small application attached to that table. Users from many different countries enter data via a small application into DB2 and generate reports out of the application.
DB1 is a relational database that has a very large application on top of it but with fewer users. If users are using the application for DB1 then they should not need to duplicate their records into DB2.
There should be one-way replication from the multiple seperate DB1s into DB2. How quickly this happens is not too important.
The important things are:
No backwards replication occurs from DB2s > DB1s (Data only flows from DB1s into one of the DB2s)
Create, Update, and Delete actions should occur in DB2 based on the results
of a comparisson with DB1 (the one way replication)
Current Approach:
I currently have a flat sql view on each DB1 database that has the same schema as the table in the DB2 db's that the data needs to go into.
The servers are also joined as linked servers.
My though was to do a sort of manually written replication script on one of the DB2 databases that calls the views from the DB1s and does the CUD actions on a timed basis.
It seems to me that there should be an easier way though!?
Any thoughts on how to do this would be very much appreciated.
Keep in mind that since several of the DB1s exist on a SQL 2012 server that there may be some issues as 2012 might not be allowed to be a publisher for replication to a 2014 server.
Background
I have a production SQL Server 2005 server to which 4 different applications connect and make changes.
There are no foreign keys and in some cases no primary keys.
Unfortunately throwing the whole thing out and starting from scratch is not an option.
So my solution is to start migrating each of the applications to a service layer approach so that there is only one application directly connecting to the database.
However there are problems that need to be fixed before that service layer is written and all the applications are migrated over.
So rather than make changes and hope they don't break any one of the 4 badly written applications (with no way of quickly testing all functionality) my solution is to start auditing the database
Problem
How do I audit what stored procedures, tables, columns, views are being accessed/updated/called by each user on SQL Server 2005.
I can find out which tables are being updated but I have no idea which columns and by what users.
I also don't know if certain tables are being accessed only through stored procedures/views.
I know that SQL Server 2008 has better auditing features but if I could do this without spending money that would be great. That said if the best solution is to upgrade or buy software that's also an option.
Check out SQL Server 2008's CDC feature. You can't use this directly in 2005 but you can write a trigger for each table to log all data changes to a new audit table. i.e. you'd have an audit table for each table in your db, with all the same columns plus some additional columns saying what the operation was and when it occurred.
If the nature of your applications means you can get user information and/or application information from CURRENT_USER and APP_NAME() you could include that information in the audit table too.
And check out this answer for more goodness.
I’m after a bit of advice on the best way to go about this is SQL server 2008R2 express. I have a number of applications that are in separate databases on the same server. They are all “plugins” that use a central staff/structure list that will be in a separate database. The application is in the process of being migrated from JET.
What I’m looking for is the best way of all the “plugin” databases being able to see the central database and use those tables in standard queries and views etc.
As I’m using express that rules out any replication solution and so far the only option I can think of is to use triggers or a stored procedure to “push” out all the changes to the plugins. The information needs to be populated on a near enough real time basis however the number of changes will be very small maybe up to 100 a day and the biggest table only has about 1000 rows at the moment (the staff names table).
Hopefully that will cover all everything but if anyone needs any more details then just ask
Thanks
Apologies if I've misunderstood, but from your description it sounds like all these databases are hosted on the same instance of SQL Server - it's your mention of replication that makes me uncertain.
Assuming that's the case, you should be able to replace any copies of tables from the central database which are held in the "plugin" databases with views or synonyms which reference the central tables directly, since SQL server allows you to make references between databases on the same server using three-part naming (database_name.schema_name.object_name)
For example, if each plugin db has a table StaffNames, you could replace this with a view by dropping the table, then creating a view:
drop table StaffNames
go
create view StaffNames
as
select * from <centraldbname>.<schema - probably dbo>.StaffNames
go
and your code should continue to work seamlessly, as long as permissions are set up.
Alternatively, you could replace all the references to the shared tables in the plugin databases with three-part name references to the central database, but the view method requires less work.
This involves data replication, kind of:
We have many sites with SQL Express installed, there is an 'audit' database on each site that has one table in 1st normal form (to make life simple :)
Now I need to get this table from each site, and copy the contents (say, with a Date Time Value > 1/1/200 00:00, but this will change obviously) and copy it to a big 'super table' in sql server proper, that also has the primary key as the Site Name (That needs injecting in) and the current primary key from the SQL Express table)
e.g. Many SQL Express DBs with the following table columns
ID, Definition Name, Definition Type, DateTime, Success, NvarChar1, NvarChar2 etc etc etc
And the big super table needs to have:
SiteName, ID, Definition Name, Definition Type, DateTime, Success, NvarChar1, NvarChar2 etc etc etc
Where items in bold are the primary key(s)
Is there a Microsoft (or non MS I suppose) app/tool/thing to manager copying all this data accross already, or do we need to write our own?
Many thanks.
You can use SSIS (which comes with SQL Server) to populate, it can be set up with variables to change the connection string to the various databases. I have one that loops through the whole list and does the same process using three differnt files from three differnt vendors. You could so something simliar to loop through the different site databases. Put the whole list of database you want to copy the audit data from in a table and loop through it changing the connection string each time.
However, why on earth would you want one mega audit table per site? If every table in the database populates the audit table as changes happen, then the audit table eventually becomes a huge problem for performance. Every insert, update and delete has to hit this table and then you are proposing to add an export on top of that. This seems to me to be a guaranteed structure for locking and deadlocks and all sorts of nastiness. Do yourself a favor and limit each audit table to the table it is auditing.
Things to consider:
Linked servers and sp_msforeachdb as part of a do-it-yourself solution.
SQL Server Replication (by Microsoft) (which I believe can pull data from SQL Server Express)
SQL Server Integration Services which can pull data from SQL Server Express instances.
Personally, I would investigate Integration Services first.
Good luck.
You could do this with SymmetricDS. SymmetricDS is open source, web-enabled, database independent, data synchronization/replication software. It uses web and database technologies to replicate tables between relational databases in near real time. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outage.
As of right now, however, you would need to implement a custom IDataLoaderFilter extension point (in Java) to add the extra column. The metadata would be available though because your SiteName would be the external_id.
I have to migrate data in sqlserver from one table to another. Most of the standalone table were possible with select and insert into sp's.
The issue is in one table from the old database there is a column with a primary key say unit.
Now this unit is divided into two parts in the new database for example 'industry' and 'resource'.i.e industry + resource together constitute the unit.
How can i go about with the migration of this.
I have done similar "Database Transformation" tasks with SQL Server Integration Services it does require that you have SQL Server Standard or Enterprise (not Express) but because it is part of SQL Server there is no additional licencing, if it isn't installed you may need to re-run the installer and select "Integration Services".
SSIS will allow you to create a data flow, aranging the data more or less how you want it. There are several tutorials on MSDN.