we currently have a java based application which uses JPA via eclipselink against a single MS sql server, this is running on a system in the internal network.
This works fine.
Now we need to add a second instance of the sql database in a DMZ with it's own application and JPA eclipselink. (External access to a subset of the application)
Both applications should see the same data, which means I will have to do some replication between the two.
MS SQL has a merge replication system for this which would cover our needs.
But here I struggle with the JPA cache.
When the data is changed directly by the db replication, how/when do I invalidate the local jpa cache?
Or is there a java/jpa/jdbc based replication available for such setups?
Any other ideas on how to handle such a situation?
EMF L2 cache is cleared using
emf.getCache().evictAll()
Related
I would like to create a simple console app that automatically copies the database from one server to another... I want the logic to be loosely coupled with the models...
The logic should work as long as the app has access to both servers.
I don't want to create a backup, I want to access the live database on one server and copy it to another without first backing it up.
Can .NET EF Core do that for SQL Server database?
Using backup is by far the best way. It also guarantees transactional consistency. If transactional consistency is for some reason not important, you can use bcp: https://learn.microsoft.com/en-us/sql/relational-databases/import-export/import-and-export-bulk-data-by-using-the-bcp-utility-sql-server?view=sql-server-2017
I have a simple question if someone can use the H2 Database while development with spring boot and Spring Data JPA and after completing the development, shifting to database like oracle .?
You can use the in-memory database for production considering the limitations.
From Spring Docs (31.1.1),
It is often convenient to develop applications by using an in-memory embedded database. Obviously, in-memory databases do not provide persistent storage. You need to populate your database when your application starts and be prepared to throw away data when your application ends.
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html
It is IMHO OK until you need to do some performance testing. Performance of H2 is worse than that of, e.g., PostgreSQL and Oracle.
I'm relatively new to Azure and am having trouble finding what options are out there for connecting to an existing SQL database to push data into it.
The situation is that we have an external client who needs to connect to our Azure SQL database to push data into it, on an on-going basis. We can't give them permission to get into our database, so we're looking at what we can do allow data in. At this point the best option seems to be to create a web service deployed in Azure that will validate the data and then push it into our database.
The question I have is, are there other options to do this in an easier way? Are there Azure services or processes that can be set up to automatically process a file and pull the data into a database? Any other go-between options when each side has their own database and for security reasons can't just open up access to it?
Azure Data Factory works great for basic ETL. If neither party can grant direct access, you can use an intermediate repository like Blob Storage to drop csv/xml/json files for ingestion. If they'll grant you access to pull, you can setup a linked service that more or less functions the same as a linked server in MSSQL. As of the last release ADF now supports Azure hosted SSIS packages too.
I would do this via SSIS using SQL Studio Managemenet Studio (if it's a one time operation). If you plan to do this repeatedly, you could schedule the SSIS job to execute on schedule. SSIS will do bulk inserts using small batches so you shouldn't have transaction log issues and it should be efficient (because of bulk inserting). Before you do this insert though, you will probably want to consider your performance tier so you don't get major throttling by Azure and possible timeouts.
I have a WinForms business application that connects to a SQL Server on a server within the business network. Recently we have added an ASP.NET web site so some of the information within the system can be accessed from the Internet. This is hosted on the same server as the SQL Server.
Due to the bandwidth available to the business network from the Internet we want to host the web site with a provider but it needs access to the SQL Server database.
95% of data changes are made by the business using the WinForms application. The web site is essentially a read only view of the data but it is possible to add some data to the system which accounts for the other 5%.
Is replication the best way to achieve the desired result e.g. SQL Server within the business network remains the master database as most changes are made to this and then replicate this to the off site server? If so which type of replication would be the most suitable and would this support replicating the little data entered from the ASP.NET web site back to the main server?
The SQL Server is currently 2005 but can be upgraded as required for any replication requirements.
Are there other solutions to this problem?
Yes, since the web application is causing 5% (max) transaction; you can separate it.
I mean, you can have a different DB which is a carbon copy of the master one and have web application point to this DB.
You can setup a bi-directional transaction replication. So that, transaction made to the master DB will get replicated as well as transaction made to the secondary DB will be replicated as well.
No need of upgrading; as SQL Server 2005 supports replication.
For further information check MSDN on replication here: Bidirectional Transactional Replication
In a Nutshell, here are the steps you would do:
Take a full backup pf the master DB
Restore the DB to newly created DB server
Configure trans replication between them.
For better performance, you can also have the primary DB mirrored onto someother DB server.
We have an application that requires our customers to have a SQL server instance on site. At their request, the application needs to synchronize the data in their database with a copy in our datacenter.
We're using .Net 3.5 SP1. We need to synchronize the data exactly, including IDENTITY columns.
We'd prefer to use something like LINQ to SQL that would let us make some simple select and insert/update calls against mapped entities. However, the IDENTITY columns seem to be a problem with LINQ and similar approaches.
We can do this all with built-up SQL statements and turn IDENTITY INSERT on / off as needed, but I'd prefer a more elegant solution.
Thanks!
** Edit - We DO need to write our own solution, and we do need to use .Net 3.5 SP1 to do it. I won't spend your time explaining all the reasons why, but please limit suggestions to options within the .Net playground.
Microsoft Sync Framework can be your solution. This is framework description from Microsoft:
Microsoft Sync Framework is a data synchronization platform from Microsoft that can be used to synchronize data across multiple data stores. Sync Framework includes a transport-agnostic architecture, into which data store-specific synchronization providers, modelled on the ADO.NET data provider API, can be plugged in.
Sync Framework is a comprehensive data synchronization solution that enables developers to build solutions that support synchronization of any database, on any data protocol over any network topology. msdn.microsoft.com
For your convinience providing link to good tutorial on the subject
If it is just a couple of tables that need to be synchronized and there is not a lot of data in the tables (now and future) you could develop some sort of bulk copy from your servers and bulk insert routine on the customer's server.
Since you said you can't use SQL Server replication services or SSIS, then perhaps a backup/restore procedure could be written. You could take a scheduled backup of your database and make it available to calling applications which could then copy the backup, restore it to another instance on the customers server, then pull all data you need via any number of methods and it would exist locally on the customers servers.
Beyond that, I think you may be asking for a maintenance and synchronization nightmare if you can't base your solution on tools that are made to do this sort of thing.