I have a WCF App that has 5-6 EF Models in it. In a production env, only one instance of this app will be running.
But in development there are 5 developers working on it at the same time. (Against the same Database.)
We are noticing that sometimes tables on our SQL Server 2008 R2 DB get locked. It seems to be when someone is doing step over debugging and has to leave it on a step for a few minutes.
I am curious why EF would keep a lock on a table. How would someone keep an lock on a table using EF? What Can I do to prevent this?
NOTE: This same application accesses a WCF Data Services (OData) endpoint to get some of its data (from the same database). I don't see how OData would be locking the db, but I thought I would mention it in case it is important.
There is only one solution. Each developer will have locally installed database and run debugging session in his own environment! Anything else is wrong development environment. Use SQL Server Express or SQL Server Developer edition.
All we can do is venture various guesses. Eg. if your data model is missing proper indexes then record lookups turn into table scans and scans escalate locks to table level.
The real solution is to investigate the blocking properly. What is causing the block, exactly, what resource is waiting on? What session/transaction/statement is holding the resource needed by what other session/transaction/statement.
Use the Activity Monitor or who_is_active or sp_blitz. Read the Waits and Queues whitepaper.
Related
I have a .NET core app which I'am running on AWS Elastic Container Services (ECS).
- The app runs on two different instances.
- Database is SQL server
The app runs the database migrations on startup, which has worked really well. But then i had to migrate a lot of data which meant that the migration took longer time. This resulted in duplicates of the data being moved.
This happens because both apps first checks the database if the migration has been executed, both finds out that it hasn't, then both starts running the migration which takes time. After it is done it adds the migration to the database.
How do people solve this?
Possible solutions I and others have thought of
Start with only one instance of the app, then scale up.
this would work, but then I will have to manually scale down and up for each time there is a migration. (It is possible to do it automatically, but it would take time)
Wrap long running migrations in transactions and at the start set the migration as done in the database. Check if it is in the database before commiting the change. If the transaction fails, remove the migration from the database.
Lock the database? EF Core lock the database during migration . Seems weird.
Make the migration a part of the deployment process. This seems to be best practice, but it would mean that the Build server would need to know the Database secrets. I'am not to afraid to give it, but it would mean i would have to maintain a duplicate set.
What does people out there do? Am I missing some obvious solution?
Thanks
We also used to have our applications perform the migration, but even Microsoft recommends avoiding this in a multi-instance environment:
We recommend production apps should not call Database.Migrate at application startup. Migrate shouldn't be called from an app in server farm. For example, if the app has been cloud deployed with scale-out (multiple instances of the app are running).
Database migration should be done as part of deployment, and in a controlled way.
Like everything there are different ways to go about solving the problem. Our team is small and so we generate migration scripts through the EF CLI tooling and then run them manually as part of a deployment/maintenance routine. This could of course be automated if your process warrants it.
As far as I can tell, scaling an Azure DB from the retired tiers to the new tiers is simply a matter of using the scale function in the Azure portal.
What I cannot seem to find anywhere is a definitive answer as to whether there are any connection string changes required (or any other issues that could cause unavailability) when scaling from the retired to new tiers.
I have a production database that needs to be upgraded, service interruption would be very bad.
The scale operation will not change the connection string. You could face some (very small, but) finite amount of downtime while the switchover happens.
Please refer to the documentation for details. Note that you will be have to suspend geo-replication (if already enabled) for the duration of the upgrade.
Techincaly it will be the same server, same connection string, same everything, but version and features.
But I would be concerned about the following statement from docu:
The duration of upgrade depends on the size, edition and number of
databases in the server. The upgrade process can run for hours to days
for servers especially for servers that has databases:
Larger than 50 GB, or
At a non-premium service tier
Which is kind of concerning.
What I would do, if possible is:
Put my service into read-only mode (put on hold any writes to the DB)
Create new db in same server from the existing one with the command - CREATE DATABASE AS COPY OF ...
When creation of DB is ready, export the new db to backpac and delete the DB when export is ready.
Perform upgrade.
In theory you could do the process without putting your system into Read-Only mode, but I am just taking more precautions measures.
And yes, you also have to aware that you are upgrading your Azure SQL DB Server not just a single Database.
I would like to know which is the best way to make a copy and keep the copies synchronized of a on premises SQL Server 2008 (not R2) database to SQL Azure.
Think of the SQL Azure as a failover kind of structure...
Notes:
The database runs fine in SQL Azure
I have already figured out how to get the rest of the app running on Azure
Please consider suggestions of the type "Upgrade to SQL Server 2012 because of X" if the gain (reliability, efficiency, time to replicate, etc...) are worth it
I`m looking for instant replication (as fast as possible)
Yes it will have to sync back eventually. If the on-premises deploy crash and the cloud get activated and changed, sync back will be necessary, but i think it does not need to be automatic... of it is, better!
The Database consist of 900+ tables (legacy system)
http://www.windowsazure.com/en-us/manage/services/sql-databases/getting-started-w-sql-data-sync/
http://msdn.microsoft.com/en-us/library/hh456371.aspx
I think the best bet is to use SQL Data Sync, it should give you bidirectional and we use it currently to sync data around the world in terms of datacenters and one local on premise database. It will only give you 5 mins sync timing but this will probably do, otherwise the next best options is to use SQL Server VMs and do the old fashion way. But with SQL Azure Data Sync we have found to be reasonable reliable and been running it for a good six months syncing across 4 database in four data centres in Azure.
Some problems though with it,
It uses Triggers.
It will obivously add load and connections to your current SQL Database.
The new control panel in Azure is a nightmare for it, so I would use the old panel for the moment.
It is in preview last time I looked, so it might not be 100% suitable
for you.
I would imagine there is some better third party solutions out there but off the shelf and in Azure SQL Data sync is well worth a look for the situation you a describing.
I have just lost pretty much a day and a half trying to get pull replication going for an off site server. After experiencing the pain that was this experience I am now thinking it shouldn't be this hard so maybe I am doing it wrong. I never did get it to work, I had to go to push replication.
Here is the situation. We have a virtual server being hosted off site that will a database for a public web application. We want to push all the data from a few of our tables in our internal database to this off site location and it has to be done almost instantaneously so that the web information is current. We don't want to set up VPN because if that machine gets compromised we don't want that vulnerability.
If sql server replication is not the best method how would you do it?
FYI: Publisher = SQL Server 2005 & Subscriber = SQL Server 2008 Web Edition
Well, if you want it to be fast and easy to manage, one solution is to set a merge replication topology, having your main server as a publisher, and the hosted server as a suscriber. Replication can then be done through http, thus without vpn.
Be careful: web replication is not as straighforward as opening a page in your browser! You can find some interesting infos here
I do not have SQL Studio on this machine, but I guess you can parameter your subscription in such a way that only downloads will be replicated to the suscriber, while uploads will be ignored.
By running the replication script (it's a BAT file) from the suscriber every minute (through any scheduled tasks manager), you can have a quasi-instantaenous update of your suscriber's tables.
I am building an Asp.net MVC site where I have a fast dedicated server for the web app but the database is stored in a very busy Ms Sql Server used by many other applications.
Also if the web server is very fast, the application response time is slow mainly for the slow response from the db server.
I cannot change the db server as all data entered in the web application needs to arrive there at the end (for backup reasons).
The database is used only from the webapp and I would like to find a cache mechanism where all the data is cached on the web server and the updates are sent to the db asynchronously.
It is not important for me to have an immediate correspondence between read db data and inserted data: think like reading questions on StackOverflow and new inserted questions that are not necessary to show up immediately after insertion).
I thought to build an in between WCF service that would exchange and sync the data between the slow db server and a local one (may be an Sqllite or an SqlExpress one).
What would be the best pattern for this problem?
What is your bottleneck? Reading data or Writing data?
If you are concerning about reading data, using a memory based data caching machanism like memcached would be a performance booster, As of most of the mainstream and biggest web sites doing so. Scaling facebook hi5 with memcached is a good read. Also implementing application side page caches would drop queries made by the application triggering lower db load and better response time. But this will not have much effect on database servers load as your database have some other heavy users.
If writing data is the bottleneck, implementing some kind of asyncronyous middleware storage service seems like a necessity. If you have fast and slow response timed data storage on the frontend server, going with a lightweight database storage like mysql or postgresql (Maybe not that lightweight ;) ) and using your real database as an slave replication server for your site is a good choise for you.
I would do what you are already considering. Use another database for the application and only use the current one for backup-purposes.
I had this problem once, and we decided to go for a combination of data warehousing (i.e. pulling data from the database every once in a while and storing this in a separate read-only database) and message queuing via a Windows service (for the updates.)
This worked surprisingly well, because MSMQ ensured reliable message delivery (updates weren't lost) and the data warehousing made sure that data was available in a local database.
It still will depend on a few factors though. If you have tons of data to transfer to your web application it might take some time to rebuild the warehouse and you might need to consider data replication or transaction log shipping. Also, changes are not visible until the warehouse is rebuilt and the messages are processed.
On the other hand, this solution is scalable and can be relatively easy to implement. (You can use integration services to pull the data to the warehouse for example and use a BL layer for processing changes.)
There are many replication techniques that should give you proper results. By installing a SQL Server instance on the 'web' side of your configuration, you'll have the choice between:
Making snapshot replications from the web side (publisher) to the database-server side (suscriber). You'll need a paid version of SQLServer on the web server. I have never worked on this kind of configuration but it might use a lot of the web server ressources at scheduled synchronization times
Making merge (or transactional if requested) replication between the database-server side (publisher) and web side(suscriber). You can then use the free version of MS-SQL Server and schedule the synchronization process to run according to your tolerance for potential loss of data if the web server goes down.
I wonder if you could improve it adding a MDF file in your Web side instead dealing with the Sever in other IP...
Just add an SQL 2008 Server Express Edition file and try, as long as you don't pass 4Gb of data you will be ok, of course there are more restrictions but, just for the speed of it, why not trying?
You should also consider the network switches involved. If the DB server is talking to a number of web servers then it may be being constrained by the network connection speed. If they are only connected via a 100mb network switch then you may want to look at upgrading that too.
the WCF service would be a very poor engineering solution to this problem - why make your own when you can use the standard SQLServer connectivity mechanisms to ensure data is transferred correctly. Log shipping will send the data across at selected intervals.
This way, you get the fast local sql server, and the data is preserved correctly in the slow backup server.
You should investigate the slow sql server though, the performance problem could be nothing to do with its load, and more to do with the queries and indexes you're asking it to work with.