For our senior design project my group is making a Silverlight application that utilizes graph theory concepts and stores the data in a database on the back end. We have a situation where we add a link between two nodes in the graph and upon doing so we run analysis to re-categorize our clusters of nodes. The problem is that this re-categorization is quite complex and involves multiple queries and updates to the database so if multiple instances of it run at once it quickly garbles data and breaks (by trying to re-insert already used primary keys). Essentially it's not thread safe, and we're trying to make it safe, and that's where we're failing and need help :).
The create link function looks like this:
private Semaphore dblock = new Semaphore(1, 1);
// This function is on our service reference and gets called
// by the client code.
public int addNeed(int nodeOne, int nodeTwo)
{
dblock.WaitOne();
submitNewNeed(createNewNeed(nodeOne, nodeTwo));
verifyClusters(nodeOne, nodeTwo);
dblock.Release();
return 0;
}
private void verifyClusters(int nodeOne, int nodeTwo)
{
// Run analysis of nodeOne and nodeTwo in graph
}
All copies of addNeed should wait for the first one that comes in to finish before another can execute. But instead they all seem to be running and conflicting with each other in the verifyClusters method. One solution would be to force our front end calls to be made synchronously. And in fact, when we do that everything works fine, so the code logic isn't broken. But when it's launched our application will be deployed within a business setting and used by internal IT staff (or at least that's the plan) so we'll have the same problem. We can't force all clients to submit data at different times, so we really need to get it synchronized on the back end. Thanks for any help you can give, I'd be glad to supply any additional information that you could need!
I wrote a series to specifically address this situation - let me know if this works for you (sequential asynchronous workflows):
Part 2 (has a link back to the part1):
http://csharperimage.jeremylikness.com/2010/03/sequential-asynchronous-workflows-part.html
Jeremy
Wrap your database updates in a transaction. Escalate to a table lock if necessary
Related
Good morning everyone,
We are implementing a method that stores a document into SqlServer with EF Core.
This method called SAVE is called by multiple endpoints of the controller and is used both from a createDocument endpoint and another endpoint putDocument.
An extra requirement is that this method contains two calls that store the document and its properties into 2 different repositories, therefore we wanted to achieve that if one of the two calls to the repositories fails, it rolls back the changes.
Moreover, the whole code is hosted on multiple machines that's why we also implemented the library of distributed lock of macallon (GitHub) in order to avoid that the same document is accessed/modified by two different machines at the same time.
Is it correct to use a mixup of these solutions? Being a noob for me a transactionScope is already a lock, in a sense that during the transaction other machines cannot even access the same row in the Db but please help me to understand. Thanks
public void Save(Document document){
using var scope = new TransactionScope(TransactionScopeOption.RequiresNew,
TransactionScopeAsyncFlowOption.Enabled);
try
{
using (var lockHandler = documentLock.CreateLockForGuid(document.Guid))
{
repositoryOne.Save(document);
repositoryTwo.Save(document);
scope.Complete();
}
}
}
lockhandler is just a wrapper which calls the distributedLock, SqlDistributedReaderWriterLock with the document.Guid as its name
I have a WPF application that uses entity framework. I am going to be implementing a repository pattern to make interactions with EF simple and more testable. Multiple clients can use this application and connect to the same database and do CRUD operations. I am trying to think of a way to synchronize clients repositories when one makes a change to the database. Could anyone give me some direction on how one would solve this type of issue, and some possible patterns that would be beneficial for this type of problem?
I would be very open to any information/books on how to keep clients synchronized, and even be alerted of things other clients are doing(The only thing I could think of was having a server process running that passes messages around). Thank you
The easiest way by far to keep every client UI up to date is just to simply refresh the data every so often. If it's really that important, you can set a DispatcherTimer to tick every minute when you can get the latest data that is being displayed.
Clearly, I'm not suggesting that you refresh an item that is being edited, but if you get the fresh data, you can certainly compare collections with what's being displayed currently. Rather than just replacing the old collection items with the new, you can be more user friendly and just add the new ones, remove the deleted ones and update the newer ones.
You could even detect whether an item being currently edited has been saved by another user since the current user opened it and alert them to the fact. So rather than concentrating on some system to track all data changes, you should put your effort into being able to detect changes between two sets of data and then seamlessly integrating it into the current UI state.
UPDATE >>>
There is absolutely no benefit from holding a complete set of data in your application (or repository). In fact, you may well find that it adds detrimental effects, due to the extra RAM requirements. If you are polling data every few minutes, then it will always be up to date anyway.
So rather than asking for all of the data all of the time, just ask for what the user wants to see (dependant on which view they are currently in) and update it every now and then. I do this by simply fetching the same data that the view requires when it is first opened. I wrote some methods that compare every property of every item with their older counterparts in the UI and switch old for new.
Think of the Equals method... You could do something like this:
public override bool Equals(Release otherRelease)
{
return base.Equals(otherRelease) && Title == otherRelease.Title &&
Artist.Equals(otherRelease.Artist) && Artists.Equals(otherRelease.Artists);
}
(Don't actually use the Equals method though, or you'll run into problems later). And then something like this:
if (!oldRelease.Equals(newRelease)) oldRelease.UpdatePropertyValues(newRelease);
And/Or this:
if (!oldReleases.Contains(newRelease) oldReleases.Add(newRelease);
I'm guessing that you get the picture now.
In his book on DbContext, #RowanMiller shows how to use the DbSet.Local property to avoid 1.) unnecessary roundtrips to the database and 2.) passing around collections (created with e.g. ToList()) in the application (page 24). I then tried to follow this approach. However, I noticed that from one using [} – block to another, the DbSet.Local property becomes empty:
ObservableCollection<Destination> destinationsList;
using (var context = new BAContext())
{
var query = from d in context.Destinations …;
query.Load();
destinationsList = context.Destinations.Local; //Nonzero here.
}
//Do stuff with destinationsList
using (var context = new BAContext())
{
//context.Destinations.Local zero here again;
//So no way of getting the in-memory data from the previous using- block here?
//Do I have to do another roundtrip to the database here to get the same data I wanted
//to cache locally???
}
Then, what is the point on page 24? How can I avoid the passing around of my collections if the DbSet.Local is only usable inside the using- block? Furthermore, how can I benefit from the change tracking if I use these short-lived context instances not handing over any cache data to each others under the hood? So, if the contexts should be short-lived for freeing resources such as connections, have I to give up the caching for this? I.e. I can’t use both at the same time (short-lived connections but long-lived cache)? So my only option would be to store the results returned by the query in my own variables, exactly what is discouraged in the motivation on page 24?
I am developing a WPF application which maybe will also become multi-tiered in the future, involving WCF. I know Julia has an example of this in her book, but I currently don’t have access to it. I found several others on the web, e.g. http://msdn.microsoft.com/en-us/magazine/cc700340.aspx (old ObjectContext, but good in explaining the inter-layer-collaborations). There, a long-lived context is used (although the disadvantages are mentioned, but no solution to these provided).
It’s not only that the single Destinations.Local gets lost, as you surely know all other entities fetched by the query are, too.
[Edit]:
After some more reading in Julia Lerman’s book, it seems to boil down to that EF does not have 2nd level caching per default; with some (considerable, I think) effort, however, ones can add 3rd party caching solutions, as is also described in the book and in various articles on MSDN, codeproject etc.
I would have appreciated if this problem had been mentioned in the section about DbSet.Local in the DbContext book that it is in fact a first level cache which is destroyed when the using {} block ends (just my proposal to make it more transparent to the readers). After first reading I had the impression DbSet.Local would always return the same reference (Singleton-style) also in the second using {} block despite the new DbContext instance.
But I am still unsure whether the 2nd level cache is the way to go for my WPF application (as Julia mentions the 2nd level cache in her article for distributed applications)? Or is the way to go to get my aggregate root instances (DDD, Eric Evans) of my domain model into memory by one or some queries in a using {} block, disposing the DbContext and only holding the references to the aggregate instances, this way avoiding a long-lived context? It would be great if you could help me with this decision.
http://msdn.microsoft.com/en-us/magazine/hh394143.aspx
http://www.codeproject.com/Articles/435142/Entity-Framework-Second-Level-Caching-with-DbConte
http://blog.3d-logic.com/2012/03/31/using-tracing-and-caching-provider-wrappers-with-codefirst/
The Local property provides a “local view of all Added, Unchanged, and Modified entities in this set”. Like all change tracking it is specific to the context you are currently using.
The DB Context is a workspace for loading data and preparing changes.
If two users were to add changes at the same time, they must not know of the others changes before they saved them. They may discard their prepared changes which suddenly would lead to problems for other other user as well.
A DB Context should be short lived indeed, but may be longer than super short when necessary. Also consider that you may not save resources by keeping it short lived if you do not load and discard data but only add changes you will save. But it is not only about resources but also about the DB state potentially changing while the DB Context is still active and has data loaded; which may be important to keep in mind for longer living contexts.
If you do not know yet all related changes you want to save into the database at once then I suggest you do not use the DB Context to store your changes in-memory but in a data structure in your code.
You can of course use entity objects for doing so without an active DB Context. This makes sense if you do not have another appropriate data class for it and do not want to create one, or decide preparing the changes in them make more sense. You can then use DbSet.Attach to attach the entities to a DB Context for saving the changes when you are ready.
This seriously is one of the biggest thorns in my side. SFDC does not allow you to use complex objects or collections of objects as parameters to a future call. What is the best workaround for this?
Currently what I have done is passed in multiple parallel arrays of primitives which form a complete object based on the index. Meaning if I need to pass a collections of users, I may pass 3 string arrays, say - Name[], Id[], and Role[]. Name[0], Id[0]. and Role[0] are the first user, etc. This means I have to build all these arrays and build the future method to reconstruct the relevant objects on the other end as well.
Is there a better way to do this?
As to why, once an Apex "transaction" is complete, the VM is destroyed. And generally speaking, salesforce will not serialize your object graph for resuming at a future time.
There may be a better way to get this task done. Can the future method query for the objects it needs to act on? Perhaps you can pass the List of Ids and the future method can use this in a WHERE clause. If it's a large number of objects, batch apex may be useful to avoid governor limits.
I would suggest creating a new custom object specifically for storing the information required in your custom apex class. You can then insert these into the database and then query for the records in the #future method before using them for the callout.
Then, once the callout has completed successfully you can then delete those records from the database to keep things nice and tidy.
My answer is essentially the same. What I do is prepare a custom queue object with all relevant Ids (User/Contact/Lead/etc.) along with my custom data that then gets handled from the #Future call. This helps with governor limits since you can pull from the queue only what your callout and future limitations will permit you to handle in a single thread. For Facebook, for example, you can batch up 20 profile updates per single callout. Each #Future allows 10 callouts and each thread permits 10 #Future calls which equals 2000 individual Facebook profile updates - IF you're handling your batches properly and IF you have enough Salesforce seats to permit this number of #Future calls. It's 200 #Future calls per user per 24 hours last I checked.
The road gets narrow when you're performing triggered callouts, which is what I assume you're trying to do based on your need to callout in an #Future method in the first place. If you're not in a trigger, then you may be able to handle your callouts as long as you do them before processing any DML. In other words, postpone any data saves in any particular thread until you're done calling out.
But since it sounds like you need to call out from a trigger, batching it up in sObjects is really the way to go. It's a bit of work, but essentially serializing your existing heap data is the road to travel here. Also consider doing this from an hourly scheduled Batch Apex call since with the queue approach you'll be able to process all of your callouts eventually. If you run into governor limits (or rather, avoid hitting them) in a particular thread, it will wake up an hour later and finish the work left in your queue. Launching that process looks something like this:
String jobId = System.schedule('YourScheduleName', '0 0 0-23 * * ?', new ScheduleableClass());
This will instantiate an instance of ScheduleableClass once an hour which would pull the work from your queue object and process the maximum amount of callouts.
Good luck and sorry for the frustration.
Just wanted to give my answer on how I do this very easily in case anyone else stumbles across this question. Apex has functions to easily serialize and de-serialize objects to and from JSON encoding. Let's say I have a list of cases that I need to do something with in a future call:
String jsonCaseList = '';
List<Case> caseList = [SELECT id, Other fields FROM Case WHERE some conditions];
//Populate the list
//Serialize your list
jsonCaseList = JSON.serialize(caseList);
//Pass jsonCaseList as a string parameter to your future call
futureCaseActivity(jsonCaseList);
#future
public static void futureCaseActivity(string jsonCases){
//De-serialize the string back into a list of cases
List<Case> futureCaseList = (List<Case>)JSON.deserialize(jsonCases, List<Case>);
//Do whatever you want with your cases
for(Case c : futureCaseList){
//Stuff
}
Update futureCaseList;
}
Anyway, seems like a much better option than adding database clutter with a new custom object and prevents needing to query the database again for info you already have, which just makes me hurt inside.
Almost forgot to add the link: https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_json_json.htm
In my project I want remove some rows first then afterwards insert new rows.
But some times what happens is it inserts the new rows first then afterwards removes the starting rows.
To solve this problem I need to manage the operations in a proper sequence.
Please help me out.
This is a common pattern/problem with Silverlight as pretty much "everything" is asynchronous (for good reasons).
Depending on how your Adds and Removes are triggered, you could queue up tasks (e.g. a list of delegates) and have each task execute the next one off the list when they complete.
The alternative is going to sound a little complex, but the solution we came up with is to create a SequentialAsynchronousTaskManager class that operates in a similar way to the SilverlightTest class which uses EnqueueConditional() methods to add wait conditions and EnqueueCallback()s to execute code.
It basically holds a list of delegates (which can be simple Lambda expressions) and either executes it regularly until it returns true (EnqueueConditional) or just executes some code (EnqueueCallback).