Most of the operations in my silverlight client are things that add/update/insert/delete multiple entities in one go.
E.g:
CreateStandardCustomer adds a Customer, Address, Person and Contract record.
CreateEnterpriseCustomer adds a Customer, Address, 2x Person and a CreditLimit record.
It looks like with a DomainService you can only do one thing at a time, e.g. add a customer record, add an address etc. How can I do a batch operation?
You might say to simply add the relevant records from the Silverlight client and call the SubmitChanges() method. However this is difficult to validate against (server side) because only certain groups of records can be added/update/deleted at a time. E.g. in the example above, an Address record added alone would not be valid in this system.
Another example would be something like Renew which updates a Customer record and adds a Renewal. These operations aren't valid individually.
Thanks for your help,
Kurren
EDIT: The server side validation needs to check that the correct operations in the batch has taken place. E.g. From the example above we Renew then a Renewal should be created and a Customer should have been updated (one without the other is invalid).
I may be missing something here, but you update a batch of entities the same way you do individual entities: namely perform all the operations on your context and then call SubmitChanges on that context. At the server your insert/delete/update methods for the types will be called as appropriate for all the changes you're submitting.
We use RIA/EF in Silverlight to do exactly that. It doesn't matter if you just create a single entity in your client context (complete with graph) or 100, because as soon as you submit those changes the complete changeset for that context is operated upon.
EDIT: Failing setting up your entity metadata with Required and Composition attributes on the appropriate properties, you can also use the DomainService.ChangeSet object to inspect what has been submitted and make decisions on what changes you want to accept or not.
Related
I have a model in my GAE based application, which is being updated quite frequently. And in some cases, it happens that same entity is being updated nearly at the same time. The entity update functionality in the app works like that:
User enters an ID and other properties of the entity which are to be updated.
Retrieve entity from DB against this ID (to make sure ID was valid).
Do bunch of validation on the properties which are going to be updated (e.g. if group_id property is being updated then make sure group is present in DB against this ID and ID is integer.)
After validations, call put() on the entity which was retrieved at step #2.
As, i mentioned same entity can be updated multiple times nearly at the same time, so i came across the classical race condition issue. i.e. let's say 2 update calls were made sequentially, in first call entity was retrieved and validations were in process but, at the same time second call triggered it retrieves the same and updates it properties. The second call also executes put() and update entity in DB. NOTE that first call was not completed yet (due to some time delay), it completes now and calls put() and updates entity in DB.
The end result in DB will be off first update call, but expected result was of second call!
I researched on GAE about this and found pre-put hooks. I think i can use "updated" timestamps to solve this issue i.e. to make sure second call only updates an entity after first call does it. But, i want to use some better approach here e.g. some DB's (like in AWS) provide tags with each DB row and we can ask DB itself to validate this tag before actually putting the record. I am curious as is there any way in GAE to do so. i.e. to ask DB to do conditional put() instead of using pre_put hooks manually?
we have an custom object in our instance that effectively is a junction object. Right now, if a relationship is removed, the record in the junction object is deleted.
We want to change this behavior to such that the junction object is marked as deleted, but not physically deleted (please understand that I cannot go into details of why, there are good business reasons to do so). Since we have multiple clients accessing our instance through SOAP and REST APIs I would like to implement a solution whereby I override the standard delete functionality of the custom object to just check a custom field is_deleted, instead of deleting the record.
Is this possible?
Cheers,
Dan
I suppose you can't just put an on-delete trigger on the object?
If you can, then just add the trigger code to update the field, and then attach an error to the record being deleted (so the deletion doesn't go through). There are plenty of examples in the official docs for how to do this.
Remember to keep everything bulkified (process all the records being deleted at once, from a list)...
On a side note, the deleted records in SalesForce are kept in the Recycle Bin on the org for 15 days after deletion. So you can also select them from the object, by using the SELECT... ALL ROWS query form.
I don't think you can really override delete action. You could override a button (with a Visualforce page) but that won't help you in any way if delete is fired from API.
I suspect you want to pretend to API (SOAP, REST etc) users that record was deleted while in reality retaining it somewhere? Smells like some shady business practice to be honest but whatever, let's assume it really is legit... For sure you can't suddenly throw errors at the operation because your end users will notice.
I think I'd go with a hidden 1-to-1 matching "shadow" object and sync each action to it. You'd need a trigger on insert/update/delete/undelete of your junction that would replicate the action (difference being this custom "soft delete" flag). This has lots of concerns like storage usage but well.
One thing that comes to mind is that (if I recall correctly) the triggers on junction object don't fire if you delete one of masters. So if it's a real junction object (you wrote "acts like") you'd have to deal with this scenario too and put logic into master objects' triggers.
If it's not a real junction object (i.e. it has OwnerId field visible) and your sharing rules permit - maybe you could transfer the ownership of record to some special user/queue outside of roles hierarchy so it becomes invisible... But I doubt it'll work, in the end the delete should appear to complete succesfully, right? Maybe in combination with some #future that'd immediately undelete them & transfer... Still - messy!
I'm working on a basic syncing algorithm for a user's notes. I've got most of it figured out, but before I start programming it, I want to run it by here to see if it makes sense. Usually I end up not realizing one huge important thing that someone else easily saw that I couldn't. Here's how it works:
I have a table in my database where I insert objects called SyncOperation. A SyncOperation is a sort of metadata on the nature of what every device needs to perform to be up to date. Say a user has 2 registered devices, firstDevice and secondDevice. firstDevice creates a new note and pushes it to the server. Now, a SyncOperation is created with the note's Id, operation type, and processedDeviceList. I create a SyncOperation with type "NewNote", and I add the originating device ID to that SyncOperation's processedDeviceList. So now secondDevice checks in to the server to see if it needs to make any updates. It makes a query to get all SyncOperations where secondDeviceId is not in the processedDeviceList. It finds out its type is NewNote, so it gets the new note and adds itself to the processedDeviceList. Now this device is in sync.
When I delete a note, I find the already created SyncOperation in the table with type "NewNote". I change the type to Delete, remove all devices from processedDevicesList except for the device that deleted the note. So now when new devices call in to see what they need to update, since their deviceId is not in the processedList, they'll have to process that SyncOperation, which tells their device to delete that respective note.
And that's generally how it'd work. Is my solution too complicated? Can it be simplified? Can anyone think of a situation where this wouldn't work? Will this be inefficient on a large scale?
Sounds very complicated - the central database shouldn't be responsible for determining which devices have recieved which updates. Here's how I'd do it:
The database keeps a table of SyncOperations for each change. Each SyncOperation is has a change_id numbered in ascending order (that is, change_id INTEGER PRIMARY KEY AUTOINCREMENT.)
Each device keeps a current_change_id number representing what change it last saw.
When a device wants to update, it does SELECT * FROM SyncOperations WHERE change_id > current_change_id. This gets it the list of all changes it needs to be up-to-date. Apply each of them in chronological order.
This has the charming feature that, if you wanted to, you could initialise a new device simply by creating a new client with current_change_id = 0. Then it would pull in all updates.
Note that this won't really work if two users can be doing concurrent edits (which edit "wins"?). You can try and merge edits automatically, or you can raise a notification to the user. If you want some inspiration, look at the operation of the git version control system (or Mercurial, or CVS...) for conflicting edits.
You may want to take a look at SyncML for ideas on how to handle sync operations (http://www.openmobilealliance.org/tech/affiliates/syncml/syncml_sync_protocol_v11_20020215.pdf). SyncML has been around for a while, and as a public standard, has had a fair amount of scrutiny and review. There are also open source implementations (Funambol comes to mind) that can also provide some coding clues. You don't have to use the whole spec, but reading it may give you a few "ahah" moments about syncing data - I know it helped to think through what needs to be done.
Mark
P.S. A later version of the protocol - http://www.openmobilealliance.org/technical/release_program/docs/DS/V1_2_1-20070810-A/OMA-TS-DS_Protocol-V1_2_1-20070810-A.pdf
I have seen the basic idea of keeping track of operations in a database elsewhere, so I dare say it can be made to work. You may wish to think about what should happen if different devices are in use at much the same time, and end up submitting conflicting changes - e.g. two different attempts to edit the same note. This may surface as a change to the user interface, to allow them to intervene to resolve such conflicts manually.
Let's say that I have two tables in a DB: Expenses and Account. Expenses is the data that I'm interested in and that table has a foreign key to Account. This DB is remote, accessed via Restful-esque commands, and I want to mirror just the data I need for my app in a Core Data data store on the iPhone. The actual DB I'm working with is much bigger than this example. ~30 tables and the Expenses table has ~7 FKs. I'm working closely with the person doing the API design, so I can modify the way I make my requests or the data returned, if necessary.
What is the best strategy for loading this data into Core Data?
My first thought was to have the request for the expense bring back the ids for the FK.
<expense>
<date>1/1/2011</date>
<cost>1.50</cost>
<account_id>123</account_id>
</expense>
This works fine if I already have an account with id '123' in my data store. If I don't, then I've got to make additional web requests every time I encounter an id I don't have… which is going to be incredibly slow. I can get around this by making requests in a specific order, i.e. request all new accounts before requesting expenses, so that I way I know all the FK rows exist. I feel this would become much too cumbersome once the DB starts reaching moderate complexity.
My second thought was to have the data returned from the request follow FKs and return data from the FK.
<expense>
<date>1/1/2011</date>
<cost>1.50</cost>
<account>
<id>123</id>
<name>Bob's Big Boy</name>
<address>1234 Main Street</address>
</account>
</expense>
This looks better and guarantees that I'll have all the data I need when I need it. If I don't already have an account '123' I can create a new account object from that XML. My concern with this method, though, is that as the database grows in complexity, these XML files could become excessively large. The Expenses table has ~7 foreign keys, each of those tables has multiple FKs. It feels like a simple request for just a single Expense could end up returning a huge chunk of data.
How have other people solved this issue?
I am assuming that at any given time you only want to cache part of the server DB in the local app and that the data cached may change overtime.
You probably want to use "stub" entities to represent related objects that you haven't actually downloaded yet. You would set up the entities like this:
Expense{
date:Date
cost:Number
account<<-->AccountStub.expenses
}
AccountStub{
id:Number
expenses<-->>Expenses.account
}
Account:AccountStub{
name:String
address:String
}
The AccountStub entity has the bare minimum info needed to identify the Account in the server DB based on info provided from the Expense table. It serves as a placeholder in the object graph for the full fledged Account object (you can think of it as a type of fault if you like.)
Since Expenses has the relationship with AccountStub and Account inherits from AccountStub you can swap out an Account for an AccountStub (and vice versa) as needed.
You will need to provide a custom subclass for AccountStub and Account such that AccountStub can trigger the downloading of account data and the creation of an Account object when that data is actually required. Then the new Account object should be swapped out for AccountStub in all its relationships (that may take rather a lot of code.)
To use, you would first obtain the data for an Expense object and create that object. You would attempt to fetch for an AccountStub with the ID provided from the Expense table data. Set the fetch to include subentries. If an AccountStub or Account object exist with that ID you will add the Expense object to the relationship. If not, the you create an AccountStub object with that ID and add it to the relationship. Now you have a basic object graph showing the relationship of an Expense object to an AccountStub object. To access the account data of an Expense, you would first check if the related account is a stub or a full account. If it is a stub, then you need to load the full account data before preceding.
The advantage of this system is that you can maintain a fairly complex object graph without having to actually have all the data locally. E.g. you can maintain several relationships and walk those relationships. E.g you could expand your model like this:
AccountStub{
id:Number
expenses<-->>Expenses.account
owner<<--AccountOwnerStub.accounts
}
AccountOwnerStub{
id:Number
accounts<-->>AccountStub.owner
}
AccountOwner{
name:String
address:String
bill:Number
}
If you wanted to find the name of an Expense object's account owner, you would just walk the relationship across the stubs with account.owner.name the Account object itself would would remain just a stub.
If you need to conserve room locally, you can revert an object back to a stub without compromising the graph.
This would take some work and you would have to keep an eye on the stubs but it would let you mirror a complex external DB without having to keep all the data on hand.
I need to design and implement something similar to what Martin Fowler calls the "Unit of Work" pattern. I have heard others refer to it as a "Shopping Cart" pattern, but I'm not convinced the needs are the same.
The specific problem is that users (and our UI team) want to be able to create and assign child objects (with referential integrity constraints in the database) before the parent object is created. I met with another of our designers today and we came up with two alternative approaches.
a) First, create a dummy parent object in the database, and then create dummy children and dummy assignments. We could use negative keys (our normal keys are all positive) to distinguish between the sheep and the goats in the database. Then when the user submits the entire transaction we have to update data and get the real keys added and aligned.
I see several drawbacks to this one.
It causes perturbations to the indexes.
We still need to come up with something to satisfy unique constraints on columns that have them.
We have to modify a lot of existing SQL and code that generates SQL to add yet another predicate to a lot of WHERE clauses.
Altering the primary keys in Oracle can be done, but its a challenge.
b) Create Transient tables for objects and assignments that need to be able to participate in these reverse transactions. When the user hits Submit, we generate the real entries and purge the old.
I think this is cleaner than the first alternative, but still involves increased levels of database activity.
Both methods require that I have some way to expire transient data if the session is lost before the user executes submit or cancel requests.
Has anyone solved this problem in a different way?
Thanks in advance for your help.
I don't understand why these objects need to be created in the database before the transaction is committed, so you might want to clarify with your UI team before proceeding with a solution. You may find that all they want to do is read information previously saved by the user on another page.
So, assuming that the objects don't need to be stored in the database before the commit, I give you plan C:
Store initialized business objects in the session. You can then create all the children you want, and only touch the database (and set up references) when the transaction needs to be committed. If the session data is going to be large (either individually or collectively), store the session information in the database (you may already be doing this).