I've read about entities lifecycle, and the locking strategies, and I watched some videos about this but I'm still not sure I understand.I understand there is also a locking mechanism in the underlying RDBMS (I'm using mysql).
I would like to know at what point a transaction is committed / entity is detached and how does it affect other transactions from a locking point of view. At what point does an user have to wait till a transaction finishes ? I've made two different scenarios below. For the sake of understanding I'm asserting the table in the scenarios contains a lot of rows and the for loops takes 10 minute to complete.
Scenario 1:
#Stateless
public class AService implements AServiceInterface {
#PersistenceContext(unitName = "my-pu")
private EntityManager em;
#Override
public List<Aclass> getAll() {
Query query = em.createQuery(SELECT_ALL_ROWS);
return query.getResultList();
}
public void update(Aclass a) {
em.merge(a);
}
}
and a calling class:
public aRadomClass{
#EJB
AServiceInterface service;
public void method(){
List<Aclass> listAclass = service.getAll();
for(Aclass a : listAclass){
a.setProperty(methodThatTakesTime());
service.update(a);
}
}
}
Without specifying a locking strategy : If another user wants to makes an update to one row in the table and the for loop already began but is not finished. Does he have to wait till the for loop is completed ?
Scenario 2:
#Stateless
public class AService implements AServiceInterface {
#PersistenceContext(unitName = "my-pu")
private EntityManager em;
#Override
public List<Aclass> getAllAndUpdate() {
Query query = em.createQuery(SELECT_ALL_ROWS);
List<Aclass> listAclass = query.getResultList();
for(Aclass a : listAclass ){
a.setProperty(methodThatTakesTime());
em.merge(a);
}
}
}
Same question.
It is important what kind of class is your aRandomClass. If it is also an EJB, you should take a look in the transaction propagation. If it is a servlet, then the transaction is closed automatically right after your EJB method exits (no matter which one). That is done using dynamic proxies. So in scenario 1 the EJB container will open and close multiple transactions: one for service.getAll() and one for each service.update(a) call. In scenario 2, if method getAllAndUpdate() is called only once, a single transaction will be opened and it will be closed on method exit.
Related
Hey there I am having a hard time understanding how shared state (ValueState, ListState, ..) work in flink. If multiple instances of a task are running in parallel how does flink prevent race conditions?
in this example from the doc, if the operator is parallelized, how does flink guarantee that there are no race conditions between the read and update of the keyHasBeenSeen value?
public static class Deduplicator extends RichFlatMapFunction<Event, Event> {
ValueState<Boolean> keyHasBeenSeen;
#Override
public void open(Configuration conf) {
ValueStateDescriptor<Boolean> desc = new ValueStateDescriptor<>("keyHasBeenSeen", Types.BOOLEAN);
keyHasBeenSeen = getRuntimeContext().getState(desc);
}
#Override
public void flatMap(Event event, Collector<Event> out) throws Exception {
if (keyHasBeenSeen.value() == null) {
out.collect(event);
keyHasBeenSeen.update(true);
}
}
}
There isn't any shared state in Flink. Having shared state would add complexity and impair scalability.
The value and update methods are scoped to the key of the current event. For any given key, all events for that key are processed by the same instance of the operator/function. And all tasks (a task is a chain of operator/function instances) are single threaded.
By keeping things simple like this, there's nothing to worry about.
I'm having trouble with transactions in Spring Boot using #Transactional annotation. The latest Spring is connected to a MS SQL Database.
I have following service, which periodically executes transactional method according to some criteria:
#Service
public class SomeService {
SomeRepository repository;
public SomeService(SomeRepository someRepository) {
this.repository = someRepository;
}
#Scheduled(fixedDelayString="${property}") //10 seconds
protected scheduledIteration() {
if(something) {
insertDataInNewTransaction(getSomeData());
}
}
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
protected void insertDataInNewTransaction(List<Data> data) {
//insert data to db
repository.saveAll(data);
//call verify proc
repository.verifyData();
}
}
The algorithm supposed to process data, insert them into table and perform check (db procedure). If the procedure throws an exception, the transaction should be rollbacked. I'm sure, that the procedure does not perform commit of the transaction.
The problem I'm facing is, that calling the method does not begin new transaction (or does but it's auto-commited), because I've tried following:
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
protected void insertDataInNewTransaction(List<Data> data) {
int counter = 0;
for(Data d : data) {
repository.save(d);
counter++;
//test
if(counter == 10) {
throw new Exception("test");
}
}
}
After the test method is executed, the first 10 rows remain in the table, where they were supposed to be rollbacked. During debugging I've noticed, that calling repository.save() in the loop inserts to the table outside transaction, because I can see the row from DB IDE while debugger sitting on next row. This gave me an idea, that the problem is caused by auto-commit, as it's MS SQL default. So I have tried to add following properties, but without any difference:
spring.datasource.hikari.auto-commit=false
spring.datasource.auto-commit=false
Is there anything I'm doing wrong?
If you use Spring Proxy AOP, then you need to turn the method insertDataInNewTransaction as public.
Remember that if the method is public, but it is invoked from the same bean, it will not create a new transaction (because spring proxies won't be call).
Short answer:
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
public void insertDataInNewTransaction(List<Data> data) {
//insert data to db
repository.saveAll(data);
//call verify proc
repository.verifyData();
}
But if you really need a new separate transaction use Propagation.REQUIRES_NEW instead of Propagation.REQUIRED.
I have a default standalone.xml configuration where there is a maximum of 20 connections to be active at the same time in the pool of connections to the database. With good reasons, I guess. We run an Oracle database.
There's a reasonable amount of database traffic as there is third party API traffic, e.g. SOAP and HTTP calls in the enterprise application I'm developing.
We often do something like the following:
#PersistenceContext(unitName = "some-pu")
private EntityManager em;
public void someBusinessMethod() {
someEntity = em.findSomeEntity();
soap.callEndPoint(someEntity.getSomeProperty()); // may take up to 1 minute
em.update(someEntity);
cdiEvent.fire(finishedBusinessEvent);
}
However, in this case the database connection is acquired when the entity is fetched and is released after the update (actually when the entire transaction is done). About transactions, everything is container managed, no additional annotations. I know that you shouldn't "hold" the database connection longer than necessary, and this is exactly what I'm trying to solve. For one I wouldn't know how to programmatically release the connection nor do I think it would be a good idea, because you still want to be able to roll back for the entire transaction.
So? How to attack this problem? There's a number of options I tried:
Option 1, using ManagedExecutorService:
#Resource
private ManagedExecutorService mes;
public void someBusinessMethod() {
someEntity = em.findSomeEntity();
this.mes.submit(() -> {
soap.callEndPoint(someEntity.getSomeProperty()); // may take up to 1 minute
em.update(someEntity);
cdiEvent.fire(finishedBusinessEvent);
});
}
Option 2, using #Asynchronous:
#Inject
private AsyncBean asyncBean;
public void someBusinessMethod() {
someEntity = em.findSomeEntity();
this.asyncBean.process(someEntity);
}
public class AsyncBean {
#Asynchronous
public void process() {
soap.callEndPoint(someEntity.getSomeProperty()); // may take up to 1 minute
em.update(someEntity);
cdiEvent.fire(finishedBusinessEvent);
}
}
This in fact solved the database connection pooling issue, e.g. the connection is released as soon as the soap.callEndPoint happened. But it did not feel really stable (can't pinpoint the problems here). And of course the transaction is finished once you enter the a-sync processing, so whenever something went wrong during the soap call there was nothing roll backed.
wrapping up...
I'm about to move the long running IO tasks (soap and http calls) to a separate part of the application offloaded via queue's and feeding the result back in the application via queue's once again. In this case everything is done via transactions and no connections are held up. But this is a lot of overhead, thus before doing so I'd like to hear your opinion / best practices how to solve this problem!
Your queue solution is viable, but perhaps not necessary if you only perform read operations before your calls, you could split the transaction into 2 transactions (as you would also do with the queue) by using a DAO pattern.
Example:
#Stateless
private DaoBean dao;
#TransactionAttribute(TransactionAttributeType.NEVER)
public void someBusinessMethod() {
Entity e = dao.getEntity(); // creates and discards TX
e = soap.callEndPoint(e.getSomeProperty());
dao.update(e); // creates TX 2 and commits
}
This solutions has a few caveats.
The business method above can not be called while a transaction is already active because it would negate the purpose of the DAO (one TX suspended with NOT_SUPPORTED).
You will have to handle or ignore the possible changes that could have occurred on the entity during the soap call (#Version ...).
The entity will be detached in the business method, so you will have to eager load everything you need in the soap call.
I can't tell you if this would work for you as it depends on what is done before the business call. While still complex, it would be easier than a queue.
You were kind of heading down the right track with Option 2, it just needs a little more decomposition to get the transaction management happening in a way that keeps them very short.
Since you have a potentially long running web service call you're definitely going to need to perform your database updates in two separate transactions:
short find operation
long web service call
short update operation
This can be accomplished by introducing a third EJB as follows:
Entry point
#Stateless
public class MyService {
#Inject
private AsyncService asyncService;
#PersistenceContext
private EntityManager em;
/*
* Short lived method call returns promptly
* (unless you need a fancy multi join query)
* It will execute in a short REQUIRED transaction by default
*/
public void someBusinessMethod(long entityId) {
SomeEntity someEntity = em.find(SomeEntity.class, entityId);
asyncService.process(someEntity);
}
}
Process web service call
#Stateless
public class AsyncService {
#Inject
private BusinessCompletionService businessCompletionService;
#Inject
private SomeSoapService soap;
/*
* Long lived method call with no transaction.
*
* Asynchronous methods are effectively run as REQUIRES_NEW
* unless it is disabled.
* This should avoid transaction timeout problems.
*/
#Asynchronous
#TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void process(SomeEntity someEntity) {
soap.callEndPoint(someEntity.getSomeProperty()); // may take up to 1 minute
businessCompletionService.handleBusinessProcessCompletion(someEntity);
}
}
Finish up
#Stateless
public class BusinessCompletionService {
#PersistenceContext
private EntityManager em;
#Inject
#Any
private Event<BusinessFinished> businessFinishedEvent;
/*
* Short lived method call returns promptly.
* It defaults to REQUIRED, but will in effect get a new transaction
* for this scenario.
*/
public void handleBusinessProcessCompletion(SomeEntity someEntity) {
someEntity.setSomething(SOMETHING);
someEntity = em.merge(someEntity);
// you may have to deal with optimistic locking exceptions...
businessFinishedEvent.fire(new BusinessFinished(someEntity));
}
}
I suspect that you may still need some connection pool tuning to cope effectively with your peak load. Monitoring should clear that up.
I have a class.Class contain a method ParseJSONResponse().I want that method should get executed on daily basis at midnight.How I can achieve this in salesforce.
I know there is schedule apex mechanism is available in salesforce to perform such a thing but I need no. of steps or code to achieve this.I am new to salesforce.Any help would be appreciated.
public with sharing class ConsumeCloudArmsWebserviceCallout{
public void ParseJSONResponse(){
// handling customerList and inserting records for it
DateTime lastModifiedDate =Common.getSynchDateByDataObject(CloudArmsWebserviceCallout.DataObject.CustomerContact);
List<Account> lstAccounts = ConsumeCustomers.CreateCustomers(lastModifiedDate);
ConsumeContacts.CreateContacts(lastModifiedDate);
Common.updateSynchByDataObject(CloudArmsWebserviceCallout.DataObject.CustomerContact);
}
}
You can implement the Schedulable interface directly to your ConsumeCloudArmsWebserviceCallout class:
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_scheduler.htm
In order to perform callout -which apparently you will according to your class name- you can use the Queueable interface:
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_queueing_jobs.htm
public with sharing class ConsumeCloudArmsWebserviceCallout implements Schedulable {
public void ParseJSONResponse(){
// handling customerList and inserting records for it
DateTime lastModifiedDate =Common.getSynchDateByDataObject(CloudArmsWebserviceCallout.DataObject.CustomerContact);
List<Account> lstAccounts = ConsumeCustomers.CreateCustomers(lastModifiedDate);
ConsumeContacts.CreateContacts(lastModifiedDate);
Common.updateSynchByDataObject(CloudArmsWebserviceCallout.DataObject.CustomerContact);
}
//Method implemented in order to use the Schedulable interface
public void execute(SchedulableContext ctx) {
ConsumeCloudQueueable cloudQueueable = new ConsumeCloudQueueable();
ID jobID = System.enqueueJob(cloudQueueable);
}
//Inner class that implements Queueable and can perform callouts.
private class ConsumeCloudQueueable implements Queueable, Database.AllowsCallouts {
public void execute(QueueableContext context) {
ConsumeCloudArmsWebserviceCallout cloudArms = new ConsumeCloudArmsWebserviceCallout();
cloudArms.ParseJSONResponse();
}
}
}
Then go onto the class page on Salesforce setup.
There you will find a schedule class button.
You will be able to schedule all classes implementing the Schedulable interface.
What will happen is that it will schedule your class daily.
Then your schedule will only en-queue a ConsumeCloudQueueable class that will do the job whenever Salesforce runs it (pretty much straight away).
Once the job runs, it will execute whatever is on you ParseJSONResponse() method.
Let me know if you have any question.
Cheers,
Seb
and welcome to salesforce development.
If I assume something wrong let me know. So you are looking to
Every Day at Midnight
Fire a job that makes a callout to another system
Then Parses the results and creates stuff
You are looking for Apex Scheduler code: http://www.salesforce.com/us/developer/docs/apexcode/Content/apex_scheduler.htm
global class ConsumeCloudArmsWebserviceCallout_Job implements Schedulable {
global void execute(SchedulableContext sc) {
new ConsumeCloudArmsWebserviceCallout().ParseJSONResponse();
}
}
Then you can schedule the job in yourname -> Developer Console. Debug -> Open Execute Anonymous Window.
system.schedule('Consumer Cloud Arms Service', '0 0 0 * * ?', new ConsumeCloudArmsWebserviceCallout_Job());
Keep in mind:
you can only make 5 callouts in one apex transation. Meaning if you need to do more, you need to use a batch job or #future calls.
Be careful with large data sets, if your job is expensive (creating and modifying lots of data) you need to be sure that you don't run in to CPU limits, So a batch job requesting smaller portions of data from the service you are calling out may be needed.
I don't think you will be running in to those issues, but they will catch you off guard sometimes.
EDIT: fixed the call to a new object, less psudo-code
I'm execute method Datastore.delete(key) form my GWT web application, AsyncCallback had call onSuccess() method .Them i refresh http://localhost:8888/_ah/admin immediately , the Entity i intent to delete still exist. Smilar to, I refresh my GWT web application immediately the item i intent to delete still show on web page.Note the the onSuccess() had been call.
So, how can i know when the Entity already deleted ?
public void deleteALocation(int removedIndex,String symbol ){
if(Window.confirm("Sure ?")){
System.out.println("XXXXXX " +symbol);
loCalservice.deletoALocation(symbol, callback_delete_location);
}
}
public AsyncCallback<String> callback_delete_location = new AsyncCallback<String>() {
public void onFailure(Throwable caught) {
Window.alert(caught.getMessage());
}
public void onSuccess(String result) {
// TODO Auto-generated method stub
int removedIndex = ArryList_Location.indexOf(result);
ArryList_Location.remove(removedIndex);
LocationTable.removeRow(removedIndex + 1);
//Window.alert(result+"!!!");
}
};
SERver :
public String deletoALocation(String name) {
// TODO Auto-generated method stub
Transaction tx = Datastore.beginTransaction();
Key key = Datastore.createKey(Location.class,name);
Datastore.delete(tx,key);
tx.commit();
return name;
}
Sorry i'm not good at english :-)
According to the docs
Returns the Key object (if one model instance is given) or a list of Key objects (if a list of instances is given) that correspond with the stored model instances.
If you need an example of a working delete function, this might help. Line 108
class DeletePost(BaseHandler):
def get(self, post_id):
iden = int(post_id)
post = db.get(db.Key.from_path('Posts', iden))
db.delete(post)
return webapp2.redirect('/')
How do you check the existence of the entity? Via a query?
Queries on HRD are eventually consistent, meaning that if you add/delete/change an entity then immediately query for it you might not see the changes. The reason for this is that when you write (or delete) an entity, GAE asynchronously updates the index and entity in several phases. Since this takes some time it might happen that you don't see the changes immediately.
Linked article discusses ways to mitigate this limitation.