I have one requirement in which i have to insert record in one table and update the record in another table.
There are 100 records for which i have to do above insertion and updation.
But If any record fails due to any error in insertion or updation only that record should be rollback not others.Means if There are 100 records and 60 gets success and 61 gets fails due to any error then only 61 records should be rolled back not all other 60 records.
I am using Spring boot JPA and #Transactional Annotation.But if put this annotation on starting of my method,Then in case of any error it roll back all records.
I am attaching my code below.Please suggest how i can achieve this.
In below code i have persistRecord() method which have all my code for insertion and updation.I have not marked this with #Transactional annotation insteed below method is annotated with #Transactional annotation
logger.debug("insertSalesTargetData method called of service class");
String userid = (String) webSession.getAttribute("username");
logger.debug("Userid :" + userid);
HashMap<String, String> persistRecordStatus = new HashMap<String, String>();
Query qCountCheck = em.createNativeQuery(QueryConstant.CHECK_SALES_TARGET_DATA_QUERY);
List<Object> existingRecordList = null;
persistRecordStatus.put("TOTAL_RECORD", String.valueOf(xlsRowslist.size()));
int failedRecordCount = 0;
int successRecordCount = 0;
for (int i = 0; i < xlsRowslist.size(); i++) {
ArrayList rowList = xlsRowslist.get(i);
// try {
double weekNoDouble = Double.parseDouble((String) rowList.get(0));// Week No
String fromDate = (String) rowList.get(1);// fromDate
String toDate = (String) rowList.get(2);// toDate
double catCodeDouble = Double.parseDouble((String) rowList.get(3));// catCode
int catCode = (int) catCodeDouble;
int weekNo = (int) weekNoDouble;
String target = (String) rowList.get(4);// target
String salesGoalId = fromDate + toDate + catCode;
salesGoalId = salesGoalId.replace("-", "");
// Check if the sales goal id already exist in the database or not
qCountCheck.setParameter(1, salesGoalId);// SALES_GOAL_ID
existingRecordList = qCountCheck.getResultList();
logger.debug("Count List Size " + existingRecordList.size());
if (existingRecordList != null && existingRecordList.size() > 0) {
if (existingRecordList.get(0) != null) {
BigDecimal row = (BigDecimal) existingRecordList.get(0);
// try {
logger.debug("Persisting record no " + i);
persistRecord(row, salesGoalId, target, fromDate, toDate, userid, catCode, weekNo);
logger.debug("Record no " + i + " persisted");
persistRecordStatus.put("SUCCESS_RECORD", String.valueOf(successRecordCount++));
/*
* } catch (Exception e) { persistRecordStatus.put("FAILED_RECORD",
* String.valueOf(failedRecordCount++)); }
*/
}
} else {
persistRecordStatus.put("FAILED_RECORD",String.valueOf(failedRecordCount++));
}
/*
* } catch (Exception e) { logger.debug("Exception in processing record no " + i
* + " " + e.toString()); persistRecordStatus.put("FAILED_RECORD",
* String.valueOf(failedRecordCount++)); }
*/
}
return persistRecordStatus;
}
You are doing some kind of batch updates ?
In batch processing, you treat "failures" by putting all stuff involved (e.g. the input record, an error code, ...) on an error store for subsequent review by some user. And you commit that along with all the "successful" updates.
Alternatively, put each insert/update pair in its own transaction. (and still alternatively, if your DBMS / transactional framework supports savepoints, take a savepoint after each insert/update pair and upon failure, rollback to the last savepoint and then commit (and figure out a way to not lose the remaining 40 unprocessed entries from your input). In either case, don't come complaining if you get performance trouble.
There is no having things both ways in transactional batch processing.
Related
Strange issue. I have a managed OPD.Net application that calls a stored procedure to insert records. When I call the procedure normally and rollback the transaction, the records are not saved to the table (duh!). When I use the procedure with parameter array binding and still roll back the transaction, the records ARE saved to the table. Somehow, the records are getting committed even though I do a rollback!
TEST Schema:
CREATE TABLE TEST
(
ID NUMBER(15,0),
VALUE VARCHAR2(50 CHAR)
)
/
CREATE SEQUENCE TEST_ID_SEQ
INCREMENT BY 1
START WITH 50
MINVALUE 1
MAXVALUE 999999999999999
NOCYCLE
NOORDER
CACHE 100
/
CREATE OR REPLACE PROCEDURE TEST_INSERT
(
iVALUE IN VARCHAR2,
oID OUT NUMBER
)
AS
BEGIN
oID := TEST_ID_SEQ.NEXTVAL;
INSERT INTO TEST
(
ID,
VALUE
)
VALUES
(
oID,
iVALUE
);
END;
/
Test Code:
using Oracle.ManagedDataAccess.Client;
using Oracle.ManagedDataAccess.Types;
using System;
using System.Data;
namespace OdpTestArrayBinding
{
class Program
{
private const string cConnectioString = "Data Source=DB_DEV;User Id=TMP;Password=sqlsql";
static void Main(string[] args)
{
using (OracleConnection lConnectionA = new OracleConnection(cConnectioString))
{
lConnectionA.StateChange += ConnectionStateChanged;
lConnectionA.Open();
Console.WriteLine($"[Connection={lConnectionA.GetHashCode()}] Connection opened.");
int lStartCount = CountTestTableRows(lConnectionA);
Console.WriteLine($"[Connection={lConnectionA.GetHashCode()}] Number of rows in table at start is {lStartCount}.");
using (OracleTransaction lTransaction = lConnectionA.BeginTransaction())
{
Console.WriteLine($"[Connection={lConnectionA.GetHashCode()}] Transaction started.");
try
{
using (OracleCommand lCmd = new OracleCommand())
{
lCmd.Connection = lConnectionA;
lCmd.BindByName = true;
lCmd.CommandType = System.Data.CommandType.StoredProcedure;
lCmd.CommandText = "TEST_INSERT";
lCmd.Parameters.Add("iVALUE", OracleDbType.Varchar2, ParameterDirection.Input);
// The OracleDbType of the output does not seem to matter, the actual value is always OracleDecimal
lCmd.Parameters.Add("oID", OracleDbType.Int64, ParameterDirection.Output);
lCmd.ArrayBindCount = 3;
lCmd.Parameters["iVALUE"].Value = new string[] { "Foo", "Bar", "Boo" };
// Not required.
//lCmd.Parameters["oID"].Value = new long[] { -1, -1, -1 };
lCmd.ExecuteNonQuery();
OracleDecimal[] lOutIds = (OracleDecimal[])lCmd.Parameters["oID"].Value;
Console.WriteLine($"[Connection={lConnectionA.GetHashCode()}] Inserted 3 rows using stored procedure, out ID vales are {string.Join(",", lOutIds)}.");
}
ListRows(lConnectionA, lStartCount + 3);
using (OracleConnection lConnectionB = new OracleConnection(cConnectioString))
{
lConnectionB.StateChange += ConnectionStateChanged;
lConnectionB.Open();
Console.WriteLine($"[Connection={lConnectionB.GetHashCode()}] Connection opened.");
ListRows(lConnectionB, lStartCount);
}
}
finally
{
lTransaction.Rollback();
Console.WriteLine($"[Connection={lConnectionA.GetHashCode()}] Transaction rolled back.");
}
}
}
Console.WriteLine("Press the ENTER key to continue...");
Console.ReadLine();
}
private static void ConnectionStateChanged(object sender, StateChangeEventArgs e)
{
Console.WriteLine($"[Connection={sender.GetHashCode()}] State changed from {e.OriginalState} to {e.CurrentState}.");
}
private static int CountTestTableRows(OracleConnection aConnection)
{
using (OracleCommand lCmd = new OracleCommand())
{
lCmd.Connection = aConnection;
lCmd.BindByName = true;
lCmd.CommandType = System.Data.CommandType.Text;
lCmd.CommandText = "SELECT COUNT(*) FROM TEST";
return Convert.ToInt32(lCmd.ExecuteScalar());
}
}
private static void ListRows(OracleConnection aConnection, int aExpectedRowCount)
{
int lCount = CountTestTableRows(aConnection);
Console.Write($"[Connection={aConnection.GetHashCode()}] Number of rows in table {lCount}");
if (lCount == aExpectedRowCount)
Console.WriteLine(" (Test passed, actual and expected row count are the same).");
else
Console.WriteLine($" (Test FAILED!, expected {aExpectedRowCount} rows).");
}
}
}
First run console output:
[Connection=47973293] State changed from Closed to Open.
[Connection=47973293] Connection opened.
[Connection=47973293] Number of rows in table at start is 0.
[Connection=47973293] Transaction started.
[Connection=47973293] Inserted 3 rows using stored procedure, out ID vales are 50,51,52.
[Connection=47973293] Number of rows in table 3 (Test passed, actual and expected row count are the same).
[Connection=21040294] State changed from Closed to Open.
[Connection=21040294] Connection opened.
[Connection=21040294] Number of rows in table 3 (Test FAILED!, expected 0 rows).
[Connection=21040294] State changed from Open to Closed.
[Connection=47973293] Transaction rolled back.
[Connection=47973293] State changed from Open to Closed.
I have tried Oracle.ManagedDataAccess.dll 4.121.2.20141216 (ODAC RELEASE 3) and 4.121.2.20150926 (ODAC RELEASE 4), both give the same result. Any ideas or workarounds?
Can you try to change your code to use this example.
Open only a using block for your connection.
public void RunOracleTransaction(string connectionString)
{
using (OracleConnection connection = new OracleConnection(connectionString))
{
connection.Open();
OracleCommand command = connection.CreateCommand();
OracleTransaction transaction;
// Start a local transaction
transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted);
// Assign transaction object for a pending local transaction
command.Transaction = transaction;
try
{
command.CommandText =
"INSERT INTO Dept (DeptNo, Dname, Loc) values (50, 'TECHNOLOGY', 'DENVER')";
command.ExecuteNonQuery();
command.CommandText =
"INSERT INTO Dept (DeptNo, Dname, Loc) values (60, 'ENGINEERING', 'KANSAS CITY')";
command.ExecuteNonQuery();
transaction.Commit();
Console.WriteLine("Both records are written to database.");
}
catch (Exception e)
{
transaction.Rollback();
Console.WriteLine(e.ToString());
Console.WriteLine("Neither record was written to database.");
}
}
}
Pleaase check your command : you have forget assign the transaction on your command?
lCmd.Connection = lConnectionA;
// Assign transaction to your command
lCmd.Transaction = lTransaction;
You should create a TransactionScope that wraps your database connection and within your TransactionScope, try enlisting in the ambient transaction:
<your connection object>.EnlistTransaction(Transaction.Current);
I have a large SQLite database (~134 GB) that has multiple tables each with 14 columns, about 330 million records, and 4 indexes. The only operation used on the database is "Select *" as I need all the columns(No inserts or updates). When I query the database, the response time is slow when the result set is big (takes 160 seconds for getting ~18,000 records).
I have improved the use of indexes multiple times and this is the fastest response time I got.
I am running the database as a back-end database for a web application on a server with 32 GB of RAM.
is there a way to use RAM (or anything else) to speed up the query process?
Here is the code that performs the query.
async.each(proteins,function(item, callback) {
`PI[item] = []; // Stores interaction proteins for all query proteins
PS[item] = []; // Stores scores for all interaction proteins
PIS[item] = []; // Stores interaction sites for all interaction proteins
var sites = {}; // a temporarily holder for interaction sites
var query_string = 'SELECT * FROM ' + organism + PIPE_output_table +
' WHERE ' + score_type + ' > ' + cutoff['range'] + ' AND (protein_A = "' + item + '" OR protein_B = "' + item '") ORDER BY PIPE_score DESC';
db.each(query_string, function (err, row) {
if (row.protein_A == item) {
PI[item].push(row.protein_B);
// add 1 to interaction sites to represent sites starting from 1 not from 0
sites['S1AS'] = row.site1_A_start + 1;
sites['S1AE'] = row.site1_A_end + 1;
sites['S1BS'] = row.site1_B_start + 1;
sites['S1BE'] = row.site1_B_end + 1;
sites['S2AS'] = row.site2_A_start + 1;
sites['S2AE'] = row.site2_A_end + 1;
sites['S2BS'] = row.site2_B_start + 1;
sites['S2BE'] = row.site2_B_end + 1;
sites['S3AS'] = row.site3_A_start + 1;
sites['S3AE'] = row.site3_A_end + 1;
sites['S3BS'] = row.site3_B_start + 1;
sites['S3BE'] = row.site3_B_end + 1;
PIS[item].push(sites);
sites = {};
}
}
The query you posted uses no variables.
It will always return the same thing: all the rows with a null score whose protein column is equal to its protein_a or protein_b column. You're then having to filter all those extra rows in Javascript, fetching a lot more rows than you need to.
Here's why...
If I'm understanding this query correctly, you have WHERE Score > [Score]. I've never encountered this syntax before, so I looked it up.
[keyword] A keyword enclosed in square brackets is an identifier. This is not standard SQL. This quoting mechanism is used by MS Access and SQL Server and is included in SQLite for compatibility.
An identifier is something like a column or table name, not a variable.
This means that this...
SELECT * FROM [TABLE]
WHERE Score > [Score] AND
(protein_A = [Protein] OR protein_B = [Protein])
ORDER BY [Score] DESC;
Is the same as this...
SELECT * FROM `TABLE`
WHERE Score > Score AND
(protein_A = Protein OR protein_B = Protein)
ORDER BY Score DESC;
You never pass any variables to the query. It will always return the same thing.
This can be seen here when you run it.
db.each(query_string, function (err, row) {
Since you're checking that each protein is equal to itself (or something very like itself), you're likely fetching every row. And it's why you have to filter all the rows again. And that is one of the reasons why your query is so slow.
if (row.protein_A == item) {
BUT! WHERE Score > [Score] will never be true, a thing cannot be greater than itself except for null! Trinary logic is weird. So only if Score is null can that be true.
So you're returning all the rows whose score is null and the protein column is equal to protein_a or protein_b. This is a lot more rows than you need, I guess you have a lot of rows with null scores.
Your query should incorporate variables (I'm assuming you're using node-sqlite3) and pass in their values when you execute the query.
var query = " \
SELECT * FROM `TABLE` \
WHERE Score > $score AND \
(protein_A = $protein OR protein_B = $protein) \
ORDER BY Score DESC; \
";
var stmt = db.prepare(query);
stmt.each({$score: score, $protein: protein}, function (err, row) {
PI[item].push(row.protein_B);
...
});
First all I sorry on my english.
I development in gwt and I have appengine server.
I have question
1) how I can select row by date.
I write this
String query = "select key from " + MyClass.class.getName();
Query q = pm.newQuery(query);
String filter = "date<= 'Tue Mar 04 19:34:12 IST 2014'";
q.setFilter(filter);
int totalAgendaToNextWeekCounter += ((List<Integer>) q.execute()).size();
but is not work i get always 0.
thank you everybody
It will help to convert the date parameter from a string to a java.util.Date, and pass it as a query parameter, in something like:
Query q = pm.newQuery(query);
int totalAgendaToNextWeekCounter = 0;
try {
java.text.DateFormat dateformat = java.text.DateFormat.getDateInstance();
java.util.Date paramDate = dateformat.parse("2014-03-04T19:34:12");
String query = "select key from " + MyClass.class.getName();
q.setFilter( "date <= :paramDate" );
totalAgendaToNextWeekCounter = ((List<Integer>) q.execute(paramDate)).size();
}
catch (java.text.ParseException ex) {
java.util.logging.Logger.getLogger(logName).severe("");
}
finally {
q.closeAll();
}
This may not fit exactly in your program, but the important thing is not to pass the date parameter as a string.
I'm relatively new to Apex, but I have a question about a batch job that I am creating. I am trying to insert AccountTeamMember records based on my company's territory alignment. The code seems to be working fine, but with one flaw: it is only inserting 100 AccountTeamMember records per user (it should be closer to 400, as that is how many I have loaded in my dev sandbox). Does anyone know what I can do to get an AccountTeamMember record inserted for all accounts per user, rather than 100 of the ~400? Is it something to do with the query including parent-child relationships and governor limits since it is such an even number (100)?
Here is the relevant code:
//list to hold new account teams
List<AccountTeamMember> acctMembers = new List<AccountTeamMember>();
//list to hold new account sharing rules
List<AccountShare> acctSharingRules = new List<AccountShare>();
global Database.querylocator start(Database.BatchableContext BC){
String query = 'SELECT (SELECT User__c FROM Territory_Users__r), (SELECT Account__c FROM Territory_Accounts__r) FROM Territory_Master__c';
return Database.getQueryLocator(query);}
global void execute(Database.BatchableContext BC, List<sObject> scope){
for (sObject s : scope) {
Territory_Master__c tm = (Territory_Master__c) s;
Territory_User__c[] userList = tm.getSObjects('Territory_Users__r');
Territory_Account__c[] accountList = tm.getSObjects('Territory_Accounts__r');
if (userList != null && accountList != null){
for(Territory_User__c uu : userList){
for(Territory_Account__c aa: accountList){
AccountTeamMember addRecord = new AccountTeamMember();
addRecord.AccountId = aa.Account__c;
addRecord.TeamMemberRole = 'Sales Rep';
addRecord.UserId = uu.User__c;
acctMembers.add(addRecord);
AccountShare addSharing = new AccountShare();
addSharing.AccountId = aa.Account__c;
addSharing.OpportunityAccessLevel = 'Read';
addSharing.CaseAccessLevel = 'Read';
addSharing.AccountAccessLevel = 'Edit';
addSharing.UserOrGroupId = uu.User__c;
acctSharingRules.add(addSharing);
}
}
}
}
//DML
if(acctMembers.size() > 0){
insert acctMembers;
}
if(acctSharingRules.size() > 0){
insert acctSharingRules;
}
}
Thanks,
Trey
FYI: This is the final result based on the answer to the question:
global Database.querylocator start(Database.BatchableContext BC){
String query = 'SELECT Id FROM Territory_Master__c';
return Database.getQueryLocator(query);}
global void execute(Database.BatchableContext BC, List<sObject> scope){
for(sObject s : scope){
Territory_Master__c tm = (Territory_Master__c) s;
List<Territory_User__c> userList = [SELECT User__c FROM Territory_User__c WHERE Territory_Master__c = :tm.Id];
List<Territory_Account__c> accountList = [SELECT Account__c FROM Territory_Account__c WHERE Territory_Master__c = :tm.Id];
if (userList != null && accountList != null){
for(Territory_User__c uu : userList){
for(Territory_Account__c aa: accountList){
AccountTeamMember addRecord = new AccountTeamMember();
addRecord.AccountId = aa.Account__c;
addRecord.TeamMemberRole = 'Sales Rep';
addRecord.UserId = uu.User__c;
acctMembers.add(addRecord);
acctSharingRules.add(new AccountShare(
AccountId = aa.Account__c,
OpportunityAccessLevel = 'Read',
CaseAccessLevel = 'Read',
AccountAccessLevel = 'Edit',
UserOrGroupId = uu.User__c)
);
}
}
}
}
//DML
if(acctMembers.size() > 0){
insert acctMembers;
}
if(acctSharingRules.size() > 0){
insert acctSharingRules;
}
}
I suspect that your subqueries got limited and it's your job to make sure you've finished with this record before moving on to the next one.
http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_calls_soql_relationships.htm
Subquery results are like regular query results in that you might need
to use queryMore() to retrieve all the records if there are many
children. For example, if you issue a query on accounts that includes
a subquery, your client application must handle results from the
subquery as well
(the example is for Java calling Salesforce so don't copy-paste the code, try to understand the concept. As far as I know Apex doesn't have this queryMore method, it's for integrations only)
Option 1
Funny enough, it should in theory work if you'd change the for loops to this:
for(Territory_User__c uu : tm.getSObjects('Territory_Users__r')){
for(Territory_Account__c aa: tm.getSObjects('Territory_Accounts__r'))
That's because for loops written in that way should automatically call their internal queryMore().
If it won't work (I haven't tested it), you'll have to make a bit more complex changes.
Option 2
Remove the subqueries from the main query, you'll have to put them in the execute(). Something like this:
for(sObject s : scope){
Territory_Master__c tm = (Territory_Master__c) s;
for(Territory_User__c uu : [SELECT User__c FROM Territory_User__c WHERE Territory_Master__c = :tm.Id]){
for(Territory_Account__c aa: [SELECT Account__c FROM Territory_Account__c WHERE Territory_Master__c = :tm.Id]){
acctMembers.add(new AccountTeamMember(
AccountId = aa.Account__c,
TeamMemberRole = 'Sales Rep',
UserId = uu.User__c)
);
acctSharingRules.add(new AccountShare(
AccountId = aa.Account__c,
OpportunityAccessLevel = 'Read',
CaseAccessLevel = 'Read',
AccountAccessLevel = 'Edit',
UserOrGroupId = uu.User__c)
);
}
}
}
Side notes
Is there a chance you'll hit a limit of 50K rows retrieved across all objects (Territory Master/User/Account? If that's the case you might have to limit the scope of your batch job (optional second parameter passed to the Database.executeBatch()).
This trick can make your script execute bit faster and use less statements (so you won't hit another governor limit):
acctSharingRules.add(new AccountShare(
AccountId = aa.Account__c,
OpportunityAccessLevel = 'Read',
CaseAccessLevel = 'Read',
AccountAccessLevel = 'Edit',
UserOrGroupId = uu.User__c)
);
There's nothing obviously wrong with your code there, so if I were you I'd try to narrow down the cause of the problem. The source of the problem is going to be one of:
Your query is only returning 100 items in accountList or userList or both
The insert can only handle 100 items at a time and is silently failing to insert the rest
The insert is failing for the items after 100
If you haven't already done so, you should familiarize yourself with using Debug Logs in Salesforce.com (Setup | Monitoring | Debug Logs). Turn them on, then run this code with some System.debug calls in there to figure out:
How many items are in userList and accountList?
If it's more than 100, when you go to insert the share rows, are some of those inserts failing? If you use Database.insert() instead of just the insert statement then you'll get a SaveResult[] back that will tell you for each row you attempted to insert whether it was successful or errored out.
So I can't say off the top of my head why this is failing, and I don't see any particular limit that should apply here, but the above should help you debug it at least.
I am trying to count the number of unread messages in my DB Table but is proving to be very difficult. I've even read tutorials online but to no avail.
What I'm doing should be simple.
Here's what I'm trying to do:
COUNT NUMBER OF ROWS IN NOTIFICATIONSTABLE
WHERE USERID = #0 AND MESSAGEWASREAD = FALSE
Can somebody please point me in the right direction? Any help will be appreciated.
Thank you
#helper RetrievePhotoWithName(int userid)
{
var database = Database.Open("SC");
var name = database.QuerySingle("select FirstName, LastName, ProfilePicture from UserProfile where UserId = #0", userid);
var notifications = database.Query("SELECT COUNT(*) as 'counter' FROM Notifications WHERE UserID = #0 AND [Read] = #1", userid, false);
var DisplayName = "";
if(notifications["counter"] < 1)
{
DisplayName = name["FirstName"] + " " + name["LastName"];
}
else
{
DisplayName = name["FirstName"] + ", you have " + notifications["counter"] + " new messages.";
}
<img src="#Href("~/Shared/Assets/Images/" + name["ProfilePicture"] + ".png")" id="MiniProfilePicture" /> #DisplayName
database.Close();
}
SELECT COUNT(*) FROM NotificationsTable WHERE
UserID = #UserID AND MessageWasRead = 0;
Sql Count Function
Okay so this is based on what I think should be done. I don't know the underlying types, so it is going to be my best guess.
var notifications = database.QuerySingle("Select COUNT(*) as NumRecs....");
if((int)notifications["NumRecs"] > 0)) .......
I changed the query for notifications to QuerySingle. You don't need a recordest, you only need a scalar value, so that should (hopefully remove your problem with the implicit conversion in the equals you were having.
I would also check to see if your database object implements IDisposable (place it in a using statement if so) as you are calling close, and this won't actually call close (I know it's not dispose but it might have dispose too) if you encounter and exception before the close function is called.
int unreadMessageCount = db.Query("SELECT * FROM Notification WHERE UserId=#0 AND Read=#1",UserId,false).Count();
string displayname = name["FirstName"] + " " + name["LastName"] + unreadMessageCount>0?",you have " + unreadMessageCount :"";