I have an ASP.NET MVC app I've developed for a client. MVC 5, EF 6.
The web server and db server are separate. All of a sudden a bunch of data was deleted from the db server. Users were performing no delete functions. Is there any scenario that would cause this? The records seem to be random. No stored procs, triggers, etc.... running. The app was working fine for months. Any scenario where SQL Server (2014 version) would delete records in a table? No errors were displayed to the user.
**** UPDATE ****
The only "delete" related code that I rolled out recently was this...
[Authorize]
public class WorkResultsController : Controller
{
private readonly ABC_WorkingContext db = new ABC_WorkingContext();
public ActionResult DeleteEvent(int id, bool? performRedirectAfterDelete = true)
{
if (!WorkFormServices.IsEditOrDeleteEnabled(id)) return this.HttpNotFound();
var #event = this.db.Events.Find(id);
try
{
// first remove any Work questions related to this event
var WorkResultsToDelete = this.db.WorkResults.Where(e => e.EventId == id).ToList();
foreach (var row in WorkResultsToDelete) this.db.WorkResults.Remove(row);
this.db.Events.Remove(#event);
this.db.SaveChanges();
if (performRedirectAfterDelete == true) return this.RedirectToAction("Index", "WorkScheduler");
return this.Json(
new { success = true, responseText = "Delete success!" },
JsonRequestBehavior.AllowGet);
}
catch (Exception)
{
return this.Json(
new { success = false, responseText = "Delete failed!!" },
JsonRequestBehavior.AllowGet);
}
}
I want to delete only WorkResults records related to the specific ID. So, I believe this is working correctly. Do you see any unintended deletes that could happen?
I agree with Min - a DB won't just delete data. This is more than likely a code (app or DB side) issue or a breach of some kind.
I would check:
app code - is there a bad SQL call/statement (related to the tables you're missing data from) that could have deleted more than it should
Stored Procs, Triggers - same as above - an SQL mistake could wreak havoc
Table relationships - are any unwanted cascade deletes set up?
EF - are there any unwanted cascades set up in this between entities?
Logins - for sanity - change the passwords for the logins your app uses...this could be a breach maybe - hard to tell without seeing the pattern of missing data
First, no commercial DB deletes random data by itself. If it really deletes its client's data, its maker would be sued by client.
So, there are DELETE queries in somewhere or someone executed DELETE operation on SQL SERVER Studio. You can monitor DB queries. Check your queries and find which query delete your data. And ask DBA or DB Operator if they executed some queries.
In my experience, there is no "THERE IS NO SUCH QUERY".
Related
Using Microsoft.EntityFrameworkCore version 6.0.5, with .NET 6, I'm doing multiple database operations inside of a transaction. When I call CommitAsync, it fails saying:
Cannot issue SAVE TRANSACTION when there is no active transaction.
How is there no active transaction when I've clearly created one?
var task = await Context.Database.BeginTransactionAsync(cancellationToken);
try {
var newEntity = ...
... Multiple database operations ...
await Context.SaveChangesAsync(cancellationToken);
... Other DB operations that now make use of the just created objects.
await Context.SaveChangesAsync(cancellationToken);
await task.CommitAsync(cancellationToken);
return newEntity;
} catch (Exception) {
await task.RollbackAsync(cancellationToken);
throw;
}
I'm creating multiple object in different tables that reference each other, and calling other people's code that might be doing a save operation of its own, thus the transaction.
Strangely I tracked this down to SQL server corruption. I had renamed a table and for some reason, if I saved changes to that table, it was internally (in the SQL server) trying to use that old name. I nuked the table and recreated and then I had no more transaction issues.
I am working with a SQL Server table that contains 80 million (80,000,000) rows. Data space = 198,000 MB. Not surprisingly, queries against this table often churn or timeout. To add to the issues, the table rows get updated fairly frequently and new rows also get added on a regular basis. It thus continues to grow like a viral outbreak.
My issue is that I would like to write Entity Framework 5 LINQ to Entities queries to grab rows from this monster table. As I've tried, timeouts have become outright epidemic. A few more things: the table's primary key is indexed and it has non-clustered indexes on 4 of its 19 columns.
So far, I am writing simple LINQ queries that use Transaction Scope and Read Uncommitted Isolation Level. I have tried increasing both the command timeout and the connection timeout. I have written queries that return FirstOrDefault() or a collection, such as the following, which attempts to grab a single ID (an int) from seven days before the current date:
public int GetIDForSevenDaysAgo(DateTime sevenDaysAgo)
{
using (var txn = new TransactionScope(TransactionScopeOption.Required, new TransactionOptions { IsolationLevel = IsolationLevel.ReadUncommitted }))
{
var GetId = from te in _repo.GetTEvents()
where te.cr_date > sevenDaysAgo
orderby te.cr_date
select te.id;
return GetId.FirstOrDefault();
}
}
and
public IEnumerable<int> GetIDForSevenDaysAgo(DateTime sevenDaysAgo)
{
using (var txn = new TransactionScope(TransactionScopeOption.Required, new TransactionOptions { IsolationLevel = IsolationLevel.ReadUncommitted }))
{
var GetId = from te in _repo.GetTEvents()
where te.cr_date > sevenDaysAgo
orderby te.cr_date
select te.id;
return GetId.Take(1);
}
}
Each query times out repeatedly regardless of the timeout settings. I'm using the repository pattern with Unity DI and fetching the table with IQueryable<> calls. I'm also limiting the repository call to eight days from the current date (hoping to only grab the needed subset of this mammoth table). I'm using Visual Studio 2013 with Update 5 targeting .NET v4.5 and SQL Server 2008 R2.
I generated the SQL statement that EF generates and it didn't look incredibly more complicated than the LINQ statements above. And my brain hurts.
So, have I reached some sort of tolerance limit for EF? Is the table simply too big? Should I revert to Stored Procedures/domain methods when querying this table? Are there other options I should explore? There was some discussion around removing some of the table's rows, but that probably won't happen anytime soon. I did read a little about paging, but I'm not sure if that would help or not. Any thoughts or ideas would be appreciated! Thank you!
As I can see you only selecting data and don't change it. So why do you need to use TransactionScope? You need it only when you have 2 or more SaveChanges() in your code and you want them to be in one transaction. So get rid of it.
Another thing that i whould use in your case is disable change tracking and auto detection of changes on your context. But be carefull if you don't rectreade your context on each request. It can presist old data.
To do it you should write this lines near your context initialization:
context.ObjectTrackingEnabled = false;
context.DeferredLoadingEnabled = false;
The other thing that you should think about is pagenation and Cache. But as i can see in your example you trying to get only one row. So can't say anything particular.
I recommend you to read this article to further optimisation.
It's not easy to say if you have to go with stored procedures or EF since we speak for a monster. :-)
The first thing I would do is to run the query in SSMS displaying the Actual Execution Plan. Sometimes it provides information about indexes missing that might increase performance.
From you example, I 'm pretty sure you need an index on that date column.
In other words, -if you have access- be sure that table design is optimal for that amount of data.
My thought is that if a simple query hangs, what more EF can do?
We have this web service with the following algorithm (the input is an id and a name)
public bool SetCustomer(int externalId,string name)
{
using (var db = new BusinessEntities())
{
var c = db.Customer.FirstOrDefault(c => c.externalId== externalId) ?? new Customer(){externalId = externalId};
c.name = name;
db.SaveChanges();
}
}
The problem is, if someone calls at the same moment this web service with the same id, then 2 customers are created.
We cannot add a unique constraint, because some other process in the app might create customer with the same externalId.
We thought about a solution with a dictionary of object that would be used as lock argument, but it doesn't seems right.
We don't want to lock the full table customer because we have a lot of parallel calls, so this will cause timeout.
Can you use an intermediate table to log insertion jobs.
Then do the jobs (creating the customer) with a mono threaded task.
This may require an asynchronous solution if your web method has to return a result.
Or even something more elaborated like Asynchronous Triggers from Sql Service Broker.
I found something :
http://michaeljswart.com/2011/09/mythbusting-concurrent-updateinsert-solutions/
it's not with entity framework but I don't think EF provide access to this kind of features.
thanks
I am using EclipseLink-2.6.1 with Amazon RDS database instance. The following code is used to insert new entities into database:
tx = em.getTransaction();
tx.begin();
for (T item : persistCollection) {
em.merge(item);
}
tx.commit();
The object which is being persisted has composite primary key (not a generated one). Locally, queries run super fast, but when inserting into remote DB it is really slow process (~20 times slower). I have tried to implement JDBC batch writing but had no success with it (eclipselink.jdbc.batch-writing and rewriteBatchedStatements=true). When logging queries which are being executed I only see lots of SELECTS and not one INSERT (SELECTS are probably here because objects are detached at first).
My question is how to proceed on this problem? (I would like to have batch writing and then see how the performance changes, but any help is appreciated)
Thank you!
Edit:
When using em.persist(item) loop is finished almost instantly but after tx.commit() there are lots (I guess for every persisted item) queries like :
[EL Fine]: sql: ServerSession(925803196) Connection(60187547) SELECT NAME FROM TICKER WHERE (NAME = ?), bind => [AA]
My model has #ManyToOne relationship with ticker_name. Why are there again so many slow SELECT queries?
Let's assume we have a bunch of LINQ2SQL InsertOnSubmit statements against a given DataContext. If the SubmitChanges call is successful, is there any way to automatically generate a list of SQL commands (or even LINQ2SQL statements) that could undo everything that was submitted at a later time? It's like executing a rollback even though everything worked as expected.
Note: The destination database will either be Oracle or SQL Server, so if there is specific functionality for both databases that will achieve this, I'm happy to use that as well.
Clarification:
I do not want the "rollback" to happen automatically as soon as the inserts have succesfully completed. I want to have the ability to "undo" the INSERT statements via DELETE (or some other means) up to 24 hours (for example) after the original program finished inserting data. We can ignore any possible referential integrity issues that may come up.
Assume a Table A with two columns: Id (autogenerated unique id) and Value (string)
If the LINQ2SQL code performs two inserts
INSERT INTO Table A VALUES('a') // Creates new row with Id = 1
INSERT INTO Table A VALUES('z') // Creates new row with Id = 2
<< time passes>>
At some point later I would want to be able "undo" this by executing
DELETE FROM A Where Id = 1
DELETE FROM A Where Id = 2
or something similar. I want to be able to generate the DELETE statements to match the INSERT ones. Or use some functionality that would let me capture a transaction and perform a rollback later.
We cannot just 'reset the database' to a certain point in time either as other changes not initiated by our program could have taken place since.
It is actually quite easy to do this, because you can pass in a SqlConnection into the LINQ to SQL DataContext on construction. Just run this connection in a transaction and roll that transaction back as soon as you're done.
Here's an example:
string output;
using (var connection = new SqlConnection("your conn.string"))
{
connection.Open();
using (var transaction = connection.StartTransaction())
{
using (var context = new YourDataContext(connection))
{
// This next line is needed in .NET 3.5.
context.Transaction = transaction;
var writer = new StringWriter();
context.Log = writer;
// *** Do your stuff here ***
context.SubmitChanges();
output = writer.ToString();
}
transaction.Rollback();
}
}
I am always required to provide a RollBack script to our QA team for testing before any change script can be executed in PROD.
Example: Files are sent externally with a bunch of mappings between us, the recipient and other third parties. One of these third parties wants to change, on an agreed date, the mappings between the three of us.
Exec script would maybe update some exisiting, delete some now redundant and insert some new records - scope_identity used in subsequent relational setup etc etc.
If, for some reason, after we have all executed our changes and the file transport is fired up, just like in UAT, we see some errors not encountered in UAT, we might multilaterally make the decision to roll back the changes. Hence the roll back script.
SQL has this info when you BEGIN TRAN until you COMMIT TRAN or ROLLBACK TRAN. I guess your question is the same as mine - can you output that info as a script.
Why do you need this?
Maybe you should explore the flashback possibilities of Oracle. It makes it possible to travel back in time.
It makes it possible to reset the content of a table or a database to how it once was at a specific moment in time (or at a specific system change number).
See: http://www.oracle.com/technology/deploy/availability/htdocs/Flashback_Overview.htm