I am using the following pattern to update my SQL Server records using Linq to SQL:
List<int> allIds;
using (MyDataContext dc= new MyDataContext())
{
dc.CommandTimeout = 0;
allIds = dc.MyTables.Select(x => x.Id).ToList();
}
Parallel.ForEach(allIds, x => ComputeAndSave(x));
and
ComputeAndSave(int x, MyDataContext dc)
{
var myRecord= dc.MyTables.Select(x => x).Where(x => x.Id == id).FirstOrDefault();
myRecord.Total = myRecord.Total + 1; //some modification(s)
dc.SubmitChanges();
}
I use the same pattern in multiple parts of my application to update all rows in some tables. And in all these cases, memory used by SQL Server slowly increases and increases and increases. The only way I find to release all that memory is to stop and restart the SQL Server Service. Why is there a memory leak here and how can it be fixed?
Many thanks!
SQL Server is supposed to take all available memory. It is supposed to run on headless servers.
In SSMS, open the server properties and set a reasonable amount of max server memory.
Related
I'm running big dependency scan on legacy db and see that some objects have obsolete ref links, if you run this code in SSMS for View that points to not existing table like in my case, you will get your output on Results tab AND error info in Messages . Like in my case below.
I tried to check all env things I know and output of this stored procedure, but didn't see any indication.
How I can capture this event as I'm running this in looped dynamic SQL script and capture output in my table for further processing?
Updated:
it just text in Message box ,on error, you still have output on
Results tab
this is sp, it loop thru object list I took from sys.object and run this string as my sample to get all dependencies, load all into table. This call to
sql_reference_entities is the only way to get inter database
dependency on column level. So I need stick to this 100$>
--
Select *
From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')
--
----update------
This behavior was fixed in SQL Server 2014 SP3 and SQL Server 2016 SP2:
Starting from Microsoft SQL Server 2012, errors raised by
sys.dm_sql_referenced_entities (such as when an object has undergone a
schema change) cannot be caught in a TRY...CATCH Transact-SQL block.
While this behavior is expected in SQL Server 2012 and above, this
improvement introduces a new column that's called is_incomplete to the
Dynamic Management View (DMV).
KB4038418 - Update adds a new column to DMV sys.dm_sql_referenced_entities in SQL Server 2014 and 2016
----update-------
The tldr is that you can't capture these on the server side, and must use a client program in C#, PowerShell or some other client that can process info messages.
That DMV is doing something strange that I don't fully understand. It's generating errors (which a normal UDF is not allowed to do), and those errors do not trigger a TRY/CATCH block or set ##error. EG
create table tempdb.dbo.foo(id int)
go
create view dbo.v_View_Obs_Table
as
select * from tempdb.dbo.foo
go
drop table tempdb.dbo.foo
go
begin try
Select * From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')
end try
begin catch
select ERROR_MESSAGE(); --<-- not hit
end catch
However these are real errors, as you can see running this from client code:
using System;
using System.Data.SqlClient;
namespace ConsoleApp6
{
class Program
{
static void Main(string[] args)
{
using (var con = new SqlConnection("Server=.;database=AdventureWorks;integrated security=true"))
{
con.Open();
con.FireInfoMessageEventOnUserErrors = true;
con.InfoMessage += (s, a) =>
{
Console.WriteLine($"{a.Message}");
foreach (SqlError e in a.Errors)
{
Console.WriteLine($"{e.Message} Number:{e.Number} Class:{e.Class} State:{e.State} at {e.Procedure}:{e.LineNumber}");
}
};
var cmd = con.CreateCommand();
cmd.CommandText = "Select * From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')";
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read() || (rdr.NextResult() && rdr.Read()))
{
Console.WriteLine(rdr[0]);
}
}
Console.ReadKey();
}
}
}
}
outputs
Invalid object name 'tempdb.dbo.foo'.
Invalid object name 'tempdb.dbo.foo'. Number:208 Class:16 State:3 at v_View_Obs_Table:4
0
The dependencies reported for entity "dbo.v_View_Obs_Table" might not include references to all columns. This is either because the entity references an object that does not exist or because of an error in one or more statements in the entity. Before rerunning the query, ensure that there are no errors in the entity and that all objects referenced by the entity exist.
The dependencies reported for entity "dbo.v_View_Obs_Table" might not include references to all columns. This is either because the entity references an object that does not exist or because of an error in one or more statements in the entity. Before rerunning the query, ensure that there are no errors in the entity and that all objects referenced by the entity exist. Number:2020 Class:16 State:1 at :1
I want to copy a database to a sql server to another, but i just want to copy structure (views, stored procedures, tables, fields, indexes, etc), no rows.
I tried to generate a script from sql server management but the script is very verbose (task menu > create as)
Follow Below steps for generate script :
Right Click on Database
Select task
Select Generate Script from Task
Follow the steps
Finally finish for complete this process
You can either use the SQL Server Management Object API (see task "creating, altering and removing databases"):
C# Code for generate sql script :
public string GenerateScript()
{
var sb = new StringBuilder();
var srv= new Server(#"Your Database Server Name");
var db= server.Databases["Your Database name"];
var scrpt = new Scripter(srv);
scrpt.Options.ScriptDrops = false;
var obj= new Urn[1];
foreach (Table tbl in db.Tables)
{
obj[0] = tbl.Urn;
if (tbl.IsSystemObject == false)
{
StringCollection sc = scripter.Script(obj);
foreach (var st in sc)
{
sb.Append(st);
}
}
}
return sb.ToString();
}
You case use Copy database wizard
Some limitations of the it are :
1.The Copy Database Wizard is not available in the Express edition.
1.The Copy Database Wizard cannot be used to copy or move databases that:
Are System.
Are marked for replication.
Are marked Inaccessible, Loading, Offline, Recovering, Suspect, or in Emergency Mode.
Have data or log files stored in Microsoft Azure storage.
I am migrating the SQL DMO logic to a SQL SMO logic and I am not sure how to achieve the same functionality for a few of the attributes.
This is the DMO:
oBCP = New SQLDMO.BulkCopy2
With oBCP
.ImportRowsPerBatch = ImportRowsPerBatch
.DataFilePath = Path.Combine(gtSysTempDir, "file.dat")
.LogFilePath = Path.Combine(gtSysTempDir, "file.log")
.ErrorFilePath = Path.Combine(gtSysTempDir, "file.err")
.MaximumErrorsBeforeAbort = 1
.DataFileType = SQLDMO.SQLDMO_DATAFILE_TYPE.SQLDMODataFile_TabDelimitedChar
End With
This is the SMO so far:
trans = New Transfer
With trans
.ImportRowsPerBatch = ImportRowsPerBatch
.TargetDatabaseFilePath = Path.Combine(gtSysTempDir, "file.dat")
.TargetLogFilePath = Path.Combine(gtSysTempDir, "file.log")
'.ErrorFilePath = Path.Combine(gtSysTempDir, "file.err")
.MaximumErrorsBeforeAbort = 1
.DataFileType = SQLDMO.SQLDMO_DATAFILE_TYPE.SQLDMODataFile_TabDelimitedChar
End With
What are the equivalents, if they exist, of ImportRowsPerBatch (BatchSize?), ErrorFilePath, MaxErrorsBeforeAbort, and DataFileType?
Thanks.
Despite what the MSDN says, Transfer is not actually the equivalent of BulkCopy2; it's intended for scripting/copying entire databases whereas BulkCopy2 is only for bulk copying data. The settings don't correspond at all -- in particular, Transfer.TargetDatabaseFilePath is supposed to be the location of a database file, whereas BulkCopy2.DataFilePath is the location of the file where imported/exported data goes. Some bulk copy scenarios are handled by Transfer, but most aren't.
If you're not actually copying an entire database, you have several alternatives, none of which use SMO (which has no class for bulk copying as such):
Directly invoking a BULK INSERT statement, which is probably what BulkCopy2 does in the background (but I haven't verified this);
Using the SqlBulkCopy class, which will require some extra code (but which also offers far more flexibility as the data to be imported can come from any source);
Building a command line and invoking bcp.
Of these, BULK INSERT is the most straightforward.
I'm new to nhibernate (but used hibernate for java before).
I built a session factory for our sql server database (sql server enterprise edition 8)
ISessionFactory factory2 = Fluently.Configure()
.Database(MsSqlConfiguration.MsSql2008
.ConnectionString(#"user id=xx; password=xxx;server=xxx;initial catalog=xxx")
.ShowSql()
)
.Mappings(m => m.FluentMappings
.AddFromAssemblyOf<Program>())
.ExposeConfiguration(cfg => new SchemaValidator(cfg).Validate())
.BuildSessionFactory();
So I use the ShowSql() method to log the queries to the console.
In my programm I load / create two objects and want to persist them and then do a update on a column:
using (var session = sf.OpenSession())
{
session.FlushMode = FlushMode.Always;
using (var ta = session.BeginTransaction())
{
Console.ReadKey();
PMA pm = session.CreateCriteria<PMA>()
.Add(Restrictions.Eq("Name", "HANSER")).List<PMA>().FirstOrDefault();
if (pm == null)
{
pm = new PMA();
pm.Prio = "1";
pm.Name = "HANSER";
pm.Datum = DateTime.Now;
session.Save(pm);
}
Clip clip = new Clip();
clip.PMA = pm;
clip.sys_created = DateTime.Now;
clip.sys_name = "system name";
clip.Title = "Test";
session.Save(clip);
Console.ReadKey();
clip.Title = "PETERSEN";
session.SaveOrUpdate(clip);
session.Transaction.Commit();
session.Flush();
session.Dispose();
Console.ReadKey();
}
}
The first insert for the pm object will be logged on the console, but the other insert and the update for the clip object don't appear in the console. When I look in the database, I see there is everything right, everything will be inserted and updated. But I want to see the query. I try to set flush mode to always and make a session.Flush() to the session at the end and then a session.Dispose(), but nothing changes.
When I use postgres (only change the sessionfactory), I see all query logs.
How can I let nhibernate log all queries for sql server ?
When using ADO.NET batching (on by default in SQL Server, which supports it), DML queries are not logged to the console.
EDIT: I think it's a problem on the subquery on the LINQ-generated query, it get all the records... But I don't know how could I fix it
I have made a simple ASP.NET MVC 2 application that does SELECT queries on a view, I get really poor performance, and while doing a simple benchmark with jMeter (10 conccurents connection) while disabling the cache (I don't want everything to rely on the non customizable/extreme OutputCache)
I see that the SQL Server get overloaded, consuming a LOT of CPU (up to 100%) and all its reserved memory space (512MB)
Here is the action code that cause the problems (manual transactions because it cause DeadLock with the other program that insert new data on the database) :
public ActionResult Index(int page = 0)
{
IronViperEntities db = new IronViperEntities();
db.Connection.Open();
DbTransaction transaction = db.Connection.BeginTransaction(IsolationLevel.ReadUncommitted);
var messages = (from globalView in db.GlobalViews orderby globalView.MessagePostDate descending select globalView).Skip(page*perPage).Take(perPage);
transaction.Commit();
db.Connection.Close();
ViewData["page"] = page;
ViewData["messages"] = messages;
return View();
}
Here is the query executed on the database :
SELECT TOP (100)
[Extent1].[MessageId] AS [MessageId],
[Extent1].[MessageUuid] AS [MessageUuid],
[Extent1].[MessageData] AS [MessageData],
[Extent1].[MessagePostDate] AS [MessagePostDate],
[Extent1].[ChannelName] AS [ChannelName],
[Extent1].[UserName] AS [UserName],
[Extent1].[UserUuid] AS [UserUuid],
[Extent1].[ChannelUuid] AS [ChannelUuid]
FROM ( SELECT [Extent1].[MessageId] AS [MessageId], [Extent1].[MessageUuid] AS [MessageUuid], [Extent1].[MessageData] AS [MessageData], [Extent1].[MessagePostDate] AS [MessagePostDate], [Extent1].[ChannelName] AS [ChannelName], [Extent1].[UserName] AS [UserName], [Extent1].[UserUuid] AS [UserUuid], [Extent1].[ChannelUuid] AS [ChannelUuid], row_number() OVER (ORDER BY [Extent1].[MessagePostDate] DESC) AS [row_number]
FROM (SELECT
[GlobalView].[MessageId] AS [MessageId],
[GlobalView].[MessageUuid] AS [MessageUuid],
[GlobalView].[MessageData] AS [MessageData],
[GlobalView].[MessagePostDate] AS [MessagePostDate],
[GlobalView].[ChannelName] AS [ChannelName],
[GlobalView].[UserName] AS [UserName],
[GlobalView].[UserUuid] AS [UserUuid],
[GlobalView].[ChannelUuid] AS [ChannelUuid]
FROM [dbo].[GlobalView] AS [GlobalView]) AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > 0
ORDER BY [Extent1].[MessagePostDate] DESC
View Code :
SELECT dbo.Messages.Id AS MessageId, dbo.Messages.Uuid AS MessageUuid, dbo.Messages.Data AS MessageData, dbo.Messages.PostDate AS MessagePostDate,
dbo.Channels.Name AS ChannelName, dbo.Users.Name AS UserName, dbo.Users.Uuid AS UserUuid, dbo.Channels.Uuid AS ChannelUuid
FROM dbo.Messages INNER JOIN
dbo.Users ON dbo.Messages.UserId = dbo.Users.Id INNER JOIN
dbo.Channels ON dbo.Messages.ChannelId = dbo.Channels.Id
I don't think the server hardware is a problem, I can run equivalent Rails/Grails application without any performance issue. (Dual Core, 3Gb of RAM)
A select count(*) on GlobalView returns ~270.000 lines, indexes are daily rebuilt and a explain show it uses all the clustered indexes.
I get an HTTP average response time of 8000ms, the SQL Server Management Studio shows an average CPU time for this SQL query of 866ms and an average logical IO of 7,592.03.
Database file size if ~180MB
I am using Windows Server 2008 R2 Enterprise Edition, ASP.NET MVC 2 with IIS 7.5 and SQL Server 2008 R2 Express Edition with Advanced Services. They are the only things running on this server.
What can I do ?
Thank you
I guess you got the query from SQL Server Profiler. Save the result, and pass it into the Database Engine Tuning Advisor. That might help you create additional indexes and statistics.
Just out of curiosity: wouldn't appending a .ToList() to the end of the var messages = ... line help?
I've found the probleme,
I replaced "orderby globalView.MessagePostDate descending" by "orderby globalView.MessageId descending", because there isn't any index on MessagePostDate, and that is muuuuch better !
Thank you