Dapper & SQLinq Package Deferred / Buffer - dapper

I am using Dapper with the SQLinq Nuget package.
Here is some sample code I am using.
Dapper with SQLinq runs the query before I run a .ToList() (for example).
I know with Dapper you can specify "buffer" to make it run deferred, but I do not see how to apply that with the SQLinq NuGet package for Dapper.
using (var sqlCnn = base.GetConnection())
{
var viewData = sqlCnn.Query(from s in new SQLinq<Week_Returns_stats_V>(). . .
public SqlConnection GetConnection(bool mars = false)
{
if (_sqlCnn != null)
{
if (_sqlCnn.State != ConnectionState.Open) CloseConnection();
}
if (mars)
{
var scsb = new SqlConnectionStringBuilder(_cnnString)
{
MultipleActiveResultSets = true
};
}
_sqlCnn = new SqlConnection(_cnnString);
return _sqlCnn;
}

I found how to include buffered after the query.
public static IEnumerable<T> Query<T>
(this IDbConnection dbconnection,
SQLinq<T> query,
IDbTransaction transaction = null,
bool buffered = true,
int? commandTimeout = default(int?),
CommandType? commandType =
default(CommandType?)) where T : new();
var viewData = sqlCnn.Query(from s in new SQLinq<Returns_stats()
select new
{
AVGPercentReturnX100 = s.AVGPercentReturnX100,
PercentProfitableX100 = s.PercentProfitableX100
}, buffered:true).AsEnumerable()

Related

Convert SqlDataReader to object for .NET Framework 4

I am working on a class Library with .NET Framework 4.0. I have managed to pull a row using ADO.NET, but I'm unable to read individual values. I want the end result in class object. I have tried reading individual value dbReader.GetValue(dbReader.GetOrdinal("BranchCode")) but getting empty result.
Branch class:
public class Branch
{
public Branch() { }
public int BranchId { get; set; }
public string BranchCode { get; set; }
public string BranchName { get; set; }
}
DataReader class:
public void Initialize()
{
try
{
string connectionString = "xyz";
SqlConnection dbConnection = new SqlConnection(connectionString);
SqlCommand cmd = new SqlCommand("Select * from dbo.Branch", dbConnection);
dbConnection.Open();
SqlDataReader dbReader = cmd.ExecuteReader();
while (dbReader.Read())
{
var x1 = dbReader.GetValue(dbReader.GetOrdinal("BranchId"));
var x2 = dbReader.GetValue(dbReader.GetOrdinal("BranchCode"));
var x3 = dbReader.GetValue(dbReader.GetOrdinal("BranchName"));
}
var dd = "Dd";
}
catch(Exception ex)
{
throw ex;
}
}
You have a number of issues with your code.
You need to actually create the Branch objects and do something with them. For example, return List.
To read the values, the easiest way is to do (typeNameHere) dbReader["ColumnName"]
You should SELECT exactly the right columns not SELECT *
Don't catch then re-throw exceptions with throw ex; as it wipes the stakc trace.
public List<Branch> Initialize()
{
string connectionString = "xyz";
const string query = #"
Select
b.BranchId,
b.BranchCode,
b.BranchName
from dbo.Branch b;
";
using (SqlConnection dbConnection = new SqlConnection(connectionString))
using (SqlCommand cmd = new SqlCommand(query, dbConnection))
{
dbConnection.Open();
using (SqlDataReader dbReader = cmd.ExecuteReader())
{
var list = new List<Branch>();
while (dbReader.Read())
{
var b = new Branch();
b.BranchId = (int)dbReader["BranchId"];
b.BranchCode = (string)dbReader["BranchCode"];
b.BranchName = (string)dbReader["BranchName"];
list.Add(b);
}
return list;
}
}
}

Insert data using Stored procedure at Entity frame work core 3.1 [duplicate]

I am using EF Core 1.0 in an ASP.NET Core App. Can you please point me to the proper way of executing stored procedures? The old method with ObjectParameters and ((IObjectContextAdapter)this).ObjectContext.ExecuteFunction is not working.
Support for stored procedures in EF Core 1.0 is resolved now, this also supports the mapping of multiple result-sets.
Check here for the fix details
And you can call it like this in c#
var userType = dbContext.Set().FromSql("dbo.SomeSproc #Id = {0}, #Name = {1}", 45, "Ada");
Stored procedure support is not yet (as of 7.0.0-beta3) implemented in EF7. You can track the progress of this feature using issue #245.
For now, you can do it the old fashioned way using ADO.NET.
var connection = (SqlConnection)context.Database.AsSqlServer().Connection.DbConnection;
var command = connection.CreateCommand();
command.CommandType = CommandType.StoredProcedure;
command.CommandText = "MySproc";
command.Parameters.AddWithValue("#MyParameter", 42);
command.ExecuteNonQuery();
To execute the stored procedures, use FromSql method which executes RAW SQL queries
e.g.
var products= context.Products
.FromSql("EXECUTE dbo.GetProducts")
.ToList();
To use with parameters
var productCategory= "Electronics";
var product = context.Products
.FromSql("EXECUTE dbo.GetProductByCategory {0}", productCategory)
.ToList();
or
var productCategory= new SqlParameter("productCategory", "Electronics");
var product = context.Product
.FromSql("EXECUTE dbo.GetProductByName #productCategory", productCategory)
.ToList();
There are certain limitations to execute RAW SQL queries or stored procedures. You can’t use it for INSERT/UPDATE/DELETE. if you want to execute INSERT, UPDATE, DELETE queries, use the ExecuteSqlCommand
var categoryName = "Electronics";
dataContext.Database
           .ExecuteSqlCommand("dbo.InsertCategory #p0", categoryName);
The support for stored procedure in EF Core is similar to the earlier versions of EF Code first.
You need to create your DbContext class by inherting the DbContext class from EF. The stored procedures are executing using the DbContext.
First step is to write a method that create a DbCommand from the DbContext.
public static DbCommand LoadStoredProc(
this DbContext context, string storedProcName)
{
var cmd = context.Database.GetDbConnection().CreateCommand();
cmd.CommandText = storedProcName;
cmd.CommandType = System.Data.CommandType.StoredProcedure;
return cmd;
}
To pass parameters to the stored procedure use the following method.
public static DbCommand WithSqlParam(
this DbCommand cmd, string paramName, object paramValue)
{
if (string.IsNullOrEmpty(cmd.CommandText))
throw new InvalidOperationException(
"Call LoadStoredProc before using this method");
var param = cmd.CreateParameter();
param.ParameterName = paramName;
param.Value = paramValue;
cmd.Parameters.Add(param);
return cmd;
}
Finally for mapping the result into a list of custom objects use the MapToList method.
private static List<T> MapToList<T>(this DbDataReader dr)
{
var objList = new List<T>();
var props = typeof(T).GetRuntimeProperties();
var colMapping = dr.GetColumnSchema()
.Where(x => props.Any(y => y.Name.ToLower() == x.ColumnName.ToLower()))
.ToDictionary(key => key.ColumnName.ToLower());
if (dr.HasRows)
{
while (dr.Read())
{
T obj = Activator.CreateInstance<T>();
foreach (var prop in props)
{
var val =
dr.GetValue(colMapping[prop.Name.ToLower()].ColumnOrdinal.Value);
prop.SetValue(obj, val == DBNull.Value ? null : val);
}
objList.Add(obj);
}
}
return objList;
}
Now we’re ready for execute the stored procedute with the ExecuteStoredProc method and maps it to the a List whose type that’s passed in as T.
public static async Task<List<T>> ExecuteStoredProc<T>(this DbCommand command)
{
using (command)
{
if (command.Connection.State == System.Data.ConnectionState.Closed)
command.Connection.Open();
try
{
using (var reader = await command.ExecuteReaderAsync())
{
return reader.MapToList<T>();
}
}
catch(Exception e)
{
throw (e);
}
finally
{
command.Connection.Close();
}
}
}
For example, to execute a stored procedure called “StoredProcedureName” with two parameters called “firstparamname” and “secondparamname” this is the implementation.
List<MyType> myTypeList = new List<MyType>();
using(var context = new MyDbContext())
{
myTypeList = context.LoadStoredProc("StoredProcedureName")
.WithSqlParam("firstparamname", firstParamValue)
.WithSqlParam("secondparamname", secondParamValue).
.ExecureStoredProc<MyType>();
}
I tried all the other solutions but didn't worked for me. But I came to a proper solution and it may be helpful for someone here.
To call a stored procedure and get the result into a list of model in EF Core, we have to follow 3 steps.
Step 1.
You need to add a new class just like your entity class. Which should have properties with all the columns in your SP. For example if your SP is returning two columns called Id and Name then your new class should be something like
public class MySPModel
{
public int Id {get; set;}
public string Name {get; set;}
}
Step 2.
Then you have to add one DbQuery property into your DBContext class for your SP.
public partial class Sonar_Health_AppointmentsContext : DbContext
{
public virtual DbSet<Booking> Booking { get; set; } // your existing DbSets
...
public virtual DbQuery<MySPModel> MySP { get; set; } // your new DbQuery
...
}
Step 3.
Now you will be able to call and get the result from your SP from your DBContext.
var result = await _context.Query<MySPModel>().AsNoTracking().FromSql(string.Format("EXEC {0} {1}", functionName, parameter)).ToListAsync();
I am using a generic UnitOfWork & Repository. So my function to execute the SP is
/// <summary>
/// Execute function. Be extra care when using this function as there is a risk for SQL injection
/// </summary>
public async Task<IEnumerable<T>> ExecuteFuntion<T>(string functionName, string parameter) where T : class
{
return await _context.Query<T>().AsNoTracking().FromSql(string.Format("EXEC {0} {1}", functionName, parameter)).ToListAsync();
}
Hope it will be helpful for someone !!!
"(SqlConnection)context"
-- This type-casting no longer works. You can do: "SqlConnection context;
".AsSqlServer()"
-- Does not Exist.
"command.ExecuteNonQuery();"
-- Does not return results. reader=command.ExecuteReader() does work.
With dt.load(reader)... then you have to switch the framework out of 5.0 and back to 4.51, as 5.0 does not support datatables/datasets, yet. Note: This is VS2015 RC.
Currently EF 7 or EF Core does not support the old method of importing Stored procedures in designer and calling them directly. You can have a look at the roadmap to see what is going to be supported in the future:
EF core roadmap.
So for now it is better to use SqlConnection to call stored procedures or any raw query, since you do not need the entire EF for this job. Here are two examples:
Call stored procedure that return single value. String in this case.
CREATE PROCEDURE [dbo].[Test]
#UserName nvarchar(50)
AS
BEGIN
SELECT 'Name is: '+#UserName;
END
Call stored procedure that return a list.
CREATE PROCEDURE [dbo].[TestList]
AS
BEGIN
SELECT [UserName], [Id] FROM [dbo].[AspNetUsers]
END
To call these stored procedure it is better to create static class that holds all of these functions, for example, I called it DataAccess class, as follows:
public static class DataAccess
{
private static string connectionString = ""; //Your connection string
public static string Test(String userName)
{
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
// 1. create a command object identifying the stored procedure
SqlCommand cmd = new SqlCommand("dbo.Test", conn);
// 2. set the command object so it knows to execute a stored procedure
cmd.CommandType = CommandType.StoredProcedure;
// 3. add parameter to command, which will be passed to the stored procedure
cmd.Parameters.Add(new SqlParameter("#UserName", userName));
// execute the command
using (var rdr = cmd.ExecuteReader())
{
if (rdr.Read())
{
return rdr[0].ToString();
}
else
{
return null;
}
}
}
}
public static IList<Users> TestList()
{
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
// 1. create a command object identifying the stored procedure
SqlCommand cmd = new SqlCommand("dbo.TestList", conn);
// 2. set the command object so it knows to execute a stored procedure
cmd.CommandType = CommandType.StoredProcedure;
// execute the command
using (var rdr = cmd.ExecuteReader())
{
IList<Users> result = new List<Users>();
//3. Loop through rows
while (rdr.Read())
{
//Get each column
result.Add(new Users() { UserName = (string)rdr.GetString(0), Id = rdr.GetString(1) });
}
return result;
}
}
}
}
And Users class is like this:
public class Users
{
public string UserName { set; get; }
public string Id { set; get; }
}
By the way you do not need to worry about the performance of opening and closing a connection for every request to sql as the asp.net is taking care of managing these for you.
And I hope this was helpful.
I had a lot of trouble with the ExecuteSqlCommand and ExecuteSqlCommandAsync, IN parameters were easy, but OUT parameters were very difficult.
I had to revert to using DbCommand like so -
DbCommand cmd = _context.Database.GetDbConnection().CreateCommand();
cmd.CommandText = "dbo.sp_DoSomething";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#firstName", SqlDbType.VarChar) { Value = "Steve" });
cmd.Parameters.Add(new SqlParameter("#lastName", SqlDbType.VarChar) { Value = "Smith" });
cmd.Parameters.Add(new SqlParameter("#id", SqlDbType.BigInt) { Direction = ParameterDirection.Output });
I wrote more about it in this post.
Since it is agreed in my team that we are going to use generic UnitOfWork pattern, I have taken a bit of everyone's solutions when creating mine.
I am posting a bit of UnitOfWork code as well so you can get a grasp of why I needed to implement it like so.
public interface IUnitOfWork : IDisposable
{
DbContext Context { get; }
Task<List<T>> ExecuteStoredProc<T>(string storedProcName, Dictionary<string, object> procParams) where T : class;
}
Interface implementation:
public class UnitOfWork : IUnitOfWork
{
public DbContext Context { get; private set; }
/// <summary>
/// Execute procedure from database using it's name and params that is protected from the SQL injection attacks.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="storedProcName">Name of the procedure that should be executed.</param>
/// <param name="procParams">Dictionary of params that procedure takes. </param>
/// <returns>List of objects that are mapped in T type, returned by procedure.</returns>
public async Task<List<T>> ExecuteStoredProc<T>(string storedProcName, Dictionary<string, object> procParams) where T : class
{
DbConnection conn = Context.Database.GetDbConnection();
try
{
if(conn.State != ConnectionState.Open)
await conn.OpenAsync();
await using (DbCommand command = conn.CreateCommand())
{
command.CommandText = storedProcName;
command.CommandType = CommandType.StoredProcedure;
foreach (KeyValuePair<string, object> procParam in procParams)
{
DbParameter param = command.CreateParameter();
param.ParameterName = procParam.Key;
param.Value = procParam.Value;
command.Parameters.Add(param);
}
DbDataReader reader = await command.ExecuteReaderAsync();
List<T> objList = new List<T>();
IEnumerable<PropertyInfo> props = typeof(T).GetRuntimeProperties();
Dictionary<string, DbColumn> colMapping = reader.GetColumnSchema()
.Where(x => props.Any(y => y.Name.ToLower() == x.ColumnName.ToLower()))
.ToDictionary(key => key.ColumnName.ToLower());
if (reader.HasRows)
{
while (await reader.ReadAsync())
{
T obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in props)
{
object val =
reader.GetValue(colMapping[prop.Name.ToLower()].ColumnOrdinal.Value);
prop.SetValue(obj, val == DBNull.Value ? null : val);
}
objList.Add(obj);
}
}
reader.Dispose();
return objList;
}
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine(e.Message, e.InnerException);
}
finally
{
conn.Close();
}
return null; // default state
}
Example usage is like so:
public class MyService : IMyService
{
private readonly IUnitOfWork _uow;
public MyService(IUnitOfWork uow)
{
_uow = uow;
}
public async Task<List<TreeViewModel>> GetTreeOptions()
{
var procParams = new Dictionary<string, object>()
{
{"#Id", 2}
};
var result = await _uow.ExecuteStoredProc<TreeViewModel>("FetchTreeProcedure", procParams);
return result;
}
}
I found this extention very usefull: StoredProcedureEFCore
Then the usage is like this
List<Model> rows = null;
ctx.LoadStoredProc("dbo.ListAll")
.AddParam("limit", 300L)
.AddParam("limitOut", out IOutParam<long> limitOut)
.Exec(r => rows = r.ToList<Model>());
long limitOutValue = limitOut.Value;
ctx.LoadStoredProc("dbo.ReturnBoolean")
.AddParam("boolean_to_return", true)
.ReturnValue(out IOutParam<bool> retParam)
.ExecNonQuery();
bool b = retParam.Value;
ctx.LoadStoredProc("dbo.ListAll")
.AddParam("limit", 1L)
.ExecScalar(out long l);
Using MySQL connector and Entity Framework Core 2.0
My issue was that I was getting an exception like fx. Ex.Message = "The required column 'body' was not present in the results of a 'FromSql' operation.". So, in order to fetch rows via a stored procedure in this manner, you must return all columns for that entity type which the DBSet is associated with, even if you don't need to access all of it for your current request.
var result = _context.DBSetName.FromSql($"call storedProcedureName()").ToList();
OR with parameters
var result = _context.DBSetName.FromSql($"call storedProcedureName({optionalParam1})").ToList();
I'm using Entity Framework Core with my ASP.Net Core 3.x WebAPI. I wanted one of my end points just to execute a particular Stored Procedure, and this is the code I needed:
namespace MikesBank.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class ResetController : ControllerBase
{
private readonly MikesBankContext _context;
public ResetController(MikesBankContext context)
{
_context = context;
}
[HttpGet]
public async Task<ActionResult> Get()
{
try
{
using (DbConnection conn = _context.Database.GetDbConnection())
{
if (conn.State != System.Data.ConnectionState.Open)
conn.Open();
var cmd = conn.CreateCommand();
cmd.CommandText = "Reset_Data";
await cmd.ExecuteNonQueryAsync();
}
return new OkObjectResult(1);
}
catch (Exception ex)
{
return new BadRequestObjectResult(ex.Message);
}
}
}
}
Notice how I need to get my DbContext which has been injected, but I also need to Open() this connection.
Create the special class according the fields in your Select query of your stored procedure.
For example I will call this class ResulData
Add to context of you EF
modelBuilder.Entity<ResultData>(e =>
{
e.HasNoKey();
});
And this a sample function to get data using the store procedure
public async Task<IEnumerable<ResultData>> GetDetailsData(int id, string name)
{
var pId = new SqlParameter("#Id", id);
var pName = new SqlParameter("#Name", name);
return await _context.Set<ResultData>()
.FromSqlRaw("Execute sp_GetDeailsData #Id #Name", parameters: new[] { pId, pName })
.ToArrayAsync();
}
I used StoredProcedureEFCore nuget package by https://github.com/verdie-g/StoredProcedureEFCore,EnterpriseLibrary.Data.NetCore,EFCor.SqlServer,EFCore.Tools
I tried DbFirst approach with {Repository pattern}.. i think so
startup.cs
ConfigureServices(IServiceCollection services){
services.AddDbContext<AppDbContext>(opt => opt
.UseSqlServer(Configuration.GetConnectionString("SampleConnectionString")));
services.AddScoped<ISomeDAL, SomeDAL>();
}
public class AppDbContext : DbContext{
public AppDbContext(DbContextOptions<AppDbContext> options) : base(options)
{}
}
ISomeDAl Interface has {GetPropertiesResponse GetAllPropertiesByCity(int CityId);}
public class SomeDAL : ISomeDAL
{
private readonly AppDbContext context;
public SomeDAL(AppDbContext context)
{
this.context = context;
}
public GetPropertiesResponse GetAllPropertiesByCity(int CityId)
{
//Create Required Objects for response
//wont support ref Objects through params
context.LoadStoredProc(SQL_STATEMENT)
.AddParam("CityID", CityId).Exec( r =>
{
while (r.Read())
{
ORMapping<GenericRespStatus> orm = new ORMapping<GenericRespStatus>();
orm.AssignObject(r, _Status);
}
if (r.NextResult())
{
while (r.Read())
{
Property = new Property();
ORMapping<Property> orm = new ORMapping<Property>();
orm.AssignObject(r, Property);
_propertyDetailsResult.Add(Property);
}
}
});
return new GetPropertiesResponse{Status=_Status,PropertyDetails=_propertyDetailsResult};
}
}
public class GetPropertiesResponse
{
public GenericRespStatus Status;
public List<Property> PropertyDetails;
public GetPropertiesResponse()
{
PropertyDetails = new List<Property>();
}
}
public class GenericRespStatus
{
public int ResCode { get; set; }
public string ResMsg { get; set; }
}
internal class ORMapping<T>
{
public void AssignObject(IDataReader record, T myClass)
{
PropertyInfo[] propertyInfos = typeof(T).GetProperties();
for (int i = 0; i < record.FieldCount; i++)
{
if (propertyInfos.Any(obj => obj.Name == record.GetName(i))) //&& record.GetValue(i) != DBNull.Value
{
propertyInfos.Single(obj => obj.Name == record.GetName(i)).SetValue(myClass, Convert.ChangeType(record.GetValue(i), record.GetFieldType(i)));
}
}
}
}
If you are executing a stored procedure from Informix using EntityFrameworkCore you need to include the command EXECUTE PROCEDURE
var spresult = _informixContext.procdata.FromSql("EXECUTE PROCEDURE dummyproc ({0},{1},{2})", parameters: new[] { p0, p1,p2 }).ToList();
Nothing have to do... when you are creating dbcontext for code first approach initialize
namespace below the fluent API area make list of sp and use it another place where you want.
public partial class JobScheduleSmsEntities : DbContext
{
public JobScheduleSmsEntities()
: base("name=JobScheduleSmsEntities")
{
Database.SetInitializer<JobScheduleSmsEntities>(new CreateDatabaseIfNotExists<JobScheduleSmsEntities>());
}
public virtual DbSet<Customer> Customers { get; set; }
public virtual DbSet<ReachargeDetail> ReachargeDetails { get; set; }
public virtual DbSet<RoleMaster> RoleMasters { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
//modelBuilder.Types().Configure(t => t.MapToStoredProcedures());
//modelBuilder.Entity<RoleMaster>()
// .HasMany(e => e.Customers)
// .WithRequired(e => e.RoleMaster)
// .HasForeignKey(e => e.RoleID)
// .WillCascadeOnDelete(false);
}
public virtual List<Sp_CustomerDetails02> Sp_CustomerDetails()
{
//return ((IObjectContextAdapter)this).ObjectContext.ExecuteFunction<Sp_CustomerDetails02>("Sp_CustomerDetails");
// this.Database.SqlQuery<Sp_CustomerDetails02>("Sp_CustomerDetails");
using (JobScheduleSmsEntities db = new JobScheduleSmsEntities())
{
return db.Database.SqlQuery<Sp_CustomerDetails02>("Sp_CustomerDetails").ToList();
}
}
}
}
public partial class Sp_CustomerDetails02
{
public long? ID { get; set; }
public string Name { get; set; }
public string CustomerID { get; set; }
public long? CustID { get; set; }
public long? Customer_ID { get; set; }
public decimal? Amount { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
public int? CountDay { get; set; }
public int? EndDateCountDay { get; set; }
public DateTime? RenewDate { get; set; }
public bool? IsSMS { get; set; }
public bool? IsActive { get; set; }
public string Contact { get; set; }
}
I'm using Entity Framework Core. The support for stored procedures and adhoc queries doesn't feel as fluid as it does in Framework.
Here are some examples for future reference:
Populate a list of entities from the results of a stored procedure:
The [dbo].[GetUnarchivedJobs] stored procedure returns a list of records that matches the Job entity.
We can use the FromSqlInterpolated() method on the Jobs property to call the stored procedure and have a list of Job returned.
NoTracking() is used to speed up the performance, and in this case we wont be updating the entities.
using Microsoft.EntityFrameworkCore;
public class DbContext : Microsoft.EntityFrameworkCore.DbContext
{
protected DbSet<Job> Jobs { get; set; }
// Populate a list of entities from the results of a stored procedure
public Task<List<Job>> GetUnarchivedJobs(int maxQty, CancellationToken cancellationToken) =>
Jobs.FromSqlInterpolated($"EXEC [dbo].[GetUnarchivedJobs] #MaxQty = {maxQty}")
.AsNoTracking()
.ToListAsync(cancellationToken)
;
public DbContext(DbContextOptions<DbContext> options) : base(options) { }
}
Send an array of integers to a stored procedure:
The [dbo].[SetJobListArchiveFlags] stored procedure has a single parameter of type integer_list_tbltype.
We need to create a DataTable to match the integer_list_tbltype type which has a single column named n.
The int values need to be added to the DataTable.
A SqlParameter is used to pass the populated DataTable to the stored procedure.
Because we are not populating any entities, we need to use methods on the Database property to call the stored procedure.
using Microsoft.Data.SqlClient;
using Microsoft.EntityFrameworkCore;
using System.Data;
public class DbContext : Microsoft.EntityFrameworkCore.DbContext
{
// Send an array of integers to a stored procedure
public async Task<int> MarkJobsAsArchived(IEnumerable<int> jobIds, CancellationToken cancellationToken)
{
// This DataTable matches the `integer_list_tbltype` db type
var table = new DataTable();
table.Columns.Add("n", typeof(int));
foreach (var id in jobIds) table.Rows.Add(id);
var parameter = new SqlParameter("#jobIds", SqlDbType.Structured);
parameter.Value = table;
parameter.TypeName = "integer_list_tbltype";
var rowsUpdatedCount = await Database.ExecuteSqlRawAsync("EXEC [dbo].[SetJobListArchiveFlags] #jobIds", new[] { parameter }, cancellationToken);
return rowsUpdatedCount;
}
}
From EF core 3.0 onwards, we can also use the following commands:
dbContext.Database.ExecuteSqlInterpolated():
var name = "Ramin";
var outArticleCount = new SqlParameter(
"#outArticleCount",
SqlDbType.Int) { Direction = ParameterDirection.Output };
dbContext.Database.ExecuteSqlInterpolated(
$"exec dbo.MySp {name}, {outputArticleCount} out");
dbContext.Database.ExecuteSqlInterpolatedAsync():
dbContext.Database.ExecuteSqlRaw():
SqlParameter[] parameters =
{
new SqlParameter("#name", SqlDbType.Varchar { Direction = ParameterDirection.Input, Value = "Ramin" },
new SqlParameter("#outArticleCount", SqlDbType.Int { Direction = ParameterDirection.Output }
}
dbContext.Database.ExecuteSqlRaw("exec dbo.MySp #name, #outArticleCount OUTPUT", parameters);
dbContext.Database.ExecuteSqlRawAsync()
Raw and Interpolated versions of above methods are equal but Interpolated versions are safe against SQL Injection attacks (read more here and here).
It's embarrassing how lacking the docs are on this. 😩
It took me a lot of hit and trial to figure this out 😪, so I wanted to document 📝 it here so someone else doesn't have to spend time figuring this out. I'm using Entity Framework Core Version 6.0.10.
Summary
There are 2 kinds of Stored Procedures that return result. And the approach to call them and get result is different from each other.
SPs that return table-based output.
SPs that return output in the output parameters.
SPs that return table based output
If your SP looks like this:
CREATE PROCEDURE dbo.MyTestSPToGetShopType #ShopId AS VARCHAR(25)
AS
SELECT ShopId, ShopType
FROM Shop
WHERE SHOPID = #ShopId
👇
Step 1:
Create a class based on your output
public class MyTestSpResult
{
public string ShopId { get; set; }
public string ShopType { get; set; }
}
Step 2:
Register this in your DbContext's protected override void OnModelCreating(ModelBuilder builder) method:
builder.Entity<MyTestSpResult>(e => e.HasNoKey());
Step 3:
Call the SP from the code, and get the result:
var myTestSpResult = await _dbContext.Set<MyTestSpResult>()
.FromSqlInterpolated($"EXEC dbo.MyTestSPToGetShopType #ShopId = \"A03\"")
.ToArrayAsync();
var myShopType = myTestSpResult[0].ShopType;
Alternate Way
Steps 1 and 2 are same as above.
Step 3:
Register the class from Step 1 in your DbContext:
public virtual DbSet<MyTestSpResult> MyTestSpResult { get; set; }
Step 4:
Call the SP from the code, and get the result:
var mySPResult = await _dbContext.MyTestSpResult
.FromSqlInterpolated($"EXEC dbo.MyTestSPToGetShopType #ShopId = \"A03\"")
.ToArrayAsync();
var myShopType = mySPResult[0].ShopType;
SPs that return output in the output parameters
If your SP looks like this:
CREATE PROCEDURE dbo.MyOtherTestSPToGetShopType
(
#ShopId VARCHAR(25)
,#ShopType VARCHAR(25) OUTPUT
)
AS
SELECT #ShopType = ShopType
FROM Shop
WHERE SHOPID = #ShopId
👇
Step 1:
Create the SQL parameters:
var shopId = new SqlParameter()
{
ParameterName = "#ShopId",
Value = "A03",
Direction = System.Data.ParameterDirection.Input,
SqlDbType = System.Data.SqlDbType.VarChar,
Size = 25
};
var shopType = new SqlParameter()
{
ParameterName = "#ShopType",
Direction = System.Data.ParameterDirection.Output,
SqlDbType = System.Data.SqlDbType.VarChar,
Size = 25
};
Step 2:
Call the SP from code using those parameters, and get the result:
await _dbContext.Database
.ExecuteSqlInterpolatedAsync($"EXEC dbo.MyOtherTestSPToGetShopType #ShopId = {shopId}, #ShopType = {shopType} OUT");
var myShopType = shopType.Value as string;
We should create a property with DbQuery not DbSet in the model for the db context like below...
public class MyContextContext : DbContext
{
public virtual DbQuery<CheckoutInvoiceModel> CheckoutInvoice { get; set; }
}
After than a method that can be used to return result
public async Task<IEnumerable<CheckoutInvoiceModel>> GetLabReceiptByReceiptNo(string labReceiptNo)
{
var listing = new List<CheckoutInvoiceModel>();
try
{
var sqlCommand = $#"[dbo].[Checkout_GetLabReceiptByReceiptNo] {labReceiptNo}";
listing = await db.Set<CheckoutInvoiceModel>().FromSqlRaw(sqlCommand).ToListAsync();
}
catch (Exception ex)
{
return null;
}
return listing;
}
From above example, we can use any one option you like.
Hope this helpful for you!

Entity Framework Core - ExecuteSqlCommand: MAX function returning wrong value

I'm trying to do a simple thing: to get the MAX of a column. Is quite simple, you know, just run SELECT MAX(column) FROM table;
The problem is when I try to do this in my .NET Core 2.1 project, using Entity Framework.
I have a function that should return the next value of a column.
private int getColumNextValue(string table, string column)
{
string query = $"SELECT MAX({column}) + 1 FROM {table};";
return base.context.Database.ExecuteSqlCommand(query);
}
The query is being generated correclty:
However, it's returning -1 instead of the real value.
But when I run the exact same query in Sql Server Management Studio, the result is correct:
What's going on?
For ExecuteSqlCommand, it only return the number of rows affected. It would not run the query and return the result.
For a workaround, you could try like:
public class ApplicationDbContext : IdentityDbContext
{
public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
: base(options)
{
}
public DbSet<User> User { get; set; }
public async Task<T> ExecuteScalarAsync<T>(string rawSql, params object[] parameters)
{
var conn = Database.GetDbConnection();
using (var command = conn.CreateCommand())
{
command.CommandText = rawSql;
if (parameters != null)
foreach (var p in parameters)
command.Parameters.Add(p);
await conn.OpenAsync();
return (T)await command.ExecuteScalarAsync();
}
}
}
And use ExecuteScalarAsync like
public async Task<IActionResult> Index()
{
string query = $"SELECT MAX(SEQ) + 1 FROM [User];";
var result = await _context.ExecuteScalarAsync<int>(query);
return View();
}
I used QueryFirstOrDefault<int> to solve my problem.
I wrote this helper method for reusable purpose:
private int GetColumNextValue(string table, string column)
{
using (SqlConnection conn = new SqlConnection(this.configuration.GetConnectionString("MyConnection")))
{
string query = $"SELECT MAX({column}) + 1 FROM {table};";
return conn.QueryFirstOrDefault<int>(query);
}
}
I hope it can help other people.

Entity Framework 6 Disable Interception temporarily

I am using an IDbCommandTreeInterceptor to enable soft deletes on my model.
System.Data.Entity.Infrastructure.Interception.DbInterception.Add(
new SoftDeleteInterception());
I want to be able to disable the interceptor temporarily so that I can select a "deleted" entity for auditing purposes.
However, It seems like the DbInterception collection is assembly-wide.
Is there any way to create a new DbContext without interception on?
Or even a way to add the interceptor to the DbContext every time it is created?
I have extended my db context class with additional property
[DbConfigurationType(typeof(DbConfig))]
public partial class YourEntitiesDB
{
public bool IgnoreSoftDelete { get; set; }
}
Then in the TreeCreated(...) method i check this flag and if true then it just doesn't go further to the QueryVisitor
public class SoftDeleteInterceptor : IDbCommandTreeInterceptor
{
public SoftDeleteInterceptor()
{
}
public void TreeCreated(DbCommandTreeInterceptionContext interceptionContext)
{
var db = interceptionContext.DbContexts.FirstOrDefault() as YourEntitiesDB;
if (db!=null && db.IgnoreSoftDelete)
{
// Ignore soft delete interseptor (Used in archives)
return;
}
if (interceptionContext.OriginalResult.DataSpace == DataSpace.CSpace)
{
var queryCommand = interceptionContext.Result as DbQueryCommandTree;
if (queryCommand != null)
{
var newQuery = queryCommand.Query.Accept(new SoftDeleteQueryVisitor());
interceptionContext.Result = new DbQueryCommandTree(
queryCommand.MetadataWorkspace,
queryCommand.DataSpace,
newQuery);
}
var deleteCommand = interceptionContext.OriginalResult as DbDeleteCommandTree;
if (deleteCommand != null)
{
var column = SoftDeleteAttribute.GetSoftDeleteColumnName(deleteCommand.Target.VariableType.EdmType);
if (column != null)
{
var setClauses = new List<DbModificationClause>();
var table = (EntityType)deleteCommand.Target.VariableType.EdmType;
if (table.Properties.Any(p => p.Name == column))
{
setClauses.Add(DbExpressionBuilder.SetClause(
DbExpressionBuilder.Property(
DbExpressionBuilder.Variable(deleteCommand.Target.VariableType, deleteCommand.Target.VariableName),
column),
DbExpression.FromBoolean(true)));
}
var update = new DbUpdateCommandTree(
deleteCommand.MetadataWorkspace,
deleteCommand.DataSpace,
deleteCommand.Target,
deleteCommand.Predicate,
setClauses.AsReadOnly(),
null);
interceptionContext.Result = update;
}
}
}
}
}
In order to use it i just set the flag to true when needed
YuorEntitiesDB DB = new YuorEntitiesDB();
DB.IgnoreSoftDelete = true;
DB.Records.Where(...)

Is there a utility to dump an existing log4j log file into a relational database?

Seems, like a very basic thing, but I could not find it.
I have a bunch of log4j/log4net log files. I would like to dump them into a database in order to be able to analyze them with ease.
I thought I would find a tool to do it in no time, apparently I am wrong.
Does anyone know of such a tool?
OK, so I found no utility. Had to write my own. Of course, it is strictly tailored to my immediate needs (time is money), however, it can save you a bit of time to start your own, in case of a need. Here is the complete code in C#:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Text.RegularExpressions;
namespace ConsoleApplication3
{
class Program
{
public class LogEntry
{
private const string PATTERN = #"^(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d\.\d{4}) (\S+) \[(\d+)\] (\w+) (\S+) - (.*)$";
private static readonly Regex s_regex = new Regex(PATTERN, RegexOptions.Compiled);
public DateTime TS;
public string Machine;
public int Thread;
public string Level;
public string Logger;
public string Message;
public static LogEntry TryCreate(string line)
{
var match = s_regex.Match(line);
return match.Success ? new LogEntry
{
TS = DateTime.ParseExact(match.Groups[1].Value, "yyyy-MM-dd HH:mm:ss.ffff", CultureInfo.InvariantCulture),
Machine = match.Groups[2].Value,
Thread = int.Parse(match.Groups[3].Value),
Level = match.Groups[4].Value,
Logger = match.Groups[5].Value,
Message = match.Groups[6].Value,
} : null;
}
public void AppendToMessage(string line)
{
Message += Environment.NewLine + line;
}
}
static void Main()
{
const string SQL = #"
INSERT INTO log ( ts, machine, thread, level, logger, message, journalId)
VALUES (#ts, #machine, #thread, #level, #logger, #message, #journalId)
";
using (var connection = new SqlConnection("server=localhost;database=misc;uid=SantaClaus;pwd=MerryChristmas"))
{
connection.Open();
using (var command = new SqlCommand(SQL, connection))
{
var tsParam = new SqlParameter("#ts", SqlDbType.DateTime);
var machineParam = new SqlParameter("#machine", SqlDbType.NVarChar, 32);
var threadParam = new SqlParameter("#thread", SqlDbType.Int);
var levelParam = new SqlParameter("#level", SqlDbType.NVarChar, 10);
var loggerParam = new SqlParameter("#logger", SqlDbType.NVarChar, 128);
var messageParam = new SqlParameter("#message", SqlDbType.NVarChar, -1);
var journalIdParam = new SqlParameter("#journalId", SqlDbType.Int);
command.Parameters.Add(tsParam);
command.Parameters.Add(machineParam);
command.Parameters.Add(threadParam);
command.Parameters.Add(levelParam);
command.Parameters.Add(loggerParam);
command.Parameters.Add(messageParam);
command.Parameters.Add(journalIdParam);
// Call Prepare after setting the Commandtext and Parameters.
command.Prepare();
int i = 0;
foreach (var file in Directory.GetFiles(#"c:\tmp\dfbje01"))
{
journalIdParam.Value = OpenJournal(connection, file);
command.Transaction = connection.BeginTransaction();
foreach (var e in GetLogEntries(file))
{
tsParam.Value = e.TS;
machineParam.Value = e.Machine;
threadParam.Value = e.Thread;
levelParam.Value = e.Level;
loggerParam.Value = e.Logger;
messageParam.Value = e.Message;
command.ExecuteNonQuery();
++i;
if (i == 1000)
{
i = 0;
command.Transaction.Commit();
command.Transaction = connection.BeginTransaction();
}
}
command.Transaction.Commit();
CloseJournal(connection, journalIdParam.Value);
}
}
}
}
private static void CloseJournal(SqlConnection connection, object id)
{
const string SQL = "UPDATE journal SET done = 1 WHERE id = #id";
using (var command = new SqlCommand(SQL, connection))
{
command.Parameters.Add(new SqlParameter("#id", id));
command.ExecuteNonQuery();
}
}
private static object OpenJournal(SqlConnection connection, string filePath)
{
const string SQL = "INSERT INTO journal (filePath) OUTPUT inserted.id VALUES (#filePath)";
using (var command = new SqlCommand(SQL, connection))
{
command.Parameters.Add(new SqlParameter("#filePath", filePath));
return command.ExecuteScalar();
}
}
private static IEnumerable<LogEntry> GetLogEntries(string filePath)
{
LogEntry prev = null;
foreach (var line in File.ReadLines(filePath))
{
var logEntry = LogEntry.TryCreate(line);
if (logEntry != null)
{
if (prev != null)
{
yield return prev;
}
prev = logEntry;
}
else if (prev != null)
{
prev.AppendToMessage(line);
}
else
{
// Oops
Console.WriteLine(line);
}
}
if (prev != null)
{
yield return prev;
}
}
}
}
Mind trying out the filtering, search, colorizing features of the latest developer snapshot of Chainsaw? It has a good number of features which may avoid the need to use a DB. If you use a VFSLogFilePatternReceiver, it can parse and tail any regular text file, including those created by log4net.
Latest developer snapshot of Chainsaw is available here:
http://people.apache.org/~sdeboy

Resources