In the earlier versions of Entity Framework, we were able to reach the Context out of ObjectQuery in order to read Parameters, Connection, etc. as below:
var query = (ObjectQuery<T>)source;
cmd.Connection = (SqlConnection)((EntityConnection)query.Context.Connection).StoreConnection;
cmd.Parameters.AddRange(
query.Parameters.Select(x => new SqlParameter(
x.Name, x.Value ?? DBNull.Value)
).ToArray()
);
When I look at the DbSet<T> object, I am unable to find any equivalent of this. My purpose here is to create extensions which will manipulate the query and get the result out of it.
Here is an instance: http://philsversion.com/2011/09/07/async-entity-framework-queries
Or should I write the extension for DbContext class and work with Set method?
Any idea?
Edit
Here is what I did so far. Basic implementation so far but certainly not ready for production. Any suggestions on this?
public static async Task<IEnumerable<T>> QueryAsync<T>(this DbContext #this, System.Linq.Expressions.Expression<Func<T, bool>> predicate = null)
where T : class {
var query = (predicate != null) ? #this.Set<T>().Where(predicate) : #this.Set<T>();
var cmd = new SqlCommand();
cmd.Connection = (SqlConnection)(#this.Database.Connection);
cmd.CommandText = query.ToString();
if (cmd.Connection.State == System.Data.ConnectionState.Closed) {
cmd.Connection.ConnectionString = new SqlConnectionStringBuilder(cmd.Connection.ConnectionString) {
AsynchronousProcessing = true
}.ToString();
cmd.Connection.Open();
}
cmd.Disposed += (o, e) => {
cmd.Clone();
};
var source = ((IObjectContextAdapter)#this).ObjectContext.Translate<T>(
await cmd.ExecuteReaderAsync()
);
return source;
}
This is a nice workaround, although I don't think you can make it much more generally applicable than what you already have.
A few things to keep in mind:
- Depending on the EF query, e.g. if you are using Include or not, the columns returned in the reader might not match the properties in the type T you are passsing.
- Depending on whether you have inheritance in your model, the T that you pass to translate may not always be the right thing to materialize for every row returned.
- After the task returned by ExecuteReaderAsync completes, you still have to retrieve each row, which depending on the execution plan for the query and the latency you are getting with the server is potentially also a blocking operation.
Async support is not coming to EF in 5.0 but we worked with other teams to make sure we have all the necessary building blocks included in .NET 4.5 and the feature is pretty high in our priority list. I encourage you to vote for it in our UserVoice site.
Related
I'm writing a new ASP.NET Core Web API, and one of my requirements is to be able to leverage EF Core 3.1 to grab the next value of a sequence defined in my SQL Server as the ID for a record I need to store.
I'm struggling to find a way to do this - in EF 6.x, I used a method directly on the DbContext descendant like this:
public int GetNextSequenceValue()
{
var rawQuery = Database.SqlQuery<int>("SELECT NEXT VALUE FOR dbo.TestSequence;");
var task = rawQuery.SingleAsync();
int nextVal = task.Result;
return nextVal;
}
and for EF Core up to 2.1, I would have been able to use Database.ExecuteSqlCommand() to run a SQL snippet and get back results. But it seems, in EF Core 3.x, I'm out of luck....
I know there are the .FromSqlRaw() and .FromSqlInterpolated methods on the DbSet - but since I only need to return the next value of a sequence (an INT), that's not going to fly. And I also know these methods also exist on the context.Database level which looks like it would be really close to what I had in EF 6.x - but here, those methods will only return the number of rows affected - I haven't found a way to send back the new value from the SEQUENCE.
Can it really be that in EF Core 3.x, I have to actually resort back to way-old ADO.NET code to fetch that value?? Is there REALLY no way to execute an arbitrary SQL snippet and get back some results from the context??
If you want to run an arbitrary TSQL batch and return a scalar value, you can do it like this:
var p = new SqlParameter("#result", System.Data.SqlDbType.Int);
p.Direction = System.Data.ParameterDirection.Output;
context.Database.ExecuteSqlRaw("set #result = next value for some_seq", p);
var nextVal = (int)p.Value;
Looks like executing raw SQL is not priority for EF Core, so up to now (EF Core 3.1) it's providing publicly just few basic limited methods. FromSql requires entity type or keyless entity type, and ExecuteSqlRaw / ExecuteSqlInterpolated are the "modern" bridge to ADO.NET ExecuteNonQuery which returns the affected rows.
The good thing is that EF Core is built on top of a public service architecture, so it can be used to add some missing functionalities. For instance, services can be used to build the so called IRelationalCommand, which has all the DbCommand execute methods, in particular ExecuteScalar needed for SQL in question.
Since EF Core model supports sequences, there is also a service for building the IRelationalCommand needed to retrieve the next value (used internally by HiLo value generators).
With that being said, following is a sample implementation of the custom method in question using the aforementioned concepts:
using System;
using System.Globalization;
using Microsoft.EntityFrameworkCore.Diagnostics;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Storage;
using Microsoft.EntityFrameworkCore.Update;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomExtensions
{
public static long GetNextSequenceValue(this DbContext context, string name, string schema = null)
{
var sqlGenerator = context.GetService<IUpdateSqlGenerator>();
var sql = sqlGenerator.GenerateNextSequenceValueOperation(name, schema ?? context.Model.GetDefaultSchema());
var rawCommandBuilder = context.GetService<IRawSqlCommandBuilder>();
var command = rawCommandBuilder.Build(sql);
var connection = context.GetService<IRelationalConnection>();
var logger = context.GetService<IDiagnosticsLogger<DbLoggerCategory.Database.Command>>();
var parameters = new RelationalCommandParameterObject(connection, null, null, context, logger);
var result = command.ExecuteScalar(parameters);
return Convert.ToInt64(result, CultureInfo.InvariantCulture);
}
}
}
In your fluent api configs you can create migration that set ID automatically to be next value from Sequence
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasSequence<int>("OrderNumbers");
modelBuilder.Entity<Order>()
.Property(o => o.OrderNo)
.HasDefaultValueSql("NEXT VALUE FOR shared.OrderNumbers");
}
For creating sequence:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasSequence<int>("OrderNumbers", schema: "shared")
.StartsAt(1000)
.IncrementsBy(5);
}
Read more from here: https://www.talkingdotnet.com/use-sql-server-sequence-in-entity-framework-core-primary-key/
For people suffering with oracle version of this problem, here's a solution:
var p = new OracleParameter("result", OracleDbType.Decimal, null, System.Data.ParameterDirection.Output);
Database.ExecuteSqlRaw($"BEGIN :result := my_seq.nextval; END;", p);
var nextVal = p.Value;
Ugly, but the best thing I could come up with:
var connection = repcontext.Database.GetDbConnection();
connection.Open();
using var cmd = connection.CreateCommand();
cmd.CommandText = "SELECT NEXT VALUE FOR AA.TransSeq;";
var obj = cmd.ExecuteScalar();
connection.Close();
seqnum = (int)obj;
This code should work in a variety of situations:
public static class DbSequence
{
private const string sqlCode = "SELECT NEXT VALUE FOR {0}.{1};";
public static T GetNextSeq<T>(this DbContext dbContext, string seqName)
{
var sqlCnn = dbContext.Database.GetDbConnection();
bool cnnClosed = sqlCnn.State != ConnectionState.Open;
if (cnnClosed) sqlCnn.Open();
try
{
using (var sqlCmd = sqlCnn.CreateCommand())
{
sqlCmd.Transaction = dbContext.Database.CurrentTransaction?.GetDbTransaction();
sqlCmd.CommandText = string.Format(sqlCode, "dbo", seqName);
var result = sqlCmd.ExecuteScalar();
if ((result == null) || (result == DBNull.Value)) throw new InvalidOperationException();
return (T)result;
}
}
finally
{
if (cnnClosed) sqlCnn.Close();
}
}
}
This code works when the connection is closed, opening it when needed and closing it after itself. It should also work when a transaction has been initiated. According to this source: https://learn.microsoft.com/en-us/sql/t-sql/statements/create-sequence-transact-sql?view=sql-server-ver16#general-remarks sequences run outside of transactions. Still, if it's available, I set up a transaction for the command. I also use generics and extension methods.
I'm trying to parse query results returned from an Azure Elastic Scale MultiShardConnection. It doesn't look like it inherits from SqlConnection or DbConnection so the Dapper methods are not available. This make sense when you consider that it's executing a fan-out query that is union'ed together. What I was hoping to do was to use existing Dapper functionality to just handle the parser of the reader results to a type.
Are those mapping features available if I don't use Dapper for the original connection?
Below are the types I'm working with:
MultiShardConnection : IDisposable
MultiShardCommand : DbCommand
MultiShardDataReader : DbDataReader, IDataReader, IDisposable, IDataRecord
Here's an example query where I'm trying to use the Dapper mapper.
Customer customer = null;
using (MultiShardConnection conn = GetMultiShardConnection())
using (MultiShardCommand cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT * FROM [Customer] WHERE ...";
cmd.CommandTimeout = 120;
cmd.CommandTimeoutPerShard = 120;
using (MultiShardDataReader reader = await cmd.ExecuteReaderAsync())
{
while (reader.Read())
{
// Replace this with mapper...
customer = new Customer()
{
CustomerId = reader.GetInt32(0)
//etc...
};
}
}
}
return customer;
Update
I ended up needing to use sp_execute_fanout
using (var con = GetConnection())
{
await con.OpenAsync();
return (await con.QueryAsync<Customer>("sp_execute_fanout ", new
{
shard_map_manager_server = "my_server.database.windows.net",
shard_map_manager_database = "my_shardmapmananger",
user_id = "my_username",
password = "my_password",
shard_map_name = "my_shardmap",
statement = "SELECT * FROM Customer"
}, commandTimeout: 120, commandType: CommandType.StoredProcedure)).FirstOrDefault();
}
Currently, MultiShardConnection is not integrated with Dapper. The reason is exactly as you point out that it does not implement DbConnection. As an alternative solution, I would recommend to try elastic database query (EDQ): https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-query-overview/. With EDQ, you can simply connect to a single database in Azure DB and use regular Dapper over the EDQ external tables to query across your shards. EDQ is now available on all service tiers in Azure SQL DB.
Let us know how that works for you.
Thanks,
Torsten
I tried the solution from OP today, because I also wanted to use Dapper to map query results from multiple shards, but I noticed that sp_execute_fanout is deprecated and replaced by sp_execute_remote.
Before you can use this sp, you need to create an external source as a reference to the Shard Map Manager.
You can then use the name of this external source as the data source name ('TestExtScr' in my example) and do something like this in your code:
using (var con = new SqlConnection(connString))
{
return await con.QueryAsync<Customer>("sp_execute_remote", new
{
data_source_name = "TestExtSrc",
statement = "SELECT foo FROM bar"
}, commandTimeout: 120, commandType: CommandType.StoredProcedure);
}
This way you can use Dapper while querying multiple shards.
I know I'm bumping an old post, but when I was looking for a solution to my problem, this post kept showing up. So I added this bit in case someone in the future is looking for the same thing :) .
Is there any way to define the SQL conversion component for additional functions to Linq2Entities.
For example:
myQuery.Where(entity => entity.Contains('foo', SearchFlags.All))
Ideally I am looking for something that doesn't require editing and building a new version the EntityFramework.dll directly. Is there any way to allow extension methods to entity framework that can support SQL generation.
So far I have a template which would represent the method I need to replace for LINQ to Entities:
public static bool Contains(this object source, string searchTerms, SearchFlags flags)
{
return true;
}
Of course this causes the error:
LINQ to Entities does not recognize the method 'Boolean
CONTAINS(System.Object, System.String, SearchFlags)' method, and this method
cannot be translated into a store expression.
To be clear, I don't want to do:
myQuery.AsEnumerable().Where(entity => entity.Contains('foo', SearchFlags.All))
Because I want to be able to execute code in SQL space and not return all the entities manually.
I also cannot use the .ToString() of the IQueryable and execute it manually because I need Entity Framework to populate the objects from several .Include joins.
I don't understand your Q clearly. However if your problem is that you can't use your own methods or other linq to objects method, just use .AsEnumerable() and do your other jobs through linq to objects, not L2E:
myQuery.AsEnumerable().Where(entity => entity.Contains('foo', SearchFlags.All))
And if you need to use your myQuery several times somewhere else, first load it to memory, then use it as many as you want:
var myQuery = from e in context.myEntities
select d;
myQuery.Load();
// ...
var myOtherQuery = from d in context.myEntities.Local
select d;
// Now any L2O method is supported...
I ended up doing the following (which works but is very far from perfect):
All my entities inherit from an IEntity which defines long Id { get; set; }
I then added a redundant restriction
context.myEntities.Where(entity => entity.Id != 0) this is
redundant since the identity starts at 1, but Linq2Entities doesn't
know that.
I then call .ToString() on the IQueryable after I have done all
my other queries, since it is of type DBQuery<Entity> it returns
the SQL Command Text, I do a simple replace with my query restriction.
In order to get all the .Include(...) to work I actually execute
two different sql commands. There is no other more pretty way to tap into this because of query execution plan caching causes issues otherwise (even when disabled).
As a result my code looks like this:
public IQueryable<IEntity> MyNewFunction(IQueryable<IEntity> myQueryable, string queryRestriction)
{
string rawSQL = myQueryable.Select(entity => entity.Id).ToString().Replace("[Extent1].Id <> 0", queryRestriction);
List<long> ids = // now execute rawSQL, get the list of ids;
return myQuerable.Where(entity => ids.Contains(entity.Id));
}
In short, other than manually executing the SQL or running a similar SQL command and appending the restriction using the existing commands the only way to write your own methods to Linq-to-Entities is to manually alter and build your own EntityFramework.dll from the EF6 source.
I'm exploring query notifications with the SQLDependency class. Building a simple working example is easy, but I feel like I'm missing something. Once I step past a simple one-table/one-dependency example I'm left wondering how can I figure out which dependency triggered my callback?
I'm having a bit of trouble explaining, so I included the simple example below. When AChange() is called I cannot look at the sql inside the dependency, and i don't have a reference to the associated cache object.
So what's a boy to do?
Option 1 - create a distinct function for each object i want to track and hard code the cache-key (or relevant information) in the callback. This feels dirty & eliminates the posibility of adding new cache items without deploying new code--ewww.
Option 2 - Use the Dependency Id property and a parallel tracking structure
Am I just missing something? Is this a deficiency in the SQLDependency structure? I've I've looked at 20 different articles on the topic and all of them seem to have the same hole. Suggestions?
Code Sample
public class DependencyCache{
public static string cacheName = "Client1";
public static MemoryCache memCache = new MemoryCache(cacheName);
public DependencyCache() {
SqlDependency.Start(connString);
}
private static string GetSQL() {
return "select someString FROM dbo.TestTable";
}
public void DoTest() {
if (memCache["TEST_KEY"] != null ) {
Debug.WriteLine("resources found in cache");
return;
}
Cache_GetData();
}
private void Cache_GetData() {
SqlConnection oConn;
SqlCommand oCmd;
SqlDependency oDep;
SqlDataReader oRS;
List<string> stuff = new List<string>();
CacheItemPolicy policy = new CacheItemPolicy();
SqlDependency.Start(connString);
using (oConn = new SqlConnection(connString) ) {
using (oCmd = new SqlCommand(GetSQL(), oConn) ) {
oDep = new SqlDependency(oCmd);
oConn.Open();
oRS = oCmd.ExecuteReader();
while(oRS.Read() ) {
resources.Add( oRS.GetString(0) );
}
oDep.OnChange += new OnChangeEventHandler (AChange);
}
}
memCache.Set("TEST_KEY", stuff, policy);
}
private void AChange( object sender, SqlNotificationEventArgs e) {
string msg= "Dependency Change \nINFO: {0} : SOURCE {1} :TYPE: {2}";
Debug.WriteLine(String.Format(msg, e.Info, e.Source, e.Type));
// If multiple queries use this as a callback how can i figure
// out WHAT QUERY TRIGGERED the change?
// I can't figure out how to tell multiple dependency objects apart
((SqlDependency)sender).OnChange -= Cache_SqlDependency_OnChange;
Cache_GetData(); //reload data
}
}
First and foremost: the handler has to be set up before the command is executed:
oDep = new SqlDependency(oCmd);
oConn.Open();
oDep.OnChange += new OnChangeEventHandler (AChange);
oRS = oCmd.ExecuteReader();
while(oRS.Read() ) {
resources.Add( oRS.GetString(0) );
}
Otherwise you have a window when the notification may be lost and your callback never invoked.
Now about your question: you should use a separate callback for each query. While this may seem cumbersome, is actually trivial by using a lambda. Something like the following:
oDep = new SqlDependency(oCmd);
oConn.Open();
oDep.OnChange += (sender, e) =>
{
string msg = "Dependency Change \nINFO: {0} : SOURCE {1} :TYPE: {2}";
Debug.WriteLine(String.Format(msg, e.Info, e.Source, e.Type));
// The command that trigger the notification is captured in the context:
// is oCmd
//
// You can now call a handler passing in the relevant info:
//
Reload_Data(oCmd, ...);
};
oRS = oCmd.ExecuteReader();
...
And remember to always check the notification source, info and type. Otherwise you run the risk of spinning ad-nauseam when you are notified for reasons other than data change, like invalid query. As a side comment I would add that a good cache design does not refresh the cache on invalidation, but simply invalidates the cached item and lets the next request actually fetch a fresh item. With your 'proactive' approach you are refreshing cached items even when not needed, refresh multiple times before they are accessed etc etc. I left out from the example error handling and proper thread synchronization (both required).
Finally, have a look at LinqtoCache which does pretty much what you're trying to do, but for LINQ queries.
Relatively new to wcf and Silverlight. I had been banging my head against the wall trying to digest WCF and the contract concept. I found a web post that showed a way to make WCF work. As you'll see it doesn't follow the classical WCF form (no datacontracts).
Here is an example of the code. Note that the ImageData object is not part of a data contract but simply declared as a normal object in a class cs file with get and set properties.
Despite the fact that this isn't "regulation" it does work. The service reference to the service automatically moves over the ImageData class and a number of other pieces of information to make this work.
This worked fine. However I hit an error I've hit twice before where I'm told that some obscure assembly is not present to allow the service to be accessed. This has happened twice before. In all three cases I was able recreate the service and the service reference (but I had to change the names of both because it seemed they already existed in the project). The first two times this fixed my problem.
This time I used the same fix but now the code, instead of returning a list of ImageData as is indicated in the signature of the function is now returning a type called "GetImageDataReponse" and I no longer can iterate through the list of ImageData Objects that it used to return. In looking at the definition of "GetImageDataReponse" I find that it looks like another VS2010 generated function that has the list of ImageData embedded in it. It seems to have added another layer of abstraction over the return type (List).
I'm not sure what is causing the odd "missing assembly" errors which seem to occur randomly and cause me to rebuild the service. Now I'm not sure why I am getting these new return types from functions that have a clear return type specified and worked before.
Any help would be appreciated. I hope to be able to prevent having to rebuild the service and the service reference in the first place.
Below is the code that used to work and then the defination of "GetImageDataReponse" in the reference.cs file.
Thanks,
Fig000
[OperationContract]
public List<ImageData> GetImageData(int imageID)
{
List<ImageData> ImageDataList = new List<ImageData>();
SqlConnection con = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["CN_eLogWeb"].ConnectionString.ToString());
SqlDataReader dr = null;
string sql = #"SELECT isnull(i.ANI,'') as ANI, isnull(i.DonorCorpID,0) as DonorCorpID,isnull(i.DonorUnitID,0) as DonorUnitID,isnull(i.AgencyID,0) as AgencyID,
isnull(i.PickupDT,'1/1/1900') as PickupDT, isnull(u.InBoundSortedStatusID,1) as InboundSortedStatusID, isnull(u.FollowupRequired,0) as FollowupRequired
from InboundSorted i
left join UserSortSessionInfo u on i.IncomingID=u.IncomingID
left join Image img on i.ImageID=img.ImageID where img.ImageID=" + imageID;
con.Open();
SqlCommand cmd = new SqlCommand(sql, con);
dr = cmd.ExecuteReader();
if (dr.HasRows)
{
while (dr.Read())
{
ImageData oImageData = new ImageData
{
ANI = Convert.ToString(dr["ANI"]),
DonorCorpID = Convert.ToInt32(dr["DonorCorpID"]),
DonorUnitID = Convert.ToInt32(dr["DonorUnitID"]),
AgencyID = Convert.ToInt32(dr["AgencyID"]),
PickupDT = Convert.ToDateTime(dr["PickupDT"]),
Status = GetInboundSortedStatus(Convert.ToInt32(dr["InboundSortedStatusID"])),
FollowupRequired = Convert.ToBoolean(dr["FollowupRequired"])
};
ImageDataList.Add(oImageData);
}
}
con.Close();
return ImageDataList;
}
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "4.0.0.0")]
[System.ServiceModel.MessageContractAttribute(WrapperName="GetImageDataResponse", WrapperNamespace="", IsWrapped=true)]
public partial class GetImageDataResponse {
[System.ServiceModel.MessageBodyMemberAttribute(Namespace="", Order=0)]
public System.Collections.Generic.List<acSilverlightLib.ImgServiceRef2.ImageData> GetImageDataResult;
public GetImageDataResponse() {
}
public GetImageDataResponse(System.Collections.Generic.List<acSilverlightLib.ImgServiceRef2.ImageData> GetImageDataResult) {
this.GetImageDataResult = GetImageDataResult;
}
}
if (dr.HasRows)
{
while (dr.Read())
{
ImageData oImageData = new ImageData
{
ANI = Convert.ToString(dr["ANI"]),
DonorCorpID = Convert.ToInt32(dr["DonorCorpID"]),
DonorUnitID = Convert.ToInt32(dr["DonorUnitID"]),
AgencyID = Convert.ToInt32(dr["AgencyID"]),
PickupDT = Convert.ToDateTime(dr["PickupDT"]),
Status = GetInboundSortedStatus(Convert.ToInt32(dr["InboundSortedStatusID"])),
FollowupRequired = Convert.ToBoolean(dr["FollowupRequired"])
};
ImageDataList.Add(oImageData);
}
}
con.Close();
return ImageDataList;
}
Did you make any changes to ImageData between versions?
An issue that can cause this is missing a Default (parameter-less) constructor on ImageData, or one of its properties.