I have the following problem:
A several tables with "data", "token_data" columns that switch their values over time
Phases:
In the current phase 0, there is only "data" column (clear data).
In phase 1 there will be "data", "token_data" columns.
In the phase 2, there will be "token_data", "clear_data" columns.
In the last phase 3, there should be only "data" column (by that
time it should be tokenized).
We currently have all dapper/db models with phase 0 in mind.
Is there a way to prepare Dapper models for all 4 phases? I was looking for OptionalColumn attribute, but couldn't find one.
Ideally there would be a global config switch that would control which Dapper model property represents the tokenized "data" column.
Like:
// Not good
[Column("Name")]
public string Name
{
get { return AppSettings.TokenizationEnabled ? this.TokenName : _name; }
set { _name = value; }
}
It's not 100% clear what you need to do. For example, why you can'y just created a class with all properties, and, depending on the phase, return the correct data for that phase. Something like:
class MyData {
public int Phase;
public String Data { private get; public set; }
public String Token_Data { private get; public set; }
public String Clean_Data { private get; public set; }
public String GetData()
{
switch(Phase): {
case 1: return Token_Data; break;
case 2: return Clean_data; break;
default: return Data; break
}
}
Aside from that, anyway, I think the feature called "Type Switching per Row" can help you: https://github.com/StackExchange/Dapper#type-switching-per-row
Related
I have a system in which I generate the transaction ids based on a table. The number must have a specific format therefore using database sequences is not an option. Moreover, the number transactions types is variable which means creating variable number of sequences. The table has the following structure:
public class TransactionSequence : BaseEntity<int>, IAggregateRoot
{
public int Year { get; set; }
public string Prefix { get; set; }
public long Sequence { get; set; }
public string Service { get; set; }
public int Length { get; set; }
public bool IsCurrent { get; set; }
}
The code for the service which generates the number is as shown below:
public class NumberingService : INumberingService
{
public static int yearLength = 2;
public static int monthLength = 2;
public static int dayLength = 2;
private readonly IRepository<TransactionSequence> _repository;
private readonly NumberingConfiguration _numbering;
public NumberingService(IRepository<TransactionSequence> repository,
NumberingConfiguration numbering)
{
_repository = repository;
_numbering = numbering;
}
public async Task<Result<string>> GetNextNumberAsync(string service, int maxlength, UserEntity sysUser, string prefix = "")
{
string transactionId = string.Empty;
try
{
var spec = new CurrentYearNumberingSpec(service);
var sequence = await _repository.GetBySpecAsync(spec);
if (sequence == null)
{
await AddServiceNumberingAsync(service, maxlength, sysUser, prefix);
sequence = await _repository.GetBySpecAsync(spec);
}
sequence.Sequence = sequence.Sequence + 1;
await _repository.UpdateAsync(sequence);
int month = DateTime.Now.Month;
int day = DateTime.Now.Day;
var length = GetLength(sequence);
transactionId = sequence.Prefix + (sequence.Year % 100).ToString("D" + 2) + month.ToString("D" + 2) + day.ToString("D" + 2) + sequence.Sequence.ToString("D" + length);
}
catch (Exception ex)
{
return Result<string>.Error(ex.Message);
}
return Result<string>.Success(transactionId, "Retrieved the next number in the sequence succesfully!");
}
private static int GetLength(TransactionSequence sequence)
{
return sequence.Length - sequence.Prefix.Length - dayLength - monthLength - yearLength;
}
}
Note: I am only showing an excerpt of the code that contains the relative info!
The problem:
Since the system is highly concurrent, each request tries to obtain a transaction id when submitted. Thus, there is a high contention for TransactionSequence row which is currently active since it will be generating the transaction id via subsequent updates. This means that there will absolutely be locking.
Solutions I tried:
1- Optimistic Concurrency via ROWVERSION with retries, this had the worst performance since optimistic concurrency makes sense only if the collision possibility is rare! But since the collision is almost guaranteed this solution had the worst performance. Either that or I did not implement it correctly!
2- Locking via SemaphoreSlim, this had an acceptable performance but its problem was that it would not scale in load balanced scenarios.
3- Distributed Locking via Redis, this had an approximate performance to SemaphoreSlim but still not the performance I am looking for!
4- Queuing via RabbitMQ with prefetch size of 1, This had a better performance than the aforementioned solutions but still I wonder if there is an optimal solution!
5- Using HiLo algorithm, I did not implement this but I have read about it as in the link below:
CQS with Database-Generated Ids
I want to know if there is a better or a well-known solution to this problem
My Environment:
ASP .NET CORE 6
EF CORE 6
SQL SERVER 2019
I hope this was clear enough, and thanks in advance!
I've got a WCF service that passes around status updates via a struct like so:
[DataContract]
public struct StatusInfo
{
[DataMember] public int Total;
[DataMember] public string Authority;
}
...
public StatusInfo GetStatus() { ... }
I expose a property in a ViewModel like this:
public class ServiceViewModel : ViewModel
{
public StatusInfo CurrentStatus
{
get{ return _currentStatus; }
set
{
_currentStatus = value;
OnPropertyChanged( () => CurrentStatus );
}
}
}
And XAML like so:
<TextBox Text="{Binding CurrentStatus.Total}" />
When I run the app I see errors in the output window indicating that the Total property cannot be found. I checked and double checked and I typed it correctly. The it occurred to me that the errors specifically indicate that the 'property' cannot be found. So adding a property to the struct made it work just fine. But this seems odd to me that WPF can't handle one-way binding to fields. Syntactically you access them the same in code and it seem silly to have to create a custom view model just for the StatusInfo struct. Have I missed something about WPF binding? Can you bind to a field or is property binding the only way?
Binding generally doesn't work to fields. Most binding is based, in part, on the ComponentModel PropertyDescriptor model, which (by default) works on properties. This enables notifications, validation, etc (none of which works with fields).
For more reasons than I can go into, public fields are a bad idea. They should be properties, fact. Likewise, mutable structs are a very bad idea. Not least, it protects against unexpected data loss (commonly associated with mutable structs). This should be a class:
[DataContract]
public class StatusInfo
{
[DataMember] public int Total {get;set;}
[DataMember] public string Authority {get;set;}
}
It will now behave as you think it should. If you want it to be an immutable struct, that would be OK (but data-binding would be one-way only, of course):
[DataContract]
public struct StatusInfo
{
[DataMember] public int Total {get;private set;}
[DataMember] public string Authority {get;private set;}
public StatusInfo(int total, string authority) : this() {
Total = total;
Authority = authority;
}
}
However, I would first question why this is a struct in the first place. It is very rare to write a struct in .NET languages. Keep in mind that the WCF "mex" proxy layer will create it as a class at the consumer anyway (unless you use assembly sharing).
In answer to the "why use structs" reply ("unknown (google)"):
If that is a reply to my question, it is wrong in many ways. First, value types as variables are commonly allocated (first) on the stack. If they are pushed onto the heap (for example in an array/list) there isn't much difference in overhead from a class - a small bit of object header plus a reference. Structs should always be small. Something with multiple fields will be over-sized, and will either murder your stack or just cause slowness due to the blitting. Additionally, structs should be immutable - unlesss you really know what you are doing.
Pretty much anything that represents an object should be immuatable.
If you are hitting a database, the speed of struct vs class is a non-issue compared to going out-of-process and probably over the network. Even if it is a bit slower, that means nothing compared to the point of getting it right - i.e. treating objects as objects.
As some metrics over 1M objects:
struct/field: 50ms
class/property: 229ms
based on the following (the speed difference is in object allocation, not field vs property). So about 5x slower, but still very, very quick. Since this is not going to be your bottleneck, don't prematurely optimise this!
using System;
using System.Collections.Generic;
using System.Diagnostics;
struct MyStruct
{
public int Id;
public string Name;
public DateTime DateOfBirth;
public string Comment;
}
class MyClass
{
public int Id { get; set; }
public string Name { get; set; }
public DateTime DateOfBirth { get; set; }
public string Comment { get; set; }
}
static class Program
{
static void Main()
{
DateTime dob = DateTime.Today;
const int SIZE = 1000000;
Stopwatch watch = Stopwatch.StartNew();
List<MyStruct> s = new List<MyStruct>(SIZE);
for (int i = 0; i < SIZE; i++)
{
s.Add(new MyStruct { Comment = "abc", DateOfBirth = dob,
Id = 123, Name = "def" });
}
watch.Stop();
Console.WriteLine("struct/field: "
+ watch.ElapsedMilliseconds + "ms");
watch = Stopwatch.StartNew();
List<MyClass> c = new List<MyClass>(SIZE);
for (int i = 0; i < SIZE; i++)
{
c.Add(new MyClass { Comment = "abc", DateOfBirth = dob,
Id = 123, Name = "def" });
}
watch.Stop();
Console.WriteLine("class/property: "
+ watch.ElapsedMilliseconds + "ms");
Console.ReadLine();
}
}
I can only guess why they only support properties: perhaps because it seems to be a universal convention in the .NET framework never to expose mutable fields (probably to safeguard binary compatibility), and they somehow expected all programmers to follow the same convention.
Also, although fields and properties are accessed with the same syntax, data binding uses reflection, and (so I've heard) reflection must be used differently to access fields than to access properties.
I have the following Global class file:
Global.cs
public static class Global
{
private static string _globalVar = "";
public static string GlobalVar
{
get { return _globalVar; }
set { _globalVar = value; }
}
}
I set the new value of string GlobarVar in Form1.cs as '1234'.
Form1.cs
public Form1()
{
InitializeComponent();
Global.GlobalVar = "1234";
}
I tried to display the value to Form2.cs using the message box
public Form2()
{
InitializeComponent();
MessageBox.Show(Global.GlobalVar); // displays blank values
}
Am I missing something?
Four options:
You're not constructing Form1 before you construct Form2
Something else is setting Global.GlobalVar back to null or an empty string
Your forms are in different app domains, so they'll have entirely separate Global types
You're running the application twice; static variables don't live on across different processes
It's hard to tell which of these is the case, but personally I'd try to avoid using global state to start with. It's a pain for testability and reasoning about how your program works.
Try your property page (file Global.cs) like these:
public class Global
{
private static string _globalVar;
public string GlobalVar
{
get { return _globalVar; }
set { _globalVar = value; }
}
}
Whats the best way to save variables like userid that is stored and reachable from different pages in WP7.
There's the querystring method, but can be kind of a pain to implement.
When navigating, pass the parameter like a HTTP querystring.
Then, on the otherside, check if the key exists, and extract the value. The downside of this is if you need to do more than 1, you need to type it in yourself, and it only supports strings.
So to pass an integer, you'd need to convert it. (And to pass a complex object, you need to take all the pieces you need to recompile it on the other side)
NavigationService.Navigate(new Uri("/PanoramaPage1.xaml?selected=item2", UriKind.Relative));
protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
string selected = String.Empty;
//check to see if the selected parameter was passed.
if (NavigationContext.QueryString.ContainsKey("selected"))
{
//get the selected parameter off the query string from MainPage.
selected = NavigationContext.QueryString["selected"];
}
//did the querystring indicate we should go to item2 instead of item1?
if (selected == "item2")
{
//item2 is the second item, but 0 indexed.
myPanorama.DefaultItem = myPanorama.Items[1];
}
base.OnNavigatedTo(e);
}
Here's a sample app that uses a querystring.
http://dl.dropbox.com/u/129101/Panorama_querystring.zip
A easier (and better) idea is to define a variable globally, or use a static class. In App.xaml.cs, define
using System.Collections.Generic;
public static Dictionary<string,object> PageContext = new Dictionary<string,object>;
Then, on the first page, simply do
MyComplexObject obj;
int four = 4;
...
App.PageContext.Add("mycomplexobj",obj);
App.PageContext.Add("four",four);
Then, on the new page, simply do
MyComplexObj obj = App.PageContext["mycomplexobj"] as MyComplexObj;
int four = (int)App.PageContext["four"];
To be safe, you should probably check if the object exists:
if (App.PageContext.ContainsKey("four"))
int four = (int)App.PageContext["four"];
You may use an App level variable (defined in App.xaml.cs) and access it from anywhere within your app. If you want to persist, shove it into Isolated Storage and read it on App launch/activate. There are helpers available to JSon serialize/deserialize your reads/writes from the Isolated Storage.
Check out Jeff's post (here) on tips to use Isolated Storage.
Hope this helps!
Well "best" is always subjective, however, I think an application service is a good candidate for this sort of thing:-
public interface IPhoneApplicationService : IApplicationService
{
string Name {get; set;}
object Deactivating();
void Activating(object state);
}
public class AuthenticationService : IPhoneApplicationService
{
public static AuthenticationService Current {get; private set; }
public void StartService(ApplicationServiceContext context)
{
Current = this;
}
public void StopService()
{
Current = null;
}
public string Name {get; set;}
public object Deactivating()
{
// Return an serialisable object such as a Dictionary if necessary.
return UserID;
}
public void Activating(object state)
{
UserID = (int)state;
}
public int UserID { get; private set; }
public void Logon(string username, string password)
{
// Code here that eventually assigns to UserID.
}
}
You place an instance of this in your App.xaml:-
<Application.ApplicationLifetimeObjects>
<!--Required object that handles lifetime events for the application-->
<shell:PhoneApplicationService
Launching="Application_Launching" Closing="Application_Closing"
Activated="Application_Activated" Deactivated="Application_Deactivated"/>
<local:AuthenticationService Name="AuthServ" />
</Application.ApplicationLifetimeObjects>
Now you do need to tweak the App.xaml.cs:-
private void Application_Activated(object sender, ActivatedEventArgs e)
{
var state = PhoneApplicationService.Current.State;
foreach (var service in ApplicationLifetimeObjects.OfType<IPhoneApplicationService>())
{
if (state.ContainsKey(service.Name))
{
service.Activating(state[service.Name]);
}
}
}
private void Application_Deactivated(object sender, DeactivatedEventArgs e)
{
var state = PhoneApplicationService.Current.State;
foreach (var service in ApplicationLifetimeObjects.OfType<IPhoneApplicationService>())
{
if (state.ContainsKey(service.Name))
{
state[service.Name] = service.Deactivating();
}
else
{
state.Add(service.Name, service.Deactivating());
}
}
}
You can now access you UserID anywhere in your app with:-
AuthenticationService.Current.UserID
This general pattern can be used to maintain seperation of key application wide services (you don't load a whole bunch of incohesive properties into your App class). It also provides the hooks for maintaining state between activations which is essential.
I get the above error sometimes during the read. The exception originates from ASP.NET SqlDataReader whenever you try to read data before calling the Read() method. Since EF does all these internally, I am wondering what else can cause this error. could it be network (or) db connectivity?
thanks
Additional Bounty Info (GenericTypeTea):
I've got the same error after upgrading to EF Code First RC (4.1):
"Invalid attempt to read when no data
is present"
This is the code in question:
using (var context = GetContext())
{
var query = from item in context.Preferences
where item.UserName == userName
where item.PrefName == "TreeState"
select item;
// Error on this line
Preference entity = query.FirstOrDefault();
return entity == null ? null : entity.Value;
}
The table structure is as follows:
Preference
{
Username [varchar(50)]
PrefName [varchar(50)]
Value [varchar(max)] Nullable
}
The table is standalone and has no relationships. This is the DbModelBuilder code:
private void ConfigurePreference(DbModelBuilder builder)
{
builder.Entity<Preference>().HasKey(x => new { x.UserName, x.PrefName });
builder.Entity<Preference>().ToTable("RP_Preference");
}
Exactly the same code works perfectly in CTP5. I'm guessing this is an RC bug, but any ideas of how to fix it would be appreciated.
This error occurs when there is a large amount of data in the RC release. The difference between the RC and CTP5 is that you need to specify the [MaxLength] property that contains a large amount of data.
Are you re-using contexts? I would guess this is happening as a result of something you are doing within GetContext
If GetContext() provides a stale context, in which the DataReader is closed/corrupted, I could see the above happening.
I cannot reproduce your problem on EF4.1 RC1.
POCO:
public class Preference
{
public string UserName { get; set; }
public string PrefName { get; set; }
public string Value { get; set; }
}
Context:
public class PreferenceContext : DbContext
{
public DbSet<Preference> Preferences {get;set;}
public PreferenceContext()
: base("Data Source=localhost;Initial Catalog=_so_question_ef41_rc;Integrated Security=SSPI;") {
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
ConfigurePreference(modelBuilder);
base.OnModelCreating(modelBuilder);
}
private void ConfigurePreference(DbModelBuilder builder)
{
builder.Entity<Preference>().HasKey(x => new { x.UserName, x.PrefName });
builder.Entity<Preference>().ToTable("RP_Preference");
}
}
My little Console App:
class Program
{
static void Main(string[] args)
{
string userName = "Anon";
for (int i = 0; i < 10000; i++)
{
var p = GetPreference(userName);
}
}
private static string GetPreference(string userName)
{
using (var context = new PreferenceContext())
{
var query = from item in context.Preferences
where item.UserName == userName
where item.PrefName == "TreeState"
select item;
// Error on this line
Preference entity = query.FirstOrDefault();
return entity == null ? null : entity.Value;
}
}
}
I do 10,000 reads, and no error. You will need to post more complete code to continue.
Increase the CommandTimeout on the context.
I had the same issue with EF4 - In my case I was (trying to) return the list of entities within the using{} section. This is the same as you are doing in your question:
return entity == null ? null : entity.Value;
} // end using
I moved the return to after the } and it worked.
I think I had the problem because the code was in a function which had already queried the database in another using block, I suspect the table was locking but not reporting the error, ending the using block before the return released the database lock.
Steve