Multi-Mapper to create object hierarchy - dapper

Question

I've been playing around with this for a bit, because it seems like it feels a lot like the documented posts/users example, but its slightly different and isn't working for me.
Assuming the following simplified setup (a contact has multiple phone numbers):
public class Contact
{
public int ContactID { get; set; }
public string ContactName { get; set; }
public IEnumerable<Phone> Phones { get; set; }
}
public class Phone
{
public int PhoneId { get; set; }
public int ContactID { get; set; } // foreign key
public string Number { get; set; }
public string Type { get; set; }
public bool IsActive { get; set; }
}
I'd love to end up with something that returns a Contact with multiple Phone objects. That way, if I had 2 contacts, with 2 phones each, my SQL would return a join of those as a result set with 4 total rows. Then Dapper would pop out 2 contact objects with two phones each.
Here is the SQL in the stored procedure:
SELECT *
FROM Contacts
LEFT OUTER JOIN Phones ON Phones.ReferenceId=Contacts.ReferenceId
WHERE clientid=1
I tried this, but ended up with 4 Tuples (which is OK, but not what I was hoping for... it just means I still have to re-normalize the result):
var x = cn.Query<Contact, Phone, Tuple<Contact, Phone>>("sproc_Contacts_SelectByClient",
(co, ph) => Tuple.Create(co, ph),
splitOn: "PhoneId", param: p,
commandType: CommandType.StoredProcedure);
and when I try another method (below), I get an exception of "Unable to cast object of type 'System.Int32' to type 'System.Collections.Generic.IEnumerable`1[Phone]'."
var x = cn.Query<Contact, IEnumerable<Phone>, Contact>("sproc_Contacts_SelectByClient",
(co, ph) => { co.Phones = ph; return co; },
splitOn: "PhoneId", param: p,
commandType: CommandType.StoredProcedure);
Am I just doing something wrong? It seems just like the posts/owner example, except that I'm going from the parent to the child instead of the child to the parent.

Answer 1

You are doing nothing wrong, it is just not the way the API was designed. All the Query APIs will always return an object per database row.
So, this works well on the many -> one direction, but less well for the one -> many multi-map.
There are 2 issues here:
If we introduce a built-in mapper that works with your query, we would be expected to "discard" duplicate data. (Contacts.* is duplicated in your query)
If we design it to work with a one -> many pair, we will need some sort of identity map. Which adds complexity.
Take for example this query which is efficient if you just need to pull a limited number of records, if you push this up to a million stuff get trickier, cause you need to stream and can not load everything into memory:
var sql = "set nocount on
DECLARE #t TABLE(ContactID int, ContactName nvarchar(100))
INSERT #t
SELECT *
FROM Contacts
WHERE clientid=1
set nocount off
SELECT * FROM #t
SELECT * FROM Phone where ContactId in (select t.ContactId from #t t)"
What you could do is extend the GridReader to allow for the remapping:
var mapped = cnn.QueryMultiple(sql)
.Map<Contact,Phone, int>
(
contact => contact.ContactID,
phone => phone.ContactID,
(contact, phones) => { contact.Phones = phones };
);
Assuming you extend your GridReader and with a mapper:
public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
(
this GridReader reader,
Func<TFirst, TKey> firstKey,
Func<TSecond, TKey> secondKey,
Action<TFirst, IEnumerable<TSecond>> addChildren
)
{
var first = reader.Read<TFirst>().ToList();
var childMap = reader
.Read<TSecond>()
.GroupBy(s => secondKey(s))
.ToDictionary(g => g.Key, g => g.AsEnumerable());
foreach (var item in first)
{
IEnumerable<TSecond> children;
if(childMap.TryGetValue(firstKey(item), out children))
{
addChildren(item,children);
}
}
return first;
}
Since this is a bit tricky and complex, with caveats. I am not leaning towards including this in core.

Answer 2

FYI - I got Sam's answer working by doing the following:
First, I added a class file called "Extensions.cs". I had to change the "this" keyword to "reader" in two places:
using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;
namespace TestMySQL.Helpers
{
public static class Extensions
{
public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
(
this Dapper.SqlMapper.GridReader reader,
Func<TFirst, TKey> firstKey,
Func<TSecond, TKey> secondKey,
Action<TFirst, IEnumerable<TSecond>> addChildren
)
{
var first = reader.Read<TFirst>().ToList();
var childMap = reader
.Read<TSecond>()
.GroupBy(s => secondKey(s))
.ToDictionary(g => g.Key, g => g.AsEnumerable());
foreach (var item in first)
{
IEnumerable<TSecond> children;
if (childMap.TryGetValue(firstKey(item), out children))
{
addChildren(item, children);
}
}
return first;
}
}
}
Second, I added the following method, modifying the last parameter:
public IEnumerable<Contact> GetContactsAndPhoneNumbers()
{
var sql = #"
SELECT * FROM Contacts WHERE clientid=1
SELECT * FROM Phone where ContactId in (select ContactId FROM Contacts WHERE clientid=1)";
using (var connection = GetOpenConnection())
{
var mapped = connection.QueryMultiple(sql)
.Map<Contact,Phone, int> (
contact => contact.ContactID,
phone => phone.ContactID,
(contact, phones) => { contact.Phones = phones; }
);
return mapped;
}
}

Answer 3

Check out https://www.tritac.com/blog/dappernet-by-example/
You could do something like this:
public class Shop {
public int? Id {get;set;}
public string Name {get;set;}
public string Url {get;set;}
public IList<Account> Accounts {get;set;}
}
public class Account {
public int? Id {get;set;}
public string Name {get;set;}
public string Address {get;set;}
public string Country {get;set;}
public int ShopId {get;set;}
}
var lookup = new Dictionary<int, Shop>()
conn.Query<Shop, Account, Shop>(#"
SELECT s.*, a.*
FROM Shop s
INNER JOIN Account a ON s.ShopId = a.ShopId
", (s, a) => {
Shop shop;
if (!lookup.TryGetValue(s.Id, out shop)) {
lookup.Add(s.Id, shop = s);
}
shop.Accounts.Add(a);
return shop;
},
).AsQueryable();
var resultList = lookup.Values;
I got this from the dapper.net tests: https://code.google.com/p/dapper-dot-net/source/browse/Tests/Tests.cs#1343

Answer 4

Multi result set support
In your case it would be much better (and easier as well) to have a multi resultset query. This simply means that you should write two select statements:
One that returns contacts
And one that returns their phone numbers
This way your objects would be unique and wouldn't duplicate.

Answer 5

Here's a reusable solution that is pretty easy to use. It is a slight modification of Andrews answer.
public static IEnumerable<TParent> QueryParentChild<TParent, TChild, TParentKey>(
this IDbConnection connection,
string sql,
Func<TParent, TParentKey> parentKeySelector,
Func<TParent, IList<TChild>> childSelector,
dynamic param = null, IDbTransaction transaction = null, bool buffered = true, string splitOn = "Id", int? commandTimeout = null, CommandType? commandType = null)
{
Dictionary<TParentKey, TParent> cache = new Dictionary<TParentKey, TParent>();
connection.Query<TParent, TChild, TParent>(
sql,
(parent, child) =>
{
if (!cache.ContainsKey(parentKeySelector(parent)))
{
cache.Add(parentKeySelector(parent), parent);
}
TParent cachedParent = cache[parentKeySelector(parent)];
IList<TChild> children = childSelector(cachedParent);
children.Add(child);
return cachedParent;
},
param as object, transaction, buffered, splitOn, commandTimeout, commandType);
return cache.Values;
}
Example usage
public class Contact
{
public int ContactID { get; set; }
public string ContactName { get; set; }
public List<Phone> Phones { get; set; } // must be IList
public Contact()
{
this.Phones = new List<Phone>(); // POCO is responsible for instantiating child list
}
}
public class Phone
{
public int PhoneID { get; set; }
public int ContactID { get; set; } // foreign key
public string Number { get; set; }
public string Type { get; set; }
public bool IsActive { get; set; }
}
conn.QueryParentChild<Contact, Phone, int>(
"SELECT * FROM Contact LEFT OUTER JOIN Phone ON Contact.ContactID = Phone.ContactID",
contact => contact.ContactID,
contact => contact.Phones,
splitOn: "PhoneId");

Answer 6

Based on Sam Saffron's (and Mike Gleason's) approach, here is a solution which will allow for multiple children and multiple levels.
using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;
namespace TestMySQL.Helpers
{
public static class Extensions
{
public static IEnumerable<TFirst> MapChild<TFirst, TSecond, TKey>
(
this SqlMapper.GridReader reader,
List<TFirst> parent,
List<TSecond> child,
Func<TFirst, TKey> firstKey,
Func<TSecond, TKey> secondKey,
Action<TFirst, IEnumerable<TSecond>> addChildren
)
{
var childMap = child
.GroupBy(secondKey)
.ToDictionary(g => g.Key, g => g.AsEnumerable());
foreach (var item in parent)
{
IEnumerable<TSecond> children;
if (childMap.TryGetValue(firstKey(item), out children))
{
addChildren(item, children);
}
}
return parent;
}
}
}
Then you can have it read outside of the function.
using (var multi = conn.QueryMultiple(sql))
{
var contactList = multi.Read<Contact>().ToList();
var phoneList = multi.Read<Phone>().ToList;
contactList = multi.MapChild
(
contactList,
phoneList,
contact => contact.Id,
phone => phone.ContactId,
(contact, phone) => {contact.Phone = phone;}
).ToList();
return contactList;
}
The map function can then be called again for the next child object using the same parent object. You can also implement splits on the parent or child read statements independently of the map function.
Here is a 'single to N' additional extension method
public static TFirst MapChildren<TFirst, TSecond, TKey>
(
this SqlMapper.GridReader reader,
TFirst parent,
IEnumerable<TSecond> children,
Func<TFirst, TKey> firstKey,
Func<TSecond, TKey> secondKey,
Action<TFirst, IEnumerable<TSecond>> addChildren
)
{
if (parent == null || children == null || !children.Any())
{
return parent;
}
Dictionary<TKey, IEnumerable<TSecond>> childMap = children
.GroupBy(secondKey)
.ToDictionary(g => g.Key, g => g.AsEnumerable());
if (childMap.TryGetValue(firstKey(parent), out IEnumerable<TSecond> foundChildren))
{
addChildren(parent, foundChildren);
}
return parent;
}

Answer 7

Once we decided to move our DataAccessLayer to stored procedures, and these procedures are often return multiple linked results (example below).
Well, my approach is almost the same, but maybe little bit more comfortable.
This is how your code may look like:
using ( var conn = GetConn() )
{
var res = await conn
.StoredProc<Person>( procName, procParams )
.Include<Book>( ( p, b ) => p.Books = b.Where( x => x.PersonId == p.Id ).ToList() )
.Include<Course>( ( p, c ) => p.Courses = c.Where( x => x.PersonId == p.Id ).ToList() )
.Include<Course, Mark>( ( c, m ) => c.Marks = m.Where( x => x.CourseId == c.Id ).ToList() )
.Execute();
}
Let's break it down...
Extension:
public static class SqlExtensions
{
public static StoredProcMapper<T> StoredProc<T>( this SqlConnection conn, string procName, object procParams )
{
return StoredProcMapper<T>
.Create( conn )
.Call( procName, procParams );
}
}
Mapper:
public class StoredProcMapper<T>
{
public static StoredProcMapper<T> Create( SqlConnection conn )
{
return new StoredProcMapper<T>( conn );
}
private List<MergeInfo> _merges = new List<MergeInfo>();
public SqlConnection Connection { get; }
public string ProcName { get; private set; }
public object Parameters { get; private set; }
private StoredProcMapper( SqlConnection conn )
{
Connection = conn;
_merges.Add( new MergeInfo( typeof( T ) ) );
}
public StoredProcMapper<T> Call( object procName, object parameters )
{
ProcName = procName.ToString();
Parameters = parameters;
return this;
}
public StoredProcMapper<T> Include<TChild>( MergeDelegate<T, TChild> mapper )
{
return Include<T, TChild>( mapper );
}
public StoredProcMapper<T> Include<TParent, TChild>( MergeDelegate<TParent, TChild> mapper )
{
_merges.Add( new MergeInfo<TParent, TChild>( mapper ) );
return this;
}
public async Task<List<T>> Execute()
{
if ( string.IsNullOrEmpty( ProcName ) )
throw new Exception( $"Procedure name not specified! Please use '{nameof(Call)}' method before '{nameof( Execute )}'" );
var gridReader = await Connection.QueryMultipleAsync(
ProcName, Parameters, commandType: CommandType.StoredProcedure );
foreach ( var merge in _merges )
{
merge.Result = gridReader
.Read( merge.Type )
.ToList();
}
foreach ( var merge in _merges )
{
if ( merge.ParentType == null )
continue;
var parentMerge = _merges.FirstOrDefault( x => x.Type == merge.ParentType );
if ( parentMerge == null )
throw new Exception( $"Wrong parent type '{merge.ParentType.FullName}' for type '{merge.Type.FullName}'." );
foreach ( var parent in parentMerge.Result )
{
merge.Merge( parent, merge.Result );
}
}
return _merges
.First()
.Result
.Cast<T>()
.ToList();
}
private class MergeInfo
{
public Type Type { get; }
public Type ParentType { get; }
public IEnumerable Result { get; set; }
public MergeInfo( Type type, Type parentType = null )
{
Type = type;
ParentType = parentType;
}
public void Merge( object parent, IEnumerable children )
{
MergeInternal( parent, children );
}
public virtual void MergeInternal( object parent, IEnumerable children )
{
}
}
private class MergeInfo<TParent, TChild> : MergeInfo
{
public MergeDelegate<TParent, TChild> Action { get; }
public MergeInfo( MergeDelegate<TParent, TChild> mergeAction )
: base( typeof( TChild ), typeof( TParent ) )
{
Action = mergeAction;
}
public override void MergeInternal( object parent, IEnumerable children )
{
Action( (TParent)parent, children.Cast<TChild>() );
}
}
public delegate void MergeDelegate<TParent, TChild>( TParent parent, IEnumerable<TChild> children );
}
That's all, but if you wanna do quick test, here is models and procedure for you:
Models:
public class Person
{
public Guid Id { get; set; }
public string Name { get; set; }
public List<Course> Courses { get; set; }
public List<Book> Books { get; set; }
public override string ToString() => Name;
}
public class Book
{
public Guid Id { get; set; }
public Guid PersonId { get; set; }
public string Name { get; set; }
public override string ToString() => Name;
}
public class Course
{
public Guid Id { get; set; }
public Guid PersonId { get; set; }
public string Name { get; set; }
public List<Mark> Marks { get; set; }
public override string ToString() => Name;
}
public class Mark
{
public Guid Id { get; set; }
public Guid CourseId { get; set; }
public int Value { get; set; }
public override string ToString() => Value.ToString();
}
SP:
if exists (
select *
from sysobjects
where
id = object_id(N'dbo.MultiTest')
and ObjectProperty( id, N'IsProcedure' ) = 1 )
begin
drop procedure dbo.MultiTest
end
go
create procedure dbo.MultiTest
#PersonId UniqueIdentifier
as
begin
declare #tmpPersons table
(
Id UniqueIdentifier,
Name nvarchar(50)
);
declare #tmpBooks table
(
Id UniqueIdentifier,
PersonId UniqueIdentifier,
Name nvarchar(50)
)
declare #tmpCourses table
(
Id UniqueIdentifier,
PersonId UniqueIdentifier,
Name nvarchar(50)
)
declare #tmpMarks table
(
Id UniqueIdentifier,
CourseId UniqueIdentifier,
Value int
)
--------------------------------------------------
insert into #tmpPersons
values
( '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Иван' ),
( '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Василий' ),
( '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Алефтина' )
insert into #tmpBooks
values
( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Математика' ),
( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Физика' ),
( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Геометрия' ),
( NewId(), '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Книга Биология' ),
( NewId(), '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Книга Химия' ),
( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга История' ),
( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга Литература' ),
( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга Древне-шумерский диалект иврита' )
insert into #tmpCourses
values
( '30945b68-a6ef-4da8-9a35-d3b2845e7de3', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Математика' ),
( '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Физика' ),
( '92bbefd1-9fec-4dc7-bb58-986eadb105c8', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Геометрия' ),
( '923a2f0c-c5c7-4394-847c-c5028fe14711', '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Биология' ),
( 'ace50388-eb05-4c46-82a9-5836cf0c988c', '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Химия' ),
( '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'История' ),
( '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Литература' ),
( '73ac366d-c7c2-4480-9513-28c17967db1a', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Древне-шумерский диалект иврита' )
insert into #tmpMarks
values
( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 98 ),
( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 87 ),
( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 76 ),
( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 89 ),
( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 78 ),
( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 67 ),
( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 79 ),
( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 68 ),
( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 75 ),
----------
( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 198 ),
( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 187 ),
( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 176 ),
( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 189 ),
( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 178 ),
( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 167 ),
----------
( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 8 ),
( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 7 ),
( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 6 ),
( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 9 ),
( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 8 ),
( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 7 ),
( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 9 ),
( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 8 ),
( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 5 )
--------------------------------------------------
select * from #tmpPersons
select * from #tmpBooks
select * from #tmpCourses
select * from #tmpMarks
end
go

Answer 8

I wanted to share my solution to this issue and see if anyone has any constructive feedback on the approach I've used?
I have a few requirements in the project I'm working on which I need to explain first up:
I have to keep my POCO's as clean as possible as these classes will be publicly shared in an API wrapper.
My POCO's are in a seperate Class Library because of the above requirement
There are going to be multiple object hierarchy levels that will vary depending on data (so I cannot use a Generic Type Mapper or I'd have to write tons of them to cater for all possible eventualities)
So, what I have done is to get SQL to handle the 2nd - nth Level heirarchy by returning a Single JSON string as a column on the original row as follows (stripped out the other columns / properties etc to illustrate):
Id AttributeJson
4 [{Id:1,Name:"ATT-NAME",Value:"ATT-VALUE-1"}]
Then, my POCO's are built up like the below :
public abstract class BaseEntity
{
[KeyAttribute]
public int Id { get; set; }
}
public class Client : BaseEntity
{
public List<ClientAttribute> Attributes{ get; set; }
}
public class ClientAttribute : BaseEntity
{
public string Name { get; set; }
public string Value { get; set; }
}
Where the POCO's inherit from BaseEntity. (To illustrate I've chosen a fairly simple, single level heirarchy as shown by the "Attributes" property of the client object. )
I then have in my Data Layer the following "Data Class" which inherits from the POCO Client.
internal class dataClient : Client
{
public string AttributeJson
{
set
{
Attributes = value.FromJson<List<ClientAttribute>>();
}
}
}
As you can see above, whats happening is that SQL is returning a column called "AttributeJson" which is mapped to the property AttributeJson in the dataClient class. This has only a setter which deserialises the JSON to the Attributes property on the inherited Client class. The dataClient Class is internal to the Data Access Layer and the ClientProvider (my data factory) returns the original Client POCO to the calling App / Library like so :
var clients = _conn.Get<dataClient>();
return clients.OfType<Client>().ToList();
Note that I'm using Dapper.Contrib and have added a new Get<T> Method that returns an IEnumerable<T>
There are a couple things to note with this solution:
There's an obvious performance trade off with the JSON serialisation - I've benchmarked this against 1050 rows with 2 sub List<T> properties, each with 2 entities in the list and it clocks in at 279ms - which is acceptable for my projects needs - this is also with ZERO optimisation on the SQL side of things so I should be able to shave a few ms there.
It does mean additional SQL queries are required to build up the JSON for each required List<T> property, but again, this suits me as I know SQL pretty well and am not so fluent on dynamics / reflection etc.. so this way I feel like I have more control over things as I actually understand whats happening under the hood :-)
There may well be a better solution than this one and if there is I would really appreciate hearing your thoughts - this is just the solution I came up with that so far fits my needs for this project (although this is experimental at the stage of posting).

Answer 9

use this:
public class Product
{
public int ProductId { get; set; }
public string ProductName { get; set; }
public Category Category { get; set; }
}
public class Category
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
public ICollection<Product> Products { get; set; }
}
using (var connection = new SQLiteConnection(connString))
{
var sql = #"select productid, productname, p.categoryid, categoryname
from products p
inner join categories c on p.categoryid = c.categoryid";
var products = await connection.QueryAsync<Product, Category, Product>(sql, (product, category) => {
product.Category = category;
return product;
},
splitOn: "CategoryId");
products.ToList().ForEach(product => Console.WriteLine($"Product: {product.ProductName}, Category: {product.Category.CategoryName}"));
Console.ReadLine();
}
took from: Managing relationShips