I need to relate some tables using non-PK columns in two of them. The tables and FKs are set up as follows:
So a Message can be assigned to many Users and Groups via the MessageUserGroups table. Additionally, a User can be assigned to many Groups via the UserGroups table.
The AdGuid field (a uniqueidentifier) in Users and Groups is not the primary key for legacy reasons. However, I would like to use the UserId and GroupId fields in MessageUserGroups to relate Messages to Users and Groups through their AdGuid fields.
Additionally, I would like to have a PostedMessages property on User which relates Message.AuthorId to User.AdGuid, and have Messages properties on User and Group which relates to Messages through MessageUserGroups.
How do I explain this to Fluent NHibernate?
Currently the mappings look like:
public MessageMap()
{
Id(m => m.Id);
References(m => m.Author).Nullable().ForeignKey("FK_Messages_Author").Column("AuthorId").Fetch.Join().PropertyRef(u => u.AdGuid);
HasManyToMany<users.user>(m => m.Users)
.Cascade.All()
.ParentKeyColumn("MessageId")
.ChildKeyColumn("UserId")
.Table("MessageUserGroups")
.FetchType.Join();
HasManyToMany<users.group>(m => m.Groups)
.Cascade.All()
.ParentKeyColumn("MessageId")
.ChildKeyColumn("GroupId")
.Table("MessageUserGroups")
.FetchType.Join();
}
public UserMap()
{
Id(u => u.Id);
Map(u => u.AdGuid);
HasManyToMany<staff.Message>(u => u.Messages)
.Cascade.All()
.ParentKeyColumn("AdGuid")
.ChildKeyColumn("UserId")
.Inverse()
.LazyLoad()
.Table("MessageUserGroup")
HasMany<staff.Message>(u => u.PostedMessages)
.Cascade.All()
.KeyColumn("AuthorId")
.Inverse()
.LazyLoad()
.Table("Messages")
HasManyToMany<Group>(u => u.Groups)
.Cascade.All()
.Not.LazyLoad()
.Cascade.All()
.ParentKeyColumn("UserID")
.ChildKeyColumn("GroupID")
.Table("UserGroups")
}
public GroupMap()
{
Id(g => g.Id);
Map(g => g.AdGuid);
HasManyToMany<staff.Message>(g => g.Messages)
.Cascade.All()
.ParentKeyColumn("AdGuid")
.ChildKeyColumn("GroupId")
.Inverse()
.LazyLoad()
.Table("MessageUserGroup")
HasManyToMany<User>(g => g.Users)
.Inverse()
.Cascade.All()
.ParentKeyColumn("GroupId")
.ChildKeyColumn("UserId")
.Table("UserGroups")
}
A little lengthy, I know. I can successfully query for Messages using NH. However, when lazy-loading the Messages properties of User or Group, NH generates this SQL (extra properties removed for brevity):
SELECT users0_.MessageId as MessageId1_, users0_.UserId as UserId1_, user1_.Id
as id38_0_, user1_.AdGuid as guid38_0_ FROM MessageUserGroups users0_
LEFT OUTER JOIN Users user1_ on users0_.UserId=user1_.Id WHERE users0_.MessageId=#p0
That join is invalid because it's trying to compare the uniqueidentifer MessageUserGroups.UserId against the bigint User.Id.
Likewise, when lazy-loading User.PostedMessages, this SQL is generated (again, abbreviated):
SELECT postedmess0_.AuthorId as AuthorId1_, postedmess0_.Id as Id1_, postedmess0_.Id as Id43_0_, postedmess0_.AuthorId as AuthorId43_0_
FROM Staff.Messages postedmess0_
WHERE postedmess0_.AuthorId=#p0
And #p0 is set to 89, which is the Id of the user I am testing with, but AuthorId needs to be a uniqueidentifier.
After both of those I get an SqlException: Operand type clash: uniqueidentifier is incompatible with bigint which is to be expected.
It looks like (Fluent) NHibernate always wants to join on the PKs, even though the docs suggest that specifying the ParentKeyColumn and ChildKeyColumn should allow me to specify any columns I like. Is that really possible? Am I misreading?
ParentKeyColumn and ChildKeyColumn specify the column names in the link table between and not the columns of the entity-tables joined. What you need is PropertyRef and ChildPropertyRef on the hasmanytomany to specify the properties to join to
Related
I have a Logstash pipeline that fetches data from MS SQL view that joins to tables A and B and put the denormalised data into ES.
Initially, INSERTS or UPDATES could happen only for table A. Therefore, to configure Logstash to pick up only newly inserted or updated records since last iteration of the polling loop, I have defined the tracking_column field which refers updatedDate timestamp column in table A:
jdbc {
#Program Search
jdbc_connection_string => "jdbc:sqlserver://__DB_LISTNER__"
jdbc_user => “admin”
jdbc_password => “admin”
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_driver_library => "/usr/share/logstash/drivers/mssql-jdbc-6.2.2.jre8.jar"
sql_log_level => "info"
tracking_column => "updated_date_timestamp"
use_column_value => true
tracking_column_type => "timestamp"
schedule => "*/1 * * * *"
statement => "select *, updateDate as updated_date_timestamp from dbo.MyView where (updateDate > :sql_last_value and updateDate < getdate()) order by updateDate ASC"
last_run_metadata_path => "/usr/share/logstash/myfolder/.logstash_jdbc_last_run"
}
Now, the UPDATES can also happen in table B. With this new requirement I am confused how can I configure Logstash to track changes on the table B as well in the same pipeline. Can I define multiple tracking_columns for the same pipeline?
Another two options I have in mind but not sure about them are:
Generate a composite value from updateDate fields of table A and B, that will be referenced by the tracking_column. But I am not sure how the SQL query should look like then?
Create another pipeline that will track changes for table B only. Though, the drawback, I see for this approach, is that the existing and new pipelines will do duplicate work on the initial iterations in order to process all the records from the DB view.
Please, advise me how should I go from here?
I found this ES discussion that suggests to use a function to select greatest value of provided dates in the SQL query. For the SQL server there is GREATEST function, but it is not recognised by SQL server I am currently using. Long story short, as a workaround I found iff() function which I use for dates comparing. So my SQL query looks like this:
select *, iif(A.updatedDate>B.updatedDate, A.updatedDate, B.updatedDate) as updated_date_timestamp from dbo.MyView where (iif(A.updatedDate>B.updatedDate, A.updatedDate, B.updatedDate) > :sql_last_value and iif(A.updatedDate>B.updatedDate, A.updatedDate, B.updatedDate) < getdate()) order by iif(A.updatedDate>B.updatedDate, A.updatedDate, B.updatedDate) ASC, id ASC
I use a union to join two datasets and then the following query to setup for pagination correctly
$paginationQuery = $this->find('all')
->contain(['EmailAddresses' => [
'foreignKey' => false,
'queryBuilder' => function($q) {
return $q->where(['Members__id' => 'EmailAddresses.member_id']);
}
]])
->select( $selectMainUnion )
->from([$this->getAlias() => $query])
->order(['Members__last_name' => 'ASC', 'Members__first_name' => 'ASC']);
I have also tried
$paginationQuery = $this->find('all')
->contain(['EmailAddresses'])
->select( $selectMainUnion )
->from([$this->getAlias() => $query])
->order(['Members__last_name' => 'ASC', 'Members__first_name' => 'ASC']);
and tried
$query->loadInto($query, ['EmailAddresses']); where $query is the result of the union.
Neither of these result in email addresses added to $paginationQuery.
Is there a way to do this?
Adding to clarify the code
$selectMain =['Members.id',
'Members.member_type',
'Members.first_name',
'Members.middle_name',
'Members.last_name',
'Members.suffix',
'Members.date_joined'];
foreach($selectMain as $select) {
$selectMainUnion[] = str_replace('.', '__', $select);
}
$this->hasMany('EmailAddresses', [
'foreignKey' => 'member_id',
'dependent' => true,
]);
Looking at the SQL in DebugKit SQL Log, there is no reference to the EmailAddresses table.
Generally containments do work fine irrespective of the queries FROM clause, whether that's a table or a subquery should be irrelevant. The requirement for this to work however is that the required primary and/or foreign key fields are being selected, and that they are in the correct format.
By default CakePHP's ORM queries automatically alias selected fields, ie they are being selected like Alias.field AS Alias__field. So when Alias is a subquery, then Alias.field doesn't exist, you'd have to select Alias.Alias__field instead. So with the automatic aliases, your select of Members__id would be transformed to Members.Members__id AS Members__Members__id, and Members__Members__id is not something the ORM understands, it would end up as Members__id in your entities, where the eager loader would expect id instead, ie the name of the primary key which is used to inject the results of the queried hasMany associated records (this happens in a separate query), your custom queryBuilder won't help with that, as the injecting happens afterwards on PHP level.
Long story short, to fix the problem, you can either change how the fields of the union queries are selected, ie ensure that they are not selected with aliases, that way the pagination query fields do not need to be changed at all:
$fields = $table->getSchema()->columns();
$fields = array_combine($fields, $fields);
$query->select($fields);
This will create a list of fields in the format of ['id' => 'id', ...], looks a bit whacky, but it works (as long as there's no ambiguity because of joined tables for example), the SQL would be like id AS id, so your pagination query can then simply reference the fields like Members.id.
Another way would be to select the aliases of the subquery, ie not just select Member__id, which the ORM turns into Member__Member__id when it applies automatic aliasing, but use Members.Member__id, like:
[
'Member__id' => 'Members.Member__id',
// ...
]
That way no automatic aliasing takes place, on SQL level it would select the field like Members.Member__id AS Member__id, and the field would end up as id in your entities, which the eager loader would find and could use for injecting the associated records.
I'm trying to achieve a query similar to this:
SELECT r.*, (SELECT COUNT(UserID) FROM RoleUsers ru WHERE ru.RoleId = r.Id) AS Assignments
FROM Roles r
To retrieve the number of the users per each role.
The simplest and the most straightforward option to implement desired output:
this.DbContext.Set<Role>().Include(x => x.RoleUser)
.Select(x => new { x, Assignments = x.RoleUsers.Count() });
Retrieves all the roles, and then N queries to retrieve count:
SELECT COUNT(*)
FROM [dbo].[RoleUsers] AS [r0]
WHERE #_outer_Id = [r0].[RoleId]
Which is not an option at all. I tried also to use GroupJoin, but it loads all the required data set in one query and performs grouping in memory:
this.DbContext.Set<Role>().GroupJoin(this.DbContext.Set<RoleUser>(), role => role.Id,
roleUser => roleUser.RoleId, (role, roleUser) => new
{
Role = role,
Assignments = roleUser.Count()
});
Generated query:
SELECT [role].[Id], [role].[CustomerId], [role].[CreateDate], [role].[Description], [role].[Mask], [role].[ModifyDate], [role].[Name], [assignment].[UserId], [assignment].[CustomerId], [assignment].[RoleId]
FROM [dbo].[Roles] AS [role]
LEFT JOIN [dbo].[RoleUser] AS [assignment] ON [role].[Id] = [assignment].[RoleId]
ORDER BY [role].[Id]
Also, I was looking into a way, to use windowing functions, where I can just split count by partition and use distinct roles, but I have no idea how to wire up windowing function in EF:
SELECT DISTINCT r.*, COUNT(ra.UserID) OVER(PARTITION BY ru.RoleId)
FROM RoleUsers ru
RIGHT JOIN Roles r ON r.Id = ru.RoleId
So, is there any way to avoid EntitySQL?
Currently there is a defect in EF Core query aggregate translation to SQL when the query projection contains a whole entity, like
.Select(role => new { Role = role, ...}
The only workaround I'm aware of is to project to new entity (at least this is supported by EF Core) like
var query = this.DbContext.Set<Role>()
.Select(role => new
{
Role = new Role { Id = role.Id, Name = role.Name, /* all other Role properies */ },
Assignments = role.RoleUsers.Count()
});
This translates to single SQL query. The drawback is that you have to manually project all entity properties.
this.DbContext.Set<Role>()
.Select(x => new { x, Assignments = x.RoleUsers.Count() });
you dont need to add include for RoleUser since you are using Select statement. Furhtermore, I guess that you are using LazyLoading where this is expected behavior. If you use eager loading the result of your LINQ will run in one query.
you can use context.Configuration.LazyLoadingEnabled = false; before your LINQ query to disable lazy loading specifically for this operation
I'm a little surprised I haven't found any information on the following question, so please excuse if I've missed it somewhere in the docs. Using SQL Server (2016 locally and Azure) and EFCore Code First we're trying to create a computed table column with a persisted value. Creating the column works fine, but I don't have a clue how to persist the value. Here's what we do:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]))");
});
And here is what we'd actually like to get in T-SQL:
CREATE TABLE [dbo].[SomeClass]
(
[FirstColumnName] [NVARCHAR](10)
, [SecondColumnName] [NVARCHAR](10)
, [Checksum] AS (CHECKSUM([FirstColumnName], [SecondColumnName])) PERSISTED
);
Can anyone point me in the right direction?
Thanks in advance, Tobi
UPDATE: Based on a good idea by #jeroen-mostert I also tried to just pass the PERSISTED string as part of the formula:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]) PERSISTED)");
});
And also outside of the parentheses:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName])) PERSISTED");
});
However und somehow surprisingly, the computed column is still generated with Is Persisted = No, so the PERSISTED string simply seems to be ignored.
Starting with EF Core 5, the HasComputedColumnSql method has a new optional parameter bool? stored to specify that the column should be persisted:
modelBuilder.Entity<SomeClass>()
.Property(p => p.Checksum)
.HasComputedColumnSql("checksum([FirstColumnName], [SecondColumnName])", stored: true);
After doing some reading and some tests, I ended up trying the PERSISTED inside the SQL query and it worked.
entity.Property(e => e.Duration_ms)
.HasComputedColumnSql("DATEDIFF(MILLISECOND, 0, duration) PERSISTED");
The generated migration was the following:
migrationBuilder.AddColumn<long>(
name: "duration_ms",
table: "MyTable",
nullable: true,
computedColumnSql: "DATEDIFF(MILLISECOND, 0, duration) PERSISTED");
To check on the database whether it is actually persisted I ran the following:
select is_persisted, name from sys.computed_columns where is_persisted = 1
and the column that I've created is there.
" You may also specify that a computed column be stored (sometimes called persisted), meaning that it is computed on every update of the row, and is stored on disk alongside regular columns:"
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]), stored: true);
});
This is taken (and slightly modified) from Microsoft Docs.: https://learn.microsoft.com/en-us/ef/core/modeling/generated-properties?tabs=data-annotations#computed-columns
I am making a query inside a form.
The idea is : Now I have a dropdown with all the users, ok?
I have a query like this:
'query_builder' => function(EntityRepository $er) {
return $er->createQueryBuilder('u')->orderBy('u.lastName', 'ASC');
}
This works flawless!
Instead of displaying all the users, I only need the users that are associated with a category. This definition is in the "user_data" table (user_id + category_id).
So, I need to do something like:
SELECT * FROM users
WHERE user_id IN (SELECT user_id FROM user_data WHERE category_id='2')
I don't have any entity that looks like UserData, I only have User.php, but inside this file I found this:
/**
*#ORM\ManyToMany(targetEntity="My\Bundle\Entity\Data", inversedBy="users")
*#ORM\joinTable(name="user_data")
*/
protected $datas;
[...]
So I see that a relationship is build, but I don't get it how to use it in order to build my query based on the 2 tables...
Anyone can help me with this issue? :)
Thanks!
Try this,
'query_builder' => function(EntityRepository $er) {
return $er->createQueryBuilder('u')
->innerJoin('u.datas', 'd')
->where('d.category = :category') // Check your Data entity to get the right field name
->setParameter('category', $yourCategoryId) // you could use options here
->orderBy('u.lastName', 'ASC');
}
Also, update your question with all your entities (Data, ...)