I am writing migrations script in cakephp 3 using Phinx.
I need to add a column with primary key while updating the table(using update() command) using migrations.
But when I run the script, it created the column but does not include the primary key.
$table->addColumn('book_id', 'integer', [
'default' => null,
'limit' => 11,
'null' => true
])->addPrimaryKey('book_id');
$table->update();
Thanks
Per the Docs:
Dealing with primary key can only be done on table creation
operations. This is due to limitations for some database servers the
plugin supports.
public function change(): void
{
$this->table('table_name')
->changePrimaryKey(['column1', 'column2'])
->save();
}
Related
There are a lot of questions of how to use fluent API to set a columns as an identity column using:
.UseIdentityColumn()
However, I need the opposite - the primary key field, needs to be the table identity (for UPSERTS) however, I'd like to remove the identity spec.
In SSMS I can set the identity specification to no but I'd like to see if this is possible within the Fluent API.
I've tried:
Edit to update - the full code is:
builder.Entity<ModelClass>().HasKey(x => x.Id);
builder.Entity<ModelClass>()
.Property(x => x.Id)
.ValueGeneratedNever()
.HasAnnotation("DatabaseGenerated", DatabaseGeneratedOption.None);
So, the ID column still needs to be key to the table .HasKey(x => x.Id) - but without the identity spec
The property Id is a key, and then told to not generate values. For me, in SQL server the table still has a "Identity Specification" = "yes" and then seed and increment values set.
1- update your entity using fluent API as below
entity.HasKey(e => e.Id).HasAnnotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.None);
2- Create new migration and make sure that your property has below configuration in new generated migration code in UP method
.Annotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.None)
then update your DB
.HasAnnotation("Npgsql:ValueGenerationStrategy",
Npgsql.EntityFrameworkCore.PostgreSQL.Metadata.NpgsqlValueGenerationStrategy.SerialColumn)
I use a union to join two datasets and then the following query to setup for pagination correctly
$paginationQuery = $this->find('all')
->contain(['EmailAddresses' => [
'foreignKey' => false,
'queryBuilder' => function($q) {
return $q->where(['Members__id' => 'EmailAddresses.member_id']);
}
]])
->select( $selectMainUnion )
->from([$this->getAlias() => $query])
->order(['Members__last_name' => 'ASC', 'Members__first_name' => 'ASC']);
I have also tried
$paginationQuery = $this->find('all')
->contain(['EmailAddresses'])
->select( $selectMainUnion )
->from([$this->getAlias() => $query])
->order(['Members__last_name' => 'ASC', 'Members__first_name' => 'ASC']);
and tried
$query->loadInto($query, ['EmailAddresses']); where $query is the result of the union.
Neither of these result in email addresses added to $paginationQuery.
Is there a way to do this?
Adding to clarify the code
$selectMain =['Members.id',
'Members.member_type',
'Members.first_name',
'Members.middle_name',
'Members.last_name',
'Members.suffix',
'Members.date_joined'];
foreach($selectMain as $select) {
$selectMainUnion[] = str_replace('.', '__', $select);
}
$this->hasMany('EmailAddresses', [
'foreignKey' => 'member_id',
'dependent' => true,
]);
Looking at the SQL in DebugKit SQL Log, there is no reference to the EmailAddresses table.
Generally containments do work fine irrespective of the queries FROM clause, whether that's a table or a subquery should be irrelevant. The requirement for this to work however is that the required primary and/or foreign key fields are being selected, and that they are in the correct format.
By default CakePHP's ORM queries automatically alias selected fields, ie they are being selected like Alias.field AS Alias__field. So when Alias is a subquery, then Alias.field doesn't exist, you'd have to select Alias.Alias__field instead. So with the automatic aliases, your select of Members__id would be transformed to Members.Members__id AS Members__Members__id, and Members__Members__id is not something the ORM understands, it would end up as Members__id in your entities, where the eager loader would expect id instead, ie the name of the primary key which is used to inject the results of the queried hasMany associated records (this happens in a separate query), your custom queryBuilder won't help with that, as the injecting happens afterwards on PHP level.
Long story short, to fix the problem, you can either change how the fields of the union queries are selected, ie ensure that they are not selected with aliases, that way the pagination query fields do not need to be changed at all:
$fields = $table->getSchema()->columns();
$fields = array_combine($fields, $fields);
$query->select($fields);
This will create a list of fields in the format of ['id' => 'id', ...], looks a bit whacky, but it works (as long as there's no ambiguity because of joined tables for example), the SQL would be like id AS id, so your pagination query can then simply reference the fields like Members.id.
Another way would be to select the aliases of the subquery, ie not just select Member__id, which the ORM turns into Member__Member__id when it applies automatic aliasing, but use Members.Member__id, like:
[
'Member__id' => 'Members.Member__id',
// ...
]
That way no automatic aliasing takes place, on SQL level it would select the field like Members.Member__id AS Member__id, and the field would end up as id in your entities, which the eager loader would find and could use for injecting the associated records.
I'm a little surprised I haven't found any information on the following question, so please excuse if I've missed it somewhere in the docs. Using SQL Server (2016 locally and Azure) and EFCore Code First we're trying to create a computed table column with a persisted value. Creating the column works fine, but I don't have a clue how to persist the value. Here's what we do:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]))");
});
And here is what we'd actually like to get in T-SQL:
CREATE TABLE [dbo].[SomeClass]
(
[FirstColumnName] [NVARCHAR](10)
, [SecondColumnName] [NVARCHAR](10)
, [Checksum] AS (CHECKSUM([FirstColumnName], [SecondColumnName])) PERSISTED
);
Can anyone point me in the right direction?
Thanks in advance, Tobi
UPDATE: Based on a good idea by #jeroen-mostert I also tried to just pass the PERSISTED string as part of the formula:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]) PERSISTED)");
});
And also outside of the parentheses:
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName])) PERSISTED");
});
However und somehow surprisingly, the computed column is still generated with Is Persisted = No, so the PERSISTED string simply seems to be ignored.
Starting with EF Core 5, the HasComputedColumnSql method has a new optional parameter bool? stored to specify that the column should be persisted:
modelBuilder.Entity<SomeClass>()
.Property(p => p.Checksum)
.HasComputedColumnSql("checksum([FirstColumnName], [SecondColumnName])", stored: true);
After doing some reading and some tests, I ended up trying the PERSISTED inside the SQL query and it worked.
entity.Property(e => e.Duration_ms)
.HasComputedColumnSql("DATEDIFF(MILLISECOND, 0, duration) PERSISTED");
The generated migration was the following:
migrationBuilder.AddColumn<long>(
name: "duration_ms",
table: "MyTable",
nullable: true,
computedColumnSql: "DATEDIFF(MILLISECOND, 0, duration) PERSISTED");
To check on the database whether it is actually persisted I ran the following:
select is_persisted, name from sys.computed_columns where is_persisted = 1
and the column that I've created is there.
" You may also specify that a computed column be stored (sometimes called persisted), meaning that it is computed on every update of the row, and is stored on disk alongside regular columns:"
modelBuilder.Entity<SomeClass>(entity =>
{
entity.Property(p => p.Checksum)
.HasComputedColumnSql("(checksum([FirstColumnName], [SecondColumnName]), stored: true);
});
This is taken (and slightly modified) from Microsoft Docs.: https://learn.microsoft.com/en-us/ef/core/modeling/generated-properties?tabs=data-annotations#computed-columns
I have been struggling to get the final SAMPLE (ASP.Net, EF Core, SQL) to work against a real SQL Server. Every sample I can find does not use real SQL they always opt for in-memory data store
I changed the connection string
"Data Source=.;Initial Catalog=IS4;Integrated Security=True;"
and ran
dotnet ef database update -c ApplicationDbContext
This created me a SQL database with 25 tables.
I tweaked Startup.cs to change
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(connectionString));
and b.UseSqlite to b.UseSqlServer
.AddConfigurationStore(options =>
{
options.ConfigureDbContext = b =>
b.UseSqlServer(connectionString,
sql => sql.MigrationsAssembly(migrationsAssembly));
})
// this adds the operational data from DB (codes, tokens, consents)
.AddOperationalStore(options =>
{
options.ConfigureDbContext = b =>
b.UseSqlServer(connectionString,
sql => sql.MigrationsAssembly(migrationsAssembly));
// this enables automatic token cleanup. this is optional.
options.EnableTokenCleanup = true;
// options.TokenCleanupInterval = 15;
});
I ran the server with "/seed" on the command line but the Seed functionality doesn't work
First it complains CLIENT can't have a NULL ID when it calls SaveChanges(). If I change the code to add the ID
if (!context.Clients.Any())
{
Console.WriteLine("Clients being populated");
int i = 1;
foreach (var client in Config.GetClients().ToList())
{
var x = client.ToEntity();
x.Id = i++;
context.Clients.Add(x);
}
context.SaveChanges();
}
else
{
Console.WriteLine("Clients already populated");
}
I then get
"Cannot insert the value NULL into column 'Id', table 'IS4.dbo.ClientGrantTypes".
When I watch the video's it says it can be migrated from SQLite to full SQL simply by changing the connection string which is obviously not true, given all the other changes I have done, so I must be doing (or missing) something else.
Any thoughts?
Could it be that all the tables with an "Id INT" column should all be IDENTITY columns and they are not!
I checked the migrations code and it has
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "ApiResources",
columns: table => new
{
Id = table.Column<int>(nullable: false)
.Annotation("Sqlite:Autoincrement", true),
Description = table.Column<string>(maxLength: 1000, nullable: true),
DisplayName = table.Column<string>(maxLength: 200, nullable: true),
I am guessing
.Annotation("Sqlite:Autoincrement", true),
doesn't work with full SQL and therefore all the tables need identity properties setting.
Interestingly if you run the other template to add the AdminUI
dotnet new is4admin
It seems to add a couple of SQL scripts
CREATE TABLE "Clients" (
"Id" INTEGER NOT NULL CONSTRAINT "PK_Clients" PRIMARY KEY AUTOINCREMENT,
"AbsoluteRefreshTokenLifetime" INTEGER NOT NULL,
"AccessTokenLifetime" INTEGER NOT NULL,
which does make them identity columns.
I was faced with this issue today and did a couple of searches online and stumbled upon this https://entityframeworkcore.com/knowledge-base/46587067/ef-core---do-sqlserver-migrations-apply-to-sqlite-
The link pointed out to switch the annotation portion in the migration class UP method after
Id = table.Column(nullable: false)
from
.Annotation("Sqlite:Autoincrement", true);
to
.Annotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn)
And you will need to import
using Microsoft.EntityFrameworkCore.Metadata;
Then you build, and the migration will be successful.
To resolve this particular issue I used SSMS.
right click on table
select script to drop and create
add IDENTITY after the NOT NULL
Execute
However you are correct, it is using sqlite annotations in the sql file and in the migrations.
To fully resolve this issue, you need to create an implementation of all 3 necessary database contexts: identity, persisted grant, and configuration.
That requires an implementation of design time factories for each of those contexts as well.
Then you can run add-migration in the package manager console for each of those contexts, and then run update database, or run the application with the migrate function when seeding.
So to recap:
Create implementations for the 3 db contexts
Create Design time factory implementations for those db contexts
Add the migrations
Update the database with those migrations
I am running migrations on my production system which uses a Postgress database and when I run it I get this error:
django.db.utils.ProgrammingError: multiple primary keys for table "website_experience" are not allowed
But works well on my development SQL database. Here's the model I'm working with:
class Experience (models.Model):
title = models.CharField(max_length = 60)
company = models.CharField(max_length = 60)
city = models.CharField(max_length = 60)
start_date = models.DateField(blank=False, default=datetime.now)
end_date = models.DateField(blank=True, null=True)
description = models.CharField(max_length = 1000)
creative_user = ForeignKey(CreativeUserProfile, models.CASCADE)
Initially, the field creative_user (which is my extended User model) was a primary key, but changed it to be a ForeignKey to express One to Many relationship between One CreativeUser having Many work Experience.
Here is the migration before and after making the change to ForeignKey
class Migration(migrations.Migration):
dependencies = [
('website', '0003_auto_20170510_1436'),
]
operations = [
migrations.CreateModel(
name='Experience',
fields=[
('title', models.CharField(max_length=60)),
('company', models.CharField(max_length=60)),
('city', models.CharField(max_length=60)),
('startDate', models.DateField()),
('endDate', models.DateField(blank=True, null=True)),
('creative_user', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, primary_key=True, serialize=False, to='website.CreativeUserProfile')),
],
),
]
This expresses the creation of Experience model and that creative_user was primary key on model. Then after making it a ForeignKey the migration looked like:
class Migration(migrations.Migration):
dependencies = [
('website', '0004_experience'),
]
operations = [
migrations.AddField(
model_name='experience',
name='id',
field=models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
preserve_default=False,
),
migrations.AlterField(
model_name='experience',
name='creative_user',
field =models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='website.CreativeUserProfile'),
),
]
As I said this all works on dev but migrating on Postgress DB thinks I have multiple primary keys. Can anyone shine some light on what wrong I'm doing?
Thanks.
Maybe is a issue related to the order of migration changes. I had this in my migration file:
operations = [
migrations.AddField(
model_name='unsubscriber',
name='id',
field=models.AutoField(default=None, primary_key=True, serialize=False),
preserve_default=False,
),
migrations.AlterField(
model_name='unsubscriber',
name='phone',
field=models.IntegerField(verbose_name='Teléfono'),
),
]
In the example I wanted to change the primary_key from phone to the new field called id, as you can see this migration is trying to create the new field as PK without changing the old one.
Just changing the order to this must work:
operations = [
migrations.AlterField(
model_name='unsubscriber',
name='phone',
field=models.IntegerField(verbose_name='Teléfono'),
),
migrations.AddField(
model_name='unsubscriber',
name='id',
field=models.AutoField(default=None, primary_key=True, serialize=False),
preserve_default=False,
),
]
It solves the problem.
I hope it helps.
I had the same issue and managed to resolve it by deleting all the migration files from the point the affected table was created, and then run makemigrations and migrate.
Your migration file "0004_experience" created a oneToOneField named "creative_user" that was set as the primary key.
My guess is, Changing from a onToOne to oneToMany relationship called for creation of a new unique field (an auto increment field "id" and set it as the primary key) in the later migration, because "creative_user" was nolonger unique.
Since the latest migration depends on migrations before it, you ended up with two primary keys.
Deleting these conflicting migration files will sort you out.
I have deleted all the migration files except init.py and run migration commands again.
python manage.py makemigrations
python manage.py migrate appName
which solved my problem.