In my asp net core 3.1-project, Im trying out MS Identity, by adding Authentication when I create a new template (Asp net core 3.1+react)
I was hoping to add my Identity tables to a different schema, using this simple Guide,
ie
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.HasDefaultSchema("notdbo");
}
But when I run "Update database" in package manager, it still creates a database in dbo-schema
Here is some reasons can cause this:
The user on on connection string should have access to the schema
If you have pending changes on model, this can cause issue. As initial migrations was based on the dbo schema and now you are trying to change the schema when having pending changes. Try to:
Option1:
Add-Migration (as mentioned in the comment) - This creates a new migration class as per specified name with the Up() and Down() methods.
Update-Database
Option2:
Remove the pending changes to the model
Run update-database. This will Executes the last migration file created by the Add-Migration command and applies changes to the database schema.
Change the schema, and you should be able to recreating the database
You need to add a valid migration first, before you do the Update-Database. The catch is that there is already one migration pending in the code, so that's why your change will not be detected.
I have an ASP.NET Core application that use SQL Server and EF Core. It contains two tables. I successfully created an Entity Framework Core migration and updated the database with it for the first table called Pies.
Then after I added another table called Feedbacks, I created a second migration.
However, when I attempt to update the database, it fails and I get this error:
Error Number:2714,State:6,Class:16
There is already an object named 'Pies' in the database.
Here is the DbContext class:
public class AppDbContext : DbContext
{
public AppDbContext(DbContextOptions<AppDbContext> options) : base(options)
{
}
public DbSet<Pie> Pies { get; set; }
public DbSet<Feedback> Feedbacks { get; set; }
}
Since you have messed up the record history of EF Core migrations by deleting a migration file and your snapshot is tainted as well because you re-ran the migrations after deleting one.
Here are the steps you should take:
Delete all remaining migrations
Delete the database
Create a new initial migration
Create a new database.
In the future to avoid these error's use a version control system like git. And before creating any migrations commit. Note you will need to remove the incorrectly applied migration to your database from the _EfMigrations table in your database, since git doesn't version control your database.
This error occurs when there is a difference between the data in the database table and the files in the database.
You most likely created the database once and then deleted the migraine folder
I'm enjoying the new Build tool in Visual Studio Online. Allows me to do almost everything that I do my local build server. But one thing that I'm missing is integration database tests: for every build run I re-create test database from scripts and run DB-tests against it.
In Visual Studio Online I can't seem to find any database instance available for my needs.
I have tried creating Azure SQL database (via PowerShell) for every build run and then delete it after the build is complete. But it takes forever (comparing to the rest of the build process) to create a database. And even when PowerShell scripts are done, database is not yet ready to accept requests - I need to constantly check if it is actually ready. So this scenario becomes too complex and not reliable.
Are there other options to do database (SQL Server) integration tests in Visual Studio Online?
Update: I think I'm not very clear of what I need - I need a free (very cheap) SQL Server instance to connect to that runs on build agent in VSO. Something like SQL Express or SQL CE or LocalDB, where I can connect to and re-create database to run C# tests against. Re-creating database or running tests is not a problem, having a valid connection string is a problem.
Update Oct 2016: I've blogged about how I do integration testing in VSTS
The TFS build servers come with MSSQL Server 2012 and MSSQL Server 2014 LocalDBs preinstalled.
Source: TFS Service - Software on the hosted build server
So, just put the following one-liner into your solution's post-build event to create a MYTESTDB LocalDB instance for your needs. This will allow you to connect to (LocalDB)\MYTESTDB an run the database integration tests just fine.
"C:\Program Files\Microsoft SQL Server\120\Tools\Binn\SqlLocalDB.exe" create "MYTESTDB" 12.0 -s
Source: SqlLocalDB Utility
In Azure DevOps, with .net Core and EF Core, I use a different technique.
I use a SQLite in memory database to execute both Integration and End to End tests.
Currently in .net Core you can use both InMemory database and SQLite with in memory option, to run any integration test in the default Azure DevOps CI Agent.
InMemory: https://learn.microsoft.com/en-us/ef/core/miscellaneous/testing/in-memory
Note that the InMemory database is not a relational database, it is a multi-purpose one, and just to mention one limitation:
InMemory will allow you to save data that would violate referential
integrity constraints in a relational database
SQLite in memory mode https://learn.microsoft.com/en-us/ef/core/miscellaneous/testing/sqlite
This approach offers a more realistic platform to test against.
Now, I went a bit further, I didn't want to just be able to run integration tests with database dependency in Azure DevOps, I wanted to also be able to host my WebAPIs in the CI Agent, and to share the database between the API DBcontext and my Persister object (Persister object is a helper class that allow me to automatically generate any kind of entity and save them to the database).
A quick note on Integration Tests and Ent to End tests:
Integration Tests
An example of integration test involving a database, could be a test of the Data Access Layer. In this case, normally, you would create a DBContext when starting a test, fill the target database with some data, use the component under test to manipulate the data, and again use the DBContext to make sure the assertions are satisfied.
This scenario is quite straight forward, in the same code you can share the same DBContext to generated the data and inject it to the component.
End to End Tests
Imagine you have like in my case a RESTful .net Core WebAPI you want to test, making sure all your CRUD operations are working as expected, and you want to test that filtering, pagination and so on are also correct.
In this case, it much more complex share the same DBContext between test (data setup and/or verification) and the WebAPI stack.
Before .net EF Core and WebHostBuilder
So far, the only way I knew was possible, was to have a dedicated server, VM or docker image, responsible serve the API, which had to be also accessible from the web or Azure DevOps.
Setup my integration tests to either re-create the DB, or be clever/limited enough to ignore completely the existing data, and make sure that each test was resilient to data corruption and fully reliable (no false negative or positive results).
Then I had to configure my build definition to run the tests.
Leveraging SQLite in memory with cache=shared and WebHostBuilder
Below I first describe the two majour technologies I use, then I add some code to show how to do it.
SQLite file::memory:?cache=shared
SQLite allow you to work in memory, instead of using a traditional file, this already gives us a huge performance boost, removing the I/O bottleneck, but on top of this, using the option cache=shared, we can use multiple connections within the same process to access the same data. If you need more than one database you can specify a name.
More info: https://www.sqlite.org/inmemorydb.html
WebHostBuilder
.net Core offers Host builders, WebHostBuilder allow us to create a server that startup and host our WebAPI, so that can be reached like if they were hosted on a real server.
When you use the WebHostBuilder in a test class, this two, are living within the same process.
More info: https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.hosting.webhostbuilder?view=aspnetcore-2.2
The Solution
When initialising an E2E test, create a new client to connect the api, create a dbcontext that you will use to seed the database and maybe assert.
Test initialisation:
[TestClass]
public class CategoryControllerTests
{
private TestServerApiClient _client;
private Persister<Category> _categoryPersister;
private Builder<Category> _categoryBuilder;
private IHouseKeeperContext _context;
protected IDbContextTransaction Transaction;
[TestInitialize]
public void TestInitialize()
{
_context = ContextProvider.GetContext();
_client = new TestServerApiClient();
ContextProvider.ResetDatabase();
_categoryPersister = new Persister<Category>(_context);
_categoryBuilder = new Builder<Category>();
}
[TestCleanup]
public void Cleanup()
{
_client?.Dispose();
_context?.Dispose();
_categoryPersister?.Dispose();
ContextProvider.Dispose();
}
[...]
}
TestServerApiClient class:
public class TestServerApiClient : System.IDisposable
{
private readonly HttpClient _client;
private readonly TestServer _server;
public TestServerApiClient()
{
var webHostBuilder = new WebHostBuilder();
webHostBuilder.UseEnvironment("Test");
webHostBuilder.UseStartup<Startup>();
_server = new TestServer(webHostBuilder);
_client = _server.CreateClient();
}
public void Dispose()
{
_server?.Dispose();
_client?.Dispose();
}
}
ContextProvider class is used to generate the DBContext which can be used to seed data or perform database queries for assertions.
public static class ContextProvider
{
private static bool _requiresDbDeletion;
private static IConfiguration _applicationConfiguration;
public static IConfiguration ApplicationConfiguration
{
get
{
if (_applicationConfiguration != null) return _applicationConfiguration;
_applicationConfiguration = new ConfigurationBuilder()
.AddJsonFile("Config/appsettings.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();
return _applicationConfiguration;
}
}
private static ServiceProvider _serviceProvider;
public static ServiceProvider ServiceProvider
{
get
{
if (_serviceProvider != null) return _serviceProvider;
var serviceCollection = new ServiceCollection();
serviceCollection.AddSingleton<IConfiguration>(ApplicationConfiguration);
var databaseType = ApplicationConfiguration?.GetValue<DatabaseType>("DatabaseType") ?? DatabaseType.SQLServer;
_requiresDbDeletion = databaseType == DatabaseType.SQLServer;
IocConfig.RegisterContext(serviceCollection, null);
_serviceProvider = serviceCollection.BuildServiceProvider();
return _serviceProvider;
}
set
{
_serviceProvider = value;
}
}
/// <summary>
/// Generate the db context
/// </summary>
/// <returns>DB Context</returns>
public static IHouseKeeperContext GetContext()
{
return ServiceProvider.GetService<IHouseKeeperContext>();
}
public static void Dispose()
{
ServiceProvider?.Dispose();
ServiceProvider = null;
}
public static void ResetDatabase()
{
if (_requiresDbDeletion)
{
GetContext()?.Database?.EnsureDeleted();
GetContext()?.Database?.EnsureCreated();
}
}
}
IocConfig class is an helper class I use in my framework to setup the dependency injection. The menthod used above, RegisterContext, is responsile to register the DBContext and set it up as desired, and because this is the same class used by the WebAPI, uses the configuration DatabaseType to determine what to do.
Inside this class probably you can find most of the "complexity".
When using SQLite in memory, you have to remember that:
The connection is not opened and closed automatically like when using SQL Server (that's why i used: context.Database.OpenConnection();)
If no connection is active, the database is deleted (that's why I used services.AddSingleton<IHouseKeeperContext>(s ... it is important that one connection is left open so that the database is not destroyed, but on the other hand you have to be careful to close all connections when a test ends, so that the database is eventually destroyed and the next test will correctly create a new empty one.
The rest of the class handles the SQL Server configuration for both Production and Testing setup. I can at any time setup the tests to use a real instance of SQL Server, all tests will keep being fully independend from the others but it will definitely be slow, and maybe suitable only for a nightly build (if needed, and it depends on the size of your system).
public class IocConfig
{
public static void RegisterContext(IServiceCollection services, IHostingEnvironment hostingEnvironment)
{
var serviceProvider = services.BuildServiceProvider();
var configuration = serviceProvider.GetService<IConfiguration>();
var connectionString = configuration.GetConnectionString(Constants.ConfigConnectionStringName);
var databaseType = DatabaseType.SQLServer;
try
{
databaseType = configuration?.GetValue<DatabaseType>("DatabaseType") ?? DatabaseType.SQLServer;
}catch
{
MyLoggerFactory.CreateLogger<IocConfig>()?.LogWarning("Missing or invalid configuration: DatabaseType");
databaseType = DatabaseType.SQLServer;
}
if(hostingEnvironment != null && hostingEnvironment.IsProduction())
{
if(databaseType == DatabaseType.SQLiteInMemory)
{
throw new ConfigurationErrorsException($"Cannot use database type {databaseType} for production environment");
}
}
switch (databaseType)
{
case DatabaseType.SQLiteInMemory:
// Use SQLite in memory database for testing
services.AddDbContext<HouseKeeperContext>(options =>
{
options.UseSqlite($"DataSource='file::memory:?cache=shared'");
});
// Use singleton context when using SQLite in memory if the connection is closed the database is going to be destroyed
// so must use a singleton context, open the connection and manually close it when disposing the context
services.AddSingleton<IHouseKeeperContext>(s => {
var context = s.GetService<HouseKeeperContext>();
context.Database.OpenConnection();
context.Database.EnsureCreated();
return context;
});
break;
case DatabaseType.SQLServer:
default:
// Use SQL Server testing configuration
if (hostingEnvironment == null || hostingEnvironment.IsTesting())
{
services.AddDbContext<HouseKeeperContext>(options =>
{
options.UseSqlServer(connectionString);
});
services.AddSingleton<IHouseKeeperContext>(s => {
var context = s.GetService<HouseKeeperContext>();
context.Database.EnsureCreated();
return context;
});
break;
}
// Use SQL Server production configuration
services.AddDbContextPool<HouseKeeperContext>(options =>
{
// Production setup using SQL Server
options.UseSqlServer(connectionString);
options.UseLoggerFactory(MyLoggerFactory);
}, poolSize: 5);
services.AddTransient<IHouseKeeperContext>(service =>
services.BuildServiceProvider()
.GetService<HouseKeeperContext>());
break;
}
}
[...]
}
Sample Test, where first I use the persister to generated data which is seeded in the database, then I use the API to get data, the test can be also reversed, using a POST request to set data and then using the DBContext to read the db and make sure the creation was successful.
[TestMethod]
public async Task GET_support_orderBy_Id()
{
_categoryPersister.Persist(3, (c, i) =>
{
c.Active = 1 % 2 == 0;
c.Name = $"Name_{i}";
c.Description = $"Desc_i";
});
var response = await _client.GetAsync("/api/category?&orderby=Id");
var categories = response.To<List<Category>>();
Assert.That.All(categories).HaveCount(3);
Assert.IsTrue(categories[0].Id < categories[1].Id &&
categories[1].Id < categories[2].Id);
response = await _client.GetAsync("/api/category?$orderby=Id desc");
categories = response.To<List<Category>>();
Assert.That.All(categories).HaveCount(3);
Assert.IsTrue(categories[0].Id > categories[1].Id &&
categories[1].Id > categories[2].Id);
}
Conclusions
I love the fact that I can run E2E tests in Azure DevOps for free, performances are incredibly good and this gives me a lot of confidence, ideal when you want to setup a continuous delivery environment.
Here is a screenshot of part of the build execution of this code in Azure DevOps (free version).
Sorry this ended up being longer than expected.
There is a "Redgate SQL CI" extension for VSTS in the marketplace you may want to try. See this link for details:
Within the extension, there are four actions available:
•Build – builds your database into a NuGet package from the database
scripts folder in source control
•Test – runs your tSQLt tests against the database
•Sync – synchronizes the package to an integration database
•Publish – publishes the package to a NuGet stream
You should push the integration tests (anything that needs an instance of your application) to be run in an environment as part of your release pipeline.
In your build just do compile and unit tests. If that competes you should trigger a Release and as part of your release pipeline your first step should be to deploy your database to an azure server.
Instead of trying to use SQL Azure you can create a VM in azure that already exists that has SQL server installed. Use remote scripting to deploy the database and execute your tests.
Even if you are not using the release tools to release this would work for you.
I ran ELMAH sql scripts in test DB(It created ELmah_Error table and 3 stored procedures) and configured ELMAH in MVC application using Nuget.
I modified web.config as specified and I'm able to log exceptions into
http://mysite/elmah.axd
But, instead i want to log the exceptions into Sql Server.
I added below class to achieve that
public class ElmahHandleErrorAttribute : System.Web.Mvc.HandleErrorAttribute
{
public override void OnException(System.Web.Mvc.ExceptionContext context)
{
LogException(e);
}
private static void LogException(Exception e)
{
// Call to Database and insert the exception info
}
}
Final step was to:
public static void RegisterGlobalFilters(GlobalFilterCollection filters)
{
filters.Add(new ElmahHandleErrorAttribute ());
}
Is it the correct way to use ELMAH to log all exceptions or AM I missing something?
Once you have the database setup, all you need to do is add the following to the <elmah> section your web.config to setup the Elmah to log to the SQL Database:
<elmah>
<errorLog type="Elmah.SqlErrorLog, Elmah" connectionStringName="<DBConnString>"
applicationName="<YourApp>"
</elmah>
Replace <DBConnString> and <YourApp> with appropriate values for your configuration.
Once you have done this you will not need to use your custom ElmahHandleErrorAttribute class.
I am not sure which NuGet package you installed, but I would recommend using the Elmah.MVC package as it integrates Elmah into MVC exceptionally well by setting up all of the ErrorHandlers and ErrorFilters for you.
I have winforms application which uses nhibernate and active records to access the database. My question is how do I update the database schema through the application? The CreateSchema method deletes the database and creates a new one. Is there anyway to update just few parts of the database through some method.
You can use schemaupdate, providing you have the configuration
public void CanUpdateDatabase()
{
SchemaUpdate su = new SchemaUpdate(config);
su.Execute(true, true);
}
it works depending on the db dialect ( in my experience works well with MSSQL+Oracle )