I'm working on a ASP.NET Core Razor pages project that includes IdentityServer and IdentityFramework with various customisations to the identity model -- all quite standard stuff.
What's not standard, though, is that someone has deicded to not use EntityFramework migrations but use DbUp instead.
I can see that changes to the C# model correspond to SQL files -- in the same way that they would correspond to EF migrations.
The mystery is how the DbUp SQL files were produced.
The filename are of the format M<timestamp>_<name>.sql -- where has the initial M come from?
None of the projects has the dbup-add-migration package installed (only dbup-core and dbup-sqlserver).
So: if, say, I edit a model class then what do I do to make the corresponding MyyyyMMdd_name.sql file?
Related
At a conference yesterday, I learned about the importance of putting your database in source control. They showed us how to make a new Database project and import the database.
What I was wondering about is how I would change an existing project running on Entity Framework to utilize the database project's power?
Schema updates have always been done by using Entity Framework Migrations. I get that the Database project will be able to deploy database updates for me and save those update scripts to source control, but I would like to keep Entity Framework for querying my data (if that makes any sense at all).
Is it possible (or even: recommended) to use Entity Framework to access the database but manage the database using a Database project in Visual Studio ? How do you go about this?
I've tried searching for similar questions and using Google to find if anyone else is having the same problem, but no dice so far.
I should also state that I am considering using this in databases that also have stored procedures in them. These are not controlled through Entity Framework at all, and therefore are not in source control yet.
Thank you for your time.
What I was wondering about is how I would change an existing project
running on Entity Framework to utilize the database project's power?
Answer: I suggest you to see this course from Plural Sight : https://www.pluralsight.com/courses/code-first-entity-framework-legacy-databases
Is it possible (or even: recommended) to use Entity Framework to access the database but manage the database using a Database project
in Visual Studio ? How do you go about this?
Answer: Yes, it's possible and recommended. Your data project becomes the source of truth about the structure of your database. This is very powerful to keep control of all the changes and state of your database in one place (Visual Studio). The course from the first answer will teach you how.
I should also state that I am considering using this in databases that
also have stored procedures in them. These are not controlled through
Entity Framework at all, and therefore are not in source control yet.
Answer: I don't see any problem using stored procedures. The tool from the Plural Sight course will create the procedure in your source control and the reverse engineering will create a class/method for easy use of the proc.
I just came across the below alternative, which I didn't test though:
Generate Entity Framework Core classes from a SQL Server database project - .dacpac file
I believe this should be something to be considered
I developed an application like that, having 2 projects: application itself and the SSDT project for the database. The database changes were deployed via change scripts, and EF migrations were disabled in the application.
Everything worked fine, although it did bring a bit of an overhead. For example, it was a bit of a hassle to introduce major database updates / refactorings into the EF layer. For some reason, I was unable to reverse engineer database changes directly into the app, so I had to do it half-manually: creating new project, generate EF context for the entire database, and then copying new / changed files into the main application.
(Then again, it was almost 5 years ago. With luck, EF scaffolding has improved since then.)
I am in process of making the c# .NET standard library which eventually should be published on nuget.
Since library encapsulates communication with database, I wanted to use EF core (Code First) to setup database model.
Users of the library should be able to generate database model for arbitrary database(i.e. user should specify connection string) from code, which is contained in library, i.e. migrations should be located in the library.
My concern is: Is this even supported scenario by EF core.
So far browsing the the internet resulted with no info about this scenario.
My questions are:
Is this scenario somehow supported in EF core.
If not then what are the ways to implement library which contains database model(either in the form of SQL script or migration code), user of the library should be able to specify connection string and generate database model.
I'm doing a small pilot project trying to implement Sql Server Data Tools sqlproj projects in order to better bring our databases under source control. In my organization, we have separate no-trust domains for test environments of various purposes, so these domains of course have their own isolated active directory accounts.
The documentation is still somewhat sparse and I don't really know where to go for more information on this toolset, especially considering the extraordinary amount of churn in Visual Studio's history of database assets.
So far, the only idea I've really had would be to make separate sqlproj projects specifically for the security objects each separate domain, separate from the other schema objects. My hope is that somehow I can tie my actual database schema to those at deploy time and also to somehow switch which security project I'm using in the build. I have no idea if that's feasible though.
Has anyone that uses Visual Studio sqlproj projects had to deal with this? Is there a best practice for this kind of thing?
If you have different settings for each environment then the easiest is to either leave them out and not delete them when you deploy or to have a post deploy script that sets them up manually.
Normally for handling different configurations I would suggest using sql cmd variables (on the properties of the project there is a page for setting these up) but when you create a login you cannot use a variable to create it so that falls over!
There is an example on how to setup a post deploy wrapper for just this case:
http://schottsql.blogspot.co.uk/2013/05/ssdt-setting-different-permissions-per.html
Good luck with ssdt, there are some strange quirks but it enables so much!
Twist to the standard “SQL database change workflow best practices”
Background
ASP.NET/C# Web App
MS SQL
Environments
Production
UAT
Test
Dev
We create patch scripts (XML and sql) that are source controlled in Mercurial. We have cmd line utility that installs patches to DB (utitlity.exe install –patch) from a Release folder the build packages. Patches have meta data that helps with when patch should run and we log patches installed in a table in the target DB. All these were covered in the 3 year old question:
SQL Server database change workflow best practices
Our Problem/Twist
I think this works well for tables, views, functions and stored procedures. We struggle with application configuration data. Here are some touch points on application configurations.
New client. BA performs system study and fit analysis. Out of this comes a configuration word document of what application configurations need to be setup. Note some of these may also come in phases over time. We need to get these new configurations into the system for the developer and client UAT.
Developer works on feature request or bug fix. A new configuration change comes out of that change. The configuration needs to make it into the system for testing and promotion to UAT and up.
QA finds that the developer missed an associated configuration change. That configuration needs to make it into the system for promotion to UAT and up.
Build goes to UAT. Client performs acceptance testing but find they really want to change another unassociated configuration and have it promoted with the changes. In other words they found they want to change a business process by a configuration. The configuration needs to make it into the system for promotion to PRD.
As the client operates in PRD they may tweak application settings. These configurations need to make it into the system for future development and testing.
The general issue is making sure we are accounting for all the configurations and accidently not miss any during promotions which causes grief.
Our Attempts At A Process
a. We have had member of the QA team to write patches (xml and sql) and check those in. This requires a build to make sure those get into the package. With this approach it really just took care of item 1 above and we fell apart on the other items. The nice thing is for the items that made it into the patches it was just an install with the utility.
b. A developer threw together a Config page on the application. All the configurations could be uploaded and downloaded via XML document but it requires the app to be running. For item 1, member of QA team would manually setup configurations in the application and then would download the Config.xml file. This XML file would be used to upload configurations in other environments. We would use text diff tool to look at differences between config.xml files from different environments. This addressed item 1 and the others items but had problems. Problems were not all configurations made it into the XML document (just needs to be fixed by developer), some of the configurations didn’t have a UI in the application so you still had to manually go to the database on some, comparing the XML document with text diff was difficult at time (looked mostly due to sorting but I’m sure there are other issues), XML was not very human readable and finally the XML document did not allow for deleting existing incorrect or outdated configs.
c. Recently we went with option B, but over time for a new client we just started manually tracking configs and promoting them manually by hand (UI and DB) through the promotions. Needless to say lots of human errors.
So we have been looking at solutions. Eventually it would be great to get as much automation in as possible. I’m looking at going with the scripting approach and just focusing on process, documentation and looking at using Redgate data compare in addition to what we had been doing with compare on config.xml. With Redgate we have to create views though and there is no way to create update scripts from that approach except to manually update the scripts. It does at least allow a comparison without the app running. I’m also looking at pulling out the configs from our normal patches and making it a system independent of the build (utility.exe –patch –config). When I say focus on process it will be things like if we compare and find a config change either reported by client or not, we still script it, just means we have to have a process in place to quickly revalidate config install before promoting to the next level. As for documentation looking at making the original QA document a living document instead of just an upfront document. The goal is to try and enhance clarity and reduce missing configurations during promotion. Unfortunately it doesn’t improve speed of delivery.
Does anyone have any recommendations or best practices to pass along. Thanks.
Can I ask exactly what you mean by application configuration. I'm interpreting that as both:
Config files in the web application
Static reference data inside the database
Full disclosure I work for Red Gate. You might be interested in taking a look at Deployment Manager, it's a deployment tool that deploys applications, databases and configuration. It's free for up to 5 projects and target servers.
The approach it uses is to package application code and the database state into packages. These packages can be deployed into dev, test, staging and production environments. The same package is deployed to each environment.
Any application configuration that needs to change between environments is handled in one of the ways below:
Variable substitution in web.config. The tool allows you to specify override values for variables in these files, and set these per environment/server
Substituting the web.config file per environment.
Custom powershell scripts that are run pre/post deploy. You could use these to execute custom SQL based on the environment or server.
Static data within the database, using SQL Source Control's static
data feature. I've written a blog post about how to supply
different sets of static data to different environments/customers.
This allows you to source control the application configurations and deploy them to different environments.
I know this is a duplicate question.
But even i couldn't find any tool to create JPA entity file from database source.
I used eclipse to do this. I only got DTO kind of classes with getters & setters.
Can anyone suggest some tools that create DAO files too.
TIA
You can use Telosys Tools,
an Eclipse plugin working from an existing database with customizable Velocity templates
See: https://sites.google.com/site/telosystools/
Eclipse Marketplace : http://marketplace.eclipse.org/content/telosys-tools
For JPA generation, use the JPA set of templates available on GitHub : https://github.com/telosys-tools
you can use netbeans instead of eclipse. After you create own project, you can select "JPA controller classes from entity classes" in netbeans menu. For example http://ensode.net/roller/dheffelfinger/entry/automated_dao_generation_from_jpa
Minuteproject is a generator tool and can generate JPA2 artifacts from Database.
You can instruct the generator to apply convention for your java code that do not follow your DB convention, but the mapping will be correct. (Example strip DB name prefix; table starting with ADMIN_ such as ADMIN_ENVIRONMENT are Environment (w/out Admin) as java class) For the moment there 20+ conventions that help you reshape your model to be less DB look-and-feel by more Java-OO-friendly.
If you want DAO minuteproject has a track BSLA that generate spring-integrated DAO