I'm work on project with django framework and use control version system to synchronize my code with other peoples. But i don't know how organize work with database.
In django, any people, worked on project, may changes django models, and tell 'syncdb' to synchronize model objects with db.
But other people don't about this changes, and it's code revision may not works.
Please, tell me some ways to solve this problem (maybe, different db or something another).
Thanks, and excuse my english :)
You have to actually talk to the people on your project.
If someone changes any database model, they have to actually tell everyone else about the change. This is not a Django problem.
Think of any SQL database -- without Django. When the DBA drop's a table, they have to tell everyone that they changed the database. Otherwise all programs that use the table break.
The model definition is special, and whoever can change this must tell everyone else.
You must have an initial backup of the DB under verison control. And after that you have to put all the modification scripts on the same version control. Something like this:
/Database (in the repository)
Initial backup
Script1_date.sql
Script2_date.sql
...
I'm not sure to understand your problem; but remember that on Django, syncdb only creates new tables. It doesn't alter an existing table.
If, for example you just add a new field, a syncdb won't do anything.
Actually, looking at the alternatives, I'm often surprised that nobody mentions South
http://south.aeracode.org/
It seems to be the best migration app out there... perhaps I am missing something important, but I find it pretty nice to work with...
take also a look at deltasql.
you can test it at http://www.gpu-grid.net/deltasql (username: admin password: testdbsync )
and download from http://sourceforge.net/projects/deltasql
ciao :-)
I'm curious...what happens if you put your MDF and LDF files under source control? Of course if your tables are empty and you just have the structure of the database...
Sounds like you want migrations.
As an example:
http://www.aswmc.com/dbmigration/
You may also want to add functional unit tests that actually test that the schema is as expected, that way when the tests fail, you can see that it is a schema change, and audit whether it will affect other parts of the app. If it doesn't, fix your test to take into account the new schema.
Related
I am trying with the help of Entity Framework to set up this without having to deal with the code-related part of SQL.
I created a model and added a migration via package manager console and it all worked well it updated and created the table.
The thing I want to ask is how does the entity know which migration I want to add.
I used:
add-migration (and put here the name of the migration file)
But the thing I don't understand is how does it know which model I want for my table?
Or put it in other words if I would have 2 models before I did any migrations which model would get chosen?
Would really appreciate it if someone could help me out.
Thanks in advance
Seems you are using entity framework migrations and got confused how it works. Here is the explanations:
Question: But the thing I don't understand how does it know which model I want for my table?
If you look into your project folder there is the directory
Migrations. Inside it all the migrations history logs written
into.When we made any changes on data model, EF Core compares the current model against a snapshot of the old model to determine the
differences, and generates migration source files; the files can be
tracked in your project's source control like any other source file.
Once a new migration has been generated, it can be applied to a database in various ways. EF Core records all applied migrations in a
special history table, allowing it to know which migrations have been
applied and which haven't
Question: If I would have 2 models before I did any migrations which model would get chosen?
As said earlier, as it keep track previous migrations history, so in your old model it compares the differences and overrite latest
changes that were not written on older files. This is how it works.
Hope above explanations guided you accordingly and redeem your confusions. You can also have a look on official documents here
I tried adding a new table into one of my models in Web2Py. In addition I added a new field to an existing table. I tried loading a page that used those tables and it didn't work, claimed those things don't exist. Okay so I get migrations to False here.
db = DAL('sqlite://storage.sqlite',pool_size=1,check_reserved=['all'], migrate = False)
Reloaded the page, no change. Then I tried doing something like this in the tables it wouldn't understand
db.define_table(....,migrate=False,fake_migrate=True)
and I changed the DAL call to be
db = DAL(...,fake_migrate_all=True)
As the web2py manual said. Still no change. So then I said well okay I will have to dumb the whole database. So I took everything out of my database folder and I tried to reload it with a clean slate.
Now it just doesn't load at all.
According to database administration none of the tables exist although if I check again in the database folder they are all there. If I try to load the application it immediately reports that none of my called tables exist. I have all the code backed up on a repo but I can't uninstall the current app because I don't have that kind of read access on the server this is running on.
Is there anything I can do?
Edit: By the way this is happening on SQLite
Have you already tried, besides dumbs DB, clean up the database folder? If you do not do this, web2py will goes insane, cuz the files says that there are tables, but db don't. Besides, take a look in here, about fixing broken migrations and some caveats about sqllite.
Is there any way to add a SQL Server Database Diagram to source control? I can't seem to find a way to script it out of the database. If so, is there a way to get that diagram into a Visual Studio Database Project for easy deployment?
to script it to a file try:
http://www.codeproject.com/KB/database/ScriptDiagram2005.aspx
I would not do this.
I've been allowed to publish my variation on a set of scripts that do just that, providing easy two-way import/export between files and diagrams stored in a database.
https://github.com/timabell/database-diagram-scm
Just run the batch file, pointing at your database of choice and you'll get a set of files, one for each diagram. Unfortunately the data is still binary, but it's a start.
It builds on what others have already done, definitely a shoulders of giants job. :-)
I don't know what the advantage of storing the diagram in source control would be. The database diagrams put an illustration to your database relationships which should be defined elsewhere. So long as you put the creation scripts for your DB in source safe, the diagram should render just fine when you create it and add your tables to it.
There isn't really an easy way of doing this. I typically do one of a few things for this.
Simply print the document to PDF using something like CutePDF
Use Visio and the Reverse Engineering option to generate the document, then save the visio file
Use Enterprise Architect or similar tool for the process.
I personally use option 3, due to the lifecycle that I take my applications through. But the real thing is what are you looking to store, if it is a static version of the database diagram, any of the above are valid.
Can anyone provide some real examples as to how best to keep script files for views, stored procedures and functions in a SVN (or other) repository.
Obviously one solution is to have the script files for all the different components in a directory or more somewhere and simply using TortoiseSVN or the like to keep them in SVN, Then whenever a change is to be made I load the script up in Management Studio etc. I don't really want this.
What I'd really prefer is some kind of batch script that I can run periodically (nightly?) that would export all the stored procedures / views etc that had changed in a given timeframe and then commit them to SVN.
Ideas?
Sounds like you're not wanting to use Revision Control properly, to me.
Obviously one solution is to have the
script files for all the different
components in a directory or more
somewhere and simply using TortoiseSVN
or the like to keep them in SVN
This is what should be done. You would have your local copy you are working on (Developing new, Tweaking old, etc) and as single components/procedures/etc get finished, you would commit them individually until you have to start the process over.
Committing half-done code just because it's been 'X' time since it was last committed is sloppy and guaranteed to cause anyone else using the repository grief.
I find it best to treat Stored Procedures just like any other compilable code: Code lives in the repository, you check it out to make changes and load it in your development tool to compile or deploy the code.
You can create a batch file and schedule it:
delete the contents of your scripts directory
using something like ExportSQLScript to export all objects to script/scripts
svn commit
Please note: That although you'll have the objects under source control, you'll not have the data or it's progression (is that a renamed field, or 1 new field and 1 deleted?).
This approach is fine for maintaining change history. But, of course, you should never be automatically committing to the "production build" (unless you like broken builds).
Although you didn't ask for it: This approach also won't produce a set of scripts that will upgrade a current DB. You'll only have initial creation scripts. Recording data progression and creation upgrade scripts is beyond basic source control systems.
I'd recommend Redgate SQL Compare for this - it allows you to compare database versions and generate change scripts - it's also fairly easily scriptable.
Based on your expanded question, you really want to use DDL triggers. Check out this article that details how to create a changelog system for your database.
Not sure on your price range, however DB Ghost could be an option for you.
I don't work for this company (or own the product) but in my researching of the same issue, this product looked quite promising.
I should've been a little more descriptive. The database in question is for an internal ERP system and thus we don't have many versions of our database, just Production/Testing/Development. When we've done a change request, some new fancy feature or something, we simply execute a script or series of scripts to update the procedures in question on the Testing database, if that is all good, then we do the same to Production.
So I'm not really after a full schema script per se, just something that can keep track of the various edits to the stored procedures over time. For example, PROCESS_INVOICE does stuff. It gets updated in some minor way in March. Some time later in say May it is discovered that in a rare case customers get double invoiced (or some other crazy corner case). I'd like to be able to see what has happened over time to this procedure. Currently the way the development environment is setup here I don't have that, which I'm trying to change.
I can recommend DBPro which is part of Visual Studio Team Edition. Have been using it for a few months for storing all parts of the database in Team Foundation Server as well as for deployment and database compares, etc.
Of course, as someone else mentioned, it does depend on your environment and price range.
I wrote a utility for dumping all of the relevant parts of my db into a directory structure that I use SVN on. I never got around to trying to incorporate it into the Manager but, if you're interested, it's here: http://www.reluctantdba.com/dbas-and-programmers/sqltools/svnforsql2005.aspx
It's free and, since I regularly run it, you know any bugs get fixed quickly.
You can always try integrating SourceSafe with SQL Server. Here's a quick start : link . To work with it you've got to have Managment Studio Developers Edition.
I'm considering using Django for a project I'm starting (fyi, a browser-based game) and one of the features I'm liking the most is using syncdb to automatically create the database tables based on the Django models I define (a feature that I can't seem to find in any other framework).
I was already thinking this was too good to be true when I saw this in the documentation:
Syncdb will not alter existing tables
syncdb will only create tables for models which have not yet been installed. It will never issue ALTER TABLE statements to match changes made to a model class after installation. Changes to model classes and database schemas often involve some form of ambiguity and, in those cases, Django would have to guess at the correct changes to make. There is a risk that critical data would be lost in the process.
If you have made changes to a model and wish to alter the database tables to match, use the sql command to display the new SQL structure and compare that to your existing table schema to work out the changes.
It seems that altering existing tables will have to be done "by hand".
What I would like to know is the best way to do this. Two solutions come to mind:
As the documentation suggests, make the changes manually in the DB;
Do a backup of the database, wipe it, create the tables again (with syncdb, since now it's creating the tables from scratch) and import the backed-up data (this might take too long if the database is big)
Any ideas?
Manually doing the SQL changes and dump/reload are both options, but you may also want to check out some of the schema-evolution packages for Django. The most mature options are django-evolution and South.
EDIT: And hey, here comes dmigrations.
UPDATE: Since this answer was originally written, django-evolution and dmigrations have both ceased active development and South has become the de-facto standard for schema migration in Django. Parts of South may even be integrated into Django within the next release or two.
UPDATE: A schema-migrations framework based on South (and authored by Andrew Godwin, author of South) is included in Django 1.7+.
As noted in other answers to the same topic, be sure to watch the DjangoCon 2008 Schema Evolution Panel on YouTube.
Also, two new projects on the map: Simplemigrations and Migratory.
One good way to do this is via fixtures, particularly the initial_data fixtures.
A fixture is a collection of files that contain the serialized contents of the database. So it's like having a backup of the database but as it's something Django is aware of it's easier to use and will have additional benefits when you come to do things like unit testing.
You can create a fixture from the data currently in your DB using django-admin.py dumpdata. By default the data is in JSON format, but other options such as XML are available. A good place to store fixtures is a fixtures sub-directory of your application directories.
You can load a fixure using django-admin.py loaddata but more significantly, if your fixture has a name like initial_data.json it will be automatically loaded when you do a syncdb, saving the trouble of importing it yourself.
Another benefit is that when you run manage.py test to run your Unit Tests the temporary test database will also have the Initial Data Fixture loaded.
Of course, this will work when when you're adding attributes to models and columns to the DB. If you drop a column from the Database you'll need to update your fixture to remove the data for that column which might not be straightforward.
This works best when doing lots of little database changes during development. For updating production DBs a manually generated SQL script can often work best.
I've been using django-evolution. Caveats include:
Its automatic suggestions have been uniformly rotten; and
Its fingerprint function returns different values for the same database on different platforms.
That said, I find the custom schema_evolution.py approach handy. To work around the fingerprint problem, I suggest code like:
BEFORE = 'fv1:-436177719' # first fingerprint
BEFORE64 = 'fv1:-108578349625146375' # same, but on 64-bit Linux
AFTER = 'fv1:-2132605944'
AFTER64 = 'fv1:-3559032165562222486'
fingerprints = [
BEFORE, AFTER,
BEFORE64, AFTER64,
]
CHANGESQL = """
/* put your SQL code to make the changes here */
"""
evolutions = [
((BEFORE, AFTER), CHANGESQL),
((BEFORE64, AFTER64), CHANGESQL)
]
If I had more fingerprints and changes, I'd re-factor it. Until then, making it cleaner would be stealing development time from something else.
EDIT: Given that I'm manually constructing my changes anyway, I'll try dmigrations next time.
django-command-extensions is a django library that gives some extra commands to manage.py. One of them is sqldiff, which should give you the sql needed to update to your new model. It is, however, listed as 'very experimental'.
So far in my company we have used the manual approach. What works best for you depends very much on your development style.
We generally have not so many schema changes in production systems and somewhat formalized rollouts from development to production servers. Whenever we roll out (10-20 times a year) we do a fill diff of the current and the upcoming production branch reviewing all the code and noting what has to be changed on the production server. The required changes might be additional dependencies, changes to the settings file and changes to the database.
This works very well for us. Having it all automated is a niche vision but to difficult for us - maybe we could manage migrations but we still would need to handle additional library, server, whatever dependencies.
Django 1.7 (currently in development) is adding native support for schema migration with manage.py migrate and manage.py makemigrations (migrate deprecates syncdb).