Let's consider the What is the best strategy for dealing with hot fixes? question from the Flyway FAQ section. In this question:
Application version 7 (and DB version 7) is deployed in production
Work starts on app version 8
DB version 8 is developed and deployed in the acceptance test environment
Bug is identified in production
DB version 7.1 is developed and must be acceptance-tested
When flyway:migrate will be invoked against the acceptance test environment, it will notice that v8 has already been executed and so that there is no need to execute v7.1.
On one side it makes sense since v7.1 might not be compatible with v8, and it is not up to Flyway to analyze this. Fail-fast is entirely understandable.
On the other side, the only way to deploy v7.1 to the acceptance test environment is to clean the database and run flyway:migrate with target = v7.1, thereby discarding data that might have had its use.
Is there a feature I'm not aware of that handles this case or is clean + migrate.target=v7.1 the only option?
More than a different feature, it's about a different process.
If you do wish to keep your data in your acceptance environment, I would recommend shipping v8 of the database with the hotfix and the actual change can then be v8.1. The features of the v8 schema might remain unused until the corresponding code gets deployed. In most cases however, this causes no harm.
Related
My team and I have been working on a project that is due to release early next year. A burning question that has been plaguing us is how to handle preproduction releases. For example, we have dev and staging environments that we deploy to semi-regularly so management and QA can take a peek at the progress of our project.
Since we have a few separate systems, we're trying to sync and schedule releases between systems so things operate smoothly. In production, we'd take a versioning approach to this, but we aren't at that stage yet.
How do teams handle pre-production releases? My first instinct was to just utilize semver but avoid any major bumps (e.g 1.X.X would be the production release)
Any opinions or advice on this is highly appreciated
You may use SemVer technique appended with a "snapshot" followed by a timestamp. For example if the pilot version in production is 1.0.0 the pre production version can be 1.0.0-SNAPSHOT+'TIMESTAMP'. Where TIMESTAMP="The time whenever the package was generated". In this way the developer(s) will be aware of the feature that was deployed in the staging environment(s).
Let me know if this answers your question.
TLDR
Keep using 0.y.z while in development phase (and your API is not in prod), worry about it when in Production
Long Story
If you DO want to follow semver:
Btw, the answer is based on https://semver.org/:
SemVer 2.0 is designed as set of rules to clearly communicate the nature of the changes contained in your new release.
X.Y.Z
X - changes here represent no backward compatibility
Y - backward compatible new features
Z - backward compatible bugfixes
But it also says on item 4:
"4. Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable."
You may mark your version with a pre-release tag after the version, as suggested by Rohit, (you may, but you don't have to). The full spec could be:
<version core>"-"<pre-release>"+"<build>
Where:
<version core>: X.Y.Z (as stated before)
<pre-release>: your pre-release tag (check semver site for details on what it should contain)
<build>: build identifier
The spec also gives you some hints on how to deal with versioning during development phase in the FAQ:
How should I deal with revisions in the 0.y.z initial development phase?
The simplest thing to do is start your initial development release at 0.1.0 and then increment the minor version for each subsequent release.
"How do I know when to release 1.0.0?If your software is being used
in production, it should probably already be 1.0.0. If you have a
stable API on which users have come to depend, you should be 1.0.0. If
you’re worrying a lot about backwards compatibility, you should
probably already be 1.0.0."
Doesn’t this discourage rapid development and fast iteration? Major
version zero is all about rapid development. If you’re changing the
API every day you should either still be in version 0.y.z or on a
separate development branch working on the next major version.
This basically says that you can stay on 0.y.z if you don't have heavy dependency on your API. If your clients treat your API as their "production" or "production ready", maybe you should move to 1.0.0 and follow the rules.
The moment you go to production you should be already on 1.0.0
I'm working on a legacy SqlServer database with no version control. I've tried importing it into a VS 2017 database project, but it takes more than an hour to load ("your project will be ready after 1200000 operations are completed"), and usually crashes out in less time than it took to load.
Does anyone have any suggestions for a version control system I can try that will cope with real-life databases?
Baseline your Database and call this Version 0.1.0
As you need to make changes to it, like add columns, data etc. Script this and add this to the source control of your choice. Call this file something like:
Version-0.1.1.sql
As you make more and more changes the amounts of files will be added to.
Version-0.1.2.sql
Version-0.1.3.sql
Version-0.1.4.sql
Of course you will test these before you deploy to live. As you are working with what is a legacy system I would probably shy away from investment in expensive tools for what is a legacy system in the first place.
To bring a database to a particular version you would run the scripts in the order. Obviously in each script you have failover etc that handles anything that may go wrong within the scripts.
It is a manual process but the best points are it's cheap, easily to understand, does not require much expense and it's a methodical system to manage change.
Note: Obviously deploy scripts to a UAT version before directly on Live.
I have had a lot of success using flyway with both sql server and postgres. It allows you to create numbered versions as betelgeuce described in his answer, but also offers additional protection of ensuring your earlier versions haven't been changed before deploying any new changes
I have a website on AppEngine that is 99% static. It is running on Python 2.7 runtime. Now the time has come to evolve this webapp, and since I have almost none Python code in it, I'd prefer to write it in Go instead.
Can I change runtime from Python 2.7 to Go, while keeping the project intact? Specifically, I want to keep the same app-ID, the same custom domain attached to it, the same SSL certificate, and so on.
What do I have to do in order to do that? I surely have to change runtime in the app.yaml. Is there anything else?
Bonus question: will such change happen without a downtime?
I'd be grateful for any links to documentation on exactly that (swapping runtime on a live app). I can't find any.
Specify a runtime as well as a new value for version. When deployed you'll have an older version that is Python and a newer version that is Go. There won't be any downtime (same as when deploying a newer version of Python).
Rather than trusting links/docs (that may be out of date or not 100% exactly what you're trying to do), why not create a new GAE-Std project for testing purposes and try it yourself. Having a GAE-Std test project is good for testing new function (especially by other testers who won't have access to the dev environ on your laptop).
The GAE services offer complete code isolation. So it should be possible to simply deploy a new version of the service, which can be written in a different language or even use a different GAE (standard/flex) environment. Personally I didn't go through a language change, but I did go through a split of a single-service app into a multi-service one, I see no reason for which the same principles wouldn't apply.
Maybe develop the new version as a separate app first, to be able to test it properly without risking an accidental impact on the old version and only after that bring the code as a new version in the old app. That'd be using the GAE project isolation. You can, in fact, test the entire version migration as a separate app if you so desire without even touching the existing app. I am using this technique - a separate app ID - to implement a staging environment for my app, completely isolated from my production app, see How to copy / clone entire Google App Engine Project
Make sure to not switch traffic to the new version at deployment time. This keeps the app working with the old version. Test first that the new version works as expected using Targeted routing. Then maybe use Splitting traffic across multiple versions to perform A/B testing with just a small percentage of the traffic going to the new version. Finally, when happy with the results, switch all traffic to the new version.
You need to pay special attention to the app-level configs (dispatch, cron, queue, datastore indexes), shared by all services/versions. They need to be functionally equivalent in the 2 versions. The service isolation doesn't apply to them, only project isolation can ensure no impact to the old version.
There should be no need to make any change to the app ID, custom domain mapping or SSL config. The above mentioned tests should confirm that.
A few potentially interesting posts related to re-working services/modules:
Converting App Engine frontend versions to modules
Google App Engine upgrading part by part
Migrating to app engine modules, test versions first?
Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
I want to test my thick client against my RESTful appengine application. I regularly increment the appengine version number so I need to keep updating my test config. Is there the equivalent of http://latest.application.appspot.com that I could point my config too?
Thanks
Skirting around your question, but in my head, I've stopped thinking of the "version" in the typical software release version (which like you, I started out thinking), but rather, it's "a different application using the same datastore".
I found out the software release version (1.0, 1.1, 1.2 etc) doesn't make much sense because 1) I don't tend to use older versions 2) my main usage would be to regression test, but this doesn't work well, because it's quite possible for a change in your model in v1.1 to break the code in v1.0.
The versions feature comes in hand to have different functional versions. For example, maybe the default application.appspot.com runs production level code, but debug.application.appspot.com has more logging enabled. Perhaps a third version has administrator functionality enabled, etc.
No, there's no way to do this. Versions aren't sequenced - they're all entirely distinct deploys, only one of which is set as the default.
What you are likely looking for is the CURRENT_VERSION_ID environment variable. It stores the deployment revision as dot-separated string: version_name.deployment_revision, e.g. staging.12345678910111213141516. You could just use it directly in your config:
import os
API_VERSION = os.environ['CURRENT_VERSION_ID'].split('.')[1]
This is a subject of common discussion, but through all my research I have not actually found a sound answer to this.
I develop my websites offline, and then launch them live through my hosting account.
I utilize codeigniter, and on that basis there are some fundamental differences between my offline and online copies, namely base urls and database configurations. As such I cannot simply develop and test my websites offline and then upload them as it requires small configuration changes which are easy to overlook and good lead to a none working live website.
The other factor is that when I am developing offline, I might add a database table or a column whilst creating some functionality. When I upload my local developments to my host, they often do not work as I have forgotten to upload the new database structure. Obviously this cannot happen - there cannot be any opportunity for a damaged or broken live website.
Further to this, I'd like to be able to have logs of my development - version control of sorts such that if i develop a feature, and then something else stops working I can easily look backwards to at least see the code changes which could have caused the change.
My fourth requirement is as follows: if i go away on holiday for a week without my development laptop, and then get a bug report, I have no way of fixing it. If i fix it on the live copy, not only is it dangerous, but i'll inevitably not update it on my local copy - as such when i update my live copy next time, that change will be lost. Is there a way that on any computer i can access my development setup, edit and test, launch to the live site, whilst also committing it such that my laptop local copy is up to date.
So yes.. in general im looking for a solution to make my development processes more efficient/suitable. Any ideas?
Thanks
Don't deploy by simply copying. Deploy by using a script (I use Apache Ant) that will automate the copy of specific files for each environment, the replacement of some values, etc.
This just needs rigor. Make a todo list while developing, and check that every modification on the server is done. You might also test the deploy procedure on a pre-production server which has an similar configuration as the production server, make sure everything is OK, and then apply the same, tested procedure on the production server
Just use a version control system. SVN or Git are two free candidates.
Make your version control server available from anywhere. If it's an open-source project, free hosting solutions exist. Of course, if you don't have a development computer wvailable, you'll have to checkout the whole project, and probably install some tools to be able to develop, test and deploy. Just try to make it as easy as possible, or always have your laptop available. If you plan to work, have your toolbox with you. If you don't plan to work, then don't work. When you have finished some development, commit to the server. When you go back to your laptop, update your working copy from the server.
Small additions and clarifications to JB
Use any VCS, which can work (in a good way) with branches - your local and prod systems are good candidates for separate branches, where you share common code but have branch-specific config. It'll require some changes in your everyday workflow (code in "test", merge finished with "prod", deploy /by tools, not hand/ only after merge...), but it's fair price
Changing of workflow, again. As JB noted - don't deploy by hand, don't deploy wrong branch, don't deploy "prod" before finished merge. But now build-tools are rather smart, you can check such pre-condition inside builder
Just use VCS, maybe DVCS will be somehow better. I say strong "No-no" for Git as first VCS, but you have wide choice even without it - SVN (poor branch|merge comparing to DVCS), Bazaar (not a tool of my dream, but, who knows), Mercurial, Fossil SCM, Monotone
Don't work on live, never do anyting outside your SCM. One source of changes is a rule of happy developer. Or don't work at all at free-time, or have codebase always reacheable for you (free code-hosting /GoogleCode, SourceForge, BitBucket, Github, Assembla, LaunchPad/ or own server), get it as needed, change, save, deploy