I'm having the same issue as nuget feeds and promotions, eight years later!
In this case I'm talking more generically; we're using ProGet as our package manager, and have nugets, universal packages, and even some docker containers to consider in the package promotion process.
One of the ideas is to have several Nuget feeds; a ci feed where every successful integration publishes a package, a qa feed that you only publish versions you want qa to test and then a release feed, where you copy only packages from the qa feed that they successfully tested.
So, say we have a build in the ci feed that works, it's version 1.2.3-ci-xyz. We want to promote that to the QA feed, without a rebuild, and re-package it as 1.2.3-rc-1. That package passes QA and is ready to be promoted into the prod feed, with no rebuild, and ship to production. It should ship as 1.2.3. (right?)
The question is, if we're not doing any rebuilds, the package binaries will still have the version 1.2.3-ci-xyz. That'll show up anywhere a version is displayed or queried in the app.
And that's where I get stuck. What's the proper pattern here? Does it matter what version is shipped, as long as we know what it is?
meaning, we promote 1.2.3-ci-xyz from lower feeds to higher feeds, without repackaging with different versions?
Wouldn't it be incorrect for package 1.2.3 to include a binary 1.2.3-ci-xyz?
do we always build with the next 3-digit number, and forget about the ci/rc suffix?
I'll share this answer from our internal support channel :)
This is how we (Inedo) typically handle this in our libraries. The short answer is:
We set the Assembly Version to Major.Minor.Patch
We set the Assembly File Version to Major.Minor.Patch.Build
We set the Package version to Major.Minor.Patch-ci.Build (we then repackage to Major.Minor.Patch-rc.Build then to Major.Minor.Patch)
We also use the Assemble Informational Version to display a friendly version (ex: Version 6.0.0 (Build 36-v6))
This allows us to repackage without rebuilding. We also will detect these pre-release dependencies during our product builds and prevent them from being released to production. You can see our ProGet v6 build as an example: ProGet 6.0.0 Build 36.
The longer answer I feel is answered pretty well in our blog post Best Practices for Versioning NuGet Packages in the Enterprise.
Related
My team and I have been working on a project that is due to release early next year. A burning question that has been plaguing us is how to handle preproduction releases. For example, we have dev and staging environments that we deploy to semi-regularly so management and QA can take a peek at the progress of our project.
Since we have a few separate systems, we're trying to sync and schedule releases between systems so things operate smoothly. In production, we'd take a versioning approach to this, but we aren't at that stage yet.
How do teams handle pre-production releases? My first instinct was to just utilize semver but avoid any major bumps (e.g 1.X.X would be the production release)
Any opinions or advice on this is highly appreciated
You may use SemVer technique appended with a "snapshot" followed by a timestamp. For example if the pilot version in production is 1.0.0 the pre production version can be 1.0.0-SNAPSHOT+'TIMESTAMP'. Where TIMESTAMP="The time whenever the package was generated". In this way the developer(s) will be aware of the feature that was deployed in the staging environment(s).
Let me know if this answers your question.
TLDR
Keep using 0.y.z while in development phase (and your API is not in prod), worry about it when in Production
Long Story
If you DO want to follow semver:
Btw, the answer is based on https://semver.org/:
SemVer 2.0 is designed as set of rules to clearly communicate the nature of the changes contained in your new release.
X.Y.Z
X - changes here represent no backward compatibility
Y - backward compatible new features
Z - backward compatible bugfixes
But it also says on item 4:
"4. Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable."
You may mark your version with a pre-release tag after the version, as suggested by Rohit, (you may, but you don't have to). The full spec could be:
<version core>"-"<pre-release>"+"<build>
Where:
<version core>: X.Y.Z (as stated before)
<pre-release>: your pre-release tag (check semver site for details on what it should contain)
<build>: build identifier
The spec also gives you some hints on how to deal with versioning during development phase in the FAQ:
How should I deal with revisions in the 0.y.z initial development phase?
The simplest thing to do is start your initial development release at 0.1.0 and then increment the minor version for each subsequent release.
"How do I know when to release 1.0.0?If your software is being used
in production, it should probably already be 1.0.0. If you have a
stable API on which users have come to depend, you should be 1.0.0. If
you’re worrying a lot about backwards compatibility, you should
probably already be 1.0.0."
Doesn’t this discourage rapid development and fast iteration? Major
version zero is all about rapid development. If you’re changing the
API every day you should either still be in version 0.y.z or on a
separate development branch working on the next major version.
This basically says that you can stay on 0.y.z if you don't have heavy dependency on your API. If your clients treat your API as their "production" or "production ready", maybe you should move to 1.0.0 and follow the rules.
The moment you go to production you should be already on 1.0.0
I'm maintainer of azcopy chocolatey package. https://chocolatey.org/packages/azcopy
Microsoft released version 10 of azcopy (from version 8).
The tool is now a single exe (in a zip file).
The old one was a MSI installer.
The command line is completely different between v8 and v10.
I've got two choices :
Create a new azcopy10 package for this package and have both living and at some point deprecate the old one
Simply "upgrade" the azcopy package to v10 as I don't expect MS to upgrade v8 anymore. If people are willing to keep v8 they still can avoid upgrade (I was going this way)
In case I just upgrade the azcopy, how do I manage the upgrade ?
If v8 is installed, do I uninstall it ? Is it allowed to uninstall a package in the chocolateyInstall.ps1 of a new version ?
Do I just manage the actual installation of v10 and let v8 if already install?
So, here are my thoughts on this...
Are these tools intended to be used side by side? You say that there are differences in how the command line functions, so it sounds like maybe that is the case. If you can use them side by side, then that in my opinion would be a case for a new, standalone package, named azcopy10.
That leaves the question on what to do with the existing azcopy package though. Should this be changed to be azcopy8? If so, should the existing azcopy package become a meta package, that points to either azcopy8 or azcopy10? That would certainly fit with how some other packages out there work.
However, this leads us back to the question of what to do if azcopy is already installed...
If they "can" work side by side, then simply blowing away the existing installation probably isn't a good idea, as some people might want to have both installed. As a result, a package parameter might be required to handle the uninstall if someone specifically chooses to uninstall it.
Long story short... There are lots of choices here, and there is unlikely to be one "correct" answer, as you will never please everyone. Likely, this doesn't help you though :(
If a Chef recipe (or any of it's cookbook dependencies) use the package resource without specifying a version, then the latest version of the package is installed. If you want to control and test exactly what you are installing, then you must always supply the package version. What can you do when the cookbooks that you depend on do not take the same precautions?
See for example the default recipe in the ark cookbook. If this recipe is used on a production server, it could install packages that have not been tested. This is just one example (with over 5m downloads) so I am wondering how people are getting around this problem.
What can you do when the cookbooks that you depend on do not take the same precautions?
I don't think there is a simple answer. This is basically "the Chef way" ...
(Actually, I would suggest that hard-wiring package versions could do more harm than good. One of the good things about using a (good) distribution's package repo is that they regularly release updates with patches for security issues and bugs. But if you wired fixed package versions into your recipes or roles/nodes or something, you would prevent any such patches from propagating to your system.)
However, if it is critical to you that package versions are stable then maybe ...
Clone and hack the cookbooks in question to use specific versions. (Actually, you probably need to do this anyway, to avoid being bitten by unstable cookbooks!)
Use a distro (such as RHEL or its "clones") that values long-term stability, and only pushes out package updates that are "really important".
Create your own private mirror of the distro's package repos with only the "good" versions of the critical packages in it.
Modify Chef so that the package resources pick/install specified versions by default. (I don't imagine this would be easy. But if you did come up with a good solution at this level, it would be a pretty useful addition to Chef! IMO.)
UPDATE
Actually, there is a way to do this for (at least) Debian-based systems; see the apt cookbook and in particular the references to "pinning".
Or with yum, you could "lock" particular versions using "yum versionlock ..." as described here: https://www.zulius.com/how-to/yum-install-specific-package-version/
UPDATE - 2
Another possible trick would be to "inject" a version attribute into the "unsafe" package resources. Something like this:
# first, include_recipe a recipe that specifies 'package "foo"' without
# a version attribute
# then ...
r = resources("package[foo]")
r.variables['version'] = "1.2.3"
With a little ingenuity, one could create a "package version lock" recipe that pulled the versions from a databag, and dealt with missing resource exceptions and version attributes that were actually provided. But I don't know if this is "A Good Idea" (tm).
Chef's package resource uses the package manager of the node's operating system (like apt, yum, etc). These tools always install the most recent version that is available through the repositories. That's why chef's package resource also installs this version.
What the ark cookbook is that it downloads the source code and then compiles it - obvious that you can specify the version to install (through the passed URL).
So it depends on your actual need. If you want to install the version that is available through the distro's or your own package repo, then it's totally fine (and that's what most cookbooks do). If you want to compile everything from source (where you usually have the option to specify the version, the coverage of chef cookbooks supporting this is lower.
Personally, I'd suggest that you set up an own apt/yum/whatever repo for the software for which have specific version requirements.
In short : I'm not managing this.
In a more complete answer:
All distro/release go throught a validation phase before releasing new packages, I'm confident over it and it helps me keep in sync with security fixes.
As far as I know all package managers takes care of not upgrading a package in a breaking way if it is a dependencies of a package installed manually, again you have to trust the package maintainer about this.
i.e.: the package ressource without version won't update make nor gcc if it is a dependency of one package you installed with a fixed version.
For exemple under ubuntu, if you set the nagios package to manual, it will never try to update the libc package over a breaking change, and so I could break other package isntallation as dependencies are not satisfied.
If you're absolutely concerned about it, you have some choices:
Rewrite each pacakge ressource to use fixed version of packages
Fork any cookbook to fix thooses issues (you can write a foodcritic rule to help you detect package ressources without specific versions)
Have your own repos, stable and testing, and move packages inside stable repo once tested and use testing repo on your staging/QA environement.
The 3 is the most conservative as you choose what is in the repo in the stable branch and it won't change magically. The drawback is security fixes you'll have to manage.
Hope it would help.
We had the same issue in our cookbook. So we decided to use data bags.
Data bags can be easily changed, for example:
knife data bag from file my_data_bag host1
OR
knife data bag edit my_data_bag host1
Your recipe will be able to see the specified version from the data bag using the code like this:
my_bag = data_bag_item('my_data_bag', 'host1')
Chef::Log.info("You have changed the version to: #{my_bag['version']}")
package 'java' do
version my_bag['version']
action :install
end
So finally you don't need to modify Cookbook or Recipe. All you need is to pass the version to the data bag.
In our project we have multiple components developed by separate teams having separate git repos.
All components have commit job and packaging job and publishes the artifacts to artifactory.
The problem comes when we want to deploy all the components as a system.
Since all these components deploys to separate servers and then interact with each other for functioning.. a lot of time inconsistencies arises due to some newer version of a component being deployed to one of the server.
For ex. I have components A,B,C and want to move following versions A1, B1, C1 in the deployment and testing pipeline. How I can ensure that no newer version of a component is deployed to QA environment (servers). I am using Jenkins as my CI/CD tool. It seems I need some integration or lightweight configuration management tool to manage the versioning of my system as a whole comprising of all components which I can promote in the deployment pipeline.
I hope I could describe my question. Suggestions to tackle this situation will be really helpful.
Thanks,
We use this pattern:
for every customer which uses our products there is one "project": It contains nearly no code, just configuration. We use this name scheme: coreapp_customerslug.
the project depends on N applications. The project pins all exact versions of the dependencies.
During CI we do this:
install project P and all the pinned dependencies
Then update all dependencies to their latest version.
Run all tests
If all tests succeed, update the versions of the dependencies and increment the version of the project.
Now the project has a new and stable release.
deploy the new release (at the moment we don't do this automatically, but in the near future).
With this pattern ("project" is an container of the apps) you can handle the version problem. If you have several servers, the update process should be fast, to avoid different versions at the same time.
Update
The CI maintains the pinned versions. We use python and pip and the file requirements.txt gets updated by a script. We use the version schema YYYY.N. N gets incremented if all tests are ok.
Attention: If app1 has latest version N, this does not mean that it works in all projects. If you have two projects: P1 and P2, this can happen: app1 with latest version N works well in project P1, but fails in P2. This means you can't create a new stable version of project P2. Sometimes this is annoying, but this keeps a constant update alive. We always use the latest version of our apps in ours projects.
In creating a package with Data Types and Items, what are the best practices to follow for subsequently upgrading a package in order to retain the data that's created on the site while adding a change to the associated Type?
Should the package name remain the same or should it include version information to retain uniqueness by version? The version number of the package does not seem to enforce any type of upgrade policies.
Currently during development the package is uninstalled then the newer package is subsequently installed however after the package reaches production I believe this type of upgrade workflow will not be sufficient without affecting the associated data.
As long as you increase the package-version but keep its unique GUID and Name, you're able to install the same package again, forcing C1 to overwrite the existing files, configuration etc.
This has a few caveats though, mainly if you need to delete some old files or configuration entries during installation, since that's what you normally do during un-installation. But you can still do it, writing your own installaton-steps code.
It is classes inheriting from some core base package-classes that you can invoke during installation. It can contain logic like checking if certain other packages are installed, do custom special logic, or clean up from earlier upgrades.
But yeah, long story short, as long as the version number increases you can "upgrade" a package, by installing the newer version, while the older one is still installed.