making tools, finding tools, and the upgrade process

[Tim Penhey]

Hi all,

Friday seems to be my day of emails...

I'd like to get a better understanding around the whole tool process,
and hopefully write up the result into a document that we can put into
the tree in the doc directory.

I'm going to start with what my current understanding is (which I hope
isn't too wrong), with some questions intermingled.


Overview
--------

Simply put, the tools have been cut down to a single executable (jujud),
which is then compressed, and put into a location that is available to
the cloud instances that are started in the current environment.

An advantage of Go here is that the jujud executable is entirely self
contained, with all the dependencies statically linked.

The jujud executable runs as both the machine and unit agents, as well
as the other juju commands accessible through the hook scripts.

Released versions of the tools are uploaded into public storage as the
main package is released.

Q) Is it just in the fallback S3 storage?  Or is it a storage for each
of the main cloud instances (hp cloud, aws, ...)?

When an environment is bootstrapped, the agent-version is set to the
version of the juju command.  The bootstrapped machine then looks for
the best tools for that agent-version.

It is expected that the entire deployed environment will use the same
tools, at least the same version in the situation where the architecture
may be different.


Series vs. OS
-------------

Right now we have series encoded in the version of the tools, where at
this stage, series relates to an Ubuntu release.  Due to the stand-alone
nature of the go executables, 1.9.12-precise-amd64 is likely to be
identical to 1.9.12-quantal-amd64, and 1.9.12-raring-amd64.

However, series isn't just going to relate to a series of Ubuntu.  As
juju moves to being a multi-OS tool, it may occur that we have
1.9.12-win8-amd64, or 1.9.12-rhel7-amd64.

It seems that series should not necessarily be the discriminating
factor, but instead OS?  Perhaps internally we have a series -> OS dict.

Would it make more sense to have 1.9.12-ubuntu-amd64 for a tools version
that is used for ubuntu based series?  And perhaps a
1.9.12-windows-amd64 for windows?  With something that may look like:
  {"precise": "ubuntu",
   "quantal": "ubuntu",
   "raring": "ubuntu",
   "win7": "windows",
   "win8": "windows",
   "rhel7": "rhel",
   ... }

I can see that we do still want series information as the base piece of
information used, as that is how we choose charms.


Finding Tools
-------------

There are three options to FindTools, and I'm not entirely clear on
which situations they are used for.

 * compatible version
 * highest version
 * development version

Why do we have these options in the first place, and why do we care
about distinguishing between them?  What use-cases are they trying to solve?

Also, there seems to be weird edge-cases around how choices are made
between the private tool bucket, and the public tool bucket.

Under what situations do we choose tools with versions that are lower or
higher than the agent-version?


Upgrading The Environment
-------------------------

There is an upgrade command for juju, whose purpose it is to upgrade the
tools and the agent-version that the currently deployed machine and unit
agents are using.

What are the deciding steps on upgrading an environment?
When would a user upgrade a running environment?

What are the steps during an upgrade?  Meaning, if I as a user go "juju
upgrade", what are the steps?  How long is it expected to complete an
upgrade roll-out to an environment?



I hope we can have those that have the understanding jump in and clarify
some of these points.  I think it would be a helpful addition to the
in-tree documentation.

Cheers,
Tim