bigtop-1.2 charms are stable; what about upgrades?

[Kevin Monroe]

Hey folks!

After a week of various tests and papercuts, the bigtop-1.2 charms and
bundles are now in stable channels.  See [0] for the list of software
versions.

This brings me to a question related to upgrades.  Currently, our charms
run an install method, and upon success, set an "installed" state that
guards the charm from subsequently running the install again.  During a
charm upgrade that alters the apt repository, we would need to invoke this
install method again to get the new apt package.

This is pretty simple to do -- we can watch for things like
"Bigtop().bigtop_apt" and trigger the install method when that value
changes.  However, I'm not convinced users want this to happen
automatically.  I propose that upgrades be a 2 step process.  Users would
upgrade the charm code (which is decoupled from the software payload),
followed by issuing an action to upgrade the software.  It would look like
this:

- juju upgrade-charm <charm>
- juju run-action <charm/unit> upgrade

This approach is how other big software groups are handling upgrades, and I
think it makes sense for us as well.  However, I'm open to feedback /
suggestions.

Regardless of how we upgrade, we are fortunate in that we have a "spec"
(sent on relations) that defines compatibility among our core Bigtop
charms.  This means, for example, if the namenode payload is upgraded, we
can detect and set things like the resourcemanager/plugin/slave charms to
blocked state until those charms are upgraded.  This gives users a very
clear view of version incompatibilities in their deployment.

Let me know your thoughts; if there are no objections, we'll get to work on
upgrade actions.  Thanks!
-Kevin Monroe

[0] - Application versions installed by charm:

hadoop-namenode 2.7.3
hadoop-plugin 2.7.3
hadoop-resourcemanager 2.7.3
hadoop-slave 2.7.3
* hbase 1.1.3-1
kafka 0.10.1.1-1
mahout 0.12.2-1
pig 0.15.0
spark 2.1.0-1
zeppelin 0.7.0
zookeeper 3.4.6-1

* HBase is getting a hotfix in bigtop-1.2.1 to bring it up to 1.1.9.  No
charm changes are required; when 1.1.9 hits the bigtop repo, that will be
the version installed by the hbase charm.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bigdata/attachments/20170425/4071a676/attachment.html>