Snapcraft's state tracking improvements

[Kyle Fazzari]

Hey everyone.

I feel it coming on... this is going to be a tome. tl;dr... snapcraft
could be smarter than it is. But would that lead to its doing more for
you than you want? We'd like to find out.

I've spoken to a few of you individually about this, and I want to open
this topic up for wider conversation. We have a number of bugs logged
against snapcraft for its state tracking shortcomings. For example, a
few of you may have run into these issues:

- You add a stage package to a part in an already-built snap, and run
`snapcraft` again. It happily says all the steps have already run, and
generates a snap that doesn't contain your stage package.

- You add (or modify) a file in the local source for a part of an
already-built snap, and run `snapcraft` again`. It again reports that
everything has already run, and generates a snap that doesn't contain
your modifications.

We used to have even more of these issues, but snapcraft has slowly been
getting smarter about things like this. You may have noticed that v2.23
fixed that first problem, for example. Well, sort of. Let me explain.

Snapcraft's state tracking allows us to do some handy things. For
example, as you probably know, when you run the `stage` or `prime`
steps, you're combining files from multiple parts into a single
directory. Thanks to state tracking, snapcraft is able to disentangle
which files came from which part, and allow you to remove a single
part's files using per-step cleaning.

However, this is complicated by the ability to have inter-part
dependencies (using the `after` keyword). If part A depends on part B
and part B is being cleaned, then snapcraft knows part A needs to be
cleaned as well. But should it go ahead and clean it for you? What if
you were offline and part A required the internet to build? Or
re-building part A took forever and you didn't want to just wipe it? We
weren't sure when we implemented this ability, so we decided to be safe
and force you to say exactly what you wanted. For example, if you simply
said `snapcraft clean part B` we error out, saying something like "Hey
you're trying to clean part B, but part A depends on it. If you intend
for both to be cleaned, please say so."

This same mindset can be found elsewhere:

- If you change something in the YAML that makes a part out of date
(e.g. stage packages) it tells you it's out of date and why, but doesn't
clean/rebuild it for you.

- Trying to build part A which depends on part B before B itself has
been staged leads to "Hey, part A has unsatisfied dependencies." If you
run `snapcraft stage A B` it'll do the right thing.

- Not-yet-released features will know when local sources for a part have
changed, and build the updates, but what should it do if that part has
another depending upon it?

The point is: snapcraft is/will be smart enough to help a lot more than
it currently does in these types of situations. The question I'm posing
to you all is: how much do you want its help?

I see four options for how to handle such situations:


*Option 1*: Error out and make you be specific. This is the current
behavior.

*Option 2*: Automatically take care of everything. If you modify a part
with dependencies, snapcraft will rebuild those dependencies as it sees
fit without your needing to say so. Similarly, if you clean a part with
dependencies, snapcraft will clean those dependencies as it sees fit
without your needing to say so.

*Option 3*: Prompt. "You've modified part A, but part B depends upon it.
Would you like to rebuild it as well? (Y/n)" and the like.

*Option 4*: Add the ability to configure this behavior between options
1-3. Perhaps the sensible default would be option 3.


Our goal here is to do the least surprising thing, which is why we're
asking for input from our users. Please let your voice be heard!

-- 
Kyle Fazzari (kyrofa)
Software Engineer
Canonical Ltd.
kyle at canonical.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/snapcraft/attachments/20161213/864ea39f/attachment.sig>