Controllers running out of disk space

[Uros Jovanovic]

Hi all,

I'd like to start a discussion on how to handle storage issues on
controller machines, especially what we can do when storage is getting 95%
or even 98% full. There are many processes that are storing data, we have
at least:
- charms and resources in the blobstore
- logs in mongo
- logs on disk
- audit.log

Even if we put all models in the controller into read-only mode to prevent
upload of new charms or resources to blobstore, we still have to deal with
logs which can fill the space quickly.

While discussing about this with Rick, given the work on model migrations,
maybe the path forward would be to allow admin to put the whole controller
in "maintenance" mode and put all agents on "hold".

How to deal with storage issue after that is open to discussion, maybe we
need tools to "clear" blobstore, or export/compress/truncate logs, etc.

Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20161117/0c5a3677/attachment.html>