Controllers running out of disk space

[John Meinel]

Juju records a longer history in the database, but for ease of debugging we
also save to a plain text file, which gets rotated. It is set to rotate at
300MB and save at most 2 backups. Which means it should go to potentially
~1GB but not grow beyond that. We could compress on rotation, which we
don't currently do, but it is capped at how much we will save on disk.

John
=:->

On Tue, Nov 22, 2016 at 11:50 AM, Jacek Nykis <jacek.nykis at canonical.com>
wrote:

> On 21/11/16 23:26, Menno Smits wrote:
> > On 18 November 2016 at 05:07, Nate Finch <nate.finch at canonical.com>
> wrote:
> >
> >> Resources are also stored in mongo and can be unlimited in size (not
> much
> >> different than fat charms, except that at least they're only pulled
> down on
> >> demand).
> >>
> >> We should let admins configure their max log size... our defaults may
> not
> >> be what they like, but I bet that's not really the issue, since we cap
> them
> >> smallish (the logs stored in mongo are capped, right?)
> >>
> >> Do we still store all logs from all machines on the controller?  Are
> they
> >> capped?  That has been a problem in the past with large models.
> >>
> >
> > ​All logs for all agents for all models are stored in mongodb on the
> > controller. They are pruned every 5 minutes to limit the space used by
> > logs. Logs older than 3 days are always removed. The logs stored for all
> > models is limited to 4GB with logs being removed fairly so that logs for
> a
> > busy model don't cause logs for a quiet model to be prematurely removed.
> >
> > The 3 day and 4GB limits are currently fixed but have been implemented in
> > such a way to allow them to become configuration options without too much
> > fuss.
> >
> > - Menno
>
> Hi Menno,
>
> Do you know if log storage in mongodb needs to be enable in any way?
>
> I can see that my juju 2.0.1 controller still stores logs on disk
> without compression:
> /var/log/juju$ ls -lh logsink*
> -rw------- 1 syslog syslog 300M Oct 11 07:15
> logsink-2016-10-11T07-15-12.443.log
> -rw------- 1 syslog syslog 300M Nov 9 08:24
> logsink-2016-11-09T08-24-35.019.log
> -rw------- 1 syslog syslog 287M Nov 22 09:36 logsink.log
>
> I think it's https://pad.lv/1494661
>
> --
> Regards,
> Jacek
>
>
>
> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20161122/4b02a986/attachment.html>