Last modified: 2014-08-15 15:48:47 UTC
The python stats collector diamond does not compress its log under /var/log/diamond/ . That might lead to /var/ filling up entirely (it is only 2GB). We might need diamond to compress logs, or a cronjob to gzip them if it can't compress them. Also want to make sure we dont keep too many archives.
This used to happen really consistently (see bug 66458). We 'fixed' it by essentially making diamond log only on errors, which shouldn't happen often. The role also realm-branches in labs to only keep one day of logs, vs prod where we keep more. Closing as wontfix for now.
You might have leftover archive.log and diamond.log files from before the fixes. If so, feel free to rm -rf them, diamond won't re-create.