Last modified: 2014-07-17 21:42:37 UTC
I had to delete those files and restart the diamond service on the Tools host because they were clogging up /var/log. Chase explained in #wikimedia-operations: | <scfc_de> chasemp: What is /var/log/diamond/diamond.log and is it necessary? | <chasemp> scfc_de: it's the local log of the statistics polling, it's not | necessary in that it won't work without it, but it doesn't logging | those details to stdout seemingly so upstart logging isn't useful in | those cases. it's handy but not could-never-live-without So I'd propose that on Labs, by default this log isn't generated. A Puppet variable could override this default if there is wider demand for that.
On a randomly chosen instance, I see: -rw-r--r-- 1 diamond nogroup 9810916 Jul 2 23:59 archive.log.2014-07-02 -rw-r--r-- 1 diamond nogroup 9810168 Jul 3 23:59 archive.log.2014-07-03 -rw-r--r-- 1 diamond nogroup 9811659 Jul 4 23:59 archive.log.2014-07-04 -rw-r--r-- 1 diamond nogroup 9806164 Jul 5 23:59 archive.log.2014-07-05 -rw-r--r-- 1 diamond nogroup 9813206 Jul 6 23:59 archive.log.2014-07-06 -rw-r--r-- 1 diamond nogroup 16792013 Jul 7 14:06 diamond.log.2014-07-06 -rw-r--r-- 1 diamond nogroup 6444463 Jul 7 15:45 archive.log -rw-r--r-- 1 diamond nogroup 1237334 Jul 8 16:46 diamond.log.2014-07-08 drwxr-xr-x 2 diamond root 4096 Jul 11 19:42 . -rw-r--r-- 1 diamond nogroup 126 Jul 11 19:43 diamond.log Since it's the 17th, I presume this has somehow been fixed?
Yes, Chase and Yuvi fiddled with diamond quite a bit with the conclusion (IIRC) that it doesn't log at all on Labs.
It does log, but only logs errors. We killed the archive handler that logged all the *metrics* being sent, which was causing the huge log files.
