Logging is a critical thing for all system administrators, if you log too much and you don’t manage the files you could fill up a partition or even worst stop some service, if you don’t log enough you’ll lose information when something goes wrong, in general a good solution for this is to send all the logs to a central server that will store for the time you need them, and keep just 1,2 days of log into the local machine.
You could do this configuration easily with rsyslog or syslog-ng to send/receive the logs and logrotate to rotate the files locally on your machines, today I want to show you some open source programs that can receive the logs, store them on filesystem or database and analyse them presenting the results via Web dashboards.
These are large applications most suited for big company, or in general to everyone that want to keep and manage a lot of data they are: Apache Flume, Logstash, Greylog2 and Scribe