|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
|
Re: [PLUG] Central Syslog
|
> Date: Thu, 5 Feb 2009 14:55:59 -0500 (EST)
> From: Matt Mossholder <matt@mossholder.com>
> ----- "Michael Leone" <turgon@mike-leone.com> wrote:
>> Is this where I get to be a smart mouth and say:
>>
>> Well, it's Linux. Write one. (Isn't everyone supposed to be skilled
>> enough to do that? I know I am not, but I figured that was just a
>> personal failing ...)
>>
>> Oh, and let the rest of us know when it's available ...
>> :-)
>
> The problem isn't writing a new one.. it is patching every syslog
> enabled program to use the new system.
>
> Even more problematic to migrate all the apps off of using the
> braindead, non-customizable facility/priority based message
> identifier system.
I agree with the points above, but there are actually several
alternatives available. I haven't used any extensively, but I know at
least syslog-ng is a semi-drop-in replacement that's backwards
compatible. I suspect most or all are, else they would be impossible to
use as Matt points out.
First, I was impressed with the 'newsyslog' tool I was reading about in
_Absolute FreeBSD_. It's very similar to the default Linux syslog, but
with tweaks like a usable time format, better file rotation features and
better handling of multiple hosts. It seems like a cross between
old-style syslog syntax and syslog-ng capabilities.
On the Linux side, (Ubuntu/Debian at least) there are several daemons:
sysklogd - System Logging Daemon # Default
dsyslog - advanced modular syslog daemon
dsyslog-dbg - advanced modular syslog daemon - debug
dsyslog-module-mysql - advanced modular syslog daemon - MySQL support
rsyslog - enhanced multi-threaded syslogd
rsyslog-doc - documentation for rsyslog
rsyslog-mysql - MySQL output plugin for rsyslog
rsyslog-pgsql - PostgreSQL output plugin for rsyslog
syslog-ng - Next generation logging daemon
And tons of checker/watcher/parsers:
epylog - New logs analyzer and parser
libparse-syslog-perl - Perl module for parsing syslog entries
log-analysis - Analyse system's logs to find out problems
logcheck - mails anomalies in the system logfiles to the administrator
logcheck-database - database of system log rules for the use of log checkers
logtool - Syslog-style logfile parser with lots of output options
logwatch - log analyser with nice output written in Perl
syslog-summary - summarize the contents of a syslog log file
swatch - Log file viewer with regexp matching, highlighting, & hooks
tenshi - log monitoring and reporting tool
syslog-ng is probably capable of any craziness you want to do, it's just
a matter of taking the time to learn and configure it.
On the $$$ side, Splunk and LogLogic come immediately to mind (I know a
senior guy at LogLogic too). Someone else listed Splunk as free, maybe
they have a free/limited version, I've never looked. But we partner
with them at work and AFAIK they aren't cheap. Neither are we for that
matter, though we only do *security* log monitoring, not all log
collection/centralization/archiving. [1]
I think you can spend a lot of time to roll it yourself, or spent a lot
of money to buy something, and then spend a lot of time to configure it
all...
But to get back to the OP's question, about "best practices" and
architecture, yeah, that's a tough one. OP also does not mention if
Windows is involved. Given this list and the choices he mentioned, I'll
assume not. (It's possible to use various free or $$$ tools for forward
Event logs via syslog. Windows logs are really, REALLY ugly, verbose,
inconsistent and almost useless though. You can make a strong argument
that Unix-like OS syslog is inconsistent, but it's usually at least
concise and useful.)
Here are some places to start:
# I know Tina and used to work with her. Very bright.
http://www.sage.org/pubs/12_logging/
Building a Logging Infrastructure
Abe Singer and Tina Bird
http://www.loganalysispros.com/
http://www.loganalysis.org/ (Tina & Marcus, site is getting really old)
http://en.wikipedia.org/wiki/Syslog
http://www.faqs.org/rfcs/rfc3164.html RFC3164: The BSD syslog Protocol
http://www.faqs.org/rfcs/rfc3227.html Guidelines for Evidence Collection
and Archiving
Some useful stuff here, from LogLogic and Splunk forums and elsewhere:
http://www.google.com/search?q=syslog+%22best+practices%22
http://www.syslog.org/wiki/Main/SyslogBestPractices
http://www.owasp.org/index.php/Log_review_and_management
Later,
JP
[1] For anyone who cares, I work for BT MSS (British Telecom Managed
Security Services), but it is "powered by Counterpane" (Bruce Schneier's
company). I've worked for Counterpane since 2002 and now for BT since
they bought us a while ago. http://en.wikipedia.org/wiki/BT_Counterpane
I do back-end engineering and tools work though, not front-end or
customer side architecture design or implementation.
----------------------------|:::======|-------------------------------
JP Vossen, CISSP |:::======| http://bashcookbook.com/
My Account, My Opinions |=========| http://www.jpsdomain.org/
----------------------------|=========|-------------------------------
"Microsoft Tax" = the additional hardware & yearly fees for the add-on
software required to protect Windows from its own poorly designed and
implemented self, while the overhead incidentally flattens Moore's Law.
___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug
|
|