Using rsyslog to log messages from Tomcat Log4j cluster M/Cs in Lan

[raja_s_patil]

Hello,

We are trying to establish cluster of Tomcat Servers (Multiple Instances on Multiple Machines).
Tomcat is Logging with help of log4j but there are multiple log files one per instance which is not desirable.
When enquired on other forums it was susgested to use syslog on Logging machine and create combined log files.
On Kubuntu 12.04 I found that instead of syslog it is using rsyslog for logging.

Can somebody help me in this situation by guiding me how can I use rsyslog to accomplish the task of creating combined log file ?

Thanks and warm regards.

Raja Patil

[kubicle]

You can find a lot of "rsyslog remote logging howtos" on the internet, but I'll summarize quickly the "necessary" steps, so you can get the idea (and ask further questions if you need to):

1. Log host (server): Enable remote syslog reception by editing /etc/rsyslog.conf. You can choose UDP or TCP and port (default is 514), uncomment the two lines under "# provides UDP syslog reception" or "# provides TCP syslog reception"
2. Log host (server): Create a log rule to gather received remote log messages to a log file of your choosing (like /var/log/tomcats.log)...the rule can be placed in /etc/rsyslog.d/
3. Tomcat hosts (clients): Create a log rule to forward tomcat log messages to your log server...the rule can be placed in /etc/rsyslog.d/
4. Make sure that your host can receive UDP or TCP packets on the port you've chosen (default 514) on your network (so if you block internal traffic, adjust firewall rules accordingly)
5. (optional) Log host: You may wish to create a logrotate rule to rotate your log file...rule can be placed in /etc/logrotate.d/

EDIT: log4j might require some extra configuration (depending on your current setup and configuration)...I think SyslogAppender can be used to send log messages to a syslog daemon. [if you log directly to remote syslog daemon (on your log server), step 3 is not necessary]

More info:
'man rsyslog.conf'
'man logrotate.conf'
Internet howtos

[raja_s_patil]

Thanks kubicle,

I will do further study on hints given and comeback here again if needed. I was bit confused between syslog and rsyslog which is being used in Kubuntu.

Just for acadaemic interest, are they more or less similar or differ a lot ?

Raja Patil

[raja_s_patil]

Thanks Kubicle,

I found this http://www.rsyslog.com/doc/features.html which explains additional features of rsyslog.
Hope it helps others.

Raja Patil

[kubicle]

Just for acadaemic interest, are they more or less similar or differ a lot ?

You probably already found that out, but rsyslog should be a drop-in replacement for syslog (with more features), they both use the same syslog protocol for example, and configuration is quite similar. IIRC, syslog only supports udp remote logging while rsyslog supports tcp/udp...but I'm not 100% on that.

[raja_s_patil]

Thanks Cubical,

Today I tried once again log4j-rsyslogd integration after about weeks gap. The logs are being routed as expected to rsyslogd.
However the log pattern is as follows

Dec 21 23:58:22 localhost local1: 23:58:22 710 localhost-startStop-1- DEBUG
< ======= By rsyslog =======>< ======= By log4j =======>

In log4j pattern I have ommited date since logs will be roated daily. However rsyslogd is adding
Timestamp, srchost, Facility to message received from client.

I would like to remove timestamp and facility to reduce the clutter. Since all tomcats in cluster will be passing
log messages to common log server there should be some info about which instance of TC has created this message

Somebody please guide me how shall I proceed ?

Thanks and warm regards.

R. S. Patil

[raja_s_patil]

BTW i solved Timestamp & facility appending problem by defining own template and log lines are showing only HostName.

However Can somebody help me in defining names for TC instances and refering them in either Log4j xml or in rsyslog conf files ?

Thanks and warm regards.

Raja Patil