In my last post I discussed how a logging server uses a message’s priority value to sort incoming log messages. In this installment I’ll talk about testing connectivity, as well as how to get various gear on the wire to submit their log entries to a centralized server.
Requirements
In order for a system to submit log entries, it has to have support for Syslog. Log entries need to be transmitted in clear text to port UDP/514 on the logging server. If you are using rsyslog on the server, TCP/514 is acceptable as well.
Submitted long entries should include the following info:
- A priority value (in <PRI> format) at the beginning of the payload
- The name of the application submitting the log entry
- The process ID used by the application
- The body of the log messages
But to be honest, Syslog is not very fussy. It will attempt to record anything sent to its listening port as a long entry. If a priority value is not present, it will record the entry to whatever file is used for facility 1. Typically this is /var/log/messages.
Testing Connectivity
When deploying a new setup, I like to verify connectivity between the first few clients and the logging server. If logging is not working, you will want to be able to isolate the problem area. Typical problems include:
- Client incorrectly configured
- Firewall in the way (on the client, the wire, or the logging server)
- Server incorrectly configured
You will want to monitor the messages file on the logging server to ensure the test log entry is received. On the logging server, run the following command:
tail -f /var/log/messages
Tail will open the log file read only and print the last five lines. As new log entries are received, they will get printed to the screen as well.
To generate a test log entry, I like to use Netcat. Netcat can be used from any Windows, Linux or UNIX system. From the test system, run the command:
echo ‘this is a UDP test log entry’ | nc -u -w 1 <IP address of logging server> 514
You should see the echoed portion of the command show up in the /var/log/messages file on the logging server. If not, launch a packet sniffer and see if you can determine where the failure is occurring. If you wish to test TCP connectivity as well, simply run the command again leaving out the Netcat “-u” switch:
echo ‘this is a TCP test log entry’ | nc -w 1 <IP address of logging server> 514
If both entries are received, we are ready to start pointing devices at our logging server.
Network hardware
Most network hardware supports Syslog via UDP/514. It is just a matter of going through the documentation and determining the proper command set for sending log entries to a remote server.
If you are using Cisco IOS, run the following from global configuration mode:
logging <IP address of logging server>
If you wish to change the logging facility from “local use 7” to something else, the command is:
logging facility <facility short name>
So to change logging facility to “local use 3”, the command would be:
logging facility local3
Linux and UNIX systems
There are a number of Syslog alternatives available for Linux and UNIX. In this section I’ll cover how to get Syslog and rsyslog to forward their log messages to a remote server.
Syslog
If the client is running Syslog, you will need to edit the /etc/syslog.conf file. Add the following line to the bottom of the file:
*.* @<IP address of logging server>
So an example would be:
*.* @192.168.1.150
Note the white space between the wild card match and the remote IP address MUST BE TAB CHARACTERS. If you use spaces, Sysylog will not be able to parse the file. Save and exit the file, then restart Syslog to activate the changes.
rsyslog
With rsyslog we have the option of forwarding our log messages via UDP or TCP. In either case we will need to edit the /etc/rsyslog.conf file. To forward long entries using UDP, add the following line to the end of the file:
*.* @<IP address of logging server>:<port>
So an example would be:
*.* @192.168.1.150:514
If we wish to use TCP instead, we simply use two “@” symbols:
*.* @@192.168.1.150:514
Once complete save and exit the file. You will need to restart rsyslog to activate your changes.
Windows systems
As mentioned in a previous post, Windows does not include support for Syslog. This means you will need third party software to convert your logs in real time and submit them to a logging server. The Loganalysis Web site has a list of possible solutions.
For the purposes of this post, I’ll cover Snare. It is free for use, can be commercially licensed, and have a very simple deployment process.
Once you download the software, you need to configure it for the system. This is shown in Figure #1. Snare needs to know the location of the logging server as well as what facility and severity level to use. Once complete, click the “Latest Events” menu potion to see which specific Event Viewer log entries Snare is forwarding to the logging server.
Exec Summary
In this post I discussed testing connectivity to a logging server, as well as how to configure clients to centralize their logs. In the next post I’ll talk about what to look for in a daily reporting tool, as well as real time alerting.
Related posts:
