Improved OSSEC Log Parsing with Logstash

logstash-ossecThe ELK stack (Elasticsearch-Logstash-Kibana) provides a cost effective alternative to commercial SIEMs for ingesting and managing OSSEC alert logs. Previously I wrote a blog – OSSEC Log Management with Elasticsearch – that discusses the design of an ELK based log system.

Since then some readers have asked for and suggested ways to parse additional fields from the OSSEC alert log stream. For example, the IP addresses of systems that cause certain security events is buried down in the Details field. So I have created a Logstash configuration file that does just that.

Continue reading

Elasticsearch Client Programming (2/2) – Python

Elasticsearch PythonThe first article in this two part series focused on developing Elasticsearch clients with Perl. Elasticsearch also has an excellent Python library which lets you search for and analyze your data with one of the many mathematics and machine learning libraries available for Python.

In this article I’ll cover how to create an Elasticsearch client using Python that has the same capabilities as the Perl client from the part 1 article.

Continue reading

Elasticsearch Client Programming (1/2) – Perl

Elasticsearch PerlSince creating a log management system for the OSSEC HIDS with Elasticsearch, I have been busy applying this useful search technology in other projects. Elasticsearch is a marvelous system for ingesting streaming data that gets indexed on the fly and quickly searching your data.

The Elasticsearch community provides client libraries that expose their search API in several popular languages, including Perl and Python. This article is the first of a two part series where I show you how to write an Elasticsearch search client application in both of these languages, starting with Perl.

Continue reading

TCP/IP Sockets with Time Out Capabilities

Socket Time OutRecently I had a question from one of my readers about how to close connections on a server when there are no requests received after a certain period of time.  The question was asked with regard to the tcpsockets classes I covered in my blog TCP Network Programming Design Patterns in C++, none of which support time out capabilties.

Timing out on both receive and connect operations are common use cases. So in this article I’ll update my tcpsockets classes to provide these capabilties.

Continue reading

Installing OSSEC from RPMs

OSSEC InstallWhen I first started playing around with OSSEC I downloaded the agent and server source packages then proceeded to install them by hand. This method was fine when I had a server and 1 or 2 agent systems, but for a large network of systems it is tedious and error prone.

The OSSEC Project offers RPM packages that can be installed with yum on RedHat derived Linux distributions. Using these packages and the ossec-authd system, you can script the installation of OSSEC and automatically register agents with the server.

Continue reading