Learning syslog-ng: a table of contents for my tutorial series

Last year, one of the returning questions I received was how to learn syslog-ng. My answer was that read the first few chapters of the documentation, read my blogs related to your use case, and then read a few relevant parts from the rest of the documentation. Our documentation is praised by users, but it is still a reference documentation. I was asked if a less detailed, more to the point, preferably video tutorial is available.

syslog-ng logo

Your request was heard. In the past couple of months, I published a tutorial series in blog and video format, which brings you from basic logging concepts to using syslog-ng to collect, parse, enrich log messages and store them to Elasticsearch. Of course, these 5-10 minute videos are not enough to learn anything in depth, but they introduce you to all major syslog-ng functionalities.

Even if you are a seasoned syslog-ng user, there is a good chance that you will learn something new from this introductory tutorial series: the “if” statement, in-line configuration elements, the inlist() filter or the JSON template function, just to name a few.

If you have roughly 1.5 hours, then I recommend going through all the videos from the beginning to the end. You can reach the play list on YouTube at: https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb

If you would rather pick only a few topics from the tutorial series, here is a table of contents, with short summaries, pointers to the blog and video versions and the related parts from the documentation. Unfortunately, the documentation for the latest version is not available yet, pointers are included to the web version of the syslog-ng version 3.37 documentation.

Of course, once you read/watched my syslog-ng tutorials, reading the blogs and relevant parts of the documentation is still highly recommended.

Introduction

The introduction gives you an overview of the tutorial series and defines what syslog-ng is.

Basic concepts

In this part, we cover some of the basic concepts behind syslog-ng. We talk about why central log collection is important, and then discuss the four major roles of syslog-ng: log collection, processing, filtering and finally storage. We conclude this part with a short introduction to various message formats.

Syslog-ng editions, and where to get them from

In this part we cover the various syslog-ng editions (open source, commercial and appliance), and where to get them from. The focus of this tutorial series is the Open Source Edition (OSE), but to avoid confusion, I also briefly introduce the other two.

Configuration and testing

This is the first practical part of the tutorial series. It introduces you to the syslog-ng configuration, shows you how to stop and start syslog-ng, and how to send a test message.

Sources

In this part we learn about syslog-ng source definitions and how to check the syslog-ng version and its enabled features. The tutorial shows you the source syntax and lists some of the more popular source drivers. The documentation lists all the sources and all their parameters.

Destinations and log path

In this part we learn about syslog-ng destinations and the log path. At the end of the session, we will also perform a quick syntax check. As usual, the tutorial shows you the destination and log path syntax and lists some of the more popular destination drivers. The documentation lists all the destinations and all their parameters. The part about the log path also includes many concepts that we only talk about in later parts of the tutorial.

Networking

In this part we learn about syslog-ng network logging, and why relays are important in a logging infrastructure. At the end of the session, we will send test messages to a syslog-ng network source.

Macros and templates

In this part we learn about syslog-ng macros and templates. At the end of the session, we will know how to do a simple log rotation using macros.

Filters

In this part we learn about syslog-ng filters. At the end of the session, we will see a more complex filter using an “if” statement and a template function.

Parsing

In this part we learn about message parsing using syslog-ng. We only scratch the surface, so reading the documentation is recommended, especially if you want to try PatternDB.

Enriching log messages

In this part we learn about enriching log messages. Enriching in this case means that you can create additional name-value pairs based on message content. There are several ways how you can enrich log messages using syslog-ng.

Elasticsearch (and Opensearch, Zinc, Humio, etc.)

In this part we learn about how to send log messages to Elasticsearch. Note that while I keep referring to the driver as “Elasticsearch destination”, you can use it with several other software utilizing the Elasticsearch API, such as Opensearch, Zinc, Humio and probably more. This part shows you not only how to send log messages to Elasticsearch, but also combines many of the previously learned syslog-ng features into a single configuration.

Updating syslog-ng, syslog-ng 4

In this part we learn about updating syslog-ng, and some of the new features of syslog-ng 4.