OpenObserve has an Elasticsearch compatible API for log ingestion, but syslog-ng is not mentioned in the documentation. My plan was to document how to modify the syslog-ng elasticsearch-http() destination, based on API documentation. However, as it turned out, OpenObserve has a ready to use syslog-ng configuration example in the web UI. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-openobserve-using-syslog-ng syslog-ng logo

This year I started publishing a syslog-ng tutorial series both on my blog and on YouTube: https://peter.czanik.hu/posts/syslog-ng-tutorial-toc/ And while the series was praised as the best possible introduction to syslog-ng, viewers also mentioned that one interesting element is missing from it: namely, it does not tell users how to develop a syslog-ng configuration. So, in this blog, learn how to develop a syslog-ng configuration from the ground up! I will explain not just the end result, but also the process and the steps to take to develop a configuration.

Even if most people ask me to compare systemd-journald vs. syslog-ng, I would say that they complement each other. Systemd-journald excels at collecting local log messages, including those of various system services. The focus of syslog-ng is on central log collection and forwarding the logs to a wide variety of destinations after processing and filtering. Combining the two gives you the most flexibility. Read more at https://www.syslog-ng.com/community/b/blog/posts/systemd-journald-vs-syslog-ng syslog-ng logo

Many users are annoyed by the version number included in the syslog-ng configuration. However, it ensures backward compatibility in syslog-ng. It is especially useful when updating to syslog-ng 4 from version 3, but also when updating within the same major version. Read more about it at https://www.syslog-ng.com/community/b/blog/posts/backward-compatibility-in-syslog-ng-by-using-the-version-number-in-syslog-ng-conf syslog-ng logo

One of the highlights of the syslog-ng 4.3.0 release is parallelize(). Normally, syslog-ng processes incoming messages from a TCP connection in a single thread. While this works fine with many connections, it is a bottleneck when using a single or very few high-traffic connections. Using parallelize() allows syslog-ng to process log messages from a single high-traffic TCP connection in multiple threads, thus increasing processing performance on multi-core machines. As you will see, parallelize() helps when you have a single high-traffic TCP connection.

One of the returning questions I received recently: why contribute to the syslog-ng upstream? I guess it is a question many open-source projects receive regularly. There are many generic answers. Here I would like to focus more on syslog-ng, focusing on various parts of it. Of course, the generic answers also apply. Syslog-ng is an open-source project, free to use, modify, and extend. By contributing, you can give something back and improve syslog-ng for everyone.

A few weeks ago, I posted about sngbench, a shell script to measure syslog-ng performance. The performance of syslog-ng is influenced by many factors, including the hardware and OS it runs on, and syslog-ng itself. This blog summarizes some of my findings using the script. https://www.syslog-ng.com/community/b/blog/posts/what-i-learned-about-syslog-ng-performance-using-sngbench syslog-ng logo

No matter how awkward you feel when you hear about UDP syslog in the age of encrypted TCP connections, UDP syslog is here to stay in some special cases. The scalability issues of UDP log collection were first addressed in syslog-ng Open Source Edition (OSE) (the so-reuseport() parameter), and later a more advanced solution arrived to syslog-ng Premium Edition (PE) (the udp-balancer() source). The good news is that a new, open-source implementation is now available as part of syslog-ng 4.

One of the returning syslog-ng questions I receive is how many log messages can a given hardware handle. My typical answer is that it depends on the configuration. I have now an answer, or rather a tool to answer your question: sngbench.sh. It is a shell script that runs from localhost and uses loggen, the bundled benchmarking and testing tool of syslog-ng. It comes with two configurations: a performance-optimized and a realistic one.

In version 4 of syslog-ng, the role of Python became even more important. Previously, all parts of syslog-ng could be extended using Python code, but no actual Python code was provided with syslog-ng. Version 4.0 added a Kubernetes module implemented in Python, while version 4.2 added support for Hypr. But how can we ensure that all Python dependencies are met? In my latest blog I describe the current situation and ask you for feedback!