One Identity Cloud PAM is one of the latest security products by One Identity. It provides asset management as well as secure and monitored remote access for One Identity Cloud users to hosts on their local network. Last year, I showed you how collect One Identity Cloud PAM Network Agent log messages on Windows and create alerts when somebody connects to a host on your local network using PAM Essentials. This time, I will show you how to work with the Linux version of the Network Agent.

Finally, a new syslog-ng release! As you can see from its version number, this is a bug fix release. It took a bit longer than expected, as we wanted to release it in sync with syslog-ng PE, the commercial variant of syslog-ng. 4.8.2 serves not just as the foundation of the new syslog-ng PE release, but also provides fixes to 4.8.1, which is included in major Linux distributions. This update ensures that all our recent bug fixes reach the majority of our users.

While no dates are set to stone yet, we expect a couple of syslog-ng releases in the near future. As version 4.8.1 is used in major Linux distributions and has a couple of known bugs, we will release 4.8.2 to address those. However, we are also working on 4.9.0, which will bring many changes. Read more at https://www.syslog-ng.com/community/b/blog/posts/a-call-for-testing-the-upcoming-syslog-ng-releases syslog-ng logo

From my previous Active Roles blogs, you could learn how to forward regular Active Roles logs from Windows Event Log to a central syslog-ng server, where it parses, filters, stores and forwards the logs. In this blog, I show you how to work with Active Roles debug logs, that is reading them using syslog-ng Agent for Windows and forwarding them to a central syslog-ng server for long(er) term storage. Debug logs are typically huge and the Active Roles debug logs are no exceptions, so you must make sure that you collect them only when really necessary.

The April syslog-ng newsletter is now on-line: Testing Elasticsearch 9.0.0 beta1 with syslog-ng Working with parsed Active Roles logs in syslog-ng Running syslog-ng PE in RHEL UBI It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2025-04-elasticsearch-beta-active-roles-rhel-ubi syslog-ng logo

Last week, I posted about running nightly syslog-ng container images on arm64. However, you can also install syslog-ng directly on the host (in my case, a Raspberry Pi 3), running the latest Raspberry OS. Read more at https://www.syslog-ng.com/community/b/blog/posts/installing-nightly-syslog-ng-arm64-packages-on-a-raspberry-pi syslog-ng logo

Recently we enabled nightly syslog-ng builds and container builds for arm64. It means that from now on, you can run the latest syslog-ng on 64bit ARM platforms. For this test, I used a Raspberry Pi 3 running the latest Raspberry Pi OS. As I use Podman everywhere else (I am an openSUSE / Fedora guy), I also installed it here for container management. Read more at https://www.syslog-ng.com/community/b/blog/posts/nightly-arm64-syslog-ng-container-builds-are-now-available syslog-ng logo

For many years, the development of syslog-ng happened on the master branch in Git. However, if you follow that branch, you might have noticed that there has not been much activity on it lately. That is because we introduced a new branch in git called “develop”. https://www.syslog-ng.com/community/b/blog/posts/introducing-the-develop-branch-of-the-syslog-ng-git-repo syslog-ng logo

The March syslog-ng newsletter is now on-line: Test syslog-ng on EPEL 10! Collecting Active Roles logs centrally using the syslog-ng Windows Agent syslog-ng OSE 4.8.1 is now in EPEL 10, quick fix for Elasticsearch It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2025-03-epel-10-elasticsearch-active-roles syslog-ng logo

In my previous OneIdentity Active Roles blog, you learned how to forward Active Roles logs to a central syslog-ng server to parse and store the logs. In this blog, I’ll show you how to: Work with parsed Active Roles logs. Store logs to various document stores. Prepare long-term storage. Send alerts for some critical events. Even if this blog about commercial software, the name-value pairs concept I describe in this blog in depth is the same in the open source syslog-ng.

One Identity Active Roles allows you to easily and securely manage Active Directory (AD), Entra ID and M365 Identity objects. While Active Roles stores its log messages into Windows Event Log, most log management and log analytics applications expect to receive log messages over the syslog protocol. This is where syslog-ng Premium Edition (PE) can help you. The syslog-ng Windows Agent can collect and forward Active Roles log messages from Windows Event Log, while the syslog-ng server can collect, process, store and forward Active Roles log messages to multiple destinations.

The December syslog-ng newsletter is now on-line: A syslog-ng container image based on Alpine Linux Call for testing: syslog-ng in openSUSE Leap 16.0 Experimental syslog-ng container image based on Alma Linux It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2025-01-alpine-linux-leap-16-0-alma-linux syslog-ng logo

Last December, I added support for EPEL 10 in my unofficial syslog-ng Git snapshot repository. This week, I call for testing the official syslog-ng EPEL 10 package. Once I saw in my unofficial syslog-ng repo that syslog-ng compiles fine on EPEL 10, I also started to work on the official package. I hit a roadblock immediately: ivykis (a mandatory dependency of syslog-ng) was missing from EPEL 10. So, right before the Christmas holidays, I submitted two missing dependencies I maintain (ivykis and riemann-c-client) to EPEL 10.

CentOS Stream 10 and EPEL 10 just became available, and as usual, I tried to build syslog-ng as soon as possible. For now it is available in my git snapshot repository, but I am also planning to make it available in EPEL 10 soon. Read more at https://www.syslog-ng.com/community/b/blog/posts/test-syslog-ng-on-epel-10 syslog-ng logo

Recently I was asked why the syslog-ng newsletter looks odd. At first I did not even understand what is the problem. Then I realized that I kept using the same format for the past 14 years, that was optimized for UNIX terminals :-) So, what is the problem? 14 years ago I was kindly asked by syslog-ng users to use plain text e-mails instead of HTML formatting. Of course it also means that there is no easy way to emphasize titles in the newsletter.

The December syslog-ng newsletter is now on-line: FreeBSD audit source for syslog-ng Version 4.8.1 of syslog-ng is now available Where should I present syslog-ng and sudo? It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-12-freebsd-audit-4-8-1-conferences syslog-ng logo

Last week I introduced you to my latest project: a syslog-ng container based on Alma Linux. This week I added a syslog-ng Prometheus exporter to the container, so you can also monitor syslog-ng, if you enable it. syslog-ng logo

The official syslog-ng container image is based on Debian Stable. However, we’ve been getting requests for an RPM-based image for many years. So, I made an initial version available based on Alma Linux and now I need your feedback about it! This image uses the “init” variant of Alma Linux 9 containers as a base image. What does this mean? Well, it uses systemd service management inside, making it possible to run multiple services from a single container.

The November syslog-ng newsletter is now on-line: A call for syslog-ng testing Working with Quickwit Huge improvements for syslog-ng in MacPorts It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-11-testing-quickwit-macports syslog-ng logo

Last week, I submitted syslog-ng to openSUSE Leap 16.0. While the distro is still in a pre-alpha stage, everything already works for me as expected. Well, except for syslog-ng, where I found a number of smaller problems. As such, this blog is a call for testing, both for syslog-ng on openSUSE Leap 16.0 and also for the distribution itself. Read the rest at https://www.syslog-ng.com/community/b/blog/posts/call-for-testing-syslog-ng-in-opensuse-leap-16-0 syslog-ng logo

Recently, someone suggested I should check out Alpine Linux and prepare a syslog-ng container image based on it. While not supported by the syslog-ng project, an Alpine-based syslog-ng container image already exist as part of the Linuxserver project. Read more at https://www.syslog-ng.com/community/b/blog/posts/a-syslog-ng-container-image-based-on-alpine-linux syslog-ng logo

Recently I was asked the same question both at my workplace and at EuroBSDCon, the conference where I was presenting: where do you talk next? I had no definite answer. Of course, I am looking forward to the FOSDEM CfP, but I am also looking for new conferences to present syslog-ng and sudo. Do you have any recommendations? You can read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/where-should-i-present-syslog-ng-and-sudo syslog-ng logo Sudo logo

The September syslog-ng newsletter is now available: Improved FreeBSD and MacOS support in 4.8.0 Setting the version number in the syslog-ng configuration Switching containers from Debian Testing to Stable You can read it at: https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-10-4-8-0-release-version-number-debian-stable syslog-ng logo

Version 4.8.1 of syslog-ng was released last week. It is a bugfix release, and it contains fixes for problems also reported by members of the Fedora community. The Fedora 41 release is near, so package updates now need some additional testing, and “karma” in Bodhi. You can find information on how to install syslog-ng 4.8.1 from a testing repo on Fedora 41 beta at https://bodhi.fedoraproject.org/updates/FEDORA-2024-4e812b8a23. This is also the place where you can provide feedback and karma.

Two weeks ago, I was at EuroBSDcon and received a feature request for syslog-ng. The user wanted to collect FreeBSD audit logs together with other logs using syslog-ng. Writing a native driver in C is time consuming. However, creating an integration based on the program() source of syslog-ng is not that difficult. This blog shows you the current state of the FreeBSD audit source, how it works, and its limitations. It is also a request for feedback.

EuroBSDCon was fantastic, as always :-) I talked to many interesting people during the four days about sudo and syslog-ng, and of course also about many other topics. I gave a sudo tutorial, and it went well, with some “students” already planning which features to implement at home. There were many good talks, including one from Dr. Marshall Kirk McKusick, who was with the FreeBSD project right from the beginning, and worked on BSD even earlier.

Last week I wrote about a campaign that we started to resolve issues on GitHub. Some of the fixes are coming from our enthusiastic community. Thanks to this, there is a new syslog-ng-devel port in MacPorts, where you can enable almost all syslog-ng features even for older MacOS versions and PowerPC hardware. Some of the freshly enabled modules include support for Kafka, GeoIP or OpenTelemetry. From this blog entry, you can learn how to install a legacy or an up-to-date syslog-ng version from MacPorts.

The September syslog-ng newsletter is now on-line: You can also contribute to the syslog-ng OSE documentation The $TRANSPORT macro of syslog-ng Rolling RPM platforms added to the syslog-ng package build system It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-09-documentation-transport-macro-rolling-rpms syslog-ng logo

Last time we looked at how syslog-ng can send logs to Quickwit using its Elasticsearch compatible API. This time we are going to look at how to use the OpenTelemetry protocol to send logs to Quickwit with syslog-ng. Read more at https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-quickwit-using-the-opentelemetry-destination-of-syslog-ng syslog-ng logo

We are always looking for new ways to store log messages. Quickwit is a new contender, designed for log storage, and among others, it also provides an Elasticsearch-compatible API. From this blog, you can learn about Quickwit, and how to forward log messages from syslog-ng to it using the Elasticsearch-compatible API. Read more at https://www.syslog-ng.com/community/b/blog/posts/first-steps-with-quickwit-and-syslog-ng syslog-ng logo

For many years, the official syslog-ng container and development containers were based on Debian Testing. We are switching to Debian Stable now. Learn about the history and the reasons for the change now. Read more at https://www.syslog-ng.com/community/b/blog/posts/we-are-switching-syslog-ng-containers-from-debian-testing-to-stable syslog-ng logo

The August syslog-ng newsletter is now on-line: Version 4.8.0 of syslog-ng improves FreeBSD and MacOS support syslog-ng Prometheus exporter Experimental syslog-ng packages for Amazon Linux 2023 It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-08-4-8-0-release-prometheus-amazon-linux syslog-ng logo

Last week One Identity released version 4.8.0 of its open-source log management application. Learn about some of the new features and bug fixes: why upgrade to the latest syslog-ng version, not only on FreeBSD :-) Read more at https://www.syslog-ng.com/community/b/blog/posts/version-4-8-0-of-syslog-ng-improves-freebsd-and-macos-support syslog-ng logo

The syslog-ng configuration starts with a version number declaration. Up until recently, if it was missing, syslog-ng did not start. With syslog-ng 4.8, this is changing. From this blog, you can learn why version information is useful, what workaround you can use if you do not want to edit your syslog-ng configuration on each update, and what changed in version 4.8. You can read the rest of my blog at https://www.

The up-to-date syslog-ng Administration Guide received a new look and easier navigation, as well as better search experience. Best of all, you can now also easily contribute to the syslog-ng documentation. Almost everything is available from the opening page at https://syslog-ng.github.io/ However, here are some direct links for you: The syslog-ng 4.X documentation: https://syslog-ng.github.io/admin-guide/README The syslog-ng documentation contributor guide: https://syslog-ng.github.io/doc-guide/README#how-to-contribute-to-the-documentation The syslog-ng developer guide: https://syslog-ng.github.io/dev-guide/README (which still has some outdated parts)

Do you want to know how your log messages arrived to syslog-ng? The new $TRANSPORT macro provides you with part of the answer. It shows you the protocol variant for network sources, or the kind of local source used. Read more at https://www.syslog-ng.com/community/b/blog/posts/the-transport-macro-of-syslog-ng syslog-ng logo

Prometheus is an open-source monitoring system that collects metrics from your hosts and applications, allowing you to visualize and alert on them. The syslog-ng Prometheus exporter allows you to export syslog-ng statistics, so that Prometheus can collect it. While an implementation in Go has been available for years on GitHub (for more information, see this blog entry), that solution uses the old syslog-ng statistics interface. And while that Go-based implementation still works, syslog-ng 4.

Last year, I received many requests about syslog-ng for Amazon Linux 2023, but I could not find an easy way to create syslog-ng packages. Recently, however, I found that Fedora Copr supports building packages for Amazon Linux 2023. So, with a little bit of experimentation, I got a cut down version of syslog-ng compiled. Read more at https://www.syslog-ng.com/community/b/blog/posts/experimental-syslog-ng-packages-for-amazon-linux-2023 syslog-ng logo

The May syslog-ng newsletter is now on-line: The official syslog-ng OSE documentation got a new look The syslog-ng Administration Guide received a new look and easier navigation. Not only that, but it is also up-to-date now. Besides, there are now contributor guides available both for the documentation and for syslog-ng developers. The admin guide is available at: https://syslog-ng.github.io/admin-guide/README You can reach all syslog-ng OSE-related documentation at: https://syslog-ng.github.io/ If you find any issues, pull requests and problem reports are welcome.

Your favorite Linux distribution is X. You test everything there. However, your colleagues use distro Y, and another team distro Z. Nightmares start here: the same commands install a different set of syslog-ng features, configuration defaults and use different object names in the default configuration. I ran into these problems while working with Gábor Samu on his HPC logging blog. From this blog you can learn about some of the main differences in packaging and configuration of syslog-ng in various Linux distributions and FreeBSD, and how to recognize these when configuring syslog-ng on a different platform.

Version 1.9.16 of sudo will feature a new option for logging: json_compact. Why is this important? This new format can easily be read and parsed by a log management software, like syslog-ng. Note that in this blog I am showing you a sudo feature which has not yet been released officially. You have to compile sudo yourself. By all means, if you have any other application writing JSON-formatted log messages, you can apply most of what you read here with slight modifications.

I love high performance computers, and some of my best friends work in high performance computing (HPC). Obviously, sometimes we also talk about logging. Recently we not just talked, but I also helped Gábor in his first steps with syslog-ng. He summarized his experiences in a blog: Logs are one of those indispensable things in IT when things go wrong. Having worked in technical support for software products in a past life, I’ve likely looked at hundreds (or more) logs over the years, helping to identify issues.

Version 4.2 of syslog-ng introduced a healthcheck option to syslog-ng-ctl. It prints three syslog-ng-related metrics on screen – if it can reach syslog-ng, that is. You can use it from scripts to monitor the health of syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-health-check syslog-ng logo

One Identity Cloud PAM Essentials is the latest security product by One Identity. It provides asset management as well as secure and monitored remote access for One Identity Cloud users to hosts on their local network. I had a chance to test PAM Essentials while still in development. While there, I also integrated it with syslog-ng. From my previous blog, you could learn what PAM Essentials is, and how you can collect its logs using syslog-ng.

One Identity Cloud PAM Essentials is the latest security product by One Identity. It provides asset management as well as secure and monitored remote access for One Identity Cloud users to hosts on their local network. I had a chance to test PAM Essentials while still in development. While there, I also integrated it with syslog-ng. From this blog, you can learn what PAM Essentials is, and how you can collect its logs using syslog-ng.

The March syslog-ng newsletter is now on-line: Native MacOS source in syslog-ng Using OpenTelemetry between syslog-ng instances Collecting even more logs on MacOS using syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-03-macos-opentelemetry syslog-ng logo

Version 4.6 of syslog-ng introduced windows-eventlog-xml-parser(), a dedicated parser for XML-formatted event logs from Windows. It makes the EventData portion of log messages more useful, as it combines two arrays into a list of name-value pairs. https://www.syslog-ng.com/community/b/blog/posts/dedicated-windows-xml-eventlog-parser-in-syslog-ng syslog-ng logo

Sometimes you have many log messages from an app, but none of them have the exact content you need. This is where the grouping-by() parser of syslog-ng can help. It allows you to aggregate information from multiple log messages into a single message. In this blog, I will show you how to parse sshd logs using the patterndb parser of syslog-ng, and then create an aggregate message from the opening and closing log message using grouping-by.

Most log messages fit on a single line. However, Windows and some developer tools and services, like Tomcat, write multi-line log messages. These can come in various formats. For example, new log messages start with a date in a specific format. You use the multi-line-prefix() of the syslog-ng file() source to send multi-line messages as single messages instead of line by line. I must admit that I have never seen multi-line logs in production.

The February syslog-ng newsletter is now on-line: Version 4.5.0 of syslog-ng is now available with OpenObserve JSON API support Syslog-ng PE can now send logs to Google BigQuery Syslog-ng can now do a full configuration check How build services make life easier for upstream developers It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-02-openobserve-configuration-check-build-services syslog-ng logo

Do you have to forward large amounts of logs between two syslog-ng instances? OTLP (OpenTelemetry protocol) support in syslog-ng was contributed by Axoflow, and it can solve this problem. Just like the ewmm() destination, syslog-ng-otlp() forwards most name-value pairs, however, unlike a tcp() connection, it scales well with multiple CPU cores. Support for OpenTelemetry was added to syslog-ng a couple of releases ago. OpenTelemetry is an observability framework, mainly used in Linux / Cloud / Kubernetes environments.

You know that support for MacOS is important when every third visitor at the syslog-ng booth of Red Hat Summit asks if syslog-ng works on MacOS. With the upcoming syslog-ng version 4.6.0, syslog-ng not only compiles on MacOS, but it also collects local log messages natively. From this blog you can learn how to compile syslog-ng yourself, options of the MacOS source, and also a bit of history. https://www.syslog-ng.com/community/b/blog/posts/native-macos-source-in-syslog-ng syslog-ng logo

The January syslog-ng newsletter is now on-line: Why use a http()-based destination in syslog-ng? An overview of Cloudflare’s logging pipeline Working with multiple systemd-journal namespaces in syslog-ng Logging to Humio / Logscale simplified in syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-01-http-cloudflare-systemd-journal-humio-logscale syslog-ng logo

Many Linux distributions provide build services under various names: openSUSE Build Service (OBS), Fedora Copr, and so on. These resources are indispensable for upstream developers, and also for their users. I will demonstrate this through some examples from the syslog-ng project. Note: this blog is loosely based on a talk idea I had for the FOSDEM Distributions Devroom. There is no deep technical information about syslog-ng in this blog. This is more like a history of syslog-ng packaging, and how the fantastic tools by openSUSE and Fedora made it a lot easier and made me an active part of these communities.

One of the most frequent syslog-ng feature requests is now resolved. Welcome the –check-startup option, allowing you to check the syntax and also spot spelling mistakes! You can learn more about it at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-can-now-do-a-full-configuration-check syslog-ng logo

Recently, syslog-ng 4.5.0 was released with many new features. These include sending logs to OpenObserve using its JSON API, support for Google Pub/Sub, a new macro describing message transport mechanisms like RFC 3164 + TCP, an SSL option to ignore validity periods, and many more. You can find a full list of new features and bug fixes in the release notes at: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.5.0 In this blog, you can find some pointers on how to install the very latest syslog-ng version and learn how you can configure syslog-ng to use the OpenObserver JSON API: https://www.

The November syslog-ng newsletter is now on-line: Sending logs to Splunk using syslog-ng Developing a syslog-ng configuration Systemd-journald vs. syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2023-11-splunk-configuration-journald syslog-ng logo

Logging is not just syslog anymore. Still, many syslog-ng users stick to using one of the syslog protocols for log transport and flat files for log storage. While most SIEMs and log analytics tools can receive syslog messages or read them using their own agents, in most cases, you can use the http() destination of syslog-ng as well to send logs to them. You gain extreme performance and an architecture that is easier to maintain.

The October syslog-ng newsletter is now on-line: Why contribute to syslog-ng upstream? Accelerating single TCP connections in syslog-ng: parallelize() Backward compatibility in syslog-ng by using the version number in syslog-ng.conf It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2023-10-contribute-parallelize-compatibility syslog-ng logo

Network traffic is expensive in the cloud, and even a single syslog-ng instance can easily saturate the full bandwidth of a network connection. Compressing HTTP traffic was introduced in syslog-ng Version 4.4.0 and depending on your use case, you can cut down on your expenses on your networking or send more logs using the same budget or bandwidth. Development of this feature was done using a locally installed OpenResty web server, and later tested using Sumologic.

You can read about many interesting syslog-ng features in my blogs. However, it can happen that when you want to try them at home, you fail because the feature is missing. How can you solve such problems? In this blog, I discuss some of the possible solutions from installing sub-packages through using unofficial repositories, to upgrading your OS. This blog focuses on RPM packages for openSUSE / SLES, Fedora / RHEL, and FreeBSD, because these are the packages I know – I am their maintainer.

OpenObserve has an Elasticsearch compatible API for log ingestion, but syslog-ng is not mentioned in the documentation. My plan was to document how to modify the syslog-ng elasticsearch-http() destination, based on API documentation. However, as it turned out, OpenObserve has a ready to use syslog-ng configuration example in the web UI. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-openobserve-using-syslog-ng syslog-ng logo

This year I started publishing a syslog-ng tutorial series both on my blog and on YouTube: https://peter.czanik.hu/posts/syslog-ng-tutorial-toc/ And while the series was praised as the best possible introduction to syslog-ng, viewers also mentioned that one interesting element is missing from it: namely, it does not tell users how to develop a syslog-ng configuration. So, in this blog, learn how to develop a syslog-ng configuration from the ground up! I will explain not just the end result, but also the process and the steps to take to develop a configuration.

Even if most people ask me to compare systemd-journald vs. syslog-ng, I would say that they complement each other. Systemd-journald excels at collecting local log messages, including those of various system services. The focus of syslog-ng is on central log collection and forwarding the logs to a wide variety of destinations after processing and filtering. Combining the two gives you the most flexibility. Read more at https://www.syslog-ng.com/community/b/blog/posts/systemd-journald-vs-syslog-ng syslog-ng logo

Many users are annoyed by the version number included in the syslog-ng configuration. However, it ensures backward compatibility in syslog-ng. It is especially useful when updating to syslog-ng 4 from version 3, but also when updating within the same major version. Read more about it at https://www.syslog-ng.com/community/b/blog/posts/backward-compatibility-in-syslog-ng-by-using-the-version-number-in-syslog-ng-conf syslog-ng logo

One of the highlights of the syslog-ng 4.3.0 release is parallelize(). Normally, syslog-ng processes incoming messages from a TCP connection in a single thread. While this works fine with many connections, it is a bottleneck when using a single or very few high-traffic connections. Using parallelize() allows syslog-ng to process log messages from a single high-traffic TCP connection in multiple threads, thus increasing processing performance on multi-core machines. As you will see, parallelize() helps when you have a single high-traffic TCP connection.

One of the returning questions I received recently: why contribute to the syslog-ng upstream? I guess it is a question many open-source projects receive regularly. There are many generic answers. Here I would like to focus more on syslog-ng, focusing on various parts of it. Of course, the generic answers also apply. Syslog-ng is an open-source project, free to use, modify, and extend. By contributing, you can give something back and improve syslog-ng for everyone.

A few weeks ago, I posted about sngbench, a shell script to measure syslog-ng performance. The performance of syslog-ng is influenced by many factors, including the hardware and OS it runs on, and syslog-ng itself. This blog summarizes some of my findings using the script. https://www.syslog-ng.com/community/b/blog/posts/what-i-learned-about-syslog-ng-performance-using-sngbench syslog-ng logo

No matter how awkward you feel when you hear about UDP syslog in the age of encrypted TCP connections, UDP syslog is here to stay in some special cases. The scalability issues of UDP log collection were first addressed in syslog-ng Open Source Edition (OSE) (the so-reuseport() parameter), and later a more advanced solution arrived to syslog-ng Premium Edition (PE) (the udp-balancer() source). The good news is that a new, open-source implementation is now available as part of syslog-ng 4.

One of the returning syslog-ng questions I receive is how many log messages can a given hardware handle. My typical answer is that it depends on the configuration. I have now an answer, or rather a tool to answer your question: sngbench.sh. It is a shell script that runs from localhost and uses loggen, the bundled benchmarking and testing tool of syslog-ng. It comes with two configurations: a performance-optimized and a realistic one.

In version 4 of syslog-ng, the role of Python became even more important. Previously, all parts of syslog-ng could be extended using Python code, but no actual Python code was provided with syslog-ng. Version 4.0 added a Kubernetes module implemented in Python, while version 4.2 added support for Hypr. But how can we ensure that all Python dependencies are met? In my latest blog I describe the current situation and ask you for feedback!

The May syslog-ng newsletter is now on-line: Learning syslog-ng, the easier way Why syslog over UDP loses messages and how to avoid that Upgrade problems from syslog-ng 3 to 4 It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2023-05-learning-udp-upgrading syslog-ng logo

Version 4 of syslog-ng was released last December. Quite a few people use it already in production. How can you install it for a test drive? It might be already available in your Linux distribution. There are also several unofficial repositories with the latest syslog-ng. From this blog, you can learn how to check your syslog-ng version, where to check if it is not yet installed, and a few additional resources, if you want to install the latest version from unofficial repositories.

Version 4 of syslog-ng works perfectly well in version 3 compatibility mode. However, if you want to use the syslog-ng 4 features, you need to be aware of some significant changes. If you have a simple configuration, like those in Linux distributions, then simply rewriting the version string is most likely enough. However, if you use PatternDB or JSON parsing, any Python code, or an Elasticsearch, or MongoDB destination, you have to be aware of the changes.

The April syslog-ng newsletter is now on-line: Installing a syslog-ng 4 development snapshot on FreeBSD Getting data to Splunk Streaming deduplication in syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2023-04-freebsd-splunk-deduplication syslog-ng logo

Getting data to Splunk can be challenging. Syslog is still the most important data source, and it can provide you with hard-to-solve problems (for example, like high volume, non-compliant messages, unreliable network protocol (UDP), and more). The syslog-ng Premium Edition (PE) and syslog-ng Store Box (SSB) by One Identity can make these challenges manageable. https://www.syslog-ng.com/community/b/blog/posts/getting-data-to-splunk syslog-ng logo

Last year, one of the returning questions I received was how to learn syslog-ng. My answer was that read the first few chapters of the documentation, read my blogs related to your use case, and then read a few relevant parts from the rest of the documentation. Our documentation is praised by users, but it is still a reference documentation. I was asked if a less detailed, more to the point, preferably video tutorial is available.

Version 4 of syslog-ng is now available. The good news is that it is fully backwards compatible. If the version string in your configuration is set to a 3.X version, it will work as expected even after updating to version 4. Of course you might run into corner cases, but I had no problems even with complex configurations. Today, we learn about updating syslog-ng, and some of the new features of syslog-ng 4.

One of the most popular destinations in syslog-ng is Elasticsearch (and OpenSearch, Zinc, Humio, etc.). The 12th part of my syslog-ng #tutorial shows you how to send log messages to Elasticsearch. You can watch the video on YouTube: and the complete playlist at https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-12-elasticsearch-and-opensearch-zinc-humio-etc syslog-ng logo

This is the eleventh part of my syslog-ng tutorial. Last time, we learned about message parsing using syslog-ng. Today, we learn about enriching log messages. You can watch the video on YouTube: and the complete playlist at https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-11-enriching-log-messages syslog-ng logo

This is the tenth part of my syslog-ng tutorial. Last time, we learned about syslog-ng filters. Today, we learn about message parsing using syslog-ng. You can watch the video on YouTube: and the complete playlist at https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-10-parsing syslog-ng logo

This is the ninth part of my syslog-ng tutorial. Last time, we learned about macros and templates. Today, we learn about syslog-ng filters. At the end of the session, we will see a more complex filter and a template function. You can watch the video on YouTube: and the complete playlist at https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-9-filters syslog-ng logo

Unless there is a serious problem, FreeBSD ports usually contains the latest stable syslog-ng release. However, sometimes people want to compile a git snapshot to test a new feature or bugfix. To do that, one way is to generate a syslog-ng release tgz on FreeBSD and edit the syslog-ng port files yourself. However, this needs some practice. As such, an easier solution is to use my weekly development snapshots. Learn how from my latest blog at: https://www.

This is the eighth part of my syslog-ng tutorial. Last time, we learned about network logging. Today, we learn about syslog-ng macros and templates. At the end of the session, we will know how to do a simple log rotation using macros. You can watch the video on YouTube: and the complete playlist at https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-8-macros-and-templates syslog-ng logo

This is the seventh part of my syslog-ng tutorial. Last time, we learned about syslog-ng destinations and the log path. Today, we learn about syslog-ng network logging. At the end of the session, we will send test messages to a syslog-ng network source. You can watch the video on YouTube: Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-7-networking syslog-ng logo

Version 4.0.1 of syslog-ng was released a month ago. Unfortunately, the new release does not compile on FreeBSD. It was a temporary problem in the environment generating the source tgz. The next release is still almost a month away, but you can compile syslog-ng 4.0.1 yourself from my unofficial ports Makefile. Learn how from my latest blog at https://www.syslog-ng.com/community/b/blog/posts/installing-syslog-ng-4-0-1-on-freebsd syslog-ng logo

This is the sixth part of my syslog-ng tutorial. Last time, we learned about syslog-ng source definitions and how to check the syslog-ng version. Today, we learn about syslog-ng destinations and the log path. At the end of the session, we will also perform a quick syntax check. You can watch the video on YouTube: Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-6-destinations-and-log-path syslog-ng logo

This is the fifth part of my syslog-ng tutorial. Last time we had an overview of the syslog-ng configuration and had our first steps working with syslog-ng. Today we learn about syslog-ng source definitions and how to check the syslog-ng version and its enabled features. You can watch the video on YouTube: Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-5-sources syslog-ng logo

This is the fourth part of my syslog-ng tutorial. I hope that since the previous part of my tutorial, you successfully installed syslog-ng. In this part we will finally work with syslog-ng, not just learn about the theoretical background. We will do basic configuration and testing. You can watch the video on YouTube: Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-4-configuration-and-testing syslog-ng logo

Installing syslog-ng on Mac is easy, if you use Homebrew for 3rd party packages. Previously, you had to install dependencies and then compile syslog-ng from source. Now, a single command takes care of everything! homebrew logo Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-is-now-available-in-homebrew syslog-ng logo

Welcome to the third part of my syslog-ng tutorial. Today we cover the various syslog-ng editions (open source, commercial and appliance), and where to get them from. The focus of this tutorial series is the Open Source Edition (OSE), but to avoid confusion, I also briefly introduce the other two. You can watch the video on YouTube: Or you can read the rest of my blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-3-syslog-ng-editions-and-where-to-get-them-from This is a boring, but important part, do not skip it!

Welcome to the second part of my syslog-ng tutorial series. In this part, we cover some of the basic concepts behind syslog-ng. Last time we defined syslog-ng as an enhanced logging daemon with a strong focus on portability and high-performance central log collection. Let us pull this sentence apart, as all words are here for a reason. The original syslog implementation was pretty simple: it collected log messages from applications and sorted them to various files.

Welcome to the first part of my syslog-ng tutorial series. In this part, I give you a quick introduction what to expect from this series and try to define what syslog-ng is. I plan to release parts of my tutorial around every week. Of course, the Christmas holidays and the upcoming conference season may cause some delays. Each part will be released as a blog accompanied by a video. It is up to you, which version you follow.

The November syslog_ng newsletter is now on-line: Testing syslog-ng 4.0 syslog-ng Store Box federated single sign-on support via OpenID Connect (OIDC) Nightly syslog-ng container images Type support: working with sudo logs in syslog-ng 4.0 It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-11-4-0-oidc-nightly-sudo syslog-ng logo

From now on, as I want to reach as many as possible, you can also read sudo and syslog-ng news from me on Mastodon. You can find my account at: https://fosstodon.org/@PCzanik Mastodon is a decentralized network of servers. I chose a server called “Fosstodon” as it is focused on open source software. Some of the projects I participate in are already there: BastilleBSD and openSUSE. As usual, next to my usual syslog-ng and sudo posts, you will also sometimes hear from me about OpenPOWER and ARM with some occasional photos from my hiking trips :-)

Each new MacOS release brings some surprises when it comes to compiling syslog-ng. MacOS Ventura has been released recently, while Homebrew has also been updated. So here are some updated instructions for MacOS Ventura (and also for the last MacOS minor release before Ventura). https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-ventura syslog-ng logo

How to get started with syslog-ng? There are two main resources: the syslog-ng documentation and the syslog-ng blogs. You should learn the concepts and basics from the documentation. The blogs document use cases and you can use the docs as a reference. syslog-ng logo Read the rest of my blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-how-to-get-started-with-learning-syslog-ng

The syslog-ng team started publishing container images many years ago. For quite a while, it was a manual process, however, a few releases ago, publishing a container image became part of the release process. Recently, nightly container images have also become available, so you can test the latest features and bug fixes easily. The syslog-ng images are still available under the Balabit namespace on the Docker hub. Balabit was bought by One Identity almost five years ago, and we stopped using the old company name years ago.

The September syslog-ng newsletter is now on-line: 3.38.1 released, 4.0 almost feature complete syslog-ng Store Box SQL source Why is my syslog-ng disk-buffer file so huge even when it is empty? Nightly syslog-ng builds for Debian and Ubuntu It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-09-3-38-sql-disk-buffer-nightly syslog-ng logo

Version 4.0 of syslog-ng is right around the corner. It hasn’tyet been released; however, you can already try some of its features. The largest and most interesting change is type support. Right now, name-value pairs within syslog-ng are represented as text, even if the PatternDB or JSON parsers could see the actual type of the incoming data. This does not change, but starting with 4.0, syslog-ng will keep the type information, and use it correctly on the destination side.

The July syslog-ng newsletter is now on-line: RHEL 9 syslog-ng news How does the syslog-ng disk-buffer work? Installing syslog-ng on Microsoft Linux It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-06-rhel-9-disk-buffer-microsoft-linux syslog-ng logo

“Pass the SALT” (PTS) is a small IT security conference in Lille, France. It has less participants than speakers at the RSA conference. I gave talks at both events. RSA is a lot more prestigious event, but I still prefer PTS. Why? Small Is Beautiful As you could guess from my introduction, PTS is a small event. It is run by volunteers. It is also a free event thanks to sponsors.

A three parts blog series: The syslog-ng disk buffer is one of the most often used syslog-ng options to ensure message delivery. However, it is not always necessary and using the safest variant has serious performance impacts. If you utilize disk-buffer in your syslog-ng configuration, it is worth to make sure that you use a recent syslog-ng version. From this blog, you can learn when to use the disk-buffer option, the main differences between reliable and non-reliable disk-buffer, and why is it worth to use the latest syslog-ng version.

Yes, Microsoft has its own Linux distribution, called CBL-Mariner. It is an internal Linux distribution by Microsoft used for cloud infrastructure and edge products and services. And even if it is not installed in the OS by default, CBL-Mariner also includes syslog-ng. Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/installing-syslog-ng-on-microsoft-linux to learn how to install syslog-ng on it and what features are available. syslog-ng logo

Red Hat Enterprise Linux 9 became generally available recently. Version 3.35 of syslog-ng has been part of EPEL 9 (the semi-official extra software repo for RHEL maintained by Fedora packagers) for a while and now I enabled a few more destination drivers. I also enabled RHEL 9 support in my unofficial Git snapshot packages, so I can support RHEL 9 together with other RHEL and Fedora versions on the next syslog-ng release.

I’m not superstitious, so I never really cared about black cats, Friday the 13th, and other signs of (imagined) trouble. Last Friday (which was the 13th) I had an article printed in a leading computer magazine in Hungary, and I gave my first IRL talk at a conference in well over two years. Best of all, I also met many people, some for the first time in real life. Free Software Conference: sudo talk Last Friday, I gave a talk at the Free Software Conference in Szeged.

Recently, I started my own blog, and as Google Analytics seems to miss a good part of visitors, I wanted to analyze my web server logs myself. I use syslog-ng to read Apache logs, process them, and store them to Elasticsearch. Along the way, I resolve the IP address using a Python parser, analyze the Agent field of the logs, and also use GeoIP to locate the user on the map.

What hardware to use for a syslog-ng server? It is a frequent question with no definite answer. It depends on many factors: the number and type of sources, the number of logs, the way logs are processed, and so on. My experience is that for the majority users even a Raspberry Pi would be enough. But of course, not for everyone. You can read the rest of my blog at https://www.

This year the syslog-ng project will participate in the Google Summer of Code (GSoC) as a mentor organization again. If you are a university student or otherwise eligible to participate in the GSoC program, you can choose to develop a new feature for syslog-ng. Read my blog to learn why to choose syslog-ng and how to get started: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-in-gsoc-2022 syslog-ng logo

For many years, you could use the match() filter of syslog-ng to parse log messages with regular expressions. However, the primary function of match() is filtering. Recent syslog-ng versions now have a dedicated regular expression parser, the regexp-parser(). So, you should use match() only if your primary use case is filtering. Otherwise, use the regexp-parser for parsing, as it is a lot more flexible. You can read the rest of my blog at https://www.

The latest pull request to syslog-ng adds a really useful feature: the flip-parser(): https://github.com/syslog-ng/syslog-ng/pull/3971 It allows you to flip the message text, reverse it, or both. As I also reported a couple of minor problems related to UTF-8 character handling, this PR most likely will not be merged today. However, you can compile it yourself, or if you use openSUSE Tumbleweed, use my packages from the openSUSE Build Service. You can read the rest of my blog at https://www.

There are multiple ways in syslog-ng to limit message rate. The throttle() option of syslog-ng destinations tries to make sure that all messages are delivered without exceeding a specified message rate. The rate-limit() filter introduced in syslog-ng 3.36 drops surplus log messages, making sure that a processing pipeline or destination is not overloaded with log messages. Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/the-difference-between-throttle-and-rate-limit-in-syslog-ng syslog-ng logo

Most of syslog-ng works perfectly well on MacOS; however, there is no native driver to collect local log messages. Due to this, in the past, the system() source did not work on MacOS, thus the default syslog-ng configuration failed to start. Version 3.36 of syslog-ng includes a workaround: it follows /var/log/system.log. You can read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/the-system-source-of-syslog-ng-now-also-works-on-macos syslog-ng logo

The March syslog-ng newsletter is now on-line: syslog-ng future: the path to syslog-ng 4 MQTT source Another use for the syslog-ng elasticsearch-http destination: Zinc Sending logs to Elastic Cloud using syslog-ng syslog-ng 3.36 is now available It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-03-syslog-ng-4-mqtt-source-zinc-elastic-cloud-3-36 syslog-ng logo

As explained in my previous post, we do have some features already in mind for syslog-ng 4, even though the work on creating a long term set of objectives for the syslog-ng project is not finished yet. One of the themes that I have working code for already, is typing. syslog-ng traditionally assumes that log data, even if it comes in a structured form (like RFC5424 structured data or JSON) is primarily textual in nature.

Version 3.36 of syslog-ng brings us many interesting new features. There is now basic support for system() source on MacOS, TLS 1.3 ciphers can now be restricted, TLS keylog support was added, symlink creation to the latest file, and there are many new possibilities in syslog parsing. From this blog, you can learn about some of the new 3.36 features, and we will test symlink creation, which is a community-contributed feature.

Last week, the ivykis library, the most important core dependency of syslog-ng landed in EPEL 9 successfully. There are still plenty of dependencies missing, but this way, I could submit a slightly cut down version of syslog-ng to EPEL 9. Hopefully the rest of the dependencies will arrive in EPEL 9 as well. I plan to update the syslog-ng package as soon as the dependencies arrive. Luckily, these are only needed to enable some less frequently used syslog-ng destination drivers, no core functionality is affected.

Recently I got some complaints that it is difficult to figure out how to contact the syslog-ng team to get help or report problems. Most of this information is available both on the syslog-ng website and at the syslog-ng repository on GitHub, but collecting here all information might be still useful for some people. Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/contacting-the-syslog-ng-team-reporting-problems-asking-questions syslog-ng logo

General availability of Elasticsearch 8 was announced last week. There were quite a few rumors that it will break compatibility with third party tools. I tested it as soon as I had a little time: I am happy to share that anything I tested with the elasticsearch-http() destination of syslog-ng still seems to work perfectly well with the latest version of Elasticsearch. You can read the rest of my blog at https://www.

Those who follow me on LinkedIn might have seen an automatic post about my work anniversary. Well, almost nothing of that post is true, but I still consider it to be my real starting date. However, the official date is also impressive: 11.5 years, almost three times the industry average spent at the same workplace. So, why do I say that the LinkedIn post is not true? Well, because all its major facts are wrong.

The February syslog-ng newsletter is now on-line: syslog-ng relaunch Sequence – making PatternDB creation for syslog-ng easier Syslog-ng on MacOS Monterey Installing syslog-ng on CentOS Stream 9 It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-01-reboot-sequence-monterey-centos-9 syslog-ng logo

Seemingly a boring topic, Balázs Scheidler finds open source licensing fascinating. It allows him to work on syslog-ng even though Balabit was acquired. He writes: “I mentioned in the previous post that I would like to focus on syslog-ng and put it more into the spotlight. I also mentioned that Balabit, the company I was a founder of and the commercial sponsor behind syslog-ng, was acquired by One Identity ~4 years ago.

This weekend I am going to give a talk about sudo in the security track of FOSDEM. I will talk a few words about logging at each major point I mention, but I cannot go into too much detail there. So, consider this blog both as a teaser and an extension to my FOSDEM talk. You will learn how to work with JSON formatted logs in syslog-ng and also about new sudo features along the way.

Balázs Scheidler, founder of the syslog-ng project, started a new blog where he details why and how he started to work on syslog-ng even more actively. He also asks for your feedback! “syslog-ng has been around for decades: I started coding the first version of syslog-ng in September 1998, circa 24 years ago. The adoption of syslog-ng skyrocketed soon after that: people installed it in place of the traditional syslogd across the globe.

The Elastic Cloud is a service by Elastic providing Elasticsearch and related services in an easy-to-use package. Last year someone reported an issue that it does not work properly with syslog-ng. I did not have time to investigate at that time. Now I started a free trial and soon my log messages from syslog-ng started to appear in Kibana in Elastic Cloud. From this blog you can learn how to configure syslog-ng for the Elastic Cloud.

There is a new drop-in replacement for Elasticsearch, at least if you don’t mind the limitations and the alpha status. However, it definitely lives up to the promise that it provides an Elasticsearch-compatible API for data ingestion. I tested it with the elasticsearch-http() destination of syslog-ng, and it worked perfectly after I modified the URL in the configuration example I found. So, what is Zinc? It is a search engine written in Go that provides an Elasticsearch-compatible API for data ingestion.

The syslog-ng application is part of all major Linux distributions, and you can usually install syslog-ng from the official repositories. If you use just the core functionality of syslog-ng, use the package in your distribution repository (apt-get install syslog-ng), and you can stop reading here. However, if you want to use the features of newer syslog-ng versions (for example, send log messages to MQTT or Apache Kafka), you have to either compile the syslog-ng from source, or install it from unofficial repositories.

For the past few days, the IT news has been abuzz with announcements from CES. As usual, I’m following them on Engadget. I must admit, that there were just a very few announcements which really caught my attention. And my favorite announcement is the most boring of them all :-) Foldable tablet by ASUS: I still use my Google Pixel C tablet almost every day. It’s almost six years old and waiting for replacement.

Version 3.35.1 of syslog-ng introduced an MQTT source. Just for some fun in the last syslog-ng blog post of the year, I created an endless loop using syslog-ng and the Mosquitto MQTT broker. Of course, it does not have much practical value other than possibly a bit of stress testing, but hopefully provides a fun introduction to MQTT-related technologies in syslog-ng. Read my blog at https://www.syslog-ng.com/community/b/blog/posts/creating-an-endless-loop-using-mqtt-and-syslog-ng syslog-ng logo

The December syslog-ng newsletter is now on-line: Sending logs to Panther using syslog-ng Reducing the complexity of log management Sending logs to Humio using the elasticsearch-http() destination of syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-12-humio-log-management-panther syslog-ng logo

CentOS Stream 9 has been around for a while, but it was officially announced just a few days ago. I already tested some earlier snapshots and they had some rough edges. The current version installed without random crashes, has networking and runs smoothly. EPEL – the semi-official repository by Fedora maintainers – is already there, but practically empty, syslog-ng or it’s dependencies are not yet there. As someone asked about syslog-ng support, I had a first try at building it.

It is easy to over-complicate log management. Almost all departments in a company need to log messages for their daily activities. However, installing several different log management and analysis systems in parallel is a nightmare both from a security and an operations perspective and wastes many resources. You cannot always reduce the number of log analysis systems, but you can reduce the complexity of log management. Let me show you, how.

Each new MacOS release brings some surprises when it comes to compiling syslog-ng. Just a couple of months ago, I provided you with a couple of pointers on how to compile syslog-ng on MacOS. Since then, MacOS Monterey was released and Homebrew was updated. So, here are some updated instructions for MacOS Monterey. You can read my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-monterey syslog-ng logo

Better late than never I just put online the November syslog-ng newsletter. Topics include: syslog-ng version 3.35.1 is now available Sending logs from syslog-ng store box to Splunk MacOS support Syslog-ng 3.34: MQTT destination with TLS and WebSocket support It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-11-3-35-ssb-macos-mqtt-destination-updates

One of the most popular applications to feed Splunk with syslog messages is syslog-ng. However not everyone is happy to work on the command line anymore. This is where syslog-ng store box (SSB), an appliance built around syslog-ng, can help. The SSB GUI provides you not only with an easyto-use interface to configure most syslog-ng features, but also a search interface and complete log life cycle management. It can forward log messages to several destinations, recently also to Splunk’s HTTP Event Collector (HEC).

Sequence – making PatternDB creation for syslog-ng easier We are well into the 21st century, but most of the log messages still arrive in an unstructured format. For well over a decade, syslog-ng had a solution to turn unstructured messages into name-value pairs, called PatternDB. However, creating a pattern database for PatternDB from scratch is a source of major pain. Or rather, it was: sequence-rtg – a fork of the sequence log analyzer – provides a new hope!

One of the most popular syslog-ng destinations is Elasticsearch. Humio, a log management provider, supports a broad range of ingest options and interfaces, including an Elasticsearch-compatible API. Last week, Humio announced Humio Community Edition, which provides the full Humio experience for free, with some limitations on daily ingestion and retention time. I tested the Community Edition, and it works perfectly well with syslog-ng. If you come from the Humio side, you might wonder what syslog-ng is.

Panther is an open-source log management system, which is also available as a service for a time-limited trial. It is still in beta phase, but it looks promising. You can see the “beta” sign on its opening page: https://app.panther.support/ I tested the time-limited cloud service version, but you can also install it locally, either from Dockerhub, or you can build the containers locally from the source. Even if it is still in beta phase, Panther comes with detailed documentation.

Version 3.33 of syslog-ng arrived with basic MQTT support. Version 3.34 has added many important features to it: user authentication, TLS support and WebSocket support. These features give you both security and flexibility while sending log messages to an MQTT broker. This blog helps you to make your first steps securing your MQTT connection: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-34-mqtt-destination-with-tls-and-websocket-support

Dear syslog-ng users, This is the 94th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. Topics include: Version 3.34.1 of syslog-ng available Syslog-ng updated in OpenBSD ports OpenSearch and syslog-ng Creating a new http()-based syslog-ng destination: Seq It is available at: https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-09-3-34-openbsd-opensearch-http-destination

Better late than never I just put online the July syslog-ng newsletter. Topics include: Sending alerts to Discord and others from syslog-ng using Apprise: blocks and Python templates Rocky Linux, AlmaLinux, CentOS & syslog-ng MongoDB support improved in syslog-ng 3.32 It is available at https://www.syslog-ng.com/community/b/blog/posts/insider-2021-07-alerting-centos-alternatives-mongodb

For the past couple of months, Yash Mathne has been working on testing syslog-ng on MacOS as a GSoC (Google Summer of Code) student. He worked both on x86 and on the freshly released ARM hardware. And we have some good news here to share: while there is still room for improvement, most of syslog-ng works perfectly well on MacOS. Read my blog for some historical background and the GSoC report: https://www.

Collecting process accounting logs on Linux with syslog-ng Process accounting logs are collected into binary log files on Linux. You can turn them into human readable format locally, using various tools. You can also use syslog-ng to read those files. Lean how syslog-ng can parse those binary logs, create name-value pairs from them and store the results from my latest blog: https://www.syslog-ng.com/community/b/blog/posts/collecting-process-accounting-logs-on-linux-with-syslog-ng

One of the most popular destinations in syslog-ng is Elasticsearch. Due to the license change of the Elastic stack, some people changed quickly to Grafana/Loki and other technologies. However, most syslog-ng users decided to wait and see. Version 1.0.0 of OpenSearch, a fork of the Elastic code base from before the license change is now available. Elastic also published a new release last week. For this blog, I tested the latest and greatest from both product lines and I’m sharing my experiences.

Yes, it’s a syslog-ng blog from me, and it’s not on https://syslog-ng.com/ :-) The reason is simple: this is not a technical blog. This is my story about how I found the Turris Omnia Linux router and how this lead to working together with the Turris guys. The beginnings When I ordered my Turris Omnia, I did not know that it ran syslog-ng. All I knew that it was an ARM device and that it ran Linux.

Using the udp-balancer() source of syslog-ng PE UDP-based log collection is so last century. We had TCP-based log collection for decades and TLS encryption to secure connections. Still, UDP is in wide use, especially at large companies and industrial automation, where every change is slow. In most cases, UDP logging is used by networking devices, but sometimes it is just left there from ancient times and people are reluctant to change it.

Syslog-ng 3.33: the MQTT destination Version 3.33 of syslog-ng introduced an MQTT destination. It uses the paho-c client library to send log messages to an MQTT broker. The current implementation supports version 3.1 and 3.1.1 of the protocol over non-encrypted connections, but this is only a first step. From this blog, you can learn how to configure and test the mqtt() destination in syslog-ng. Read my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-33-the-mqtt-destination