Below you will find pages that utilize the taxonomy term “syslog-ng”
Other
The syslog-ng insider 2022-07: RHEL 9; disk-buffer; Microsoft Linux;
The July syslog-ng newsletter is now on-line:
RHEL 9 syslog-ng news How does the syslog-ng disk-buffer work? Installing syslog-ng on Microsoft Linux It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-06-rhel-9-disk-buffer-microsoft-linux
syslog-ng logo
Posts
My Favorite IT Security Event: Pass the SALT
“Pass the SALT” (PTS) is a small IT security conference in Lille, France. It has less participants than speakers at the RSA conference. I gave talks at both events. RSA is a lot more prestigious event, but I still prefer PTS. Why?
Small Is Beautiful As you could guess from my introduction, PTS is a small event. It is run by volunteers. It is also a free event thanks to sponsors.
Other
The syslog-ng disk-buffer
A three parts blog series: The syslog-ng disk buffer is one of the most often used syslog-ng options to ensure message delivery. However, it is not always necessary and using the safest variant has serious performance impacts. If you utilize disk-buffer in your syslog-ng configuration, it is worth to make sure that you use a recent syslog-ng version.
From this blog, you can learn when to use the disk-buffer option, the main differences between reliable and non-reliable disk-buffer, and why is it worth to use the latest syslog-ng version.
Other
Installing syslog-ng on Microsoft Linux
Yes, Microsoft has its own Linux distribution, called CBL-Mariner. It is an internal Linux distribution by Microsoft used for cloud infrastructure and edge products and services. And even if it is not installed in the OS by default, CBL-Mariner also includes syslog-ng.
Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/installing-syslog-ng-on-microsoft-linux to learn how to install syslog-ng on it and what features are available.
syslog-ng logo
Other
RHEL 9 syslog-ng news
Red Hat Enterprise Linux 9 became generally available recently. Version 3.35 of syslog-ng has been part of EPEL 9 (the semi-official extra software repo for RHEL maintained by Fedora packagers) for a while and now I enabled a few more destination drivers. I also enabled RHEL 9 support in my unofficial Git snapshot packages, so I can support RHEL 9 together with other RHEL and Fedora versions on the next syslog-ng release.
Posts
Friday the 13th: a lucky day :-)
I’m not superstitious, so I never really cared about black cats, Friday the 13th, and other signs of (imagined) trouble. Last Friday (which was the 13th) I had an article printed in a leading computer magazine in Hungary, and I gave my first IRL talk at a conference in well over two years. Best of all, I also met many people, some for the first time in real life.
Free Software Conference: sudo talk Last Friday, I gave a talk at the Free Software Conference in Szeged.
Other
Analyzing Apache HTTPD logs in syslog-ng
Recently, I started my own blog, and as Google Analytics seems to miss a good part of visitors, I wanted to analyze my web server logs myself. I use syslog-ng to read Apache logs, process them, and store them to Elasticsearch. Along the way, I resolve the IP address using a Python parser, analyze the Agent field of the logs, and also use GeoIP to locate the user on the map.
Other
Hardware for a syslog-ng server
What hardware to use for a syslog-ng server? It is a frequent question with no definite answer. It depends on many factors: the number and type of sources, the number of logs, the way logs are processed, and so on. My experience is that for the majority users even a Raspberry Pi would be enough. But of course, not for everyone.
You can read the rest of my blog at https://www.
Other
Syslog-ng in GSoC 2022
This year the syslog-ng project will participate in the Google Summer of Code (GSoC) as a mentor organization again. If you are a university student or otherwise eligible to participate in the GSoC program, you can choose to develop a new feature for syslog-ng.
Read my blog to learn why to choose syslog-ng and how to get started: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-in-gsoc-2022
syslog-ng logo
Other
Using the regexp-parser of syslog-ng
For many years, you could use the match() filter of syslog-ng to parse log messages with regular expressions. However, the primary function of match() is filtering. Recent syslog-ng versions now have a dedicated regular expression parser, the regexp-parser(). So, you should use match() only if your primary use case is filtering. Otherwise, use the regexp-parser for parsing, as it is a lot more flexible.
You can read the rest of my blog at https://www.
Other
New syslog-ng parser: flip-parser()
The latest pull request to syslog-ng adds a really useful feature: the flip-parser(): https://github.com/syslog-ng/syslog-ng/pull/3971
It allows you to flip the message text, reverse it, or both. As I also reported a couple of minor problems related to UTF-8 character handling, this PR most likely will not be merged today. However, you can compile it yourself, or if you use openSUSE Tumbleweed, use my packages from the openSUSE Build Service.
You can read the rest of my blog at https://www.
Other
The difference between throttle() and rate-limit() in syslog-ng
There are multiple ways in syslog-ng to limit message rate. The throttle() option of syslog-ng destinations tries to make sure that all messages are delivered without exceeding a specified message rate. The rate-limit() filter introduced in syslog-ng 3.36 drops surplus log messages, making sure that a processing pipeline or destination is not overloaded with log messages.
Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/the-difference-between-throttle-and-rate-limit-in-syslog-ng
syslog-ng logo
Other
The system() source of syslog-ng now also works on MacOS
Most of syslog-ng works perfectly well on MacOS; however, there is no native driver to collect local log messages. Due to this, in the past, the system() source did not work on MacOS, thus the default syslog-ng configuration failed to start. Version 3.36 of syslog-ng includes a workaround: it follows /var/log/system.log.
You can read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/the-system-source-of-syslog-ng-now-also-works-on-macos
syslog-ng logo
Other
The syslog-ng insider 2022-03: syslog-ng 4; MQTT source; Zinc; Elastic Cloud; 3.36;
The March syslog-ng newsletter is now on-line:
syslog-ng future: the path to syslog-ng 4 MQTT source Another use for the syslog-ng elasticsearch-http destination: Zinc Sending logs to Elastic Cloud using syslog-ng syslog-ng 3.36 is now available It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-03-syslog-ng-4-mqtt-source-zinc-elastic-cloud-3-36
syslog-ng logo
Other
syslog-ng 4 theme: typing
As explained in my previous post, we do have some features already in mind for syslog-ng 4, even though the work on creating a long term set of objectives for the syslog-ng project is not finished yet. One of the themes that I have working code for already, is typing.
syslog-ng traditionally assumes that log data, even if it comes in a structured form (like RFC5424 structured data or JSON) is primarily textual in nature.
Other
Syslog-ng 3.36 news: better TLS 1.3, basic MacOS support, and many more
Version 3.36 of syslog-ng brings us many interesting new features. There is now basic support for system() source on MacOS, TLS 1.3 ciphers can now be restricted, TLS keylog support was added, symlink creation to the latest file, and there are many new possibilities in syslog parsing.
From this blog, you can learn about some of the new 3.36 features, and we will test symlink creation, which is a community-contributed feature.
Other
A minimalist syslog-ng package is heading to EPEL 9
Last week, the ivykis library, the most important core dependency of syslog-ng landed in EPEL 9 successfully. There are still plenty of dependencies missing, but this way, I could submit a slightly cut down version of syslog-ng to EPEL 9. Hopefully the rest of the dependencies will arrive in EPEL 9 as well. I plan to update the syslog-ng package as soon as the dependencies arrive. Luckily, these are only needed to enable some less frequently used syslog-ng destination drivers, no core functionality is affected.
Other
Contacting the syslog-ng team: reporting problems, asking questions
Recently I got some complaints that it is difficult to figure out how to contact the syslog-ng team to get help or report problems. Most of this information is available both on the syslog-ng website and at the syslog-ng repository on GitHub, but collecting here all information might be still useful for some people.
Read the rest of my blog at https://www.syslog-ng.com/community/b/blog/posts/contacting-the-syslog-ng-team-reporting-problems-asking-questions
syslog-ng logo
Other
Elasticsearch 8 and syslog-ng
General availability of Elasticsearch 8 was announced last week. There were quite a few rumors that it will break compatibility with third party tools. I tested it as soon as I had a little time: I am happy to share that anything I tested with the elasticsearch-http() destination of syslog-ng still seems to work perfectly well with the latest version of Elasticsearch.
You can read the rest of my blog at https://www.
Posts
12 years of syslog-ng (and sudo)
Those who follow me on LinkedIn might have seen an automatic post about my work anniversary. Well, almost nothing of that post is true, but I still consider it to be my real starting date. However, the official date is also impressive: 11.5 years, almost three times the industry average spent at the same workplace.
So, why do I say that the LinkedIn post is not true? Well, because all its major facts are wrong.
Other
The syslog-ng Insider 2022-02: Reboot; Sequence; Monterey; CentOS 9;
The February syslog-ng newsletter is now on-line:
syslog-ng relaunch Sequence – making PatternDB creation for syslog-ng easier Syslog-ng on MacOS Monterey Installing syslog-ng on CentOS Stream 9 It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2022-01-reboot-sequence-monterey-centos-9
syslog-ng logo
Other
syslog-ng-future.blog? Is this a fork or what?
Seemingly a boring topic, Balázs Scheidler finds open source licensing fascinating. It allows him to work on syslog-ng even though Balabit was acquired. He writes:
“I mentioned in the previous post that I would like to focus on syslog-ng and put it more into the spotlight. I also mentioned that Balabit, the company I was a founder of and the commercial sponsor behind syslog-ng, was acquired by One Identity ~4 years ago.
Other
Working with JSON logs from sudo in syslog-ng
This weekend I am going to give a talk about sudo in the security track of FOSDEM. I will talk a few words about logging at each major point I mention, but I cannot go into too much detail there. So, consider this blog both as a teaser and an extension to my FOSDEM talk. You will learn how to work with JSON formatted logs in syslog-ng and also about new sudo features along the way.
Other
syslog-ng relaunch
Balázs Scheidler, founder of the syslog-ng project, started a new blog where he details why and how he started to work on syslog-ng even more actively. He also asks for your feedback!
“syslog-ng has been around for decades: I started coding the first version of syslog-ng in September 1998, circa 24 years ago. The adoption of syslog-ng skyrocketed soon after that: people installed it in place of the traditional syslogd across the globe.
Other
Sending logs to Elastic Cloud using syslog-ng
The Elastic Cloud is a service by Elastic providing Elasticsearch and related services in an easy-to-use package. Last year someone reported an issue that it does not work properly with syslog-ng. I did not have time to investigate at that time. Now I started a free trial and soon my log messages from syslog-ng started to appear in Kibana in Elastic Cloud.
From this blog you can learn how to configure syslog-ng for the Elastic Cloud.
Other
Another use for the syslog-ng elasticsearch-http destination: Zinc
There is a new drop-in replacement for Elasticsearch, at least if you don’t mind the limitations and the alpha status. However, it definitely lives up to the promise that it provides an Elasticsearch-compatible API for data ingestion. I tested it with the elasticsearch-http() destination of syslog-ng, and it worked perfectly after I modified the URL in the configuration example I found.
So, what is Zinc? It is a search engine written in Go that provides an Elasticsearch-compatible API for data ingestion.
Other
Installing the latest syslog-ng on Ubuntu and other DEB distributions
The syslog-ng application is part of all major Linux distributions, and you can usually install syslog-ng from the official repositories. If you use just the core functionality of syslog-ng, use the package in your distribution repository (apt-get install syslog-ng), and you can stop reading here. However, if you want to use the features of newer syslog-ng versions (for example, send log messages to MQTT or Apache Kafka), you have to either compile the syslog-ng from source, or install it from unofficial repositories.
Posts
CES 2022: my favorite announcement comes from AMD, and why it's interesting for syslog-ng
For the past few days, the IT news has been abuzz with announcements from CES. As usual, I’m following them on Engadget. I must admit, that there were just a very few announcements which really caught my attention. And my favorite announcement is the most boring of them all :-)
Foldable tablet by ASUS: I still use my Google Pixel C tablet almost every day. It’s almost six years old and waiting for replacement.
Other
Creating an endless loop using MQTT and syslog-ng
Version 3.35.1 of syslog-ng introduced an MQTT source. Just for some fun in the last syslog-ng blog post of the year, I created an endless loop using syslog-ng and the Mosquitto MQTT broker. Of course, it does not have much practical value other than possibly a bit of stress testing, but hopefully provides a fun introduction to MQTT-related technologies in syslog-ng.
Read my blog at https://www.syslog-ng.com/community/b/blog/posts/creating-an-endless-loop-using-mqtt-and-syslog-ng
syslog-ng logo
Other
The syslog-ng insider 2021-12: Humio; Log Management; Panther;
The December syslog-ng newsletter is now on-line:
Sending logs to Panther using syslog-ng Reducing the complexity of log management Sending logs to Humio using the elasticsearch-http() destination of syslog-ng It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-12-humio-log-management-panther
syslog-ng logo
Other
Installing syslog-ng on CentOS Stream 9
CentOS Stream 9 has been around for a while, but it was officially announced just a few days ago. I already tested some earlier snapshots and they had some rough edges. The current version installed without random crashes, has networking and runs smoothly. EPEL – the semi-official repository by Fedora maintainers – is already there, but practically empty, syslog-ng or it’s dependencies are not yet there. As someone asked about syslog-ng support, I had a first try at building it.
Other
Reducing the complexity of log management
It is easy to over-complicate log management. Almost all departments in a company need to log messages for their daily activities. However, installing several different log management and analysis systems in parallel is a nightmare both from a security and an operations perspective and wastes many resources. You cannot always reduce the number of log analysis systems, but you can reduce the complexity of log management. Let me show you, how.
Other
Syslog-ng on MacOS Monterey
Each new MacOS release brings some surprises when it comes to compiling syslog-ng. Just a couple of months ago, I provided you with a couple of pointers on how to compile syslog-ng on MacOS. Since then, MacOS Monterey was released and Homebrew was updated. So, here are some updated instructions for MacOS Monterey.
You can read my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-monterey
syslog-ng logo
Other
The syslog-ng Insider 2021-11: 3.35; SSB; MacOS; mqtt() destination updates;
Better late than never I just put online the November syslog-ng newsletter. Topics include:
syslog-ng version 3.35.1 is now available Sending logs from syslog-ng store box to Splunk MacOS support Syslog-ng 3.34: MQTT destination with TLS and WebSocket support It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-11-3-35-ssb-macos-mqtt-destination-updates
Other
Sending logs from syslog-ng store box to Splunk
One of the most popular applications to feed Splunk with syslog messages is syslog-ng. However not everyone is happy to work on the command line anymore. This is where syslog-ng store box (SSB), an appliance built around syslog-ng, can help. The SSB GUI provides you not only with an easyto-use interface to configure most syslog-ng features, but also a search interface and complete log life cycle management. It can forward log messages to several destinations, recently also to Splunk’s HTTP Event Collector (HEC).
Other
Sequence – making PatternDB creation for syslog-ng easier
Sequence – making PatternDB creation for syslog-ng easier We are well into the 21st century, but most of the log messages still arrive in an unstructured format. For well over a decade, syslog-ng had a solution to turn unstructured messages into name-value pairs, called PatternDB. However, creating a pattern database for PatternDB from scratch is a source of major pain. Or rather, it was: sequence-rtg – a fork of the sequence log analyzer – provides a new hope!
Other
Sending logs to Humio using the elasticsearch-http() destination of syslog-ng
One of the most popular syslog-ng destinations is Elasticsearch. Humio, a log management provider, supports a broad range of ingest options and interfaces, including an Elasticsearch-compatible API. Last week, Humio announced Humio Community Edition, which provides the full Humio experience for free, with some limitations on daily ingestion and retention time. I tested the Community Edition, and it works perfectly well with syslog-ng.
If you come from the Humio side, you might wonder what syslog-ng is.
Other
Sending logs to Panther using syslog-ng
Panther is an open-source log management system, which is also available as a service for a time-limited trial. It is still in beta phase, but it looks promising. You can see the “beta” sign on its opening page: https://app.panther.support/ I tested the time-limited cloud service version, but you can also install it locally, either from Dockerhub, or you can build the containers locally from the source.
Even if it is still in beta phase, Panther comes with detailed documentation.
Other
Syslog-ng 3.34: MQTT destination with TLS and WebSocket support
Version 3.33 of syslog-ng arrived with basic MQTT support. Version 3.34 has added many important features to it: user authentication, TLS support and WebSocket support. These features give you both security and flexibility while sending log messages to an MQTT broker.
This blog helps you to make your first steps securing your MQTT connection: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-34-mqtt-destination-with-tls-and-websocket-support
Other
The syslog-ng Insider 2021-09: 3.34; OpenBSD; OpenSearch; http() destination;
Dear syslog-ng users,
This is the 94th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. Topics include:
Version 3.34.1 of syslog-ng available Syslog-ng updated in OpenBSD ports OpenSearch and syslog-ng Creating a new http()-based syslog-ng destination: Seq It is available at: https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2021-09-3-34-openbsd-opensearch-http-destination
Other
The syslog-ng insider 2021-07: Alerting; CentOS alternatives; MongoDB;
Better late than never I just put online the July syslog-ng newsletter. Topics include:
Sending alerts to Discord and others from syslog-ng using Apprise: blocks and Python templates Rocky Linux, AlmaLinux, CentOS & syslog-ng MongoDB support improved in syslog-ng 3.32 It is available at https://www.syslog-ng.com/community/b/blog/posts/insider-2021-07-alerting-centos-alternatives-mongodb
Other
GSoC report: syslog-ng MacOS support
For the past couple of months, Yash Mathne has been working on testing syslog-ng on MacOS as a GSoC (Google Summer of Code) student. He worked both on x86 and on the freshly released ARM hardware. And we have some good news here to share: while there is still room for improvement, most of syslog-ng works perfectly well on MacOS.
Read my blog for some historical background and the GSoC report: https://www.
Other
Collecting process accounting logs on Linux with syslog-ng
Collecting process accounting logs on Linux with syslog-ng Process accounting logs are collected into binary log files on Linux. You can turn them into human readable format locally, using various tools. You can also use syslog-ng to read those files.
Lean how syslog-ng can parse those binary logs, create name-value pairs from them and store the results from my latest blog: https://www.syslog-ng.com/community/b/blog/posts/collecting-process-accounting-logs-on-linux-with-syslog-ng
Other
Elasticsearch 7.14 and Opensearch 1.0 Are Available and Work Fine With Syslog-ng
One of the most popular destinations in syslog-ng is Elasticsearch. Due to the license change of the Elastic stack, some people changed quickly to Grafana/Loki and other technologies. However, most syslog-ng users decided to wait and see. Version 1.0.0 of OpenSearch, a fork of the Elastic code base from before the license change is now available. Elastic also published a new release last week.
For this blog, I tested the latest and greatest from both product lines and I’m sharing my experiences.
Posts
Turris, syslog-ng and me
Yes, it’s a syslog-ng blog from me, and it’s not on https://syslog-ng.com/ :-) The reason is simple: this is not a technical blog. This is my story about how I found the Turris Omnia Linux router and how this lead to working together with the Turris guys.
The beginnings When I ordered my Turris Omnia, I did not know that it ran syslog-ng. All I knew that it was an ARM device and that it ran Linux.
Other
Using the udp-balancer() source of syslog-ng PE
Using the udp-balancer() source of syslog-ng PE UDP-based log collection is so last century. We had TCP-based log collection for decades and TLS encryption to secure connections. Still, UDP is in wide use, especially at large companies and industrial automation, where every change is slow. In most cases, UDP logging is used by networking devices, but sometimes it is just left there from ancient times and people are reluctant to change it.
Other
Syslog-ng 3.33: the MQTT destination
Syslog-ng 3.33: the MQTT destination Version 3.33 of syslog-ng introduced an MQTT destination. It uses the paho-c client library to send log messages to an MQTT broker. The current implementation supports version 3.1 and 3.1.1 of the protocol over non-encrypted connections, but this is only a first step.
From this blog, you can learn how to configure and test the mqtt() destination in syslog-ng.
Read my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-33-the-mqtt-destination