HPC and me
Recently I found that quite a few of my Twitter and Mastodon followers are working in high-performance computing (HPC). At first I was surprised because I’m not a HPC person, even if I love high performance computers. Then I realized that there are quite few overlaps, and one of my best friends is also deeply involved in HPC. My work, logging, is also a fundamental part of HPC environments.
Let’s start with a direct connection to HPC: one of my best friends, Gabor Samu, is working in HPC. He is one of the product managers for one of the leading commercial HPC workload managers: IBM Spectrum LSF Suites. I often interact with his posts both on Twitter and Mastodon.
I love high performance computers and non-x86 architectures. Of course, high performance computers aren’t the exclusive domain of HPC today. Just think of web and database servers, CAD and video editing workstations, AI, and so on. But there is definitely an overlap. Some of the fastest HPC systems are built around non-x86 architectures. You can find many of those on the top500 list. ARM and POWER systems made it even into the top10 list, and occupied the #1 position for years.
The European Union is developing its own CPUs for HPC as part of the European Processor Initiative. Their target is a low power consumption but high performance system. They are working on ARM and RISC-V systems.
It’s not just HPC where non-x86 architectures can show significant performance benefits. For many years, IBM’s POWER9 CPU was the fastest to run syslog-ng. Running syslog-ng on X86 was not even half as fast. Currently my fastest measurement was on an AMD system, but I would not be surprised if I could measure higher syslog-ng message rates on POWER10 or Ampere systems (if I had access).
Logging is a fundamental part of HPC environments. With the scale of HPC systems, and the price of down time, logging is required for being able to isolate / identity issues rapidly. These systems run around the clock and are used to solve grand scale challenges for humanity like vaccine development, weather modeling, or analyzing data from the LHC. Logging gives you here better visibility into the system and makes you able to identify problems very quickly.
One of the most active syslog-ng users, Faxm0dem, is running thousands of syslog-ng instances in France, at CC-IN2P3. If you take a look at the powered by syslog-ng page, you will see that they are not the only ones. I learned at various conferences that there are many more HPC labs where syslog-ng is in use, but unfortunately they are not sharing infrastructure details publicly, only in private discussions.
TL;DR: I’m not surprised any more, if I see new HPC focused followers :-)
