(Replying to PARENT post)

Yeah, fluentd... I had to look into its in_stdin plugin, and I can tell you one thing: it simply doesn't handle regexes properly. In fact, none of its input plugins can correctly handle the situation where a multiline regex doesn't match the whole line, unless they do what in_tail did (split the lines yourself and disallow the multiline regexes).

Edit: Sorry, I meant "Fluent bit". No idea how fluentd handle this scenario, but I was told it was too slow (being written in Ruby) so that's why the switch to fluentbit was made.

👤Joker_vD🕑5y🔼0🗨️0

(Replying to PARENT post)

Last time I tried, fluent-bit did not support IPv6. Maybe it does now? (Maybe it does not: https://github.com/fluent/fluent-bit/issues/2056 has been opened recently). Anyway, compared to the solution in the article, it seemed way too fanciful.

Fluentd would not just be slow, but also run out of memory. I am no ruby-head, but a then-colleague of mine helped configure it. It still ran out of memory. I had no patience for a log system that did not work out of the box on one computer basically just logging failed ssh login attempts.

👤taneliv🕑5y🔼0🗨️0

(Replying to PARENT post)

I've done multiline processing with logstash. It works but it's not necessarily efficient or simple. Depends a lot on the pattern and whether the multiline stack traces you want to extract turns out to be 700 lines long sometimes (yes, speaking from experience).

I'd personally recommend to not bother with processing multiline output into a single message. Lots of trouble for no benefits. It's just a stream of lines at the end of the day, it will look the same in tail and kibana.

👤user5994461🕑5y🔼0🗨️0