https://f0o.dev/tags/elasticsearch/ Recent content in Elasticsearch on Daniel 'f0o' Preussker Hugo -- gohugo.io en-us Fri, 03 Dec 2021 00:00:00 +0000 https://f0o.dev/posts/2021/12/logging-insanity/ Fri, 03 Dec 2021 00:00:00 +0000 https://f0o.dev/posts/2021/12/logging-insanity/ <h3 id="how-complex-can-you-make-log-forwarding">How complex can you make log forwarding?</h3> <p>You&rsquo;d imagine not very much, after all - what&rsquo;s there to complicate?</p> <p>You get the logs, move them to a processor and then just write them into the persistence - right?</p> <p>You&rsquo;d be surprised how complex you can make that&hellip;</p> <p>This is a <strong>simplified</strong> view of what I discovered in the wild.</p> <pre tabindex="0"><code> +---------------+ +-----------------+ | Application | ---- Stdout ----&gt; | Docker Engine | +---------------+ +-----------------+ | \./ +---------------+ +-----------------+ | Logspout | &lt;--- Listens ---&gt; | All stdout/s | +---------------+ +-----------------+ | \./ +---------------+ +-----------------+ | Fluentbit | --- Forward ----&gt; | HAProxy | +---------------+ +-----------------+ | \./ +---------------+ +-----------------+ | ElasticSearch | &lt;---- Writes ---- | Fluentd | +---------------+ +-----------------+ </code></pre><p>I absolutely fell from my chair when I saw this. What in everything that&rsquo;s holy&hellip;.</p>