Side Project on Kubernetes Log Management

Hi Hackers!

I have been working on Kubernetes systems for a while now and if you know it, due to the nature of kubernetes clusters, you have to spend a lot of money on the observability front. And especially for storing and searching the logs.

We've been using Datadog -switched from Splunk- at where I work with the aim of cost reduction but still the cost is very hefty.

So, I said to myself, what if there would be something that'd be very simple and cheap and comes with much less overhead and maintenance then Grafana or ELK stack. My idea was to get rid of the logging cost which is the biggest one and to still use Datadog's metric and traces in tandem to it.

And, I built AegisCL - https://www.aegiscl.com/

So, what it does is that, just under a couple of seconds, builds an S3-Athena pair in your AWS environment and start archiving your kubernetes logs while giving you Datadog like visibility into it. Cost-wise, it is extremely(extremely) cheap -also I offer it for free- and very reliable.

I will be pitching it to my manager so that maybe we can use it in my company too.

As an humble hacker like you, I'm seeking feedback!