Ask IH: How do you manage all the logs from your SaaS product or service?

I've always used Papertrail.

I use the ELK stack as well. Make sure you've got the ability to add space quickly as those logs sure grow quickly.

This is something my company often sets up for clients as well as for our own products such as https://kwiki.io - and we usually do it as @pulkownik said, using ELK stack, namely Kibana.

You are not limited by language or technology stack. Your process would be something like:

1. make sure your app(s) write out logs in a consistent format
2. set up ELK
3. set up filebeat or similar app to ingest the logs and send them to ELK
4. set up Kibana dashboard together with any (grok) filters, analysis etc. Or you can just leave the logs as-is

I recommend this way, you will appreciate it, especially later when your app/platform grows. Using elk and stuff like tagging requests using fingerprinting you can get very nice tracebacks as requests flow through your multi-layered microserviced system, easily detect issues, debug etc.

Finally, definitely don't keep them in a database, that's not what database is for ;)

Elastic stack (formerly known as ELK) is the standard for all logs- but of course you'd have to stand it up and mange it yourself but atleast its oss. There are many managed solutions in the space such as loggly, scalyr, etc.

What do you mean by logs? Application logs? I have used ELK (Elasticsearch Logstah Kibana) stack and it works pretty well. You can customize it as you want.

This comment was deleted 8 years ago