How I made a bit of code run 3000x faster...
A lesson I learned this month. Sometimes it is far better to eschew the fancy tools and abstraction layers around you and get 'down and dirty' with raw code at the base layer to get the performance you need.
Full details on this post on my personal blog.
Trending on Indie Hackers
Apologies for the broken link everyone - I mustn't have got the Markdown format correct, but here is the raw link to the post:
https://devan.codes/blog/2020/9/21/making-my-code-run-3000-times-faster
Great read. I resonate with tech like Ruby ORM, I use elixir's Ecto personally.
Did you write this scheduled event directly into the MySQL console (as it reads)? Or do you put it somewhere in your migrations checked into version control?
Thanks Jon - I'd really like to get some time to play with Phoenix/Elixir sometime! I wrote the schedule directly in the MySQL console, and documented on our DevOps Wiki for the team to know to include it as part of their routine maintenance checks. Good idea to get it into our repo though, so I might upload the SQL script in there for safekeeping too.
Don't forget you can also just do a single
select(:file_size)(at least in ActiveRecord) so you're not getting full objects. Or if you don't need objects at all you could usepluck.But in general, caching is always going to be fastest (I also like dropping directly down to the DB to do it). You could also create a materialized view to do this, if you didn't want to change your model to accommodate
upload_usage.Great tips David - Thanks! 👍
@DevanSabaratnam thanks for sharing your experience. It's a good example when to rely on abstractions such ORM or go down and just use a native query.
Did you eventually stick MySQL jobs or moved it into a codebase and triggered from a code ?
P.S. The link doesn't seem to work, but I found the post via your profile.
Thanks Artur! We just left it as a scheduled job on AWS RDS as it seemed easier than trying to trigger via our background worker server. That server has enough on its plate already... 😅
And thanks for the heads up about the broken link. I can't seem to edit my post any longer but will post a comment with the correct link.