3
8 Comments

How would you schedule a task in your node application?

Scheduling task in your node applications (e.g. sending emails at a scheduled time) - how would you do it? (1) Use node-cron but risk failure since it'll just be 1 server to manage cron jobs (2) AWS Lambda/Cloudwatch (3) Redis cron-tab?

on June 17, 2020
  1. 2

    I am interested in the answer too:-) I have many scheduled emails to send too.

    I am currently using php scripts, mysql, and cronjob for many years on the same server as the website. It has never been a problem, except in a few days a year with very heavy volume, then it takes a while to send out all the emails.

    1. 1

      Do you spend a lot on server cost, for it to run 24/7?

      1. 1

        It depends on your traffic. I am just using regular VPS hosting, ranging from $60 to $120 per month. But I will be moving to DigitalOcean or something similar after the summer for ease of scaling, I think. All hosting setup run 24/7.

  2. 1

    I would go with Redis and a queue (Kue or Bull). You can use something like PM2 to spin up a separate process to handle sending, that way should something go wrong, it won't crash your server. You can obviously use a lambda for that last part. Definitely use a queue with Redis,either way, you'll be able to throttle the calls, handle failures... etc

  3. 1

    For this kind of thing I love the aws lambda solution. At my last job I wrote a terraform module to spin up a lambda function, cloudwatch event, etc.

    We used it everywhere. I don't have that code anymore but going to have to write something similar again soon.

    1. 1

      Thanks! I was trying to weigh the different alternatives since if I go with AWS Lambda I'll have to configure a CloudWatch rule that watches out for AWS API calls via CloudTrail... that seems quite hefty for V1.0 of my app so I'll probably shelve it for now.

  4. 1

    I recently migrated to Hashicorp Nomad for a lightweight alternative to Kubernetes. So far I am loving it.

    One of the features of my site is displaying charts which have some amount of processing that is required in order to generate the data. Some of these run every hour and some once per day. It's all in Node.js and I schedule them using a periodic stanza https://www.nomadproject.io/docs/job-specification/periodic/

    The way I have set it up is that the stanza executes a "task runner" which then sends a load of HTTP requests which are load balanced between multiple instances of the same service. This could just as easily push to a queue but I'm keeping it simple for now. The periodic stanza could directly trigger the task that does the work instead of my task runner approach but as it is quite a lot of different charts that could lead to a lot of containers being brought up and I want to keep costs low - this is all currently running on a single Lightsail instance.

    It's pretty cool because the task runner itself could be allocated to any client (i.e. "distributed cron")

    This setup is potentially overkill - it depends if you think you will need to scale up in the medium/long term. My site is on the back of a product currently in beta so traffic is currently low but will hopefully shoot up quite a bit on year end when they go live. I'm using the time now to prepare for that.

    1. 2

      Haven't heard of this alternative, so thanks for bringing it up! Let me know how it fares :)