How does Mailgun send emails?

Agreed with the others, setting up an SMTP server should probably be avoided unless you're scaling an email-specific business. One thing you could do if you're building an MVP is use Amazon's SES which is essentially just a thin API on top of an SMTP server, and is much cheaper than the commercial Email Service Providers. https://aws.amazon.com/ses/

With your own email servers, you will spend a lot of time worrying about reliability and deliverability. These two things are the primary value adds that ESPs provide, in addition to better UIs and APIs. In short, deliverability relies on reputation of not only your domain name, but also the physical server from which it was sent. You can read more about that here: https://litmus.com/community/learning/19-deliverability-basics

E-mails are sent through the SMTP protocol, so you need a SMTP server in order to send emails. The most popular one is Postfix, but there are several others. Mailgun, for example, uses Haraka as their SMTP server, Haraka is built on NodeJS and focus on scalability.

As mentioned by others, don't implement your own SMTP unless you have good experience running mail servers. While php's mail() function is easy for example, getting that email past the spam filters and in to the inbox is another story - one best managed by paying a company to deal with those problems for a few pennies per thousand emails.

I've used Mailgun before and it worked great.

SMTP
https://sendgrid.com/docs/API_Reference/SMTP_API/getting_started_smtp.html