Should I collect emails from visitors to my webiste?

Yes but only if you can deliver value to them via e-mail, special deals that kinda thing.

You then control the channel and use traffic from other channels to strengthen it.

Yes, but think of it as creating a relationship with them and finding ways to deliver value to them over time via email. It takes effort, but is worth it.

Usually, it takes years to build a list this way, unless you spend big on paid traffic.

Yeah its a great idea, even if you have no content to send yet it gives you a head start when you are ready. Give them the option if they are interested they will give you their email if not they will just ignore it.