I've been monitoring my users for when they successfully have the app run on a PR with a little Heroku Dataclip (Like an SQL query, where you can pull up the results on my phone in the morning).
For a few users, I've added small comments on their PR introducing myself & thanking them for the install, then inviting them to mention me if they need any assistance.
It has worked really well! I'm getting really actionable feedback from people using the product & they're super lovely!