We just hosted our second user conference. About 50 people came to our hometown of St. Louis and spent 3 days learning pro tips for our product.
I know this isn't a representative sample of our customers but it was great getting face-to-face feedback. I learned a lot about what they need, and they all left with an even better opinion of our product/company. I also had a chance to preview a new design we're working on and it was a relief that people seemed to like it.
Having said all that, planning events (even a relatively small one like this) is hard and expensive. We ended up losing money on the whole thing and it took a ton of time. Based on this experience, I think the argument for doing a user conference for a business like ours is that it's inspiring/motivating for our employees. I don't think there's any way we ever make money off this even considering second-order effects.