Even though we don't have a ton of traffic yet, I have done some performance optimizations so that we can handle the first loads of traffic reasonably. The first implementation was so rough and crude that it was slower than molasses.
Now we take advantage of caching important information so that we're not refetching logos unnecessarily, and when users specify a size we only go and resize it once. The first time it can take about 500ms-1000ms (depending on how long it takes to process the original and convert to the new size), but subsequent requests are only 30-50ms.