Emerging Spaces: Machine Learning-based News Trend Detection Service

This is the new case study by SPD Group. Learn how ML can help a business to introduce new features and elevate the level of customer experience. In this case Machine Learning is used to introduce an innovative feature - news trend detection tool.

Overview
The client is a leading financial data provider that covers the global venture capital, private equity, and public markets. We remained its business partner for over 13 years, starting with a small team and growing into a primary full-cycle technology provider. One of our teams has already been crawling financial news for this company. Over time, the volume of information has increased from approximately 100 sources of news to 2,000-3,000 sources. It was impossible for the team to process such a large amount of information by relying only on human experts, so it was decided to automate this process. While working on a solution, it became clear that it has a clear business value and thus could be an additional microservice for our partner.

Business goal
Developing a new microservice with a visual interface on the website that could help clients to find trends to invest in. Broadening the partner’s functionality with News Trend Detection Service will help acquire new clients as well as increasing the trust of existing ones.

Challenge
The biggest challenge is definitely the volume of information — around 50,000 articles in a day should be processed; that’s 18 million a year! The right set of Machine Learning algorithms must be used and adjusted to handle this volume properly. From a technical perspective, the team needed more powerful servers to analyze such a gigantic amount of information, as Machine Learning algorithms require a lot of processing power.

Read the full case study here: https://spd.group/cases/ml-based-news-trend-detection-service/