What we learned analyzing 2000+ of our project management tickets

Project management is a chore, am I right?

No, don't agree with me! I'm wrong haha

Project management is one of the best methods to collaborate in an accountable way. It's what most startups with multiple early employees and resources use to build things from the ground up. It's also a treasure trove of unstructured data to gain insights from.

For many fledgling startups, the traction of work deployed across product, sales, marketing, development, finance and management teams can determine whether your business survives long enough to publish an MVP, get your first billable customers or find funding from VC's/investors.

Our company Relevance AI is a vector-based data experimentation platform for data scientists & data teams.

TL;DR -

• You think you may know how your project management functions work by taking a look at tickets, until you get a bird's eye view of your entire tickets.
• Content & Documentation were indeed some of longest to complete tickets.
• Our latest Clustering product produced the greatest quantity of tickets.
• Our fastest tickets to complete were all Product related.

Hypothesis

We wanted to achieve two outcomes with this experiment.

1 - Find out what types of tasks take the shortest & longest amount of time to complete, then take these insights and flag them for our Ops team to action
2 - Use it as an excuse to take our Clustering platform for a test drive.

Initially, we predicted that tasks around documentation as well as content creation would take the longest amount of time given the need for deep research, whilst tech support tickets would take the shortest amount of time.

##Process

Our initial dataset was compiled from over 2000+ tickets taken from our chosen project management system, Linear.

These were then exported into an CSV file and subsequently uploaded as a dataset in the Relevance AI platform, then vectorised & clustered.

We built out two apps to work within this experiment:

1. An app that utilizes vector embeddings from OpenAI.
2. An that utilizes vector embeddings from SentenceTransformers.

In order, the next steps performed were:

1. Ingesting the exported csv from our project management tool
2. Vectorizing the ticket titles
3. Clustering on the vectors created from the ticket titles, ranging from 30-120 number of clusters
4. Validating those cluster quantitatively using our Cluster Report card which looks at range of statistical scores such as Silhouette score, Dunn index, etc.
5. Validating those clusters qualitatively by viewing the distance between words using both the cluster app and our 3d projector tool.

Results

-The highest quantity of tickets were related to the development of our Clustering product.

• 76 tickets total with an average completion time of 11.7 hours per ticket.

• Out of 76 tickets total, only 38 tickets were completed in that month.

• Our fastest tickets were completed in 5.3 hours (note our team often works on tickets simultaneously). The majority of those tickets belonged to the Product team.

• Our slowest tickets were related to writing vector-based content, at an average of 13.4 hours.
• Out of 22 tickets total, only 14 have been completed in that month.

• If we filter the data within our dashboard by the “Backend” team, the most completed tickets were related to “Chunk vector search”

Takeaways

Largely the bulk of our work went towards developing our Clustering product, given it was our primary product in development. We assume that in most SaaS businesses, this will ideally be the case unless you have a development bottleneck or have a low iteration, non-complex product to offer

Our hypothesis regarding content being generated was correct. As our product is geared towards data scientists and uses complex, highly technical concepts, as well as heavy use of coding, it makes sense that our content needs a lot of time.

Questions

What I'd love to ask IndieHackers are:

1. What areas within your project management tasks are blackholes where traction occurs in the longest timecycles per task and why?

2. What areas within your project management tasks are the most efficient and why?

3. Have you made any changes recently to improve production & turnover of tasks in your business where bottlenecks occurred?

If you'd like to learn more about data science, analysis or our platform in general, please connect with us via:
LinkedIn
Twitter
Slack