Yesterday, I decided to run a study, to find whether Ai-generated titles are better than the actual titles of 10 random published blog posts. Since I couldn’t change the titles of articles I didn’t own, to see how they perform, I decided to generate titles for the same article, and ask SEO experts about it.
I gathered a list of articles that ranked high for a keyword I search for in a random niche, and then I generated a title using my software Enwrite, to see how they compare.
Currently, I only have some preliminary results, but they are very interesting:
2 people preferred the AI-generated title for 7/10 titles
1 person preferred the AI-generated title for 6/10 titles
1 person preferred the AI-generated title for 5/10 titles
1 person preferred the AI-generated title for 3/10 titles
Average score: 6/10
But, what’s more interesting are the response breakdown:
Question 1: 3/5 people preferred the AI-generated response
Question 2: 3/5 people preferred the AI-generated response
Question 3: 0/5 people preferred the AI-generated response
Question 4: 1/5 people preferred the AI-generated response
Question 5: 2/5 people preferred the AI-generated response
Question 6: 5/5 people preferred the AI-generated response
Question 7: 4/5 people preferred the AI-generated response
Question 8: 4/5 people preferred the AI-generated response
Question 9: 2/5 people preferred the AI-generated response
Question 10: 4/5 people preferred the AI-generated response
It is important to note that questions 3 and 4 were subjectively bad generations, and if I was an actual blogger, I would have never used them, but I had to include them in the test for objectivity reasons.
I will be releasing results as more entries come in. Stay tuned!
We are still looking for participants in the study who work in SEO - so if you are interested, please visit https://www.enwrite.co/study.
For comparisons like these, it would be good to include some characteristics of your SEO experts, like:
That way you can understand your studies a bit better. Especially in later stages :)
To be honest, at the moment, I'm trying to make this as low friction as possible, in order to get as many responses and have evidence that my product actually works.
It would be interesting to know this information though, but definitely at a later stage!