Understand any codebase with AI
Hey guys, I've been diving deeper into open source recently and have come across larger codebases that truly challenged my understanding. This led me to create Semantic (https://sem.sh/) -- a tool that assists me in comprehending these complex systems.
Let me know if this helps you guys too!
Hey, as an AI engineer, I find this interesting! I guess what you want to do/are doing is a vector database on top of OpenAI embeddings and ChatGPT on top? 🗂️🔝 That would be the usual very simple approach. I've also heard that AST-based embeddings have been discussed and used in the literature. 📚🌳
I think the value that you can generate is mostly determined by the size of the context window that the model can process. 🪟 In general, I think programmers are a very hard audience to please with a very low willingness to pay and a tendency to go with open source projects. ⌨️🧑💻 This is because of security concerns but also because they like to stay in control. 🔒 Just some general concerns. Feel free to connect, I know some people that worked on the same idea but ultimately needed to pivot. 🤝🌐
Hi! You are exactly correct the way it works right now is just a vector database with embeddings talking to chatgpt. The embeddings I am using now are trained on AST.
The reason I built this was because other same ideas that are on the market right now couldn't process extremely large codebases without crashing. This is a very helpful comment, I would love to hear more about the people who pivoted and your experience with programmers as an audience! Looking forward to chatting more :)
https://www.linkedin.com/in/nathanlu/
How deep does it go? Are you talking AST or just plugging code snippets into chatGPT.
Right now it is just finding relevant documents, summarizing it, and sending it to chatGPT. The response isn't the most accurate though so I am definitely planning to do something with AST and callgraphs
Can you explain a bit deeper what this is doing exactly?
Yes for sure! It lets you link a public Github repository and you can inquire about how things work in the codebase. For example I was going through a distributed database opensource and had the tool explain how raft consensus was implemented