CocoIndex - Open source ETL to transform data for AI; with real-time incremental processing
🎉 We are thrilled to announce the open-source release of https://github.com/cocoindex-io/cocoindex 🥥 🌴 . It is an ETL framework to transform data for AI, with real-time incremental processing - keep index up to date with low latency on source update.
It is the world’s first open source compute engine that supports:
- incremental processing
- custom logic
- heavy transformations beyond SQL. E.g., LLM inference in the ETL, vector embeddings, knowledge graph / structured extraction.
Key features:
- Keep index fresh on source updates with low latency with live updates.
- Support custom logic like lego; with data flow programming model.
- Native built-ins to support use cases for vector embeddings, knowledge graphs, and other data transformations.
- Super performant RUST engine, with Python SDK
Get started: https://cocoindex.io/docs/getting_started/quickstart
