I built a self-driving perception system in India with no team, no funding, and no clean roads. Here is what actually happened.

want to tell you about the moment my system confidently labeled every single Indian driver as an aggressive maniac.
Not one driver. Every driver. 0 out of 100. AGGRESSIVE. On every test run.
The system was not wrong by its own logic. That was the problem.

I am Sherin. I run a computer vision company out of Kerala. About eight months ago I got tired of watching AV research happen exclusively in San Francisco and decided to build my own perception stack for Indian roads.
No investors. No research lab. Just me, a laptop mounted in a car, and roads that would give any self-driving engineer a quiet existential crisis.
The system is called Orvex. Four cameras running simultaneously. Real-time object detection, persistent tracking, per-object distance and time-to-collision, collision warnings, lane analysis, the works. Built from scratch. Runs locally at 15 to 17ms latency.
I drove it through Bangalore and Kerala traffic and watched it fall apart in the most instructive ways possible.

The behavior scorer failure taught me more than six months of training runs did.
I had built a driver safety scoring module. 0 to 100 scale. It evaluated hard braking, rapid acceleration, tight gap acceptance, lateral aggression.
Every Indian driver scored zero. Every single one.
Because the module was built on assumptions from Western road safety research where hard braking is abnormal and cutting gaps is dangerous. In Indian traffic, those things are just how you get anywhere. Threading a gap between an auto-rickshaw and a KSRTC bus at a signal is not aggression. It is Tuesday.
I had to disable the scorer completely and rethink what "safe driving" even means in this context. That realization alone was worth the entire project.

The scale of the problem only became visible when I actually instrumented it.
By mid-session in one test drive, my tracker had assigned over 1100 unique object IDs. In roughly 50 minutes of driving.
For context, nuScenes, the dataset most AV researchers train on, averages 30 to 40 annotated objects per scene. I was hitting 33 simultaneously active tracked objects in a parking lot.
Nobody benchmarks for this. The entire published literature on perception systems implicitly assumes a traffic density that simply does not exist in Indian cities. You do not discover that by reading papers. You discover it by putting a camera on a car and driving to the market.

The optical flow channel was an accident that became load-bearing.
I added a motion analysis feed almost as an afterthought. It ended up being one of the most useful parts of the system.
In dense traffic, your detection pipeline misses things. Objects partially behind a truck. A motorcycle moving too fast to clear the confidence threshold. The optical flow channel catches motion vectors for all of it, classified or not. It became a soft early warning layer I had not designed for but could not remove once I saw it working.
Some of the best features in this project came from watching the system fail and asking why.

Now the thing I have not announced yet.
I have been talking to a friend in Bangalore. Through his connections we have access to 200 electric vehicles in active fleet service across the city.
We are going to build a full autonomous vehicle system from scratch together.
Not buy a stack. Not fine-tune something off the shelf. From scratch. Our own data collection pipeline, our own annotation infrastructure, our own models trained on Indian roads. The fleet gives us what most AV startups spend years trying to get: a real operational environment with genuine scale across one of the most complex urban road networks on the planet.
The EV platform matters more than it sounds. Clean electrical architecture, drive-by-wire integration is a solved problem on modern EVs, no combustion complexity. The integration surface is predictable. That is not nothing when you are a small team.
Phase one is data infrastructure. Phase two is rebuilding Orvex properly with everything learned from the road tests. Phase three is the car.

What I have figured out about building hard technical products with no runway:
Ship something embarrassing onto real roads as fast as possible. The behavior scorer failure, the TTC model limitations, the tracker overload under density, I did not find any of that in simulation. I found it on a Tuesday afternoon in Bangalore traffic.
Simulation is useful. I use CARLA, I have built custom Indian road scenarios, synthetic data has real value. But simulation optimizes for the things you already know to model. The road shows you what you forgot to think about.
The benchmark numbers do not matter until they do. Orvex underperforms several open-source baselines on standard benchmarks. It also catches real collision threats on roads those benchmarks have never seen. At some point you have to decide what you are actually optimizing for.
Small teams building hard problems in ignored markets have a genuine advantage: nobody is competing with you on your specific problem. The downside is nobody has solved it for you either.

This is a long game. I know that.
But the fleet is real. The footage is real. The failures were real and fixable.
That is more than most AV projects built in a garage can say.

Happy to talk to anyone building perception systems, edge AI, or AV infrastructure outside the usual Western markets. The problems are genuinely interesting and the community working on them is smaller than it should be.