Build multiplayer like Figma
Adding multiplayer (collaborative editing) to software is hard. Like, real hard.
I know. I've spent half of the last 5 years getting it right.
Showing live cursors on others' screens is easy. It's getting the syncable data correct that's hard.
I've had to build this myself for my online/offline collaborative writing app, because there just isn't much out there beyond white papers and partial projects. And the algorithms are only a part of it. The server infrastructure is another can of worms. Mine is moving to the edge, close to the users.
I'm wondering if it would be valuable to builders to have a syncable document service they could build apps on (or add features to existing apps) that enabled realtime features.
Figma had it easy (comparatively). They don't do offline. They don't do collaborative rich text. My service can do everything Figma's does and those things, too. (Note: I'm talking about the syncing/realtime/multiplayer data, not the design tools. You'll have to build your own app on top, but it is a hard piece you don't have to worry about.)
I know, I know, this is backwards. You ought to find your ideal customer first, then create solutions to solve their biggest problems. But here I am with a solution looking for a problem. Thing is, I had that problem, and I would have loved to short-track the last 5 years and let someone else handle it for me.
There are lots of questions you'll have, if you have any interest. How would I trust you to care for my data? How much would it cost? How easy is it?
But first, are there problems needing a solution like this? What would you build with it? Would you trust someone else with it?
How did you solve it? Are you using websockets?
Very curious about the technical implementation and complications. Mostly because I've been using websockets for ages for quite an array of things and it always worked out great. But I can imagine synchronous mutations to something might be a pain.
Websockets gets you the realtime, but it doesn't solve the concurrent updates (or offline updates). Here's the steps I went through with Dabble.
When I first launched Dabble, each scene in the novel was a separate document stored to CouchDB which synced automatically. But if you wrote something offline, even a single word, on one computer, but then wrote 5k words on another computer online, when that offline computer came online the old version of the document would overwrite what you wrote online and you'd lose everything. Except the one word. I was able to restore much in support going back through the logs, but it was not fun for me or the customer.
Next idea, compare the timestamps and only overwrite if it is newer. This lost my ability to use the automatic syncing that was easy with couchdb, and it also only helped a little, since the offline writing may have been 5k words and the online newer writing only one word. Still an issue.
I knew if I was to ever have concurrent editing, this model wouldn't work anyway, so I started researching and learning about Operational Transformation (OT). There isn't a lot of open source libraries or concrete details about implementation (or wasn't 4 years ago), so I struggled through writing my own using JSON Patch. As you read more about OT, you'll learn there are edge cases and implementing it is hard.
Still I had it mostly working, and issues were few and far between. But as you gain more customers, those few issues turn into more issues, simply because of scale. Just like if you have 1 server, failure isn't something you worry about much, but if you have 10k servers, it is an inevitability.
I've improved things over time and modified the algorithm in such a way as to be 100% correct for JSON Patch operations (which is the standard I started with for the operations in OT). I've also implemented a field-level last-write-wins algorithm like Figma uses for data that doesn't need the full OT since LWW is more efficient. And I've got it architected for optimized deployment at the edge to make things as fast as possible. I'm super thrilled with the technology I've developed and see great value in it for future products, but I've still got my hands full with Dabble, so I'm curious if there may be opportunity to help others get to where I have without the work and to benefit from it in the process, win-win.
That sounds like quite the journey! Huge respect for your perseverance!
Thank you!