Collaborative editing?

Feature Concepts
1

Hi,

I am thinking about a feature at the moment and would like to understand if you someone would be interested. I am thinking about collaborative editing for the content editor like in Google Docs.

This will also effect the current content editors like rich text and markdown. Therefore your input is really needed here.

I am trying to understand this topic in general and found a very great library called: yjs (https://docs.yjs.dev/). The basic idea is to have data structures like maps, arrays and so on that are then synced over the wire to other users. As a developer it is very easy to implement, because you don’t need to understand the details and the library manages the complex details. It also works over web rtc as peer 2 peer so there is no server involved or any extra load on the system.

We could use this approach for the content editor and allow synchronization over all content fields. The behavior would depends on the editor and each input would be handled as a single value. So by default it 2 users change the same input field only one change would be accepted.

Fortunately there are also integrations for rich text editors such as slate: https://github.com/BitPhinix/slate-yjs/. This would allow a better user experience for editors that are built into Squidex and built on top of slate. This would mean to replace the rich text and markdown editor with a slate implementation (see https://www.slatejs.org/examples/markdown-preview for markdown).

We can also leverage yjs to reimplement to show the current users that are on a page and to improve the chat system to reduce server load.

3 Likes
2

UPDATE

The peer to peer system does not work as good as intended, so there is a server needed.

Currently we have the following options:

  1. Hocuspocus: Node based server, available as cloud version or as a self hosted. We can build a docker container for that: See: https://tiptap.dev/hocuspocus/getting-started. Most popular solution on Github with good enough performance. This requires an extra service or cloud setup.

  2. y-sweet: Rust based alternative with high performance, not as popular yet and the performance is not really needed for our use case. Therefore hocuspocus might be better. This requires an extra service or cloud setup.

  3. SignalR: .NET alternative, that is still in development. Definitely nothing we can start with, but we can port to SignalR eventually at a later point of time. https://github.com/yjs/ycs/issues/5

Disadvantages

The big advantage with this yjs tech, is how easy it is to build real-time features with that. We already have 3 systems that would be replaced with that:

  1. Notifications (You can get notifications in Squidex)
  2. Comments (You can add comments for a content item)
  3. Editing Users (You can see who is currently online)

For ease of deployment, these systems work with polling, which also flood the logs.

It does not make sense to keep these in the system as is, because the comments cannot be evolved (e.g. when you want to comment on a particular field). It is also a burden to keep maintaining them and have 2 systems in place.

So the only good decision would be to replace everything with yjs. This means:

  • If the SignalR implementation is not ready and you have no hocuspocus server running (docker image will be provided), these features will be not available anymore.
  • If the SignalR implementation is ready and you have more than 1 server, you probably need redis. it is needed to send an update from UserA who is connected to Server 1 to UserB who is connected to Server 2.
3

UPDATE 2

If we already move notifications and comments we would also loose the ability to get email notifications, which actually sucks :frowning:

4

UPDATE 3

I am contributing to a yjs server for .NET now, which solves all the problems with notifications. The base work is done here. I have to dig into the editor now. It will probably be prosemirror (https://prosemirror.net/). The current markdown editor is also based on prosemirror but not updated since a while. Therefore we will probably move to our editor. This will take while.

So the current goal is:

  1. Provide an improved collaboration feature for the UI that also shows cursors and focused element. Deploy that and release a new version.
  2. Build a new editor base on Prosemirror for rich text and html editing. Provide a fallback with editor extensions if you want to have the current experience back. The work for the markdown editor is already done: https://github.com/Squidex/squidex-samples/tree/master/editors/markdown.
  3. Integrate the editors and the content form with the collaboration system.

Breaking Changes that will occur

  1. Some endpoints for comments and notifications will be removed. For example the ability to create new comments from the API. Rules are not affected by that.
  2. The markdown editor will be replaced with a new unified editor. The previous editor will available as editor extension.
  3. The richtext editor will be replaced with a new unified editor. The previous editor will available as editor extension.
  4. You will very likely need some kind of backplane for the collaboration system, for example redis, if you have more than one Squidex instance. This is needed to share changes between multiple servers via pubsub. It would be possible to implement that with MongoDB but the load on the database could be relatively high.
5
6