Change synchronisation model to event-sourcing

[ftsell]

Event-Sourcing is an architecture to synchronize distributed systems based on past actions.
The Vuex store is already internally event-sourced since the state is built by folding over all commits. If we could utilize the commit-log and simply send every commit over the wire to other clients the store state would automatically be synchronized.

To implement this I propose a protocol which will still be transported over socket.io.
It would essentially need three actions:

• commit: whenever one of the peers in the system commits something into the store, this commit should be transferred over the wire as well.
• request_log: when a new peer joins the group, that peer needs to request old commits.
• log: this type of message should include the complete commit-log so that a new peer can build its state.
  If possible this should be implemented as a Vuex store module so that not the complete store is synchronized but only a portion of it.

In this new module the backend would not need to save old room state but instead only serve as a bridge between peers in a room.

[jaagut]

Thanks for your idea of using event-sourcing. Generally, I like the concept and I think we already discussed this during the design phase of the backend. But we saw a few drawbacks in our case, since synchronization-issues are still possible:

Synchronization-issues could occur, when two clients are committing (sending) a change-event (before receiving the other event).
Those two clients can be in a different state, when first applying their own change, then the later received one. All other users will presumably receive the events in the same order as the server (unless routing is difficult).
This means, besides using event-sourcing, synchronization-issues are still possible and clients could be in different states. To solve this problem some sort of ground-truth is needed, such that all clients are guaranteed to be in the same state.

Our solution for the backend is, by just repeating/propagating all received states to all clients in the same order as received by the server. The client that 'creates' the changed state has to ignore its own message when receiving it.
(As I write this, I notice a sync-problem that is currently possible, when a client does not receive states in the same order as the server is sending them... Maybe we should add a counter.)

So, I don't see an immediate advantage over the current implementation and switching to eventsourcing means huge changes in the backend as well as in the frontend. I think before doing this, we should resolve the other known issues first.

[ftsell]

The problem you describe with out-of-order transmissions is exactly one of the problems event-sourcing would solve though. The whole point is to not transmit the complete updated state in one event but rather small incremental changes. So if one client changes the playback speed and another clicks pause, those two different events can simply both be applied (no matter the order).
Since the possibility still exists that two clients click pause at the same time, we would still have to write code to resolve these conflicts. And if they cannot be resolved the most-used strategy I found was to try and resubmit an event.

I agree though that it is a lot of work and might bring with it new problems. I'll see if I can implement a proof-of-concept to test whether it actually works better.

[jaagut]

In case of the backend, I think, we can implement and test this step by step by supporting both variants for some time...

[Flova]

I think event sourcing is cool, but it is not that relevant for our use case. All clients end up in the same state at the end, because of the server-side time signatures. We do not need the whole history at this point. Ignoring states that are older than the current state in the frontend is enough for our use case. There are far more open issues in the front end that have a bigger impact on the user experience (e.g. the communication issues with video.js). These issues create real problems and I don't think we would have a benefit while solving this issue here. Especially in contrast to the afford required to restructure the whole backend.
To solve unlikely the issue of two independent values being changed (playback rate and pause like in your example) we could easily apply the current principle to two different API points (one for payback speed and one for time/play etc.). I could add an independent state for this if it is needed, but I think it adds complexity for a quite unlikely scenario, that is not that fatal or unexpected by the user.

Nevertheless, I am fine with it if somebody wants to add the event sourcing and all features are implemented/tested to the current level.