Deciding the frontend framework

[pavish]

This discussion is for analysing and choosing the frontend framework/libraries for Mathesar. The priority is on choosing the approach that best suits our roadmap and feature list.

We are looking for framework/libraries for rendering our application code, and are not going to be analyzing libraries for charting, custom selectors etc., in this discussion thread.

Our use case includes developing the app itself, custom ui components for spreadsheet like views, calendar, kanban views, custom date pickers etc.,

We are looking for frameworks/libraries/approaches that satisfy the following:

1. Highly responsive.
2. Seamless user interactions.
3. Fast initial load time.
4. Does not require it's own server. Eg., NextJS, Sapper etc., Needs to be integrated with our current backend which is in Python running Django webserver.
5. Scalable & Maintable.
6. Good community support.
7. Production ready.
8. Will not be shunned-upon/disappear over the course of the next 5 years.

We are also looking for good testing support, but we would be probably be using testing tools like cypress, which are development agnostic and can apply to any approach. This is yet to be discussed.

We are not going to be discussing about tooling in this thread, since it's dependent upon the framework/library of choice.

The initial analysis covers libraries based on popularity, recommendations, HN threads & internal considerations. Some of them are server rendered, and some are client rendered. Chances are that we may require combining two frameworks to suit our needs, like it's done by most modern applications.

We will not be going into detail about each framework in the thread. We will add links to their respective sites and references. The following contains only the pros and cons of each of the above mentioned framework/libraries.

The points mentioned below are my own views and this discussion thread is to help obtain different set of views and opinions from the rest of my team and the developer community in general.

Table of contents

• React
• Svelte
• Vue
• Django templates
• Hotwire: Turbo + Stimulus
• Sockpuppet
• HTMX
• Django Unicorn

• My personal opinion

---

React

Website: https://reactjs.org

Pros

• Client rendered, better responsiveness.
• Scalable and maintable.
• Faster development time.
• Reliable, has a good community and user base.
• Stable component libraries readily available.

Cons

• Uses a virtual DOM. Virtual DOMs were necessary when React first came out and it was what made react stand apart, but at the current state of browser evolution, virtual DOMs are unnecessary and more of a bottle-neck. Being just fast enough is not okay anymore, when browsers are so much more powerful.
• Initial loading time depends upon app size.
• First actual data rendering will require a separate request to the server. A network roundtrip we'd like to avoid.

Svelte

Website: https://svelte.dev

Pros

• Client rendered, better responsiveness.
• No virutal DOM.
• Scalable and maintainable.
• Has support for animation, transistion directives and stores as part of the library itself.
• Component first approach, compiles to JS, can be adapted and integrated with any server framework.

Cons

• No readily available stable component libraries. There are good few like Carbon, but not enough options.
• Initial load time depends on app size, although since it's a component library that combines to JS, we can reduce this by integrating with server rendered frameworks. Takes additional effort to counter this drawback.
• First actual data rendering will require a separate request to the server. A network roundtrip we'd like to avoid.

Vue

Website: https://vuejs.org

Pros

• Client rendered, better responsiveness.
• Good community support.
• Component first approach, can be integrated with any server rendered framework.

Cons

• Uses a virtual DOM.
• Initial loading size depends on app size, but can be overcome by separating it out into components and integrating with server rendered frameworks. Takes additional effort.
• Component templates are not declarative enough as JSX or Svelte. Will cause scalability and maintainability issues when application grows.
• First actual data rendering will require a separate request to the server. A network roundtrip we'd like to avoid.

Django templates

Website: https://docs.djangoproject.com/en/3.2/topics/templates/

Pros

• Server rendered, faster initial data load.
• Suits our backend without the need to do anything additional.

Cons

• Will still require a client side library for components and to handle responsiveness.
• Moving to a different page means a reload, which we would like to strictly avoid.

Hotwire: Turbo + Stimulus

Website: https://hotwire.dev

Pros

• Server rendered, faster initial data load.
• Turbo's State and logic is maintained on server, rather than both on server and client, which simplifies routing and forms.
• Stimulus can be used for higher responsiveness on client, offering the advantage of both server rendered and client rendered libraries.

Cons

• No stable integration with Django. Available option: turbo-django is not production ready.
• Very small community. Unreliable to start using right away, may need a couple years to grow and mature.
• Will still require an additional client library for ui components. Stimulus is not extensible enough to build complex components like spreadsheets. Even if built, adds a lot more complexity than there needs to be.

Sockpuppet

Website: https://sockpuppet.argpar.se

Pros

• Server rendered, faster initial data load.
• HTML over websockets, faster data load in general.
• Integrates well with Django. Uses stimulus for client interactivity.

Cons

• Very small community. Promising but unreliable to start using right away, would require time to grow and mature.
• Everything is over websockets, may cause issues at scale since websockets need open connections from the client to the server. We might need websockets but we would like to control when and why to use them, certainly not for page rendering as a whole, without tested and tried products out there.
• Would still require a client library for ui components.

HTMX

Website: https://htmx.org

Pros

• Simplifies ajax, transistions and websockets as part of html directive, amplifying hypertext.
• Integrates well with server rendered frameworks.

Cons

• Not a library that can actually be used for the UI of the product as a whole. Can be used to simplify developing with Django templates.
• Will still require an additional client library for ui components. The need for htmx disappears when we do have a client library.

Django Unicorn

Website: https://www.django-unicorn.com

Pros

• Built to work with Django.
• Server rendered, faster initial load time.
• Component first approach.
• Automatic ajax calls for client interactivity.

Cons

• Very small community. Unreliable to start working with right away.
• Better alternative to plain django templates, but does not replace UI libraries.
• Will still require a client library for ui components.

---

My personal opinion

Svelte + Django templates

Based on the above analysis and considering that we need to be building custom UI components for most of the usecases of Mathesar, it is imperative that we will need a client library.

The idea is to use Django templates to serve static files for faster initial load and then take over the control from the client after page load.

How?

• Django will serve the initial html file. Based on the url requested, it will render the respective data as json, which can be accessed by Svelte as default props on page load.
• Static layouts and loading states can be rendered by django directly.
• The client will take over the routing after it loads. Since the initial data for rendering content is already present, no additional network call needed.
• Future requests on url change are made to the API endpoints which return json.
• This provides the combined benefits of both client and server rendered frameworks.
• Improves intial load speed and client interactivity and responsiveness.
• Since we will always need a client library for ui components, using the same for the application itself makes the most sense.

Isn't this what the HTML over wire approaches do?

• Yes, the concept is pretty similar. Serve html, take over from the client after page loads.
• The stark difference is that data is handled as json instead of html. Static layouts can be rendered as html, but data for tables, spreadsheets etc., is json.
• Rendering data as json on the server reduces the time taken to serve the file, since the size will be much smaller compared to rendering and serving it as html.
• Future page transitions will only make API calls. No more HTML expect first page load.
• Hyderating the DOM will not be required on the client, since the state can be directly inferred from the json.

Why Svelte?

• Svelte compiles to JS and can be initialized as a standalone component, offering more extensibility and control over the application architecture.
• Can be easily integrated with Django, as mentioned above.
• It does not use a virtual DOM and tops React and Vue in terms of performance on several aspects. Some references: 1, 2, 3, 4
• Has a good community and is steadily gaining popularity. Currently the top rated frontend library in State Of JS 2021 report.

What about lack of stable component libraries for Svelte?

• Most of the components needed for Mathesar are going to have to be custom built, based on our usecases.
• Since svelte is just a library, we can also include and work with vanilla js components with ease, if required.
• I have built a fairly complex log management product with Svelte before, and all the components from the custom select menu, the date selector, the search view etc., were custom built. I don't see any reason why it cannot be done again. We could ideally build a UI component library, enabling us to contribute back to the Svelte community.
• The time taken to build those components are fairly low, considering that no matter what framework we choose most of our components need to be custom built.

---

This thread is open for different set of views and opinions, and factors that we may have failed to consider. Also, please feel free to add more frameworks/libraries in the thread, that can also be analyzed.

[kgodey]

@pavish Your recommendation of Django templates + Svelte sounds good to me. I have not used Svelte before, but I did some reading on it, and it seems like a solid option.

I agree that HTMX/Django Unicorn, and other "sprinkling of JavaScript" approaches are not a good fit for Mathesar because of the amount of rapid interactivity that we would like in the product. I'm curious about some of the higher performing libraries from your references like HyperApp, AppRun, etc. but I don't think it's worth investigating them further. Svelte seems fine and I think your experience with Svelte will help us get up and running faster, which is good.

[ghislaineguerin]

@pavish I like your recommendation. I also think the idea of building a UI component library is a good one.