Adding, editing and deleting records

[ghislaineguerin]

The specs related to issues #193 #194 #192 are ready for review. Please take a look at the attached document and follow the review guidelines.

@mathemancer @kgodey @pavish, your feedback is required.
The deadline for feedback is at the end of this week. Please plan your review so that at least one round of changes can be made and reviewed if needed.

Document Link

https://hackmd.io/@mathesar/SymUiJcsO

[kgodey]

My first round of review is complete.

[pavish]

My first round of review is also complete.

[ghislaineguerin]

@pavish @kgodey
I've read your feedback, and I agree with the need to make the 'new row' easy to access. Usually, for most table-based interfaces, new rows are added to the bottom of the list. Spreadsheets will sometimes allow rows to be inserted in arbitrary positions, but this doesn't apply to our application. The order of the table records is (I assume) determined by the insert order, where the most recent records are at the bottom.
An optimal solution is that we always navigate to the last record page when adding a new one. These are some reasons for this:

• Having the new record on top and then having it disappear when inserted is not a good experience.
• By navigating to the last record, a user can add multiple records subsequently without requiring an additional step (click add row again).
• Error handling across multiple rows will be more difficult if records can be added to any page.
• If we ever move away from paginated views it will still be valid.

[pavish]

• I strongly feel that only allowing users to add new records in last page is very restrictive. We will have to allow them to add records no matter what page they are in.
• I agree with point 2. If the user navigates to the last page, we can allow adding multiple subsequent records. But on other pages, we should have an option to let them add records.
• If any filter, or sort option is applied, even if user is in last page, the record may disappear (since it depends on the filter), so point 1 will have to be handled no matter the scenario.

[ghislaineguerin]

@pavish I think they can start the add record process from any page, but what you are saying is that you don't want them to be taken to the last page, rather remain on the current page right? Why do we think persisting table location is so important to users?
I do see how it's convenient, I'm just not sure if taking users to the last row hinders their experience as they'd still have to go there in case they want to make corrections.
Also, I am aware that the records will disappear if filtered, but if there are errors then the changes stop being saved and reordering of records doesn't happen until errors are fixed. Again, once that process is done, it's fine to have the records disappear.

[kgodey]

@ghislaineguerin I'm not sure why navigating to the last page is necessary given the points you've made. The record won't be reordered according to table display properties until all errors are handled and it's saved, whichever page it happens to be on.

[ghislaineguerin]

@kgodey @pavish
Just talked with Kriti and we agreed on balancing both the need for a persistent position for new rows, with the possibility of adding them from anywhere in the table. I explained that persistence was important so that users could work through multiple records, in the event of bulk-adding or pasting data. The tricky bit was supporting this persistent position and auto-save at the same time, and deciding when the new records are ready.

Given that, we might try having a group of persistent records that then get re-ordered on table refresh or once the user clicks outside of the delimited area. I will update the prototype to reflect this.

[mathemancer]

I finished my first round of review. TBH, I'm convinced by the confusion around the interplay between auto-save and filtering/sorting that adding records as rows rather than fields in a form is a problem. If we have a "add record" button that creates some kind of pop-up with a form, we gain:

1. order independence
2. filter independence
3. An interface that users are familiar with w.r.t. required fields
4. An easy-to-find save button

While I understand that folks are used to editing tables in a spreadsheet-like fashion, I think that adding records via a form would lead to a more intuitive UI that helps them understand the advantages and constraints of working with an actual DB backend in this way. (even things like fkey constraints will feel more natural).

[kgodey]

TBH, I'm convinced by the confusion around the interplay between auto-save and filtering/sorting that adding records as rows rather than fields in a form is a problem.

@mathemancer I think your feedback would be most useful if you focused on describing the problem with the existing design rather than proposing a specific solution (see the first point in our design review guidelines)

Also, you might want to wait to respond until @ghislaineguerin is done updating the document from the first round of feedback.

[ghislaineguerin]

@mathemancer @kgodey @pavish
I have updated the prototype based on feedback regarding the placement of the new row. The new UI should:

• Allow users to add new rows from any page
• Allow users to make changes to new rows while adding them (prevent filter or sort from relocating the row)
  However, I think we should try to maintain the new row to the bottom of the table view as it is the standard for similar data grid implementations.
  The problem remains with not assigning row numbers as the records aren't really in their final position.

I tend to agree with @mathemancer opinion on forms offering a better experience for data entry, but I think we can optimize table edits as well, allowing users to choose their preferred method and working around the constraints of each one. I like table edit as it allows to easily paste or bulk add data and visualize it, which I feel might be fairly common use of Mathesar, but forms might offer more opportunities to introduce advanced features and increase their discoverability.

[mathemancer]

@kgodey is correct, I was being too prescriptive, rather than descriptive. I'll comment more tomorrow when I can devote more attention to it.

[pavish]

@ghislaineguerin Can we have one more view on the prototype where the table has a lot of data, and then we are trying to add a new row. That will give better context on how the UX would feel like, especially since we'll also have a page count and pagination present.

[ghislaineguerin]

@pavish I did at the last page. If you click on the top tab and go to awards table, then hit "Add" it should scroll down to the empty row

[kgodey]

@ghislaineguerin I assume we should only be looking at the prototype and not the rest of the document (since the screenshots don't match).

My main concern is with making it really clear to the user what's going on with the save status of their data. I like that we have a "saving changes" indicator above the table, but I think we should make it really clear to the user when we're not even attempting to save because all the required fields haven't been filled out. Displaying errors on the row level would help with this.

I did some quick mockups in Figma to convey what I'm thinking. Please note that I'm not proposing this exact solution, I just mocked up an idea because I figured that a visual explanation would be easier to follow.

[kgodey]

Other general thoughts:

• It would be good to have icons for things like "delete", "add", and "saving changes" next to the text, I think it would make it easier to differentiate quickly.
• I think we should make the text "Delete Records", "Add Records", etc. to make it clear what the buttons are for, there are a lot of things we could be deleting or adding.
• I'm wondering if having a colored background or other row level indicators for both the "unsaved" state and the "saving in progress" state would be useful. Again, these are just ideas, the general problem I'm trying to solve is that the cues we're offering the users about things changing might be too subtle.
  • e.g. we could put the cells in a disabled state while they are being synced to the backend so that the user doesn't try to edit the cell while it's being updated, or we could have a gray background.
  • we could have a green background or something like that for cells that have been added but aren't in their final position yet.

[ghislaineguerin]

@kgodey @pavish @mathemancer
Additional review notes have been added to the document to cover the following:

• Accessing new row control
• Order and Filter Independence
• Required Fields
• Save Status
• Disabled State while Syncing

Edit: Requesting re-review and will move the deadline to Monday next week.

[pavish]

@ghislaineguerin If we make it floating, it will obstruct the last row, especially while editing. In our case, the last row is always an empty one with '*', so I guess it should be fine.

I am still not convinced about showing the total records. What is the reasoning behind showing it, since we already have it on the pagination pane? I would like to understand it's use better.

Upon clicking add record, we will automatically scroll to the bottom. And we don't hide our scrollbar. So, does this navigator add any benefit?

[ghislaineguerin]

@pavish I'm still missing a place where we can show selected records, count, and other row related info. We might incorporate that into the pagination bar. I will give it some more thought as I agree that it might not be the best placement, but since we are selecting now, it would be good to also have a place where we show that kind of count.

[pavish]

@ghislaineguerin Okay, makes sense.

[mathemancer]

The only concern I have remaining is that adding/deleting rows is fundamentally different from adding/deleting columns, but they're presented in the same drop-down in the graphics shown. That could lead to some confusion for users.

[kgodey]

@mathemancer In the newest prototype, there's no dropdown, just an Add Record button and a delete button (both of which only apply to records).

[ghislaineguerin]

Review Conclussions

• We agreed that the optimal experience is for users to add records from any page, irrespective of which table location they are on.
• We also agreed that filters, sorting, and other table properties shouldn't affect new records until the user navigates away or refreshes the content. This solved the problems related to allowing users to continue editing or solve errors in new records.
• Additional controls for indicating the number of records and providing navigation to the top or bottom of a list of records will be detailed in a separate issue.

[ghislaineguerin]

@kgodey @pavish @mathemancer
I have pushed the specs to https://wiki.mathesar.org/design/process/documents/add-update-delete-records-specs