2021-08-03 planning discussion

[kgodey]

Discussion topics

• Retrospective: Summary of last week's work.
• Active Work: Work planned for the next week.
• Updates: General updates everyone should be aware of.
• Questions: Questions for the team to answer.

About

Each Tuesday, we start a discussion about Mathesar's progress. Everyone is welcome to add a new topic to discuss or to comment on existing topics.

[kgodey]

Retrospective

This post lists issue updates from 2021-07-27 until now by milestone.

2021-07 improvements

• Issues closed:
  • [@kgodey] Fix GH actions for PRs from forks #485

03. Working with Schemas

• Issues closed:
  • [@andyharley-cci] AttributeError if there's no schemas synced to the database #437
  • [@eito-fis] DB syncing is not triggered when objects are fetched without involving the API #365
• Issues in progress:
  • [@dhruvkb] Implement schema list page #171
  • [@pavish] Implement new top navigation and layout #362
• New issues created:
  • Schema list page interactivity #518, created by @pavish

04. Tables from File Import

• Issues closed:
  • [@obulat] New CSV imports use autogenerated column names. #491
• Issues in progress:
  • [@mathemancer] API updates for confirm data step in CSV file import. #201
• New issues created:
  • New CSV imports use autogenerated column names. #491, created by @kgodey

05. Editable Tables

• Issues closed:
  • [@pavish] Implement record deletion #196
• Issues in progress:
  • [@pavish] Implement record editing #195

06. Working with Tables

• Issues closed:
  • [@andyharley-cci] Table creation returns 500 if there's a table with the same name #364
  • [@kgodey] Unique columns #302
  • [@eito-fis] Allow user to create a new table via pasting from spreadsheet #190
• Issues in progress:
  • [@eito-fis] API to upload file from URL #203
  • [@eito-fis] Table API should support creating tables without names #449
  • [@eito-fis] Create canonical ordering for tables and return it by default #397
  • [@ghislaineguerin] Design for editing column properties and table constraints #324
  • [@kgodey] CRUD API for table constraints #326
• New issues created:
  • DB function to get rows with duplicate values #515, created by @kgodey
  • Add filter to show records with duplicate values in a set of columns #501, created by @kgodey
  • Implement setting a default value for a column #500, created by @kgodey
  • Implement setting a column as non-nullable or unique #499, created by @kgodey
  • Implement showing and changing a column's type #498, created by @kgodey
  • Implement renaming a column #497, created by @kgodey
  • Implement deleting a column #496, created by @kgodey
  • Implement adding a new column #495, created by @kgodey
  • DB and API work for adding a default value to columns #494, created by @kgodey

07. Initial Data Types

• Issues in progress:
  • [@ghislaineguerin] Design for money data type #239
  • [@powellc] Handle BOOLEAN data type in the backend #387
  • [@mathemancer] Handle NUMERIC data type in the backend. #377
  • [@ghislaineguerin] Design for Number data types #238
  • [@ghislaineguerin] Design for text data type #234

2021-08 improvements

• Issues closed:
  • [@aaditkamat] Svelte errors displayed during the preprocessing step while running the UI container #511
• Issues in progress:
  • [@mr-gabe49] Select component keyboard accessibility #483
  • [@pavish] Serve built static files in production #358
  • [@andyharley-cci] Set up staging environment for Mathesar #225
  • [@Dhruvi16] Create button component in the component library. #322
• New issues created:
  • Uncaught 404s in record viewsets and pagination classes #517, created by @eito-fis
  • Svelte errors displayed during the preprocessing step while running the UI container #511, created by @aaditkamat
  • Split up API and serializer code by object #510, created by @kgodey

10. Multiple Database Support

• Issues in progress:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395

24. Deployment

• New issues created:
  • Evaluate potential security issues with file uploads #509, created by @kgodey
  • Decide which Postgres versions we want to support. #505, created by @kgodey

26. Post-MVP

• New issues created:
  • Backend spec for updating data in views that don't meet PostgreSQL "updatable" status #504, created by @kgodey

[kgodey]

Active Work

This post lists active work planned for this week, based on the Active GitHub project. It also includes a current summary of milestones involved in active work.

[2021-08-09] 03. Working with Schemas

• 10 issues already closed.
• Issues currently in progress:
  • [@dhruvkb] Implement schema list page #171
  • [@pavish] Implement new top navigation and layout #362
• Issues not yet started:
  • [@pavish] Switch to selected schema in file import view #316
  • [@dhruvkb] Schema list page interactivity #518

[2021-08-11] 04. Tables from File Import

• 26 issues already closed.
• Issues currently in progress:
  • [@mathemancer] API updates for confirm data step in CSV file import. #201
• Issues not yet started:
  • [@andyharley-cci] Handle non-unicode encoded datafiles #421
  • [@pavish] Import status indicators #296
  • [@pavish] File import preview page #295

[2021-08-16] 05. Editable Tables

• 6 issues already closed.
• Issues currently in progress:
  • [@pavish] Implement record editing #195
• Issues not yet started:
  • [@pavish] Implement adding new records #197

[2021-08-23] 06. Working with Tables

• 9 issues already closed.
• Issues currently in progress:
  • [@eito-fis] API to upload file from URL #203
  • [@eito-fis] Table API should support creating tables without names #449
  • [@eito-fis] Create canonical ordering for tables and return it by default #397
  • [@ghislaineguerin] Design for editing column properties and table constraints #324
  • [@kgodey] CRUD API for table constraints #326
• Issues not yet started:
  • [@eito-fis] Extend column API to support duplicating columns #452
  • [@eito-fis] DB and API work for adding a default value to columns #494
  • [@eito-fis] Add filter to show records with duplicate values in a set of columns #501
  • [@eito-fis] DB function to get rows with duplicate values #515
• Issues remaining in backlog:
  • 14 frontend issues

[2021-08-30] 07. Initial Data Types

• 2 issues already closed.
• Issues currently in progress:
  • [@ghislaineguerin] Design for money data type #239
  • [@powellc] Handle BOOLEAN data type in the backend #387
  • [@mathemancer] Handle NUMERIC data type in the backend. #377
  • [@ghislaineguerin] Design for Number data types #238
  • [@ghislaineguerin] Design for text data type #234
• Issues not yet started:
  • [@ghislaineguerin] Design for date & time data types #240
  • [@powellc] Implement filtering options for Boolean types #388
  • [@powellc] Implementation for boolean data type #247
  • [@ghislaineguerin] Design for email data type #235
  • [@ghislaineguerin] Design for URI data type #236
  • [@ghislaineguerin] Design for boolean data type #237
  • [@ghislaineguerin] Design for duration data type #241
  • [@mathemancer] Handle SMALLINT data type in the backend. #378
  • [@mathemancer] Handle INTEGER data type in the backend. #379
  • [@mathemancer] Handle BIGINT data type in the backend. #380
  • [@mathemancer] Handle DECIMAL data type in the backend. #381
  • [@mathemancer] Handle REAL data type in the backend. #382
  • [@mathemancer] Handle DOUBLE PRECISION data type in the backend. #383
  • [@kgodey] Store friendly to DB type mapping in the backend #367
  • [@kgodey] Ensure that the API only accepts DB types as input everywhere we use types. #369
  • [@kgodey] Extend the v0/databases/'s types list to show both Mathesar and DB type information. #368
  • [@rootxnet] Implementation for text data type #244
  • [@mathemancer] Backend work for custom MATHESAR_TYPES.PERCENTAGE type. #434
• Issues remaining in backlog:
  • 37 backend and database issues
  • 8 frontend issues

[2021-08-31] 2021-08 improvements

• 4 issues already closed.
• Issues currently in progress:
  • [@mr-gabe49] Select component keyboard accessibility #483
  • [@pavish] Serve built static files in production #358
  • [@andyharley-cci] Set up staging environment for Mathesar #225
  • [@Dhruvi16] Create button component in the component library. #322
• Issues not yet started:
  • [@mathemancer] Schema sqlalchemy engine caching is per-instance #280
• Issues remaining in backlog:
  • 2 backend and database issues
  • 9 frontend issues
  • 6 design issues
  • 1 infrastructure issues

[2021-09-06] 10. Multiple Database Support

• 6 issues already closed.
• Issues currently in progress:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395
• Issues remaining in backlog:
  • 4 backend and database issues
  • 4 frontend issues
  • 2 design issues

[kgodey]

Updates

Roadmap Updates

The roadmap has been updated to account for editing records taking more time than originally anticipated.

• Please review the new order and deadlines.
• Feel free to participate in the discussion about how to approach editing: Loading data on the frontend #514

Presentation

The Mathesar team is presenting on progress so far to the larger CCI research meeting. Please let me know if you'd like an invite. It's at 11AM Eastern Time on Thursday, Aug 5.

Contributors

• We have two new contributors this week with merged PRs. Thank you, @aaditkamat and @obulat!
• I'd also like to mention @dhruvkb's work on Create the schema list page #513, which involved implementing a whole new page in Mathesar and is almost merged.
• @andyharley-cci got his first backend PR merged today. :)

Contractors

• @powellc is working with us part-time on backend work from now until the end of October. Welcome, @powellc!
• @rootxnet is working with us on Implementation for text data type #244 and its sub-issues. Welcome, @rootxnet!

Comments or feedback on any of the above is welcome.

[kgodey]

Questions

@mathemancer, @eito-fis, @ghislaineguerin, @pavish: Your responses are required. Others are also welcome to respond.

Retrospective

• What went well this week?
• What could have gone better this week?
• Are there any improvements to our process you'd like to suggest?

Active Work

• Does your workload for this week look okay?
• Are there any blockers to meeting our upcoming deadlines?

Discussion

• Any feedback on the weekly discussion (either format or content)?
• Anything else you'd like to add?

[kgodey]

What went well this week?

• I got to actually write some code for API for adding and removing table constraints #488
• It's nice that we have more contributors.

What could have gone better this week?

Backend work is a little slower than I'd like. I'd like to try to find more time to help with it.

Are there any improvements to our process you'd like to suggest?

No.

Does your workload for this week look okay?

Yes. I'm also going to be speccing out more issues, which isn't represented in the active work.

Are there any blockers to meeting our upcoming deadlines?

Not that I'm aware of.

Any feedback on the weekly discussion (either format or content)?

I'm hoping that putting the questions in one post will make it easier for people to find them. I wasn't getting a lot of responses when they were at the end of other long posts.

[pavish]

What went well this week?

• I'm glad we got new contributors, and for the extra frontend help (Thanks @dhruvkb !).
• This week was not very productive for me. I spent a lot of time thinking on ways to handle editing with the current data loading approach, before posting it for suggestions.

What could have gone better this week?

• It would have been better if we had planned the data loading strategy earlier, considering all our features.
• I still find that it's okay, we do not have a lot of dependent features on top of spreadsheet, so changes should be simple.
• I would like to hit some milestones and speed up frontend.

Are there any improvements to our process you'd like to suggest?

None, at the moment.

Does your workload for this week look okay?

Yes. It looks good to me.

Are there any blockers to meeting our upcoming deadlines?

I do not see any blockers.

Any feedback on the weekly discussion (either format or content)?
Anything else you'd like to add?

None.

[mathemancer]

What went well this week?

Data Checking API flow is finally done!

What could have gone better this week?

I want to get faster with PR reviews; will be focusing on that.

Are there any improvements to our process you'd like to suggest?

None

Does your workload for this week look okay?

I think it looks good.

Are there any blockers to meeting our upcoming deadlines?

Not to my knowledge.

Any feedback on the weekly discussion (either format or content)?

I think the new format is working well.

Anything else you'd like to add?

Nope!

[powellc]

• What went well this week?

Starting to get more comfortable with the codebase and got a few PR reviews under my belt and some data type tests run successfully :D

• What could have gone better this week?

Need to find a good rhythm for getting ticket work done and communicated out. Still a little new to everything

• Are there any improvements to our process you'd like to suggest?

Nope!

• Does your workload for this week look okay?

Yup, going to finish up reviewing the current boolean type implementation and move on to adding filtering on booleans.

• Are there any blockers to meeting our upcoming deadlines?

Nope.

• Any feedback on the weekly discussion (either format or content)?

Love it!

• Anything else you'd like to add?

Nada 😄

[eito-fis]

What went well this week?

Finally getting some PR reviews in!

What could have gone better this week?

Progress on some easier PRs was slow - would like to get the PR right in fewer review cycles if it's simple.

Are there any improvements to our process you'd like to suggest?

N / A

Does your workload for this week look okay?

Seems fine to me.

Are there any blockers to meeting our upcoming deadlines?

N / A

Any feedback on the weekly discussion (either format or content)?

Having all questions in once place is definitely nice.

Anything else you'd like to add?

N / A

[ghislaineguerin]

What went well this week?

I had been on vacation, so not an entire week. But it was nice to be back and catch up quickly as we do a lot of async communication.
I also liked how we planned and practiced for the presentation. It felt like a very productive meeting.

What could have gone better this week?

It would have been easier if I didn't have so many reviews open. Closing them has been challenging as many sub-parts for data types are still not defined or become more evident as I add new parts.

Are there any improvements to our process you'd like to suggest?

I should try to identify those parts before I start designing and plan to include them in my planning.

Does your workload for this week look okay?

It does.

Are there any blockers to meeting our upcoming deadlines?

No blockers.

Any feedback on the weekly discussion (either format or content)?

No, I think it works well.

Anything else you'd like to add?

Not for now.