[DEPR] bok-choy: remove supporting tooling & archive repo

[feanil]

Overview

We propose the removal all remaining bok-choy test suites, all tooling code supporting bok-choy test suites, and the bok-choy test framework itself.

Target acceptance date: 2022-02-18

Earliest possible release without bok-choy: Nutmeg

Context

Bok-Choy is a home-grown library allowing acceptance-level tests against Web pages to be written in Python. Over the years, it has been used in a few different places:

• edX’s end-to-end testing suite for LMS & CMS
• edx-platform accessibility tests
• various repositories' acceptance tests suites:
  • edx-platform
  • edx-ora2
  • edx-notifications
  • ecommerce
  • credentials
  • insights
  • xblock-sdk
  • xblock-drag-and-drop-v2 (very minimal usage)

As we have moved from Django-templated (“legacy”) frontends to React micro-frontends (“MFEs”), bok-choy has fallen out favor, as the framework does not work or make sense to use with MFEs. Even with legacy frontends, we have found their bok-choy test suites to be so slow and flaky that their usefulness has been called into question.

In 2020, we decided to remove the vast majority of bok-choy acceptance tests from edx-platform. In 2021, edX decided to do the same for edx-platform accessibility tests.

Replacement

There is no direct Open edX replacement for bok-choy. Instead, we encourage community members to write acceptance and integration tests using third-party libraries. For example, the new Authentication Micro-Frontend uses Cypress for end-to-end tests.

Removal steps

• 2U: archive the edx-e2e-tests repository.
• For each repository listed under “Context”:
  • Delete all the remaining bok-choy tests.
    • or, if appropriate, refactor end-to-end tests to not use bok-choy instead of deleting them.
  • Remove any settings files, test scripts, or tooling configuration specific to bok-choy.
  • Remove bok-choy from the repository’s requirements list.
• Remove all references to bok-choy and edx-e2e-tests from devstack. Remove the chrome and firefox services.
• Search for “bok-choy” and “bok_choy” in the openedx GitHub organization, removing any dangling references to the test library or old test suites.
  • 2U: Do the same for the edx GH org.
• Archive the bok-choy repository.

--

Original Jira Issue: https://openedx.atlassian.net/browse/DEPR-174
Communicated to community: https://discuss.openedx.org/t/deprecation-removal-bok-choy-test-suites-framework/6547

[kdmccormick]

PR to mark bok-choy as deprecated in its README and ReadTheDocs: openedx-unsupported/bok-choy#364

[kdmccormick]

No concerns have been raised on the GitHub issue, the DEPR ticket in Jira, or the deprecation announcement on the forums, so Bok-Choy is now officially deprecated! A notice has been added to the repository's README and ReadTheDocs page (the ReadTheDocs notice may take some time to manifest).

[kdmccormick]

Big PR of bok-choy cruft removed from edx-platform: openedx/edx-platform#29813

[feanil]

@jmbowman what's left to do on this issue?

[jmbowman]

Finish moving the e2e tests to Cypress, and look for any remaining usage in other repos that we may have overlooked in our first pass. I think the e2e switchover is very close to complete, but I'm not sure at the moment what the exact ETA is.

[kdmccormick]

@jmbowman just curious, any movement on the e2e switchover, or a relevant ticket we could link to this one?

[dianakhuang]

The e2e tests for edx.org have been switched over to using Cypress. I have seen usages of bok choy in credentials and ecommerce, but we can at least start looking into cleaning out some of edx-platform.

[robrap]

Here are some github searches to see some current uses:

Search code:

• https://github.com/search?q=org%3Aopenedx+bokchoy&type=code
• https://github.com/search?q=org%3Aopenedx+bok_choy&type=code

Search repo files, like:

• https://github.com/openedx/edx-platform/find/master (search for "bokchoy")

Also in docs:

• https://openedx.atlassian.net/wiki/spaces/AC/pages/23003228/Everything+About+Database+Migrations (search for "bokchoy")
  • Note: is there a replacement for testing squashed migrations? See notes under "Click for more on testing...".

[kdmccormick]

Updates:

• bok-choy is used in ecommerce, but ecommerce is also deprecated, so that's fine.
• bok-choy is used in credentials. I opened an issue over there to either port or remove the tests. Archiving the bok-choy repo will not break credentials, so that issue shouldn't block archival of bok-choy.
• I have one final bok-choy PR open intended to push the deprecation notice to bok-choy's page on pypi.
  • Update: Merged. PyPI is updated.

Next steps:

• Ensure that bok_choy settings files (both .py and .yml) are no longer used in edx-platform. In general, production.py and minimal.yml should be suitable replacements.
• Figure out why the deprecation notice in docs/index.rst isn't showing up on https://bok-choy.readthedocs.io/en/latest/. Perhaps the docs aren't being pushed there automatically?
• Go through the remaining repos that install bok-choy: xblock-adventure, xblock-drag-and-drop-v2, xblock-utils, edx-analytics-dashboard, and a few others. You can get a list by searching the org for 'bok-choy=='.
• For each repo that installs bok-choy:
  • Is it deprecated?
    • Yes -> Great, just make a note of that one this issue, and move on.
    • No -> Does it actually use bok-choy?
      • Yes -> Open an issue for removing bok-choy, like this one.
      • No -> Open & merge a PR to remove bok-choy from the list of requirements.
• Archive the bok-choy repo.
• Archive the edx-custom-a11y-rules repo, which is used exclusively as a bok-choy add-on.
• Search for dangling references to bok-choy and remove them. For example, there are several obselete bok_choy settings files in edx-platform which can be removed.

[kdmccormick]

PyPI is now updated:

[robrap]

@kdmccormick: I'm not sure what the refactor should be, but the bok_choy.yml file is used in the following github workflows:

• https://github.com/openedx/edx-platform/blob/2edbdcf7f48eddeff9fc6f1967e3f11be8c8e390/.github/workflows/migrations-check-mysql8.yml#L71-L80
• https://github.com/openedx/edx-platform/blob/2edbdcf7f48eddeff9fc6f1967e3f11be8c8e390/.github/workflows/migrations-check.yml#L84
• https://github.com/openedx/edx-platform/blob/90f4f6543deb39517b75b61b15ba8ff557a873dc/.github/workflows/static-assets-check.yml#L67-L68

[kdmccormick]

@robrap as it turns out, that was actually what led me back to this DEPR ticket today 😄

I'm currently experimenting with replacing those with the minimal.yml files that @feanil recently introduced. What we really need is a broader configuration cleanup, but in the short term I'm hoping minimal.yml will do that trick.

[dianakhuang]

There is a team at 2U that is looking to convert the remaining bok-choy tests as well. We should be able to replace the usages of bok_choy.yml with minimal.yml.

[kdmccormick]

I've got to step away from this to focus on other work. As far as I know, these steps are what's needed to fully close out the DEPR: #13 (comment)

[dianakhuang]

@feanil What's the current status of this? Can it be moved into 'removed'?

[feanil]

We've got one more cleanup PR left before we can fully remove bok-choy. The work is in progress now: openedx-unsupported/ecommerce#4093

[feanil]

@salman2013 I've archived the bok-choy and edx-custom-a11y-rules repos. There are still a bunch of mentions of bokchoy in the platform, can you review them and open PRs for any that should be cleaned up? https://github.com/search?q=org%3Aopenedx%20bokchoy&type=code Once that's done, I think we can close this.

[kdmccormick]

Some more references, from the various spellings of bok-choy

"bok-choy": https://github.com/search?q=org%3Aopenedx+%22bok-choy%22&type=code

"bok choy": https://github.com/search?q=org%3Aopenedx+%22bok+choy%22&type=code

"bok_choy": https://github.com/search?q=org%3Aopenedx+%22bok_choy%22&type=code

[feanil]

@salman2013 can you take a look at the searches that Kyle provided and clean those up as well?

[salman2013]

@kdmccormick @feanil There is a PRs to remove remaining bok-choy references.

• Remove bok-choy reference edx-platform#34216
• Remove bok-choy reference openedx-unsupported/configuration#7101
• Remove bok-choy reference openedx-backstage#34
• Remove bok-choy references. docs.openedx.org#466
• Remove bok-choy references FeedbackXBlock#59
• Remove bok-choy references openedx-unsupported/registrar#594

[salman2013]

@kdmccormick These search result are found in release notes so I leave them intentionally. Could you please check if this should also be removed then i can do that.

[kdmccormick]

Thanks @salman2013 . I'll take a look at the edx-platform, feedbackblock, and registrar PRs.

These search result are found in release notes so I leave them intentionally. Could you please check if this should also be removed then i can do that.

Leaving those there is the right thing to do 👍🏻

[feanil]

I had a question on the FeedbackXBlock Pr but everything else is merged.

[robrap]

Here is a 2U specific search, but one that other orgs can use if they want to remove their own bokchoy references. It uses 4 versions of the term found above:
https://github.com/search?q=org%3Aedx+%28%22bok-choy%22+OR+%22bok+choy%22+OR+%22bok_choy%22+OR+%22bokchoy%22%29++NOT+is%3Aarchived&type=code

[timmc-edx]

Or if you want to be really thorough, GitHub search now supports regular expressions like /bok.?choy/ :-D