(fix) O3-3503: Fix pagination of problem datasource fetch results #333

CynthiaKamau · 2024-06-28T08:25:04Z

Requirements

This PR has a title that briefly describes the work done including the ticket number. If there is a ticket, make sure your PR title includes a conventional commit label. See existing PR titles for inspiration.
My work conforms to the OpenMRS 3.0 Styleguide and design documentation.
My work includes tests or is validated by existing tests.

Summary

Currently, the implementation of the problem data source fetches only 50 results due to pagination constraints. This limitation affects the visibility of data across three concept classes. This pull request aims to enhance the data fetching process to retrieve all results from the endpoint, ensuring comprehensive data coverage.

Screenshots

Screen.Recording.2024-06-28.at.11.17.26.mov

Related Issue

https://openmrs.atlassian.net/browse/O3-3503

Other

github-actions · 2024-06-28T08:27:58Z

Size Change: +2 B (0%)

Total Size: 1.14 MB

ℹ️ View Unchanged

Filename	Size	Change
`dist/151.js`	300 kB	0 B
`dist/225.js`	2.57 kB	0 B
`dist/277.js`	1.84 kB	0 B
`dist/3.js`	481 B	0 B
`dist/300.js`	642 B	0 B
`dist/335.js`	968 B	0 B
`dist/353.js`	3.02 kB	0 B
`dist/41.js`	3.37 kB	0 B
`dist/422.js`	6.8 kB	0 B
`dist/540.js`	2.63 kB	0 B
`dist/55.js`	758 B	0 B
`dist/572.js`	252 kB	+1 B (0%)
`dist/617.js`	86.9 kB	0 B
`dist/635.js`	14.3 kB	0 B
`dist/733.js`	107 kB	0 B
`dist/901.js`	11.8 kB	0 B
`dist/99.js`	691 B	0 B
`dist/993.js`	3.09 kB	0 B
`dist/main.js`	340 kB	+1 B (0%)
`dist/openmrs-esm-form-engine-lib.js`	3.67 kB	0 B

_{compressed-size-action}

samuelmale · 2024-07-01T06:32:27Z

Loading all available concepts appears to be very resource-intensive. I suggest implementing a "background search" that uses user input as the search term.

CynthiaKamau · 2024-07-01T06:44:06Z

Loading all available concepts appears to be very resource-intensive. I suggest implementing a "background search" that uses user input as the search term.

This will be a bit hard given that problems come from three different concept classes, also for questions that use problems as obs, search might not work

samuelmale · 2024-07-01T08:30:18Z

This would work perfectly for a single concept class:

const searchTerm = "WHO";

/openmrs/ws/rest/v1/concept?searchType=fuzzy&v=custom:(uuid,display,conceptClass:(uuid,display))&class=8d4918b0-c2cc-11de-8d13-0010c6dffd0f&q=${searchTerm}

To support multiple classes I think we can filter the returned results by the classes:

const classes = [
  '8d4918b0-c2cc-11de-8d13-0010c6dffd0f',
  '8d492954-c2cc-11de-8d13-0010c6dffd0f',
  '8d492b2a-c2cc-11de-8d13-0010c6dffd0f'
];

const url = `${restBaseUrl}/concept?searchType=fuzzy&v=custom:(uuid,display,conceptClass:(uuid,display))&q=${searchTerm}`;

openmrsFetch(url).then(({ data }) => {
  return data.results.filter(
    (concept) => classes.includes(concept.conceptClass.uuid)
  );
});

CynthiaKamau · 2024-07-01T08:39:48Z

This would work perfectly for a single concept class:

const searchTerm = "WHO";

/openmrs/ws/rest/v1/concept?searchType=fuzzy&v=custom:(uuid,display,conceptClass:(uuid,display))&class=8d4918b0-c2cc-11de-8d13-0010c6dffd0f&q=${searchTerm}

To support multiple classes I think we can filter the returned results by the classes:

const classes = [
  '8d4918b0-c2cc-11de-8d13-0010c6dffd0f',
  '8d492954-c2cc-11de-8d13-0010c6dffd0f',
  '8d492b2a-c2cc-11de-8d13-0010c6dffd0f'
];

const url = `${restBaseUrl}/concept?searchType=fuzzy&v=custom:(uuid,display,conceptClass:(uuid,display))&q=${searchTerm}`;

openmrsFetch(url).then(({ data }) => {
  return data.results.filter(
    (concept) => classes.includes(concept.conceptClass.uuid)
  );
});

The issue with searching through the results is that the data returned contains 50 results, and if you filter, you will only get one result, unless the limit is increased, which will still not return the expected results

samuelmale · 2024-07-01T09:01:09Z

Makes sense. @ibacher should we consider supporting multiple concept classes (here) for fuzzy searches?

brandones · 2024-07-23T03:20:56Z

Ping @samuelmale @ibacher ?

samuelmale · 2024-07-23T10:36:47Z

Since we lack the ideal backend support, I think @CynthiaKamau's solution can work for now ie. fetching concepts of all the relevant classes in parallel.

Question: Does this problem exist in AFE?

CynthiaKamau · 2024-07-23T11:05:09Z

Since we lack the ideal backend support, I think @CynthiaKamau's solution can work for now ie. fetching concepts of all the relevant classes in parallel.

Question: Does this problem exist in AFE?

In AFE, diagnosis/problems are handled as a datasource, so

/concept?searchType=

In AFE, the input is searchable, in RFE, there instances where its a select box

samuelmale · 2024-07-23T11:22:22Z

In AFE, the input is searchable, in RFE, there instances where its a select box

IIRC the ui-select-ext supports searching, can we instead switch to doing a search? cc: @kajambiya

brandones · 2024-08-05T12:51:41Z

In AFE, the input is searchable, in RFE, there instances where its a select box

IIRC the ui-select-ext supports searching, can we instead switch to doing a search? cc: @kajambiya

ping @kajambiya

samuelmale · 2024-08-08T12:42:01Z

Filed ticket: https://openmrs.atlassian.net/browse/O3-3770

brandones · 2024-08-19T15:40:18Z

@samuelmale Thanks for the ticket. What should we do about this PR? Close in favor of doing O3-3770? Merge until we have a better solution?

mogoodrich · 2024-08-19T19:55:05Z

Actually @brandones @samuelmale ... @chibongho just recently merged the following PR, worth taking a look at: openmrs/openmrs-esm-core@45bdc9b

also, linked ticket - https://openmrs.atlassian.net/browse/O3-3189

samuelmale · 2024-08-19T20:46:21Z

Actually @brandones @samuelmale ... @chibongho just recently merged the following PR, worth taking a look at: openmrs/openmrs-esm-core@45bdc9b

Thanks for sharing, @mogoodrich. I think this is particularly useful when dealing with paginable components like data tables.

What should we do about this PR? Close in favor of doing O3-3770? Merge until we have a better solution?

@brandones I’m inclined to implement or fix the search capability rather than eagerly loading all concepts from different classes in one shot.

brandones · 2024-08-20T13:10:01Z

Ok, thanks @samuelmale . @CynthiaKamau , will you do that, do you have the time?

CynthiaKamau · 2024-08-30T11:00:33Z

Ok, thanks @samuelmale . @CynthiaKamau , will you do that, do you have the time?

Yes, working on updating it

… classes

CynthiaKamau · 2024-08-30T12:23:02Z

Actually @brandones @samuelmale ... @chibongho just recently merged the following PR, worth taking a look at: openmrs/openmrs-esm-core@45bdc9b

Thanks for sharing, @mogoodrich. I think this is particularly useful when dealing with paginable components like data tables.

What should we do about this PR? Close in favor of doing O3-3770? Merge until we have a better solution?

@brandones I’m inclined to implement or fix the search capability rather than eagerly loading all concepts from different classes in one shot.

@samuelmale would switching to this format suffice , i wouldn't really need to make any changes ?

{
              "label": "Test problem",
              "type": "obs",
              "id": "patient_problem",
              "questionOptions": {
                "rendering": "ui-select-extended",
                "isSearchable": true,
                "concept": "160540AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
                "datasource": {
                  "name": "problem_datasource"
                }
              }
            }

samuelmale · 2024-09-01T19:08:13Z

@CynthiaKamau The question's configuration looks okay; does it work as expected?

CynthiaKamau · 2024-09-02T06:27:17Z

@CynthiaKamau The question configuration looks okay; does it work as expected?

Is it related to this ticket ?

samuelmale · 2024-09-17T10:43:28Z

Is it related to this ticket ?

Affirmative! I also noticed that you're working on the talked about ticket.

CynthiaKamau requested review from kajambiya, samuelmale and arodidev June 28, 2024 08:25

CynthiaKamau requested review from brandones and ibacher July 4, 2024 06:13

brandones removed request for brandones, kajambiya and arodidev July 4, 2024 12:44

brandones changed the title ~~O3-3503 Fix problem datasource fetch results based on problem concept…~~ (fix) O3-3503: Fix pagination of problem datasource fetch results Aug 19, 2024

O3-3503 Fix problem datasource fetch results based on problem concept…

47b0b33

… classes

CynthiaKamau force-pushed the O3-3503 branch from ed1a935 to 3836126 Compare September 2, 2024 07:36

Increase debounce delay to allow user to search

8c39210

CynthiaKamau force-pushed the O3-3503 branch from 3836126 to 8c39210 Compare September 2, 2024 07:38

CynthiaKamau closed this Sep 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(fix) O3-3503: Fix pagination of problem datasource fetch results #333

(fix) O3-3503: Fix pagination of problem datasource fetch results #333

CynthiaKamau commented Jun 28, 2024 •

edited by brandones

Loading

github-actions bot commented Jun 28, 2024 •

edited

Loading

samuelmale commented Jul 1, 2024

CynthiaKamau commented Jul 1, 2024

samuelmale commented Jul 1, 2024

CynthiaKamau commented Jul 1, 2024

samuelmale commented Jul 1, 2024

brandones commented Jul 23, 2024

samuelmale commented Jul 23, 2024 •

edited

Loading

CynthiaKamau commented Jul 23, 2024

samuelmale commented Jul 23, 2024 •

edited

Loading

brandones commented Aug 5, 2024

samuelmale commented Aug 8, 2024

brandones commented Aug 19, 2024

mogoodrich commented Aug 19, 2024 •

edited

Loading

samuelmale commented Aug 19, 2024

brandones commented Aug 20, 2024

CynthiaKamau commented Aug 30, 2024

CynthiaKamau commented Aug 30, 2024 •

edited

Loading

samuelmale commented Sep 1, 2024 •

edited

Loading

CynthiaKamau commented Sep 2, 2024

samuelmale commented Sep 17, 2024

(fix) O3-3503: Fix pagination of problem datasource fetch results #333

(fix) O3-3503: Fix pagination of problem datasource fetch results #333

Conversation

CynthiaKamau commented Jun 28, 2024 • edited by brandones Loading

Requirements

Summary

Screenshots

Related Issue

Other

github-actions bot commented Jun 28, 2024 • edited Loading

samuelmale commented Jul 1, 2024

CynthiaKamau commented Jul 1, 2024

samuelmale commented Jul 1, 2024

CynthiaKamau commented Jul 1, 2024

samuelmale commented Jul 1, 2024

brandones commented Jul 23, 2024

samuelmale commented Jul 23, 2024 • edited Loading

CynthiaKamau commented Jul 23, 2024

samuelmale commented Jul 23, 2024 • edited Loading

brandones commented Aug 5, 2024

samuelmale commented Aug 8, 2024

brandones commented Aug 19, 2024

mogoodrich commented Aug 19, 2024 • edited Loading

samuelmale commented Aug 19, 2024

brandones commented Aug 20, 2024

CynthiaKamau commented Aug 30, 2024

CynthiaKamau commented Aug 30, 2024 • edited Loading

samuelmale commented Sep 1, 2024 • edited Loading

CynthiaKamau commented Sep 2, 2024

samuelmale commented Sep 17, 2024

CynthiaKamau commented Jun 28, 2024 •

edited by brandones

Loading

github-actions bot commented Jun 28, 2024 •

edited

Loading

samuelmale commented Jul 23, 2024 •

edited

Loading

samuelmale commented Jul 23, 2024 •

edited

Loading

mogoodrich commented Aug 19, 2024 •

edited

Loading

CynthiaKamau commented Aug 30, 2024 •

edited

Loading

samuelmale commented Sep 1, 2024 •

edited

Loading