-
Notifications
You must be signed in to change notification settings - Fork 30
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Updating documentation * Updates to docs
- Loading branch information
1 parent
001bd9f
commit b8d641f
Showing
8 changed files
with
81 additions
and
71 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,27 @@ | ||
# User Roles and Responsibilities | ||
|
||
There are different types of users/roles that can be created in MedPerf: | ||
Here we introduce user roles at MedPerf. Depending on the objectives and expectations a user may have multiple roles. | ||
|
||
## Benchmark Committee | ||
|
||
Include regulatory bodies, groups of experts (e.g., clinicians, patient representative groups), and data or Model Owners wishing to drive the evaluation of their model or data. Benchmark Committee do not have admin privileges on the system, but they have elevated permissions regarding the benchmark they define, such as deciding which model and Data Providers will participate. | ||
May include healthcare stakeholders (e.g., hospitals, clinicians, patient advocacy groups, payors, etc.), regulatory bodies, data providers and model owners wishing to drive the evaluation of AI models on real world data. While the *Benchmark Committee* does not have admin privileges on MedPerf, they have elevated permissions regarding benchmark assets (e.g., task, evaluation metrics, etc.) and policies (e.g., participation of model owners, data providers, anonymizations) | ||
|
||
![](./images/benchmark_committee.png) | ||
|
||
## Data Providers | ||
|
||
Include hospitals, medical practices, research organizations, and healthcare insurance providers that own medical data, register medical data, and execute benchmark requests. | ||
May include hospitals, medical practices, research organizations, and healthcare payors that own medical data, register medical data, and execute benchmarks. | ||
|
||
![](./images/data_owners.png) | ||
|
||
## Model Owners | ||
|
||
Include ML researchers and software vendors that own a trained medical ML model and want to evaluate its performance. | ||
May include ML researchers and software vendors that own a trained medical ML model and want to evaluate its performance against a benchmark. | ||
|
||
![](./images/model_owners.png) | ||
|
||
## Platform Providers | ||
|
||
Organizations like [MLCommons](https://mlcommons.org/en/){target="\_blank"} that operate a platform that enables a benchmark committee to run benchmarks by connecting Data Providers with Model Owners. | ||
Organizations like [MLCommons](https://mlcommons.org/en/){target="\_blank"} that operate the MedPerf platform enabling benchmark committees to develop and run benchmarks. | ||
|
||
![](./images/platform%20provider.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters