diff --git a/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb b/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb index ae67502..558963b 100644 --- a/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb +++ b/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb @@ -1605,7 +1605,7 @@ "id": "852f94ae", "metadata": {}, "source": [ - "## 5. Set up settings\n", + "### 5. Set up settings\n", "\n", "Above, we just used default settings already implemented for us. This included what annotations to record and how they are displayed in the app. \n", "\n", @@ -1838,7 +1838,7 @@ "id": "7a4f78c5", "metadata": {}, "source": [ - "### 5.1 Autofill\n", + "#### 5.1 Autofill\n", "\n", "Sometimes there are parts of the dashboard that can calculate annotations on the fly. In this case, say we want to record what color the histogram plot was last plotted with. It would be tedious and error prone for a reviewer to have to keep copying data into the correpsonding input. \n", "\n", @@ -2264,7 +2264,7 @@ "source": [ "**NOTE**: Some reviewers may have components that listen to each other, so if you keep one but not the other, some of the predefined interactivity may lead to error messages. You may run into similar errors for autofill buttons which may be pulling data from components that are excluded. \n", "\n", - "Also, the data needed to rendered excluded components will still be required when setting the review data (`reviewer.set_review_data(...)`)." + "Also, the data needed to render excluded components will still be required when setting the review data (`reviewer.set_review_data(...)`)." ] }, { diff --git a/tutorial_notebooks/Quick_Example_AnnoMate_Reviewer.ipynb b/tutorial_notebooks/Quick_Example_AnnoMate_Reviewer.ipynb new file mode 100644 index 0000000..29e645a --- /dev/null +++ b/tutorial_notebooks/Quick_Example_AnnoMate_Reviewer.ipynb @@ -0,0 +1,808 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "d2aedd2d-b1d5-4d2d-8e95-61ff06292438", + "metadata": { + "tags": [] + }, + "source": [ + "# Quick Example AnnoMate Reviewer\n", + "\n", + "This notebook demonstrates how to run an example reviewer built on AnnoMate.\n", + "\n", + "- If you want to know more details on what AnnoMate reviewers are and extensive documentation of its features, see `Intro_to_AnnoMate_Reviewers`\n", + "- If you want to know how to **create** your own custom reviewer, see `Developer_AnnoMate_Tutorial`" + ] + }, + { + "cell_type": "markdown", + "id": "670b8f2a-af59-4f15-8551-fa038ea8849e", + "metadata": { + "tags": [] + }, + "source": [ + "## Installation" + ] + }, + { + "cell_type": "markdown", + "id": "9e1078a8-0e56-4642-9b6f-3610ddf1377f", + "metadata": {}, + "source": [ + "Using pypi (recommended)\n", + "1. Create an environment: `conda create --name --file requirements.txt python==3.8`\n", + "1. Activate environment: `conda activate `\n", + "1. Install AnnoMate: `pip install AnnoMate`" + ] + }, + { + "cell_type": "markdown", + "id": "49d4fe42-17fe-4ea6-aaa7-85b7c0718788", + "metadata": {}, + "source": [ + "Using git:\n", + "1. Download the repository: `git clone git@github.com:getzlab/AnnoMate.git` \n", + "1. `cd AnnoMate`\n", + "1. Create an environment: `conda create --name --file requirements.txt python==3.8`\n", + "1. Activate environment: `conda activate `\n", + "1. Install package: `pip install -e .`" + ] + }, + { + "cell_type": "markdown", + "id": "e637c8ae-2ca2-487a-9522-de96383ae80f", + "metadata": {}, + "source": [ + "Running from docker container: You need to open at least 2 ports if you are using Mac or Windows:\n", + "1. A port to open jupyter lab (in this case, ``)\n", + "2. A port to open the dash app (``)\n", + "```\n", + "docker run -it -p : -p : gcr.io/broad-getzlab-workflows/annomate:dev_branch_v385\n", + "cd AnnoMate/tutorial_notebooks\n", + "jupyter lab --ip 0.0.0.0 --port --no-browser --allow-root\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "6429e242-eb91-4419-ba41-3b63e6f1ae6d", + "metadata": {}, + "source": [ + "## Running this notebook\n", + "\n", + "Make sure to update your kernel to the conda environment `` you just created.\n" + ] + }, + { + "cell_type": "markdown", + "id": "31fae173", + "metadata": {}, + "source": [ + "## Run an Example Reviewer\n", + "\n", + "Here we are setting up a basic reviewer that displays fake sample data." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "a9657940", + "metadata": {}, + "outputs": [], + "source": [ + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ef37cc47", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import os\n", + "from AnnoMate.Reviewers.ExampleReviewer import ExampleReviewer" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "b0aaeb3b-a736-41d5-9d3b-5cd00fa6d40d", + "metadata": {}, + "outputs": [], + "source": [ + "# get data\n", + "fn = 'example_data/AnnoMate_Tutorial/data_to_review_example.tsv'\n", + "df = pd.read_csv(fn, sep='\\t')\n", + "df = df.set_index('sample_id')" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "04a55062-3ae8-4ecb-b4eb-de3a65ba220e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Setting auto_export_path to ./example_reviewer_data/data.auto_export\n", + "Using ./example_reviewer_data/data.auto_export for auto exporting.\n", + "Dash app running on http://0.0.0.0:8056/\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataInterface.py:41: UserWarning:\n", + "\n", + "Loading existing data pkl file\n", + "\n" + ] + }, + { + "data": { + "application/javascript": [ + "window.open('http://0.0.0.0:8056/')" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# set up and run reviewer\n", + "my_reviewer = ExampleReviewer()\n", + "output_pkl_path = './example_reviewer_data'\n", + "\n", + "my_reviewer.set_review_data(data_path=output_pkl_path, \n", + " description='Example reviewer description',\n", + " sample_df=df,\n", + " preprocessing_str='Testing preprocessing')\n", + "\n", + "my_reviewer.set_review_app(mut_file_col='mutations_file', \n", + " sample_cols=['gender', 'age', 'tissue_origin'])\n", + "\n", + "my_reviewer.set_default_review_data_annotations_configuration()\n", + "my_reviewer.set_default_autofill()\n", + "my_reviewer.run(collapsable=True, port=8056, mode='tab')" + ] + }, + { + "cell_type": "markdown", + "id": "2a06283e-b1cb-452a-8d8d-46e2ebf538ea", + "metadata": {}, + "source": [ + "Accessing the Dash App:\n", + "> **If you are using a VM:** Make sure to open a ssh tunnel to the specified port number\n", + "> - `~C` to open the ssh prompt (will look like `ssh>`)\n", + "> - Type and enter `-L :localhost:`\n", + "\n", + "> **If you are running from a docker container:** Make sure to open the port for accessing the dash app (`reviewer.run(port=, ...)`) when you open the container\n", + "> - `docker run -it -p : -p : gcr.io/broad-getzlab-workflows/annomate:dev_branch_v385`\n", + "> - You can add more ports by adding more `-p :` arguments\n", + "> - If you are on Linux, you can replace `-p` arguments with `--network host`" + ] + }, + { + "cell_type": "markdown", + "id": "8476f28b-85f4-48b5-8a99-ccd6503dfa67", + "metadata": {}, + "source": [ + "### Use a table to navigate between samples rather than dropdown\n", + "\n", + "The table will also include the annotations in addition to the columns included in the dataframe passed to `review_data_table_df`. This table in the dashboard has native sorting and filtering, which can help the user find or prioritize samples to review." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "8158587c-270a-45f0-a769-52f6dbab95ae", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Setting auto_export_path to ./example_reviewer_data/data.auto_export\n", + "Using ./example_reviewer_data/data.auto_export for auto exporting.\n", + "Dash app running on http://0.0.0.0:8056/\n" + ] + }, + { + "data": { + "application/javascript": [ + "window.open('http://0.0.0.0:8056/')" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "my_reviewer.run(\n", + " review_data_table_df=df[['gender', 'age', 'tissue_origin']], \n", + " review_data_table_page_size=7,\n", + " port=8056,\n", + " mode='tab'\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "cc355478-4b2f-4dea-b8f4-f2743a3737d2", + "metadata": {}, + "source": [ + "### Disable test mode" + ] + }, + { + "cell_type": "markdown", + "id": "8da6a3c8-18eb-4382-9aab-4031be0ad167", + "metadata": {}, + "source": [ + "On initial load, you'll notice a `Freeze data` button. Once you are happy with your input data and are ready to start reviewing your data, press this button to disable test mode and freeze your data. Note that once this button is pressed, you will not be able to unfreeze your data / revert back to test mode." + ] + }, + { + "cell_type": "markdown", + "id": "a015ccb4", + "metadata": {}, + "source": [ + "### Retrieving annotations made in the dashbaord\n", + "After submitting annotations in the dashboard, you can view your progress by accessing the annotation table or history table:" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "8bff62a2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
NotesFlagHistogram colorA dropdown menuTest removing annotation
sample_0RemoveNaN
sample_1RemoveNaN
sample_2Nothing to sayKeepNaN
sample_3RemoveNaN
sample_4Not goodRemoveNaN
\n", + "
" + ], + "text/plain": [ + " Notes Flag Histogram color A dropdown menu \\\n", + "sample_0 Remove \n", + "sample_1 Remove \n", + "sample_2 Nothing to say Keep \n", + "sample_3 Remove \n", + "sample_4 Not good Remove \n", + "\n", + " Test removing annotation \n", + "sample_0 NaN \n", + "sample_1 NaN \n", + "sample_2 NaN \n", + "sample_3 NaN \n", + "sample_4 NaN " + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "my_reviewer.get_annot().head()" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "852bf6fa-9016-493a-89e9-cfa29e37d9c1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
indextimestampsource_data_fnNotesFlagHistogram colorA dropdown menuTest removing annotation
0sample_02024-05-29 11:21:53.051037./example_reviewer_data/data.pklRemoveNaNNaNNaN
0sample_12024-05-29 11:21:56.466574./example_reviewer_data/data.pklRemoveNaNNaNNaN
0sample_22024-05-29 11:22:02.213915./example_reviewer_data/data.pklNothing to sayKeepNaNNaNNaN
0sample_32024-05-29 11:22:05.466076./example_reviewer_data/data.pklRemoveNaNNaNNaN
0sample_42024-05-29 11:22:12.998607./example_reviewer_data/data.pklNot goodRemoveNaNNaNNaN
\n", + "
" + ], + "text/plain": [ + " index timestamp source_data_fn \\\n", + "0 sample_0 2024-05-29 11:21:53.051037 ./example_reviewer_data/data.pkl \n", + "0 sample_1 2024-05-29 11:21:56.466574 ./example_reviewer_data/data.pkl \n", + "0 sample_2 2024-05-29 11:22:02.213915 ./example_reviewer_data/data.pkl \n", + "0 sample_3 2024-05-29 11:22:05.466076 ./example_reviewer_data/data.pkl \n", + "0 sample_4 2024-05-29 11:22:12.998607 ./example_reviewer_data/data.pkl \n", + "\n", + " Notes Flag Histogram color A dropdown menu \\\n", + "0 Remove NaN NaN \n", + "0 Remove NaN NaN \n", + "0 Nothing to say Keep NaN NaN \n", + "0 Remove NaN NaN \n", + "0 Not good Remove NaN NaN \n", + "\n", + " Test removing annotation \n", + "0 NaN \n", + "0 NaN \n", + "0 NaN \n", + "0 NaN \n", + "0 NaN " + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "my_reviewer.get_history()" + ] + }, + { + "cell_type": "markdown", + "id": "7fea16ef", + "metadata": {}, + "source": [ + "Now you can export this table to a file that you can then share, upload to Terra, or use for further analysis." + ] + }, + { + "cell_type": "markdown", + "id": "defbae62-fe6d-4e0d-b0a3-7f3ac522fc23", + "metadata": {}, + "source": [ + "## Make modifications to the Dashboard in the notebook" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "34d7128c-36bd-4da3-92c0-8962770dc0a9", + "metadata": {}, + "outputs": [], + "source": [ + "from AnnoMate.Data import DataAnnotation\n", + "import AnnoMate.AnnotationDisplayComponent as adc\n", + "import numpy as np" + ] + }, + { + "cell_type": "markdown", + "id": "ada89ed7-0ffd-4fe1-993d-ddc9f5e2b2b8", + "metadata": {}, + "source": [ + "### Add another table component\n", + "\n", + "AnnoMate includes a helper function to quickly add in a table using a tsv file provided for each sample. In this case, we display the file in column `Treatmet file` in `df` corresponding to the selected sample for review.\n", + "\n", + "See `Developer_AnnoMate_Tutorial` for more details on how to make a fully custom interactive component." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "9fd80758", + "metadata": {}, + "outputs": [], + "source": [ + "my_reviewer.app.add_table_from_path(\n", + " data_table_source='df', # reference which table in the Data object to use.\n", + " table_title='Treatment file',\n", + " component_id='treatment-component-id',\n", + " table_fn_col='treatments_file', \n", + " table_cols=['treatment_name', 'response', 'tx_start']\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4dfb58a3-b5a8-4dff-a5a8-1a4ae4ff5184", + "metadata": {}, + "source": [ + "### Update existing annotation options and display\n", + "\n", + "By default, the `ExampleReviewer` `Flag` annotation had only two options: Keep and Remove. Here, we update the viable options for annotations, and set a default value." + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "53cb2bf6-a874-425c-bc6c-0afff25a05a1", + "metadata": {}, + "outputs": [], + "source": [ + "my_reviewer.add_review_data_annotation(\n", + " annot_name='Flag', \n", + " review_data_annotation=DataAnnotation(\n", + " annot_value_type='string', \n", + " options=['Keep', 'Remove', 'Report'], \n", + " )\n", + ")\n", + "my_reviewer.add_annotation_display_component(\n", + " annot_name='Flag', \n", + " annot_display_component=adc.RadioitemAnnotationDisplay(default_display_value='Keep')\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0d62cad4-2508-4528-a456-f0c45e83d05f", + "metadata": {}, + "source": [ + "### Add more annotations\n", + "\n", + "Here we add a dropdown menu to track another annotation that is not included in the pre-built ExampleReviewer class" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "85edf720-e28f-40c2-8ef2-4ee0917d065d", + "metadata": {}, + "outputs": [], + "source": [ + "# Dropdown/Select\n", + "my_reviewer.add_review_data_annotation(\n", + " annot_name='A dropdown menu', \n", + " review_data_annotation=DataAnnotation(\n", + " annot_value_type='string', \n", + " options=['Option A', 'Option B', 'Option C']\n", + " )\n", + ")\n", + "my_reviewer.add_annotation_display_component(\n", + " annot_name='A dropdown menu', \n", + " annot_display_component=adc.SelectAnnotationDisplay())" + ] + }, + { + "cell_type": "markdown", + "id": "936bd35f-0920-4c71-9b2e-31db813541aa", + "metadata": {}, + "source": [ + "### Remove annotation from display\n", + "\n", + "Note this will only remove the annotation input from the dashboard. It will not delete the column in the annotation table (`reviewer.get_annot()`)." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "f5d59a4c-e5c5-482b-bcc3-544b909a4fa6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# adding an annotation\n", + "my_reviewer.add_review_data_annotation(\n", + " annot_name='Test removing annotation', \n", + " review_data_annotation=DataAnnotation(\n", + " annot_value_type='float', \n", + " )\n", + ")\n", + "my_reviewer.add_annotation_display_component(\n", + " annot_name='Test removing annotation', \n", + " annot_display_component=adc.NumberAnnotationDisplay())\n", + "\n", + "'Test removing annotation' in my_reviewer.annot_app_display_types_dict.keys()" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "95ff8056-3b62-489e-8bf6-8809bcdfedeb", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# removing an annotation from display\n", + "removed_annot_display_component = my_reviewer.remove_annotation_display_component('Test removing annotation')\n", + "'Test removing annotation' in my_reviewer.annot_app_display_types_dict.keys()" + ] + }, + { + "cell_type": "markdown", + "id": "10700832-b886-4568-a0ae-d582e367d0a5", + "metadata": {}, + "source": [ + "### Add annotation autofill button\n", + "\n", + "To expedite recording annotations, the user can create a button that will fill the annotation inputs. Here, when the button `Mut vafs` is clicked, the `Histogram color` annotation will be filled with whatever the current state of the radioitem `mut-figure-color-radioitem` is.\n", + "\n", + "Users can also prefill with a fixed value (i.e. a string, number)" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "eff8b6fe-f965-4a78-8097-6aa19fc3a2cc", + "metadata": {}, + "outputs": [], + "source": [ + "from dash.dependencies import State\n", + "my_reviewer.add_autofill(\n", + " autofill_button_name='Mut vafs',\n", + " fill_value=State('mut-figure-color-radioitem', 'value'),\n", + " annot_name='Histogram color'\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e5ebf97b-fb62-4768-bbb3-af4516ab5c16", + "metadata": {}, + "source": [ + "### Change which components and the order the components are displayed" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "ad199e40-f8ef-4528-b6f3-f2335d9c9a06", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "odict_keys(['sample-info-component', 'Maf file', 'Mut vafs', 'Treatment file'])" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# View available components\n", + "my_reviewer.app.more_components.keys()" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "id": "71659b78-7085-4e07-8fca-85efbc9bfc31", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Setting auto_export_path to ./example_reviewer_data/data.auto_export\n", + "Using ./example_reviewer_data/data.auto_export for auto exporting.\n", + "Dash app running on http://0.0.0.0:8056/\n" + ] + } + ], + "source": [ + "my_reviewer.run(\n", + " port=8056,\n", + " components_name_order=['sample-info-component', 'Treatment file', 'Mut vafs']\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "fe4762ca-2cb4-41f2-a486-74d4cfd4c08e", + "metadata": {}, + "source": [ + "**NOTE**: Some reviewers may have components that listen to each other, so if you keep one but not the other, some of the predefined interactivity may lead to error messages. You may run into similar errors for autofill buttons which may be pulling data from components that are excluded. \n", + "\n", + "Also, the data needed to render excluded components will still be required when setting the review data (`reviewer.set_review_data(...)`)." + ] + }, + { + "cell_type": "markdown", + "id": "e07c9eca", + "metadata": {}, + "source": [ + "# Final words\n", + "\n", + "You can always go back and change the parameters of setting the app and annotation configurations, adding or editing the components of the app, etc.\n", + "\n", + "The one thing that will NOT change after disabling test mode is `set_review_data()`. This review data can only be frozen once per `data_path`. You may re-set your review data as much as you'd like in test mode, but once you click the 'Freeze data' button, disabling test mode, no more changes to the input data can be made (annotaions can still be made and components can still be added). If you wish to start over, you must manually delete the data path. Else, reference a new path.\n", + "\n", + "This feature allows you to make these changes to the app on the fly, and you can restart the notebook and it will not change the annotations you have already made.\n", + "\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python [conda env:test]", + "language": "python", + "name": "conda-env-test-py" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}