Skip to content

Commit

Permalink
Disable plots (stanford-crfm#1778)
Browse files Browse the repository at this point in the history
  • Loading branch information
yifanmai authored and danielz02 committed Sep 7, 2023
1 parent 0f6a120 commit a556927
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ This reads the pre-existing files in `benchmark_output/runs/v1/` that were writt

Additionally, for each group and group-relavent metric, it will output a pair of files: `benchmark_output/runs/v1/groups/latex/<group_name>_<metric_name>.tex` and `benchmark_output/runs/v1/groups/latex/<group_name>_<metric_name>.json`. These files contain the statistics for that metric from each run within the group.

<!--
# TODO(#1441): Enable plots
## Using `helm-create-plots`
The `helm-create-plots` reads the `groups` directory created by `helm-summarize` and creates plots, equivalent to those use in the HELM paper. Run the following:
Expand All @@ -76,6 +79,8 @@ helm-create-plots --suite v1
This reads the pre-existing files in `benchmark_output/runs/v1/groups` that were written by `helm-summarize` previously,
and creates plots (`.png` or `.pdf`) at `benchmark_output/runs/v1/plots`.
-->

## Using `helm-server`

Finally, the `helm-server` command launches a web server to visualize the output files of `helm-run` and `helm-benchmark`. Run:
Expand Down
3 changes: 3 additions & 0 deletions src/helm/benchmark/static/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@
<li class="nav-item"><a class="nav-link active" href="?models=1">Models</a></li>
<li class="nav-item"><a class="nav-link active" href="?scenarios=1">Scenarios</a></li>
<li class="nav-item"><a class="nav-link active" href="?groups=1">Results</a></li>
<!--
TODO(#1441): Enable plots.
<li class="nav-item"><a class="nav-link active" href="?plots=1">Plots</a></li>
-->
<li class="nav-item"><a class="nav-link active" href="?runs=1">Raw runs</a></li>
</ul>
</div>
Expand Down

0 comments on commit a556927

Please sign in to comment.