Skip to content

Commit

Permalink
make model.model_metrics.qps_metric.warmup_steps can be passed throug…
Browse files Browse the repository at this point in the history
…h config (pytorch#2255)

Summary:
Pull Request resolved: pytorch#2255

`ThroughputMetric` already support warm_steps, however it's default to always 100.
Thsi diff is to make qps_metric.warmup_steps configurable so that it can be control through model yaml file or command argument, for example,

```
buck2 run mode/{opt,amd-gpu} //aps_models/ads/icvr:icvr_launcher -- mode=local_ctr_cvr_cmf_rep_1000x_v1_no_atom +model.model_metrics.qps_metric.warmup_steps=1
```

Reviewed By: Yuzhen11

Differential Revision: D60388121

fbshipit-source-id: 7963e8e4f87bde651cd4a39c9b68310ef408acb6
  • Loading branch information
joebos authored and facebook-github-bot committed Jul 30, 2024
1 parent 5ddb831 commit 2771a90
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 0 deletions.
1 change: 1 addition & 0 deletions torchrec/metrics/metric_module.py
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,7 @@ def generate_metric_module(
batch_size=batch_size,
world_size=world_size,
window_seconds=metrics_config.throughput_metric.window_size,
warmup_steps=metrics_config.throughput_metric.warmup_steps,
)
else:
throughput_metric = None
Expand Down
2 changes: 2 additions & 0 deletions torchrec/metrics/metrics_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ class RecComputeMode(Enum):

_DEFAULT_WINDOW_SIZE = 10_000_000
_DEFAULT_THROUGHPUT_WINDOW_SECONDS = 100
_DEFAULT_THROUGHPUT_WARMUP_STEPS = 100


@dataclass
Expand Down Expand Up @@ -113,6 +114,7 @@ class StateMetricEnum(StrValueMixin, Enum):
@dataclass
class ThroughputDef:
window_size: int = _DEFAULT_THROUGHPUT_WINDOW_SECONDS
warmup_steps: int = _DEFAULT_THROUGHPUT_WARMUP_STEPS


@dataclass
Expand Down

0 comments on commit 2771a90

Please sign in to comment.