Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bugs] Fix attn mask #852

Merged
merged 5 commits into from
Jul 19, 2024
Merged

[Bugs] Fix attn mask #852

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ce9bb62
[WIP]: Fix sequence parallel memory bottleneck in DPO
RangiLyu Jul 11, 2024
8753d9a
loss mask before split
RangiLyu Jul 11, 2024
1f58ee7
refactor orpo
RangiLyu Jul 12, 2024
03712bf
fix attention_mask in preference_collate_fn
HIT-cwh Jul 19, 2024
f3813ba
Merge branch 'main' into fix_attn_mask
HIT-cwh Jul 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions xtuner/dataset/collate_fns/preference_collate_fn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def preference_collate_fn(instances: Sequence[Dict],
labels = torch.stack(labels)

if use_varlen_attn:
attention_mask = None
attention_mask = torch.ones_like(input_ids).bool()
position_ids = torch.stack(position_ids, dim=0)
else:
# Some tokenizers have the same eos token and pad token, so input_ids
Expand All @@ -74,8 +74,10 @@ def preference_collate_fn(instances: Sequence[Dict],
input_ids = pad_for_sequence_parallel(input_ids, pad_index)
labels = pad_for_sequence_parallel(labels, IGNORE_INDEX)
position_ids = pad_for_sequence_parallel(position_ids, 0)
if attention_mask is not None:
attention_mask = pad_for_sequence_parallel(attention_mask, 0)
# We use attention_mask to distinguish `input_ids` from
# (sequence parallel) pad tokens in `get_var_len_atten_logps` method of
# class `DPO` and `ORPO`
attention_mask = pad_for_sequence_parallel(attention_mask, 0)
if use_varlen_attn:
(cumulative_len, attention_mask
) = pad_cumulative_len_for_sequence_parallel(cumulative_len)
Expand Down
Loading