Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

English-to-Spanish translation with a sequence-to-sequence Transformer: TransformerDecoder second attention_mask maybe wrong #1964

Open
xwklwwlf opened this issue Oct 18, 2024 · 0 comments
Assignees

Comments

@xwklwwlf
Copy link

Issue Type

Bug

Source

source

Keras Version

keras 2.15

Custom Code

Yes

OS Platform and Distribution

mac os

Python version

3.10

GPU model and memory

No response

Current Behavior?

https://github.com/keras-team/keras-io/blob/master/examples/nlp/neural_machine_translation_with_transformer.py
line 362 attention_mask=padding_mask,
if the sequence_length of encoder_inputs and decoder_inputs is different, the code will report an error.
I think attention_mask should be set to padding_mask of encoder_inputs, not padding_mask of decoder_inputs.

Standalone code to reproduce the issue or tutorial link

https://keras.io/examples/nlp/neural_machine_translation_with_transformer/
https://github.com/keras-team/keras-io/blob/master/examples/nlp/neural_machine_translation_with_transformer.py

Relevant log output

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants