Port GPT Text Generation Guide to Keras Core #1490
Conversation
|
This PR is WIP
|
|
Could you please change it to draft till you make all the changes. Thanks |
|
This PR is ready for review, could you @fchollet ? |
| perplexity = keras_nlp.metrics.Perplexity(from_logits=True, mask_token_id=0) | ||
| model.compile(optimizer="adam", loss=loss_fn, metrics=[perplexity]) | ||
|
|
||
| """ | ||
| Let's take a look at our model summary - a large majority of the |
There was a problem hiding this comment.
Why did we remove this section?
There was a problem hiding this comment.
When I run a generation script, it simply fails saying that console width is not enough to print summary.
There was a problem hiding this comment.
Hmm, probably not a reason to cut this section. Do you have the error handy?
| @@ -264,19 +252,31 @@ model.fit(train_ds, validation_data=val_ds, verbose=2, epochs=EPOCHS) | |||
| <div class="k-default-codeblock"> | |||
| ``` | |||
| Epoch 1/6 | |||
| 3169/3169 - 132s - loss: 4.5592 - perplexity: 95.8829 - val_loss: 4.1382 - val_perplexity: 63.2792 - 132s/epoch - 42ms/step | |||
|
|
|||
| /usr/local/lib/python3.10/dist-packages/keras_core/src/layers/layer.py:759: UserWarning: Layer 'position_embedding' (of type PositionEmbedding) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask. | |||
There was a problem hiding this comment.
we might want to take a look at these warning, are they valid?
|
This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you. |
|
This PR was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further. |
Closes keras-team/keras-hub#1147
cc: @mattdangerw