Section 8: ValueError: logits and labels must have the same shape, received ((None, 15, 1) vs (None,)).

[Raunak-Singh-Inventor]

In Section 8: Introduction to NLP Fundamentals in TensorFlow I am running Model 1: Feed-forward neural network (dense model) and getting

ValueError: `logits` and `labels` must have the same shape, received ((None, 15, 1) vs (None,)).

Original code:

# Build model with the Functional API 
from tensorflow.keras import layers
inputs = layers.Input(shape=(1,), dtype="string") # inputs are 1-dimensional strings
x = text_vectorizer(inputs) # turn the input text into numbers
x = embedding(x) # create an embedding of the numberized inputs
outputs = layers.Dense(1, activation="sigmoid")(x) # Create the output layer, want binary outputs so use sigmoid activation functions
model_1 = tf.keras.Model(inputs, outputs, name="model_1_dense")

# Compile the model
model_1.compile(loss="binary_crossentropy",
                optimizer=tf.keras.optimizers.Adam(),
                metrics=["accuracy"])

# Fit the model
model_1_history = model_1.fit(x=train_sentences,
                              y=train_labels,
                              epochs=5,
                              validation_data=(val_sentences, val_labels),
                              callbacks=[create_tensorboard_callback(dir_name=SAVE_DIR,
                                                                     experiment_name="model_1_dense")])

I found the fix for this was to use this code to build the model:

# Build model with the Functional API 
from tensorflow.keras import layers
inputs = layers.Input(shape=(1,), dtype="string") # inputs are 1-dimensional strings
x = text_vectorizer(inputs) # turn the input text into numbers
x = embedding(x) # create an embedding of the numberized inputs
outputs = layers.Dense(1, activation="sigmoid")(x) # Create the output layer, want binary outputs so use sigmoid activation functions
outputs = tf.squeeze(outputs, axis=-1) # the FIX
model_1 = tf.keras.Model(inputs, outputs, name="model_1_dense")

or use this code as mentioned in the offical course github notebook https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/video_notebooks/08_introduction_to_nlp_in_tensorflow_video.ipynb:

# Build model with the Functional API 
from tensorflow.keras import layers
inputs = layers.Input(shape=(1,), dtype="string") # inputs are 1-dimensional strings
x = text_vectorizer(inputs) # turn the input text into numbers
x = embedding(x) # create an embedding of the numberized inputs
x = layers.GlobalAveragePooling1D()(x)
outputs = layers.Dense(1, activation="sigmoid")(x) # Create the output layer, want binary outputs so use sigmoid activation functions
model_1 = tf.keras.Model(inputs, outputs, name="model_1_dense")

My question is: How in the video does @mrdbourke not get this error when training the model without a GlobalAveragePooling1D layer or a tf.squeeze operation?

[mrdbourke]

Hey @Raunak-Singh-Inventor,

If there are minor errors in the video that can't be explained, the course notebooks are the best place to reference (like what you've done).

The videos will try to have notes/be updated where possible if something happens like what you found but due to slight changes in the code it can't always guarantee to best the exact right answer.

However, the notebooks will be kept up to date & if something major changes, the videos will be noted/updated too.