Draft: Skip models in model_builder

[vallsv]

Hi,

Following #908 i have created this proof of concept.

It works, but i have no idea if that's something useful or not.

Let me know what you think and i can improve the implementation if you like.

• During the creation of a new model model_build(hp) is called (that's normal behaviour)
• hp provides a new function hp.skip_model(message) that can be called if the user decide that the parameters
• This raises an exception, which is converted into a status SKIPPED
• The SKIPPED status is not counted as an INVALID model, so the model is skipped silently

It looks to work fine, but there is a lake. At start, model_build is called only for a a "discovery" phase. At this time the user code can call hp.skip_model(), which interrupt unexpectedly the processing. To fix that we just could flag hp with a dicovery_phase during this phase, so skip_model could just dry run.

Exemple

def model_builder(hp):
    discovery_stage = len(hp._hps) == 0  # work around to not call `skip_model` at start

    model = models.Sequential()

    # -------------------------------------

    c1_kernel_size = hp.Choice(f'c1_kernel', values=[1, 2, 3])
    c2_kernel_size = hp.Choice(f'c2_kernel', values=[1, 2, 3, 5])
    c3_kernel_size = hp.Choice(f'c3_kernel', values=[1, 2, 3])
    c4_kernel_size = hp.Choice(f'c4_kernel', values=[1, 2, 3])
    c5_kernel_size = hp.Choice(f'c5_kernel', values=[1, 2, 3])

    # -------------------------------------

    ...

    try:
        model.add(
            layers.Conv2D(
                name="c4",
                filters=c4_filters,
                kernel_size=c4_kernel_size,
                # dilation_rate=(1, 64),
                activation='relu',
                padding='VALID',
            )
        )
        model.add(
            layers.MaxPooling2D(),
        )
    except ValueError as e:
        # Skip the model if the combinations of kernel sizes / strides doesn't fit the input size
        if not discovery_stage:
            hp.skip_model(str(e))

    ...

    # -------------------------------------

    # From https://stackoverflow.com/questions/43137288/how-to-determine-needed-memory-of-keras-model
    gbytes = get_model_memory_usage(model, BATCH_SIZE)

    if not discovery_stage:
        if gbytes > 40:
            hp.skip_model(f"Model too big: {gbytes}GiB")

    return model

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.