My final model consisted of the following layers:
| Layer | Description |
|---|---|
| Input | 32x32x1 grayscale image |
| Convolution 5x5 | 2x2 stride, valid padding, outputs 28x28x6 |
| RELU | |
| Max pooling | 2x2 stride, outputs 14x14x6 |
| Convolution 5x5 | 2x2 stride, valid padding, outputs 10x10x16 |
| RELU | |
| Max pooling | 2x2 stride, outputs 5x5x16 |
| Convolution 1x1 | 2x2 stride, valid padding, outputs 1x1x412 |
| RELU | |
| Fully connected | input 412, output 122 |
| RELU | |
| Dropout | 50% keep |
| Fully connected | input 122, output 84 |
| RELU | |
| Dropout | 50% keep |
| Fully connected | input 84, output 43 |