Design for Base Deep class

[AurumnPegasus]

Any other comments?

I have proposed 2 solutions for the structure of Deep Learning Estimators (personally, I prefer the second).
I would like any suggestions/changes I can make to the designs, and would love to hear feedback.

[fkiraly]

Change requests:

• I think we need a very careful write-up of the conceptual model. Which abstractions/objects are there? Which abstractions are represented by which base classes? What are (formal) examples?
• one related specific point: previously, I think the BaseDeepNetwork descendants always contained uncompiled networks? Whereas, in your solution 1, it seems to be a compiled network now. What is the generic signature of build_model?
• I would also follow the "3 examples rule" in design. If you define a base class or strategy pattern, list 3 examples. This also applies to examples of, say, composition.

[AurumnPegasus]

Hey,

1. I do not exactly understand your point, but I will add some examples which might help explain mine. I will do so in another commit
2. Yes, in my solution 1, BaseDeepNetwork will now return a compiled network. I will specify, in detail, in the next commit

[AurumnPegasus]

hey @fkiraly ,
I have added detailed implementation of how the classes and design will look for both of the proposed solutions. I have also written a small pro/con list for both of them in the same folder.
From a design perspective, I hope that would make things a lot clearer accompanied by the original document.
If you have any doubts or feedback, let me know.

[AurumnPegasus]

@fkiraly I have updated the doc to make it more understandable.
Let me know the feedback or any detail that needs to be added/clarified!

[TonyBagnall]

hi @AurumnPegasus great stuff, but I think if we are going for a full design rather than a quick port (which was my preference tbh, sktime-dl is simply ported from https://github.com/hfawaz/dl-4-tsc), we might want to consider pytorch too. I have never used pytorch myself, but it is probably more popular now in my areas of research. pytorch is, I believe, structured quite differently. Do you have any experience with it?

[AurumnPegasus]

Hey @TonyBagnall
Yes, I am more comfortable with pytorch since I have been generally using it for my courses/research areas. Generally, there is nothing you cant do in keras which you can do in pytorch (and vice versa), but the main difference occurs in the structure. Keras generally allows you to use layers and models as abstractions, whereas Pytorch requires you to write the class/network you want to use (hence allows greater customization) and train loops as you want.

There might be minor differences which will be there in BaseDeepNetwork if we want to enable both pytorch and keras, since we will have to write the models differently, but the application can be abstracted with minimal changes in class structure (atleast from what I understand now, maybe once we go more in depth with the problem we might face new issues)

[TonyBagnall]

it would be awesome to have a structure that allowed both pytorch and keras based estimators. Id like to keep the current implementations as they reproduce Hassan's results, but ideally I'd like researchers to use sktime to design new estimators, and pytorch functionality would I think greatly help that.

[fkiraly]

I would suggest, let's not overcomplicate the design question by having multiple backends.

We already have multiple estimator types, that is imo the challenge we first should address.

[GuzalBulatova]

Hi @AurumnPegasus, thanks for the solutions proposed here, good job 👍
To me the second one also makes more sense, and BaseDeepNetwork -> BaseDeepEstimator -> ... goes in parallel with the existing design of BaseObject -> BaseEstimator -> ...
One thing that I'm curious about though is would implementing BaseDeepEstimator multiply entities? I assume that within sktime's scope we wouldn't need the BaseDeepNetwork to be anything other than an estimator. Would other classes inherit from BaseDN?

[AurumnPegasus]

hey @GuzalBulatova
I think you have understood the idea about BaseDeepEstimator a bit wrong (or maybe I have misunderstood what you are saying)
I am proposing that BaseEstimator -> BaseDeepEstimator -> BaseDeepClassifier -> CNNClassifier
the BaseDeepNetwork class remains mostly unchanged, since it is required for the specific networks (not the specific estimators). So the BaseDeepNetwork is the parent of CNNNetwork, and the CNNClassifier creates an object of CNNNetwork to use the specific keras network

So in this case BaseDeepEstimator does not really multiply entities from my understanding, although I am not very sure what you mean by entities here

[GuzalBulatova]

Hi @AurumnPegasus I see, I got it wrong previous time. I thought you propose BaseDeepNetwork -> BaseDeepEstimator(BaseDeepNetwork, BaseEstimator) -> BaseDeepClassifier(BaseDeepEstimator) -> anydeepclassifier. And in this case I was wondering if it made sense to have a BaseDeepNetwork as a separate class.

I think I understand now what you meant. It makes sense, I would go ahead with the second option.