「OSPP - KubeEdge SIG AI」Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs #82
Conversation
qxygxt
changed the title
|
@MooreZheng @Frank-lilinjie |
|
|
||
| **Semantic Segmentation** is an image segmentation technique used to assign each pixel in an image to a specific semantic category, which can be widely applied in various fields, such as scene understanding, obstacle detection, and so on. The key challenge of this task is to consider both pixel-level details and global semantic information to accurately segment the image. | ||
|
|
||
| In recent years, developers often reuse training data multiple times to improve the model's performance. In practical applications, previous training data is often no longer available due to privacy concerns or other reasons. In such cases, if a machine learning model continues to learn new tasks, it often leads to **catastrophic forgetting**. Therefore, we hope the model has the characteristic of **lifelong learning**, where it can learn new knowledge while retaining old knowledge. The same is true of semantic segmentation tasks. The reason why semantic segmentation requires lifelong learning is that during the lifetime of a semantic segmentation system, it needs to process images of different kinds, different scenes, and different datasets, and there may be significant differences between them. Therefore, if the system is unable to engage in lifelong learning, there will be the problem of catastrophic forgetting and poor performance. |
There was a problem hiding this comment.
A good revision from the previous one. Would be improved by indicating the problem to be solved. E.g., some pictures for example would be preferred.
|
How is this solution related to ianvs lifelong learning libs? It seems to me that it runs in the unseen task processing, but it should be clearly indicated in the lifelong learning process. |
|
An architecture is needed in the proposal, showing how the solution works with ianvs modules. |
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_1.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_2.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_3.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_4.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_5.png Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_6.png Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_7.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_8.png Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]>
…ation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
| The following is the architecture diagram of this project system, and this project focuses on the `unknown task processing` module. | ||
|
|
||
| Before entering this module, unknown tasks have been [detected](https://github.com/kubeedge/ianvs/tree/4ae10f0e5e1ab958e143b04fade4acc448009857/examples/scene-based-unknown-task-recognition/lifelong_learning_bench) and samples have been labeled by some means such as manual labeling. The core concern of this module is how to use unknown task samples (i.e., incremental class samples) to update the model. | ||
|  |
There was a problem hiding this comment.
What is the meaning of the dotted line box in the figure? It is best to explain it in the figure. Secondly, why do you need to upload the reasoning results of unseen tasks to the cloud instead of directly uploading unseen samples to the cloud.
There was a problem hiding this comment.
For the first suggestion, it has been clarified in the figure.
There was a problem hiding this comment.
Ok, as for the second problem, I can see that the module 5-1 identifies unseen tasks and transmits data to the cloud for processing
There was a problem hiding this comment.
To clarify the second problem: in the framework of this project, unseen samples are detected at the edge, uploaded to the cloud for labeling, and eventually used for unseen task processing (project-focused part). The reason for this design is that in many cases, the computing and storage capacity of the edge is not enough to support model training, so unseen samples need to be uploaded to the cloud for labeling and training. The cloud is significantly better than the edge in terms of computing performance, and it can label and process unknown samples more quickly and efficiently.
There was a problem hiding this comment.
Thank you for your explanation. I understand.
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_1.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_2.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_3.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_4.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_5.png Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_6.png Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_7.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_8.png Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]> modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md Signed-off-by: qxygxt <[email protected]> Update Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md deleted: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_6.png new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_9.png
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md modified: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_9.png
|
The current version of proposal has been revised according to the comments of the reviewers (including @MooreZheng @tangming1996 @jaypume ), and this PR fixes issue #79 . |
There was a problem hiding this comment.
As mentioned in the routine meeting, we see a much better version. We also understand that the key module lies in the unseen task processing module. Thank you so much for the revision.
Nevertheless, the project scope and module design are not yet very clear to most community members. It is thus suggested to include the module designs, i.e., how would other modules be implemented.
| | ---- | :-------: | ------ | ----- | ----- | ---------------- | ----- | --------- | --------- | ------------- | ----- | ---------- | ------------------- | | ||
| | 1 | MDIL-SS | 0.6521 | 0.075 | 0.021 | Lifelonglearning | 3 | CS SYN CR | ERFNet | 0.0001 | 1 | 10 | 2023-05-28 17:05:15 | | ||
|
|
||
| ## 4 Roadmap |
There was a problem hiding this comment.
Before getting into advanved algorithm development, it is suggested to ensure all modules work with a simple version first.
|
|
||
| Before entering this module, unknown tasks have been [detected](https://github.com/kubeedge/ianvs/tree/4ae10f0e5e1ab958e143b04fade4acc448009857/examples/scene-based-unknown-task-recognition/lifelong_learning_bench) and samples have been labeled by some means such as manual labeling. The core concern of this module is how to use unknown task samples (i.e., incremental class samples) to update the model. | ||
|
|
||
|  |
There was a problem hiding this comment.
As mentioned in the routine meeting, we see a much better version. We also understand that the key module lies in the unseen task processing module. Thank you so much for the revision.
Nevertheless, the project scope and module design are not yet very clear to most community members. For example,
- We see designs in the training phase. Will this project be related to an end-to-end design, e.g., including any advanced technique in the inference phase?
- We see designs on unseen task processing. Will this project include any advanced techniques for unseen task recognition?
- As clarified in the routine meeting, the unseen task processing will be integrated into the ianvs lifelong learning as a whole. How do other modules implement to support the unseen task processing module?
It is thus suggested to include the module designs as solution details. I also believe that simple implementations will work, but the implementation still needs to be carefully considered, in order to ensure the input and output of the key module.
|
|
||
| The following diagram shows how the algorithm works in Ianvs. | ||
|
|
||
|  |
There was a problem hiding this comment.
Thank you so much for the revision and it looks much better. Nevertheless, similar to the above design, the project scope and module design are also not yet very clear to most community members in the figure. It is thus suggested to include the module designs, i.e., how would other modules be implemented.
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_10.png
…ng Algorithm Evaluation System based on Ianvs
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_10.png
Signed-off-by: qxygxt <[email protected]>
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
new file: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_11.png
Signed-off-by: qxygxt <[email protected]>
modified: docs/proposals/algorithms/lifelong-learning/Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md modified: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_11.png modified: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_8.png
qxygxt
changed the title
qxygxt
changed the title
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md modified: docs/proposals/algorithms/lifelong-learning/images/OSPP_MDIL-SS_11.png
… of a Class Incremental Learning Algorithm Evaluation System based on Ianvs.md
…arning_bench/README.md
…arning_bench/README.md modified: examples/class_increment_semantic_segmentation/lifelong_learning_bench/README.md Signed-off-by: qxygxt [email protected]
[OSPP-Proposal] Implementation of a Class Incremental Learning Algorithm Evaluation System based on Ianvs