This repository provides the official implementation for the following paper:
Colorizing line art is a pivotal task in the production of hand-drawn cel animation. In this work, we introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments. To facilitate the training of our network, we also propose a unique dataset PaintBucket-Character. This dataset includes rendered line arts alongside their colorized counterparts, featuring various 3D characters.
- 2024.03.30: The checkpoint and training code of our BasicPBC are released.
- 2024.03.29: This repo is created.
- Add trapped-ball segmentation module for unclosed line arts inference.
- Add a lightweight BasicPBC which can process images in 2K resolution without encountering Out-Of-Memory (OOM) error on 8GB RAM GPU.
-
Clone the repo
git clone https://github.com/ykdai/BasicPBC.git
-
Install dependent packages
cd BasicPBC pip install -r requirements.txt
-
Install BasicPBC Please run the following commands in the BasicPBC root path to install BasicPBC:
python setup.py develop
The details of our dataset can be found at this page. Dataset can be downloaded using the following links.
Google Drive | Baidu Netdisk | Number | Description | |
---|---|---|---|---|
PaintBucket-Character Train/Test | link | link | 11,345/3,000 | 3D rendered frames for training and testing. Our dataset is a mere 2GB in size, so feel free to download it and enjoy exploring. 😆😆 |
PaintBucket-Real Test | / | / | 200 | Hand-drawn frames for testing. |
Due to copyright issues, we do not provide download links for the real hand-drawn dataset. Please contact us through the e-mail if you want to use it. These hand-drawn frames are only for evaluation and not for any commercial activities.
You can download the pretrained checkpoints from the following links. Please place it under the ckpt
folder and unzip it, then you can run the basicsr/test.py
for inference.
Google Drive | Baidu Netdisk |
---|---|
link | link |
We also have plans to release a lightweight model, please stay tuned!🤗
To estimate the colorized frames with our checkpoint trained on PaintBucket-Character, you can run the basicsr/test.py
by using:
python basicsr/test.py -opt options/test/basicpbc_pbch_test_option.yml
The colorized results will be saved at results/
.
To inference on your own data, put your animation clips under dataset/test/your_data/
. The clip folder should contain the colorized gt
of the 1st frame and line
of all frames.
├── dataset
├── test
├── your_data
├── anime_clip1
├── gt
├── 0000.png
├── line
├── 0000.png
├── 0001.png
├── ...
Run the inference_line_frames.py
by using:
python inference_line_frames.py --folder_path dataset/test/your_data/
Find results under results/your_data/
.
Training with single GPU
To train a model with your own data/model, you can edit the options/train/basicpbc_pbch_train_option.yml
and run the following codes.
python basicsr/train.py -opt options/train/basicpbc_pbch_train_option.yml
Training with multiple GPU
You can run the following command for multiple GPU training:
CUDA_VISIBLE_DEVICES=0,1 bash scripts/dist_train.sh 2 options/train/basicpbc_pbch_train_option.yml
├── BasicPBC
├── assets
├── basicsr
├── archs
├── data
├── losses
├── metrics
├── models
├── ops
├── utils
├── dataset
├── train
├── PaintBucket_Char
├── test
├── PaintBucket_Char
├── PaintBucket_Real
├── experiments
├── options
├── test
├── train
├── paint
├── raft
├── results
├── scripts
This project is licensed under S-Lab License 1.0. Redistribution and use of the dataset and code for non-commercial purposes should follow this license.
If you find this work useful, please cite:
@article{InclusionMatching2024,
title = {Learning Inclusion Matching for Animation Paint Bucket Colorization},
author = {Dai, Yuekun and Zhou, Shangchen and Li, Qinyue and Li, Chongyi and Loy, Chen Change},
journal = {CVPR},
year = {2024},
}
If you have any question, please feel free to reach me out at [email protected]
.