Decay: An Organic Degradation

Before reading: Please refer to following paper for more information about CycleGAN.

Our project began with the question, “what if inorganic things ‘decay’ like organic things?”

TL;JUST RUN THE CODE

• opencv-python==4.2.0
• Pillow==7.1.2
• tensorflow==2.4.1
• torch==1.7.0
• torchvision==0.8.1

Type in the following command:

python train.py

If you want to train the model with your own model,

python train.py --dataset your2data --epochs 100 --decay_epochs 80

If your device supports CUDA, it will be activated automatically. If you want to explicitly turn on CUDA,

python train.py --cuda

After training, you can test your model using your custom input image. The images to be tested should be located under "./test".
To specify the model, use the following command:

python test.py --model weights/fruits2rotten/G_A2B.pth --cuda

For real-time webcam translation,

python webcam_translation.py

The model is set to be weights/fruit2rotten/G_A2B.pth by default. User input for the webcam configuration is not yet supported; it must be directly modified from the python script file.

For help, type

python train.py --help

Abstract—The importance of sustainable design is increasing. Through deep convolutional neural network, we imagined the world where everything is organic, and thus perishes.
A deep neural network was trained to predict the "decayed" appearance of an arbitrary input image.
A dataset that consists of several hundred images of fresh/rotten apples, oranges, and bananas were used to train the neural network. pix2pix, CycleGAN, and DiscoGAN were considered as our neural network model, and CycleGAN was found to perform the best.

INTRODUCTION

The dichotomy that we established between inorganic and organic materials are no longer applicable. Witnessing the immortality of inorganic materials threatening the lives and existence of organic materials, we are creating a hypothetical situation where everything in the world decays and perishes.

PROCESS

Multiple training images of fresh and rotten organic food are fed into our model, with which the model will update its parameters. Various neural network architectures were considered.

1. pix2pix
   Our first consideration was pix2pix.
   However, it was not successful, since pix2pix required a set of paired dataset. It was almost impossible to prepare several hundreds of fresh-rotten pair of images.
2. DiscoGAN
   Our next consideration was DiscoGAN. It was chosen since it not only could take in unpaired dataset but also was capable of performing geometry change. Since an apple, an orange, or a banana goes through a morphological change when rotten, DiscoGAN seemed to be a good choice.
   Following figure shows the resulting images after 50 epochs of training with apple2rotten dataset, which consists of 184 images of fresh apple and 271 images of rotten apple.
   
   0.A.jpg is the original image of an apple, 0.AB.jpg is the image that is mapped to the domain of rotten apple, and 0.ABA.jpg is the reconstructed image, which is mapped back to the original domain. As it can be seen, the translated image(0.AB.jpg) not only shows the change in colors but also in its contours.
   However, the transformation was not performing well at all for the images that are not apples. Take a look at following transformations.
   
   
   When the model attempts to convert a non-apple image into an apple image, it tries to force to translate the image into an apple; thus, drastic failure.
3. CycleGAN
   Our next consideration was CycleGAN because it could preserve the geometric characteristics of the input image. Following figures show the resulting images after 200 epochs of training with banana2rotten dataset, which consists of 342 images of fresh banana and 484 images of rotten banana.
   
   
   It seems working! However, since our dataset mostly consists of banana, the model performed well only for the yellow images.
   Therefore, we decided to train the model using all images of the fruits - apples, oranges, and bananas altogether, so that our model can be ready for more diverse colors.
   Following image shows some of the resulting images generated by the model while being trained.
   
   It can be seen that the model is trained to translate the images of red, yellow, orange, and green.
   

RESULT

Following images are generated by our model that is trained to predict the decay of the input image.

The model is trained with 505 images of fresh fruits, and 690 images of rotten fruits. It was trained for 200 epochs.