MVS-using-CNN-and-LSTM

Cloud-Assisted Multi-View Video Summarization using CNN and Bi-Directional LSTM

Paper

https://ieeexplore.ieee.org/abstract/document/8765236

The package contains the following three files.

1. FullDatasetFeatures.py

It contains the code to extract features from the overall dataset.

2. LSTM.py

It contains code to train the extracted features using Bi-directional LSTM. Splitting the code into train, test and validation is already included in this code.

3. OneFileOnlineTest.py

This is to test one video file on the trained model and write its output video.

Notice

We are really sorry to have the raw code, but to ensure the availability of an easy and understandable code we need some time. Feel free to contact me at [email protected] if you have any queries or you will be okay with raw format code. Thanks

Demo Results

Dummy link:

https://www.youtube.com/watch?v=aHvTtb8MbnQ

Contact Me

If you feel any difficulty or errors, please refer to tanveerkhattak37973[at][gmail]

Citation

@article{hussain2019cloud,
  title={Cloud-assisted multiview video summarization using CNN and bidirectional LSTM},
  author={Hussain, Tanveer and Muhammad, Khan and Ullah, Amin and Cao, Zehong and Baik, Sung Wook and de Albuquerque, Victor Hugo C},
  journal={IEEE Transactions on Industrial Informatics},
  volume={16},
  number={1},
  pages={77--86},
  year={2019},
  publisher={IEEE}
}

If you are interested in Multi-view Video Summarization domain you may want to read our recent survey:

@article{hussain2020comprehensive,
  title={A comprehensive survey of multi-view video summarization},
  author={Hussain, Tanveer and Muhammad, Khan and Ding, Weiping and Lloret, Jaime and Baik, Sung Wook and de Albuquerque, Victor Hugo C},
  journal={Pattern Recognition},
  volume={109},
  pages={107567},
  year={2020},
  publisher={Elsevier}
}

My further research [sorted year-wise] on Video Summarization (single- and multi-view video summarization) domain is as follows:

Multi-view Video Summarization

@ARTICLE{9208765,
  author={T. {Hussain} and K. {Muhammad} and A. {Ullah} and J. {Del Ser} and A. H. {Gandomi} and M. {Sajjad} and S. W. {Baik} and V. H. C. {de Albuquerque}},
  journal={IEEE Internet of Things Journal}, 
  title={Multi-View Summarization and Activity Recognition Meet Edge Computing in IoT Environments}, 
  year={2020},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/JIOT.2020.3027483}}
@article{hussain2019intelligent,
  title={Intelligent Embedded Vision for Summarization of Multiview Videos in IIoT},
  author={Hussain, Tanveer and Muhammad, Khan and Del Ser, Javier and Baik, Sung Wook and de Albuquerque, Victor Hugo C},
  journal={IEEE Transactions on Industrial Informatics},
  volume={16},
  number={4},
  pages={2592--2602},
  year={2019},
  publisher={IEEE}
}
@article{hussain2019cloud,
  title={Cloud-assisted multiview video summarization using CNN and bidirectional LSTM},
  author={Hussain, Tanveer and Muhammad, Khan and Ullah, Amin and Cao, Zehong and Baik, Sung Wook and de Albuquerque, Victor Hugo C},
  journal={IEEE Transactions on Industrial Informatics},
  volume={16},
  number={1},
  pages={77--86},
  year={2019},
  publisher={IEEE}
}
@article{후세인2018구조적인,
  title={구조적인 유사성에 기반한 다중 뷰 비디오의 효율적인 키프레임 추출},
  author={후세인 and 탄베르 and 칸살만 and 이미영 and 백성욱 and others},
  journal={한국차세대컴퓨팅학회 논문지},
  volume={14},
  number={6},
  pages={7--14},
  year={2018}
}

Single-view Video Summarization

@article{muhammad2020efficient,
  title={Efficient and Privacy Preserving Video Transmission in 5G-Enabled IoT Surveillance Networks: Current Challenges and Future Directions},
  author={Muhammad, Khan and Hussain, Tanveer and Rodrigues, Joel JPC and Bellavista, Paolo and de Macedo, Antonio Roberto L and de Albuquerque, Victor Hugo C},
  journal={IEEE Network},
  year={2020},
  publisher={IEEE}
}
@article{muhammad2020efficient,
  title={Efficient CNN based summarization of surveillance videos for resource-constrained devices},
  author={Muhammad, Khan and Hussain, Tanveer and Baik, Sung Wook},
  journal={Pattern Recognition Letters},
  volume={130},
  pages={370--375},
  year={2020},
  publisher={Elsevier}
}
@article{muhammad2019deepres,
  title={DeepReS: A Deep Learning-Based Video Summarization Strategy for Resource-Constrained Industrial Surveillance Scenarios},
  author={Muhammad, Khan and Hussain, Tanveer and Del Ser, Javier and Palade, Vasile and De Albuquerque, Victor Hugo C},
  journal={IEEE Transactions on Industrial Informatics},
  volume={16},
  number={9},
  pages={5938--5947},
  year={2019},
  publisher={IEEE}
}
@article{muhammad2019cost,
  title={Cost-effective video summarization using deep CNN with hierarchical weighted fusion for IoT surveillance networks},
  author={Muhammad, Khan and Hussain, Tanveer and Tanveer, Muhammad and Sannino, Giovanna and de Albuquerque, Victor Hugo C},
  journal={IEEE Internet of Things Journal},
  volume={7},
  number={5},
  pages={4455--4463},
  year={2019},
  publisher={IEEE}
}