🔍 Image Search Engine

• ⌛ Project Duration: June 12 to June 24
• ⚙️ Main tech: PyTorch, BentoML, NextJS, Vercel
• 🦄 Purpose: Drag and drop (or upload) an image and find similar images
• 💻 Check it out: https://image-search-pro.vercel.app/
• 📷 Video demonstration: https://www.youtube.com/watch?v=loBZkPvn29c

🤗 Website

• 💪 Frontend framework: NextJS
• 🌧️ Deployment: Vercel
• 💻 Check it out: https://image-search-pro.vercel.app/
• 📷 Video demonstration: https://www.youtube.com/watch?v=loBZkPvn29c

Usage

1. Upload (or drag and drop) an image into the website

2. Find and download similar images

🤖 Backend

• 🎰 ML Framework: PyTorch
• 🏈 Backend Framework: BentoML
• 🌧️ Deployment method: BentoCTL, AWS and Terraform
• 🔗 API Link: https://shzs1c1u45.execute-api.ap-southeast-2.amazonaws.com/predict

🧪 Test the API with Postman

1. Open up Postman and paste the aforementioned API Link
2. Make sure the request METHOD is set to "POST"
3. Go to "Body" and select "form-data"
4. Add a new file key named "image" and upload a file from your PC
5. Hit "Send"

🎰 Backend: ML

I began my project with TensorFlow, but due to model size issues, I transitioned to PyTorch. MobileNetV3 architecture was utilized to reduce the model size and improve inference speed. I employed a MobileNetV3 model for classification and a Siamese MobileNetV3 network for similarity scoring. This combination of frameworks and architectures enabled me to achieve smaller models without compromising performance

• 🐶 Classifier:

  • Architecture: MobileNetV3
  • Purpose: Classify the input images. The output label (e.g. Labrador Retriever) is used to fetch 50 images from Unsplash.

• 🏆 Siamese network:

  • Architecture: 2 MobileNetV3 models
  • Purpose: Calculate similarity scores for each of the 50 images and the input image. The 20 most similar images, i.e. the images with the best similarity scores, are returned.

🏈 Backend: server code

After experimenting with different backend frameworks such as FastAPI and Flask, I came across BentoML, which is a tool that makes ML API creation and deployment effortless.

🌧️ Backend: deployment

BentoML offers a tool called bentoctl, which is used to deploy BentoML services to production ready API endpoints in the cloud. It does this using AWS Lambda, AWS API Gateway, and AWS ECR, whilst using Terraform to automate the creation of this infrastructure.

📜 Summary

⚙️ Technologies

This project exposed me to many different technologies involved in ML application deployment, as well full stack development.

Tech used:

• Frontend: NextJS, Vercel
• Backend: PyTorch, BentoML, AWS, Terraform

💡 What I learnt

• 🏃‍♂️ Model optimization: In my quest to achieve lower inference times, I had to switch from Tensorflow to PyTorch and from ResNets to MobileNet. This process was painstaking, however it taught me the value of different model architectures and how some are better suited for application development.
• 🌧️ Deployment: I was exposed to different technologies involved in the deployment process including Docker and AWS.
• 🫡 Never give up: This may sound cliche, but it's a timeless mantra which applies no matter what field you are in. Many times throughout the project I was faced with a challenge or an error which made me want to give up. But through persistence and grit, I managed to overcome these challenges and eventually produce a functional ML application.