Skip to content

Home

Jump to bottom
YiHungWONG edited this page Jul 31, 2024 · 3 revisions

Construction-Hazard-Detection

The Construction-Hazard-Detection repository provides a system for automatically detecting and monitoring safety hazards on construction sites using AI-powered object detection and real-time video analysis. It helps construction managers and safety officers identify potential risks and ensure compliance with safety regulations.

The core functionality of the system is built around the YOLOv8 object detection model, which is used to identify various objects and potential hazards in construction site video streams. The main components of the system include:

• Object detection and hazard identification: The DangerDetector class analyzes detected objects to identify potential hazards such as workers without safety gear, workers in controlled areas, or workers in close proximity to machinery. For more details on the hazard detection logic, see the Hazard Detection Logic section.

• Real-time video processing: The StreamCapture and LiveStreamDetector classes handle the capture and processing of video streams from construction sites. The system can process multiple streams simultaneously and adapt to varying internet speeds. More information can be found in the Real-Time Video Stream Processing section.

• Notification system: When hazards are detected, the system can send alerts through various messaging platforms including LINE, Facebook Messenger, Telegram, and WeChat. This functionality is implemented in separate notifier classes for each platform. The Notification System section provides more details on this feature.

• Web-based monitoring interface: A Flask-based web application allows users to view live video streams and detection results in real-time. The implementation details are covered in the Web-Based Monitoring Interface section.

The system also includes tools for data augmentation, model training, and evaluation:

• The /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation directory contains scripts for applying various image augmentation techniques to enhance the training dataset.

• The /Construction-Hazard-Detection/examples/YOLOv8_train directory provides scripts for training, validating, and exporting YOLOv8 models.

• The /Construction-Hazard-Detection/examples/YOLOv8_evaluation directory contains tools for evaluating model performance using metrics such as Average Precision and mean Average Precision (mAP).

Key design choices in the implementation include:

  1. Modular architecture: The system is divided into separate components for video capture, object detection, hazard analysis, and notification, allowing for easy maintenance and extensibility.

  2. Flexibility in deployment: The system can be run in a Docker environment or a Python environment, with instructions provided for both setups.

  3. Configurable video streams: Users can specify details of video streams, including URL, image name, label, and machine learning model, through a YAML configuration file.

  4. Automated model management: The system includes functionality for downloading pre-trained models and periodically updating them to ensure the latest versions are used.

For developers looking to extend or customize the system, the /Construction-Hazard-Detection/examples/YOLOv8_server_api directory provides a Flask-based API for object detection, which can be integrated into other applications or services.

Object Detection and Hazard Identification

References: /Construction-Hazard-Detection/examples/YOLOv8_evaluation, /Construction-Hazard-Detection/examples/YOLOv8_train, /Construction-Hazard-Detection/src

The YOLOModelHandler class in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py serves as the central component for managing YOLO (You Only Look Once) models. It provides methods for:

  • Loading models onto appropriate devices (MPS, CUDA, or CPU)
  • Training models with specified configurations
  • Validating models on datasets
  • Generating predictions on images
  • Exporting models to various formats (e.g., ONNX)
  • Performing k-fold cross-validation

The LiveStreamDetector class in /Construction-Hazard-Detection/src/live_stream_detection.py handles real-time object detection:

  • Generates detections using either cloud API or local YOLOv8 model
  • Implements post-processing to remove overlapping labels
  • Supports sliced prediction using SAHI (Sliced Anchored Hyperrectangle Inference) for improved accuracy on large images

The DangerDetector class in /Construction-Hazard-Detection/src/danger_detector.py identifies potential safety hazards:

  • Calculates the number of people in controlled areas
  • Detects safety violations such as missing hardhats or safety vests
  • Identifies workers in dangerous proximity to machinery or vehicles

The system integrates with various messaging platforms for sending notifications:

The StreamCapture class in /Construction-Hazard-Detection/src/stream_capture.py manages video stream capture:

  • Handles stream failures and reinitialization
  • Adjusts stream quality based on internet speed

For more detailed information on specific components, refer to the following sections:

YOLOv8 Model Training and Evaluation Hazard Detection Logic Real-time Object Detection and Tracking Real-Time Video Stream Processing Hazard Detection and Analysis Notification System

YOLOv8 Model Training and Evaluation

References: /Construction-Hazard-Detection/examples/YOLOv8_evaluation, /Construction-Hazard-Detection/examples/YOLOv8_train

The YOLOModelHandler class in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py manages the training and evaluation of YOLOv8 models for construction hazard detection. Key functionalities include:

• Model loading: load_model() loads a YOLOv8 model from a .yaml or .pt file onto the appropriate device (MPS, CUDA, or CPU).

• Training: train_model() trains the model using specified data configuration, epochs, and optimizer.

• Validation: validate_model() evaluates the model on a validation dataset.

• Cross-validation: cross_validate_model() performs k-fold cross-validation, creating temporary directories for training and validation sets.

• Model export: export_model() exports the trained model to formats like ONNX for deployment.

The /Construction-Hazard-Detection/examples/YOLOv8_train/train_yolo_models.sh script automates the training process for multiple YOLOv8 models:

• Checks for pre-trained models in the ../models directory. • Downloads base models if not present. • Initiates training for each model using train.py.

For model evaluation, the /Construction-Hazard-Detection/examples/YOLOv8_evaluation directory contains scripts to assess YOLOv8 performance:

• convert_yolo_to_coco.py: Converts YOLO annotations to COCO format for evaluation.

• evaluate_sahi_yolov8.py: Uses the SAHI library for sliced prediction and computes COCO metrics.

• evaluate_yolov8.py: Evaluates models using the Ultralytics framework, providing standard metrics.

These scripts allow for customization of evaluation parameters and file paths to suit specific datasets and model requirements.

Hazard Detection Logic

References: /Construction-Hazard-Detection/src

The DangerDetector class in /Construction-Hazard-Detection/src/danger_detector.py is responsible for identifying potential safety hazards in construction sites. Key functionalities include:

• Counting people in controlled areas using calculate_people_in_controlled_area() • Detecting safety violations with detect_danger()

The hazard detection logic follows these steps:

  1. Count people in controlled areas
  2. Classify detected objects (persons, hardhat/safety vest violations, machinery/vehicles)
  3. Filter out likely drivers
  4. Check for hardhat and safety vest violations
  5. Identify persons dangerously close to machinery/vehicles

Helper functions support the main detection logic:

• is_driver(): Determines if a person is likely a vehicle driver based on position and size • overlap_percentage(): Calculates bounding box overlap • is_dangerously_close(): Checks if a person is too close to machinery/vehicles

The detect_danger() method aggregates warnings for various safety violations, providing a comprehensive assessment of potential hazards in the construction site.

Real-time Object Detection and Tracking

References: /Construction-Hazard-Detection/src

The LiveStreamDetector class in /Construction-Hazard-Detection/src/live_stream_detection.py handles real-time object detection and tracking using YOLOv8 with SAHI (Sliced and Hierarchical Inference). Key features include:

• Flexible detection options: Can generate detections locally or via a cloud API. • Authentication handling: Uses authenticate(), token_expired(), and ensure_authenticated() to manage API access tokens. • Detection generation: generate_detections_cloud() sends frames to the API, while generate_detections_local() uses a local YOLOv8 model. • Post-processing: remove_overlapping_labels() and remove_completely_contained_labels() refine detection results for Hardhat and Safety Vest categories. • Stream processing: run_detection() reads frames from the stream and processes them using generate_detections().

The StreamCapture class in /Construction-Hazard-Detection/src/stream_capture.py complements this functionality by:

• Handling video stream capture with execute_capture(), which yields frames and timestamps. • Managing stream failures through reinitialization attempts. • Adapting to network conditions using check_internet_speed() and select_quality_based_on_speed().

These classes work together to provide a robust system for real-time construction site monitoring, capable of adapting to various network conditions and processing requirements.

Real-Time Video Stream Processing

References: /Construction-Hazard-Detection/src/live_stream_detection.py, /Construction-Hazard-Detection/src/live_stream_tracker.py, /Construction-Hazard-Detection/src/stream_capture.py

The StreamCapture class in /Construction-Hazard-Detection/src/stream_capture.py handles real-time video stream processing. Key features include:

• Initialization with a stream URL and optional capture interval • Stream setup using the H264 codec • Frame capture with timestamp generation • Automatic stream reinitialization on failure • Fallback to generic capture method if reinitialization fails • Internet speed checking and quality selection based on available bandwidth

The LiveStreamDetector class in /Construction-Hazard-Detection/src/live_stream_detection.py performs object detection on the captured frames:

• Supports both local processing and cloud API integration • Handles API authentication and token management • Post-processes detection results to remove overlapping and fully contained labels • Provides flexibility to run detection locally or via cloud API

The LiveStreamDetector class in /Construction-Hazard-Detection/src/live_stream_tracker.py focuses on detection and tracking:

• Initializes with a stream URL and YOLOv8 model path • Generates detections including object IDs, bounding boxes, and timestamps • Manages video capture resources

These classes work together to process video streams, detect objects, and track them in real-time, forming the foundation for hazard detection in construction sites.

Hazard Detection and Analysis

References: /Construction-Hazard-Detection/src/danger_detector.py

The DangerDetector class in /Construction-Hazard-Detection/src/danger_detector.py is responsible for identifying potential safety hazards in construction sites. Key functionalities include:

• Calculating the number of people within a controlled area using calculate_people_in_controlled_area(). • Detecting various safety violations through the detect_danger() method.

The hazard detection process involves:

  1. Counting people in the controlled area.
  2. Classifying detected objects into categories (persons, hardhat violations, safety vest violations, machinery/vehicles).
  3. Filtering out likely drivers using the is_driver() helper function.
  4. Checking for hardhat and safety vest violations by comparing person locations with violation detections.
  5. Identifying persons dangerously close to machinery or vehicles using is_dangerously_close().

Helper functions like overlap_percentage() calculate the overlap between bounding boxes to determine proximity of objects.

The system generates warning messages for various hazards, including:

  • People in controlled areas
  • Workers without hardhats or safety vests
  • Workers too close to machinery or vehicles

These warnings are collected in a set and can be used for real-time alerts or further analysis.

Notification System

References: /Construction-Hazard-Detection/src/line_notifier.py, /Construction-Hazard-Detection/src/messenger_notifier.py, /Construction-Hazard-Detection/src/telegram_notifier.py, /Construction-Hazard-Detection/src/wechat_notifier.py

The notification system supports multiple messaging platforms for sending alerts when hazards are detected. Separate notifier classes handle communication with each platform:

Key features:

• All notifiers support sending text messages and optional images • Authentication tokens are loaded from environment variables if not provided explicitly • Image data is converted to appropriate formats before sending (e.g. PIL Image, BytesIO) • HTTP requests are made to respective platform APIs to deliver notifications

The send_notification() method is the primary interface for all notifier classes:

  • Takes recipient ID, message text, and optional image as input
  • Handles platform-specific payload construction and API calls
  • Returns status code or response data from the API

Notable implementation details:

• WeChatNotifier implements additional get_access_token() and upload_media() methods to handle WeChat Work's authentication and media upload requirements • TelegramNotifier uses the python-telegram-bot library for simplified API interactions • Error handling and logging are implemented to manage API failures gracefully

The notification system provides a flexible way to alert stakeholders across multiple messaging platforms when construction hazards are detected.

Web-Based Monitoring Interface

References: /Construction-Hazard-Detection/examples/streaming_web

The web-based monitoring interface is implemented using Flask, with routes defined in /Construction-Hazard-Detection/examples/streaming_web/routes.py. Key components include:

• Index page: Displays a grid of camera labels, each linking to a specific camera stream.

• Label page: Shows images for a particular label, retrieved from Redis using get_image_data().

• Camera page: Provides a live stream view for a specific camera and label.

The frontend is built with HTML templates and JavaScript:

• /Construction-Hazard-Detection/examples/streaming_web/templates/index.html: Renders the camera label grid.

• /Construction-Hazard-Detection/examples/streaming_web/templates/label.html: Displays labeled images in a grid layout.

• /Construction-Hazard-Detection/examples/streaming_web/templates/camera.html: Shows the live camera feed.

JavaScript files handle dynamic updates:

• /Construction-Hazard-Detection/examples/streaming_web/static/js/index.js: Updates all image sources every 5 seconds.

• /Construction-Hazard-Detection/examples/streaming_web/static/js/camera.js: Refreshes the camera image source every 5 seconds.

• /Construction-Hazard-Detection/examples/streaming_web/static/js/label.js: Manages WebSocket connections for real-time updates.

The interface uses WebSockets for efficient real-time communication, with event handlers registered in /Construction-Hazard-Detection/examples/streaming_web/sockets.py. The update_images() function runs as a background task, sending new image data to clients every 10 seconds.

Styling is handled by /Construction-Hazard-Detection/examples/streaming_web/static/css/styles.css, which implements a responsive grid layout and adjusts for different screen sizes and forced colors mode.

Data Augmentation Techniques

References: /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation

The DataAugmentation class in /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation/data_augmentation.py implements various image augmentation techniques to enhance the training dataset for YOLOv8 object detection models. The augmentation sequence is defined in the _get_augmentation_sequence() method, which applies the following transformations with varying probabilities:

  • Horizontal and vertical flipping
  • Rotation (up to 25 degrees)
  • Scaling (80% to 120%)
  • Translation (up to 20% of image size)
  • Brightness and contrast adjustments
  • Gaussian blur
  • Additive Gaussian noise

These transformations are applied using the imgaug.augmenters library. The augment_image() method processes individual images, applying the augmentation sequence while preserving bounding box annotations. Key features include:

  • Image resizing to ensure consistent input size for the model
  • Adjustment of bounding box coordinates to match resized image dimensions
  • Batch processing in augment_data() to optimize memory usage
  • Dataset shuffling in shuffle_data() using unique UUIDs for randomness

The augmentation process is parallelized using multi-threading to improve processing speed. Users can customize the augmentation sequence by modifying the _get_augmentation_sequence() method to add or remove transformations based on specific dataset requirements.

For visualization of augmented images and their bounding boxes, the BoundingBoxVisualiser class in /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation/visualise_bounding_boxes.py can be used to verify the correctness of the augmented data.

YOLOv8 Model Training

References: /Construction-Hazard-Detection/examples/YOLOv8_train

The YOLOModelHandler class in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py manages the training process for YOLOv8 models. Key methods include:

  • load_model(): Loads a YOLO model from a .yaml or .pt file onto the appropriate device (MPS, CUDA, or CPU).
  • train_model(): Trains the model using specified data configuration, epochs, and optimizer.
  • validate_model(): Evaluates model performance on a validation dataset.
  • cross_validate_model(): Performs k-fold cross-validation, creating temporary directories for training and validation sets.

The training process can be customized through command-line arguments in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py, allowing users to specify the model name, number of epochs, and data configuration. Hyperparameter tuning is achieved by modifying the YOLO configuration files.

For batch training of multiple models, /Construction-Hazard-Detection/examples/YOLOv8_train/train_yolo_models.sh automates the process:

  • Checks for pre-trained models in the ../models directory.
  • Downloads base models if not present.
  • Initiates training for each model using /Construction-Hazard-Detection/examples/YOLOv8_train/train.py.

This approach allows for efficient experimentation with different model architectures and hyperparameters. The script supports both training from scratch and fine-tuning pre-trained models, enhancing flexibility in model development.

Model Export and Deployment

References: /Construction-Hazard-Detection/examples/YOLOv8_train

The YOLOModelHandler class in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py provides the export_model() method for exporting trained YOLOv8 models to various formats, including ONNX. This method facilitates the deployment of models in different environments.

Key aspects of model export and deployment:

• The export_model() method takes the export format as an argument, allowing flexibility in choosing the output format.

• ONNX format is specifically supported, enabling deployment across a wide range of platforms and frameworks.

• The train.py script can be used to export models via command-line arguments:

python train.py --model_name 'yolov8n.pt' --export_format 'onnx' --onnx_path 'yolov8n.onnx'

• Exported models are saved in the specified path, making them easily accessible for deployment.

• The /Construction-Hazard-Detection/examples/YOLOv8_train/train_yolo_models.sh script automates the training process for multiple YOLOv8 models, including the export step.

• After exporting, models can be integrated into various deployment environments, such as web services or edge devices, depending on the chosen export format.

Server Setup and Configuration

References: /Construction-Hazard-Detection/examples/YOLOv8_server_api/app.py, /Construction-Hazard-Detection/examples/YOLOv8_server_api/config.py

The Flask application is initialized in /Construction-Hazard-Detection/examples/YOLOv8_server_api/app.py. Key setup steps include:

• Generating a secure JWT secret key using the secrets library • Loading configuration from the Config class in /Construction-Hazard-Detection/examples/YOLOv8_server_api/config.py • Initializing JWTManager for JSON Web Token authentication • Setting up the database connection using SQLAlchemy

The Config class in /Construction-Hazard-Detection/examples/YOLOv8_server_api/config.py manages application settings:

• JWT_SECRET_KEY: Retrieved from environment variables, with a fallback value • SQLALCHEMY_DATABASE_URI: Database connection string, also from environment variables • SQLALCHEMY_TRACK_MODIFICATIONS: Disabled for performance

Environment variables are loaded from a .env file using python-dotenv, enhancing portability and security by keeping sensitive information separate from the codebase.

A BackgroundScheduler is implemented to update the JWT secret key every 30 days, enhancing security.

The application registers blueprints for authentication, object detection, and model management, modularizing the API structure.

Authentication and User Management

References: /Construction-Hazard-Detection/examples/YOLOv8_server_api/auth.py, /Construction-Hazard-Detection/examples/user_management

The authentication system is implemented using Flask-JWT-Extended for token generation and management. The auth_blueprint in /Construction-Hazard-Detection/examples/YOLOv8_server_api/auth.py handles authentication routes.

Key components:

  • create_token(): Authenticates users and generates JSON Web Tokens (JWT). • Extracts username and password from request • Checks user credentials against database or cache • Generates access token on successful authentication

User management operations are defined in /Construction-Hazard-Detection/examples/user_management/user_operation.py:

The User model in /Construction-Hazard-Detection/examples/user_management/models.py represents authenticated users:

  • Attributes: id, username, password_hash
  • Methods: • set_password(): Generates and stores password hash • check_password(): Verifies provided password against stored hash

Flask routes in /Construction-Hazard-Detection/examples/user_management/app.py expose user management functionality:

These routes validate input, escape data to prevent XSS attacks, and call corresponding functions from user_operation.py.

Object Detection API

References: /Construction-Hazard-Detection/examples/YOLOv8_server_api/detection.py, /Construction-Hazard-Detection/examples/YOLOv8_server_api/models.py

The detection_blueprint Flask blueprint handles object detection requests. Key components:

• detect() function processes /detect endpoint requests, protected by JWT authentication and rate-limited to 3000 requests/minute.

• Uploaded images are converted from byte streams to NumPy arrays, then to OpenCV format.

• Object detection uses the get_sliced_prediction() function from the sahi library with the specified YOLOv8 model.

• Results are compiled into bounding boxes, confidence scores, and class labels in YOLOv8 format.

• Post-processing removes overlapping and contained labels for "Hardhat" and "Safety Vest" categories using remove_overlapping_labels() and remove_completely_contained_labels().

• overlap_percentage() and is_contained() helper functions determine bounding box relationships.

• Garbage collection is performed after processing to free memory.

The DetectionModelManager class manages YOLOv8 models:

• Loads models using AutoDetectionModel from the sahi library.

• Provides methods to access models and check for updates.

• Runs a background thread to reload models hourly if files are modified.

This implementation allows for efficient object detection with multiple YOLOv8 models, periodic model updates, and post-processing of results to improve accuracy in construction hazard detection scenarios.

Model Management and Downloading

References: /Construction-Hazard-Detection/examples/YOLOv8_server_api/model_downloader.py, /Construction-Hazard-Detection/examples/YOLOv8_server_api/models.py

The DetectionModelManager class in /Construction-Hazard-Detection/examples/YOLOv8_server_api/models.py handles loading and periodic updating of pre-trained machine learning models:

  • Loads YOLO models using AutoDetectionModel from the sahi library
  • Stores loaded models in a dictionary for easy access
  • Provides methods to retrieve specific models and their last modified times
  • Runs a background thread to check for model updates hourly and reloads if necessary

The model_downloader.py blueprint in /Construction-Hazard-Detection/examples/YOLOv8_server_api/model_downloader.py manages downloading of model files:

  • Exposes a /models/<model_name> endpoint for downloading specific models
  • Validates requested model names against an allowed list
  • Checks if local model files are up-to-date by comparing last modified times
  • Downloads updated model files from an external server when necessary
  • Implements rate limiting to prevent excessive requests (10 per minute per IP)
  • Handles various error cases, such as file not found or network issues

Together, these components ensure that the system always has access to the latest versions of pre-trained models while efficiently managing downloads and updates.

Notification Services

References: /Construction-Hazard-Detection/src

The notification services are implemented through several classes, each handling a different messaging platform:

• LineNotifier: Sends notifications via LINE Notify API • MessengerNotifier: Sends notifications through Facebook Messenger • TelegramNotifier: Handles notifications via Telegram • WeChatNotifier: Manages notifications through WeChat Work

These classes share a similar structure:

• Initialization with API tokens or credentials, often retrieved from environment variables • A send_notification method that takes a message and optional image as input • Methods for handling authentication and token management

Key features:

• LineNotifier constructs API requests with headers and payload, converting images to BytesIO buffers when needed • MessengerNotifier prepares multipart/form-data requests for sending images • TelegramNotifier uses the python-telegram-bot library to interact with the Telegram API • WeChatNotifier includes methods for retrieving access tokens and uploading media to the WeChat Work API

The notifiers handle various error cases and network issues:

• Retry logic in LiveStreamDetector for API calls • Token expiration checks and renewal in WeChatNotifier

Image handling is consistent across notifiers:

• Accept NumPy arrays as input • Convert to appropriate formats (e.g., PIL Image, BytesIO) for API requests

The notification services integrate with the Real-Time Monitoring and Notification system, allowing for immediate alerts when hazards are detected in the construction site.

LINE Chatbot Example

References: /Construction-Hazard-Detection/examples/line_chatbot

The /Construction-Hazard-Detection/examples/line_chatbot/line_bot.py script demonstrates a simple LINE chatbot using Flask and the LINE Messaging API. The chatbot's primary function is to echo back any text messages it receives.

Key components:

• Flask application: Handles incoming webhook requests from LINE. • LINE Bot API client: Interacts with the LINE platform using LineBotApi. • Webhook handler: Processes incoming events with WebhookHandler.

The main functionality is implemented in two key functions:

• callback(): Serves as the entry point for the Flask application.

  • Validates the request signature using the X-Line-Signature header.
  • Passes the request body to handler.handle() for processing.
  • Returns 'OK' if successful, or aborts with a 400 status for invalid signatures.

• handle_message(): Responds to text message events.

The chatbot uses environment variables LINE_CHANNEL_ACCESS_TOKEN and LINE_CHANNEL_SECRET for authentication with the LINE platform. These must be properly configured in the deployment environment.

This example serves as a starting point for more complex LINE chatbot implementations. Developers can extend the functionality by modifying the handle_message() function to add custom responses or integrate with other services.

Real-Time Monitoring and Notification

References: /Construction-Hazard-Detection/examples/streaming_web, /Construction-Hazard-Detection/src

The LiveStreamDetector class in /Construction-Hazard-Detection/src/live_stream_tracker.py handles real-time video stream processing and object detection. Key functionality includes:

• generate_detections(): Yields detection results, object IDs, and timestamps for each video frame. • run_detection(): Processes the video stream and outputs detection results.

The StreamCapture class in /Construction-Hazard-Detection/src/stream_capture.py manages video stream capture:

• execute_capture(): Captures frames and yields them with timestamps. • check_internet_speed() and select_quality_based_on_speed(): Adjust stream quality based on available bandwidth.

Hazard detection logic is implemented in the DangerDetector class in /Construction-Hazard-Detection/src/danger_detector.py:

• calculate_people_in_controlled_area(): Counts people within a defined safety zone. • detect_danger(): Identifies safety violations like unauthorized entry, missing safety gear, and proximity to machinery.

When hazards are detected, notifications are sent through various messaging platforms:

• LineNotifier in /Construction-Hazard-Detection/src/line_notifier.py • MessengerNotifier in /Construction-Hazard-Detection/src/messenger_notifier.py • TelegramNotifier in /Construction-Hazard-Detection/src/telegram_notifier.py • WeChatNotifier in /Construction-Hazard-Detection/src/wechat_notifier.py

These classes provide send_notification() methods to dispatch alerts with optional images.

The web interface for monitoring is implemented in the /Construction-Hazard-Detection/examples/streaming_web directory:

• Flask routes in routes.py serve pages for viewing camera feeds and labels. • WebSocket integration in sockets.py enables real-time updates of camera images. • Client-side JavaScript in /static/js handles dynamic image updates and WebSocket communication.

This system provides continuous monitoring of construction sites, real-time hazard detection, and immediate notification of safety violations through multiple channels.

Data Augmentation and Model Training

References: /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation, /Construction-Hazard-Detection/examples/YOLOv8_train

The DataAugmentation class in /Construction-Hazard-Detection/examples/YOLOv8_data_augmentation/data_augmentation.py handles image augmentation for enhancing YOLOv8 model training datasets. Key features include:

• Applying random transformations like rotation, scaling, flipping, and color jittering to input images • Preserving bounding box annotations during transformations • Processing images in batches to conserve memory • Shuffling the augmented dataset for randomness

The YOLOModelHandler class in /Construction-Hazard-Detection/examples/YOLOv8_train/train.py manages YOLOv8 model operations:

• Loading models onto appropriate devices (MPS, CUDA, or CPU) • Training models with specified data configurations and epochs • Validating models on datasets • Generating predictions on images, including SAHI (Sliced Anchored Hyperrectangle Inference) for large images • Exporting models to formats like ONNX • Performing k-fold cross-validation

The /Construction-Hazard-Detection/examples/YOLOv8_train/train_yolo_models.sh script automates training for multiple YOLOv8 models:

• Checking for pre-trained models • Downloading base models if necessary • Sequentially training models using train.py

Users can modify training parameters, augmentation techniques, and hyperparameters in YOLO configuration files to suit specific datasets or applications.

API and Server Implementation

References: /Construction-Hazard-Detection/examples/YOLOv8_server_api, /Construction-Hazard-Detection/examples/user_management

The server-side API is implemented using Flask, with the main application logic in /Construction-Hazard-Detection/examples/YOLOv8_server_api/app.py. This file initializes the Flask application, configures JWT authentication, and registers blueprints for different functionalities.

Key components of the API implementation:

  • Authentication: Handled by the auth_blueprint in /Construction-Hazard-Detection/examples/YOLOv8_server_api/auth.py. The create_token() function authenticates users and generates JWT tokens.

  • Object Detection: Implemented in the detection_blueprint in /Construction-Hazard-Detection/examples/YOLOv8_server_api/detection.py. The detect() function processes uploaded images using YOLOv8 models and returns detection results.

  • Model Management: The DetectionModelManager class in /Construction-Hazard-Detection/examples/YOLOv8_server_api/models.py handles loading and updating of YOLOv8 models.

  • Configuration: Centralized in the Config class in /Construction-Hazard-Detection/examples/YOLOv8_server_api/config.py, allowing for flexible settings management.

  • Error Handling: Implemented across multiple modules to manage various error scenarios gracefully.

User management functionality is provided in /Construction-Hazard-Detection/examples/user_management/app.py:

  • CRUD operations for users are implemented using SQLAlchemy ORM.
  • Routes for adding, deleting, and updating users are defined, with input validation and XSS prevention.
  • User operations are handled by functions in /Construction-Hazard-Detection/examples/user_management/user_operation.py.

The API design emphasizes modularity and separation of concerns, allowing for easy extension and maintenance of the codebase.

Integration with Messaging Platforms

References: /Construction-Hazard-Detection/examples/line_chatbot, /Construction-Hazard-Detection/src

The system integrates with various messaging platforms through dedicated notifier classes:

• LineNotifier: Sends notifications via LINE Notify API • MessengerNotifier: Sends notifications via Facebook Messenger • TelegramNotifier: Sends notifications through Telegram • WeChatNotifier: Sends notifications through WeChat Work

Each notifier class follows a similar pattern:

• Initialization with platform-specific credentials (tokens, secrets) • A send_notification() method to send text messages and optional images

Key features:

• LineNotifier uses the LINE Notify API to send messages and images • MessengerNotifier supports sending messages and image attachments via Facebook's Graph API • TelegramNotifier utilizes the Telegram Bot API for message and photo sending • WeChatNotifier handles authentication with WeChat Work API and supports text and image messages

The notifiers are designed to be easily integrated into the main application, allowing for flexible notification delivery across multiple platforms. This modular approach enables easy addition of new messaging platforms in the future.

An example LINE chatbot implementation is provided in /Construction-Hazard-Detection/examples/line_chatbot/line_bot.py, demonstrating how to create a simple echo bot using the LINE Messaging API and Flask.

Clone this wiki locally