Skip to main content
Python · PyTorch · YOLO · Computer Vision · Precision Livestock

​
Overview

A desktop application for real-time dairy cow lameness detection. Input is either an RTSP stream or a local video file. The backend uses YOLO11m-seg to perform instance segmentation on each frame, then accumulates frames over time to build a GEI (Gait Energy Image) and an HEI (Height Energy Image). A MobileNetV3-Large classifier takes both as input and outputs one of four lameness grades: Normal, Slight, Severe, or Stationary. The PyQt6 front end shows a live annotated video window, separate GEI and HEI visualization panels, and a per-animal statistics table. Model weights are not included in the repository and must be obtained from the maintainer.

​
Core features

  • RTSP stream or video file input; multiple formats and stream protocols supported
  • YOLO11m-seg instance segmentation with multi-animal tracking per frame
  • GEI construction: binary silhouette frames averaged over a sliding window to capture overall gait shape (centroid-aligned, aspect-ratio-preserved crop, resized to 220x318)
  • HEI construction: three-channel encoding (full, front-half, rear-half interval accumulation) to capture dorsal height variation patterns
  • MobileNetV3-Large classifier fuses GEI and HEI predictions into a four-grade output (GEI weight higher by default)
  • GPU acceleration via CUDA, with batch processing, memory pooling, and a ring buffer for temporal frame management
  • PyQt6 GUI with live annotation view, GEI/HEI energy image windows, FPS monitor, and statistics table
  • Adjustable parameters: YOLO confidence threshold, accumulation window size, GEI/HEI fusion weight, playback speed

​
Lameness grades and dual-modality fusion

The classifier outputs four grades, Normal, Slight, Severe, and Stationary, fusing GEI (gait energy image) and HEI (height energy image) features:

​
Architecture

GUI
gui_app.py · GUI entry point
main_backend.py · backend engine (DetectionWorker)
model_v3.py · MobileNetV3 model definition
lameness_models · classifier weights (obtain separately)
GEI_lameness_mobileNetv3_large.pth
HEI_lameness_mobileNetv3_large.pth
yolo_model · YOLO weights (obtain separately)
cow_yolo11m-seg_model.pt
json
class_indices.json
PY_*_camera_params.json

​
Quick start

1

Obtain model weights

The weights are not included in the repository. Contact the maintainer to get the following three files and place them at the paths shown:
2

Install PyTorch (CUDA build)

3

Install remaining dependencies

4

Launch the GUI

Set the input source (RTSP URL or video path), model paths, and camera calibration file in the interface, then click Start.

​
Notes

Model weights are not in the repository; place them in the correct directories before attempting inference. The application has been tested on Windows 10/11 (64-bit) only. GPU inference requires an NVIDIA GPU with CUDA 12.1; the app falls back to CPU mode without a GPU but runs significantly slower. For RTSP connection failures, verify the URL format, network reachability, and that the camera supports RTSP.

​
In practice

Suited for research trials or farm experiments that need a per-animal lameness grade from walking footage, with a graphical interface that is easier to operate than a command-line inference script. Results can be exported as statistical reports. The dual-modality GEI and HEI fusion is intended to combine gait shape information with dorsal height variation, giving the classifier two complementary dimensions of locomotion data.