;
};
`Python` · `PyTorch` · `YOLO` · `Computer Vision` · `Precision Livestock`
## Overview
A desktop application for real-time dairy cow lameness detection. Input is either an RTSP stream or a local video file. The backend uses YOLO11m-seg to perform instance segmentation on each frame, then accumulates frames over time to build a GEI (Gait Energy Image) and an HEI (Height Energy Image). A MobileNetV3-Large classifier takes both as input and outputs one of four lameness grades: Normal, Slight, Severe, or Stationary. The PyQt6 front end shows a live annotated video window, separate GEI and HEI visualization panels, and a per-animal statistics table.
Model weights are not included in the repository and must be obtained from the maintainer.
## Core features
* RTSP stream or video file input; multiple formats and stream protocols supported
* YOLO11m-seg instance segmentation with multi-animal tracking per frame
* GEI construction: binary silhouette frames averaged over a sliding window to capture overall gait shape (centroid-aligned, aspect-ratio-preserved crop, resized to 220x318)
* HEI construction: three-channel encoding (full, front-half, rear-half interval accumulation) to capture dorsal height variation patterns
* MobileNetV3-Large classifier fuses GEI and HEI predictions into a four-grade output (GEI weight higher by default)
* GPU acceleration via CUDA, with batch processing, memory pooling, and a ring buffer for temporal frame management
* PyQt6 GUI with live annotation view, GEI/HEI energy image windows, FPS monitor, and statistics table
* Adjustable parameters: YOLO confidence threshold, accumulation window size, GEI/HEI fusion weight, playback speed
## Lameness grades and dual-modality fusion
The classifier outputs four grades, Normal, Slight, Severe, and Stationary, fusing GEI (gait energy image) and HEI (height energy image) features:
## Architecture
## Quick start
The weights are not included in the repository. Contact the maintainer to get the following three files and place them at the paths shown:
```
GUI/lameness_models/GEI_lameness_mobileNetv3_large.pth
GUI/lameness_models/HEI_lameness_mobileNetv3_large.pth
GUI/yolo_model/cow_yolo11m-seg_model.pt
```
```bash theme={null}
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```
```bash theme={null}
cd GUI
pip install -r requirements_gui.txt
```
```bash theme={null}
python gui_app.py
```
Set the input source (RTSP URL or video path), model paths, and camera calibration file in the interface, then click Start.
## Notes
Model weights are not in the repository; place them in the correct directories before attempting inference. The application has been tested on Windows 10/11 (64-bit) only. GPU inference requires an NVIDIA GPU with CUDA 12.1; the app falls back to CPU mode without a GPU but runs significantly slower. For RTSP connection failures, verify the URL format, network reachability, and that the camera supports RTSP.
## In practice
Suited for research trials or farm experiments that need a per-animal lameness grade from walking footage, with a graphical interface that is easier to operate than a command-line inference script. Results can be exported as statistical reports. The dual-modality GEI and HEI fusion is intended to combine gait shape information with dorsal height variation, giving the classifier two complementary dimensions of locomotion data.
## Links
* GitHub: [felimet/dairycow\_lameness\_gui](https://github.com/felimet/dairycow_lameness_gui)
* Ultralytics YOLO11: [docs.ultralytics.com](https://docs.ultralytics.com)
* PyQt6 docs: [riverbankcomputing.com/software/pyqt](https://www.riverbankcomputing.com/software/pyqt/)