一、参考资料
项目源码
pytorch yolo5+Deepsort实现目标检测和跟踪
工程落地
YoloV5 + deepsort + Fast-ReID 完整行人重识别系统（三）
yolov5-deepsort-pedestrian-counting
Yolov5-Deepsort-Fastreid
二、相关介绍
Deepsort是实现目标跟踪的算法，从sort（simple online and realtime tracking）演变而来。其使用卡尔慢滤波器预测所检测对象的运动轨迹，匈牙利算法将它们与新检测的目标匹配。Deepsort易于使用，且速度快，成为AI目标检测跟踪的热门算法。
三、重要说明
yolov5可检测多种类型的目标，而Deepsort目标跟踪只能跟踪一种类型目标，例如person、car。所以，跟踪需要把yolov5的目标检测类型数量限制成单个类型检测。coco数据集定义：person=0，car=2。
# 行人跟踪python track.py --classes 0 --source demo_person.mp4 # 小汽车跟踪python track.py --classes 2 --source demo_car.mp4
yolov5提供不同检测精度的权重文件，yolov5x.pt比yolov5s.pt精度高。应用跟踪时，当两个目标重叠后再分离，yolov5s.pt会出现标注数改变。比如，目标10和目标20发生重叠分离，目标10变成了目标15，而目标20不变（目标20遮挡目标10）。此种情况，用yolov5x.pt会好很多，维持目标10不变。
yolov5限定单个类型，不需要重新训练。faster rcnn、ResNet限定单个类型，单需要重新训练。
yolov5的速度明显优于FastRCNN，且消耗GPU资源少。用FastRCNN，还没用到Deepsort，只看逐帧检测，速度比yolov5+Deepsort逐帧目标检测还要慢，且GPU使用率达到95%。
yolov5的训练速度比Faster RCNN、ResNet50、FPN快。
四、实验环境
4.1 系统环境
EnvironmentOperating System + Version: Ubuntu + 16.04GPU Type: GeForce GTX1650,4GBNvidia Driver Version: 470.63.01CUDA Version: 10.2.300CUDNN Version: 7.6.5Python Version (if applicable): 3.6.14virtualenv：20.13.0gcc：7.5.0g++：7.5.0
4.2 requirements-gpu.txt
absl-py==1.0.0cached-property==1.5.2cachetools==4.2.4certifi==.10.8charset-normalizer==2.0.10cycler==0.11.0Cython==0.29.26dataclasses==0.8distlib==0.3.4easydict==1.9filelock==3.4.1flake8==4.0.1future==0.18.2gdown==3.10.1google-auth==2.3.3google-auth-oauthlib==0.4.6grpcio==1.43.0h5py==3.1.0idna==3.3imageio==2.13.5importlib-metadata==4.2.0importlib-resources==5.4.0isort==4.3.21kiwisolver==1.3.1Markdown==3.3.5matplotlib==3.3.4mccabe==0.6.1numpy==1.19.5oauthlib==3.1.1opencv-python==4.5.5.62pandas==1.1.5Pillow==8.4.0platformdirs==2.4.0protobuf==3.19.3pyasn1==0.4.8pyasn1-modules==0.2.8pycodestyle==2.8.0pyflakes==2.4.0pyparsing==3.0.6PySocks==1.7.1python-dateutil==2.8.2pytz==.3PyYAML==6.0requests==2.27.1requests-oauthlib==1.3.0rsa==4.8scipy==1.5.4seaborn==0.11.2six==1.16.0tb-nightly==2.8.0a2027tensorboard-data-server==0.6.1tensorboard-plugin-wit==1.8.1torch==1.9.0+cu102torchvision=0.10.0+cu102tqdm==4.62.3typing_extensions==4.0.1urllib3==1.26.8virtualenv==20.13.0Werkzeug==2.0.2yacs==0.1.8yapf==0.32.0zipp==3.6.0
五、关键步骤
5.1 下载 github 仓库代码
5.1.1 下载 Yolov5_DeepSort_Pytorch
git clone /mikel-brostrom/Yolov5_DeepSort_Pytorch.git
5.1.2 项目目录
.├── deep_sort│ ├── configs│ ├── deep│ ├── deep_sort.py│ ├── __init__.py│ ├── LICENSE│ ├── __pycache__│ ├── README.md│ ├── sort│ └── utils├── inference # infer 推理的结果│ └── output├── LICENSE├── MOT16_eval│ ├── eval.sh│ ├── track_all.gif│ └── track_pedestrians.gif├── README.md├── requirementes-gpu.txt├── requirements.txt├── runs│ └── track├── track.py├── venv # virtualenv 创建的虚拟环境│ ├── bin│ ├── lib│ └── pyvenv.cfg├── yolov5 # clone yolov5 to this path│ ├── CONTRIBUTING.md│ ├── data│ ├── detect.py│ ├── Dockerfile│ ├── export.py│ ├── hubconf.py│ ├── LICENSE│ ├── models│ ├── README.md│ ├── requirements.txt│ ├── setup.cfg│ ├── train.py│ ├── tutorial.ipynb│ ├── utils│ ├── val.py│ └── weights
5.1.3 下载 yolov5
下载到Yolov5_DeepSort_Pytorch根目录下，删除之前的yolov5文件夹。
git clone /ultralytics/yolov5.git
5.1.4 修改文件夹名称
deep-person-reid改为reid
5.2 环境配置
5.2.1 （可选）创建virtualenv虚拟环境
# 进入项目路径cd Yolov5_DeepSort_Pytorch# 创建虚拟环境virtualenv --system-site-packages -p /usr/bin/python venv# 激活虚拟环境source ./venv/bin/activate
5.2.2 安装依赖包
# 安装依赖包pip install -r requirements.txt
5.3 下载预训练模型
选择目标检测模型：yolov5；
选择DeepSort模型：ReID；
5.3.1 下载yolov5预训练模型
下载地址，并放入目录Yolov5_DeepSort_Pytorchyolo5/weights
比如， yolov5s.pt
python track.py --source 0 --yolo_model yolov5/weights/yolov5n.pt --img 640yolov5/weights/yolov5s.ptyolov5/weights/yolov5m.ptyolov5/weights/yolov5l.pt yolov5/weights/yolov5x.pt --img 1280...
5.3.2 下载Deepsort预训练模型
下载地址，放入目录Yolov5_DeepSort_Pytorch/deep_sort_pytorch/deep_sort/deep/checkpoint
比如，osnet_x1_0
python track.py --source 0 --deep_sort_model osnet_x1_0nasnsetmobileresnext101_32x8d
5.4 infer推理测试
python track.py --source 0 --yolo_model yolov5/weights/yolov5n.pt --deep_sort_model osnet_x1_0 --img 640
(venv) yichao@yichao:~/MyDocuments/Yolov5_DeepSort_Pytorch$ python track.py --source 0 --yolo_model yolov5/weights/yolov5s.pt --deep_sort_model osnet_x1_0 --img 640deep_sort/deep/reid/torchreid/metrics/rank.py:12: UserWarning: Cython evaluation (very fast so highly recommended) is unavailable, now use python evaluation.'Cython evaluation (very fast so highly recommended) is 'Successfully loaded imagenet pretrained weights from "/home/yichao/MyDocuments/Yolov5_DeepSort_Pytorch/deep_sort/deep/checkpoint/osnet_x1_0_imagenet.pth"Selected model type: osnet_x1_0YOLOv5 🚀 v6.0-193-gdb1f83b torch 1.9.0+cu102 CUDA:0 (NVIDIA GeForce GTX 1650, 3904MiB)YOLOv5 🚀 v6.0-193-gdb1f83b torch 1.9.0+cu102 CUDA:0 (NVIDIA GeForce GTX 1650, 3904MiB)weight_path: yolov5/weights/yolov5s.ptweight_path: yolov5/weights/yolov5s.ptFusing layers... Model Summary: 224 layers, 7266973 parameters, 0 gradients1/1: 0... Success (inf frames 640x480 at 30.00 FPS)0: 480x640 1 person, Done. YOLO:(0.428s), DeepSort:(0.220s)0: 480x640 1 person, Done. YOLO:(0.023s), DeepSort:(0.020s)0: 480x640 1 person, Done. YOLO:(0.010s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s).........
5.5 修改源码
由于预训练模型是手动下载的，所以需要修改源码中的路径。
修改osnet_ain.py源码
# 源码路径Yolov5_DeepSort_Pytorch/deep_sort/deep/reid/torchreid/models/osnet.pycached_file = os.path.join(model_dir, filename)改为cached_file = "/home/yichao/MyDocuments/Yolov5_DeepSort_Pytorch/deep_sort/deep/checkpoint/osnet_x1_0_imagenet.pth"
5.6 infer推理速度
0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.010s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.010s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.011s), DeepSort:(0.013s)0: 480x640 1 person, Done. YOLO:(0.010s), DeepSort:(0.013s)# yolov5+Deepsort，大约24ms，即41FPS# DeepSort 的速度取决于画面中目标的数目，上述数据是在单目标的情况下进行统计的。
5.7 显存占用情况
Tue Jan 18 16:06:27 +-----------------------------------------------------------------------------+| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4||-------------------------------+----------------------+----------------------+| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. || | |MIG M. ||===============================+======================+======================|| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A || 27% 43C P0 44W / 75W | 1895MiB / 3903MiB |79%Default || | | N/A |+-------------------------------+----------------------+----------------------++-----------------------------------------------------------------------------+| Processes:|| GPU GI CI PID Type Process name GPU Memory || ID ID Usage||=============================================================================|| 0 N/A N/A1529G /usr/lib/xorg/Xorg218MiB || 0 N/A N/A5928C python 1673MiB |+-----------------------------------------------------------------------------+
六、TensorRT加速目标跟踪
YoloV5+DeepSort+TensorRT 目标检测、跟踪
yolov5_deepsort_tensorrt_cpp
七、可能存在的问题
Q1: 缺少_bz2文件
File "/usr/local/lib/python3.6/bz2.py", line 23, in <module>from _bz2 import BZ2Compressor, BZ2DecompressorModuleNotFoundError: No module named '_bz2'
错误原因：缺少 _bz2.cpython-36m-x86_64-linux-gnu.so 文件解决办法：把系统自带Python3.6的“_bz2.cpython-36m-x86_64-linux-gnu.so”文件，放到Python3.8的文件夹中。如果是Python3.8版本，也可以把文件改名后，放到Python3.6的文件夹中。sudo cp /home/yichao/miniconda3/envs/compress_model/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so /usr/local/lib/python3.6/lib-dynload/
Q2: 缺少DetectMultiBackend
Traceback (most recent call last):File "track.py", line 24, in <module>from mon import DetectMultiBackendImportError: cannot import name 'DetectMultiBackend'
错误原因：博主使用的不是最新版本的yolov5，且使用的分支是v5.0，yolov5/models/common文件中暂不支持DetectMultiBackend。解决办法：下载最新版本的yolov5，切换到最新的分支v6.0。git clone /ultralytics/yolov5.gitgit checkout -b 新分支名称（创建） tag_name
Q3: CUDA错误
File "./yolov5/models/yolo.py", line 222, in fusem.conv = fuse_conv_and_bn(m.conv, m.bn) # update convFile "./yolov5/utils/torch_utils.py", line 207, in fuse_conv_and_bnfusedconv.weight.copy_(torch.mm(w_bn, w_conv).view(fusedconv.weight.shape))RuntimeError: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasCreate(handle)`
错误原因：pytorch的版本问题，博主由于粗心，安装了CPU版本的Pytorch和pytorchvision。torch-1.8.0-cp36-cp36m-linux_x86_64.whltorchvision-0.9.0-cp36-cp36m-linux_x86_64.whl解决办法：安装GPU版本的pytorch和pytorchvison。torch-1.9.0+cu102-cp36-cp36m-linux_x86_64.whltorchvision-0.10.0+cu102-cp36-cp36m-linux_x86_64.whlpytorch与pytorchvision版本对齐，参考 [Pytorch安装教程](/m0_37605642/article/details/117855911)
Q4: pytorch版本问题
File "/home/yichao/MyDocuments/Yolov5_DeepSort_Pytorch/venv/lib/python3.6/site-packages/torchvision/extension.py", line 63, in _assert_has_ops"Couldn't load custom C++ ops. This can happen if your PyTorch and "RuntimeError: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check /pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.
错误原因：博主由于粗心，安装了cpu版本的torchvision，与GPU版本的pytorch不匹配。torchvision-0.10.0-cp36-cp36m-linux_x86_64.whl解决办法：卸载pytorchvision，安装GPU版本的pytorchvison。torchvision-0.10.0+cu102-cp36-cp36m-linux_x86_64.whl
