MonkeyTrail: A scalable video-based method for tracking macaque movement trajectory in daily living cages

Meng-Shi Liu; Jin-Quan Gao; Gu-Yue Hu; Guang-Fu Hao; Tian-Zi Jiang; Chen Zhang; Shan Yu

doi:10.24272/j.issn.2095-8137.2021.353

Volume 43 Issue 3

May 2022

Turn off MathJax

Article Contents

Article Navigation > Zoological Research > 2022 > 43(3): 343-351

Meng-Shi Liu, Jin-Quan Gao, Gu-Yue Hu, Guang-Fu Hao, Tian-Zi Jiang, Chen Zhang, Shan Yu. MonkeyTrail: A scalable video-based method for tracking macaque movement trajectory in daily living cages. Zoological Research, 2022, 43(3): 343-351. doi: 10.24272/j.issn.2095-8137.2021.353

Citation:

Meng-Shi Liu, Jin-Quan Gao, Gu-Yue Hu, Guang-Fu Hao, Tian-Zi Jiang, Chen Zhang, Shan Yu. MonkeyTrail: A scalable video-based method for tracking macaque movement trajectory in daily living cages. Zoological Research, 2022, 43(3): 343-351. doi: 10.24272/j.issn.2095-8137.2021.353

Citation:

Meng-Shi Liu, Jin-Quan Gao, Gu-Yue Hu, Guang-Fu Hao, Tian-Zi Jiang, Chen Zhang, Shan Yu. MonkeyTrail: A scalable video-based method for tracking macaque movement trajectory in daily living cages. Zoological Research, 2022, 43(3): 343-351. doi: 10.24272/j.issn.2095-8137.2021.353

PDF( 8969 KB)

MonkeyTrail: A scalable video-based method for tracking macaque movement trajectory in daily living cages

doi: 10.24272/j.issn.2095-8137.2021.353

Meng-Shi Liu^{1, 2, 3},
Jin-Quan Gao^{4, 5},
Gu-Yue Hu^{1, 2, 3},
Guang-Fu Hao^{1, 2, 3},
Tian-Zi Jiang^{1, 2, 3, 6},
Chen Zhang^{7
,
,},
Shan Yu^{1, 2, 3
,
,}

1.
Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
2.
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
3.
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 101408, China
4.
Technology Management Center, SAFE Pharmaceutical Technology Co., Ltd., Beijing 100176, China
5.
Model R&D Center, Beijing Life Biosciences Co., Ltd., Beijing 100176, China
6.
Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, Sichuan 611731, China
7.
Department of Neurobiology, School of Basic Medical Sciences, Beijing Key Laboratory of Neural Regeneration and Repair, and Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing 100069, China

Funds: This work was supported by the National Key Research and Development Program of China (2017YFA0105203, 2017YFA0105201), National Science Foundation of China (31771076, 81925011), Strategic Priority Research Program of the Chinese Academy of Sciences (CAS) (XDB32040201), Beijing Academy of Artificial Intelligence, and Key-Area Research and Development Program of Guangdong Province (2019B030335001)

More Information

Author Bio:
School of Computing, National University of Singapore, Singapore 119077, Singapore
Corresponding author: E-mail: czhang@ccmu.edu.cn; shan.yu@nlpr.ia.ac.cn
Received Date: 2022-01-19
Accepted Date: 2022-03-17

Published Online: 2022-03-17

Publish Date: 2022-05-18

Abstract

Abstract

Behavioral analysis of macaques provides important experimental evidence in the field of neuroscience. In recent years, video-based automatic animal behavior analysis has received widespread attention. However, methods capable of extracting and analyzing daily movement trajectories of macaques in their daily living cages remain underdeveloped, with previous approaches usually requiring specific environments to reduce interference from occlusion or environmental change. Here, we introduce a novel method, called MonkeyTrail, which satisfies the above requirements by frequently generating virtual empty backgrounds and using background subtraction to accurately obtain the foreground of moving animals. The empty background is generated by combining the frame difference method (FDM) and deep learning-based model (YOLOv5). The entire setup can be operated with low-cost hardware and can be applied to the daily living environments of individually caged macaques. To test MonkeyTrail performance, we labeled a dataset containing >8 000 video frames with the bounding boxes of macaques under various conditions as ground-truth. Results showed that the tracking accuracy and stability of MonkeyTrail exceeded that of two deep learning-based methods (YOLOv5 and Single-Shot MultiBox Detector), traditional frame difference method, and naïve background subtraction method. Using MonkeyTrail to analyze long-term surveillance video recordings, we successfully assessed changes in animal behavior in terms of movement amount and spatial preference. Thus, these findings demonstrate that MonkeyTrail enables low-cost, large-scale daily behavioral analysis of macaques.
- Movement trajectory tracking,
- Video-based behavioral analyses,
- Background subtraction,
- Virtual empty background,
- Occlusion

FullText(HTML)

References(34)

References

[1]	Bala PC, Eisenreich BR, Yoo SBM, Hayden BY, Park HS, Zimmermann J. 2020. Automated markerless pose estimation in freely moving macaques with OpenMonkeyStudio. Nature Communications, 11(1): 4560. doi: 10.1038/s41467-020-18441-5
[2]	Ballesta S, Reymond G, Pozzobon M, Duhamel JR. 2014. A real-time 3D video tracking system for monitoring primate groups. Journal of Neuroscience Methods, 234: 147−152. doi: 10.1016/j.jneumeth.2014.05.022
[3]	Bateson M, Martin PR. 2021. Measuring Behaviour: an Introductory Guide. 4^th ed. Cambridge: Cambridge University Press.
[4]	Beckman D, Morrison JH. 2021. Towards developing a rhesus monkey model of early Alzheimer's disease focusing on women's health. American Journal of Primatology, 83(11): e23289.
[5]	Bezard E, Dovero S, Prunier C, Ravenscroft P, Chalon S, Guilloteau D, et al. 2001. Relationship between the appearance of symptoms and the level of nigrostriatal degeneration in a progressive 1-methyl-4-phenyl-1, 2, 3, 6-tetrahydropyridine-lesioned macaque model of Parkinson's disease. Journal of Neuroscience, 21(17): 6853−6861. doi: 10.1523/JNEUROSCI.21-17-06853.2001
[6]	Caiola M, Pittard D, Wichmann T, Galvan A. 2019. Quantification of movement in normal and parkinsonian macaques using video analysis. Journal of Neuroscience Methods, 322: 96−102. doi: 10.1016/j.jneumeth.2019.05.001
[7]	Chen YC, Yu JH, Niu YY, Qin DD, Liu HL, Li G, et al. 2017. Modeling rett syndrome using TALEN-edited MECP2 mutant cynomolgus monkeys. Cell, 169(5): 945−955.E10. doi: 10.1016/j.cell.2017.04.035
[8]	Francisco FA, Nührenberg P, Jordan AL. 2019. A low-cost, open-source framework for tracking and behavioural analysis of animals in aquatic ecosystems. BioRxiv: 571232.
[9]	Gibbs RA, Rogers J, Katze MG, Bumgarner R, Weinstock GM, Mardis ER, et al. 2007. Evolutionary and biomedical insights from the rhesus macaque genome. Science, 316(5822): 222−234. doi: 10.1126/science.1139247
[10]	Gonzalez RC, Woods RE. 2002. Digital Image Processing. 2^nd ed. Prentice Hall: Upper Saddle River.
[11]	Graving JM, Chae D, Naik H, Li L, Koger B, Costelloe BR, et al. 2019. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. Elife, 8: e47994. doi: 10.7554/eLife.47994
[12]	Hashimoto T, Izawa Y, Yokoyama H, Kato T, Moriizumi T. 1999. A new video/computer method to measure the amount of overall movement in experimental animals (two-dimensional object-difference method). Journal of Neuroscience Methods, 91(1-2): 115−122. doi: 10.1016/S0165-0270(99)00082-5
[13]	Hu GY, Cui B, Yu S. 2020a. Joint learning in the spatio-temporal and frequency domains for skeleton-based action recognition. IEEE Transactions on Multimedia, 22(9): 2207−2220. doi: 10.1109/TMM.2019.2953325
[14]	Hu GY, Cui B, He Y, Yu S. 2020b. Progressive relation learning for group activity recognition. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 977–986.
[15]	Jocher G. 2021. YOLOv5.https://github.com/ultralytics/yolov5.
[16]	Johansson G. 1973. Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14(2): 201−211.
[17]	Krakauer JW, Ghazanfar AA, Gomez-Marin A, Maciver MA, Poeppel D. 2017. Neuroscience needs behavior: correcting a reductionist bias. Neuron, 93(3): 480−490. doi: 10.1016/j.neuron.2016.12.041
[18]	Lehner PN. 1987. Design and execution of animal behavior research: an overview. Journal of Animal Science, 65(5): 1213−1219. doi: 10.2527/jas1987.6551213x
[19]	Lind NM, Vinther M, Hemmingsen RP, Hansen AK. 2005. Validation of a digital video tracking system for recording pig locomotor behaviour. Journal of Neuroscience Methods, 143(2): 123−132. doi: 10.1016/j.jneumeth.2004.09.019
[20]	Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, et al. 2016a. SSD: single shot multibox detector. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer, 21–37.
[21]	Liu Z, Li X, Zhang JT, Cai YJ, Cheng TL, Cheng C, et al. 2016b. Autism-like behaviours and germline transmission in transgenic monkeys overexpressing MeCP2. Nature, 530(7588): 98−102. doi: 10.1038/nature16533
[22]	Mathis A, Mamidanna P, Cury KM, Abe T, Murthy VN, Mathis MW, et al. 2018. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 21(9): 1281−1289. doi: 10.1038/s41593-018-0209-y
[23]	Mathis A, Schneider S, Lauer J, Mathis MW. 2020. A primer on motion capture with deep learning: principles, pitfalls, and perspectives. Neuron, 108(1): 44−65. doi: 10.1016/j.neuron.2020.09.017
[24]	Nice M M. 1954. Reviewed work: The Herring Gull's World. A study of the social behaviour of birds by Niko Tinbergen. Bird-Banding, 25(2): 81−82. doi: 10.2307/4510469
[25]	Pandya JD, Grondin R, Yonutas HM, Haghnazar H, Gash DM, Zhang ZM, et al. 2015. Decreased mitochondrial bioenergetics and calcium buffering capacity in the basal ganglia correlates with motor deficits in a nonhuman primate model of aging. Neurobiology of Aging, 36(5): 1903−1913. doi: 10.1016/j.neurobiolaging.2015.01.018
[26]	Redmon J, Divvala S, Girshick R, Farhadi A. 2016. You only look once: Unified, real-time object detection. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 779–788.
[27]	Togasaki DM, Hsu A, Samant M, Farzan B, DeLanney LE, Langston JW, et al. 2005. The Webcam system: a simple, automated, computer-based video system for quantitative measurement of movement in nonhuman primates. Journal of Neuroscience Methods, 145(1-2): 159−166. doi: 10.1016/j.jneumeth.2004.12.010
[28]	Tzutalin. 2015. LabelImg.https://github.com/tzutalin/labelImg.
[29]	Ueno M, Hayashi H, Kabata R, Terada K, Yamada K. 2019. Automatically detecting and tracking free-ranging Japanese macaques in video recordings with deep learning and particle filters. Ethology, 125(5): 332−340. doi: 10.1111/eth.12851
[30]	Walton A, Branham A, Gash DM, Grondin R. 2006. Automated video analysis of age-related motor deficits in monkeys using EthoVision. Neurobiology of Aging, 27(10): 1477−1483. doi: 10.1016/j.neurobiolaging.2005.08.003
[31]	Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JM, Pashkovski SL, et al. 2015. Mapping sub-second structure in mouse behavior. Neuron, 88(6): 1121−1135. doi: 10.1016/j.neuron.2015.11.031
[32]	Wu Y, Lim J, Yang MH. 2013. Online object tracking: A benchmark. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2411–2418
[33]	Yabumoto T, Yoshida F, Miyauchi H, Baba K, Tsuda H, Ikenaka K, et al. 2019. MarmoDetector: a novel 3D automated system for the quantitative assessment of marmoset behavior. Journal of Neuroscience Methods, 322: 23−33. doi: 10.1016/j.jneumeth.2019.03.016
[34]	Yao Y, Jafarian Y, Park HS. 2019. MONET: multiview semi-supervised keypoint detection via epipolar divergence. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 753–762.