WO2022041129A1 - 一种动物行为学记录三维捕捉装置、方法、系统及应用 - Google Patents

一种动物行为学记录三维捕捉装置、方法、系统及应用 Download PDF

Info

Publication number
WO2022041129A1
WO2022041129A1 PCT/CN2020/112156 CN2020112156W WO2022041129A1 WO 2022041129 A1 WO2022041129 A1 WO 2022041129A1 CN 2020112156 W CN2020112156 W CN 2020112156W WO 2022041129 A1 WO2022041129 A1 WO 2022041129A1
Authority
WO
WIPO (PCT)
Prior art keywords
cameras
dimensional
animal
frame
skeleton
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/112156
Other languages
English (en)
French (fr)
Inventor
陈可
黄康
韩亚宁
蔚鹏飞
王立平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Priority to PCT/CN2020/112156 priority Critical patent/WO2022041129A1/zh
Publication of WO2022041129A1 publication Critical patent/WO2022041129A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
    • A01K1/00—Housing animals; Equipment therefor
    • A01K1/02—Pigsties; Dog-kennels; Rabbit-hutches or the like
    • A01K1/03—Housing for domestic or laboratory animals
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B29/00—Combinations of cameras, projectors or photographic printing apparatus with non-photographic non-optical apparatus, e.g. clocks or weapons; Cameras having the shape of other objects
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/08—Stereoscopic photography by simultaneous recording
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering

Definitions

  • the invention relates to the field of target tracking, in particular to a three-dimensional capture device, method, system and application for animal behavior recording.
  • the prior art 3D capture devices mostly use dual cameras and triple cameras for 3D capture, but the less the number of cameras, the more difficult it is to capture in all directions, the body parts of the animals being captured are easily blocked, and the cameras are large in size and difficult to install.
  • an excessively large camera may cause the animal's alertness, which is not conducive to the normal monitoring of animal behavior, and the camera has a low frame rate, making it difficult to quickly track animal behavior.
  • the feature point matching algorithm is mostly used to obtain the corresponding feature points in the images captured by the two cameras, but this method is extremely unstable, and the number and matching positions of the matching feature points in each frame of the video may be different, which leads to the instability of the reconstructed 3D point cloud. , it is difficult to analyze this 3D point cloud.
  • the present invention provides a three-dimensional capture device, method, system and application for animal behavioral recording.
  • the specific technical scheme is as follows:
  • a three-dimensional capture device for animal behavior recording comprising a camera, a fixed frame, and a cage, wherein the cage is arranged in the fixed frame;
  • the number of the cameras is multiple, and the multiple cameras are respectively arranged at multiple positions of the fixed frame to photograph all the animals in the cage.
  • At least one of the cameras is arranged at the center of the top of the fixed frame, and the cameras face vertically downward, and the rest of the cameras are arranged around the center, and The camera is disposed obliquely downward.
  • the number of the cameras includes 5, and the cage includes a stainless steel glass mixing cage;
  • the number of the cameras including 5 is only a preferred number of the content of the present invention, and there are other preferred numbers, such as 2, 3, 4, 6, 7, etc.;
  • the stainless steel-glass blend is only one preferred material of the present invention, and there are other preferred materials, such as aluminum alloy-glass blend, wood-glass blend, rubber-glass blend, and the like.
  • the fixing frame includes a rectangular frame, and a plurality of the cameras are respectively arranged on each top corner of the upper surface of the rectangular frame;
  • the rectangular frame is only a preferred shape of the content of the present invention, and there are other preferred shapes, such as a trapezoidal frame, a conical frame, a spherical frame, etc.;
  • Arrangement on each top corner of the upper surface of the rectangular frame is only a preferred arrangement of the present invention, and there are other preferred arrangements, such as arrangement on each side of the upper surface of the rectangular frame, arrangement on all on each side of the rectangular frame perpendicular to the ground, etc.
  • the fixing frame comprises an aluminum alloy fixing frame
  • aluminum alloy is only a preferred material for the content of the present invention, and there are other preferred materials, such as stainless steel, wood, and the like.
  • a three-dimensional capture method for animal behavior records comprising the following steps:
  • the feature points are marked, and a plurality of two-dimensional skeleton points of the animal are obtained by the fusion of the marks;
  • a deep neural network to complete the tracking of multiple two-dimensional skeleton points of the animal, and then combine a plurality of the two-dimensional skeleton points in each frame to form a two-dimensional skeleton sequence.
  • the network makes the animal's two-dimensional skeleton sequence more robust and accurate when the ambient light changes, the image tone changes, and the background changes slightly.
  • the data of a plurality of the three-dimensional skeletons are read into the workspace, and a line is formed between the plurality of the three-dimensional skeletons, and displayed on the screen frame by frame.
  • the method for "controlling a plurality of the cameras to synchronously capture the activity videos of animals at different positions and angles" includes: using five of the cameras, and dividing the five cameras into four pairs , one of the described cameras is the main camera, and the other four described cameras are sub-cameras, and the 5 described cameras are controlled to synchronously shoot the activity videos of animals at different positions and angles;
  • the method of "using the triangulation algorithm to map the two-dimensional skeleton sequence combined with the calibration result into the three-dimensional space to calculate the three-dimensional skeleton” includes: using the triangulation algorithm to reconstruct the two-dimensional skeleton sequence in pairs, and for each animal's skeleton. Four spatial point positions are reconstructed from each of the feature points, and finally the least squares method is used to optimize the spatial point positions to obtain the optimal spatial point positions, and the two-dimensional skeleton sequence composed of the optimal spatial point positions The three-dimensional skeleton is obtained by combining the calibration results and mapping them into three-dimensional space.
  • the implementation method of "calibrating a plurality of the cameras” includes:
  • the cameras are automatically initialized, the calibration board includes a pattern for calibration, the size of the pattern is measured, and the cameras synchronously shoot several calibration boards at different positions and angles.
  • the patterns used for calibration include checkerboard patterns, solid circle array patterns, and the like.
  • the "obtaining multiple required animal pictures according to the preset frame rate, image quality and shooting duration" further includes:
  • the frame rate is 40-70FPS
  • the picture quality is 440*280-740*580;
  • the shooting time is 10-20 minutes.
  • a three-dimensional capture system for animal behavior records comprising: a camera, a fixed frame, and a cage, wherein the cage is arranged in the fixed frame;
  • the number of the cameras is multiple, and the multiple cameras are respectively arranged at multiple positions of the fixed frame to photograph all the animals in the cage;
  • a calibration module for calibrating a plurality of the cameras
  • the picture acquisition module is used to control a plurality of the cameras to synchronously shoot the motion videos of animals at different positions and angles, obtain the required animal pictures according to the preset frame rate, picture quality and shooting time, and analyze the multiple animal pictures.
  • the corresponding feature points between the animal pictures are marked, and a plurality of two-dimensional skeleton points of the animal are obtained through the fusion of the markings;
  • the two-dimensional skeleton extraction module is used to train a deep neural network, complete the tracking of multiple two-dimensional skeleton points of the animal, and then combine a plurality of the two-dimensional skeleton points in each frame to form a two-dimensional skeleton sequence, specifically , using the DeepLabCut toolbox to train the deep neural network, so that the animal's two-dimensional skeleton sequence has stronger robustness and accuracy when the ambient light changes, the image tone changes, and the background slightly changes.
  • the three-dimensional skeleton extraction module is used to use the triangulation algorithm to map the two-dimensional skeleton sequence combined with the calibration result into the three-dimensional space to calculate the three-dimensional skeleton. more precise;
  • the processing module is used to rotate, translate, and invert a plurality of the three-dimensional skeleton coordinates without distortion, and align them to the horizontal line, and use the vertical direction of the main camera as the Z-axis to align the Z-axis to the height of the real world , so that the features of the three-dimensional skeleton are more accurate and real;
  • the visualization module is used for reading the data of a plurality of the three-dimensional skeletons into the workspace, connecting the plurality of the three-dimensional skeletons into lines, and displaying them frame by frame on the screen.
  • the method of the "picture acquisition module” includes: using five cameras, dividing the five cameras into four pairs, one of the cameras is the main camera, and the other four cameras are the main camera.
  • the described camera is a sub-camera, and 5 described cameras are controlled to synchronously shoot active videos of animals at different positions and angles;
  • the method of the "three-dimensional skeleton extraction module” includes: using a triangulation algorithm to reconstruct the two-dimensional skeleton sequence in pairs, reconstructing four spatial point positions for each of the feature points of the animal, and finally using the least squares method. Optimizing the spatial point positions is performed to obtain the optimal spatial point positions, and the two-dimensional skeleton sequence composed of the optimal spatial point positions is mapped to the three-dimensional space in combination with the calibration results to calculate the three-dimensional skeleton.
  • the "calibration module” is further used for:
  • the cameras are automatically initialized, the calibration board includes a pattern for calibration, the size of the pattern is measured, and the cameras synchronously shoot several calibration boards at different positions and angles.
  • the patterns used for calibration include checkerboard patterns, solid circle array patterns, and the like.
  • the "picture acquisition module” further includes:
  • the frame rate is 40-70FPS
  • the picture quality is 440*280-740*580;
  • the shooting time is 10-20 minutes.
  • the three-dimensional capture system for animal ethology recording is applied in the ethology recording of non-human primates.
  • a three-dimensional capture device for animal behavior recording includes a camera, a fixed frame and a cage, the cage is arranged in the fixed frame, the number of cameras is multiple, and the multiple cameras are respectively arranged in multiple parts of the fixed frame. At least one of the cameras is set at the center of the top of the fixed frame, and the camera faces vertically downward, and the rest of the cameras are set around the center position, and the cameras are set diagonally downward; Based on the technical solution of the present invention, researchers can use multiple cameras to quickly track animal behaviors from different angles and positions, and to photograph and store various body movements of animals in an all-round, complete and synchronous manner.
  • the method includes the following steps: calibrating a plurality of cameras; Activity video, according to the preset frame rate, image quality and shooting time to obtain the required multiple animal pictures, and mark the corresponding feature points between the multiple animal pictures, and obtain multiple two-dimensional animal pictures through the fusion of marks Skeleton point; train a deep neural network to complete the tracking of multiple 2D skeleton points of animals, and then combine multiple 2D skeleton points in each frame to form a 2D skeleton sequence; use the triangulation algorithm to combine the 2D skeleton sequence with calibration
  • the result is mapped to the three-dimensional space to calculate the three-dimensional skeleton; the coordinates of multiple three-dimensional skeletons are rotated, translated, and inverted without distortion, and aligned to the horizontal line, and the vertical direction of the main camera is used as the Z-axis, and the Z-axis is aligned to the real world height; read the data of multiple
  • the 3D skeleton makes the data of the 3D skeleton more accurate; through the rotation, translation and inversion of multiple 3D skeleton coordinates without distortion, the features of the 3D skeleton are more accurate and more realistic, and are visualized in the 3D space, and then This enables researchers to use the 3D skeleton data for further analysis.
  • FIG. 1 is a diagram of a three-dimensional capture device for animal behavior recording in an embodiment
  • Fig. 2 is the flow chart of the three-dimensional capture method of animal behavior record in the embodiment
  • Fig. 3 is the concrete flow chart of multi-camera calibration in Fig. 3;
  • the three-dimensional capture device for animal behavior recording includes a camera 101 , a fixed frame 102 , a cage 103 , and the cage 103 set in the fixed frame 102;
  • the number of cameras 101 is multiple, and the multiple cameras 101 are respectively arranged at multiple positions of the fixed frame 102 to photograph the animals in the whole cage 103 .
  • At least one of the cameras 101 is arranged at the center of the top of the fixed frame 102, and the cameras 101 face vertically downward, and the other cameras 101 are arranged around the center, and the cameras 101 are arranged obliquely downward.
  • the number of cameras 101 includes five, and the cage 103 includes a stainless steel glass mixing cage;
  • the number of cameras 101 including 5 is only a preferred number of the present invention, and there are other preferred numbers, such as 2, 3, 4, 6, 7, etc.;
  • Stainless steel glass blend is only one preferred material for the content of the present invention, and there are other preferred materials, such as aluminum alloy glass blend, wood glass blend, rubber glass blend, and the like.
  • the fixed frame 102 includes a rectangular frame, and a plurality of cameras 101 are respectively arranged on each top corner of the upper surface of the rectangular frame;
  • the rectangular frame is only a preferred shape of the content of the present invention, and there are other preferred shapes, such as a trapezoidal frame, a conical frame, a spherical frame, etc.;
  • the fixing frame 102 includes an aluminum alloy fixing frame
  • aluminum alloy is only a preferred material for the content of the present invention, and there are other preferred materials, such as stainless steel, wood, and the like.
  • the three-dimensional capture method of the animal behavior record comprises the following steps:
  • Step 201 multi-camera calibration; including calibrating multiple cameras 101;
  • Step 202 Multi-camera synchronous video shooting; including controlling multiple cameras 101 to synchronously shoot motion videos of animals at different positions and angles, obtaining the required multiple animal pictures according to the preset frame rate, image quality and shooting time, and making Corresponding feature points between multiple animal pictures are marked, and multiple two-dimensional skeleton points of animals are obtained through the fusion of markings;
  • Step 203 two-dimensional skeleton extraction; including training a deep neural network, completing the tracking of multiple two-dimensional skeleton points of the animal, and then combining multiple two-dimensional skeleton points in each frame to form a two-dimensional skeleton sequence, specifically, using DeepLabCut
  • the toolbox trains deep neural networks to make animal 2D skeleton sequences more robust and accurate in response to ambient light changes, image tone changes, and slight background changes.
  • Step 204 3D skeleton reconstruction; including using the triangulation algorithm to map the 2D skeleton sequence in combination with the calibration result into the 3D space to calculate the 3D skeleton, specifically, using the SVD method to perform least squares calculation to make the data of the 3D skeleton more accurate;
  • Step 205 rotating, translating, and reversing the three-dimensional skeleton; including rotating, translating, and reversing multiple three-dimensional skeleton coordinates without distortion, and aligning them to the horizontal line, taking the vertical direction of the main camera 101 as the Z axis, and aligning the Z axis to the real
  • the height of the world makes the features of the 3D skeleton more accurate and real;
  • Step 206 3D skeleton visualization; including reading the data of the multiple 3D skeletons into the workspace, connecting the multiple 3D skeletons into lines, and displaying them frame by frame on the screen.
  • the method for "controlling multiple cameras to synchronously shoot animal videos at different positions and angles" includes: using 5 cameras, dividing the 5 cameras into four pairs, one of which is the main camera, and the remaining four
  • the camera is a sub-camera, which controls 5 cameras to synchronously shoot the activity videos of animals at different positions and angles;
  • the method of "using the triangulation algorithm to map the two-dimensional skeleton sequence combined with the calibration results into the three-dimensional space to calculate the three-dimensional skeleton” includes: using the triangulation algorithm to reconstruct the two-dimensional skeleton sequence in pairs, and reconstructing each feature point of the animal separately. Four space point positions, and finally use the least squares method to optimize the space point positions, find the optimal space point positions, and map the two-dimensional skeleton sequence composed of the optimal space point positions to the three-dimensional space for calculation. Get a 3D skeleton.
  • the implementation method of "multi-camera calibration” includes:
  • Step 301 start the camera; including starting a plurality of cameras 101;
  • Step 302 camera initialization; including automatic initialization of multiple cameras 101;
  • Step 303 Measure the size of the pattern on the calibration plate; include the pattern included on the calibration plate for calibration, and measure the size of the pattern;
  • Step 304 photographing the pattern on the calibration plate; including multiple cameras 101 synchronously photographing the calibration plate at different positions and angles for several times, obtaining multiple pictures of the calibration plate pattern, and obtaining the rotation matrix and translation vector of the multiple cameras 101, specifically Typically, patterns used for calibration include checkerboard patterns, solid circle array patterns, and the like.
  • "obtaining the required multiple animal pictures according to the preset frame rate, image quality and shooting time” further includes:
  • the frame rate is 40-70FPS
  • the picture quality is 440*280-740*580;
  • a three-dimensional capture system for animal behavior recording comprising: a camera 101, a fixing frame 102, a cage 103, and the cage 103 is arranged in the fixing frame 102;
  • the number of cameras 101 is multiple, and the multiple cameras 101 are respectively arranged at multiple positions of the fixed frame 102 to photograph the animals in the whole cage 103;
  • a calibration module for calibrating the plurality of cameras 101
  • the picture acquisition module is used to control the multiple cameras 101 to synchronously shoot the motion videos of animals at different positions and angles, obtain the required multiple animal pictures according to the preset frame rate, picture quality and shooting time, and analyze the multiple animal pictures.
  • the corresponding feature points between them are marked, and multiple two-dimensional skeleton points of the animal are obtained by the fusion of the marks;
  • the 2D skeleton extraction module is used to train a deep neural network, complete the tracking of multiple 2D skeleton points of animals, and then combine multiple 2D skeleton points in each frame to form a 2D skeleton sequence.
  • the DeepLabCut tool is used.
  • the deep neural network is trained by the box, so that the animal's two-dimensional skeleton sequence has stronger robustness and accuracy when the ambient light changes, the image tone changes, and the background changes slightly.
  • the three-dimensional skeleton extraction module is used to map the two-dimensional skeleton sequence combined with the calibration result into the three-dimensional space by using the triangulation algorithm to calculate the three-dimensional skeleton.
  • the SVD method is used to perform the least square calculation to make the data of the three-dimensional skeleton more accurate;
  • the processing module is used to rotate, translate, and invert multiple three-dimensional skeleton coordinates without distortion, and align them to the horizontal line, take the vertical direction of the main camera 101 as the Z axis, and align the Z axis to the height of the real world, so that the three-dimensional The features of the skeleton are more accurate and realistic;
  • the visualization module is used to read the data of multiple 3D skeletons into the workspace, connect the multiple 3D skeletons into lines, and display them frame by frame on the screen.
  • the method of the "picture acquisition module” includes: using 5 cameras, dividing the 5 cameras into four pairs, one of the cameras is the main camera, the other four cameras are the sub-cameras, and the 5 cameras are controlled to shoot animals synchronously Activity videos in different locations and angles;
  • the method of the "3D skeleton extraction module” includes: using the triangulation algorithm to reconstruct the 2D skeleton sequence in pairs, reconstructing four spatial point positions for each feature point of the animal, and finally using the least squares method to optimize the spatial point positions, Find the optimal spatial point position, and map the two-dimensional skeleton sequence composed of the optimal spatial point positions into the three-dimensional space to calculate the three-dimensional skeleton.
  • the "calibration module” is also used for:
  • the multiple cameras 101 are activated, the multiple cameras 101 are automatically initialized, the calibration plate includes a pattern for calibration, and the size of the pattern is measured. From the picture of the pattern, the rotation matrices and translation vectors of the plurality of cameras 101 are obtained.
  • the patterns used for calibration include checkerboard patterns, solid circle array patterns, and the like.
  • the "picture acquisition module” further includes:
  • the frame rate is 40-70FPS
  • the picture quality is 440*280-740*580;
  • the three-dimensional capture system for animal behavior recording is applied in the behavior recording of non-human primates.
  • a three-dimensional capture device for animal behavior recording includes a camera 101, a fixed frame 102, and a cage 103.
  • the cage 103 is arranged in the fixed frame 102.
  • the number of cameras 101 is multiple, and the multiple cameras 101 are respectively Set at multiple positions of the fixed frame 102 to photograph the animals in the full cage 103, at least one of the cameras 101 is set at the central position of the top of the fixed frame 102, and the camera 101 faces vertically downward, and the rest of the cameras 101 are set at Around the center position, and the cameras 101 are set obliquely downward; based on the technical solution of the present invention, researchers can use multiple cameras 101 to quickly track animal behaviors from different angles and positions, and shoot a variety of animals in an all-round and complete synchronization. Body moves and stores.
  • the method includes the following steps: calibrating a plurality of cameras 101; For the active video of the angle, obtain the required multiple animal pictures according to the preset frame rate, image quality and shooting time, and mark the corresponding feature points between the multiple animal pictures, and obtain multiple animal pictures through the fusion of the marks.
  • Two-dimensional skeleton points train a deep neural network to complete the tracking of multiple two-dimensional skeleton points of animals, and then combine multiple two-dimensional skeleton points in each frame to form a two-dimensional skeleton sequence; use the triangulation algorithm to combine the two-dimensional skeleton sequences
  • the calibration result is mapped into the three-dimensional space to obtain the three-dimensional skeleton; the coordinates of the multiple three-dimensional skeletons are rotated, translated, and inverted without distortion, and aligned to the horizontal line, and the Z axis is aligned with the vertical direction of the main camera 101 as the Z axis.
  • To the height of the real world read the data of multiple 3D skeletons into the workspace, connect multiple 3D skeletons into lines, and display them frame by frame on the screen;
  • modules in the device in the implementation scenario may be distributed in the device in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the implementation scenario with corresponding changes.
  • the modules of the above implementation scenarios may be combined into one module, or may be further split into multiple sub-modules.

Landscapes

  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Environmental Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Zoology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Animal Husbandry (AREA)
  • Health & Medical Sciences (AREA)
  • Clinical Laboratory Science (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computer Graphics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

一种动物行为学记录三维捕捉装置，包括摄像头（101）、固定框（102）、笼具（103）；笼具（103）设置在固定框（102）内；多个摄像头（101）分别设置在固定框（102）的多个位置。一种动物行为学记录三维捕捉方法，包括：对多个摄像头（101）进行定标；控制多个摄像头（101）同步拍摄动物得到动物的多个二维骨架点；再把每帧中的多个二维骨架点组合形成二维骨架序列；使用三角算法将二维骨架序列映射到三维空间中计算得到三维骨架；将三维骨架之间连成线，并在屏幕上逐帧显示。动物行为学记录三维捕捉装置、方法、系统及应用能够完整同步地拍摄动物的多种身体动作，能够稳定追踪并自定义动物身体的二维骨架，能够从多个二维骨架点中稳定重建出三维骨架点，并在三维空间中可视化表示出来。

Description

一种动物行为学记录三维捕捉装置、方法、系统及应用 技术领域
本发明涉及目标跟踪领域，具体涉及一种动物行为学记录三维捕捉装置、方法、系统及应用。
背景技术
目前，主要通过拍摄二维视频的方式记录动物的活动，研究人员通过二维视频进行分析并得出结论，而动物的身体具有多自由度，行为表现丰富且高度复杂，二维视频中动物的身体部位极易被遮挡，通过拍摄二维视频的方式记录动物所得到的数据，其准确度和可信度难以保证。
现有技术的三维捕捉装置多采用双摄像头和三摄像头进行三维捕捉，但摄像头数量越少越难以做到全方位拍摄，拍摄的动物身体部位极易被遮挡，且该摄像头尺寸较大，难以安装于笼具中，过大的摄像头可能会引起动物的警觉，不利于动物行为的正常监测,而且该摄像头的帧率较低，难以快速跟踪动物行为。
目前多采用特征点匹配算法得到两摄像头拍摄图像中对应的特征点，但这种方法极不稳定，视频中每帧图像匹配特征点数量和匹配位置可能不同，导致重建出来的三维点云不稳定，难以对此三维点云进行分析。
发明内容
为了克服现有技术的不足，本发明提供了一种动物行为学记录三维捕捉装置、方法、系统及应用，具体技术方案如下所示：
一种动物行为学记录三维捕捉装置，包括摄像头、固定框、笼具，所述笼具设置在所述固定框内；
所述摄像头的数量为多个，多个所述摄像头分别设置在所述固定框的多个位置，以拍全所述笼具内的动物。
在一个具体的实施例中，所述摄像头中的至少一个设置在所述固定框顶部的中心位置，并且所述摄像头朝向竖直向下，其余所述摄像头设置在所述中心位置的四周，且所述摄像头朝向斜向下设置。
在一个具体的实施例中，所述摄像头的数量包括5个，所述笼具包括不锈钢玻璃混合笼子；
需要说明的是，所述摄像头的数量包括5个仅是本发明内容的一种优选数量，其还有其他优选数量，例如2个、3个、4个、6个、7个等；
所述不锈钢玻璃混合仅是本发明内容的一种优选材料，其还有其他优选材料，例如铝合金玻璃混合、木头玻璃混合、橡胶玻璃混合等。
所述固定框包括矩形框架，多个所述摄像头分别设置在所述矩形框架上表面的每个顶角上；
需要说明的是，所述矩形框架仅是本发明内容的一种优选形状，其还有其他优选形状，例如梯形框架、锥形框架、球形框架等；
设置在所述矩形框架上表面的每个顶角上仅是本发明内容的一种优选设置，其还有其他优选设置，例如设置在所述矩形框架上表面的每条边上、设置在所述矩形框架与地面垂直的每条边上等。
优选地，所述固定框包括铝合金固定框；
需要说明的是，铝合金仅是本发明内容的一种优选材料，其还有其他优选材料，例如不锈钢、木头等。
一种动物行为学记录三维捕捉方法，包括以下步骤:
对多个所述摄像头进行定标；
控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张所述动物图片之间相应的特征点进行标记，通过所述标记的融合得到所述动物的多个二维骨架点；
训练深度神经网络，完成所述动物的多个二维骨架点的追踪，再把每帧 中的多个所述二维骨架点组合形成二维骨架序列，具体地，利用DeepLabCut工具箱训练深度神经网络，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度。
使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架，具体地，使用SVD方法进行最小二乘计算，使所述三维骨架的数据更加准确；
将多个所述三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头垂直的方向作为Z轴，将所述Z轴对齐到真实世界的高度，使所述三维骨架的特征更准确、更真实；
将多个所述三维骨架的数据读取到工作空间，将多个所述三维骨架之间连成线，并在屏幕上逐帧显示。
在一个具体的实施例中，所述“控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频”的方法包括：使用5个所述摄像头，将5个所述摄像头分为四对，其中一个所述摄像头为主摄像头，其余四个所述摄像头为副摄像头，控制5个所述摄像头同步拍摄动物不同位置和不同角度的活动视频；
所述“使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架”的方法包括：使用三角算法将所述二维骨架序列进行两两重建，对动物的每个所述特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
在一个具体的实施例中，所述“对多个所述摄像头进行定标”的实现方法包括：
启动多个所述摄像头，多个所述摄像头自动初始化，所述标定板上包括用于标定的图案，测量所述图案的尺寸，多个所述摄像头同步拍摄不同位 置和不同角度的标定板若干次，获得多张所述标定板图案的图片，得到多个所述摄像头的旋转矩阵和平移向量，具体地，所述用于标定的图案包括棋盘格图案、实心圆阵列图案等。
在一个具体的实施例中，所述“按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片”进一步包括：
所述帧率为40-70FPS；
所述画质为440*280-740*580；
所述拍摄时长为10-20分钟。
一种动物行为学记录三维捕捉系统，包括：摄像头、固定框、笼具，所述笼具设置在所述固定框内；
所述摄像头的数量为多个，多个所述摄像头分别设置在所述固定框的多个位置，以拍全所述笼具内的动物；
还包括：
定标模块，用于对多个所述摄像头进行定标；
图片采集模块，用于控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张所述动物图片之间相应的特征点进行标记，通过所述标记的融合得到所述动物的多个二维骨架点；
二维骨架提取模块，用于训练深度神经网络，完成所述动物的多个二维骨架点的追踪，再把每帧中的多个所述二维骨架点组合形成二维骨架序列，具体地，利用DeepLabCut工具箱训练深度神经网络，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度。
三维骨架提取模块，用于使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架，具体地，使用SVD方法进行最小二乘计算，使所述三维骨架的数据更加准确；
处理模块，用于将多个所述三维骨架坐标进行无畸变的旋转、平移、 反转，并对齐到水平线，以主摄像头垂直的方向作为Z轴，将所述Z轴对齐到真实世界的高度，使所述三维骨架的特征更准确、更真实；
可视化模块，用于将多个所述三维骨架的数据读取到工作空间，将多个所述三维骨架之间连成线，并在屏幕上逐帧显示。
在一个具体的实施例中，所述“图片采集模块”的方法包括：使用5个所述摄像头，将5个所述摄像头分为四对，其中一个所述摄像头为主摄像头，其余四个所述摄像头为副摄像头，控制5个所述摄像头同步拍摄动物不同位置和不同角度的活动视频；
所述“三维骨架提取模块”的方法包括：使用三角算法将所述二维骨架序列进行两两重建，对动物的每个所述特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
在一个具体的实施例中，所述“定标模块”还用于：
启动多个所述摄像头，多个所述摄像头自动初始化，所述标定板上包括用于标定的图案，测量所述图案的尺寸，多个所述摄像头同步拍摄不同位置和不同角度的标定板若干次，获得多张所述标定板图案的图片，得到多个所述摄像头的旋转矩阵和平移向量，具体地，所述用于标定的图案包括棋盘格图案、实心圆阵列图案等。
在一个具体的实施例中，所述“图片采集模块”还包括：
所述帧率为40-70FPS；
所述画质为440*280-740*580；
所述拍摄时长为10-20分钟。
在一个具体的实施例中，所述的动物行为学记录三维捕捉系统在非人灵长类动物行为学记录中得到应用。
本发明至少具有以下有益效果：
根据本发明提供的一种动物行为学记录三维捕捉装置，包括摄像头、固定框、笼具，笼具设置在固定框内,摄像头的数量为多个，多个摄像头分别设置在固定框的多个位置，以拍全笼具内的动物,摄像头中的至少一个设置在固定框顶部的中心位置，并且摄像头朝向竖直向下，其余摄像头设置在中心位置的四周，且摄像头朝向斜向下设置；基于本发明的技术方案，研究人员能够利用多个摄像头从不同角度和不同位置快速跟踪动物行为，全方位完整同步地拍摄动物的多种身体动作并储存。
根据本发明提供的一种使用该动物行为学记录三维捕捉装置的动物行为学记录三维捕捉方法，包括以下步骤：对多个摄像头进行定标；控制多个摄像头同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张动物图片之间相应的特征点进行标记，通过标记的融合得到动物的多个二维骨架点；训练深度神经网络，完成动物的多个二维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列；使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架；将多个三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头垂直的方向作为Z轴，将Z轴对齐到真实世界的高度；将多个三维骨架的数据读取到工作空间，将多个三维骨架之间连成线，并在屏幕上逐帧显示；
基于本发明的技术方案，通过多个摄像头的同步拍摄，动物的多种身体动作行为能够被全方位完整的拍摄并保存；对多张动物图片之间相应的特征点进行标记并融合，使动物身体的二维骨架点能够被稳定追踪并自定义；训练深度神经网络，完成动物的多个二维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度；使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架，使三维骨架的数据更加准确；通过对多个三维骨架坐标进行无畸变的旋转、 平移、反转，使三维骨架的特征更准确、更真实，并在三维空间中可视化表示出来，进而使研究人员能利用三维骨架的数据进行进一步分析。
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1是实施例中动物行为学记录三维捕捉装置图；
图2是实施例中动物行为学记录三维捕捉方法流程图；
图3是图3中多摄像头定标的具体流程图；
主要元件符号说明：
101-摄像头；102-固定框；103-笼具；201-多摄像头定标；202-多摄像头同步视频拍摄；203-二维骨架提取；204-三维骨架重建；205-三维骨架旋转平移反转；206-三维骨架可视化；301-启动摄像头；302-摄像头初始化；303-测量标定板上图案的尺寸；304-拍摄标定板上的图案。
具体实施方式
实施例
本实施例提供了一种动物行为学记录三维捕捉装置、方法、系统及应用，如图1所示，该动物行为学记录三维捕捉装置包括摄像头101、固定框102、笼具103，笼具103设置在固定框102内；
摄像头101的数量为多个，多个摄像头101分别设置在固定框102的多个位置，以拍全笼具103内的动物。
本实施例中，摄像头101中的至少一个设置在固定框102顶部的中心位置，并且摄像头101朝向竖直向下，其余摄像头101设置在中心位置的四周，且摄像头101朝向斜向下设置。
本实施例中，摄像头101的数量包括5个，笼具103包括不锈钢玻璃混合笼子；
需要说明的是，摄像头101的数量包括5个仅是本发明内容的一种优选数量，其还有其他优选数量，例如2个、3个、4个、6个、7个等；
不锈钢玻璃混合仅是本发明内容的一种优选材料，其还有其他优选材料，例如铝合金玻璃混合、木头玻璃混合、橡胶玻璃混合等。
固定框102包括矩形框架，多个摄像头101分别设置在矩形框架上表面的每个顶角上；
需要说明的是，矩形框架仅是本发明内容的一种优选形状，其还有其他优选形状，例如梯形框架、锥形框架、球形框架等；
设置在矩形框架上表面的每个顶角上仅是本发明内容的一种优选设置，其还有其他优选设置，例如设置在矩形框架上表面的每条边上、设置在矩形框架与地面垂直的每条边上等。
优选地，固定框102包括铝合金固定框；
需要说明的是，铝合金仅是本发明内容的一种优选材料，其还有其他优选材料，例如不锈钢、木头等。
如图2所示，该动物行为学记录三维捕捉方法包括以下步骤:
步骤201：多摄像头定标；包括对多个摄像头101进行定标；
步骤202：多摄像头同步视频拍摄；包括控制多个摄像头101同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张动物图片之间相应的特征点进行标记，通过标记的融合得到动物的多个二维骨架点；
步骤203：二维骨架提取；包括训练深度神经网络，完成动物的多个二 维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列，具体地，利用DeepLabCut工具箱训练深度神经网络，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度。
步骤204：三维骨架重建；包括使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架，具体地，使用SVD方法进行最小二乘计算，使三维骨架的数据更加准确；
步骤205：三维骨架旋转平移反转；包括将多个三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头101垂直的方向作为Z轴，将Z轴对齐到真实世界的高度，使三维骨架的特征更准确、更真实；
步骤206：三维骨架可视化；包括将多个三维骨架的数据读取到工作空间，将多个三维骨架之间连成线，并在屏幕上逐帧显示。
本实施例中，“控制多个摄像头同步拍摄动物不同位置和不同角度的活动视频”的方法包括：使用5个摄像头，将5个摄像头分为四对，其中一个摄像头为主摄像头，其余四个摄像头为副摄像头，控制5个摄像头同步拍摄动物不同位置和不同角度的活动视频；
“使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架”的方法包括：使用三角算法将二维骨架序列进行两两重建，对动物的每个特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
本实施例中，如图3所示，“多摄像头定标”的实现方法包括：
步骤301：启动摄像头；包括启动多个摄像头101；
步骤302：摄像头初始化；包括多个摄像头101自动初始化；
步骤303：测量标定板上图案的尺寸；包括标定板上包括用于标定的图案，测量图案的尺寸；
步骤304：拍摄标定板上的图案；包括多个摄像头101同步拍摄不同位置和不同角度的标定板若干次，获得多张标定板图案的图片，得到多个摄像头101的旋转矩阵和平移向量，具体地，用于标定的图案包括棋盘格图案、实心圆阵列图案等。
本实施例中，“按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片”进一步包括：
帧率为40-70FPS；
画质为440*280-740*580；
拍摄时长为10-20分钟。
一种动物行为学记录三维捕捉系统，包括：摄像头101、固定框102、笼具103，笼具103设置在固定框102内；
摄像头101的数量为多个，多个摄像头101分别设置在固定框102的多个位置，以拍全笼具103内的动物；
还包括：
定标模块，用于对多个摄像头101进行定标；
图片采集模块，用于控制多个摄像头101同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张动物图片之间相应的特征点进行标记，通过标记的融合得到动物的多个二维骨架点；
二维骨架提取模块，用于训练深度神经网络，完成动物的多个二维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列，具体地，利用DeepLabCut工具箱训练深度神经网络，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度。
三维骨架提取模块，用于使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架，具体地，使用SVD方法进行最小二乘计算，使三维骨架的数据更加准确；
处理模块，用于将多个三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头101垂直的方向作为Z轴，将Z轴对齐到真实世界的高度，使三维骨架的特征更准确、更真实；
可视化模块，用于将多个三维骨架的数据读取到工作空间，将多个三维骨架之间连成线，并在屏幕上逐帧显示。
本实施例中，“图片采集模块”的方法包括：使用5个摄像头，将5个摄像头分为四对，其中一个摄像头为主摄像头，其余四个摄像头为副摄像头，控制5个摄像头同步拍摄动物不同位置和不同角度的活动视频；
“三维骨架提取模块”的方法包括：使用三角算法将二维骨架序列进行两两重建，对动物的每个特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
本实施例中，“定标模块”还用于：
启动多个摄像头101，多个摄像头101自动初始化，标定板上包括用于标定的图案，测量图案的尺寸，多个摄像头101同步拍摄不同位置和不同角度的标定板若干次，获得多张标定板图案的图片，得到多个摄像头101的旋转矩阵和平移向量，具体地，用于标定的图案包括棋盘格图案、实心圆阵列图案等。
本实施例中，“图片采集模块”还包括：
帧率为40-70FPS；
画质为440*280-740*580；
拍摄时长为10-20分钟。
本实施例中，该动物行为学记录三维捕捉系统在非人灵长类动物行为学记录中得到应用。
本发明至少具有以下有益效果：
根据本发明提供的一种动物行为学记录三维捕捉装置，包括摄像头101、固定框102、笼具103，笼具103设置在固定框102内,摄像头101的数量为多个，多个摄像头101分别设置在固定框102的多个位置，以拍全笼具103内的动物,摄像头101中的至少一个设置在固定框102顶部的中心位置，并且摄像头101朝向竖直向下，其余摄像头101设置在中心位置的四周，且摄像头101朝向斜向下设置；基于本发明的技术方案，研究人员能够利用多个摄像头101从不同角度和不同位置快速跟踪动物行为，全方位完整同步地拍摄动物的多种身体动作并储存。
根据本发明提供的一种使用该动物行为学记录三维捕捉装置的动物行为学记录三维捕捉方法，包括以下步骤：对多个摄像头101进行定标；控制多个摄像头101同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张动物图片之间相应的特征点进行标记，通过标记的融合得到动物的多个二维骨架点；训练深度神经网络，完成动物的多个二维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列；使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架；将多个三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头101垂直的方向作为Z轴，将Z轴对齐到真实世界的高度；将多个三维骨架的数据读取到工作空间，将多个三维骨架之间连成线，并在屏幕上逐帧显示；
基于本发明的技术方案，通过多个摄像头101的同步拍摄，动物的多种身体动作行为能够被全方位完整的拍摄并保存；对多张动物图片之间相应的特征点进行标记并融合，使动物身体的二维骨架点能够被稳定追踪并自定义；训练深度神经网络，完成动物的多个二维骨架点的追踪，再把每帧中的多个二维骨架点组合形成二维骨架序列，使动物的二维骨架序列在环境光变化、图像色调变化、背景微变时，具有更强的鲁棒性和精度；使用三角算法将二维骨架序列结合定标结果映射到三维空间中计算得到三维 骨架，使三维骨架的数据更加准确；通过对多个三维骨架坐标进行无畸变的旋转、平移、反转，使三维骨架的特征更准确、更真实，并在三维空间中可视化表示出来，进而使研究人员能利用三维骨架的数据进行进一步分析。
本领域技术人员可以理解附图只是一个优选实施场景的示意图，附图中的模块或流程并不一定是实施本发明所必须的。
本领域技术人员可以理解实施场景中的装置中的模块可以按照实施场景描述进行分布于实施场景的装置中，也可以进行相应变化位于不同于本实施场景的一个或多个装置中。上述实施场景的模块可以合并为一个模块，也可以进一步拆分成多个子模块。
上述本发明序号仅仅为了描述，不代表实施场景的优劣。
以上公开的仅为本发明的几个具体实施场景，但是，本发明并非局限于此，任何本领域的技术人员能思之的变化都应落入本发明的保护范围。

Claims (12)

  1. 一种动物行为学记录三维捕捉装置，其特征在于：包括摄像头、固定框、笼具，所述笼具设置在所述固定框内；
    所述摄像头的数量为多个，多个所述摄像头分别设置在所述固定框的多个位置，以拍全所述笼具内的动物。
  2. 根据权利要求1所述的动物行为学记录三维捕捉装置，其特征在于：所述摄像头中的至少一个设置在所述固定框顶部的中心位置，并且所述摄像头朝向竖直向下，其余所述摄像头设置在所述中心位置的四周，且所述摄像头朝向斜向下设置。
  3. 根据权利要求1所述的动物行为学记录三维捕捉装置，其特征在于：所述摄像头的数量包括5个，所述笼具包括不锈钢玻璃混合笼子；
    所述固定框包括矩形框架，多个所述摄像头分别设置在所述矩形框架上表面的每个顶角上；
    优选地，所述固定框包括铝合金固定框。
  4. 一种动物行为学记录三维捕捉方法，其特征在于，使用权利要求1或2所述的动物行为学记录三维捕捉装置，包括以下步骤：
    对多个所述摄像头进行定标；
    控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张所述动物图片之间相应的特征点进行标记，通过所述标记的融合得到所述动物的多个二维骨架点；
    训练深度神经网络，完成所述动物的多个二维骨架点的追踪，再把每帧中的多个所述二维骨架点组合形成二维骨架序列；
    使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架；
    将多个所述三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头垂直的方向作为Z轴，将所述Z轴对齐到真实世界的高度；
    将多个所述三维骨架的数据读取到工作空间，将多个所述三维骨架之间连成线，并在屏幕上逐帧显示。
  5. 根据权利要求4所述的动物行为学记录三维捕捉方法，其特征在于，所述“控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频”的方法包括：使用5个所述摄像头，将5个所述摄像头分为四对，其中一个所述摄像头为主摄像头，其余四个所述摄像头为副摄像头，控制5个所述摄像头同步拍摄动物不同位置和不同角度的活动视频；
    所述“使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架”的方法包括：使用三角算法将所述二维骨架序列进行两两重建，对动物的每个所述特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
  6. 根据权利要求4所述的动物行为学记录三维捕捉方法，其特征在于，所述“对多个所述摄像头进行定标”的实现方法包括：
    启动多个所述摄像头，多个所述摄像头自动初始化，所述标定板上包括用于标定的图案，测量所述图案的尺寸，多个所述摄像头同步拍摄不同位置和不同角度的标定板若干次，获得多张所述标定板图案的图片，得到多个所述摄像头的旋转矩阵和平移向量。
  7. 根据权利要求4所述的动物行为学记录三维捕捉方法，其特征在于，所述“按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片”进一步包括：
    所述帧率为40-70FPS；
    所述画质为440*280-740*580；
    所述拍摄时长为10-20分钟。
  8. 一种动物行为学记录三维捕捉系统，其特征在于：
    包括摄像头、固定框、笼具，所述笼具设置在所述固定框内；
    所述摄像头的数量为多个，多个所述摄像头分别设置在所述固定框的多个位置，以拍全所述笼具内的动物；
    还包括：
    定标模块，用于对多个所述摄像头进行定标；
    图片采集模块，用于控制多个所述摄像头同步拍摄动物不同位置和不同角度的活动视频，按照预设的帧率、画质和拍摄时长得到所需要的多张动物图片，并对多张所述动物图片之间相应的特征点进行标记，通过所述标记的融合得到所述动物的多个二维骨架点；
    二维骨架提取模块，用于训练深度神经网络，完成所述动物的多个二维骨架点的追踪，再把每帧中的多个所述二维骨架点组合形成二维骨架序列；
    三维骨架提取模块，用于使用三角算法将所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架；
    处理模块，用于将多个所述三维骨架坐标进行无畸变的旋转、平移、反转，并对齐到水平线，以主摄像头垂直的方向作为Z轴，将所述Z轴对齐到真实世界的高度；
    可视化模块，用于将多个所述三维骨架的数据读取到工作空间，将多个所述三维骨架之间连成线，并在屏幕上逐帧显示。
  9. 根据权利要求8所述的动物行为学记录三维捕捉系统，其特征在于，所述“图片采集模块”的方法包括：使用5个所述摄像头，将5个所述摄像头分为四对，其中一个所述摄像头为主摄像头，其余四个所述摄像头为副摄像头，控制5个所述摄像头同步拍摄动物不同位置和不同角度的活动 视频；
    所述“三维骨架提取模块”的方法包括：使用三角算法将所述二维骨架序列进行两两重建，对动物的每个所述特征点分别重建出四个空间点位置，最后使用最小二乘法进行优化空间点位置，求出最优的空间点位置，将各最优的空间点位置组成的所述二维骨架序列结合定标结果映射到三维空间中计算得到三维骨架。
  10. 根据权利要求8所述的动物行为学记录三维捕捉系统，其特征在于，所述“定标模块”还用于：
    启动多个所述摄像头，多个所述摄像头自动初始化，所述标定板上包括用于标定的图案，测量所述图案的尺寸，多个所述摄像头同步拍摄不同位置和不同角度的标定板若干次，获得多张所述标定板图案的图片，得到多个所述摄像头的旋转矩阵和平移向量。
  11. 根据权利要求8所述的动物行为学记录三维捕捉系统，其特征在于，所述“图片采集模块”还包括：
    所述帧率为40-70FPS；
    所述画质为440*280-740*580；
    所述拍摄时长为10-20分钟。
  12. 如权利要求8-11任一项所述的动物行为学记录三维捕捉系统在非人灵长类动物行为学记录中的应用。
PCT/CN2020/112156 2020-08-28 2020-08-28 一种动物行为学记录三维捕捉装置、方法、系统及应用 Ceased WO2022041129A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/112156 WO2022041129A1 (zh) 2020-08-28 2020-08-28 一种动物行为学记录三维捕捉装置、方法、系统及应用

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/112156 WO2022041129A1 (zh) 2020-08-28 2020-08-28 一种动物行为学记录三维捕捉装置、方法、系统及应用

Publications (1)

Publication Number Publication Date
WO2022041129A1 true WO2022041129A1 (zh) 2022-03-03

Family

ID=80352506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112156 Ceased WO2022041129A1 (zh) 2020-08-28 2020-08-28 一种动物行为学记录三维捕捉装置、方法、系统及应用

Country Status (1)

Country Link
WO (1) WO2022041129A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114858802A (zh) * 2022-07-05 2022-08-05 天津大学 一种织物多尺度图像采集方法与装置
WO2025117779A1 (en) * 2023-11-28 2025-06-05 Steve Miller Animal caging system with complete line of sight

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005001768A1 (en) * 2003-06-06 2005-01-06 Psychogenics, Inc. System and method for assessing motor and locomotor deficits and recovery therefrom
CN203881749U (zh) * 2014-02-12 2014-10-15 鄢俊安 多功能动物旷场实验装置
CN105987685A (zh) * 2016-07-04 2016-10-05 西北农林科技大学 一种基于双目视觉的昆虫行为学研究辅助系统
GB2559461A (en) * 2016-12-05 2018-08-08 Equi Poise Ltd Motion monitoring system
CN109588341A (zh) * 2019-01-21 2019-04-09 济宁医学院 一种行为学检测专用啮齿类动物条件性恐惧反应检测笼盒
US20190139297A1 (en) * 2017-11-07 2019-05-09 Microsoft Technology Licensing, Llc 3d skeletonization using truncated epipolar lines
CN209089644U (zh) * 2018-10-19 2019-07-12 宁波大学 动物行为箱
CN110348371A (zh) * 2019-07-08 2019-10-18 叠境数字科技（上海）有限公司 人体三维动作自动提取方法
CN110915669A (zh) * 2019-12-05 2020-03-27 中国科学院深圳先进技术研究院 动物进食行为与神经元功能监测同步化装置
CN111105486A (zh) * 2019-12-07 2020-05-05 东南大学 一种多视角小白鼠动态三维重建方法
CN111476883A (zh) * 2020-03-30 2020-07-31 清华大学 多视角无标记动物的三维姿态轨迹重建方法及装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005001768A1 (en) * 2003-06-06 2005-01-06 Psychogenics, Inc. System and method for assessing motor and locomotor deficits and recovery therefrom
CN203881749U (zh) * 2014-02-12 2014-10-15 鄢俊安 多功能动物旷场实验装置
CN105987685A (zh) * 2016-07-04 2016-10-05 西北农林科技大学 一种基于双目视觉的昆虫行为学研究辅助系统
GB2559461A (en) * 2016-12-05 2018-08-08 Equi Poise Ltd Motion monitoring system
US20190139297A1 (en) * 2017-11-07 2019-05-09 Microsoft Technology Licensing, Llc 3d skeletonization using truncated epipolar lines
CN209089644U (zh) * 2018-10-19 2019-07-12 宁波大学 动物行为箱
CN109588341A (zh) * 2019-01-21 2019-04-09 济宁医学院 一种行为学检测专用啮齿类动物条件性恐惧反应检测笼盒
CN110348371A (zh) * 2019-07-08 2019-10-18 叠境数字科技（上海）有限公司 人体三维动作自动提取方法
CN110915669A (zh) * 2019-12-05 2020-03-27 中国科学院深圳先进技术研究院 动物进食行为与神经元功能监测同步化装置
CN111105486A (zh) * 2019-12-07 2020-05-05 东南大学 一种多视角小白鼠动态三维重建方法
CN111476883A (zh) * 2020-03-30 2020-07-31 清华大学 多视角无标记动物的三维姿态轨迹重建方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114858802A (zh) * 2022-07-05 2022-08-05 天津大学 一种织物多尺度图像采集方法与装置
WO2025117779A1 (en) * 2023-11-28 2025-06-05 Steve Miller Animal caging system with complete line of sight

Similar Documents

Publication Publication Date Title
CN103236076B (zh) 基于激光影像的物体三维模型重建系统及方法
TWI530909B (zh) 影像合成系統及方法
CN108830906B (zh) 一种基于虚拟双目视觉原理的摄像机参数自动标定方法
CN113252309A (zh) 一种用于近眼显示设备的测试方法、测试装置及存储介质
CN106887023A (zh) 用于双目摄像机标定的标定板及其标定方法和标定系统
CN106910221B (zh) 一种全局标定的方法及装置
JP7657308B2 (ja) シーンの３次元モデルを生成するための方法、装置、およびシステム
CN111144356B (zh) 一种远程教学的教师视线跟随方法及装置
CN112907647B (zh) 一种基于固定式单目摄像机的三维空间尺寸测量方法
CN112101146B (zh) 一种动物行为学记录三维捕捉装置、方法、系统及应用
CN108108021A (zh) 人眼跟踪系统外参数校正治具及校正方法
CN106998430B (zh) 基于多相机的360度视频回放方法
WO2022041129A1 (zh) 一种动物行为学记录三维捕捉装置、方法、系统及应用
WO2024016828A3 (zh) 一种基于虚拟相机的图像采集方法和相关装置
CN107066975B (zh) 基于深度传感器的视频识别与跟踪系统及其方法
JP2016063394A (ja) 投影画像生成装置、投影画像生成方法および投影画像生成プログラム
CN108320311A (zh) 一种能够自动纠偏的ar沙盘校准模型计算方法
Harjoko et al. Low-cost 3D surface reconstruction using Stereo camera for small object
CN209231985U (zh) 一种基于虚拟双目视觉原理的摄像机参数自动标定装置
CN114820811B (zh) 一种基于运动球体的同步相机的基础矩阵求解及标定方法
CN113347410B (zh) 一种利用陀螺仪辅助人眼追踪的3d显示方法及装置
Chen et al. Dior: dataset for indoor-outdoor reidentification long range 3d/2d skeleton gait collection pipeline, semi-automated gait keypoint labeling and baseline evaluation methods
CN110711369B (zh) 一种绕杆训练考核系统及其考核数据获取方法
CN211462004U (zh) 一种绕杆训练考核系统
CN107094232A (zh) 全景视频直播服务装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20950799

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20950799

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10.02.2023)