CN107818574B - Fish shoal three-dimensional tracking method based on skeleton analysis - Google Patents
Fish shoal three-dimensional tracking method based on skeleton analysis Download PDFInfo
- Publication number
- CN107818574B CN107818574B CN201710914852.5A CN201710914852A CN107818574B CN 107818574 B CN107818574 B CN 107818574B CN 201710914852 A CN201710914852 A CN 201710914852A CN 107818574 B CN107818574 B CN 107818574B
- Authority
- CN
- China
- Prior art keywords
- view
- target
- skeleton
- points
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 241000251468 Actinopterygii Species 0.000 claims abstract description 22
- 238000001514 detection method Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/285—Analysis of motion using a sequence of stereo image pairs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/38—Registration of image sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fish school three-dimensional tracking method based on skeleton analysis, which relates to the technical field of information. According to the fish school three-dimensional tracking method based on skeleton analysis, effective three-dimensional tracking can be carried out on the fish school target only by two cameras, and the method not only has high accuracy, but also has high tracking speed.
Description
Technical Field
The invention relates to the technical field of information, in particular to a fish school three-dimensional tracking method based on skeleton analysis.
Background
Reference Qian Z M, Chen Y Q. feature point based 3D tracking of multiple fish from multiple-view images [ J ]. ploS one, 2017, 12 (6): e0180254 provides a three-dimensional fish school tracking method based on three views. According to the method, firstly, a target in multiple views is simplified into a characteristic point representation by utilizing skeleton analysis, then, according to an obtained characteristic point model, the tracking in a top view direction is taken as a main reference, the tracking in two side view directions is taken as a reference, the target is matched and associated, and finally, the motion track of the target in a three-dimensional space is obtained.
In the tracking method of the reference, when the top view has occlusion, the method uses the tracking results of two side view directions to correlate the targets before and after the occlusion, so that the tracking performance can be improved and the tracking result is more reliable, but at the expense of the tracking efficiency, if the target of the top view has no occlusion, the tracking in the side view is not necessary.
Disclosure of Invention
The invention aims to solve the technical problem of providing a fish school three-dimensional tracking method based on skeleton analysis, which mainly tracks in the top view direction, does not use the detection result in the side view direction for tracking, only uses the detection result in the side view direction for stereo matching with the top view tracking result, reduces the complexity of the tracking method and improves the tracking efficiency.
In order to achieve the above purpose, the present invention provides the following technical solutions:
the fish school three-dimensional tracking method based on skeleton analysis comprises the following steps:
(1) estimating the motion direction of the target by using a plurality of points near the skeleton endpoint, assuming the skeleton endpoint to be p (x, y), and defining the endpoint section es as n skeleton points es { (x) of the adjacent endpointi,yi) I 1., n }, the direction of the end point segment may be calculated according to the least square method,
a characteristic point F (p, theta) formed by combining the skeleton endpoint p and the endpoint section direction theta can represent the target;
(2) in the head and tail characteristic points of the target, the tail characteristic point of the target in a top view can be removed according to the asymmetry of the shape, and the head and tail characteristic points are still reserved in the side view direction;
(3) and carrying out data association on the feature points between adjacent frames of the top view to obtain a top view two-dimensional tracking track. Next, stereo matching is carried out by using the top-view tracking track and the characteristic points of the target in the side-view direction, so that the position of the target in a three-dimensional space can be obtained, and the stereo matching is completed by using motion continuity in order to solve the problem of uncertainty of the stereo matching;
suppose thatAndrepresenting a feature point in top and side views, respectively, if under epipolar constraint,if there are k possible matching candidate feature points in the side view, then the motion continuity constraint is defined as follows:
wherein,feature points representing time t-1 and top viewCharacteristic points of the matching side view. pc (personal computer)maxAnd dcmaxRespectively representing the maximum movement distance and the maximum deflection angle of the adjacent frame objects,andrespectively represent the feature pointsAndand w and (1-w) represent the weight of the position and the direction in the cost function, respectively. The above expression indicates that, of the k candidate feature points in the side view, a feature point having the best motion continuity with the matching point at the previous time is selected as the matching point at the current time.
The beneficial effect of adopting above technical scheme is: the fish school three-dimensional tracking method based on skeleton analysis mainly tracks in the top view direction, detection results in the side view direction are not used for tracking, and only are used for carrying out three-dimensional matching with top view tracking results, so that the complexity of the tracking method is reduced, and the tracking efficiency is improved. According to the fish school three-dimensional tracking method based on skeleton analysis, effective three-dimensional tracking can be carried out on the fish school target only by two cameras, and the method not only has high accuracy, but also has high tracking speed.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 is a flow chart of fish school three-dimensional tracking based on skeletal analysis;
FIG. 2 is a top view of stereo matching based on motion continuity;
fig. 3 is a perspective matching side view based on motion continuity.
Detailed Description
The following describes a preferred embodiment of the fish school three-dimensional tracking method based on skeleton analysis according to the present invention in detail with reference to the accompanying drawings.
Fig. 1, fig. 2 and fig. 3 show a specific embodiment of the fish school three-dimensional tracking method based on skeleton analysis of the present invention:
the fish school three-dimensional tracking method based on skeleton analysis mainly tracks in the top view direction, detection results in the side view direction are not used for tracking, and only are used for carrying out three-dimensional matching with top view tracking results, so that complexity of the tracking method is reduced, and tracking efficiency is improved. Fig. 1 shows a flow chart of the proposed method. Due to the motion region segmentation and skeleton extraction method and the documents Qian Z M, Chen Y Q.feature point based 3D tracking of multiple fish from multiple-view images [ J ]. ploS one, 2017, 12 (6): e0180254 proposes a three-view-based fish shoal three-dimensional tracking method to keep consistency, and the method is not described here.
The fish school three-dimensional tracking method based on skeleton analysis comprises the following steps:
(1) estimating the motion direction of the target by using a plurality of points near the skeleton endpoint, assuming the skeleton endpoint to be p (x, y), and defining the endpoint section es as n skeleton points es { (x) of the adjacent endpointi,yi) I 1., n }, the direction of the end point segment may be calculated according to the least square method,
a characteristic point F (p, theta) formed by combining the skeleton endpoint p and the endpoint section direction theta can represent the target;
this representation has the following advantages: (1) the data volume is small. Targets in different view directions can be effectively represented only by using two points with directions, so that the tracking difficulty is greatly reduced; (2) the shielding processing capacity is strong. Most of the shielding targets can be effectively represented, and the accuracy of shielding tracking is improved.
(2) In the head and tail characteristic points of the target, the tail characteristic point of the target in a top view can be removed according to the asymmetry of the shape, and the head and tail characteristic points are still reserved in the side view direction;
(3) and carrying out data association on the feature points between adjacent frames of the top view to obtain a top view two-dimensional tracking track. Next, stereo matching is carried out by using the top-view tracking track and the characteristic points of the target in the side-view direction, so that the position of the target in a three-dimensional space can be obtained, and the stereo matching is completed by using motion continuity in order to solve the problem of uncertainty of the stereo matching;
suppose thatAndrepresenting a feature point in top and side views, respectively, if under epipolar constraint,if there are k possible matching candidate feature points in the side view, then the motion continuity constraint is defined as follows:
wherein,feature points representing time t-1 and top viewCharacteristic points of the matching side view. pc (personal computer)maxAnd dcmaxRespectively representing the maximum movement distance and the maximum deflection angle of the adjacent frame objects,andrespectively represent the feature pointsAndw and (1-w) respectively represent the weight of the position and the direction in the cost function, and the above formula shows that, in the k candidate feature points in the side view, the feature point having the best motion continuity with the matching point at the previous moment is selected as the matching point at the current moment.
Fig. 2 and 3 show an example of stereo matching. The dashed arrows in fig. 2 and 3 indicate polar lines. Target i in the top view of FIG. 2t In the side view of FIG. 3K candidate matching targets exist on the corresponding polar line, and i is selectedtThe target for which the matching point at the previous time has the best motion continuity is taken as the matching target at the current time.
According to the fish school three-dimensional tracking method based on skeleton analysis, effective three-dimensional tracking can be carried out on the fish school target only by two cameras, and the method not only has high accuracy, but also has high tracking speed.
The above is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, many variations and modifications can be made without departing from the inventive concept of the present invention, which falls into the protection scope of the present invention.
Claims (1)
1. A fish school three-dimensional tracking method based on skeleton analysis is characterized by comprising the following steps: the fish school three-dimensional tracking method based on skeleton analysis comprises the following steps:
(1) estimating the motion direction of the target by using a plurality of points near the skeleton endpoint, assuming the skeleton endpoint to be p (x, y), and defining the endpoint section es as n skeleton points es { (x) of the adjacent endpointi,yi) I 1., n }, the direction of the end point segment may be calculated according to the least square method,
a characteristic point F (p, theta) formed by combining the skeleton endpoint p and the endpoint section direction theta can represent the target;
(2) in the head and tail characteristic points of the target, the tail characteristic point of the target in a top view can be removed according to the asymmetry of the shape, and the head and tail characteristic points are still reserved in the side view direction;
(3) performing data association on feature points between adjacent frames of the top view to obtain a top view two-dimensional tracking track; next, stereo matching is carried out by using the top-view tracking track and the characteristic points of the target in the side-view direction, so that the position of the target in a three-dimensional space can be obtained, and the stereo matching is completed by using motion continuity in order to solve the problem of uncertainty of the stereo matching;
suppose thatAndrepresenting a feature point in top and side views, respectively, if under epipolar constraint,if there are k possible matching candidate feature points in the side view, then the motion continuity constraint is defined as follows:
wherein,feature points representing time t-1 and top viewCharacteristic points of the matching side view, pcmaxAnd dcmaxRespectively representing the maximum movement distance and the maximum deflection angle of the adjacent frame objects,andrespectively represent the feature pointsAndw and (1-w) represent the weight of the position and the direction in the cost function respectively; on the upper partThe formula indicates that, of the k candidate feature points in the side view, a feature point having the best motion continuity with the matching point at the previous time is selected as the matching point at the current time.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710914852.5A CN107818574B (en) | 2017-09-21 | 2017-09-21 | Fish shoal three-dimensional tracking method based on skeleton analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710914852.5A CN107818574B (en) | 2017-09-21 | 2017-09-21 | Fish shoal three-dimensional tracking method based on skeleton analysis |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107818574A CN107818574A (en) | 2018-03-20 |
CN107818574B true CN107818574B (en) | 2021-08-27 |
Family
ID=61607068
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710914852.5A Active CN107818574B (en) | 2017-09-21 | 2017-09-21 | Fish shoal three-dimensional tracking method based on skeleton analysis |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107818574B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111476828B (en) * | 2020-03-27 | 2023-01-10 | 清华大学 | Multi-view animal group tracking method and device |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101739568A (en) * | 2009-11-04 | 2010-06-16 | 北京交通大学 | Layered observation vector decomposed hidden Markov model-based method for identifying behaviors |
CN102609954A (en) * | 2010-12-17 | 2012-07-25 | 微软公司 | Validation analysis of human target |
CN104867135A (en) * | 2015-05-04 | 2015-08-26 | 中国科学院上海微系统与信息技术研究所 | High-precision stereo matching method based on guiding image guidance |
CN105225229A (en) * | 2015-09-07 | 2016-01-06 | 三峡大学 | Fish based on vision signal cross dam movement locus locating device and method |
CN106875429A (en) * | 2017-03-02 | 2017-06-20 | 楚雄师范学院 | Shoal of fish three-dimensional tracking and system |
-
2017
- 2017-09-21 CN CN201710914852.5A patent/CN107818574B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101739568A (en) * | 2009-11-04 | 2010-06-16 | 北京交通大学 | Layered observation vector decomposed hidden Markov model-based method for identifying behaviors |
CN102609954A (en) * | 2010-12-17 | 2012-07-25 | 微软公司 | Validation analysis of human target |
CN104867135A (en) * | 2015-05-04 | 2015-08-26 | 中国科学院上海微系统与信息技术研究所 | High-precision stereo matching method based on guiding image guidance |
CN105225229A (en) * | 2015-09-07 | 2016-01-06 | 三峡大学 | Fish based on vision signal cross dam movement locus locating device and method |
CN106875429A (en) * | 2017-03-02 | 2017-06-20 | 楚雄师范学院 | Shoal of fish three-dimensional tracking and system |
Non-Patent Citations (2)
Title |
---|
An effective and robust method for tracking multiple fish in video image based on fish head detection;Zhi-Ming Qian等;《BMC Bioinformatics》;20161231;全文 * |
半遮挡目标鱼体的识别与跟踪方法研究_毛家发半遮挡目标鱼体的识别与跟踪方法研究;毛家发等;《浙江工业大学学报》;20150430;第43卷(第2期);第1-11页 * |
Also Published As
Publication number | Publication date |
---|---|
CN107818574A (en) | 2018-03-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10354129B2 (en) | Hand gesture recognition for virtual reality and augmented reality devices | |
US10366538B2 (en) | Method and device for illustrating a virtual object in a real environment | |
Tombari et al. | Classification and evaluation of cost aggregation methods for stereo correspondence | |
US8199977B2 (en) | System and method for extraction of features from a 3-D point cloud | |
EP3114833B1 (en) | High accuracy monocular moving object localization | |
CN108776989B (en) | Low-texture planar scene reconstruction method based on sparse SLAM framework | |
CN102902355A (en) | Space interaction method of mobile equipment | |
CN109961461B (en) | Multi-moving-object tracking method based on three-dimensional layered graph model | |
Rangesh et al. | Ground plane polling for 6dof pose estimation of objects on the road | |
US8929587B2 (en) | Multi-tracking of video objects | |
Li et al. | RGBD relocalisation using pairwise geometry and concise key point sets | |
US20220277595A1 (en) | Hand gesture detection method and apparatus, and computer storage medium | |
Grabner et al. | Gp2c: Geometric projection parameter consensus for joint 3d pose and focal length estimation in the wild | |
Xu et al. | Video-object segmentation and 3D-trajectory estimation for monocular video sequences | |
CN107818574B (en) | Fish shoal three-dimensional tracking method based on skeleton analysis | |
Zhang et al. | Monocular 3D localization of vehicles in road scenes | |
Fan et al. | Human-m3: A multi-view multi-modal dataset for 3d human pose estimation in outdoor scenes | |
CN105631938A (en) | Image processing method and electronic equipment | |
Chattopadhyay et al. | STAR: a content based video retrieval system for moving camera video shots | |
Qian et al. | Skeleton-based 3D tracking of multiple fish from two orthogonal views | |
CN103295220A (en) | Application method of binocular vision technology in recovery physiotherapy system | |
JP6910622B2 (en) | Image processing system | |
Yoshimoto et al. | Cubistic representation for real-time 3D shape and pose estimation of unknown rigid object | |
Duanmu et al. | A multi-view pedestrian tracking framework based on graph matching | |
Zhang et al. | Vehicle 3D localization in road scenes VIA a monocular moving camera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |