CN107818574A - Shoal of fish three-dimensional tracking based on skeleton analysis - Google Patents

Shoal of fish three-dimensional tracking based on skeleton analysis Download PDF

Info

Publication number
CN107818574A
CN107818574A CN201710914852.5A CN201710914852A CN107818574A CN 107818574 A CN107818574 A CN 107818574A CN 201710914852 A CN201710914852 A CN 201710914852A CN 107818574 A CN107818574 A CN 107818574A
Authority
CN
China
Prior art keywords
mrow
msubsup
msub
tracking
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710914852.5A
Other languages
Chinese (zh)
Other versions
CN107818574B (en
Inventor
钱志明
王志刚
寸天睿
秦海菲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chuxiong Normal University
Original Assignee
Chuxiong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chuxiong Normal University filed Critical Chuxiong Normal University
Priority to CN201710914852.5A priority Critical patent/CN107818574B/en
Publication of CN107818574A publication Critical patent/CN107818574A/en
Application granted granted Critical
Publication of CN107818574B publication Critical patent/CN107818574B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/285—Analysis of motion using a sequence of stereo image pairs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Abstract

The invention discloses a kind of shoal of fish three-dimensional tracking based on skeleton analysis, it is related to areas of information technology, the shoal of fish three-dimensional tracking based on skeleton analysis is based on the tracking in top view direction, the testing result of side-looking direction is not used in tracking, only it is used for doing Stereo matching with top view tracking result, the complexity of tracking is reduced, improves tracking efficiency.The shoal of fish three-dimensional tracking based on skeleton analysis only needs two video cameras just to carry out effective three-dimensional tracking to shoal of fish target, not only has higher accuracy, and have faster tracking velocity.

Description

Shoal of fish three-dimensional tracking based on skeleton analysis
Technical field
The present invention relates to areas of information technology, relate in particular to a kind of shoal of fish three-dimensional track side based on skeleton analysis Method.
Background technology
Bibliography Qian Z M, Chen Y Q.Feature point based 3D tracking of multiple Fish from multi-view images [J] .PloS one, 2017,12 (6)：Proposed in e0180254 a kind of based on three The shoal of fish three-dimensional tracking of view.Target in multi views is reduced to characteristic point table by this method first with skeleton analysis Show, then, according to obtained feature point model, based on the tracking in top view direction, the tracking of two side-looking directions is used as reference, Target is matched and associated, finally gives the movement locus of target in three dimensions.
In the tracking of bibliography, when top view blocks, this method using two side-looking directions with Track result is associated to blocking front and rear target, and the advantage so handled is can to improve the performance for blocking tracking, makes tracking As a result it is more reliable, but this is to track efficiency as cost to sacrifice, if the target of top view does not have to block, side view Tracking in figure is just not necessarily.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of shoal of fish three-dimensional tracking based on skeleton analysis, with top Based on the tracking of apparent direction, the testing result of side-looking direction is not used in tracking, is only used for doing Stereo matching with top view tracking result, The complexity of tracking is reduced, improves tracking efficiency.
To achieve the above object, the present invention provides following technical scheme：
The shoal of fish three-dimensional tracking based on skeleton analysis comprises the following steps：
(1) direction of motion of target is estimated using multiple points near skeleton end points, it is assumed that skeleton end points is p (x, y), Define n skeletal point the es={ (x that end points section es is neighbouring end pointsi, yi) | i=1 ..., n }, the direction of end points section can basis Least square method is calculated,
The characteristic point F (p, θ) of joint skeleton end points p and end points section direction θ compositions can be indicated to target；
(2) in two characteristic points end to end of target, target in top view can be removed according to the asymmetry of shape Tail feature point, and for side-looking direction, still retain two characteristic points end to end；
(3) data correlation is carried out to the characteristic point of the adjacent interframe of top view, obtains top view two-dimensional tracking track.Next, Stereo matching is carried out in the characteristic point of side-looking direction, you can obtain target in three dimensions using top view pursuit path and target Position, for solve Stereo matching uncertain problem, complete Stereo matching using motion continuity；
Assuming thatWithA feature in top view and side view is represented respectively Point, if under epipolar-line constraint,The k candidate feature point that may be matched in side view be present, then by motion continuity about Beam is defined as follows：
Wherein,Represent the characteristic point of t-1 moment and top viewThe characteristic point of the side view to match.pcmaxWith dcmaxThe largest motion distance and maximum deflection angle of consecutive frame target are represented respectively,WithPoint Characteristic point is not representedWithBetween position and direction change, w and (1-w) represent position and direction in cost respectively Shared weight in function.Above formula represents, in k candidate feature point of side view, selects the match point with previous moment to have Match point of the successional characteristic point of optimal movement as current time.
Beneficial effect using above technical scheme is：The shoal of fish three-dimensional tracking based on skeleton analysis is with top view side To tracking based on, the testing result of side-looking direction is not used in tracking, is only used for top view tracking result doing Stereo matching, reduces The complexity of tracking, improve tracking efficiency.The shoal of fish three-dimensional tracking based on skeleton analysis only needs two shootings Machine can just carry out effective three-dimensional tracking to shoal of fish target, not only have higher accuracy, and with faster tracking Speed.
Brief description of the drawings
The embodiment of the present invention is described in further detail below in conjunction with the accompanying drawings.
Fig. 1 is the shoal of fish three-dimensional trace flow figure based on skeleton analysis；
Fig. 2 is Stereo matching top views of the based on motion continuity；
Fig. 3 is the Stereo matching side view based on motion continuity.
Embodiment
The side of being preferable to carry out of the invention will now be described in detail with reference to the accompanying drawings the shoal of fish three-dimensional tracking based on skeleton analysis Formula.
Fig. 1, Fig. 2 and Fig. 3 show the embodiment of the shoal of fish three-dimensional tracking of the invention based on skeleton analysis：
The shoal of fish three-dimensional tracking based on skeleton analysis is based on the tracking in top view direction, the detection knot of side-looking direction Fruit is not used in tracking, is only used for top view tracking result doing Stereo matching, the complexity of tracking is reduced with this, improve with Track efficiency.Fig. 1 shows the flow chart of institute's extracting method.Due to moving region segmentation and main framing extracting method and document Qian Z M, Chen Y Q.Feature point based 3D tracking of multiple fish from multi-view Images [J] .PloS one, 2017,12 (6)：A kind of shoal of fish three-dimensional track side based on three-view diagram is proposed in e0180254 Method is consistent, and is no longer introduced here.
The shoal of fish three-dimensional tracking based on skeleton analysis comprises the following steps：
(1) direction of motion of target is estimated using multiple points near skeleton end points, it is assumed that skeleton end points is p (x, y), Define n skeletal point the es={ (x that end points section es is neighbouring end pointsi, yi) | i=1 ..., n }, the direction of end points section can basis Least square method is calculated,
The characteristic point F (p, θ) of joint skeleton end points p and end points section direction θ compositions can be indicated to target；
This representation has the following advantages that：(1) data volume is few.Just can effectively it be represented not using only two points with direction With the target in view directions, the difficulty of tracking is significantly reduced；(2) it is strong to block disposal ability.Big portion can effectively be represented Divide shelter target, improve the accuracy for blocking tracking.
(2) in two characteristic points end to end of target, target in top view can be removed according to the asymmetry of shape Tail feature point, and for side-looking direction, still retain two characteristic points end to end；
(3) data correlation is carried out to the characteristic point of the adjacent interframe of top view, obtains top view two-dimensional tracking track.Next, Stereo matching is carried out in the characteristic point of side-looking direction, you can obtain target in three dimensions using top view pursuit path and target Position, for solve Stereo matching uncertain problem, complete Stereo matching using motion continuity；
Assuming thatWithA feature in top view and side view is represented respectively Point, if under epipolar-line constraint,The k candidate feature point that may be matched in side view be present, then by motion continuity about Beam is defined as follows：
Wherein,Represent the characteristic point of t-1 moment and top viewThe characteristic point of the side view to match.pcmaxWith dcmaxThe largest motion distance and maximum deflection angle of consecutive frame target are represented respectively,WithPoint Characteristic point is not representedWithBetween position and direction change, w and (1-w) represent position and direction in cost respectively Shared weight in function, above formula represents, in k candidate feature point of side view, selects the match point with previous moment to have Match point of the successional characteristic point of optimal movement as current time.
Fig. 2 and Fig. 3 gives an example of Stereo matching.Dotted arrow in Fig. 2 and Fig. 3 represents polar curve.Fig. 2 top views Target i in figuret In Fig. 3 side viewsK candidate matches target, selection and i on corresponding polar curve be presenttIn of previous moment There is matching target of the successional target of optimal movement as current time with point.
The shoal of fish three-dimensional tracking based on skeleton analysis only needs two video cameras just to be carried out to shoal of fish target Effective three-dimensional tracking, not only has higher accuracy, and have faster tracking velocity.
The above is only the preferred embodiment of the present invention, it is noted that for the person of ordinary skill of the art, Without departing from the concept of the premise of the invention, various modifications and improvements can be made, these belong to the guarantor of the present invention Protect scope.

Claims (1)

  1. A kind of 1. shoal of fish three-dimensional tracking based on skeleton analysis, it is characterised in that：The shoal of fish three based on skeleton analysis Dimension tracking comprises the following steps：
    (1) direction of motion of target is estimated using multiple points near skeleton end points, it is assumed that skeleton end points is p (x, y), definition End points section es is n skeletal point es={ (x of neighbouring end pointsi, yi) | i=1 ..., n }, the direction of end points section can be according to minimum Square law is calculated,
    <mrow> <mi>&amp;theta;</mi> <mo>=</mo> <mi>arctan</mi> <mrow> <mo>(</mo> <mfrac> <mrow> <msub> <mi>&amp;Sigma;x</mi> <mi>i</mi> </msub> <msub> <mi>&amp;Sigma;y</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>n&amp;Sigma;x</mi> <mi>i</mi> </msub> <msub> <mi>y</mi> <mi>i</mi> </msub> </mrow> <mrow> <msup> <mrow> <mo>(</mo> <msub> <mi>&amp;Sigma;x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>-</mo> <msup> <msub> <mi>n&amp;Sigma;x</mi> <mi>i</mi> </msub> <mn>2</mn> </msup> </mrow> </mfrac> <mo>)</mo> </mrow> </mrow>
    The characteristic point F (p, θ) of joint skeleton end points p and end points section direction θ compositions can be indicated to target；
    (2) in two characteristic points end to end of target, the afterbody of target in top view can be removed according to the asymmetry of shape Characteristic point, and for side-looking direction, still retain two characteristic points end to end；
    (3) data correlation is carried out to the characteristic point of the adjacent interframe of top view, obtains top view two-dimensional tracking track.Next, use Top view pursuit path and target carry out Stereo matching in the characteristic point of side-looking direction, you can obtain the position of target in three dimensions Put, to solve the uncertain problem of Stereo matching, Stereo matching is completed using motion continuity；
    Assuming thatWithA characteristic point in top view and side view is represented respectively, such as Fruit under epipolar-line constraint,The k candidate feature point that may be matched in side view be present, then it is fixed to constrain motion continuity Justice is as follows：
    <mrow> <mi>m</mi> <mi>c</mi> <mrow> <mo>(</mo> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>t</mi> </mrow> <mrow> <mi>t</mi> <mi>o</mi> <mi>p</mi> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>=</mo> <mi>arg</mi> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mi>q</mi> </munder> <mi>c</mi> <mi>v</mi> <mrow> <mo>(</mo> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>t</mi> <mo>-</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>F</mi> <mrow> <mi>q</mi> <mo>,</mo> <mi>t</mi> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>,</mo> <mrow> <mo>(</mo> <mi>q</mi> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow>
    <mrow> <mi>c</mi> <mi>v</mi> <mrow> <mo>(</mo> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>t</mi> <mo>-</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>F</mi> <mrow> <mi>q</mi> <mo>,</mo> <mi>t</mi> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>)</mo> </mrow> <mo>=</mo> <mi>w</mi> <mrow> <mo>(</mo> <mfrac> <mrow> <mi>p</mi> <mi>c</mi> <mrow> <mo>(</mo> <msubsup> <mi>p</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>t</mi> <mo>-</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>p</mi> <mrow> <mi>q</mi> <mo>,</mo> <mi>t</mi> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>)</mo> </mrow> </mrow> <mrow> <msub> <mi>pc</mi> <mrow> <mi>m</mi> <mi>a</mi> <mi>x</mi> </mrow> </msub> </mrow> </mfrac> <mo>)</mo> </mrow> <mo>+</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <mi>w</mi> <mo>)</mo> </mrow> <mrow> <mo>(</mo> <mfrac> <mrow> <mi>d</mi> <mi>c</mi> <mrow> <mo>(</mo> <msubsup> <mi>&amp;theta;</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>t</mi> <mo>-</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;theta;</mi> <mrow> <mi>q</mi> <mo>,</mo> <mi>t</mi> </mrow> <mrow> <mi>s</mi> <mi>i</mi> <mi>d</mi> <mi>e</mi> </mrow> </msubsup> <mo>)</mo> </mrow> </mrow> <mrow> <msub> <mi>dc</mi> <mi>max</mi> </msub> </mrow> </mfrac> <mo>)</mo> </mrow> </mrow>
    Wherein,Represent the characteristic point of t-1 moment and top viewThe characteristic point of the side view to match, pcmaxAnd dcmax The largest motion distance and maximum deflection angle of consecutive frame target are represented respectively,WithRespectively Represent characteristic pointWithBetween position and direction change, w and (1-w) represent position and direction in cost letter respectively Shared weight in number.Above formula represents, in k candidate feature point of side view, selection and the match point of previous moment have most Good speed moves match point of the successional characteristic point as current time.
CN201710914852.5A 2017-09-21 2017-09-21 Fish shoal three-dimensional tracking method based on skeleton analysis Active CN107818574B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710914852.5A CN107818574B (en) 2017-09-21 2017-09-21 Fish shoal three-dimensional tracking method based on skeleton analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710914852.5A CN107818574B (en) 2017-09-21 2017-09-21 Fish shoal three-dimensional tracking method based on skeleton analysis

Publications (2)

Publication Number Publication Date
CN107818574A true CN107818574A (en) 2018-03-20
CN107818574B CN107818574B (en) 2021-08-27

Family

ID=61607068

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710914852.5A Active CN107818574B (en) 2017-09-21 2017-09-21 Fish shoal three-dimensional tracking method based on skeleton analysis

Country Status (1)

Country Link
CN (1) CN107818574B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111476828A (en) * 2020-03-27 2020-07-31 清华大学 Multi-view animal group tracking method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739568A (en) * 2009-11-04 2010-06-16 北京交通大学 Layered observation vector decomposed hidden Markov model-based method for identifying behaviors
CN102609954A (en) * 2010-12-17 2012-07-25 微软公司 Validation analysis of human target
CN104867135A (en) * 2015-05-04 2015-08-26 中国科学院上海微系统与信息技术研究所 High-precision stereo matching method based on guiding image guidance
CN105225229A (en) * 2015-09-07 2016-01-06 三峡大学 Fish based on vision signal cross dam movement locus locating device and method
CN106875429A (en) * 2017-03-02 2017-06-20 楚雄师范学院 Shoal of fish three-dimensional tracking and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739568A (en) * 2009-11-04 2010-06-16 北京交通大学 Layered observation vector decomposed hidden Markov model-based method for identifying behaviors
CN102609954A (en) * 2010-12-17 2012-07-25 微软公司 Validation analysis of human target
CN104867135A (en) * 2015-05-04 2015-08-26 中国科学院上海微系统与信息技术研究所 High-precision stereo matching method based on guiding image guidance
CN105225229A (en) * 2015-09-07 2016-01-06 三峡大学 Fish based on vision signal cross dam movement locus locating device and method
CN106875429A (en) * 2017-03-02 2017-06-20 楚雄师范学院 Shoal of fish three-dimensional tracking and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ZHI-MING QIAN等: "An effective and robust method for tracking multiple fish in video image based on fish head detection", 《BMC BIOINFORMATICS》 *
毛家发等: "半遮挡目标鱼体的识别与跟踪方法研究_毛家发半遮挡目标鱼体的识别与跟踪方法研究", 《浙江工业大学学报》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111476828A (en) * 2020-03-27 2020-07-31 清华大学 Multi-view animal group tracking method and device
CN111476828B (en) * 2020-03-27 2023-01-10 清华大学 Multi-view animal group tracking method and device

Also Published As

Publication number Publication date
CN107818574B (en) 2021-08-27

Similar Documents

Publication Publication Date Title
Wang et al. RODNet: A real-time radar object detection network cross-supervised by camera-radar fused object 3D localization
Bao et al. Monofenet: Monocular 3d object detection with feature enhancement networks
Bergmann et al. Tracking without bells and whistles
Braun et al. Pose-rcnn: Joint object detection and pose estimation using 3d object proposals
CN106682619B (en) Object tracking method and device
Zhu et al. Cross-modality 3d object detection
CN102932605A (en) Method for selecting camera combination in visual perception network
Bu et al. Pedestrian planar LiDAR pose (PPLP) network for oriented pedestrian detection based on planar LiDAR and monocular images
Islam et al. A pedestrian detection and tracking framework for autonomous cars: Efficient fusion of camera and lidar data
CN103617631B (en) A kind of tracking based on Spot detection
Zarzar et al. Efficient tracking proposals using 2d-3d siamese networks on lidar
Liu et al. Simple online and realtime tracking with spherical panoramic camera
CN106447718A (en) 2D-to-3D depth estimation method
Wang et al. Relative pose estimation for stereo rolling shutter cameras
Hultqvist et al. Detecting and positioning overtaking vehicles using 1D optical flow
Gupta et al. Far3det: Towards far-field 3d detection
CN107818574A (en) Shoal of fish three-dimensional tracking based on skeleton analysis
Baris et al. Classification and tracking of traffic scene objects with hybrid camera systems
Zhang et al. Monocular 3D localization of vehicles in road scenes
CN103903262B (en) Depth discontinuous region stereo matching algorithm based on image segmentation
CN117132952A (en) Bird&#39;s eye view angle vehicle perception system based on many cameras
CN116630376A (en) Unmanned aerial vehicle multi-target tracking method based on ByteTrack
Obrvan et al. Appearance based vehicle detection by radar-stereo vision integration
CN104537690A (en) Motor point target detection method based on maximum-time index union
CN103761725B (en) A kind of video plane detection method based on innovatory algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant