CN111833382B - Target tracking method based on moving head camera - Google Patents

Target tracking method based on moving head camera Download PDF

Info

Publication number
CN111833382B
CN111833382B CN202010696298.XA CN202010696298A CN111833382B CN 111833382 B CN111833382 B CN 111833382B CN 202010696298 A CN202010696298 A CN 202010696298A CN 111833382 B CN111833382 B CN 111833382B
Authority
CN
China
Prior art keywords
target
tracking
machine
vecx
horizontal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010696298.XA
Other languages
Chinese (zh)
Other versions
CN111833382A (en
Inventor
唐自兴
马玉涛
徐少辉
杨亮亮
杨运红
江发钦
申雷
李志洋
宋庆丰
庞继锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuhai Raysharp Technology Co ltd
Original Assignee
Zhuhai Raysharp Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuhai Raysharp Technology Co ltd filed Critical Zhuhai Raysharp Technology Co ltd
Priority to CN202010696298.XA priority Critical patent/CN111833382B/en
Publication of CN111833382A publication Critical patent/CN111833382A/en
Application granted granted Critical
Publication of CN111833382B publication Critical patent/CN111833382B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Abstract

The invention provides a target tracking method based on an oscillating camera, which specifically comprises the steps of human shape/human face detection and positioning, human shape/human face position prediction and ID distribution, determination of a tracked human shape/human face target, calculation of the distance between the target and the center of a picture in the horizontal and vertical directions, control of an oscillating machine to rotate the target to the center of the picture, judgment of whether the oscillating machine needs to return to the home position or not and the like. The invention achieves the purpose of tracking the human shape/face and the moving target block in real time by using simple mathematical calculation and a multi-frame filtering, Kalman prediction and Hungarian matching method, avoids the condition of shaking of the shaking machine caused by target detection false detection in the tracking process, and solves the problems of large operation amount, complex model and unstable tracking shaking of the shaking machine in the prior art.

Description

Target tracking method based on moving head camera
The application is as follows: 202010091403.7, filing date: 2020.02.13, title of the invention: a target tracking method based on a moving head camera is applied by division.
Technical Field
The invention relates to the technical field of visual tracking, in particular to a target tracking method based on a moving head camera.
Background
In the field of computer vision, target tracking is one of the most challenging problems, the related directions are mainly single-target tracking and multi-target tracking, and the most applied direction in the field of security protection is mainly multi-target tracking. The main tracking processes of the multi-target tracking include detection, feature extraction/motion trajectory prediction, similarity calculation and ID distribution. In the prior art, a correlation filter and a deep learning method are mainly used for tracking, and are mainly used for predicting the target position of the next frame and extracting the features of a detection frame area so as to obtain feature expression with higher discriminability.
In the prior art, some traditional object tracking algorithms are high in running speed and poor in tracking accuracy, a plurality of newly proposed object tracking algorithms are complex in bottom-layer mathematical models, and some object tracking algorithms need to perform visual feature extraction by a combined deep learning method, so that a huge calculation amount is generated in a target tracking process, and instantaneity cannot be achieved; in addition, when the target detection generates false detection, the camera swings left and right or up and down due to the false detection when the target is tracked by using the moving camera, so that the final tracking effect is poor.
The publication number is: CN109040574A, chinese patent application, discloses a method and a device for tracking a target by a rotary shaking head machine, which are used to identify a target in a video and control the shaking head machine to rotate to track the target, so as to keep the target at the center of a picture for a user to observe. The method comprises the following steps: acquiring a frame of image currently shot by the head shaking machine; determining a tracking target of the head shaking machine from the current frame image; determining the distance between the tracking target and the center of the image; determining motor rotation parameters required by the rotary oscillating machine to track the target according to the distance between the tracked target and the center of the image and the relationship between the predetermined motor rotation parameters of the oscillating machine and the image distance; and rotating the head shaking machine according to the motor rotation parameters. According to the method, the deviation of the tracking target from the central position of the image generated in each frame of picture in the moving process is calculated, and the motor rotation parameters corresponding to the tracking target kept in the central area of the image by the head shaking machine are further calculated, so that the lens of the head shaking machine can track the target and keep the target in the central area of the picture, and the user can observe the target conveniently. However, the method still has the problems of huge calculation amount generated in the target tracking process, poor real-time performance, unstable tracking and shaking of the head shaking machine.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a target tracking method based on an oscillating camera, and solves the problems of large calculation amount, complex model and unstable tracking of the oscillating machine in the prior art.
In order to realize the technical scheme, the invention provides a target tracking method based on a panning head camera aiming at a human shape/face target, which specifically comprises the following steps:
s1, human shape/face detection positioning: for human shape or face detection, detecting the human shape or face in the video by setting a detection mode and a score threshold of the human shape or face;
s2, human shape/face position prediction and ID allocation: for the current frame, the target position predicted by the Kalman filter is combined with the actual target position detected in the step S1, an IOU value between the predicted position and the actual position is calculated to serve as a weight, wherein the IOU value is the ratio of the intersection and the union of the rectangular area of the target position predicted by the Kalman filter and the rectangular area of the actual target position detected in the step S1, and an ID is allocated to the current frame by using a Hungarian matching algorithm to obtain a target to which the ID is allocated;
s3, determining the tracked human shape/human face target: according to the ID of the tracking target of the previous frame, finding the target with the same ID in the current frame as the target to be tracked of the current frame; if the current frame does not have a target with the same ID as the previous frame, reselecting the target to be tracked;
s4, calculating the distance between the target and the center of the picture in the horizontal and vertical directions: calculating the distance vecx between the tracking target and the center of the picture in the horizontal direction and the distance vecy between the tracking target and the center of the picture in the vertical direction according to the coordinates, the width and the height of the top left vertex of the rectangular area of the tracking target and the width and the height of the picture;
s5, controlling the shaking head machine to rotate the target to the center of the picture:
a. horizontal direction: when | vecx | >40 and | vecx | > (obj _ width/4), while vecx <0, where obj _ width represents the width of the target; when the conditions are met, calling a driving interface to control the oscillating machine to rotate rightwards; on the contrary, when the vecx is larger than 0, calling the driving interface to control the head shaking machine to rotate to the left; calculating the distance vecx of the target in the horizontal direction in real time, and when | vecx | < ═ 40 or | vecx | < (obj _ width/4), stopping the motor from rotating in the horizontal direction;
b. vertical direction: when | vecy | >40 and | vecy | > (obj _ height/4) and vecy is less than 0, wherein obj _ height represents the height of a target, and when the conditions are met, the driving interface is called to control the oscillating machine to rotate upwards; otherwise, when the vecy is larger than 0, calling the driving interface to control the oscillating machine to rotate downwards; calculating the distance vecy of the target in the vertical direction in real time, and when | vecy | < ═ 40 or | vecy | < (obj _ height/4), stopping the motor from rotating in the vertical direction;
s6, judging whether the segment shaking head machine needs to return to the home position:
a. when a static person is detected by 15 continuous frames, recording targets tracked by the head shaking machine of the front frame and the rear frame, calculating distance differences of horizontal _ dis and vertical _ dis of central points of the tracked targets of the front frame and the rear frame in the horizontal direction and the vertical direction, and area differences of the two targets, wherein if the distance differences of the horizontal _ dis, the vertical _ dis and the area _ dis of the 15 continuous frames are smaller than a set threshold value respectively, the head shaking machine is returned to the home position; otherwise, continuing to track the target;
b. and when the camera stops rotating and no human shape/face is detected within 30s after the rotation is stopped, the shaking head machine is returned to the home position.
Preferably, in step S3, the objects to be tracked are reselected by first calculating the horizontal center distances of all the objects from the screen in step S2 and then arranging the objects in ascending order; and then, arranging the areas of the targets according to an inequality area >4 × area j, wherein area and area represent the areas of the targets i and j respectively, and finally, taking the first arranged target as a target object to be tracked finally.
Aiming at the target tracking of the foreground moving block, the invention also provides a target tracking method based on the moving head camera, which specifically comprises the following steps:
s1, positioning the target: for the detection of the motion block, the foreground motion target detection is completed by using a background difference method;
s2, determining a tracking target block: filtering out motion blocks with the area smaller than a set area threshold; judging whether the number of the motion block targets meeting the conditions is zero, and if the number of the motion block targets meeting the conditions is zero, returning to the next frame processing; otherwise, the areas of the reserved motion blocks are arranged in descending order; counting the frame number of the video under the condition that the oscillating motor is static; judging the number of the motion blocks, and arranging the horizontal distances from the center points of the motion blocks to the center point of the picture in an ascending order to obtain the object with larger area as the final object to be tracked;
s3, calculating the moving step length of the oscillating machine corresponding to the unit pixel: keeping the head shaking machine still, recording the horizontal and vertical position coordinates of the motor of the head shaking machine at the moment, simultaneously selecting a point P in a picture to which a lens is opposite at the moment, and recording the pixel coordinates of the point; controlling the head shaking machine to rotate for a period of time through the PTZ, and then stopping the head shaking machine, recording the pixel coordinate of the point P after the point P moves to a new position, and simultaneously recording the horizontal and vertical position coordinates of the head shaking machine motor; calculating the pixel difference of the P point in the horizontal and vertical directions of the new and old positions and the coordinate difference of the horizontal and vertical directions of the oscillating machine respectively; finally, dividing the position coordinate difference of the motor by the pixel difference to obtain a step size step _ hor corresponding to the unit pixel in the horizontal direction and a step size step _ ver corresponding to the unit pixel in the vertical direction;
s4, calculating the rotating target position of the oscillating machine: according to the obtained final target block, calculating the distance vecx of the central point of the target motion block from the central point of the picture in the horizontal direction and the distance vecy in the vertical direction; according to the target motion direction recorded in the step S23 and the positive and negative values of vecx, calculating the horizontal movement step _ hor and the vertical movement step _ ver of the oscillating machine corresponding to the obtained unit pixel, and then combining the obtained pixel distance values vecx and vecy to calculate the step vecx step _ hor and vecy step _ ver that the camera should move; acquiring horizontal and vertical positions of the camera currently through a driving interface, and calculating to obtain a target position of the camera;
s5, controlling the oscillating machine to reach the target position: continuously acquiring the current position of the motor, and if a command is found to occur but the motor does not rotate, calling the driving command again; when the target position is reached in one of the horizontal direction and the vertical direction, the motor is stopped to rotate in the direction; when both directions stop, the target tracking is completed;
s6, judging whether the oscillating machine needs to return to the home position: when the camera stops rotating and no moving target is detected in 30s after the rotation is stopped, the camera is required to return to the home position.
Preferably, the step S2 is implemented by the following steps:
s21, filtering out blocks with the area smaller than 20 pixels for the motion blocks detected by the background difference method;
s22, judging whether the number of the motion block targets meeting the conditions is zero, and if the number of the motion block targets meeting the conditions is zero, returning to the next frame processing; otherwise, the areas of the reserved motion blocks are arranged in descending order;
s23, counting the frame number of the video under the static condition of the oscillating motor; meanwhile, for the motion blocks sorted according to the area, firstly, judging whether the block obj _ area with the largest area is a full-screen motion block; if the obj _ area is a full-screen motion block, making the number moveCount of non-full-screen motion blocks be 0; if not, adding 1 to moveCount; then judging whether the moveCount value is between 0 and 10 and contains critical points of 0 and 10, if so, clearing all targets to zero without tracking any block, and simultaneously allocating IDs to the targets by using Kalman filtering and Hungarian matching method to obtain the average speed v of each target in the horizontal direction; if v < -20, recording the motion direction of the target to the left, if v >20, recording the motion direction of the target to the right, and if v is in the range of [ -20,20], determining that the motion amplitude of the target is smaller; if moveCount is greater than 10 and the number of the timed video frames is greater than 10 frames, sending all the target blocks which are arranged in the descending order of the areas to the next step for processing;
s24, judging the number of the motion blocks, and if the number cnt of the motion blocks meets 1< cnt < (5), arranging the horizontal distances from the center point of the motion block to the center point of the picture in an ascending order; if cnt is greater than 5, selecting the first 5 motion blocks according to the area descending order to perform horizontal distance ascending arrangement of the center point; finally, selecting a motion block obj _ dis closest to the central point of the picture in horizontal distance;
s25, comparing the areas of obj _ area and obj _ dis to obtain the object with larger area as the final object to be tracked.
Preferably, the step S4 is implemented by the following steps:
s41, calculating the distance vecx between the center point of the target motion block and the center point of the picture in the horizontal direction and the distance vecy in the vertical direction according to the obtained final target block, judging the absolute value of vecx and vecy at the moment in order to avoid the shaking of the shaking head machine caused by the target block being positioned at the center or nearby the picture, and controlling the shaking head machine to rotate in the direction meeting the conditions when the absolute value is greater than a set threshold value; otherwise, the head shaking machine is not controlled to rotate;
s42, judging whether the moving directions of the oscillating machine and the target are opposite or the same according to the target moving direction recorded in the step S23 and the positive and negative values of vecx and vecy, and specifically: if the unit pixel step length is the same direction, the horizontal unit pixel step length step _ hor and the vertical unit pixel step length step _ ver are increased by 0.2, and if the unit pixel step length is the same direction, the step _ hor and the step _ ver are decreased by 0.2; if the motion amplitude of the target is smaller, keeping step _ hor and step _ ver unchanged;
s43, calculating the step length of the camera head which should move in the horizontal and vertical directions respectively through formulas vecx step _ hor and vecy step _ ver according to the horizontal moving step _ hor and the vertical moving step _ ver of the oscillating machine corresponding to the unit pixel calculated in the step S42, the distance vecx of the central point of the target motion block from the central point of the picture in the horizontal direction and the distance vecy in the vertical direction calculated in the step S41;
s44, obtaining the horizontal position horizontal _ position and vertical position vertical _ position of the camera through the driving interface, and calculating the horizontal and vertical target positions of the camera rotation by using the formula horizontal _ target ═ horizontal _ position-vecx _ step _ hor and vertical _ target ═ vertical _ position-vecy _ step _ ver respectively, in combination with the step vecx _ step _ hor and vecx _ step _ ver which the motor should move and which are calculated in step S43;
s45, carrying out horizontal and vertical optical coupling constraint on the target positions in the horizontal and vertical directions;
s46, carrying out boundary condition constraint on the target positions in the horizontal and vertical directions to obtain the target position to which the oscillating machine is to be rotated finally;
and S47, clearing the frame number recorded in the step S23 in the foreground motion block determined to be tracked in the step S2.
The target tracking method based on the moving head camera has the beneficial effects that: the invention achieves the purpose of tracking the human shape/face and the moving target block in real time by using simple mathematical calculation and a multi-frame filtering, Kalman prediction and Hungarian matching method, avoids the condition of shaking of the shaking machine caused by target detection false detection in the tracking process, and solves the problems of large operation amount, complex model and unstable tracking shaking of the shaking machine in the prior art.
Drawings
FIG. 1 is a schematic flow chart of example 1.
FIG. 2 is a schematic flow chart of example 2.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person skilled in the art without making any inventive step are within the scope of the present invention.
Example 1: a target tracking method based on a moving head camera.
Referring to fig. 1, a target tracking method based on a moving head camera specifically includes the following steps:
aiming at a human shape/face target, the invention provides a target tracking method based on a moving head camera, which specifically comprises the following steps:
s1, human shape/face detection positioning: for human shape or face detection, detecting the human shape or face in the video by setting a detection mode and a score threshold of the human shape or face;
s2, human shape/face position prediction and ID allocation: for the current frame, the target position predicted by the Kalman filter is combined with the actual target position detected in the step S1, an IOU value between the predicted position and the actual position is calculated to serve as a weight, wherein the IOU value is the ratio of the intersection and the union of the rectangular area of the target position predicted by the Kalman filter and the rectangular area of the actual target position detected in the step S1, and an ID is allocated to the current frame by using a Hungarian matching algorithm to obtain a target to which the ID is allocated;
s3, determining the tracked human shape/human face target: according to the ID of the tracking target of the previous frame, finding the target with the same ID in the current frame as the target to be tracked of the current frame; if the current frame does not have a target with the same ID as the previous frame, reselecting the target to be tracked; the method of reselecting the target to be tracked is to calculate the horizontal center distances of all the targets from the picture in step S2 first, and then arrange the targets in ascending order; then, the areas of the targets are arranged according to an inequality area >4 by area, wherein the area represents the area of the targets, and finally, the first arranged target is used as a target object to be tracked;
s4, calculating the distance between the target and the center of the picture in the horizontal and vertical directions: calculating the distance vecx between the tracking target and the center of the picture in the horizontal direction and the distance vecy between the tracking target and the center of the picture in the vertical direction according to the coordinates, the width and the height of the top left vertex of the rectangular area of the tracking target and the width and the height of the picture;
s5, controlling the shaking head machine to rotate the target to the center of the picture:
a. horizontal direction: when | vecx | >40 and | vecx | > (obj _ width/4), while vecx <0, where obj _ width represents the width of the target; when the conditions are met, calling a driving interface to control the oscillating machine to rotate rightwards; on the contrary, when the vecx is larger than 0, calling the driving interface to control the head shaking machine to rotate to the left; calculating the distance vecx of the target in the horizontal direction in real time, and when | vecx | < ═ 40 or | vecx | < (obj _ width/4), stopping the motor from rotating in the horizontal direction;
b. vertical direction: when | vecy | >40 and | vecy | > (obj _ height/4) and vecy is less than 0, wherein obj _ height represents the height of a target, and when the conditions are met, the driving interface is called to control the oscillating machine to rotate upwards; otherwise, when the vecy is larger than 0, calling the driving interface to control the oscillating machine to rotate downwards; calculating the distance vecy of the target in the vertical direction in real time, and when | vecy | < ═ 40 or | vecy | < (obj _ height/4), stopping the motor from rotating in the vertical direction;
s6, judging whether the segment shaking head machine needs to return to the home position:
a. when a static person is detected by 15 continuous frames, recording targets tracked by the head shaking machine of the front frame and the rear frame, calculating distance differences of horizontal _ dis and vertical _ dis of central points of the tracked targets of the front frame and the rear frame in the horizontal direction and the vertical direction, and area differences of the two targets, wherein if the distance differences of the horizontal _ dis, the vertical _ dis and the area _ dis of the 15 continuous frames are smaller than a set threshold value respectively, the head shaking machine is returned to the home position; otherwise, continuing to track the target;
b. and when the camera stops rotating and no human shape/face is detected within 30s after the rotation is stopped, the shaking head machine is returned to the home position.
The method achieves the purpose of tracking the human shape/face in real time by using simple mathematical calculation and a multi-frame filtering, Kalman prediction and Hungarian matching method, avoids the condition of shaking of the shaking machine caused by target detection false detection in the tracking process, and solves the problems of large calculation amount, complex model and unstable shaking machine tracking in the prior art.
Example 2: a target tracking method based on a moving head camera.
Referring to fig. 2, for target tracking of a foreground moving block, a target tracking method based on a panning head camera specifically includes the following steps:
s1, positioning the target: for the detection of the motion block, the foreground motion target detection is completed by using a background difference method;
s2, determining a tracking target block:
s21, filtering out blocks with the area smaller than 20 pixels for the motion blocks detected by the background difference method;
s22, judging whether the number of the motion block targets meeting the conditions is zero, and if the number of the motion block targets meeting the conditions is zero, returning to the next frame processing; otherwise, the areas of the reserved motion blocks are arranged in descending order;
s23, counting the frame number of the video under the static condition of the oscillating motor; meanwhile, for the motion blocks sorted according to the area, firstly, judging whether the block obj _ area with the largest area is a full-screen motion block; if the obj _ area is a full-screen motion block, making the number moveCount of non-full-screen motion blocks be 0; if not, adding 1 to moveCount; then judging whether the moveCount value is between 0 and 10 and contains critical points of 0 and 10, if so, clearing all targets to zero without tracking any block, and simultaneously allocating IDs to the targets by using Kalman filtering and Hungarian matching method to obtain the average speed v of each target in the horizontal direction; if v < -20, recording the motion direction of the target to the left, if v >20, recording the motion direction of the target to the right, and if v is in the range of [ -20,20], determining that the motion amplitude of the target is smaller; if moveCount is greater than 10 and the number of the timed video frames is greater than 10 frames, sending all the target blocks which are arranged in the descending order of the areas to the next step for processing;
s24, judging the number of the motion blocks, and if the number cnt of the motion blocks meets 1< cnt < (5), arranging the horizontal distances from the center point of the motion block to the center point of the picture in an ascending order; if cnt is greater than 5, selecting the first 5 motion blocks according to the area descending order to perform horizontal distance ascending arrangement of the center point; finally, selecting a motion block obj _ dis closest to the central point of the picture in horizontal distance;
s25, comparing the areas of obj _ area and obj _ dis to obtain the object with larger area as the final object to be tracked;
s3, calculating the moving step length of the oscillating machine corresponding to the unit pixel: keeping the head shaking machine still, recording the horizontal and vertical position coordinates of the motor of the head shaking machine at the moment, simultaneously selecting a point P in a picture to which a lens is opposite at the moment, and recording the pixel coordinates of the point; controlling the head shaking machine to rotate for a period of time through the PTZ, and then stopping the head shaking machine, recording the pixel coordinate of the point P after the point P moves to a new position, and simultaneously recording the horizontal and vertical position coordinates of the head shaking machine motor; calculating the pixel difference of the P point in the horizontal and vertical directions of the new and old positions and the coordinate difference of the horizontal and vertical directions of the oscillating machine respectively; finally, dividing the position coordinate difference of the motor by the pixel difference to obtain a step size step _ hor corresponding to the unit pixel in the horizontal direction and a step size step _ ver corresponding to the unit pixel in the vertical direction;
s4, calculating the rotating target position of the oscillating machine:
s41, calculating the distance vecx between the center point of the target motion block and the center point of the picture in the horizontal direction and the distance vecy in the vertical direction according to the obtained final target block, judging the absolute value of vecx and vecy at the moment in order to avoid the shaking of the shaking head machine caused by the target block being positioned at the center or nearby the picture, and controlling the shaking head machine to rotate in the direction meeting the conditions when the absolute value is greater than a set threshold value; otherwise, the head shaking machine is not controlled to rotate;
s42, judging whether the moving directions of the oscillating machine and the target are opposite or the same according to the target moving direction recorded in the step S23 and the positive and negative values of vecx and vecy, and specifically: if the unit pixel step length is the same direction, the horizontal unit pixel step length step _ hor and the vertical unit pixel step length step _ ver are increased by 0.2, and if the unit pixel step length is the same direction, the step _ hor and the step _ ver are decreased by 0.2; if the motion amplitude of the target is smaller, keeping step _ hor and step _ ver unchanged;
and S43, calculating the step vecx step _ hor which should be moved in the horizontal direction of the camera and the step vecy step _ ver which should be moved in the vertical direction according to the horizontal movement step _ hor and the vertical movement step _ ver of the oscillating machine corresponding to the unit pixel calculated in the step S42, the distance vecx in the horizontal direction from the center point of the target motion block to the center point of the picture calculated in the step S41 and the distance vecy in the vertical direction.
S44, obtaining the horizontal position horizontal _ position and vertical position vertical _ position of the camera through the driving interface, and calculating the horizontal and vertical target positions of the camera rotation by using the formula horizontal _ target ═ horizontal _ position-vecx _ step _ hor and vertical _ target ═ vertical _ position-vecy _ step _ ver respectively, in combination with the step vecx _ step _ hor and vecx _ step _ ver which the motor should move and which are calculated in step S43;
s45, carrying out horizontal and vertical optical coupling constraint on the target positions in the horizontal and vertical directions;
s46, carrying out boundary condition constraint on the target positions in the horizontal and vertical directions to obtain the target position to which the oscillating machine is to be rotated finally;
s47, clearing the frame number recorded in the step S23 in the foreground motion block determined to be tracked in the step S2;
s5, controlling the oscillating machine to reach the target position: continuously acquiring the current position of the motor, and if a command is found to occur but the motor does not rotate, calling the driving command again; when the target position is reached in one of the horizontal direction and the vertical direction, the motor is stopped to rotate in the direction; when both directions stop, the target tracking is completed;
s6, judging whether the oscillating machine needs to return to the home position: when the camera stops rotating and no moving target is detected in 30s after the rotation is stopped, the camera is required to return to the home position.
The method achieves the purpose of tracking the moving target block in real time by using simple mathematical calculation and a multi-frame filtering, Kalman prediction and Hungarian matching method, avoids the condition of shaking of the shaking machine caused by target detection false detection in the tracking process, and solves the problems of large calculation amount, complex model and unstable tracking of the shaking machine in the prior art.
The above description is only for the preferred embodiment of the present invention, but the present invention should not be limited to the embodiment and the disclosure of the drawings, and therefore, all equivalent or modifications that do not depart from the spirit of the present invention are intended to fall within the scope of the present invention.

Claims (2)

1. A target tracking method based on a moving head camera is characterized by comprising the following steps:
s1, human shape/face detection positioning: for human shape or face detection, detecting the human shape or face in the video by setting a detection mode and a score threshold of the human shape or face;
s2, human shape/face position prediction and ID allocation: for the current frame, the target position predicted by the Kalman filter is combined with the actual target position detected in the step S1, an IOU value between the predicted position and the actual position is calculated to serve as a weight, wherein the IOU value is the ratio of the intersection and the union of the rectangular area of the target position predicted by the Kalman filter and the rectangular area of the actual target position detected in the step S1, and an ID is allocated to the current frame by using a Hungarian matching algorithm to obtain a target to which the ID is allocated;
s3, determining the tracked human shape/human face target: according to the ID of the tracking target of the previous frame, finding the target with the same ID in the current frame as the target to be tracked of the current frame; if the current frame does not have a target with the same ID as the previous frame, reselecting the target to be tracked;
s4, calculating the distance between the target and the center of the picture in the horizontal and vertical directions: calculating the distance vecx between the tracking target and the center of the picture in the horizontal direction and the distance vecy between the tracking target and the center of the picture in the vertical direction according to the coordinates, the width and the height of the top left vertex of the rectangular area of the tracking target and the width and the height of the picture;
s5, controlling the shaking head machine to rotate the target to the center of the picture:
a. horizontal direction: when | vecx | >40 and | vecx | > (obj _ width/4), while vecx <0, where obj _ width represents the width of the target; when the conditions are met, calling a driving interface to control the oscillating machine to rotate rightwards; on the contrary, when vecx is larger than 0, calling a driving interface to control the oscillating machine to rotate to the left, calculating the distance vecx of the target in the horizontal direction in real time, and when | vecx | < ═ 40 or | vecx | < (obj _ width/4), stopping the rotation of the motor in the horizontal direction;
b. vertical direction: when | vecy | >40 and | vecy | > (obj _ height/4) and vecy is less than 0, wherein obj _ height represents the height of a target, and when the conditions are met, the driving interface is called to control the oscillating machine to rotate upwards; otherwise, when the vecy is larger than 0, calling the driving interface to control the oscillating machine to rotate downwards; calculating the distance vecy of the target in the vertical direction in real time, and when | vecy | < ═ 40 or | vecy | < (obj _ height/4), stopping the motor from rotating in the vertical direction;
s6, judging whether the segment shaking head machine needs to return to the home position:
a. when a static person is detected by 15 continuous frames, recording targets tracked by the head shaking machine of the front frame and the rear frame, calculating distance differences of horizontal _ dis and vertical _ dis of central points of the tracked targets of the front frame and the rear frame in the horizontal direction and the vertical direction, and area differences of the two targets, wherein if the distance differences of the horizontal _ dis, the vertical _ dis and the area _ dis of the 15 continuous frames are smaller than a set threshold value respectively, the head shaking machine is returned to the home position; otherwise, continuing to track the target;
b. and when the camera stops rotating and no human shape/face is detected within 30s after the rotation is stopped, the shaking head machine is returned to the home position.
2. A moving head camera based target tracking method according to claim 1, characterized by: in step S3, the method of reselecting the target to be tracked is to calculate the horizontal center distances of all the targets from the screen in step S2, and then arrange the targets in ascending order; and then, arranging the areas of the targets according to an inequality area >4 × area j, wherein area and area represent the areas of the targets i and j respectively, and finally, taking the first arranged target as a target object to be tracked finally.
CN202010696298.XA 2020-02-13 2020-02-13 Target tracking method based on moving head camera Active CN111833382B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010696298.XA CN111833382B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010696298.XA CN111833382B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera
CN202010091403.7A CN111242984B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202010091403.7A Division CN111242984B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera

Publications (2)

Publication Number Publication Date
CN111833382A CN111833382A (en) 2020-10-27
CN111833382B true CN111833382B (en) 2021-03-09

Family

ID=70870555

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202010696298.XA Active CN111833382B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera
CN202010091403.7A Active CN111242984B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202010091403.7A Active CN111242984B (en) 2020-02-13 2020-02-13 Target tracking method based on moving head camera

Country Status (1)

Country Link
CN (2) CN111833382B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112132041A (en) * 2020-09-24 2020-12-25 天津锋物科技有限公司 Community patrol analysis method and system based on computer vision
CN112132047A (en) * 2020-09-24 2020-12-25 天津锋物科技有限公司 Community patrol system based on computer vision
CN115242958B (en) * 2021-04-22 2023-09-26 北京君正集成电路股份有限公司 Control method and device supporting humanoid tracking shooting under complex scene
CN115471546A (en) * 2021-06-11 2022-12-13 华为技术有限公司 Target tracking method, system, device and storage medium
CN113469025A (en) * 2021-06-29 2021-10-01 阿波罗智联(北京)科技有限公司 Target detection method and device applied to vehicle-road cooperation, road side equipment and vehicle
CN114245016A (en) * 2021-12-21 2022-03-25 睿魔智能科技（深圳）有限公司 Control method and system for video shooting, computer equipment and storage medium
CN115665553B (en) * 2022-09-29 2023-06-13 深圳市旗扬特种装备技术工程有限公司 Automatic tracking method and device of unmanned aerial vehicle, electronic equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN204716578U (en) * 2015-06-12 2015-10-21 深圳市捷扬讯科电子有限公司 A kind of intelligent fan
CN106373143A (en) * 2015-07-22 2017-02-01 中兴通讯股份有限公司 Adaptive method and system
CN106650665A (en) * 2016-12-26 2017-05-10 北京旷视科技有限公司 Human face tracing method and device
US9936169B1 (en) * 2013-03-15 2018-04-03 Puretech Systems, Inc. System and method for autonomous PTZ tracking of aerial targets
CN108921880A (en) * 2018-06-11 2018-11-30 西安电子科技大学 A kind of vision multi-object tracking method based on multiple single trackers
CN109040574A (en) * 2017-06-08 2018-12-18 北京君正集成电路股份有限公司 A kind of method and device of rotation head-shaking machine tracking target

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8593545B2 (en) * 2010-06-09 2013-11-26 Olympus Imaging Corp. Imaging apparatus, imaging method, and computer-readable recording medium with switched image capturing mode
CN102118611B (en) * 2011-04-15 2013-01-02 中国电信股份有限公司 Digital video surveillance method, digital video surveillance system and digital video surveillance platform for moving object
US9973692B2 (en) * 2013-10-03 2018-05-15 Flir Systems, Inc. Situational awareness by compressed display of panoramic views
CN104796612B (en) * 2015-04-20 2017-12-19 河南弘金电子科技有限公司 High definition radar linkage tracing control camera system and linkage tracking
CN105208343B (en) * 2015-09-25 2016-09-07 珠海安联锐视科技股份有限公司 Can be used for intelligent monitor system and the method for video monitoring equipment
CN105812736A (en) * 2016-03-21 2016-07-27 北京工业大学 Self-adjustment Pan/Tilt/Zoom camera remote intelligent control system and control method
CN106504274A (en) * 2016-10-10 2017-03-15 广东技术师范学院 A kind of visual tracking method and system based under infrared camera
CN107273822B (en) * 2017-05-26 2021-06-04 西安电子科技大学 Privacy protection method based on surveillance video multi-target tracking and face recognition
CN109885099A (en) * 2017-12-06 2019-06-14 智飞智能装备科技东台有限公司 A kind of visual identifying system for unmanned plane tracking lock target
CN107992099A (en) * 2017-12-13 2018-05-04 福州大学 A kind of target sport video tracking and system based on improvement frame difference method
CN108377364B (en) * 2018-02-07 2020-09-01 深圳市亿联智能有限公司 High-efficiency video monitoring moving object tracking mode

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9936169B1 (en) * 2013-03-15 2018-04-03 Puretech Systems, Inc. System and method for autonomous PTZ tracking of aerial targets
CN204716578U (en) * 2015-06-12 2015-10-21 深圳市捷扬讯科电子有限公司 A kind of intelligent fan
CN106373143A (en) * 2015-07-22 2017-02-01 中兴通讯股份有限公司 Adaptive method and system
CN106650665A (en) * 2016-12-26 2017-05-10 北京旷视科技有限公司 Human face tracing method and device
CN109040574A (en) * 2017-06-08 2018-12-18 北京君正集成电路股份有限公司 A kind of method and device of rotation head-shaking machine tracking target
CN108921880A (en) * 2018-06-11 2018-11-30 西安电子科技大学 A kind of vision multi-object tracking method based on multiple single trackers

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Mono-camera robotic system for tracking moving objects;Diana Gornea 等;《2014 9th IEEE Conference on Industrial Electronics and Applications》;20141023;第1820-1825页 *
基于双目云台相机的目标跟踪系统建模与仿真;夏候凯顺;《系统仿真学报》;20150228;第27卷(第2期);第362-368页 *

Also Published As

Publication number Publication date
CN111833382A (en) 2020-10-27
CN111242984A (en) 2020-06-05
CN111242984B (en) 2021-01-19

Similar Documents

Publication Publication Date Title
CN111833382B (en) Target tracking method based on moving head camera
EP3084577B1 (en) Selection and tracking of objects for display partitioning and clustering of video frames
US9767568B2 (en) Image processor, image processing method, and computer program
CN106530328B (en) A method of it is followed based on video image to moving object detection and smoothly
Haritaoglu et al. w 4 s: a real-time system for detecting and tracking people in 2 1/2 d
KR100459893B1 (en) Method and apparatus for color-based object tracking in video sequences
KR100544677B1 (en) Apparatus and method for the 3D object tracking using multi-view and depth cameras
CN108875683A (en) Robot vision tracking method and system
US20120026277A1 (en) Video conference
CN103167270A (en) Person head shooting method, system and server
CN111314609A (en) Method and device for controlling pan-tilt tracking camera shooting
CN112365522A (en) Method for tracking personnel in park across borders
JP2010057105A (en) Three-dimensional object tracking method and system
CN111209781A (en) Method and device for counting number of people in room
CN111696135A (en) Intersection ratio-based forbidden parking detection method
CN109831616B (en) Human face following method and device based on monocular camera
CN104504378A (en) Method and device for detecting image information
JP7125843B2 (en) Fault detection system
Riche et al. 3D Saliency for abnormal motion selection: The role of the depth map
CN112419364A (en) Target tracking method and system based on image feature matching
CN110728696A (en) Student standing detection method of recording and broadcasting system based on background modeling and optical flow method
Grest et al. Realtime multi-camera person tracking for immersive environments
CN106447685B (en) A kind of infrared track method
Li et al. A shadow detection method based on improved Gaussian Mixture Model
CN110378927B (en) Target detection and tracking method based on skin color

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant