CN111932579A - Method and device for adjusting equipment angle based on motion trail of tracked target - Google Patents

Method and device for adjusting equipment angle based on motion trail of tracked target Download PDF

Info

Publication number
CN111932579A
CN111932579A CN202010501273.XA CN202010501273A CN111932579A CN 111932579 A CN111932579 A CN 111932579A CN 202010501273 A CN202010501273 A CN 202010501273A CN 111932579 A CN111932579 A CN 111932579A
Authority
CN
China
Prior art keywords
target
video frame
tracked
frame sequence
adjusting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010501273.XA
Other languages
Chinese (zh)
Inventor
詹瑾
黄智慧
李伟键
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Polytechnic Normal University
Original Assignee
Guangdong Polytechnic Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Polytechnic Normal University filed Critical Guangdong Polytechnic Normal University
Priority to CN202010501273.XA priority Critical patent/CN111932579A/en
Publication of CN111932579A publication Critical patent/CN111932579A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention discloses a method and a device for adjusting an equipment angle based on a tracked target motion track, wherein the method comprises the following steps: sequentially carrying out frame splitting processing on the video to obtain a video frame sequence; detecting a target to be tracked in a video frame sequence based on a target detection network model to obtain a target area of the target to be tracked; carrying out coordinate positioning on a target area of a target to be tracked in a corresponding video frame sequence; performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence; generating an adjusting instruction for adjusting the angle of the equipment based on the motion track, and sending the adjusting instruction to the equipment; adjusting the angle of the device based on the adjustment instruction. In the embodiment of the invention, the tracking angle of the equipment can be finely adjusted in real time, so that the equipment can accurately track the target.

Description

Method and device for adjusting equipment angle based on motion trail of tracked target
Technical Field
The invention relates to the technical field of image processing, in particular to a method and a device for adjusting an equipment angle based on a tracked target motion track.
Background
The research and application of target tracking are an important branch of the computer vision field, and are widely applied to various safety monitoring technical fields, the existing tracking algorithm for the target in the video is too complex, the target needs to be subjected to complex tracking calculation in the tracking process, and the existing tracking equipment is difficult to realize the motion trail prediction of more target tracking to perform micro-adjustment on the angle of the equipment in the target tracking process, so that the tracked target is always within the view angle range tracked by the equipment, the tracked target is ensured not to be easily tracked and lost, and the tracking accuracy and the tracking effect are improved.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a method and a device for adjusting the angle of equipment based on the motion track of a tracked target.
In order to solve the above technical problem, an embodiment of the present invention provides a method for adjusting an angle of a device based on a motion trajectory of a tracked target, where the method includes:
sequentially carrying out frame splitting, labeling according to a time sequence and removing redundant video frames on a video to obtain a video frame sequence;
detecting a target to be tracked in the video frame sequence based on a target detection network model to obtain a target area of the target to be tracked;
carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence, and recording coordinate position information;
performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
generating an adjusting instruction for adjusting the angle of equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the adjusting instruction to the equipment;
adjusting the angle of the device based on the adjustment instruction.
Optionally, the target detection network model is a YOLOv3 network model;
the loss functions of the YOLOv3 network model include an object confidence loss function, an object class loss function, and an object localization loss function.
Optionally, before the target-detection-based network model detects the target to be tracked in the sequence of video frames, the method further includes:
and carrying out size normalization processing on the video frame sequence, and normalizing the picture size in the video frame sequence to 416 x 416.
Optionally, the performing coordinate positioning on the target region of the target to be tracked in the corresponding video frame sequence includes:
constructing pixel coordinates of the video frame sequence based on pixel points;
and acquiring the pixel coordinate position of the target area of the target to be tracked in the corresponding video frame sequence for coordinate positioning.
Optionally, the performing, based on the sequence of video frames, a correlation calculation between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame includes:
and performing similarity correlation calculation on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Optionally, the performing, based on the sequence of video frames, correlation calculation of similarity between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame includes:
and performing correlation calculation of SIFT feature vector similarity on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Optionally, the obtaining a motion trajectory of the target to be tracked in the video frame sequence includes:
judging whether the target area of the target to be tracked of the previous video frame appears in the target area of the target to be tracked in the current video frame or not according to the correlation calculation result;
when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame;
and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
Optionally, the generating an adjustment instruction for performing an angular adjustment on a device based on a motion trajectory of the target to be tracked in the sequence of video frames includes:
predicting the appearance position of the target to be tracked in the next video frame in the video frame sequence according to the motion track of the target to be tracked in the video frame sequence;
when the appearance position of the target to be tracked in the next video frame is determined to exceed a specified area, acquiring angle data of the equipment needing to be adjusted according to the appearance position;
and generating an adjusting instruction for adjusting the angle of the equipment based on the angle data of the equipment needing to be adjusted.
Optionally, the adjusting the angle of the device based on the adjustment instruction includes:
analyzing the adjusting instruction to obtain angle data of the equipment needing to be adjusted in the adjusting instruction;
and controlling the equipment to carry out angle adjustment based on the angle data of the equipment needing to be adjusted.
In addition, an embodiment of the present invention further provides an apparatus for adjusting an angle of a device based on a motion trajectory of a tracked target, where the apparatus includes:
the video frame processing module: the video processing device is used for sequentially carrying out frame splitting, marking according to a time sequence and removing redundant video frames to obtain a video frame sequence;
a target detection module: the target detection network model is used for detecting a target to be tracked in the video frame sequence to obtain a target area of the target to be tracked;
a coordinate positioning module: the system is used for carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence and recording coordinate position information;
a motion trajectory acquisition module: the tracking method comprises the steps of performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
an instruction generation module: the adjusting instruction is used for generating an angle adjusting instruction for the equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the angle adjusting instruction to the equipment;
the angle adjusting module: and adjusting the angle of the equipment based on the adjusting instruction.
In the embodiment of the invention, by adopting the method in real time, a simpler tracking algorithm can be adopted in the target tracking process, and the target track is predicted, so that the angle fine adjustment of the tracking equipment is realized, the tracked target is always within the visual angle range tracked by the equipment, the tracked target is ensured not to be easily tracked and lost, and the tracking accuracy and the tracking effect are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for adjusting an angle of a device based on a motion trajectory of a tracked target in an embodiment of the present invention;
fig. 2 is a schematic structural composition diagram of an adjusting apparatus for adjusting an angle of a device based on a motion trajectory of a tracked target in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Examples
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a method for adjusting an angle of a device based on a motion trajectory of a tracked target according to an embodiment of the present invention.
As shown in fig. 1, a method for adjusting an angle of a device based on a motion track of a tracked object, the method includes:
s11: sequentially carrying out frame splitting, labeling according to a time sequence and removing redundant video frames on a video to obtain a video frame sequence;
in the specific implementation process of the invention, after videos acquired by video acquisition equipment are subjected to frame splitting, a video frame sequence is marked according to a time sequence, and then a video frame sequence is obtained after related redundant frames are removed, so that the subsequent calculation redundancy is reduced, and the calculation amount is reduced.
S12: detecting a target to be tracked in the video frame sequence based on a target detection network model to obtain a target area of the target to be tracked;
in the specific implementation process of the invention, the target detection network model is a YOLOv3 network model; the loss functions of the YOLOv3 network model include an object confidence loss function, an object class loss function, and an object localization loss function.
Further, before the target detection network model detects the target to be tracked in the sequence of video frames, the method further includes: and carrying out size normalization processing on the video frame sequence, and normalizing the picture size in the video frame sequence to 416 x 416.
Specifically, the target detection network model is a YOLOv3 network model; the loss function of the YOLOv3 network model is as follows:
L(O,o,C,c,l,g)＝λ1Lconf(o,c)+λ2Lcla(O,C)+λ3Lloc(l,g)；
wherein λ is1，λ2，λ3Is the equilibrium coefficient; l isconf(o, c) is a target confidence loss function; l iscla(O, C) is a target class loss function; l isloc(l, g) is the target localization loss function.
Figure BDA0002524905580000051
Figure BDA0002524905580000052
Wherein o isiE {0,1}, which represents whether the target actually exists in the predicted target boundary box i, 0 represents absence, and 1 represents existence;
Figure BDA0002524905580000053
and (4) the Sigmoid probability of whether the target exists in the predicted target rectangular box i or not is shown.
Figure BDA0002524905580000054
Figure BDA0002524905580000055
Wherein, OijE {0,1}, which represents whether the jth class target really exists in the prediction target boundary box i, 0 represents nonexistence, and 1 represents existence;
Figure BDA0002524905580000056
and (4) representing the Sigmoid probability of the j-th class target in the predicted target rectangular frame i.
Figure BDA0002524905580000057
Figure BDA0002524905580000058
Figure BDA0002524905580000061
Figure BDA0002524905580000064
Figure BDA0002524905580000062
Wherein the target location loss function uses a sum of squares of the true bias value and the predicted bias value, wherein
Figure BDA0002524905580000065
Indicating the predicted rectangular box coordinate offset,
Figure BDA0002524905580000063
indicating the coordinate offset between the GTbox and default frame that matches it, (b)x,by,bw,bh) Is a predicted target rectangular frame parameter; (c)x,cy,cw,ch) Default rectangular frame parameters; (g)x,gy,gw,gh) Is in conjunction with itAnd matching the parameters of the real target rectangular frame.
Before detecting the target to be tracked in the video frame sequence according to the standard detection network model, size normalization processing needs to be carried out on the video frame sequence, and in the invention, the picture size in the video frame sequence is normalized to 416 x 416.
By normalizing the picture size in the video frame sequence to 416 × 416, the video frame sequence more conforms to the input picture format size in the target detection network model, so that the target area of the target to be tracked is easier to detect in the video frame sequence.
S13: carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence, and recording coordinate position information;
in a specific implementation process of the present invention, the performing coordinate positioning on the target region of the target to be tracked in the corresponding video frame sequence includes: constructing pixel coordinates of the video frame sequence based on pixel points; and acquiring the pixel coordinate position of the target area of the target to be tracked in the corresponding video frame sequence for coordinate positioning.
Specifically, pixel points in an image of each frame in the video frame sequence are obtained, pixel coordinates of the image of each frame are constructed through the pixel points, and corresponding coordinate positions are obtained according to the pixel coordinates in the positions of the pixel coordinates of the target area of the target to be tracked in the image of the corresponding frame of the corresponding video frame sequence; by the method, the position of the pixel coordinate of the target area of the target to be tracked in the image of the corresponding frame of the corresponding video frame sequence can be accurately determined.
S14: performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
in a specific implementation process of the present invention, the performing correlation calculation on a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame based on a video frame sequence includes: and performing similarity correlation calculation on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Further, the performing, based on the sequence of video frames, correlation calculation of similarity between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame includes: and performing correlation calculation of SIFT feature vector similarity on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Further, the acquiring a motion trajectory of the target to be tracked in the video frame sequence includes: judging whether the target area of the target to be tracked of the previous video frame appears in the target area of the target to be tracked in the current video frame or not according to the correlation calculation result; when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame; and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
Specifically, the method includes performing correlation calculation of similarity between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame according to a video frame sequence, and then obtaining a correlation calculation result; further adopting correlation calculation of SIFT feature vector similarity; the SIFT feature is based on some locally apparent points of interest on the object, regardless of the size and rotation of the image. The tolerance to light, noise, and slight viewing angle changes is also quite high.
The specific steps of correlation calculation of SIFT feature vector similarity are as follows: and (3) detection of extreme values in the scale space: the image locations are searched for on all scales. Identifying potential interest points invariant to scale and rotation by a gaussian differential function; key point positioning: at each candidate location, the location and scale are determined by fitting a fine model. The selection of key points depends on their degree of stability; direction determination: one or more directions are assigned to each keypoint location based on the local gradient direction of the image. All subsequent operations on the image data are transformed with respect to the orientation, scale and location of the keypoints, thereby providing invariance to these transformations; description of key points: local gradients of the image are measured at a selected scale in a neighborhood around each keypoint. These gradients are transformed into a representation that allows for relatively large local shape deformations and illumination variations.
The SIFT algorithm is characterized in that: the SIFT features are local features of the image, which keep invariance to rotation, scale scaling and brightness change and also keep a certain degree of stability to view angle change, affine transformation and noise; the uniqueness (distingness) is good, the information content is rich, and the method is suitable for quick and accurate matching in a massive characteristic database; the multiplicity, even a few objects can generate a large number of SIFT feature vectors; high speed, the optimized SIFT matching algorithm can even meet the real-time requirement; and the expandability can be conveniently combined with the feature vectors in other forms.
Specifically, whether a target area of a target to be tracked of a previous video frame appears in a target area of a target to be tracked in a current video frame is judged according to the correlation calculation result; when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame; and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
S15: generating an adjusting instruction for adjusting the angle of equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the adjusting instruction to the equipment;
in a specific implementation process of the present invention, the generating an adjustment instruction for performing an angle adjustment on a device based on a motion trajectory of a target to be tracked in the sequence of video frames includes: predicting the appearance position of the target to be tracked in the next video frame in the video frame sequence according to the motion track of the target to be tracked in the video frame sequence; when the appearance position of the target to be tracked in the next video frame is determined to exceed a specified area, acquiring angle data of the equipment needing to be adjusted according to the appearance position; and generating an adjusting instruction for adjusting the angle of the equipment based on the angle data of the equipment needing to be adjusted.
Specifically, the position of the target to be tracked in the next video frame in the video frame, that is, the position coordinate of the target to be tracked, needs to be predicted according to the motion trajectory of the target to be tracked in the video frame sequence, then whether the target to be tracked is in a specified coordinate region is judged, the specified coordinate region is determined according to the prior experience, and when the target to be tracked is no longer in the coordinate region, the target may leave the video acquisition range of the device at any time, the target of the instructor is tracked and lost, when the position of the target to be tracked in the next video frame is determined to exceed the specified region, the calculation of the adjustment angle data needs to be performed according to the coordinate position information of the target to be tracked in the current video frame and the current angle information of the device, the specific calculation process is that the distance from the center of the device to the center point of the video frame where the target to be tracked is located is obtained, and the distance from the position of the target to be tracked in the current video frame to, assuming that an included angle between a distance from a current video frame position of a target to be tracked to a center point of a device and a distance from a center of the device to a center point of a video frame where the target to be tracked is located is a right angle, a first included angle formed by the distance from the center of the device to the target to be tracked and the distance from the center of the device to the center point of the video frame where the target to be tracked is located can be obtained through a trigonometric function, and the first included angle is angle data of the device to be adjusted; after angle data of the equipment needing to be adjusted are obtained, an adjusting instruction for adjusting the angle of the equipment is generated according to the angle data.
S16: adjusting the angle of the device based on the adjustment instruction.
In a specific implementation process of the present invention, the adjusting the angle of the device based on the adjustment instruction includes: analyzing the adjusting instruction to obtain angle data of the equipment needing to be adjusted in the adjusting instruction; and controlling the equipment to carry out angle adjustment based on the angle data of the equipment needing to be adjusted.
Specifically, after receiving the adjustment instruction, the device analyzes the adjustment instruction to obtain angle data of the device to be adjusted in the adjustment instruction; and then adjusting the angle data of the equipment according to the requirement to control the equipment to adjust the angle.
In the embodiment of the invention, by adopting the method in real time, a simpler tracking algorithm can be adopted in the target tracking process, and the target track is predicted, so that the angle fine adjustment of the tracking equipment is realized, the tracked target is always within the visual angle range tracked by the equipment, the tracked target is ensured not to be easily tracked and lost, and the tracking accuracy and the tracking effect are improved.
Examples
Referring to fig. 2, fig. 2 is a schematic structural composition diagram of an apparatus for adjusting an angle of a device based on a motion trajectory of a tracked target according to an embodiment of the present invention.
As shown in fig. 2, an apparatus for adjusting an angle of a device based on a motion trajectory of a tracked target, the apparatus includes:
the video frame processing module 21: the video processing device is used for sequentially carrying out frame splitting, marking according to a time sequence and removing redundant video frames to obtain a video frame sequence;
in the specific implementation process of the invention, after videos acquired by video acquisition equipment are subjected to frame splitting, a video frame sequence is marked according to a time sequence, and then a video frame sequence is obtained after related redundant frames are removed, so that the subsequent calculation redundancy is reduced, and the calculation amount is reduced.
The target detection module 22: the target detection network model is used for detecting a target to be tracked in the video frame sequence to obtain a target area of the target to be tracked;
in the specific implementation process of the invention, the target detection network model is a YOLOv3 network model; the loss functions of the YOLOv3 network model include an object confidence loss function, an object class loss function, and an object localization loss function.
Further, before the target detection network model detects the target to be tracked in the sequence of video frames, the method further includes: and carrying out size normalization processing on the video frame sequence, and normalizing the picture size in the video frame sequence to 416 x 416.
Specifically, the target detection network model is a YOLOv3 network model; the loss function of the YOLOv3 network model is as follows:
L(O,o,C,c,l,g)＝λ1Lconf(o,c)+λ2Lcla(O,C)+λ3Lloc(l,g)；
wherein λ is1，λ2，λ3Is the equilibrium coefficient; l isconf(o, c) is a target confidence loss function; l iscla(O, C) is a target class loss function; l isloc(l, g) is the target localization loss function.
Figure BDA0002524905580000101
Figure BDA0002524905580000102
Wherein o isiE {0,1}, which represents whether the target actually exists in the predicted target boundary box i, 0 represents absence, and 1 represents existence;
Figure BDA0002524905580000103
and (4) the Sigmoid probability of whether the target exists in the predicted target rectangular box i or not is shown.
Figure BDA0002524905580000104
Figure BDA0002524905580000105
Wherein, OijE {0,1}, which represents whether the jth class target really exists in the prediction target boundary box i, 0 represents nonexistence, and 1 represents existence;
Figure BDA0002524905580000106
and (4) representing the Sigmoid probability of the j-th class target in the predicted target rectangular frame i.
Figure BDA0002524905580000107
Figure BDA0002524905580000108
Figure BDA0002524905580000109
Figure BDA00025249055800001010
Figure BDA00025249055800001011
Wherein the target location loss function uses a sum of squares of the true bias value and the predicted bias value, wherein
Figure BDA00025249055800001012
Indicating the predicted rectangular box coordinate offset,
Figure BDA00025249055800001013
indicating the coordinate offset between the GTbox and default frame that matches it, (b)x,by,bw,bh) Is composed ofPredicted target rectangular box parameters; (c)x,cy,cw,ch) Default rectangular frame parameters; (g)x,gy,gw,gh) The matched real target rectangular frame parameters are obtained.
Before detecting the target to be tracked in the video frame sequence according to the standard detection network model, size normalization processing needs to be carried out on the video frame sequence, and in the invention, the picture size in the video frame sequence is normalized to 416 x 416.
By normalizing the picture size in the video frame sequence to 416 × 416, the video frame sequence more conforms to the input picture format size in the target detection network model, so that the target area of the target to be tracked is easier to detect in the video frame sequence.
The coordinate positioning module 23: the system is used for carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence and recording coordinate position information;
in a specific implementation process of the present invention, the performing coordinate positioning on the target region of the target to be tracked in the corresponding video frame sequence includes: constructing pixel coordinates of the video frame sequence based on pixel points; and acquiring the pixel coordinate position of the target area of the target to be tracked in the corresponding video frame sequence for coordinate positioning.
Specifically, pixel points in an image of each frame in the video frame sequence are obtained, pixel coordinates of the image of each frame are constructed through the pixel points, and corresponding coordinate positions are obtained according to the pixel coordinates in the positions of the pixel coordinates of the target area of the target to be tracked in the image of the corresponding frame of the corresponding video frame sequence; by the method, the position of the pixel coordinate of the target area of the target to be tracked in the image of the corresponding frame of the corresponding video frame sequence can be accurately determined.
The motion trajectory acquisition module 24: the tracking method comprises the steps of performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
in a specific implementation process of the present invention, the performing correlation calculation on a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame based on a video frame sequence includes: and performing similarity correlation calculation on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Further, the performing, based on the sequence of video frames, correlation calculation of similarity between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame includes: and performing correlation calculation of SIFT feature vector similarity on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
Further, the acquiring a motion trajectory of the target to be tracked in the video frame sequence includes: judging whether the target area of the target to be tracked of the previous video frame appears in the target area of the target to be tracked in the current video frame or not according to the correlation calculation result; when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame; and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
Specifically, the method includes performing correlation calculation of similarity between a target region of a target to be tracked in a current video frame and a target region of a target to be tracked in a previous video frame according to a video frame sequence, and then obtaining a correlation calculation result; further adopting correlation calculation of SIFT feature vector similarity; the SIFT feature is based on some locally apparent points of interest on the object, regardless of the size and rotation of the image. The tolerance to light, noise, and slight viewing angle changes is also quite high.
The specific steps of correlation calculation of SIFT feature vector similarity are as follows: and (3) detection of extreme values in the scale space: the image locations are searched for on all scales. Identifying potential interest points invariant to scale and rotation by a gaussian differential function; key point positioning: at each candidate location, the location and scale are determined by fitting a fine model. The selection of key points depends on their degree of stability; direction determination: one or more directions are assigned to each keypoint location based on the local gradient direction of the image. All subsequent operations on the image data are transformed with respect to the orientation, scale and location of the keypoints, thereby providing invariance to these transformations; description of key points: local gradients of the image are measured at a selected scale in a neighborhood around each keypoint. These gradients are transformed into a representation that allows for relatively large local shape deformations and illumination variations.
The SIFT algorithm is characterized in that: the SIFT features are local features of the image, which keep invariance to rotation, scale scaling and brightness change and also keep a certain degree of stability to view angle change, affine transformation and noise; the uniqueness (distingness) is good, the information content is rich, and the method is suitable for quick and accurate matching in a massive characteristic database; the multiplicity, even a few objects can generate a large number of SIFT feature vectors; high speed, the optimized SIFT matching algorithm can even meet the real-time requirement; and the expandability can be conveniently combined with the feature vectors in other forms.
Specifically, whether a target area of a target to be tracked of a previous video frame appears in a target area of a target to be tracked in a current video frame is judged according to the correlation calculation result; when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame; and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
The instruction generation module 25: the adjusting instruction is used for generating an angle adjusting instruction for the equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the angle adjusting instruction to the equipment;
in a specific implementation process of the present invention, the generating an adjustment instruction for performing an angle adjustment on a device based on a motion trajectory of a target to be tracked in the sequence of video frames includes: predicting the appearance position of the target to be tracked in the next video frame in the video frame sequence according to the motion track of the target to be tracked in the video frame sequence; when the appearance position of the target to be tracked in the next video frame is determined to exceed a specified area, acquiring angle data of the equipment needing to be adjusted according to the appearance position; and generating an adjusting instruction for adjusting the angle of the equipment based on the angle data of the equipment needing to be adjusted.
Specifically, the position of the target to be tracked in the next video frame in the video frame, that is, the position coordinate of the target to be tracked, needs to be predicted according to the motion trajectory of the target to be tracked in the video frame sequence, then whether the target to be tracked is in a specified coordinate region is judged, the specified coordinate region is determined according to the prior experience, and when the target to be tracked is no longer in the coordinate region, the target may leave the video acquisition range of the device at any time, the target of the instructor is tracked and lost, when the position of the target to be tracked in the next video frame is determined to exceed the specified region, the calculation of the adjustment angle data needs to be performed according to the coordinate position information of the target to be tracked in the current video frame and the current angle information of the device, the specific calculation process is that the distance from the center of the device to the center point of the video frame where the target to be tracked is located is obtained, and the distance from the position of the target to be tracked in the current video frame to, assuming that an included angle between a distance from a current video frame position of a target to be tracked to a center point of a device and a distance from a center of the device to a center point of a video frame where the target to be tracked is located is a right angle, a first included angle formed by the distance from the center of the device to the target to be tracked and the distance from the center of the device to the center point of the video frame where the target to be tracked is located can be obtained through a trigonometric function, and the first included angle is angle data of the device to be adjusted; after angle data of the equipment needing to be adjusted are obtained, an adjusting instruction for adjusting the angle of the equipment is generated according to the angle data.
The angle adjustment module 26: and adjusting the angle of the equipment based on the adjusting instruction.
In a specific implementation process of the present invention, the adjusting the angle of the device based on the adjustment instruction includes: analyzing the adjusting instruction to obtain angle data of the equipment needing to be adjusted in the adjusting instruction; and controlling the equipment to carry out angle adjustment based on the angle data of the equipment needing to be adjusted.
Specifically, after receiving the adjustment instruction, the device analyzes the adjustment instruction to obtain angle data of the device to be adjusted in the adjustment instruction; and then adjusting the angle data of the equipment according to the requirement to control the equipment to adjust the angle.
In the embodiment of the invention, by adopting the method in real time, a simpler tracking algorithm can be adopted in the target tracking process, and the target track is predicted, so that the angle fine adjustment of the tracking equipment is realized, the tracked target is always within the visual angle range tracked by the equipment, the tracked target is ensured not to be easily tracked and lost, and the tracking accuracy and the tracking effect are improved.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
In addition, the method and the apparatus for adjusting an angle of a device based on a motion trajectory of a tracked target according to the embodiments of the present invention are described in detail, and a specific example is used herein to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. A method for adjusting an angle of equipment based on a motion track of a tracked target is characterized by comprising the following steps:
sequentially carrying out frame splitting, labeling according to a time sequence and removing redundant video frames on a video to obtain a video frame sequence;
detecting a target to be tracked in the video frame sequence based on a target detection network model to obtain a target area of the target to be tracked;
carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence, and recording coordinate position information;
performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
generating an adjusting instruction for adjusting the angle of equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the adjusting instruction to the equipment;
adjusting the angle of the device based on the adjustment instruction.
2. The adaptation method according to claim 1, wherein the target detection network model is a YOLOv3 network model;
the loss functions of the YOLOv3 network model include an object confidence loss function, an object class loss function, and an object localization loss function.
3. The adjusting method according to claim 1, wherein before the target-detection-network-model-based detection of the target to be tracked in the video frame sequence, the method further comprises:
and carrying out size normalization processing on the video frame sequence, and normalizing the picture size in the video frame sequence to 416 x 416.
4. The adjustment method according to claim 1, wherein the coordinate locating of the target region of the target to be tracked in the corresponding video frame sequence comprises:
constructing pixel coordinates of the video frame sequence based on pixel points;
and acquiring the pixel coordinate position of the target area of the target to be tracked in the corresponding video frame sequence for coordinate positioning.
5. The adjustment method according to claim 1, wherein the performing the correlation calculation on the target area of the target to be tracked in the current video frame and the target area of the target to be tracked in the previous video frame based on the video frame sequence comprises:
and performing similarity correlation calculation on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
6. The adjustment method according to claim 5, wherein the performing, based on the sequence of video frames, the correlation calculation of the similarity between the target region of the target to be tracked in the current video frame and the target region of the target to be tracked in the previous video frame comprises:
and performing correlation calculation of SIFT feature vector similarity on a target region of a target to be tracked in the current video frame and a target region of a target to be tracked in the previous video frame based on the video frame sequence.
7. The adjusting method according to claim 1, wherein the obtaining a motion trajectory of the target to be tracked in the sequence of video frames comprises:
judging whether the target area of the target to be tracked of the previous video frame appears in the target area of the target to be tracked in the current video frame or not according to the correlation calculation result;
when the judgment is carried out, respectively obtaining the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame;
and obtaining the motion trail of the target to be tracked in the video frame sequence according to the coordinate positioning of the target area of the target to be tracked in the previous video frame and the target area of the target to be tracked in the current video frame.
8. The adjusting method according to claim 1, wherein the generating of the adjusting instruction for angle adjustment of the device based on the motion trajectory of the target to be tracked in the video frame sequence comprises:
predicting the appearance position of the target to be tracked in the next video frame in the video frame sequence according to the motion track of the target to be tracked in the video frame sequence;
when the appearance position of the target to be tracked in the next video frame is determined to exceed a specified area, acquiring angle data of the equipment needing to be adjusted according to the appearance position;
and generating an adjusting instruction for adjusting the angle of the equipment based on the angle data of the equipment needing to be adjusted.
9. The adjustment method according to claim 1, wherein the adjusting the angle of the device based on the adjustment instruction comprises:
analyzing the adjusting instruction to obtain angle data of the equipment needing to be adjusted in the adjusting instruction;
and controlling the equipment to carry out angle adjustment based on the angle data of the equipment needing to be adjusted.
10. An apparatus for adjusting an angle of a device based on a motion trajectory of a tracked object, the apparatus comprising:
the video frame processing module: the video processing device is used for sequentially carrying out frame splitting, marking according to a time sequence and removing redundant video frames to obtain a video frame sequence;
a target detection module: the target detection network model is used for detecting a target to be tracked in the video frame sequence to obtain a target area of the target to be tracked;
a coordinate positioning module: the system is used for carrying out coordinate positioning on a target area of the target to be tracked in a corresponding video frame sequence and recording coordinate position information;
a motion trajectory acquisition module: the tracking method comprises the steps of performing correlation calculation on a target area of a target to be tracked in a current video frame and a target area of a target to be tracked in a previous video frame based on a video frame sequence, and acquiring a motion track of the target to be tracked in the video frame sequence;
an instruction generation module: the adjusting instruction is used for generating an angle adjusting instruction for the equipment based on the motion track of the target to be tracked in the video frame sequence, and sending the angle adjusting instruction to the equipment;
the angle adjusting module: and adjusting the angle of the equipment based on the adjusting instruction.
CN202010501273.XA 2020-08-12 2020-08-12 Method and device for adjusting equipment angle based on motion trail of tracked target Pending CN111932579A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010501273.XA CN111932579A (en) 2020-08-12 2020-08-12 Method and device for adjusting equipment angle based on motion trail of tracked target

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010501273.XA CN111932579A (en) 2020-08-12 2020-08-12 Method and device for adjusting equipment angle based on motion trail of tracked target

Publications (1)

Publication Number Publication Date
CN111932579A true CN111932579A (en) 2020-11-13

Family

ID=73316494

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010501273.XA Pending CN111932579A (en) 2020-08-12 2020-08-12 Method and device for adjusting equipment angle based on motion trail of tracked target

Country Status (1)

Country Link
CN (1) CN111932579A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113033384A (en) * 2021-03-23 2021-06-25 清华大学 Wheelchair curling motion state detection and target tracking system
CN113052853A (en) * 2021-01-25 2021-06-29 广东技术师范大学 Video target tracking method and device in complex environment
CN113283279A (en) * 2021-01-25 2021-08-20 广东技术师范大学 Deep learning-based multi-target tracking method and device in video
CN115514858A (en) * 2022-10-09 2022-12-23 江苏超正科技有限公司 Target identification and motion detection method and system based on deep neural network image

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959637A (en) * 2016-06-01 2016-09-21 福州大学 Remote intelligent video monitoring system
CN106571038A (en) * 2015-10-12 2017-04-19 原熙 Method for fully automatically monitoring road
CN106682572A (en) * 2016-10-12 2017-05-17 纳恩博（北京）科技有限公司 Target tracking method, target tracking system and first electronic device
CN109597431A (en) * 2018-11-05 2019-04-09 视联动力信息技术股份有限公司 A kind of method and device of target following
CN110021034A (en) * 2019-03-20 2019-07-16 华南理工大学 A kind of tracking recording broadcasting method and system based on head and shoulder detection
CN110570456A (en) * 2019-07-26 2019-12-13 南京理工大学 Motor vehicle track extraction method based on fusion of YOLO target detection algorithm and optical flow tracking algorithm
CN110930428A (en) * 2020-02-19 2020-03-27 成都纵横大鹏无人机科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN111161311A (en) * 2019-12-09 2020-05-15 中车工业研究院有限公司 Visual multi-target tracking method and device based on deep learning

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106571038A (en) * 2015-10-12 2017-04-19 原熙 Method for fully automatically monitoring road
CN105959637A (en) * 2016-06-01 2016-09-21 福州大学 Remote intelligent video monitoring system
CN106682572A (en) * 2016-10-12 2017-05-17 纳恩博（北京）科技有限公司 Target tracking method, target tracking system and first electronic device
CN109597431A (en) * 2018-11-05 2019-04-09 视联动力信息技术股份有限公司 A kind of method and device of target following
CN110021034A (en) * 2019-03-20 2019-07-16 华南理工大学 A kind of tracking recording broadcasting method and system based on head and shoulder detection
CN110570456A (en) * 2019-07-26 2019-12-13 南京理工大学 Motor vehicle track extraction method based on fusion of YOLO target detection algorithm and optical flow tracking algorithm
CN111161311A (en) * 2019-12-09 2020-05-15 中车工业研究院有限公司 Visual multi-target tracking method and device based on deep learning
CN110930428A (en) * 2020-02-19 2020-03-27 成都纵横大鹏无人机科技有限公司 Target tracking method and device, electronic equipment and storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113052853A (en) * 2021-01-25 2021-06-29 广东技术师范大学 Video target tracking method and device in complex environment
CN113283279A (en) * 2021-01-25 2021-08-20 广东技术师范大学 Deep learning-based multi-target tracking method and device in video
CN113283279B (en) * 2021-01-25 2024-01-19 广东技术师范大学 Multi-target tracking method and device in video based on deep learning
CN113033384A (en) * 2021-03-23 2021-06-25 清华大学 Wheelchair curling motion state detection and target tracking system
CN115514858A (en) * 2022-10-09 2022-12-23 江苏超正科技有限公司 Target identification and motion detection method and system based on deep neural network image

Similar Documents

Publication Publication Date Title
CN111932579A (en) Method and device for adjusting equipment angle based on motion trail of tracked target
CN111932582A (en) Target tracking method and device in video image
CN111832514B (en) Unsupervised pedestrian re-identification method and unsupervised pedestrian re-identification device based on soft multiple labels
Tian et al. Scene Text Detection in Video by Learning Locally and Globally.
CN107240130B (en) Remote sensing image registration method, device and system
CN112464775A (en) Video target re-identification method based on multi-branch network
CN111105436B (en) Target tracking method, computer device and storage medium
CN114494373A (en) High-precision rail alignment method and system based on target detection and image registration
Chowdhary et al. Video surveillance for the crime detection using features
CN111882594A (en) ORB feature point-based polarization image rapid registration method and device
Guler et al. A new object tracking framework for interest point based feature extraction algorithms
CN110738098A (en) target identification positioning and locking tracking method
US11830218B2 (en) Visual-inertial localisation in an existing map
CN113688819B (en) Target object expected point tracking and matching method based on marked points
CN111401286B (en) Pedestrian retrieval method based on component weight generation network
US11556580B1 (en) Indexing key frames for localization
Lu et al. Research on target detection and tracking system of rescue robot
Pęszor et al. Facial reconstruction on the basis of video surveillance system for the purpose of suspect identification
Everingham et al. Automated visual identification of characters in situation comedies
CN113129332A (en) Method and apparatus for performing target object tracking
Qiu et al. An adaptive kernel‐based target tracking method based on multiple features fusion
CN116452791B (en) Multi-camera point defect area positioning method, system, device and storage medium
Akbar et al. Detection of Indonesian Vehicle Plate Location using Harris Corner Feature Detector Method
Oad et al. Object detection algorithm based on Gaussian mixture model and sift keypoint match
Li et al. TextSLAM: Visual SLAM With Semantic Planar Text Features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination