CN113052019A - Target tracking method and device, intelligent equipment and computer storage medium - Google Patents
Target tracking method and device, intelligent equipment and computer storage medium Download PDFInfo
- Publication number
- CN113052019A CN113052019A CN202110262505.5A CN202110262505A CN113052019A CN 113052019 A CN113052019 A CN 113052019A CN 202110262505 A CN202110262505 A CN 202110262505A CN 113052019 A CN113052019 A CN 113052019A
- Authority
- CN
- China
- Prior art keywords
- tracking
- area
- foreground mask
- target
- confidence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000001514 detection method Methods 0.000 claims abstract description 107
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 13
- 238000012545 processing Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000000605 extraction Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a target tracking method and device, intelligent equipment and a computer storage medium, wherein the method comprises the following steps: the method comprises the steps of collecting an image, carrying out target detection on the image to obtain a detection frame comprising a first foreground mask, tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask, determining a target tracking confidence coefficient according to a first area of the detection frame and a second area of the tracking frame, and determining a tracking result according to the target tracking confidence coefficient, so that the problem of inaccurate target tracking in the prior art is solved, and the target tracking accuracy is improved.
Description
Technical Field
The present invention relates to the field of target tracking technologies, and in particular, to a target tracking method and apparatus, an intelligent device, and a computer storage medium.
Background
At present, target tracking is an important subject in the field of determining machine vision and artificial intelligence, and can be applied to the fields of video monitoring, smart televisions, human-computer interaction and the like, because the tracker of a related filter algorithm has the advantages of low complexity and high speed, the method can be widely applied to products such as smart televisions, human-computer interaction and the like, wherein a KCF target tracker is adopted for target tracking in the prior art, but when the target scale of a tracked target is changed, shielded and goes out of the field, the phenomenon of target loss occurs in the tracking process of the KCF target tracker, and therefore the accuracy of target tracking is reduced.
Disclosure of Invention
The invention mainly aims to provide a target tracking method and device, intelligent equipment and a computer storage medium, and aims to solve the problem of inaccurate target tracking.
To achieve the above object, the present invention provides a target tracking method; in one embodiment, the target tracking method comprises the following steps:
acquiring an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask;
tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask;
determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame;
and determining a tracking result according to the target tracking confidence.
In one embodiment, the step of determining a target tracking confidence according to the first area of the detection box and the second area of the tracking box comprises:
acquiring an intersection of the first area and the second area and a union of the first area and the second area;
and acquiring the ratio of the intersection to the union, and determining the target tracking confidence according to the ratio.
In one embodiment, the step of determining a tracking result according to the target tracking confidence includes:
acquiring a third area of the first foreground mask and a fourth area of the second foreground mask;
determining a reference confidence according to a third area of the first foreground mask and a fourth area of the second foreground mask;
and determining a tracking result according to the reference confidence coefficient, the target tracking confidence coefficient and a preset threshold value.
In an embodiment, the step of determining the reference confidence according to the third area of the first foreground mask and the fourth area of the second foreground mask includes:
acquiring an intersection of the third area and the fourth area and a union of the third area and the fourth area;
and acquiring the ratio of the intersection to the union, and determining the reference confidence according to the ratio.
In an embodiment, the step of determining a tracking result according to the reference confidence, the target tracking confidence and a preset threshold includes:
acquiring an intersection between a first sub-threshold and the target tracking confidence, wherein the preset threshold comprises the first sub-threshold and a second sub-threshold;
and comparing the intersection, the reference confidence and the second sub-threshold, and determining the tracking result according to the comparison result, wherein when the intersection is greater than the reference confidence and smaller than the second sub-threshold, the tracking result is determined to be that the tracking target is not lost.
In an embodiment, after the step of tracking the first foreground mask to obtain the tracking frame including the second foreground mask, the method further includes:
acquiring a first feature of a first foreground mask and a second feature of a second foreground mask, wherein the first feature and the second feature both comprise at least one of a scale feature and a direction feature;
and if the first characteristic is matched with the second characteristic, executing the step of determining the target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
In an embodiment, after the step of determining a tracking result according to the target tracking confidence, the method further includes:
and correcting the detection frame by adopting the tracking frame, and resetting the second foreground mask corresponding to the tracking frame according to the corrected detection frame.
To achieve the above object, the present invention also provides a target tracking apparatus, including:
the detection module is used for acquiring an image and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask;
the tracking module is used for tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask;
the calculation module is used for determining a target tracking confidence coefficient according to the first area of the detection frame and the second area of the tracking frame;
and the judging module is used for determining a tracking result according to the target tracking confidence coefficient.
To achieve the above object, the present invention further provides an intelligent device, which includes a memory, a processor, and an object tracking program stored in the memory and executable on the processor, wherein the object tracking program, when executed by the processor, implements the steps of the object tracking method as described above.
To achieve the above object, the present invention also provides a computer storage medium storing an object tracking program, which when executed by a processor, implements the steps of the object tracking method as described above.
According to the target tracking method and device, the intelligent device and the computer storage medium, the detection frame comprising the first foreground mask is obtained by carrying out target detection on the collected graph, the tracking frame comprising the second foreground mask is obtained by tracking the first foreground mask, the target tracking confidence coefficient is determined according to the first area of the detection frame and the second area of the tracking frame, and the tracking result is determined according to the target tracking confidence coefficient, so that whether the target is tracked and lost is judged according to the tracking result, the tracking result is adjusted in time, the problem of inaccurate target tracking in the prior art is solved, and the target tracking accuracy is improved.
Drawings
Fig. 1 is a schematic structural diagram of an intelligent device according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a target tracking method according to the present invention;
FIG. 3 is a flowchart illustrating a second embodiment of the target tracking method according to the present invention;
FIG. 4 is a flowchart illustrating a third embodiment of a target tracking method according to the present invention;
FIG. 5 is a flowchart illustrating a fourth embodiment of the target tracking method according to the present invention;
FIG. 6 is a flowchart illustrating a fifth embodiment of the target tracking method of the present invention;
FIG. 7 is a flowchart illustrating a sixth embodiment of the target tracking method according to the present invention;
FIG. 8 is a flowchart illustrating a seventh embodiment of the target tracking method of the present invention;
FIG. 9 is a schematic diagram of a target tracking device according to the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In order to solve the problem of inaccurate target tracking in the prior art, the method comprises the steps of collecting an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask; tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask; determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame; and determining a tracking result according to the target tracking confidence, so that whether the target is lost or not is judged according to the tracking result, and the target tracking accuracy is improved.
For a better understanding of the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
It should be noted that fig. 1 is an architectural diagram of a hardware operating environment of an intelligent device.
As shown in fig. 1, the smart device may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the smart device architecture shown in FIG. 1 does not constitute a limitation of smart devices, which may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an object tracking program. Among these, an operating system is a program that manages and controls the hardware and software resources of the smart device, an object tracking program, and the execution of other software or programs.
In the smart device shown in fig. 1, the user interface 1003 is mainly used for connecting a terminal and performing data communication with the terminal; the network interface 1004 is mainly used for the background server and performs data communication with the background server; the processor 1001 may be used to invoke an object tracking program stored in the memory 1005.
In this embodiment, the smart device includes: a memory 1005, a processor 1001, and an object tracking program stored on the memory and executable on the processor, wherein:
in this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
after a task created by connected background intelligent equipment is detected, a task processing instruction is generated according to the task and is sent to a node client;
receiving a processing result of a task corresponding to the task processing instruction returned by the node client in real time; and feeding back the processing result to the background intelligent equipment in real time.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask;
tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask;
determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame;
and determining a tracking result according to the target tracking confidence.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring an intersection of the first area and the second area and a union of the first area and the second area;
and acquiring the ratio of the intersection to the union, and determining the target tracking confidence according to the ratio.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring a third area of the first foreground mask and a fourth area of the second foreground mask;
determining a reference confidence according to a third area of the first foreground mask and a fourth area of the second foreground mask;
and determining a tracking result according to the reference confidence coefficient, the target tracking confidence coefficient and a preset threshold value.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring an intersection of the third area and the fourth area and a union of the third area and the fourth area;
and acquiring the ratio of the intersection to the union, and determining the reference confidence according to the ratio.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring an intersection between a first sub-threshold and the target tracking confidence, wherein the preset threshold comprises the first sub-threshold and a second sub-threshold;
and comparing the intersection, the reference confidence and the second sub-threshold, and determining the tracking result according to the comparison result, wherein when the intersection is greater than the reference confidence and smaller than the second sub-threshold, the tracking result is determined to be that the tracking target is not lost.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
acquiring a first feature of a first foreground mask and a second feature of a second foreground mask, wherein the first feature and the second feature both comprise at least one of a scale feature and a direction feature;
and if the first characteristic is matched with the second characteristic, executing the step of determining the target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
In this embodiment, the processor 1001 may be configured to call the target tracking program stored in the memory 1005 and perform the following operations:
and correcting the detection frame by adopting the tracking frame, and resetting the second foreground mask corresponding to the tracking frame according to the corrected detection frame.
Since the intelligent device provided in the embodiment of the present application is an intelligent device used for implementing the method in the embodiment of the present application, based on the method described in the embodiment of the present application, a person skilled in the art can understand the specific structure and the deformation of the intelligent device, and thus details are not described here. All intelligent devices adopted by the method of the embodiment of the present application belong to the scope to be protected by the present application. The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
For a software implementation, the techniques described in this disclosure may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described in this disclosure. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
Based on the above structure, the embodiment of the present invention is provided, an operating system to which the target tracking method is applied in the present application includes, but is not limited to, Linux, Android, Windows7, and the like, and the target tracking method may be applied to an intelligent terminal, for example, an intelligent video processing device such as an intelligent television.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the target tracking method of the present invention, which includes the following steps:
step S110, collecting an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask.
In this embodiment, the process of performing target detection on an image is actually a process of performing foreground extraction on a moving target, and the foreground extraction process is mainly divided into two categories, namely, static foreground extraction, that is, a camera is fixed and a background part relative to the moving target is unchanged; and secondly, dynamic foreground extraction, namely, a camera tracks a certain moving target, the background part of the moving target changes along with the position change of the moving target, and the background part which does not need to be analyzed is screened out from video stream data by adopting a foreground extraction algorithm to reserve the foreground part, so that the target analysis effect is improved.
In this embodiment, a plurality of moving objects exist in a video frame, and therefore, a plurality of moving objects need to be detected, each moving object corresponds to a first foreground mask, and each first foreground mask corresponds to a detection frame; the foreground extraction algorithm at least comprises any one of a PAWCS algorithm and a SuBSENSE algorithm, the mask is used for distinguishing a foreground part and a background part, the foreground part can be considered to be composed of white pixels, and the background part is composed of black pixels; the first foreground mask, namely the foreground part, is a result of detecting a moving object in each video frame by a foreground extraction algorithm, namely extracting the moving object in the video frame, wherein the extracted moving object is the first foreground mask; the detection frame is a minimum rectangular frame comprising the first foreground mask, namely the length and the width of the rectangular frame are minimum, and the detection frame is used for determining the position of the first foreground mask; specifically, a video is acquired, the video is divided into a plurality of image frames according to a preset time interval, a foreground extraction algorithm is adopted to acquire a first foreground mask of a moving target in each image frame, a minimum detection frame is determined through the first foreground mask of the moving target, and the position of the moving target is determined through the minimum detection frame.
Step S120, tracking the first foreground mask to obtain a tracking frame including a second foreground mask.
In this embodiment, a plurality of moving objects exist in a video frame, and therefore the plurality of moving objects need to be tracked, because an error exists in the tracking process, a target tracker is adopted to track each moving object to generate a second foreground mask correspondingly, and each second foreground mask corresponds to one tracking frame; the second foreground mask is a result of the target tracker tracking the moving target in each video frame, namely a result of the tracking of the first foreground mask, and is similar to the detection frame, and the tracking frame is a minimum rectangular frame including the second foreground mask, namely, the second foreground mask is minimum in length and width; specifically, a target tracker is adopted to track a first foreground mask to obtain a second foreground mask and a tracking frame corresponding to the second foreground mask, the target tracker is a KCF target tracker, the KCF target tracker adopts a discrimination type tracking method, a target detector is trained in the tracking process, the target detector is used to detect whether the predicted position of the next frame is a target or not, then a new detection result is used to update the target detector, the target foreground mask is selected as a positive sample when the target detector is trained, and the target background mask is a negative sample and is used for distinguishing the foreground mask from the background mask.
Step S130, determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
In this embodiment, the first area is an area of a minimum detection frame including a first foreground mask, the second area is an area of a minimum tracking frame including a second foreground mask, and the target tracking confidence is an actual value of the total estimated by using a measured value, that is, the actual position of the target is estimated by using the first area of the detection frame and the second area of the tracking frame; the method comprises the steps of obtaining a minimum detection frame comprising a first foreground mask by detecting a moving target in each image frame, calculating the area of the minimum detection frame, obtaining a minimum tracking frame comprising a second foreground mask by tracking the first foreground mask of the moving target in each image frame, calculating the area of the minimum tracking frame, and determining a target tracking confidence coefficient according to the area of the minimum detection frame and the area of the minimum tracking frame.
In this embodiment, specifically, the determining of the target tracking confidence is actually determined according to the number of the pixels in the minimum detection frame and the minimum tracking frame, because the size of each pixel in the detection frame is consistent, the area of the minimum detection frame is calculated by detecting the number of the pixels in the minimum detection frame, and the area of the minimum tracking frame is calculated by detecting the number of the pixels in the minimum tracking frame, so that the target tracking confidence is determined according to the number of the pixels in the minimum detection frame and the number of the pixels in the minimum tracking frame.
And step S140, determining a tracking result according to the target tracking confidence.
In this embodiment, after determining the target tracking confidence according to the first area of the detection frame and the second area of the tracking frame, comparing the target tracking confidence with a preset threshold value to determine a tracking result, for example, the first foreground mask is accurate, when a moving target is severely shielded, the target tracking result may deviate, on the basis, comparing the calculated target tracking confidence with the preset threshold value, and when the target tracking confidence is not within the preset threshold value range, determining that the target tracking is lost.
In the technical scheme of this embodiment, a first foreground mask and a corresponding detection frame are obtained by performing target detection on an image frame within a collected time interval, a target tracker is used to track the first foreground mask to obtain a second foreground mask and a corresponding tracking frame, a first area of the detection frame and a second area of the tracking frame are calculated, a target tracking confidence is determined according to the first area of the detection frame and the second area of the tracking frame, the target tracking confidence is compared with a preset threshold, and when the target tracking confidence is not within the preset threshold, it is determined that target tracking is lost.
Referring to fig. 3, fig. 3 is a schematic flow chart of a second embodiment of the target tracking method of the present invention, wherein steps S131 to S132 in the second embodiment are detailed steps of step S130 in the first embodiment, and the second implementation includes the following steps:
step S131, an intersection of the first area and the second area and a union of the first area and the second area are obtained.
Step S132, obtaining the ratio of the intersection to the union, and determining the target tracking confidence according to the ratio.
In this embodiment, the number of pixels in the detection frame is detected, and the first area of the detection frame is calculated according to the number of pixels in the detection frame and is recorded as SD(x) Detecting the number of pixel points in the tracking frame, calculating the second area of the tracking frame according to the number of the pixel points in the tracking frame, and recording the second area as ST(x) Acquiring the intersection of the first area of the detection frame and the second area of the tracking frame, and recording as SD(x)∩ST(x) Obtaining the detection frameIs merged with the second area of the tracking frame, denoted as SD(x)∪ST(x) Acquiring the intersection of the first area of the detection frame and the second area of the tracking frame and the ratio of the union of the first area of the detection frame and the second area of the tracking frame, and recording the ratio as
Determining a target tracking confidence coefficient R (x) according to the ratio, and recording as
In the technical scheme of this embodiment, an intersection and a union of a first area of a detection frame and a second area of a tracking frame are obtained, so that a target tracking confidence is determined according to a ratio of the intersection to the union.
Referring to fig. 4, fig. 4 is a schematic flow chart of a third embodiment of the target tracking method of the present invention, steps S141 to S143 in the third embodiment are detailed steps of step S140 in the first embodiment, and the third implementation includes the following steps:
step S141, a third area of the first foreground mask and a fourth area of the second foreground mask are obtained.
Step S142, determining a reference confidence according to the third area of the first foreground mask and the fourth area of the second foreground mask.
In this embodiment, similar to the first area of the detection frame and the second area of the tracking frame, the third area of the first foreground mask is calculated according to the number of the pixel points in the first foreground mask, the fourth area of the second foreground mask is calculated according to the number of the pixel points in the second foreground mask, and the reference confidence is determined according to the third area of the first foreground mask and the fourth area of the second foreground mask.
And S143, determining a tracking result according to the reference confidence, the target tracking confidence and a preset threshold.
In this embodiment, a reference confidence level is determined according to the area of the first foreground mask and the area of the second foreground mask, a target tracking confidence level is determined according to the area of the minimum detection frame and the area of the minimum tracking frame, the reference confidence level and the target tracking confidence level are compared with a preset threshold value, whether the reference confidence level and the target tracking confidence level are within a preset threshold value range is judged, and a tracking result is determined according to a judgment result.
In the technical scheme of this embodiment, a reference confidence is determined by a third area of the first foreground mask and a fourth area of the second foreground mask, and a target tracking result is determined according to a relationship between the reference confidence, a target tracking confidence and a preset threshold.
Referring to fig. 5, fig. 5 is a schematic flow chart of a fourth embodiment of the target tracking method of the present invention, wherein steps S1421-S1422 in the fourth embodiment are detailed steps of step S142 in the third embodiment, and the fourth embodiment includes the following steps:
step S1421, obtain an intersection of the third area and the fourth area, and a union of the third area and the fourth area.
Step S1422, obtaining the ratio of the intersection to the union, and determining the reference confidence according to the ratio.
In this embodiment, the number of the pixels in the first foreground mask is detected, and the third area of the first foreground mask is calculated according to the number of the pixels in the first foreground mask and is recorded as FD(x) Detecting the number of the pixel points in the second foreground mask, calculating a fourth area of the second foreground mask according to the number of the pixel points in the second foreground mask, and recording the fourth area as FT(x) Acquiring the intersection of the third area of the first foreground mask and the fourth area of the second foreground mask, and recording as FD(x)∩FT(x) Acquiring a union of the third area of the first foreground mask and the fourth area of the second foreground mask, and recording as FD(x)∪FT(x) Obtaining the intersection of the third area of the first foreground mask and the fourth area of the second foreground mask and the union ratio of the third area of the first foreground mask and the fourth area of the second foreground mask, and recording the ratio as
Determining a reference confidence level according to the ratio, and recording the confidence level as
In the technical solution of this embodiment, a reference confidence is determined according to a ratio of the intersection to the union by obtaining an intersection of the third area and the fourth area and a union of the third area and the fourth area.
Referring to fig. 6, fig. 6 is a schematic flowchart of a fifth embodiment of the target tracking method of the present invention, where steps S1431-S1432 in the fifth embodiment are the detailed steps of step S143 in the third embodiment, and the fifth embodiment includes the following steps:
step S1431, an intersection between a first sub-threshold and the target tracking confidence is obtained, where the preset threshold includes the first sub-threshold and a second sub-threshold.
Step S1432, comparing the intersection, the reference confidence and the second sub-threshold, and determining the tracking result according to the comparison result, wherein when the intersection is greater than the reference confidence and smaller than the second sub-threshold, it is determined that the tracking result is that the tracking target is not lost.
In this embodiment, the first sub-threshold and the second sub-threshold are preset thresholds, and the first sub-threshold is a preset threshold that a third area of the first foreground mask and a fourth area of the second foreground mask meet requirements, and is recorded as T1(ii) a The second sub-threshold is a preset threshold value that the first area of the detection frame and the second area of the tracking frame meet the requirements, and is recorded as T2Acquiring an intersection T between the first sub-threshold and the target tracking confidence R (x)1Andr (x), comparing the intersection with the reference confidence and the second sub-threshold, determining whether the comparison result meets the requirement, when the intersection is greater than the reference confidence and the intersection is less than the second sub-threshold, that is c (x)<T1∩R(x)<T2If the comparison result is inconsistent, the target tracking is not lostOn demand, it indicates that target tracking has been lost, specifically, the
In the technical scheme of this embodiment, an intersection between a first sub-threshold in preset thresholds and a target tracking confidence is obtained, the intersection is compared with a reference confidence and a second sub-threshold in the preset thresholds, and when the intersection is greater than the reference confidence and smaller than the second sub-threshold, it is determined that the tracking target is lost, so as to determine a target tracking result.
Referring to fig. 7, fig. 7 is a flowchart illustrating a sixth embodiment of the target tracking method of the present invention, wherein step S230 in the sixth embodiment is located after step S120 in the first embodiment, and the sixth implementation includes the following steps:
step S210, collecting an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask.
Step S220, the first foreground mask is tracked to obtain a tracking frame including a second foreground mask.
Step S230, a first feature of a first foreground mask and a second feature of a second foreground mask are obtained, where the first feature and the second feature both include at least one of a scale feature and a direction feature.
In this embodiment, in the moving process of a moving object, because there are multiple moving objects, in the process of tracking the moving object, deviation of a tracking result may be caused by a problem of light, so that, before determining a confidence of tracking the object, object matching needs to be performed, a relationship between a foreground mask of the object and the tracking result, that is, a mapping relationship between a first foreground mask and a second foreground mask is determined, in this process, a first feature of the first foreground mask and a second feature of the second foreground mask are obtained, the first feature includes at least one of a scale feature and a direction feature, the second feature includes at least one of a scale feature and a direction feature, the scale feature is a SIFT feature, the direction feature is an ORB feature, SIFT features and ORB features of object regions of the first foreground mask and the second foreground mask detected at a frame level are respectively extracted for feature matching, when the two are successfully matched, the first foreground mask and the second foreground mask reach a mapping relation, when the tracked target cannot find a feature match with the detected foreground target of the current frame, the tracking failure is represented, and the currently tracked target is abandoned; and if the detected foreground target cannot find one of the tracking targets to realize matching, the tracking target is lost.
Step S240, if the first feature matches the second feature, determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
In this embodiment, when the first feature of the first foreground mask is matched with the second feature of the second foreground mask, for example, the SIFT feature of the first foreground mask is matched with the SIFT feature of the second foreground mask, or the ORB feature of the first foreground mask is matched with the ORB feature of the second foreground mask, it is determined that the first foreground mask and the second foreground mask have a mapping relationship, the first area of the detection frame corresponding to the first foreground mask having the mapping relationship and the second area of the tracking frame corresponding to the second foreground mask are calculated, and the target tracking confidence is determined according to the first area of the detection pattern and the second area of the tracking frame.
And step S250, determining a tracking result according to the target tracking confidence.
In the technical scheme of this embodiment, an image is acquired, the image is subjected to target detection to obtain a detection frame including a first foreground mask, the first foreground mask is tracked to obtain a tracking frame including a second foreground mask, a first feature of the first foreground mask and a second feature of the second foreground mask are obtained to perform feature matching, and when the first feature is matched with the second feature, a target tracking confidence is determined according to a first area of the detection frame and a second area of the tracking frame, so that the problem of inaccurate target tracking in the prior art is solved, and the accuracy of target tracking is improved.
Referring to fig. 8, fig. 8 is a schematic flowchart of a seventh embodiment of the target tracking method of the present invention, wherein step S350 in the seventh embodiment is located after step S140 in the first embodiment, and the seventh implementation includes the following steps:
step S310, collecting an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask.
Step S320, tracking the first foreground mask to obtain a tracking frame including a second foreground mask.
Step S330, determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
And step S340, determining a tracking result according to the target tracking confidence.
And S350, correcting the detection frame by adopting the tracking frame, and resetting the second foreground mask corresponding to the tracking frame according to the corrected detection frame.
In this embodiment, the KCF target tracker is updated according to the first foreground mask of the current image frame, specifically, the detection frame corresponding to the first foreground mask is corrected by using the tracking frame corresponding to the second foreground mask to obtain a corrected detection frame, and the first foreground mask corresponding to the corrected detection frame is tracked again to obtain a tracking frame including the second foreground mask, that is, the second foreground mask corresponding to the tracking frame is reset according to the corrected detection frame.
In the technical scheme of this embodiment, an image is collected, a target of the image is detected to obtain a detection frame including a first foreground mask, the first foreground mask is tracked to obtain a tracking frame including a second foreground mask, a target tracking confidence is determined according to a first area of the detection frame and a second area of the tracking frame, a target tracking result is determined according to the target tracking confidence, the detection frame is corrected by using the tracking frame, and a position of the second foreground mask corresponding to the tracking frame is reset according to the corrected detection frame, so that the technical problems of inaccurate target detection and tracking by using a target tracker in the prior art are solved, and the accuracy of target tracking is improved.
Based on the same inventive concept, the present invention further provides a target tracking apparatus, as shown in fig. 9, where fig. 9 is a schematic structural diagram of the target tracking apparatus of the present invention, and the target tracking apparatus includes: the detection module 10, the tracking module 20, the calculation module 30 and the judgment module 40, which will be described in the following:
the detection module 10 is used for acquiring an image and performing target detection on the image to obtain a detection frame comprising a first foreground mask;
a tracking module 20, configured to track the first foreground mask to obtain a tracking frame including a second foreground mask;
a calculating module 30, configured to determine a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame; specifically, the calculation module 30 is further configured to obtain an intersection of the first area and the second area, and a union of the first area and the second area; and acquiring the ratio of the intersection to the union, and determining the target tracking confidence according to the ratio.
The judging module 40 is used for determining a tracking result according to the target tracking confidence; specifically, the judging module 40 is further configured to obtain a third area of the first foreground mask and a fourth area of the second foreground mask; determining a reference confidence according to a third area of the first foreground mask and a fourth area of the second foreground mask; determining a tracking result according to the reference confidence coefficient, the target tracking confidence coefficient and a preset threshold; specifically, the step of determining the reference confidence according to the third area of the first foreground mask and the fourth area of the second foreground mask in the determining module 40 includes: acquiring an intersection of the third area and the fourth area and a union of the third area and the fourth area; obtaining a ratio of the intersection to the union, and determining the reference confidence according to the ratio; the step of determining the tracking result according to the reference confidence, the target tracking confidence and the preset threshold in the judgment module 40 includes: acquiring an intersection between a first sub-threshold and the target tracking confidence, wherein the preset threshold comprises the first sub-threshold and a second sub-threshold; and comparing the intersection, the reference confidence and the second sub-threshold, and determining the tracking result according to the comparison result, wherein when the intersection is greater than the reference confidence and smaller than the second sub-threshold, the tracking result is determined to be that the tracking target is not lost.
The object detection device comprises the modules in addition to the modules mentioned above: a matching module and an updating module; the matching module is configured to obtain a first feature of a first foreground mask and a second feature of a second foreground mask, where the first feature and the second feature both include at least one of a scale feature and a direction feature, and if the first feature and the second feature are matched, the computing module 30 is connected; and the updating module corrects the detection frame by adopting the tracking frame and resets the second foreground mask corresponding to the tracking frame according to the corrected detection frame.
The method comprises the steps that as an acquired image is adopted, target detection is carried out on the image to obtain a detection frame comprising a first foreground mask; tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask; determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame; and determining a tracking result according to the target tracking confidence, and updating the target tracker in real time according to the tracking result, so that the problem of inaccurate target tracking in the prior art is solved, and the target tracking accuracy is improved.
Based on the same inventive concept, the embodiment of the present application further provides a computer storage medium, where a target tracking program is stored in the computer storage medium, and when the target tracking program is executed by a processor, the steps of the target tracking method described above are implemented, and the same technical effect can be achieved, and in order to avoid repetition, details are not repeated here.
Since the computer storage medium provided in the embodiments of the present application is a computer storage medium used for implementing the method in the embodiments of the present application, based on the method described in the embodiments of the present application, a person skilled in the art can understand a specific structure and a modification of the computer storage medium, and thus details are not described here. Computer storage media used in the methods of embodiments of the present application are all intended to be protected by the present application.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable computer storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by certain computer program instructions. These determining machine program instructions may be provided to a processor of a general purpose determining machine, a special purpose determining machine, an embedded processing machine, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the determining machine or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These determining machine program instructions may also be stored in a determining machine readable memory that can direct a determining machine or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the determining machine readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These decision machine program instructions may also be loaded onto a decision machine or other programmable data processing apparatus to cause a series of operational steps to be performed on the decision machine or other programmable apparatus to produce a decision machine implemented process such that the instructions which execute on the decision machine or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed determination machine. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as tokens.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method of target tracking, the method comprising:
acquiring an image, and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask;
tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask;
determining a target tracking confidence according to the first area of the detection frame and the second area of the tracking frame;
and determining a tracking result according to the target tracking confidence.
2. The target tracking method of claim 1, wherein the step of determining a target tracking confidence from the first area of the detection box and the second area of the tracking box comprises:
acquiring an intersection of the first area and the second area and a union of the first area and the second area;
and acquiring the ratio of the intersection to the union, and determining the target tracking confidence according to the ratio.
3. The target tracking method of claim 1, wherein the step of determining a tracking result based on the target tracking confidence comprises:
acquiring a third area of the first foreground mask and a fourth area of the second foreground mask;
determining a reference confidence according to a third area of the first foreground mask and a fourth area of the second foreground mask;
and determining a tracking result according to the reference confidence coefficient, the target tracking confidence coefficient and a preset threshold value.
4. The target tracking method of claim 3, wherein the step of determining a reference confidence from a third area of the first foreground mask and a fourth area of the second foreground mask comprises:
acquiring an intersection of the third area and the fourth area and a union of the third area and the fourth area;
and acquiring the ratio of the intersection to the union, and determining the reference confidence according to the ratio.
5. The target tracking method of claim 3, wherein the step of determining a tracking result according to the reference confidence, the target tracking confidence and a preset threshold comprises:
acquiring an intersection between a first sub-threshold and the target tracking confidence, wherein the preset threshold comprises the first sub-threshold and a second sub-threshold;
and comparing the intersection, the reference confidence and the second sub-threshold, and determining the tracking result according to the comparison result, wherein when the intersection is greater than the reference confidence and smaller than the second sub-threshold, the tracking result is determined to be that the tracking target is not lost.
6. The target tracking method of claim 1, wherein after the step of tracking the first foreground mask to obtain the tracking frame including the second foreground mask, further comprising:
acquiring a first feature of a first foreground mask and a second feature of a second foreground mask, wherein the first feature and the second feature both comprise at least one of a scale feature and a direction feature;
and if the first characteristic is matched with the second characteristic, executing the step of determining the target tracking confidence according to the first area of the detection frame and the second area of the tracking frame.
7. The target tracking method of claim 1, wherein the step of determining a tracking result based on the target tracking confidence further comprises, after the step of:
and correcting the detection frame by adopting the tracking frame, and resetting the second foreground mask corresponding to the tracking frame according to the corrected detection frame.
8. An object tracking apparatus, characterized in that the apparatus comprises:
the detection module is used for acquiring an image and carrying out target detection on the image to obtain a detection frame comprising a first foreground mask;
the tracking module is used for tracking the first foreground mask to obtain a tracking frame comprising a second foreground mask;
the calculation module is used for determining a target tracking confidence coefficient according to the first area of the detection frame and the second area of the tracking frame;
and the judging module is used for determining a tracking result according to the target tracking confidence coefficient.
9. An intelligent device, characterized in that the intelligent device comprises a memory, a processor and an object tracking program stored in the memory and executable on the processor, the object tracking program, when executed by the processor, implementing the steps of the object tracking method according to any one of claims 1-7.
10. A computer storage medium, characterized in that the computer storage medium stores an object tracking program, which when executed by a processor implements the steps of the object tracking method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110262505.5A CN113052019A (en) | 2021-03-10 | 2021-03-10 | Target tracking method and device, intelligent equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110262505.5A CN113052019A (en) | 2021-03-10 | 2021-03-10 | Target tracking method and device, intelligent equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113052019A true CN113052019A (en) | 2021-06-29 |
Family
ID=76511205
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110262505.5A Pending CN113052019A (en) | 2021-03-10 | 2021-03-10 | Target tracking method and device, intelligent equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113052019A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793365A (en) * | 2021-11-17 | 2021-12-14 | 第六镜科技(成都)有限公司 | Target tracking method and device, computer equipment and readable storage medium |
| WO2023093086A1 (en) * | 2021-11-26 | 2023-06-01 | 上海商汤智能科技有限公司 | Target tracking method and apparatus, training method and apparatus for model related thereto, and device, medium and computer program product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530613A (en) * | 2013-10-15 | 2014-01-22 | 无锡易视腾科技有限公司 | Target person hand gesture interaction method based on monocular video sequence |
| US20170300778A1 (en) * | 2016-04-13 | 2017-10-19 | Sony Corporation | Object tracking device and method |
| CN110111363A (en) * | 2019-04-28 | 2019-08-09 | 深兰科技（上海）有限公司 | A kind of tracking and equipment based on target detection |
| CN111242977A (en) * | 2020-01-09 | 2020-06-05 | 影石创新科技股份有限公司 | Target tracking method of panoramic video, readable storage medium and computer equipment |
-
2021
- 2021-03-10 CN CN202110262505.5A patent/CN113052019A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530613A (en) * | 2013-10-15 | 2014-01-22 | 无锡易视腾科技有限公司 | Target person hand gesture interaction method based on monocular video sequence |
| US20170300778A1 (en) * | 2016-04-13 | 2017-10-19 | Sony Corporation | Object tracking device and method |
| CN110111363A (en) * | 2019-04-28 | 2019-08-09 | 深兰科技（上海）有限公司 | A kind of tracking and equipment based on target detection |
| CN111242977A (en) * | 2020-01-09 | 2020-06-05 | 影石创新科技股份有限公司 | Target tracking method of panoramic video, readable storage medium and computer equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793365A (en) * | 2021-11-17 | 2021-12-14 | 第六镜科技(成都)有限公司 | Target tracking method and device, computer equipment and readable storage medium |
| CN113793365B (en) * | 2021-11-17 | 2022-04-29 | 第六镜科技(成都)有限公司 | Target tracking method and device, computer equipment and readable storage medium |
| WO2023093086A1 (en) * | 2021-11-26 | 2023-06-01 | 上海商汤智能科技有限公司 | Target tracking method and apparatus, training method and apparatus for model related thereto, and device, medium and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035304B (en) | Target tracking method, medium, computing device and apparatus | |
| US20130084006A1 (en) | Method and Apparatus for Foreground Object Detection | |
| CN106971401B (en) | Multi-target tracking device and method | |
| CN109727275B (en) | Object detection method, device, system and computer readable storage medium | |
| CN109544592B (en) | Moving object detection algorithm for camera movement | |
| CN111062974B (en) | Method and system for extracting foreground target by removing ghost | |
| CN110909712B (en) | Moving object detection method and device, electronic equipment and storage medium | |
| US10671887B2 (en) | Best image crop selection | |
| CN110335313B (en) | Audio acquisition equipment positioning method and device and speaker identification method and system | |
| CN111783665A (en) | Action recognition method and device, storage medium and electronic equipment | |
| CN107578424B (en) | Dynamic background difference detection method, system and device based on space-time classification | |
| CN110428394B (en) | Method, apparatus and computer storage medium for target movement detection | |
| CN113052019A (en) | Target tracking method and device, intelligent equipment and computer storage medium | |
| CN111582032A (en) | Pedestrian detection method and device, terminal equipment and storage medium | |
| CN110610123A (en) | Multi-target vehicle detection method and device, electronic equipment and storage medium | |
| US10614336B2 (en) | Method, system, and computer-readable recording medium for image-based object tracking | |
| JP2020504383A (en) | Image foreground detection device, detection method, and electronic apparatus | |
| CN108229281B (en) | Neural network generation method, face detection device and electronic equipment | |
| CN116402852A (en) | Dynamic high-speed target tracking method and device based on event camera | |
| CN111598913A (en) | Image segmentation method and system based on robot vision | |
| KR101690050B1 (en) | Intelligent video security system | |
| US11393091B2 (en) | Video image processing and motion detection | |
| CN111667419A (en) | Moving target ghost eliminating method and system based on Vibe algorithm | |
| CN114846513A (en) | Motion analysis system and motion analysis program | |
| CN114998283A (en) | Lens blocking object detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |