CN112150505A - Target object tracker updating method and device, storage medium and electronic device - Google Patents
Target object tracker updating method and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN112150505A CN112150505A CN202010955180.4A CN202010955180A CN112150505A CN 112150505 A CN112150505 A CN 112150505A CN 202010955180 A CN202010955180 A CN 202010955180A CN 112150505 A CN112150505 A CN 112150505A
- Authority
- CN
- China
- Prior art keywords
- tracking
- detection frame
- target
- target object
- appearance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001514 detection method Methods 0.000 claims abstract description 114
- 230000015654 memory Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 12
- 238000009825 accumulation Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- GIYXAJPCNFJEHY-UHFFFAOYSA-N N-methyl-3-phenyl-3-[4-(trifluoromethyl)phenoxy]-1-propanamine hydrochloride (1:1) Chemical compound Cl.C=1C=CC=CC=1C(CCNC)OC1=CC=C(C(F)(F)F)C=C1 GIYXAJPCNFJEHY-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an updating method and device of a target object tracker, a storage medium and an electronic device. Wherein, the method comprises the following steps: acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video; determining a second tracking confidence coefficient of the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image; and under the condition that the second tracking confidence coefficient is less than or equal to the first preset threshold value, updating the parameters of the target tracker, and updating the target tracker according to the tracking confidence coefficient value, so that the technical problem that the tracking quality of the tracking track of the target object is low due to error accumulation caused by long-time non-refreshing of the tracker in the related technology is solved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an updating method and device of a target object tracker, a storage medium and an electronic device.
Background
The object tracking (MOT) technology belongs to the field of computer vision, and has wide application in pedestrian behavior analysis, vehicle unmanned driving and robot navigation positioning.
The target tracking mainly realizes the identification of all related category targets in the image frame, the association of the positions of the related category targets in the video sequence, the determination of the target IDs of the related category targets and the acquisition of the target track.
The track quality is mainly embodied in two aspects: track quality attenuation and interferent effects; track quality degradation is mainly due to error accumulation caused by long-time tracking; the interference influence mainly means that the tracker is easily influenced by similar targets (interference objects) around, track ID drift occurs, and the tracker cannot distinguish the targets from the similar interference objects. The track quality of the target track reflects the accuracy of the target track, and therefore, the acquisition of the high-quality target track is the key point of attention of people.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides an updating method and device of a target object tracker, a storage medium and an electronic device, which at least solve the technical problem of low tracking quality of a target object tracking track caused by error accumulation caused by long-time non-refreshing of the tracker in the related technology.
According to an aspect of an embodiment of the present invention, there is provided an updating method of a target object tracker, including: acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video; determining a second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image; updating the parameters of the target tracker when the second tracking confidence is less than or equal to a first predetermined threshold.
According to another aspect of the embodiments of the present invention, there is also provided an updating apparatus of a target object tracker, including: the first acquisition unit is used for acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video; a first determining unit, configured to determine a second tracking confidence level for tracking the first target object in a tth frame image according to the first tracking confidence level and a third tracking confidence level, where the third tracking confidence level is used to indicate a confidence level that a second detection frame of the first target object in the tth frame image interferes with a first detection frame of the first target object in a t-1 frame image; and the updating unit is used for updating the parameters of the target tracker under the condition that the second tracking confidence coefficient is less than or equal to a first preset threshold value.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for updating a target object tracker when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for updating the target object tracker through the computer program.
In the embodiment of the invention, a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video is obtained; determining a second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image; and under the condition that the second tracking confidence coefficient is less than or equal to the first preset threshold value, updating the parameters of the target tracker, and updating the target tracker by the tracking confidence coefficient score in consideration of the influence of the interferent, thereby solving the technical problem of low tracking quality of the tracking track of the target object caused by error accumulation caused by long-time non-refreshing of the tracker in the related technology.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a schematic diagram of an application environment of an alternative target object tracker update method according to an embodiment of the present invention;
FIG. 2 is a flow diagram of an alternative method for updating a target object tracker, according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an alternative target object tracker update apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of the embodiment of the present invention, an updating method of a target object tracker is provided, and optionally, as an optional implementation manner, the updating method of the target object tracker may be applied to, but is not limited to, a hardware environment as shown in fig. 1, which may include, but is not limited to, a terminal device 102, a network 110, and a server 112. Wherein, the terminal device 102 is used for displaying the target video.
The method can be used in the server 112, and the specific process includes the following steps: the server 112 obtains a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video; determining a second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image; and under the condition that the second tracking confidence coefficient is less than or equal to the first preset threshold value, updating the parameters of the target tracker, and updating the target tracker by the tracking confidence coefficient score in consideration of the influence of the interferent, thereby solving the technical problem of low tracking quality of the tracking track of the target object caused by error accumulation caused by long-time non-refreshing of the tracker in the related technology.
The method may also include, but is not limited to, applying to the terminal device 102 and the interaction between the terminal device 102 and the server 112.
Optionally, in this embodiment, the above-mentioned method for updating the target object tracker may be applied, but not limited to, in the server 112, for assisting the tracking of the target object in the target video of the terminal device 102. The application client may be but not limited to run in the terminal device 102, and the terminal device 102 may be but not limited to a mobile phone, a tablet computer, a notebook computer, a PC, and other terminal devices that support running the application client. The server 112 and the terminal device 102 may implement data interaction through a network, which may include but is not limited to a wireless network or a wired network. Wherein, this wireless network includes: bluetooth, WIFI, and other networks that enable wireless communication. Such wired networks may include, but are not limited to: wide area networks, metropolitan area networks, and local area networks. The above is merely an example, and this is not limited in this embodiment.
Optionally, as an optional implementation manner, as shown in fig. 2, the method for updating the target object tracker includes:
step S202, a first tracking confidence coefficient output by a target tracker tracking a first target object in the t-1 frame image in the target video is obtained.
Step S204, determining a second tracking confidence coefficient of the first target object tracked in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image.
And step S204, updating the parameters of the target tracker under the condition that the second tracking confidence coefficient is less than or equal to a first preset threshold value.
Optionally, in this embodiment, the scheme may include, but is not limited to, a single-target tracking scheme, a multi-target tracking scheme, and a scheme combining single-target tracking and multi-target tracking.
In the embodiment, for the problems of error accumulation caused by long-time non-refreshing of the tracker, target drift caused by interferents and the like, the track attenuation and the influence of the interferents are considered, an overall confidence score mechanism is provided, and a more rigorous track confidence score is provided.
Optionally, in this embodiment, determining a second tracking confidence level for tracking the first target object in the tth frame image according to the first tracking confidence level and the third tracking confidence level includes:
determining an average of the first tracking confidence and the second tracking confidence as a second tracking confidence.
According to the embodiment provided by the application, a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video is obtained; determining a second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image; and under the condition that the second tracking confidence coefficient is less than or equal to the first preset threshold value, updating the parameters of the target tracker, and updating the target tracker by the tracking confidence coefficient score in consideration of the influence of the interferent, thereby solving the technical problem of low tracking quality of the tracking track of the target object caused by error accumulation caused by long-time non-refreshing of the tracker in the related technology.
Optionally, in this embodiment, in a case that the second tracking confidence is less than or equal to the first predetermined threshold, after updating the target tracker, the method may further include:
and matching the position information of the first target object into the target track of the first target object by the updated target tracker under the condition that a fourth tracking confidence coefficient output by tracking the first target object in the t +1 th frame image in the target video is greater than or equal to a second preset threshold value.
Optionally, in this embodiment, before determining a second tracking confidence of tracking the first target object in the tth frame image according to the first tracking confidence and the third tracking confidence, the method may further include:
determining the similarity of the tracking appearance of the target according to the appearance features of the first detection frame and the appearance features of the second detection frame;
determining a tracking area overlapping ratio according to the area region of the first detection frame and the area region of the second detection frame;
and determining a third tracking confidence coefficient according to the first tracking confidence coefficient, the target tracking appearance similarity and the tracking area overlapping ratio.
Wherein determining the tracking area overlapping ratio according to the area region of the first detection frame and the area region of the second detection frame includes:
acquiring a first area region where the area region of the first detection frame and the area region of the second detection frame are intersected;
acquiring a second area region obtained by combining the area region of the first detection frame and the area region of the second detection frame;
the ratio of the first area region to the second area region is determined as a tracking area overlap ratio.
Determining a third tracking confidence degree according to the first tracking confidence degree, the target tracking appearance similarity and the tracking area overlapping ratio comprises the following steps:
determining a third tracking confidence level by:
Score3＝β·IOU(TTrack,DDet)·SiamAppearance·Score1
wherein, Score1Is the first tracking confidence, beta is a hyperparameter, SiamAppearanceTracking the appearance similarity, T, for the targetTrackAnd DDetIndicating a first and a second detection box, IOU (T)Track,DDet) Is the tracking area overlap ratio.
In this embodiment, the original tracking confidence output by the single target tracker is ScoreSOTWhen the overlap ratio of the track (the track where the first detection frame corresponding to the first target object is located) and the matched detection frame (relative to the second detection frame), namely the IOU is smaller, is higher, the possibility that the target tracked by the tracker drifts is higher, and meanwhile, the similarity of the appearance features is added for limitation. The confidence of the tracking of the perceived interferer (relative to the third confidence of the tracking) is therefore：
Scoredistractor＝β·IOU(TTrack,DDet)·SiamAppearance·ScoreKALorSOT (3)
Wherein, ScoreKALorSOTRepresents ScoreSOTBeta is a hyperparameter, SimAppearanceFor the appearance similarity of the trace-tracking frame and the detection frame, TTrackAnd DDetShowing the trace-tracking box and the matching detection box.
In summary, considering the influence of the trajectory attenuation and the interfering object, for the target X, the overall confidence of the tracking quality is:
wherein, ScoreX,tTo track the quality Score according to the last frameX,t-1And obtaining the tracking quality of the current frame, wherein lambda is a hyper-parameter.
When the score is less than a certain threshold (relative to a first predetermined threshold), the tracker is updated to counteract the accumulated error.
Optionally, in this embodiment, the method further includes:
under the condition that a third detection frame of the first target object and a fourth detection frame of the second target object exist in the t frame image, and the overlapping area between the third detection frame and the fourth detection frame is larger than or equal to a third preset threshold value, acquiring the appearance feature of the third detection frame and the appearance feature of the fourth detection frame;
determining similarity of first tracking appearance features according to the appearance features of the first detection frame and the appearance features of the third detection frame;
respectively determining the similarity of second tracking appearance features according to the appearance features of the first detection frame and the appearance features of the fourth detection frame;
and determining the maximum value of the similarity of the first tracking appearance feature and the second tracking appearance feature as the similarity of the target tracking appearance.
In this embodiment, when there is a large overlapping area between two targets (the third detection box of the first target object and the fourth detection box of the second target object), ID switching (ID switch) is easy to occur, which can be expressed as:
Λ＝Yt,whereIOU(Xt,Yt)＞γ
wherein, Xt represents the tracking result, and Yt represents the similar objects around the tracking result.
In addition to focusing on the target object, other objects overlapping with the target object should be focused on during tracking. When the tracking confidence is smaller than a certain threshold, the tracking result is considered to have deviation, and target drift is possible to occur. Therefore, selecting objects with the IOU larger than gamma as candidate targets, storing the candidate targets and the tracking results into a candidate set, solving the similarity of the appearance features between the candidate set and the tracking results, and selecting the object with the highest feature similarity in the candidate set as the tracking target. The method relieves the problem of frequent ID switching caused by dense targets in multi-target tracking to a certain extent.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
According to another aspect of the embodiments of the present invention, there is also provided an updating apparatus of a target object tracker for implementing the above updating method of a target object tracker. As shown in fig. 3, the updating apparatus of the target object tracker includes: a first acquisition unit 31, a first determination unit 33, and an update unit 35.
The first obtaining unit 31 is configured to obtain a first tracking confidence level output by the target tracker tracking the first target object in the t-1 th frame image in the target video.
The first determining unit 33 is configured to determine a second tracking confidence level for tracking the first target object in the t-th frame image according to the first tracking confidence level and a third tracking confidence level, where the third tracking confidence level is used to indicate a confidence level that a second detection frame of the first target object in the t-th frame image interferes with a first detection frame of the first target object in the t-1-th frame image.
An updating unit 35, configured to update the parameter of the target tracker when the second tracking confidence is smaller than or equal to the first predetermined threshold.
By the embodiment provided by the application, the first obtaining unit 31 obtains a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video; the first determining unit 33 determines a second tracking confidence coefficient for tracking the first target object in the t-th frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, where the third tracking confidence coefficient is used to represent a confidence coefficient that a second detection frame of the first target object in the t-th frame image interferes with a first detection frame of the first target object in the t-1-th frame image; the updating unit 35 updates the parameter of the target tracker when the second tracking confidence is equal to or less than the first predetermined threshold. The method and the device have the advantages that the influence of the interferents is considered, the target tracker is updated through the tracking confidence score, and the technical problem that in the related technology, the tracking quality of the tracking track of the target object is low due to the fact that the tracker is not refreshed for a long time and error accumulation is caused is solved.
Optionally, the first determining unit 33 may include:
and the first determining module is used for determining the average value of the first tracking confidence coefficient and the second tracking confidence coefficient as the second tracking confidence coefficient.
Optionally, the apparatus further comprises:
and the matching unit is used for matching the position information of the first target object into the target track of the first target object under the condition that the second tracking confidence coefficient is smaller than or equal to a first preset threshold value and the fourth tracking confidence coefficient output by the updated target tracker for tracking the first target object in the t +1 th frame image in the target video is greater than or equal to a second preset threshold value after the target tracker is updated.
Optionally, the apparatus further comprises:
the second determining unit is used for determining the target tracking appearance similarity according to the appearance features of the first detection frame and the second detection frame before determining the second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and the third tracking confidence coefficient;
a third determination unit configured to determine a tracking area overlapping ratio from the area region of the first detection frame and the area region of the second detection frame;
and the fourth determining unit is used for determining the third tracking confidence coefficient according to the first tracking confidence coefficient, the target tracking appearance similarity and the tracking area overlapping ratio.
Wherein, the third determining unit may include:
the first acquisition module is used for acquiring a first area region where the area region of the first detection frame and the area region of the second detection frame are intersected;
the second acquisition module is used for acquiring a second area combined by the area of the first detection frame and the area of the second detection frame;
and the second determination module is used for determining the ratio of the first area to the second area as the overlapping ratio of the tracking areas.
Wherein the fourth determining unit includes:
a third determining module for determining a third tracking confidence level by the following formula:
Score3＝β·IOU(TTrack,DDet)·SiamAppearance·Score1
wherein, Score1The first tracking confidence coefficient, beta is a hyper-parameter, SimAppeance is the similarity of the target tracking appearance, TTrack and DDet represent the first detection box and the second detection box, IOU (T)Track,DDet) Is the tracking area overlap ratio.
Optionally, the apparatus may further include:
the second obtaining unit is used for obtaining the appearance feature of the third detection frame and the appearance feature of the fourth detection frame under the condition that the third detection frame of the first target object and the fourth detection frame of the second target object exist in the t frame image and the overlapping area between the third detection frame and the fourth detection frame is larger than or equal to a third preset threshold value;
a fifth determining unit, configured to determine a similarity of the first tracking appearance feature according to the appearance feature of the first detection frame and the appearance feature of the third detection frame;
a sixth determining unit, configured to determine a second tracking appearance feature similarity according to the appearance features of the first detection frame and the appearance features of the fourth detection frame, respectively;
and the seventh determining unit is used for determining the maximum value of the similarity of the first tracking appearance feature and the similarity of the second tracking appearance feature as the similarity of the target tracking appearance.
According to a further aspect of the embodiments of the present invention, there is also provided an electronic apparatus for implementing the method for updating a target object tracker, as shown in fig. 4, the electronic apparatus includes a memory 402 and a processor 404, the memory 402 stores a computer program, and the processor 404 is configured to execute the steps in any one of the method embodiments by the computer program.
Optionally, in this embodiment, the electronic apparatus may be located in at least one network device of a plurality of network devices of a computer network.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in the t-1 frame image in the target video;
s2, determining a second tracking confidence coefficient of the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image;
s3, if the second tracking confidence is less than or equal to the first predetermined threshold, the parameter of the target tracker is updated.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 4 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 4 is a diagram illustrating the structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 4, or have a different configuration than shown in FIG. 4.
The memory 402 may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for updating a target object tracker in the embodiment of the present invention, and the processor 404 executes various functional applications and data processing by running the software programs and modules stored in the memory 402, that is, implements the method for updating a target object tracker. The memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 402 may further include memory located remotely from the processor 404, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 402 may be specifically, but not limited to, used to store information such as a first tracking confidence, a second tracking confidence, and a third tracking confidence. As an example, as shown in fig. 4, the memory 402 may include, but is not limited to, the first acquiring unit 31, the first determining unit 33, and the updating unit 35 in the updating apparatus of the target object tracker. In addition, other module units in the update apparatus of the target object tracker may also be included, but are not limited to, and are not described in this example again.
Optionally, the transmission device 406 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 406 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 406 is a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
According to a further aspect of an embodiment of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
s1, acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in the t-1 frame image in the target video;
s2, determining a second tracking confidence coefficient of the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image;
s3, if the second tracking confidence is less than or equal to the first predetermined threshold, the parameter of the target tracker is updated.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (16)
1. An updating method of a target object tracker, comprising:
acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video;
determining a second tracking confidence coefficient for tracking the first target object in the t frame image according to the first tracking confidence coefficient and a third tracking confidence coefficient, wherein the third tracking confidence coefficient is used for representing the confidence coefficient of interference between a second detection frame of the first target object in the t frame image and a first detection frame of the first target object in the t-1 frame image;
updating the parameters of the target tracker when the second tracking confidence is less than or equal to a first predetermined threshold.
2. The method of claim 1, wherein determining a second tracking confidence with which the first target object is tracked in the tth frame image based on the first tracking confidence and a third tracking confidence comprises:
determining an average of the first tracking confidence and the second tracking confidence as the second tracking confidence.
3. The method of claim 1, wherein after updating the target tracker if the second tracking confidence is less than or equal to a first predetermined threshold, the method further comprises:
and under the condition that a fourth tracking confidence coefficient output by the target tracker for tracking the first target object in the t +1 th frame image in the target video is greater than or equal to a second preset threshold value, matching the position information of the first target object into the target track of the first target object.
4. The method of claim 1, wherein prior to determining a second tracking confidence with which to track the first target object in the tth frame image based on the first tracking confidence and the third tracking confidence, the method further comprises:
determining target tracking appearance similarity according to the appearance features of the first detection frame and the second detection frame;
determining a tracking area overlapping ratio according to the area region of the first detection frame and the area region of the second detection frame;
and determining the third tracking confidence degree according to the first tracking confidence degree, the target tracking appearance similarity and the tracking area overlapping ratio.
5. The method of claim 4, wherein determining a tracking area overlap ratio from the area region of the first detection box and the area region of the second detection box comprises:
acquiring a first area region where the area region of the first detection frame and the area region of the second detection frame are intersected;
acquiring a second area combined by the area of the first detection frame and the area of the second detection frame;
a ratio of the first area region to the second area region is determined as the tracking area overlap ratio.
6. The method of claim 4, wherein determining the third tracking confidence based on the first tracking confidence, target tracking appearance similarity, and tracking area overlap ratio comprises:
determining the third tracking confidence by:
Score3＝β·IOU(TTrack,DDet)·SiamAppearance·Score1
wherein, Score1Is the first tracking confidence, beta is a hyperparameter, SiamAppearanceTracking the appearance similarity, T, for the targetTrackAnd DDetIndicating a first and a second detection box, IOU (T)Track,DDet) Is the tracking area overlap ratio.
7. The method of claim 4, wherein the method comprises:
under the condition that a third detection frame of a first target object and a fourth detection frame of a second target object exist in the t frame image, and the overlapping area between the third detection frame and the fourth detection frame is larger than or equal to a third preset threshold value, acquiring the appearance feature of the third detection frame and the appearance feature of the fourth detection frame;
determining similarity of first tracking appearance features according to the appearance features of the first detection frame and the appearance features of the third detection frame;
respectively determining second tracking appearance feature similarity according to the appearance features of the first detection frame and the fourth detection frame;
and determining the maximum value of the similarity of the first tracking appearance feature and the second tracking appearance feature as the similarity of the target tracking appearance.
8. An updating apparatus of a target object tracker, comprising:
the first acquisition unit is used for acquiring a first tracking confidence coefficient output by a target tracker tracking a first target object in a t-1 frame image in a target video;
a first determining unit, configured to determine a second tracking confidence level for tracking the first target object in a tth frame image according to the first tracking confidence level and a third tracking confidence level, where the third tracking confidence level is used to indicate a confidence level that a second detection frame of the first target object in the tth frame image interferes with a first detection frame of the first target object in a t-1 frame image;
and the updating unit is used for updating the parameters of the target tracker under the condition that the second tracking confidence coefficient is less than or equal to a first preset threshold value.
9. The apparatus of claim 8, wherein the first determining unit comprises:
a first determining module, configured to determine an average of the first tracking confidence and the second tracking confidence as the second tracking confidence.
10. The apparatus of claim 8, further comprising:
and the matching unit is used for matching the position information of the first target object into the target track of the first target object when the fourth tracking confidence coefficient output by the updated target tracker for tracking the first target object in the t +1 th frame image in the target video is greater than or equal to a second preset threshold after the target tracker is updated under the condition that the second tracking confidence coefficient is less than or equal to a first preset threshold.
11. The apparatus of claim 8, further comprising:
a second determining unit, configured to determine a target tracking appearance similarity according to the appearance feature of the first detection frame and the appearance feature of the second detection frame before determining a second tracking confidence for tracking the first target object in the t-th frame image according to the first tracking confidence and the third tracking confidence;
a third determining unit configured to determine a tracking area overlapping ratio from an area region of the first detection frame and an area region of the second detection frame;
and the fourth determining unit is used for determining the third tracking confidence coefficient according to the first tracking confidence coefficient, the target tracking appearance similarity and the tracking area overlapping ratio.
12. The apparatus of claim 11, wherein the third determining unit comprises:
the first acquisition module is used for acquiring a first area region where the area region of the first detection frame and the area region of the second detection frame are intersected;
the second acquisition module is used for acquiring a second area combined by the area of the first detection frame and the area of the second detection frame;
a second determining module, configured to determine a ratio of the first area region to the second area region as the tracking area overlapping ratio.
13. The apparatus according to claim 11, wherein the fourth determining unit comprises:
a third determining module, configured to determine the third tracking confidence level according to the following formula:
Score3＝β·IOU(TTrack,DDet)·SiamAppearance·Score1
wherein, Score1Is the first tracking confidence, beta is a hyperparameter, SiamAppearanceTracking the appearance similarity, T, for the targetTrackAnd DDetIndicating a first and a second detection box, IOU (T)Track,DDet) Is the tracking area overlap ratio.
14. The apparatus of claim 11, wherein the apparatus comprises:
a second obtaining unit, configured to obtain an appearance feature of a third detection frame of a first target object and an appearance feature of a fourth detection frame of a second target object when a third detection frame of the first target object and the fourth detection frame exist in the t-th frame image and an overlapping area between the third detection frame and the fourth detection frame is greater than or equal to a third preset threshold;
a fifth determining unit, configured to determine a first tracking appearance feature similarity according to the appearance feature of the first detection frame and the appearance feature of the third detection frame;
a sixth determining unit, configured to determine a second tracking appearance feature similarity according to the appearance features of the first detection frame and the fourth detection frame, respectively;
a seventh determining unit, configured to determine, as the target tracking appearance similarity, a determination that a value of the first tracking appearance feature similarity is the largest among the second tracking appearance feature similarities.
15. A computer-readable storage medium, comprising a stored program, wherein the program is operable to perform the method of any one of claims 1 to 7.
16. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 7 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010955180.4A CN112150505A (en) | 2020-09-11 | 2020-09-11 | Target object tracker updating method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010955180.4A CN112150505A (en) | 2020-09-11 | 2020-09-11 | Target object tracker updating method and device, storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112150505A true CN112150505A (en) | 2020-12-29 |
Family
ID=73889578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010955180.4A Pending CN112150505A (en) | 2020-09-11 | 2020-09-11 | Target object tracker updating method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112150505A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112836682A (en) * | 2021-03-04 | 2021-05-25 | 广东建邦计算机软件股份有限公司 | Method and device for identifying object in video, computer equipment and storage medium |
| WO2022237902A1 (en) * | 2021-05-14 | 2022-11-17 | 长沙智能驾驶研究院有限公司 | Method, apparatus, and device for detecting object, and computer storage medium |
| CN112836682B (en) * | 2021-03-04 | 2024-05-28 | 广东建邦计算机软件股份有限公司 | Method, device, computer equipment and storage medium for identifying object in video |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050031165A1 (en) * | 2003-08-08 | 2005-02-10 | Lockheed Martin Corporation. | Method and apparatus for tracking an object |
| CN107292907A (en) * | 2017-07-14 | 2017-10-24 | 灵动科技（北京）有限公司 | A kind of method to following target to be positioned and follow equipment |
| WO2018121286A1 (en) * | 2016-12-30 | 2018-07-05 | 纳恩博（北京）科技有限公司 | Target tracking method and device |
| US20180285647A1 (en) * | 2017-03-28 | 2018-10-04 | Qualcomm Incorporated | Methods and systems for performing sleeping object detection and tracking in video analytics |
| CN108898624A (en) * | 2018-06-12 | 2018-11-27 | 浙江大华技术股份有限公司 | A kind of method, apparatus of moving body track, electronic equipment and storage medium |
| CN109903310A (en) * | 2019-01-23 | 2019-06-18 | 平安科技（深圳）有限公司 | Method for tracking target, device, computer installation and computer storage medium |
| CN110033012A (en) * | 2018-12-28 | 2019-07-19 | 华中科技大学 | A kind of production method for tracking target based on channel characteristics weighted convolution neural network |
| CN110807377A (en) * | 2019-10-17 | 2020-02-18 | 浙江大华技术股份有限公司 | Target tracking and intrusion detection method, device and storage medium |
| CN110895819A (en) * | 2018-09-12 | 2020-03-20 | 长沙智能驾驶研究院有限公司 | Target tracking method, target tracking device, computer-readable storage medium and computer equipment |
| CN111369590A (en) * | 2020-02-27 | 2020-07-03 | 北京三快在线科技有限公司 | Multi-target tracking method and device, storage medium and electronic equipment |
-
2020
- 2020-09-11 CN CN202010955180.4A patent/CN112150505A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050031165A1 (en) * | 2003-08-08 | 2005-02-10 | Lockheed Martin Corporation. | Method and apparatus for tracking an object |
| WO2018121286A1 (en) * | 2016-12-30 | 2018-07-05 | 纳恩博（北京）科技有限公司 | Target tracking method and device |
| US20180285647A1 (en) * | 2017-03-28 | 2018-10-04 | Qualcomm Incorporated | Methods and systems for performing sleeping object detection and tracking in video analytics |
| CN107292907A (en) * | 2017-07-14 | 2017-10-24 | 灵动科技（北京）有限公司 | A kind of method to following target to be positioned and follow equipment |
| CN108898624A (en) * | 2018-06-12 | 2018-11-27 | 浙江大华技术股份有限公司 | A kind of method, apparatus of moving body track, electronic equipment and storage medium |
| CN110895819A (en) * | 2018-09-12 | 2020-03-20 | 长沙智能驾驶研究院有限公司 | Target tracking method, target tracking device, computer-readable storage medium and computer equipment |
| CN110033012A (en) * | 2018-12-28 | 2019-07-19 | 华中科技大学 | A kind of production method for tracking target based on channel characteristics weighted convolution neural network |
| CN109903310A (en) * | 2019-01-23 | 2019-06-18 | 平安科技（深圳）有限公司 | Method for tracking target, device, computer installation and computer storage medium |
| CN110807377A (en) * | 2019-10-17 | 2020-02-18 | 浙江大华技术股份有限公司 | Target tracking and intrusion detection method, device and storage medium |
| CN111369590A (en) * | 2020-02-27 | 2020-07-03 | 北京三快在线科技有限公司 | Multi-target tracking method and device, storage medium and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112836682A (en) * | 2021-03-04 | 2021-05-25 | 广东建邦计算机软件股份有限公司 | Method and device for identifying object in video, computer equipment and storage medium |
| CN112836682B (en) * | 2021-03-04 | 2024-05-28 | 广东建邦计算机软件股份有限公司 | Method, device, computer equipment and storage medium for identifying object in video |
| WO2022237902A1 (en) * | 2021-05-14 | 2022-11-17 | 长沙智能驾驶研究院有限公司 | Method, apparatus, and device for detecting object, and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10782688B2 (en) | Method, control apparatus, and system for tracking and shooting target | |
| US20200327680A1 (en) | Visual target tracking method and apparatus based on deep adversarial training | |
| CN112179330A (en) | Pose determination method and device of mobile equipment | |
| CN107872534B (en) | Information pushing method and device, server and readable storage medium | |
| CN109806585B (en) | Game display control method, device, equipment and storage medium | |
| CN110766724A (en) | Target tracking network training and tracking method and device, electronic equipment and medium | |
| CN110866497B (en) | Robot positioning and mapping method and device based on dotted line feature fusion | |
| CN111062400B (en) | Target matching method and device | |
| JP2019102049A (en) | Method and device for lane detection | |
| CN112634316B (en) | Target tracking method, device, equipment and storage medium | |
| CN111899285A (en) | Method and device for determining tracking track of target object and storage medium | |
| CN113344016A (en) | Deep migration learning method and device, electronic equipment and storage medium | |
| CN112561954B (en) | Method and device for determining tracking track of target object and storage medium | |
| CN112150505A (en) | Target object tracker updating method and device, storage medium and electronic device | |
| CN113052870A (en) | Target tracking method, target detection method and device | |
| US20200219000A1 (en) | Data Processing Method And Apparatus | |
| CN111047622A (en) | Method and device for matching objects in video, storage medium and electronic device | |
| CN108320332B (en) | Orientation determination method and device for virtual role | |
| WO2018202127A1 (en) | Information pushing method and device, storage medium, and electronic device | |
| CN110929057A (en) | Image processing method, device and system, storage medium and electronic device | |
| KR20170121950A (en) | Method and device for detecting abnormal account in online game | |
| KR20190040863A (en) | Method and apparatus for providing supervised and extended restart in random walks for ranking and link prediction in networks | |
| CN112368718B (en) | System and method for deep memory network | |
| CN111563134B (en) | Fingerprint database clustering method, system, equipment and storage medium of positioning system | |
| CN112561956B (en) | Video target tracking method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |