WO2020237674A1 - Procédé et appareil de suivi de cible, et véhicule aérien sans pilote - Google Patents

Procédé et appareil de suivi de cible, et véhicule aérien sans pilote Download PDF

Info

Publication number
WO2020237674A1
WO2020237674A1 PCT/CN2019/089668 CN2019089668W WO2020237674A1 WO 2020237674 A1 WO2020237674 A1 WO 2020237674A1 CN 2019089668 W CN2019089668 W CN 2019089668W WO 2020237674 A1 WO2020237674 A1 WO 2020237674A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image frame
targets
preset
determined
Prior art date
Application number
PCT/CN2019/089668
Other languages
English (en)
Chinese (zh)
Inventor
杨凌霄
曹子晟
胡攀
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980009924.9A priority Critical patent/CN111684491A/zh
Priority to PCT/CN2019/089668 priority patent/WO2020237674A1/fr
Publication of WO2020237674A1 publication Critical patent/WO2020237674A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention relates to the field of image technology, in particular to a target tracking method, a target tracking device and an unmanned aerial vehicle.
  • Gaussian function is mainly used to estimate the motion state of the target to realize the tracking of the target.
  • Gaussian sampling is mainly performed on the previous frame of the target to generate a large number of possible positions of the target in the current frame.
  • the Gaussian function is generally generated randomly, it is not guaranteed to accurately estimate the actual position of the target in various motion modes, so a large number of possible positions need to be generated, so that the possible positions may contain the actual position of the target with a high probability Come in.
  • the present invention provides a target tracking method, a target tracking device and an unmanned aerial vehicle to solve the problems of large calculation amount and easy tracking errors when judging the actual position of the target in related technologies.
  • a target tracking method including:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • a target tracking device which includes a processor configured to execute the following steps:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • an unmanned aerial vehicle which includes the device described in any of the foregoing embodiments.
  • each pending target and the preset target can be determined According to the similarity, at least one candidate target is determined among multiple pending targets according to the similarity.
  • multiple undetermined targets are screened based on the estimated target, so that relatively accurate and small number of candidate targets are obtained, and then the actual target is determined among the candidate targets through the preset tracking model.
  • the amount of data processed by the preset tracking model can be greatly reduced, thereby shortening the time to determine the actual target, speeding up the tracking, and because the estimated target is passed.
  • the relatively accurate results obtained by the preset correlation filtering model so the actual target is determined for a small number and relatively accurate alternative targets, which can reduce the possibility of identifying similar targets around the actual target as the actual target, thereby improving the accuracy of tracking Sex.
  • Fig. 1 is a schematic flowchart of a target tracking method according to an embodiment of the present disclosure.
  • Fig. 2 is a schematic diagram showing a pending target according to an embodiment of the present disclosure.
  • Fig. 3 is a schematic diagram showing an estimated target according to an embodiment of the present disclosure.
  • Fig. 4 is a schematic diagram showing an alternative target according to an embodiment of the present disclosure.
  • 5A and 5B are schematic diagrams of tracking a target according to the target tracking method according to an embodiment of the present disclosure.
  • Fig. 6 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 7 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 8 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • Fig. 9 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 10 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 11 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 12 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • Fig. 13 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 14 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 15 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 1 is a schematic flowchart of a target tracking method according to an embodiment of the present disclosure.
  • the target tracking method shown in the embodiments of the present disclosure can be applied to devices with image capture functions, and can be independent image capture devices, such as cameras, video cameras, etc., or devices equipped with image capture devices, such as mobile terminals (mobile phones, Tablet computers, etc.), unmanned equipment (drones, unmanned vehicles, etc.).
  • the target tracking method may include the following steps:
  • Step S1 based on the target in the first image frame, determine a plurality of pending targets in the second image frame; the first image frame and the second image frame have timing correlation in the code stream;
  • the device to which the target tracking method is applicable can continuously collect multiple frames of images, and based on the target in the first image frame in the multiple frames of images, multiple pending targets can be determined in the second image frame.
  • the possible position of the target in the second image frame can be predicted as the pending target center.
  • first image frame and the second image frame may be two consecutive images in the multi-frame image, or two non-consecutive images in the multi-frame image, which can be specifically set as required.
  • the first image frame and the second image frame have time-series correlation in the code stream, which may mean that the second image frame is located after the first image frame in time sequence. In this case, the first image frame precedes the second image
  • the movement state of the target in the first image frame can be determined first, and then according to the movement state of the target in the first image frame, the target may be predicted in the second image frame that is subsequently acquired or will be acquired. The position that appears is the center of the pending target.
  • the second image frame may be an image frame adjacent to the first image frame.
  • the first image frame is the i-th image frame
  • the second image frame is the i+1-th image frame. Frame image.
  • the second image frame may be an image frame that is not adjacent to the first image frame.
  • the first image frame is the i-th image
  • the second image frame is the i+th image.
  • k is greater than 1.
  • Fig. 2 is a schematic diagram showing a pending target according to an embodiment of the present disclosure.
  • the target to be determined can be represented by a rectangular box.
  • the center of the rectangular box coincides with the center of the target to be determined.
  • the size of the rectangular box can be the same as the circumscribed rectangle of the target to be determined, or the Region of Interest (ROI). .
  • Step S2 Estimate the predicted target of the target in the first image frame in the second image frame by using a preset correlation filter model
  • the estimated target in the second image frame of the target in the first image frame is estimated through a preset correlation filter model. Specifically, an area may be determined in the second image frame first. The target in the image frame is at least partially overlapped. Preferably, the center of the region coincides with the center of the target in the first image frame.
  • the size of the region can be set as needed, and then the features of the changed region are extracted, and the relevant filter model is preset in Convolution is performed on the extracted features, and then the result of the convolution is converted to the frequency domain through Fast Fourier Transformation (FFT) to determine the coordinates of the point with the largest response in the frequency domain in the second image frame , The final determined coordinates are the center, and the estimated target is determined in the second image frame based on the size of the target in the first image frame.
  • FFT Fast Fourier Transformation
  • the point multiplication of features can be used in the frequency domain to replace the relatively complicated convolution process, thereby reducing the time-consuming determination of the estimated target in the second image frame .
  • the preset correlation filter model is obtained by pre-training, specifically based on the learning and judgment of the target and the location information around the target, it is estimated that the target in the first image frame is in the second image frame. In the process of estimating the target, using the preset correlation filtering model can determine the estimated target relatively accurately.
  • the predicted target of the target in the first image frame in the second image frame can be quickly and relatively accurately estimated.
  • Fig. 3 is a schematic diagram showing an estimated target according to an embodiment of the present disclosure.
  • an estimated target can be obtained.
  • step S1 and step S2 is in no particular order.
  • step S1 may be executed before step S2, or the execution order may be adjusted as needed, for example, step S2 may be executed before step S1. Or perform step S1 and step S2 at the same time.
  • Step S3 determining at least one candidate target among the multiple pending targets according to the similarity between the multiple pending targets and the estimated targets;
  • Step S4 Determine the actual target among at least one candidate target through a preset tracking model.
  • the difference between each pending target and the preset target can be determined. Based on the similarity, at least one candidate target is determined among the multiple pending targets according to the similarity, for example, the pending target with the greatest similarity is selected as the candidate target, or the pending target with the similarity greater than the preset value is selected as the candidate target.
  • Fig. 4 is a schematic diagram showing an alternative target according to an embodiment of the present disclosure.
  • the undetermined target whose similarity is greater than the preset value can be selected as the candidate target. Based on this, multiple alternative targets can be determined among the undetermined targets shown in Fig. 2. Since the determined candidate target is different from The estimated target in Fig. 3 has high similarity, so the positions of multiple candidate targets in Fig. 4 are highly concentrated.
  • multiple undetermined targets are screened based on the estimated target, so that relatively accurate and small number of candidate targets are obtained, and then the actual target is determined among the candidate targets through the preset tracking model.
  • the amount of data processed by the preset tracking model can be greatly reduced, thereby shortening the time to determine the actual target, speeding up the tracking, and because the estimated target is passed.
  • the relatively accurate results obtained by the preset correlation filtering model so the actual target is determined for a small number and relatively accurate alternative targets, which can reduce the possibility of identifying similar targets around the actual target as the actual target, thereby improving the accuracy of tracking Sex.
  • 5A and 5B are schematic diagrams of tracking a target according to the target tracking method according to an embodiment of the present disclosure.
  • the actual target can be determined in multiple frames (eg, three frames) of images.
  • the target tracking method described in this embodiment may be suitable for tracking a human body
  • the target tracking method described in this embodiment may be suitable for tracking objects such as vehicles.
  • the method described in this embodiment is not limited to the above-mentioned embodiment for tracking human bodies and vehicles. It can also be based on other types of targets such as faces, and can be specifically configured as required.
  • the number of frames per second (FPS) of video processed can be increased by 20%, which means that the time to determine the actual target is shortened.
  • Speed up the tracking, and the similarity between the determined actual target and the human-labeled target can be increased by 2%, that is, the tracking accuracy is improved.
  • Fig. 6 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure. As shown in FIG. 6, the determining multiple targets to be determined in the second image frame based on the targets in the first image frame includes:
  • Step S11 Based on the target in the first image frame, a plurality of undetermined targets are determined in the second image frame by a preset Gaussian function, wherein the expected value of the preset Gaussian function is equal to the center of the target in the first image frame coordinate of.
  • multiple targets to be determined in the second image frame can be obtained by a preset Gaussian function, and the expected value of the preset Gaussian function is equal to the center of the target in the first image frame coordinate of.
  • the movement state of the target in the first image frame such as the size of the speed, the direction of the speed, the position of the target in the first image frame, etc., it is possible to generate a large number of targets in the second image frame through a preset Gaussian function. location information.
  • the generated position information can be used as the center of the target to be determined, and then the target to be determined is represented by a rectangular box.
  • the center of the rectangular box coincides with the center of the target to be determined.
  • the size of the rectangular box can be the same as the circumscribed rectangle of the target to be determined (according to the first image frame The circumscribed rectangle of the target is determined), or the region of interest (which can be determined according to the region of interest of the target in the first image frame) is the same.
  • Fig. 7 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the estimating the predicted target of the target in the first image frame in the second image frame by using a preset correlation filtering model includes:
  • Step S21 Determine a tracking area in the second image frame, wherein the tracking area at least partially overlaps with the target in the first image frame, and the size of the tracking area is the same as that in the first image frame. N times the size of the target, n>1;
  • Step S22 Estimate the predicted target of the target in the first image frame in the second image frame in the tracking area through a preset correlation filter model.
  • a tracking area with a size larger than the size of the target in the first image frame may be determined in the second image frame, and then through a preset correlation filter model, it is estimated that the target in the first image frame is in the first image frame in the tracking area. 2. The estimated target in the image frame.
  • the size of the tracking area is n times the size of the target in the first image frame, and n>1, that is, the size of the tracking area is greater than the size of the target in the first image frame, and the tracking area is the same as that in the first image frame.
  • the targets are at least partially overlapped, so compared to estimating the estimated target in an area with the same size as the target in the first image frame, since the size of the tracking area is larger, the estimated target after the target movement is more likely to be included. It can improve the accuracy of estimating the estimated target.
  • the size of the tracking area is larger than the size of the target in the first image frame, it requires more calculations to estimate the estimated target in a larger size area, but because this embodiment uses a preset correlation filter model, in the tracking area Estimate the predicted target of the target in the first image frame in the second image frame, in which the result of the convolution is converted to the frequency domain by fast Fourier transform, and the point multiplication of features can be used in the frequency domain to replace the relatively complex Convolution process, so even if the estimated target is estimated in a larger area, because the calculation process is simple, it will not produce too much extra time, and still can ensure high calculation efficiency.
  • the center of the tracking area coincides with the center of the target in the first image frame.
  • the target since the target moves from the position in the first image frame to the position in the second image frame, it starts from the center of the target in the first image frame, so the actual target of the second image frame is more It may appear near the center of the target in the first image frame, so the center of the determined tracking area can coincide with the center of the target in the first image frame, so that the estimation can be estimated near the center of the target in the first image frame.
  • the target as opposed to estimating the estimated target while being far away from the center of the target in the first image frame, is beneficial to ensure that the estimated estimated target matches the actual position of the target in the second image frame.
  • n 3.
  • the estimated target is estimated in the larger tracking area, the more calculation is required, and the speed of the target is generally not too large, that is, the position of the target in the second image frame is different from that in the first image frame.
  • the location of the target will not be too far, so setting an overly large tracking area will hardly improve the accuracy of estimating the target in the second image frame, but will increase the amount of calculation to a greater extent.
  • n can be set according to the speed of the target in the first image frame. The higher the speed, the longer the target position in the second image frame and the position in the first image frame may be. Therefore, n can be set to be larger to ensure that the tracking area can include the position of the target in the second image frame, thereby ensuring that the estimated target has a higher accuracy rate.
  • n can be determined according to the time between the second image frame and the first image frame. The longer the time between the second image frame and the first image frame, the greater the time the target corresponds to the first image frame. The distance from time to the corresponding time of the second image frame may be greater, the target is in the center of the first image frame, and the distance to the center of the target in the second image frame may be farther, then the larger n can be set , So as to ensure that the tracking area can contain the position of the target in the second image frame with a greater probability, so as to ensure that the estimated target has a higher accuracy rate.
  • Fig. 8 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure. As shown in Figure 8, the method further includes:
  • Step S5 using the feature of the tracking area as an input and the center of the actual target as an output constituting sample, and updating the first training sample set corresponding to the preset correlation filtering model;
  • Step S6 According to the updated first training sample set, the preset correlation filtering model is updated through machine learning.
  • the preset correlation filtering model may be pre-trained according to the first training sample set, the first training sample set contains multiple samples, and the input of the sample is the target in the previous frame (for example, the first image frame)
  • the characteristics of the area such as the circumscribed rectangle of the target, the area of interest of the target, the characteristics of a certain frame of the target, etc.
  • the output of the sample is the coordinates of the target center in the current frame (for example, the second image frame).
  • the characteristics of the tracking area may be used as input, and the center of the actual target may be used as the output to constitute sample a, and the first training sample set corresponding to the preset correlation filtering model may be updated.
  • the sample a can be added to the first training sample set, so that the updated first training sample set adds one sample a.
  • the preset correlation filtering model is updated through machine learning, so that the updated preset correlation filtering model is more in line with the target's motion state in the last two frames, so that the updated The preset correlation filter model can estimate the estimated target more accurately.
  • Fig. 9 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the estimation of the target in the first image frame and the predicted target in the second image frame in the tracking area by using a preset correlation filtering model includes:
  • Step S222 performing convolution on the feature by using a preset correlation filtering model
  • Step S223 Convert the convolution result to the frequency domain through fast Fourier transform
  • Step S224 Determine the coordinates corresponding to the tracking area of the point with the largest response in the frequency domain
  • Step S225 taking the coordinates as a center and determining an estimated target in the second image frame based on the size of the target in the first image frame.
  • the target in the first image frame is estimated in the tracking area through a preset correlation filter model, and the target in the second image frame is estimated.
  • the characteristics of the tracking area can be extracted first, and then the correlation
  • the filter model performs convolution on the extracted features, and then converts the result of the convolution to the frequency domain through fast Fourier transform, and then determines the coordinates of the point with the largest response in the frequency domain in the second image frame, and finally determines
  • the coordinates are the center, and the estimated target is determined in the second image frame based on the size of the target in the first image frame.
  • the point multiplication of features can be used in the frequency domain to replace the relatively complicated convolution process, thereby reducing the time-consuming determination of the estimated target in the second image frame .
  • the preset correlation filter model is obtained by pre-training, specifically based on the learning and judgment of the target and the location information around the target, it is estimated that the target in the first image frame is in the second image frame. In the process of estimating the target, using the preset correlation filtering model can determine the estimated target relatively accurately.
  • the predicted target of the target in the first image frame in the second image frame can be quickly and relatively accurately estimated.
  • Fig. 10 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S31 Determine at least one candidate target among the plurality of pending targets according to the intersection ratio of the plurality of pending targets and the estimated targets.
  • the similarity between the pending target and the estimated target can be determined according to the Intersection over Union (IoU) of the pending target and the estimated target, where the intersection ratio refers to the location information of the pending target The intersection with the location information of the estimated target, divided by the union of the location information of the target to be determined and the location information of the estimated target.
  • IoU Intersection over Union
  • the circumscribed rectangle of the target to be determined indicates the location information of the target to be determined
  • the circumscribed rectangle of the estimated target indicates the location information of the estimated target.
  • the intersection ratio of the target to be determined and the estimated target is the ratio between the circumscribed rectangle of the target to be determined and the estimated target
  • the area where the circumscribed rectangle overlaps is divided by the area where the circumscribed rectangle of the target to be determined and the circumscribed rectangle of the estimated target merge into one.
  • intersection ratio the greater the overlap between the pending target and the estimated target, that is, the higher the similarity between the pending target and the estimated target, which can be based on the intersection of multiple pending targets and the estimated target.
  • At least one candidate target is determined among the multiple pending targets, for example, the pending target with the largest intersection ratio is selected as the candidate target, or the pending target with the intersection ratio greater than the preset ratio is selected as the candidate target.
  • Fig. 11 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S32 extracting the feature of the pixel at the preset position in the target to be determined, and extracting the feature of the pixel at the corresponding position in the estimated target;
  • Step S33 According to the similarity between the feature of the pixel at the preset position and the feature of the pixel at the corresponding position, at least one candidate target is determined from the plurality of pending targets.
  • the preset position may be one or more positions, including at least the position of one pixel in the pending area and at most including the position of each pixel in the pending area.
  • the pixel at the preset position is the pixel at the center position of the target to be determined
  • the corresponding position in the estimated target refers to the pixel at the center position of the preset target.
  • the preset position is the pixel in the first row of the target to be determined.
  • the corresponding position in the target refers to the pixel in the first row of the preset target.
  • both the estimated target and the undetermined target are the results of target estimation, the higher the similarity between the feature of the pixel at the preset position in the undetermined target and the feature of the pixel at the corresponding position in the estimated target, the more the target and the estimated target are similar. It may be the same result estimated for the target. Therefore, according to the similarity between the feature of the pixel at the preset position and the feature of the pixel at the corresponding position, at least one candidate target can be determined among multiple pending targets, such as selecting similarity The highest estimated target is selected as an alternative target, or an estimated target with a similarity higher than a preset threshold is selected as an alternative target.
  • Fig. 12 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S34 Determine, among the multiple pending targets, the pending target with the greatest similarity to the estimated target as the candidate target.
  • the target with the greatest similarity to the estimated target can be determined as the candidate target. According to this, only one candidate target needs to be determined, which can effectively reduce the subsequent passing of preset tracking
  • the model determines the calculation amount of the actual target in at least one candidate target.
  • the actual target is determined in at least one candidate target through the preset tracking model, and only one candidate target needs to be determined whether it is actual Target, if it is determined that the candidate target is not an actual target, prompt information may be generated, and if it is determined that the candidate target is an actual target, tracking may be performed based on information such as the location of the actual target.
  • Fig. 13 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S35 sort the similarities between the plurality of pending targets and the estimated targets
  • Step S36 Determine the candidate target among the multiple pending targets according to the preset order of the similarity.
  • the similarities between the multiple pending targets and the estimated targets can be sorted, which can be sorted from large to small, or from small to small. Big sorting, taking the sorting from big to small as an example, the undetermined target corresponding to the similarity ranked before the preset order can be determined as the candidate target.
  • the preset order can be directly expressed in order, for example, the undetermined target corresponding to the similarity ranked before the 10th similarity is determined as the candidate target; the preset order can be expressed in proportion, for example, the top percent The undetermined target corresponding to the similarity of ten is determined as the candidate target.
  • Fig. 14 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S37 Determine the undetermined target corresponding to the similarity greater than the preset value as the candidate target.
  • each similarity can be compared with a preset value to determine a similarity greater than the preset value, which will then be greater than the preset value.
  • the undetermined target corresponding to the similarity of the value is determined as the candidate target. Among them, if there is no similarity greater than the preset value, a prompt message can be generated.
  • Fig. 15 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure. As shown in Figure 15, the method further includes:
  • Step S7 taking the feature of the target in the first image frame as input, and taking the center of the actual target as the output constituent sample, and updating the second training sample set corresponding to the preset tracking model;
  • Step S8 according to the updated second training sample set, update the preset tracking model through machine learning.
  • the preset tracking model (including but not limited to a neural network, such as a convolutional neural network) may be pre-trained according to the second training sample set, the second training sample set contains multiple samples, and the input of the sample It is the feature of the target area in the previous frame (such as the first image frame), such as the feature of the circumscribed rectangle of the target, the feature of the target area of interest, the feature of a certain frame of image where the target is located, etc.
  • the output of the sample is the current The coordinates of the target center of the frame (for example, the second image frame).
  • the characteristics of the tracking area may be used as input, and the center of the actual target may be used as the output to constitute the sample b, and the second training sample set corresponding to the preset tracking model may be updated.
  • the sample b can be added to the second training sample set, so that the updated second training sample set adds one sample b.
  • the preset tracking model is updated through machine learning, so that the updated preset tracking model is more in line with the target's motion state in the last two frames, so that the updated preset can be used later
  • the tracking model can estimate the estimated target more accurately.
  • the embodiment of the present disclosure also provides a target tracking device, including a processor, and the processor is configured to execute the following steps:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • the processor is configured to execute the following steps:
  • multiple undetermined targets are determined in the second image frame through a preset Gaussian function, wherein the expected value of the preset Gaussian function is equal to the coordinates of the center of the target in the first image frame.
  • the processor is configured to execute the following steps:
  • a tracking area is determined in the second image frame, wherein the tracking area at least partially overlaps with the target in the first image frame, and the size of the tracking area is the same as that of the target in the first image frame. N times the size, n>1;
  • the center of the tracking area coincides with the center of the target in the first image frame.
  • n 3.
  • the processor is further configured to execute the following steps:
  • the preset correlation filtering model is updated through machine learning.
  • the processor is configured to execute the following steps:
  • the processor is configured to execute the following steps:
  • At least one candidate target is determined among the plurality of pending targets according to the intersection ratio of the plurality of pending targets and the estimated targets.
  • the processor is configured to execute the following steps:
  • At least one candidate target is determined from the plurality of pending targets.
  • the processor is configured to execute the following steps:
  • the candidate target is determined to be the candidate target with the greatest similarity to the estimated target among the plurality of pending targets.
  • the processor is configured to execute the following steps:
  • the candidate target is determined among the plurality of pending targets according to the preset order of the similarity.
  • the processor is configured to execute the following steps:
  • the undetermined target corresponding to the similarity greater than the preset value is determined as the candidate target.
  • the processor is further configured to execute the following steps:
  • the preset tracking model is updated through machine learning.
  • An embodiment of the present disclosure also provides an unmanned aerial vehicle, including the target tracking device described in any of the foregoing embodiments.
  • the systems, devices, modules, or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or implemented by products with certain functions.
  • the functions are divided into various units and described separately.
  • the functions of each unit can be implemented in the same one or more software and/or hardware.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.
  • the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de suivi de cible comprenant les étapes suivantes : sur la base d'une cible dans une première trame d'image, détermination de multiples cibles à déterminer dans une seconde trame d'image ; au moyen d'un modèle de filtrage pertinent prédéfini, estimation d'une cible pré-estimée, dans la seconde trame d'image, de la cible dans la première trame d'image ; en fonction d'une similarité entre les multiples cibles à déterminer et la cible pré-estimée, détermination d'au moins une cible alternative parmi les multiples cibles à déterminer ; et détermination d'une cible réelle dans la ou les cibles alternatives au moyen d'un modèle de suivi prédéfini. Selon les modes de réalisation de la présente invention, le temps de détermination de la cible réelle peut être raccourci et une vitesse de suivi peut être accélérée, ce qui permet d'améliorer la précision de suivi.
PCT/CN2019/089668 2019-05-31 2019-05-31 Procédé et appareil de suivi de cible, et véhicule aérien sans pilote WO2020237674A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980009924.9A CN111684491A (zh) 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机
PCT/CN2019/089668 WO2020237674A1 (fr) 2019-05-31 2019-05-31 Procédé et appareil de suivi de cible, et véhicule aérien sans pilote

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/089668 WO2020237674A1 (fr) 2019-05-31 2019-05-31 Procédé et appareil de suivi de cible, et véhicule aérien sans pilote

Publications (1)

Publication Number Publication Date
WO2020237674A1 true WO2020237674A1 (fr) 2020-12-03

Family

ID=72451448

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/089668 WO2020237674A1 (fr) 2019-05-31 2019-05-31 Procédé et appareil de suivi de cible, et véhicule aérien sans pilote

Country Status (2)

Country Link
CN (1) CN111684491A (fr)
WO (1) WO2020237674A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393492A (zh) * 2021-05-27 2021-09-14 浙江大华技术股份有限公司 目标跟踪方法、装置、电子装置和存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190635A (zh) * 2018-07-25 2019-01-11 北京飞搜科技有限公司 基于分类cnn的目标追踪方法、装置及电子设备
CN109697727A (zh) * 2018-11-27 2019-04-30 哈尔滨工业大学（深圳） 基于相关滤波和度量学习的目标跟踪方法、系统及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105989367B (zh) * 2015-02-04 2019-06-28 阿里巴巴集团控股有限公司 目标获取方法及设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190635A (zh) * 2018-07-25 2019-01-11 北京飞搜科技有限公司 基于分类cnn的目标追踪方法、装置及电子设备
CN109697727A (zh) * 2018-11-27 2019-04-30 哈尔滨工业大学（深圳） 基于相关滤波和度量学习的目标跟踪方法、系统及存储介质

Also Published As

Publication number Publication date
CN111684491A (zh) 2020-09-18

Similar Documents

Publication Publication Date Title
Kamal et al. Automatic traffic sign detection and recognition using SegU-Net and a modified Tversky loss function with L1-constraint
CN107767405B (zh) 一种融合卷积神经网络的核相关滤波目标跟踪方法
CN110378264B (zh) 目标跟踪方法及装置
CN108898086B (zh) 视频图像处理方法及装置、计算机可读介质和电子设备
CN108062525B (zh) 一种基于手部区域预测的深度学习手部检测方法
CN109543641B (zh) 一种实时视频的多目标去重方法、终端设备及存储介质
WO2014174932A1 (fr) Dispositif de traitement d'image, programme, et procédé de traitement d'image
CN111931764B (zh) 一种目标检测方法、目标检测框架及相关设备
JP2018523877A (ja) オブジェクト追跡のためのシステムおよび方法
US20130342636A1 (en) Image-Based Real-Time Gesture Recognition
US20230042187A1 (en) Behavior recognition method and system, electronic device and computer-readable storage medium
CN111401293B (zh) 一种基于Head轻量化Mask Scoring R-CNN的手势识别方法
CN110796686A (zh) 目标跟踪方法及设备、存储装置
WO2021017283A1 (fr) Procédé et appareil de suivi en ligne basé sur un procédé hors ligne, dispositif informatique et support d'enregistrement
Kim et al. Fast pedestrian detection in surveillance video based on soft target training of shallow random forest
US20210342593A1 (en) Method and apparatus for detecting target in video, computing device, and storage medium
CN111008576B (zh) 行人检测及其模型训练、更新方法、设备及可读存储介质
WO2022142417A1 (fr) Procédé et appareil de suivi de cible, dispositif électronique et support d'enregistrement
WO2021047587A1 (fr) Procédé de reconnaissance de geste, dispositif électronique, support de stockage lisible par ordinateur et puce
CN110427915B (zh) 用于输出信息的方法和装置
CN113128368B (zh) 一种人物交互关系的检测方法、装置及系统
CN112381071A (zh) 一种视频流中目标的行为分析方法、终端设备及介质
WO2021169642A1 (fr) Procédé et système de détermination de rotation de globe oculaire basée sur une vidéo
CN112183649A (zh) 一种用于对金字塔特征图进行预测的算法
CN116402850A (zh) 一种面向智能驾驶的多目标跟踪方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19931398

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19931398

Country of ref document: EP

Kind code of ref document: A1