WO2020237674A1 - 目标跟踪方法、目标跟踪装置和无人机 - Google Patents

目标跟踪方法、目标跟踪装置和无人机 Download PDF

Info

Publication number
WO2020237674A1
WO2020237674A1 PCT/CN2019/089668 CN2019089668W WO2020237674A1 WO 2020237674 A1 WO2020237674 A1 WO 2020237674A1 CN 2019089668 W CN2019089668 W CN 2019089668W WO 2020237674 A1 WO2020237674 A1 WO 2020237674A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image frame
targets
preset
determined
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/089668
Other languages
English (en)
French (fr)
Inventor
杨凌霄
曹子晟
胡攀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Original Assignee
SZ DJI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SZ DJI Technology Co Ltd filed Critical SZ DJI Technology Co Ltd
Priority to PCT/CN2019/089668 priority Critical patent/WO2020237674A1/zh
Priority to CN201980009924.9A priority patent/CN111684491A/zh
Publication of WO2020237674A1 publication Critical patent/WO2020237674A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention relates to the field of image technology, in particular to a target tracking method, a target tracking device and an unmanned aerial vehicle.
  • Gaussian function is mainly used to estimate the motion state of the target to realize the tracking of the target.
  • Gaussian sampling is mainly performed on the previous frame of the target to generate a large number of possible positions of the target in the current frame.
  • the Gaussian function is generally generated randomly, it is not guaranteed to accurately estimate the actual position of the target in various motion modes, so a large number of possible positions need to be generated, so that the possible positions may contain the actual position of the target with a high probability Come in.
  • the present invention provides a target tracking method, a target tracking device and an unmanned aerial vehicle to solve the problems of large calculation amount and easy tracking errors when judging the actual position of the target in related technologies.
  • a target tracking method including:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • a target tracking device which includes a processor configured to execute the following steps:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • an unmanned aerial vehicle which includes the device described in any of the foregoing embodiments.
  • each pending target and the preset target can be determined According to the similarity, at least one candidate target is determined among multiple pending targets according to the similarity.
  • multiple undetermined targets are screened based on the estimated target, so that relatively accurate and small number of candidate targets are obtained, and then the actual target is determined among the candidate targets through the preset tracking model.
  • the amount of data processed by the preset tracking model can be greatly reduced, thereby shortening the time to determine the actual target, speeding up the tracking, and because the estimated target is passed.
  • the relatively accurate results obtained by the preset correlation filtering model so the actual target is determined for a small number and relatively accurate alternative targets, which can reduce the possibility of identifying similar targets around the actual target as the actual target, thereby improving the accuracy of tracking Sex.
  • Fig. 1 is a schematic flowchart of a target tracking method according to an embodiment of the present disclosure.
  • Fig. 2 is a schematic diagram showing a pending target according to an embodiment of the present disclosure.
  • Fig. 3 is a schematic diagram showing an estimated target according to an embodiment of the present disclosure.
  • Fig. 4 is a schematic diagram showing an alternative target according to an embodiment of the present disclosure.
  • 5A and 5B are schematic diagrams of tracking a target according to the target tracking method according to an embodiment of the present disclosure.
  • Fig. 6 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 7 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 8 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • Fig. 9 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 10 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 11 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 12 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • Fig. 13 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 14 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 15 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • Fig. 1 is a schematic flowchart of a target tracking method according to an embodiment of the present disclosure.
  • the target tracking method shown in the embodiments of the present disclosure can be applied to devices with image capture functions, and can be independent image capture devices, such as cameras, video cameras, etc., or devices equipped with image capture devices, such as mobile terminals (mobile phones, Tablet computers, etc.), unmanned equipment (drones, unmanned vehicles, etc.).
  • the target tracking method may include the following steps:
  • Step S1 based on the target in the first image frame, determine a plurality of pending targets in the second image frame; the first image frame and the second image frame have timing correlation in the code stream;
  • the device to which the target tracking method is applicable can continuously collect multiple frames of images, and based on the target in the first image frame in the multiple frames of images, multiple pending targets can be determined in the second image frame.
  • the possible position of the target in the second image frame can be predicted as the pending target center.
  • first image frame and the second image frame may be two consecutive images in the multi-frame image, or two non-consecutive images in the multi-frame image, which can be specifically set as required.
  • the first image frame and the second image frame have time-series correlation in the code stream, which may mean that the second image frame is located after the first image frame in time sequence. In this case, the first image frame precedes the second image
  • the movement state of the target in the first image frame can be determined first, and then according to the movement state of the target in the first image frame, the target may be predicted in the second image frame that is subsequently acquired or will be acquired. The position that appears is the center of the pending target.
  • the second image frame may be an image frame adjacent to the first image frame.
  • the first image frame is the i-th image frame
  • the second image frame is the i+1-th image frame. Frame image.
  • the second image frame may be an image frame that is not adjacent to the first image frame.
  • the first image frame is the i-th image
  • the second image frame is the i+th image.
  • k is greater than 1.
  • Fig. 2 is a schematic diagram showing a pending target according to an embodiment of the present disclosure.
  • the target to be determined can be represented by a rectangular box.
  • the center of the rectangular box coincides with the center of the target to be determined.
  • the size of the rectangular box can be the same as the circumscribed rectangle of the target to be determined, or the Region of Interest (ROI). .
  • Step S2 Estimate the predicted target of the target in the first image frame in the second image frame by using a preset correlation filter model
  • the estimated target in the second image frame of the target in the first image frame is estimated through a preset correlation filter model. Specifically, an area may be determined in the second image frame first. The target in the image frame is at least partially overlapped. Preferably, the center of the region coincides with the center of the target in the first image frame.
  • the size of the region can be set as needed, and then the features of the changed region are extracted, and the relevant filter model is preset in Convolution is performed on the extracted features, and then the result of the convolution is converted to the frequency domain through Fast Fourier Transformation (FFT) to determine the coordinates of the point with the largest response in the frequency domain in the second image frame , The final determined coordinates are the center, and the estimated target is determined in the second image frame based on the size of the target in the first image frame.
  • FFT Fast Fourier Transformation
  • the point multiplication of features can be used in the frequency domain to replace the relatively complicated convolution process, thereby reducing the time-consuming determination of the estimated target in the second image frame .
  • the preset correlation filter model is obtained by pre-training, specifically based on the learning and judgment of the target and the location information around the target, it is estimated that the target in the first image frame is in the second image frame. In the process of estimating the target, using the preset correlation filtering model can determine the estimated target relatively accurately.
  • the predicted target of the target in the first image frame in the second image frame can be quickly and relatively accurately estimated.
  • Fig. 3 is a schematic diagram showing an estimated target according to an embodiment of the present disclosure.
  • an estimated target can be obtained.
  • step S1 and step S2 is in no particular order.
  • step S1 may be executed before step S2, or the execution order may be adjusted as needed, for example, step S2 may be executed before step S1. Or perform step S1 and step S2 at the same time.
  • Step S3 determining at least one candidate target among the multiple pending targets according to the similarity between the multiple pending targets and the estimated targets;
  • Step S4 Determine the actual target among at least one candidate target through a preset tracking model.
  • the difference between each pending target and the preset target can be determined. Based on the similarity, at least one candidate target is determined among the multiple pending targets according to the similarity, for example, the pending target with the greatest similarity is selected as the candidate target, or the pending target with the similarity greater than the preset value is selected as the candidate target.
  • Fig. 4 is a schematic diagram showing an alternative target according to an embodiment of the present disclosure.
  • the undetermined target whose similarity is greater than the preset value can be selected as the candidate target. Based on this, multiple alternative targets can be determined among the undetermined targets shown in Fig. 2. Since the determined candidate target is different from The estimated target in Fig. 3 has high similarity, so the positions of multiple candidate targets in Fig. 4 are highly concentrated.
  • multiple undetermined targets are screened based on the estimated target, so that relatively accurate and small number of candidate targets are obtained, and then the actual target is determined among the candidate targets through the preset tracking model.
  • the amount of data processed by the preset tracking model can be greatly reduced, thereby shortening the time to determine the actual target, speeding up the tracking, and because the estimated target is passed.
  • the relatively accurate results obtained by the preset correlation filtering model so the actual target is determined for a small number and relatively accurate alternative targets, which can reduce the possibility of identifying similar targets around the actual target as the actual target, thereby improving the accuracy of tracking Sex.
  • 5A and 5B are schematic diagrams of tracking a target according to the target tracking method according to an embodiment of the present disclosure.
  • the actual target can be determined in multiple frames (eg, three frames) of images.
  • the target tracking method described in this embodiment may be suitable for tracking a human body
  • the target tracking method described in this embodiment may be suitable for tracking objects such as vehicles.
  • the method described in this embodiment is not limited to the above-mentioned embodiment for tracking human bodies and vehicles. It can also be based on other types of targets such as faces, and can be specifically configured as required.
  • the number of frames per second (FPS) of video processed can be increased by 20%, which means that the time to determine the actual target is shortened.
  • Speed up the tracking, and the similarity between the determined actual target and the human-labeled target can be increased by 2%, that is, the tracking accuracy is improved.
  • Fig. 6 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure. As shown in FIG. 6, the determining multiple targets to be determined in the second image frame based on the targets in the first image frame includes:
  • Step S11 Based on the target in the first image frame, a plurality of undetermined targets are determined in the second image frame by a preset Gaussian function, wherein the expected value of the preset Gaussian function is equal to the center of the target in the first image frame coordinate of.
  • multiple targets to be determined in the second image frame can be obtained by a preset Gaussian function, and the expected value of the preset Gaussian function is equal to the center of the target in the first image frame coordinate of.
  • the movement state of the target in the first image frame such as the size of the speed, the direction of the speed, the position of the target in the first image frame, etc., it is possible to generate a large number of targets in the second image frame through a preset Gaussian function. location information.
  • the generated position information can be used as the center of the target to be determined, and then the target to be determined is represented by a rectangular box.
  • the center of the rectangular box coincides with the center of the target to be determined.
  • the size of the rectangular box can be the same as the circumscribed rectangle of the target to be determined (according to the first image frame The circumscribed rectangle of the target is determined), or the region of interest (which can be determined according to the region of interest of the target in the first image frame) is the same.
  • Fig. 7 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the estimating the predicted target of the target in the first image frame in the second image frame by using a preset correlation filtering model includes:
  • Step S21 Determine a tracking area in the second image frame, wherein the tracking area at least partially overlaps with the target in the first image frame, and the size of the tracking area is the same as that in the first image frame. N times the size of the target, n>1;
  • Step S22 Estimate the predicted target of the target in the first image frame in the second image frame in the tracking area through a preset correlation filter model.
  • a tracking area with a size larger than the size of the target in the first image frame may be determined in the second image frame, and then through a preset correlation filter model, it is estimated that the target in the first image frame is in the first image frame in the tracking area. 2. The estimated target in the image frame.
  • the size of the tracking area is n times the size of the target in the first image frame, and n>1, that is, the size of the tracking area is greater than the size of the target in the first image frame, and the tracking area is the same as that in the first image frame.
  • the targets are at least partially overlapped, so compared to estimating the estimated target in an area with the same size as the target in the first image frame, since the size of the tracking area is larger, the estimated target after the target movement is more likely to be included. It can improve the accuracy of estimating the estimated target.
  • the size of the tracking area is larger than the size of the target in the first image frame, it requires more calculations to estimate the estimated target in a larger size area, but because this embodiment uses a preset correlation filter model, in the tracking area Estimate the predicted target of the target in the first image frame in the second image frame, in which the result of the convolution is converted to the frequency domain by fast Fourier transform, and the point multiplication of features can be used in the frequency domain to replace the relatively complex Convolution process, so even if the estimated target is estimated in a larger area, because the calculation process is simple, it will not produce too much extra time, and still can ensure high calculation efficiency.
  • the center of the tracking area coincides with the center of the target in the first image frame.
  • the target since the target moves from the position in the first image frame to the position in the second image frame, it starts from the center of the target in the first image frame, so the actual target of the second image frame is more It may appear near the center of the target in the first image frame, so the center of the determined tracking area can coincide with the center of the target in the first image frame, so that the estimation can be estimated near the center of the target in the first image frame.
  • the target as opposed to estimating the estimated target while being far away from the center of the target in the first image frame, is beneficial to ensure that the estimated estimated target matches the actual position of the target in the second image frame.
  • n 3.
  • the estimated target is estimated in the larger tracking area, the more calculation is required, and the speed of the target is generally not too large, that is, the position of the target in the second image frame is different from that in the first image frame.
  • the location of the target will not be too far, so setting an overly large tracking area will hardly improve the accuracy of estimating the target in the second image frame, but will increase the amount of calculation to a greater extent.
  • n can be set according to the speed of the target in the first image frame. The higher the speed, the longer the target position in the second image frame and the position in the first image frame may be. Therefore, n can be set to be larger to ensure that the tracking area can include the position of the target in the second image frame, thereby ensuring that the estimated target has a higher accuracy rate.
  • n can be determined according to the time between the second image frame and the first image frame. The longer the time between the second image frame and the first image frame, the greater the time the target corresponds to the first image frame. The distance from time to the corresponding time of the second image frame may be greater, the target is in the center of the first image frame, and the distance to the center of the target in the second image frame may be farther, then the larger n can be set , So as to ensure that the tracking area can contain the position of the target in the second image frame with a greater probability, so as to ensure that the estimated target has a higher accuracy rate.
  • Fig. 8 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure. As shown in Figure 8, the method further includes:
  • Step S5 using the feature of the tracking area as an input and the center of the actual target as an output constituting sample, and updating the first training sample set corresponding to the preset correlation filtering model;
  • Step S6 According to the updated first training sample set, the preset correlation filtering model is updated through machine learning.
  • the preset correlation filtering model may be pre-trained according to the first training sample set, the first training sample set contains multiple samples, and the input of the sample is the target in the previous frame (for example, the first image frame)
  • the characteristics of the area such as the circumscribed rectangle of the target, the area of interest of the target, the characteristics of a certain frame of the target, etc.
  • the output of the sample is the coordinates of the target center in the current frame (for example, the second image frame).
  • the characteristics of the tracking area may be used as input, and the center of the actual target may be used as the output to constitute sample a, and the first training sample set corresponding to the preset correlation filtering model may be updated.
  • the sample a can be added to the first training sample set, so that the updated first training sample set adds one sample a.
  • the preset correlation filtering model is updated through machine learning, so that the updated preset correlation filtering model is more in line with the target's motion state in the last two frames, so that the updated The preset correlation filter model can estimate the estimated target more accurately.
  • Fig. 9 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the estimation of the target in the first image frame and the predicted target in the second image frame in the tracking area by using a preset correlation filtering model includes:
  • Step S222 performing convolution on the feature by using a preset correlation filtering model
  • Step S223 Convert the convolution result to the frequency domain through fast Fourier transform
  • Step S224 Determine the coordinates corresponding to the tracking area of the point with the largest response in the frequency domain
  • Step S225 taking the coordinates as a center and determining an estimated target in the second image frame based on the size of the target in the first image frame.
  • the target in the first image frame is estimated in the tracking area through a preset correlation filter model, and the target in the second image frame is estimated.
  • the characteristics of the tracking area can be extracted first, and then the correlation
  • the filter model performs convolution on the extracted features, and then converts the result of the convolution to the frequency domain through fast Fourier transform, and then determines the coordinates of the point with the largest response in the frequency domain in the second image frame, and finally determines
  • the coordinates are the center, and the estimated target is determined in the second image frame based on the size of the target in the first image frame.
  • the point multiplication of features can be used in the frequency domain to replace the relatively complicated convolution process, thereby reducing the time-consuming determination of the estimated target in the second image frame .
  • the preset correlation filter model is obtained by pre-training, specifically based on the learning and judgment of the target and the location information around the target, it is estimated that the target in the first image frame is in the second image frame. In the process of estimating the target, using the preset correlation filtering model can determine the estimated target relatively accurately.
  • the predicted target of the target in the first image frame in the second image frame can be quickly and relatively accurately estimated.
  • Fig. 10 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S31 Determine at least one candidate target among the plurality of pending targets according to the intersection ratio of the plurality of pending targets and the estimated targets.
  • the similarity between the pending target and the estimated target can be determined according to the Intersection over Union (IoU) of the pending target and the estimated target, where the intersection ratio refers to the location information of the pending target The intersection with the location information of the estimated target, divided by the union of the location information of the target to be determined and the location information of the estimated target.
  • IoU Intersection over Union
  • the circumscribed rectangle of the target to be determined indicates the location information of the target to be determined
  • the circumscribed rectangle of the estimated target indicates the location information of the estimated target.
  • the intersection ratio of the target to be determined and the estimated target is the ratio between the circumscribed rectangle of the target to be determined and the estimated target
  • the area where the circumscribed rectangle overlaps is divided by the area where the circumscribed rectangle of the target to be determined and the circumscribed rectangle of the estimated target merge into one.
  • intersection ratio the greater the overlap between the pending target and the estimated target, that is, the higher the similarity between the pending target and the estimated target, which can be based on the intersection of multiple pending targets and the estimated target.
  • At least one candidate target is determined among the multiple pending targets, for example, the pending target with the largest intersection ratio is selected as the candidate target, or the pending target with the intersection ratio greater than the preset ratio is selected as the candidate target.
  • Fig. 11 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S32 extracting the feature of the pixel at the preset position in the target to be determined, and extracting the feature of the pixel at the corresponding position in the estimated target;
  • Step S33 According to the similarity between the feature of the pixel at the preset position and the feature of the pixel at the corresponding position, at least one candidate target is determined from the plurality of pending targets.
  • the preset position may be one or more positions, including at least the position of one pixel in the pending area and at most including the position of each pixel in the pending area.
  • the pixel at the preset position is the pixel at the center position of the target to be determined
  • the corresponding position in the estimated target refers to the pixel at the center position of the preset target.
  • the preset position is the pixel in the first row of the target to be determined.
  • the corresponding position in the target refers to the pixel in the first row of the preset target.
  • both the estimated target and the undetermined target are the results of target estimation, the higher the similarity between the feature of the pixel at the preset position in the undetermined target and the feature of the pixel at the corresponding position in the estimated target, the more the target and the estimated target are similar. It may be the same result estimated for the target. Therefore, according to the similarity between the feature of the pixel at the preset position and the feature of the pixel at the corresponding position, at least one candidate target can be determined among multiple pending targets, such as selecting similarity The highest estimated target is selected as an alternative target, or an estimated target with a similarity higher than a preset threshold is selected as an alternative target.
  • Fig. 12 is a schematic flowchart showing yet another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S34 Determine, among the multiple pending targets, the pending target with the greatest similarity to the estimated target as the candidate target.
  • the target with the greatest similarity to the estimated target can be determined as the candidate target. According to this, only one candidate target needs to be determined, which can effectively reduce the subsequent passing of preset tracking
  • the model determines the calculation amount of the actual target in at least one candidate target.
  • the actual target is determined in at least one candidate target through the preset tracking model, and only one candidate target needs to be determined whether it is actual Target, if it is determined that the candidate target is not an actual target, prompt information may be generated, and if it is determined that the candidate target is an actual target, tracking may be performed based on information such as the location of the actual target.
  • Fig. 13 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S35 sort the similarities between the plurality of pending targets and the estimated targets
  • Step S36 Determine the candidate target among the multiple pending targets according to the preset order of the similarity.
  • the similarities between the multiple pending targets and the estimated targets can be sorted, which can be sorted from large to small, or from small to small. Big sorting, taking the sorting from big to small as an example, the undetermined target corresponding to the similarity ranked before the preset order can be determined as the candidate target.
  • the preset order can be directly expressed in order, for example, the undetermined target corresponding to the similarity ranked before the 10th similarity is determined as the candidate target; the preset order can be expressed in proportion, for example, the top percent The undetermined target corresponding to the similarity of ten is determined as the candidate target.
  • Fig. 14 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure.
  • the determining at least one candidate target among the plurality of pending targets according to the similarity between the plurality of pending targets and the estimated target includes:
  • Step S37 Determine the undetermined target corresponding to the similarity greater than the preset value as the candidate target.
  • each similarity can be compared with a preset value to determine a similarity greater than the preset value, which will then be greater than the preset value.
  • the undetermined target corresponding to the similarity of the value is determined as the candidate target. Among them, if there is no similarity greater than the preset value, a prompt message can be generated.
  • Fig. 15 is a schematic flowchart showing another target tracking method according to an embodiment of the present disclosure. As shown in Figure 15, the method further includes:
  • Step S7 taking the feature of the target in the first image frame as input, and taking the center of the actual target as the output constituent sample, and updating the second training sample set corresponding to the preset tracking model;
  • Step S8 according to the updated second training sample set, update the preset tracking model through machine learning.
  • the preset tracking model (including but not limited to a neural network, such as a convolutional neural network) may be pre-trained according to the second training sample set, the second training sample set contains multiple samples, and the input of the sample It is the feature of the target area in the previous frame (such as the first image frame), such as the feature of the circumscribed rectangle of the target, the feature of the target area of interest, the feature of a certain frame of image where the target is located, etc.
  • the output of the sample is the current The coordinates of the target center of the frame (for example, the second image frame).
  • the characteristics of the tracking area may be used as input, and the center of the actual target may be used as the output to constitute the sample b, and the second training sample set corresponding to the preset tracking model may be updated.
  • the sample b can be added to the second training sample set, so that the updated second training sample set adds one sample b.
  • the preset tracking model is updated through machine learning, so that the updated preset tracking model is more in line with the target's motion state in the last two frames, so that the updated preset can be used later
  • the tracking model can estimate the estimated target more accurately.
  • the embodiment of the present disclosure also provides a target tracking device, including a processor, and the processor is configured to execute the following steps:
  • the actual target is determined in at least one of the candidate targets through a preset tracking model.
  • the processor is configured to execute the following steps:
  • multiple undetermined targets are determined in the second image frame through a preset Gaussian function, wherein the expected value of the preset Gaussian function is equal to the coordinates of the center of the target in the first image frame.
  • the processor is configured to execute the following steps:
  • a tracking area is determined in the second image frame, wherein the tracking area at least partially overlaps with the target in the first image frame, and the size of the tracking area is the same as that of the target in the first image frame. N times the size, n>1;
  • the center of the tracking area coincides with the center of the target in the first image frame.
  • n 3.
  • the processor is further configured to execute the following steps:
  • the preset correlation filtering model is updated through machine learning.
  • the processor is configured to execute the following steps:
  • the processor is configured to execute the following steps:
  • At least one candidate target is determined among the plurality of pending targets according to the intersection ratio of the plurality of pending targets and the estimated targets.
  • the processor is configured to execute the following steps:
  • At least one candidate target is determined from the plurality of pending targets.
  • the processor is configured to execute the following steps:
  • the candidate target is determined to be the candidate target with the greatest similarity to the estimated target among the plurality of pending targets.
  • the processor is configured to execute the following steps:
  • the candidate target is determined among the plurality of pending targets according to the preset order of the similarity.
  • the processor is configured to execute the following steps:
  • the undetermined target corresponding to the similarity greater than the preset value is determined as the candidate target.
  • the processor is further configured to execute the following steps:
  • the preset tracking model is updated through machine learning.
  • An embodiment of the present disclosure also provides an unmanned aerial vehicle, including the target tracking device described in any of the foregoing embodiments.
  • the systems, devices, modules, or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or implemented by products with certain functions.
  • the functions are divided into various units and described separately.
  • the functions of each unit can be implemented in the same one or more software and/or hardware.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.
  • the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本公开提出一种目标跟踪方法，包括：基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；通过预设跟踪模型在至少一个所述备选目标中确定实际目标。根据本公开的实施例，可以缩短确定实际目标的时间，加快跟踪速度，提高跟踪的准确性。

Description

目标跟踪方法、目标跟踪装置和无人机 技术领域
本发明涉及图像技术领域，尤其涉及目标跟踪方法、目标跟踪装置和无人机。
背景技术
在相关技术中，主要采用高斯函数估计目标的运动状态，以实现对目标的跟踪，其中主要针对目标前一帧图像进行高斯采样，产生目标在当前帧大量的可能的位置。
而由于高斯函数一般是随机产生的，并不能保证准确地估计各种各样的运动方式下目标的实际位置，所以需要产生大量的可能的位置，以便可能的位置大概率将目标的实际位置包含进来。
但是，产生可能的位置过多，后续从可能的位置中确定目标的实际位置，计算量也随之增大，会降低确定目标实际位置的效率。而且，产生可能的位置过多，还会大概率将目标附近的与目标相似的物体的运动状态包含进来，容易在后续判断过程中将目标附近的与目标相似的物体确定为目标，导致跟踪出错。
发明内容
本发明提供目标跟踪方法、目标跟踪装置和无人机，以解决相关技术中判断目标实际位置时，计算量大和容易跟踪出错的问题。
根据本公开实施例的第一方面，提出一种目标跟踪方法，包括：
基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；
通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
根据本公开实施例的第二方面，提出一种目标跟踪装置，包括处理器，所述处理器用于执行如下步骤：
基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；
通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
根据本公开实施例的第三方面，提出无人机，包括上述任一实施例所述的装置。
根据本公开的实施例，由于预估目标是通过预设相关滤波模型得到的，是相对准确的结果，所以在确定预估目标和多个待定目标之后，可以确定每个待定目标与预设目标的相似度，然后根据相似度在多个待定目标中确定至少一个备选目标。
据此，实现了基于预估目标对多个待定目标进行筛选，从而得到相对准确的，且数量较少的备选目标，进而再通过预设跟踪模型在备选目标中确定实际目标。
相对直接通过预设跟踪模型在大量的待定目标中确定实际目标，可以极大地减少预设跟踪模型处理数据的数据量，从而缩短确定实际目标的时间，加快跟踪速度，并且由于预估目标是通过预设相关滤波模型得到的相对准确的结果，所以针对数量较少，且相对准确地备选目标确定实际目标，可以降低将实际目标周围相似目标确定为实际目标的可能性，从而提高跟踪的准确性。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是根据本公开的实施例示出的一种目标跟踪方法的示意流程图。
图2是根据本公开的实施例示出的一种待定目标的示意图。
图3是根据本公开的实施例示出的一种预估目标的示意图。
图4是根据本公开的实施例示出的一种备选目标的示意图。
图5A和图5B是根据本公开的实施例示出的目标跟踪方法跟踪目标的示意图。
图6是根据本公开的实施例示出的另一种目标跟踪方法的示意流程图。
图7是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图8是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图9是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图10是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图11是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图12是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图13是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图14是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
图15是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做 出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。另外，在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
图1是根据本公开的实施例示出的一种目标跟踪方法的示意流程图。本公开的实施例所示目标跟踪方法可以适用于具有图像采集功能的设备，可以独立的图像采集设备，例如照相机，摄像机等，也可以是装配有图像采集设备的设备，例如移动终端(手机、平板电脑等)，无人驾驶设备(无人机、无人车等)。
如图1所示，所述目标跟踪方法可以包括以下步骤：
步骤S1，基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；所述第一图像帧与所述第二图像帧在码流中具有时序相关性；
在一个实施例中，所述目标跟踪方法所适用的设备，可以连续采集多帧图像，基于多帧图像中的第一图像帧中的目标，可以在第二图像帧中确定多个待定目标。
其中，可以根据第一图像帧中目标的运动状态，例如速度的大小，速度的方向，目标在第一图像帧中的位置等，在第二图像帧中预测目标可能出现的位置作为待定目标的中心。
需要说明的是，第一图像帧和第二图像帧，可以是所述多帧图像中连续的两帧图像，也可以是多帧图像中非连续的两帧图像，具体可以根据需要设置。
第一图像帧与第二图像帧在码流中具有时序相关性，可以是指第二图像帧在时序上位于第一图像帧之后，在这种情况下，第一图像帧先于第二图像帧被获取到，也就可以先确定第一图像帧中目标的运动状，进而可以根据第一图像帧中目标的运动状态，在后续获取到或即将获取到的第二图像帧中预测目标可能出现的位置作为待定目标的中心。
可选地，第二图像帧可以是与第一图像帧相邻的图像帧，例如在连续采集的n帧图像中，第一图像帧为第i帧图像，第二图像帧为第i+1帧图像。
可选地，第二图像帧可以是与第一图像帧不相邻的图像帧，例如在连续 采集的n帧图像中，第一图像帧为第i帧图像，第二图像帧为第i+k帧图像，k大于1。
图2是根据本公开的实施例示出的一种待定目标的示意图。
如图2所示，可以通过矩形框表示待定目标，矩形框的中心与待定目标的中心重合，矩形框的尺寸可以与待定目标的外接矩形，或者感兴趣区域(Region of Interest，简称ROI)相同。
步骤S2，通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
在一个实施例中，通过预设相关滤波模型，估计第一图像帧中的目标在第二图像帧中的预估目标，具体可以先在第二图像帧中确定一块区域，该区域与第一图像帧中的目标至少部分重合，优选地，该区域的中心与第一图像帧中目标的中心重合，该区域的大小可以根据需要设置，然后提取改区域的特征，通过预设相关滤波模型在提取的特征上进行卷积，再通过快速傅氏变换(Fast Fourier Transformation，简称FFT)将卷积的结果转换到频域，进而确定频域上响应最大的点在第二图像帧中对应的坐标，最后以确定的坐标为中心，基于第一图像帧中的目标的尺寸，在第二图像帧中确定预估目标。
由于通过快速傅氏变换将卷积的结果转换到频域，可以在频域中利用特征的点乘来代替相对复杂的卷积过程，从而减少在第二图像帧中确定预估目标的耗时。
另外，由于预设相关滤波模型是预先训练得到的，具体是根据对目标和目标周围的位置信息进行学习和判断得到的，所以在估计第一图像帧中的目标在第二图像帧中的预估目标的过程中，使用预设相关滤波模型可以相对准确地确定预估目标。
所以通过预设相关滤波模型，可以快速且相对准确地估计第一图像帧中的目标在第二图像帧中的预估目标。
图3是根据本公开的实施例示出的一种预估目标的示意图。
如图3所示，通过预设相关滤波模型，估计第一图像帧中的目标在第二 图像帧中的预估目标，可以得到一个预估目标。
需要说明的是，步骤S1和步骤S2的执行顺序不分先后，例如可以如图1所示，在步骤S2之前执行步骤S1，也可以根据需要调整执行顺序，例如在步骤S1之前执行步骤S2，或者同时执行步骤S1和步骤S2。
步骤S3，根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
步骤S4，通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
在一个实施例中，由于预估目标是通过预设相关滤波模型得到的，是相对准确的结果，所以在确定预估目标和多个待定目标之后，可以确定每个待定目标与预设目标的相似度，然后根据相似度在多个待定目标中确定至少一个备选目标，例如选择相似度最大的待定目标作为备选目标，或者选择相似度大于预设值的待定目标为备选目标。
图4是根据本公开的实施例示出的一种备选目标的示意图。
如图4所示，可以选择相似度大于预设值的待定目标为备选目标，据此可以在图2所示的待定目标中确定出多个备选目标，由于确定出的备选目标与图3中的预估目标相似度较高，所以在图4中多个备选目标的位置高度集中。
据此，实现了基于预估目标对多个待定目标进行筛选，从而得到相对准确的，且数量较少的备选目标，进而再通过预设跟踪模型在备选目标中确定实际目标。
相对直接通过预设跟踪模型在大量的待定目标中确定实际目标，可以极大地减少预设跟踪模型处理数据的数据量，从而缩短确定实际目标的时间，加快跟踪速度，并且由于预估目标是通过预设相关滤波模型得到的相对准确的结果，所以针对数量较少，且相对准确地备选目标确定实际目标，可以降低将实际目标周围相似目标确定为实际目标的可能性，从而提高跟踪的准确性。
图5A和图5B是根据本公开的实施例示出的目标跟踪方法跟踪目标的示 意图。
如图5A和图5B所示，在多帧(例如三帧)图像中可以分别确定实际目标。其中，如图5A所示，本实施例所述的目标跟踪方法可以适用于跟踪人体，如图5C所示，本实施例所述的目标跟踪方法可以适用于跟踪车辆等物体。
本实施例所述的方法并不限于上述实施例跟踪人体，车辆，还可以根据人脸等其他类型的目标，具体可以根据需要配置。
将本实施例所述的方法应用于具体的跟踪过程中，根据实验结果，相对于相关技术，每秒处理视频的帧数(FPS)可以提高20％，也即缩短了确定实际目标的时间，加快跟踪速度，并且确定的实际目标与人为标注目标的相似度可以提高2％，也即提高了跟踪的准确性。
图6是根据本公开的实施例示出的另一种目标跟踪方法的示意流程图。如图6所示，所述基于第一图像帧中的目标，在第二图像帧中确定多个待定目标包括：
步骤S11，基于第一图像帧中的目标，通过预设高斯函数在第二图像帧中确定多个待定目标，其中，所述预设高斯函数的期望值等于所述第一图像帧中目标的中心的坐标。
在一个实施例中，基于第一图像帧中的目标，在第二图像帧中确定多个待定目标，可以通过预设高斯函数得到，预设高斯函数的期望值等于第一图像帧中目标的中心的坐标。具体地，根据第一图像帧中目标的运动状态，例如速度的大小，速度的方向，目标在第一图像帧中的位置等，通过预设高斯函数在第二图像帧中产生大量目标可能的位置信息。
可以将产生的位置信息作为待定目标的中心，然后通过矩形框表示待定目标，矩形框的中心与待定目标的中心重合，矩形框的尺寸可以与待定目标的外接矩形(可以根据第一图像帧中目标的外接矩形确定)，或者感兴趣区域(可以根据第一图像帧中目标的感兴趣区域确定)相同。
图7是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图7所示，所述通过预设相关滤波模型，估计所述第一图像帧中的目标在 所述第二图像帧中的预估目标包括：
步骤S21，在所述第二图像帧中确定跟踪区域，其中，所述跟踪区域与所述第一图像帧中的目标至少部分重合，且所述跟踪区域的尺寸为所述第一图像帧中的目标的尺寸的n倍，n＞1；
步骤S22，通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标。
在一个实施例中，可以在第二图像帧中确定尺寸大于第一图像帧中目标的尺寸的跟踪区域，进而通过预设相关滤波模型，在跟踪区域中估计第一图像帧中的目标在第二图像帧中的预估目标。
由于跟踪区域的尺寸为第一图像帧中的目标的尺寸的n倍，且n＞1，也即跟踪区域的尺寸大于第一图像帧中目标的尺寸，并且跟踪区域与第一图像帧中的目标至少部分重合，所以相对于在与第一图像帧中目标的尺寸相同的区域中估计预估目标，由于跟踪区域尺寸较大，更大概率能够将目标运动后的预估目标包含在内，可以提高估计预估目标的准确率。
而且虽然跟踪区域的尺寸大于第一图像帧中目标的尺寸，在更大尺寸的区域内估计预估目标需要更多的计算量，但是由于本实施例通过预设相关滤波模型，在跟踪区域中估计第一图像帧中的目标在第二图像帧中的预估目标，其中通过快速傅氏变换将卷积的结果转换到频域，可以在频域中利用特征的点乘来代替相对复杂的卷积过程，所以即使在较大的区域内估计预估目标，由于计算过程简单，并不会产生过多额外耗时，仍能保证较高的计算效率。
可选的，所述跟踪区域的中心与所述第一图像帧中的目标的中心重合。
在一个实施例中，由于目标从第一图像帧中的位置运动到第二图像帧中的位置，是从第一图像帧中目标的中心开始运动的，所以第二图像帧的实际目标，更有可能出现在第一图像帧中目标的中心附近，因此确定的跟踪区域，其中心可以与第一图像帧中的目标的中心重合，从而可以在第一图像帧中目标的中心附近估计预估目标，相对于在远离第一图像帧中目标的中心的情况下估计预估目标，有利于保证估计到的预估目标与第二图像帧目标的实际位 置相符。
可选地，n＝3。
在一个实施例中，跟踪区域的尺寸与第一图像帧中的目标的尺寸的关系可以根据需要进行设置，本实施例在确保跟踪区域的尺寸比第一图像帧中的目标的尺寸大的基础上，设置n＝3，也即跟踪区域的尺寸为第一图像帧中的目标的尺寸的3倍，使得跟踪区域不会过大。
因为在越大的跟踪区域中估计预估目标，所需的计算量越大，而目标的运动速度一般不会过大，也即在第二图像帧中目标的位置与在第一图像帧中目标的位置不会过远，所以设置过大的跟踪区域几乎不会提高在第二图像帧中估计预估目标的准确率，而会较大程度上增加计算量。
所以设置n＝3，确保跟踪区域的尺寸比第一图像帧中的目标的尺寸大，又使得跟踪区域不会过大，既可以提高估计预估目标的准确率，又不会过多地增加计算量。
在一个实施例中，可以根据目标在第一图像帧中的速度大小来设置n，速度越大，说明目标在第二图像帧中的位置与在第一图像帧中的位置距离可能月远，所以可以设置n越大，以便确保跟踪区域能够包含目标在第二图像帧中的位置，从而确保估计预估目标具有较高的准确率。
需要说明的是，n可以根据第二图像帧与第一图像帧相距的时间来确定，第二图像帧与第一图像帧相距的时间越大，那么一般情况下，目标从第一图像帧对应时刻到第二图像帧对应时刻运动的距离就可能越大，目标在第一图像帧中的中心，到目标在第二图像帧中的中心距离可能就越远，那么可以将n设置的越大，以便保证跟踪区域能够更大概率包含目标在第二图像帧中的位置，从而确保估计预估目标具有较高的准确率。
图8是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图8所示，所述方法还包括：
步骤S5，以所述跟踪区域的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设相关滤波模型对应的第一训练样本集；
步骤S6，根据更新后的第一训练样本集，通过机器学习更新所述预设相关滤波模型。
在一个实施例中，预设相关滤波模型可以是根据第一训练样本集预先训练得到的，第一训练样本集中包含多个样本，样本的输入是前一帧(例如第一图像帧)中目标所在区域的特征，例如目标的外接矩形的特征，目标的感兴趣区域的特征，目标所在的某一帧图像的特征等，样本的输出是当前帧(例如第二图像帧)目标中心的坐标。
根据本实施例，在确定实际目标后，可以以跟踪区域的特征作为输入，以实际目标的中心作为输出构成样本a，更新预设相关滤波模型对应的第一训练样本集。例如可以将样本a添加至第一训练样本集中，使得更新后的第一训练样本集增加一个样本a。然后再根据更新后的第一训练样本集，通过机器学习更新预设相关滤波模型，以使更新后的预设相关滤波模型更符合目标在最近两帧中的运动状态，以便后续使用更新后的预设相关滤波模型可以更加准确地估计预估目标。
图9是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图9所示，所述通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标包括：
步骤S221，提取所述跟踪区域的特征；
步骤S222，通过预设相关滤波模型在所述特征上进行卷积；
步骤S223，通过快速傅氏变换将卷积的结果转换到频域；
步骤S224，确定频域上响应最大的点在所述跟踪区域对应的坐标；
步骤S225，以所述坐标为中心，基于所述第一图像帧中的目标的尺寸，在所述第二图像帧中确定预估目标。
在一个实施例中，通过预设相关滤波模型，在跟踪区域中估计第一图像帧中的目标在第二图像帧中的预估目标，具体可以先提取跟踪区域的特征，然后通过预设相关滤波模型在提取的特征上进行卷积，再通过快速傅氏变换将卷积的结果转换到频域，进而确定频域上响应最大的点在第二图像帧中对 应的坐标，最后以确定的坐标为中心，基于第一图像帧中的目标的尺寸，在第二图像帧中确定预估目标。
由于通过快速傅氏变换将卷积的结果转换到频域，可以在频域中利用特征的点乘来代替相对复杂的卷积过程，从而减少在第二图像帧中确定预估目标的耗时。
另外，由于预设相关滤波模型是预先训练得到的，具体是根据对目标和目标周围的位置信息进行学习和判断得到的，所以在估计第一图像帧中的目标在第二图像帧中的预估目标的过程中，使用预设相关滤波模型可以相对准确地确定预估目标。
所以通过预设相关滤波模型，可以快速且相对准确地估计第一图像帧中的目标在第二图像帧中的预估目标。
图10是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图10所示，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
步骤S31，根据多个所述待定目标与所述预估目标的交并比，在多个所述待定目标中确定至少一个备选目标。
在一个实施例中，可以根据待定目标与预估目标的交并比(Intersection over Union，简称IoU)，确定待定目标与预估目标的相似度，其中，交并比是指待定目标的位置信息与预估目标的位置信息的交集，除以待定目标的位置信息与预估目标的位置信息的并集。
例如通过待定目标的外接矩形表示待定目标的位置信息，通过预估目标的外接矩形表示预估目标的位置信息，待定目标与预估目标的交并比就是待定目标的外接矩形与预估目标的外接矩形重合的区域，除以待定目标的外接矩形与预估目标的外接矩形合并在其一的区域。
交并比越大，说明待定目标与预估目标重叠的部分越大，也即待定目标与预估目标相似度越高，从而可以根据根据多个待定目标与预估目标的交并比，在多个待定目标中确定至少一个备选目标，例如选择交并比最大的待定 目标作为备选目标，或者选择交并比大于预设比值的待定目标作为备选目标。
图11是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图11所示，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
步骤S32，提取所述待定目标中预设位置的像素的特征，以及提取所述预估目标中对应位置的像素的特征；
步骤S33，根据所述预设位置的像素的特征与所述对应位置的像素的特征的相似度，在多个所述待定目标中确定至少一个备选目标。
在一个实施例中，预设位置可以是一个或多个位置，最少包括待定区域中一个像素的位置，最多包括待定区域中每个像素的位置。
例如预设位置的像素为待定目标的中心位置的像素，那么预估目标中对应位置是指预设目标的中心位置的像素，例如预设位置为待定目标的第一行的像素，那么预估目标中对应位置是指预设目标的第一行的像素。
由于预估目标和待定目标都是针对目标估计的结果，那么待定目标中预设位置的像素的特征与预估目标中对应位置的像素的特征的相似度越高，待定目标和预估目标越有可能属于对于目标预估的相同结果，因此，可以根据预设位置的像素的特征与对应位置的像素的特征的相似度，在多个待定目标中确定至少一个备选目标，例如选择相似度最高的预估目标作为备选目标，或者选择相似度高于预设阈值的预估目标作为备选目标。
图12是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图12所示，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
步骤S34，在多个所述待定目标中确定与所述预估目标的相似度最大的待定目标为所述备选目标。
在一个实施例中，可以在多个待定目标中确定与预估目标的相似度最大的待定目标为备选目标，据此，只需确定一个备选目标，可以有效地降低后续通过预设跟踪模型在至少一个备选目标中确定实际目标的计算量。
需要说明的是，在这种情况下，由于只确定一个备选目标，那么通过预设跟踪模型在至少一个备选目标中确定实际目标，也就只需针对一个备选目标确定其是否为实际目标，若确定该备选目标不是实际目标，可以生成提示信息，若确定该备选目标是实际目标，则可以基于实际目标的位置等信息进行跟踪。
图13是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图13所示，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
步骤S35，对多个所述待定目标与所述预估目标的相似度进行排序；
步骤S36，根据所述相似度的预设次序在多个所述待定目标确定所述备选目标。
在一个实施例中，在计算出多个待定目标与预估目标的相似度后，可以对多个待定目标与预估目标的相似度进行排序，可以由大到小排序，也可以有小到大排序，以由大到小排序为例，可以将排在预设次序之前的相似度对应的待定目标确定为备选目标。
其中，预设次序可以直接以次序表示，例如将排在第10个相似度之前的相似度对应的待定目标确定为备选目标；预设次序可以以比例表示，例如将排在前百分之十的相似度对应的待定目标确定为备选目标。
图14是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。如图14所示，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
步骤S37，将大于预设值的相似度对应的待定目标确定为所述备选目标。
在一个实施例中，在计算出多个待定目标与预估目标的相似度后，可以将每个相似度与预设值进行比较，确定出大于预设值的相似度，进而将大于预设值的相似度对应的待定目标确定为备选目标。其中，若不存在大于预设值的相似度，可以生成提示信息。
图15是根据本公开的实施例示出的又一种目标跟踪方法的示意流程图。 如图15所示，所述方法还包括：
步骤S7，以所述第一图像帧中的目标的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设跟踪模型对应的第二训练样本集；
步骤S8，根据更新后的第二训练样本集，通过机器学习更新所述预设跟踪模型。
在一个实施例中，预设跟踪模型(包括但不限于神经网络，例如卷积神经网络)可以是根据第二训练样本集预先训练得到的，第二训练样本集中包含多个样本，样本的输入是前一帧(例如第一图像帧)中目标所在区域的特征，例如目标的外接矩形的特征，目标的感兴趣区域的特征，目标所在的某一帧图像的特征等，样本的输出是当前帧(例如第二图像帧)目标中心的坐标。
根据本实施例，在确定实际目标后，可以以跟踪区域的特征作为输入，以实际目标的中心作为输出构成样本b，更新预设跟踪模型对应的第二训练样本集。例如可以将样本b添加至第二训练样本集中，使得更新后的第二训练样本集增加一个样本b。然后再根据更新后的第二训练样本集，通过机器学习更新预设跟踪模型，以使更新后的预设跟踪模型更符合目标在最近两帧中的运动状态，以便后续使用更新后的预设跟踪模型可以更加准确地估计预估目标。
本公开的实施例还提出一种目标跟踪装置，包括处理器，所述处理器用于执行如下步骤：
基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；
通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
在一个实施例中，所述处理器用于执行如下步骤：
基于第一图像帧中的目标，通过预设高斯函数在第二图像帧中确定多个待定目标，其中，所述预设高斯函数的期望值等于所述第一图像帧中目标的中心的坐标。
在一个实施例中，所述处理器用于执行如下步骤：
在所述第二图像帧中确定跟踪区域，其中，所述跟踪区域与所述第一图像帧中的目标至少部分重合，且所述跟踪区域的尺寸为所述第一图像帧中的目标的尺寸的n倍，n＞1；
通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标。
在一个实施例中，所述跟踪区域的中心与所述第一图像帧中的目标的中心重合。
在一个实施例中，n＝3。
在一个实施例中，所述处理器还用于执行如下步骤：
以所述跟踪区域的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设相关滤波模型对应的第一训练样本集；
根据更新后的第一训练样本集，通过机器学习更新所述预设相关滤波模型。
在一个实施例中，所述处理器用于执行如下步骤：
提取所述跟踪区域的特征；
通过预设相关滤波模型在所述特征上进行卷积；
通过快速傅氏变换将卷积的结果转换到频域；
确定频域上响应最大的点在所述跟踪区域对应的坐标；
以所述坐标为中心，基于所述第一图像帧中的目标的尺寸，在所述第二图像帧中确定预估目标。
在一个实施例中，所述处理器用于执行如下步骤：
根据多个所述待定目标与所述预估目标的交并比，在多个所述待定目标中确定至少一个备选目标。
在一个实施例中，所述处理器用于执行如下步骤：
提取所述待定目标中预设位置的像素的特征，以及提取所述预估目标中对应位置的像素的特征；
根据所述预设位置的像素的特征与所述对应位置的像素的特征的相似度，在多个所述待定目标中确定至少一个备选目标。
在一个实施例中，所述处理器用于执行如下步骤：
在多个所述待定目标中确定与所述预估目标的相似度最大的待定目标为所述备选目标。
在一个实施例中，所述处理器用于执行如下步骤：
对多个所述待定目标与所述预估目标的相似度进行排序；
根据所述相似度的预设次序在多个所述待定目标确定所述备选目标。
在一个实施例中，所述处理器用于执行如下步骤：
将大于预设值的相似度对应的待定目标确定为所述备选目标。
在一个实施例中，所述处理器还用于执行如下步骤：
以所述第一图像帧中的目标的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设跟踪模型对应的第二训练样本集；
根据更新后的第二训练样本集，通过机器学习更新所述预设跟踪模型。
本公开的实施例还提出一种无人机，包括上述任一实施例所述的目标跟踪装置。
上述实施例阐明的系统、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本申请时可以把各单元的功能在同一个或多个软件和/或硬件中实现。本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等) 上实施的计算机程序产品的形式。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于系统实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (27)

  1. 一种目标跟踪方法，其特征在于，包括：
    基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；所述第一图像帧与所述第二图像帧在码流中具有时序相关性；
    通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
    根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
    通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
  2. 根据权利要求1所述的方法，其特征在于，所述基于第一图像帧中的目标，在第二图像帧中确定多个待定目标包括：
    基于第一图像帧中的目标，通过预设高斯函数在第二图像帧中确定多个待定目标，其中，所述预设高斯函数的期望值等于所述第一图像帧中目标的中心的坐标。
  3. 根据权利要求1所述的方法，其特征在于，所述通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标包括：
    在所述第二图像帧中确定跟踪区域，其中，所述跟踪区域与所述第一图像帧中的目标至少部分重合，且所述跟踪区域的尺寸为所述第一图像帧中的目标的尺寸的n倍，n＞1；
    通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标。
  4. 根据权利要求3所述的方法，其特征在于，所述跟踪区域的中心与所述第一图像帧中的目标的中心重合。
  5. 根据权利要求3所述的方法，其特征在于，n＝3。
  6. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    以所述跟踪区域的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设相关滤波模型对应的第一训练样本集；
    根据更新后的第一训练样本集，通过机器学习更新所述预设相关滤波模型。
  7. 根据权利要求3所述的方法，其特征在于，所述通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标包括：
    提取所述跟踪区域的特征；
    通过预设相关滤波模型在所述特征上进行卷积；
    通过快速傅氏变换将卷积的结果转换到频域；
    确定频域上响应最大的点在所述跟踪区域对应的坐标；
    以所述坐标为中心，基于所述第一图像帧中的目标的尺寸，在所述第二图像帧中确定预估目标。
  8. 根据权利要求1所述的方法，其特征在于，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
    根据多个所述待定目标与所述预估目标的交并比，在多个所述待定目标中确定至少一个备选目标。
  9. 根据权利要求1所述的方法，其特征在于，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
    提取所述待定目标中预设位置的像素的特征，以及提取所述预估目标中对应位置的像素的特征；
    根据所述预设位置的像素的特征与所述对应位置的像素的特征的相似度，在多个所述待定目标中确定至少一个备选目标。
  10. 根据权利要求1至9中任一项所述的方法，其特征在于，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
    在多个所述待定目标中确定与所述预估目标的相似度最大的待定目标为 所述备选目标。
  11. 根据权利要求1至9中任一项所述的方法，其特征在于，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
    对多个所述待定目标与所述预估目标的相似度进行排序；
    根据所述相似度的预设次序在多个所述待定目标确定所述备选目标。
  12. 根据权利要求1至9中任一项所述的方法，其特征在于，所述根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标包括：
    将大于预设值的相似度对应的待定目标确定为所述备选目标。
  13. 根据权利要求1至9中任一项所述的方法，其特征在于，所述方法还包括：
    以所述第一图像帧中的目标的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设跟踪模型对应的第二训练样本集；
    根据更新后的第二训练样本集，通过机器学习更新所述预设跟踪模型。
  14. 一种目标跟踪装置，其特征在于，包括处理器，所述处理器用于执行如下步骤：
    基于第一图像帧中的目标，在第二图像帧中确定多个待定目标；
    通过预设相关滤波模型，估计所述第一图像帧中的目标在所述第二图像帧中的预估目标；
    根据多个所述待定目标与所述预估目标的相似度，在多个所述待定目标中确定至少一个备选目标；
    通过预设跟踪模型在至少一个所述备选目标中确定实际目标。
  15. 根据权利要求14所述的装置，其特征在于，所述处理器用于执行如下步骤：
    基于第一图像帧中的目标，通过预设高斯函数在第二图像帧中确定多个待定目标，其中，所述预设高斯函数的期望值等于所述第一图像帧中目标的 中心的坐标。
  16. 根据权利要求14所述的装置，其特征在于，所述处理器用于执行如下步骤：
    在所述第二图像帧中确定跟踪区域，其中，所述跟踪区域与所述第一图像帧中的目标至少部分重合，且所述跟踪区域的尺寸为所述第一图像帧中的目标的尺寸的n倍，n＞1；
    通过预设相关滤波模型，在所述跟踪区域中估计所述第一图像帧中的目标在所述第二图像帧中的预估目标。
  17. 根据权利要求16所述的装置，其特征在于，所述跟踪区域的中心与所述第一图像帧中的目标的中心重合。
  18. 根据权利要求16所述的装置，其特征在于，n＝3。
  19. 根据权利要求16所述的装置，其特征在于，所述处理器还用于执行如下步骤：
    以所述跟踪区域的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设相关滤波模型对应的第一训练样本集；
    根据更新后的第一训练样本集，通过机器学习更新所述预设相关滤波模型。
  20. 根据权利要求16所述的装置，其特征在于，所述处理器用于执行如下步骤：
    提取所述跟踪区域的特征；
    通过预设相关滤波模型在所述特征上进行卷积；
    通过快速傅氏变换将卷积的结果转换到频域；
    确定频域上响应最大的点在所述跟踪区域对应的坐标；
    以所述坐标为中心，基于所述第一图像帧中的目标的尺寸，在所述第二图像帧中确定预估目标。
  21. 根据权利要求14所述的装置，其特征在于，所述处理器用于执行如下步骤：
    根据多个所述待定目标与所述预估目标的交并比，在多个所述待定目标中确定至少一个备选目标。
  22. 根据权利要求14所述的装置，其特征在于，所述处理器用于执行如下步骤：
    提取所述待定目标中预设位置的像素的特征，以及提取所述预估目标中对应位置的像素的特征；
    根据所述预设位置的像素的特征与所述对应位置的像素的特征的相似度，在多个所述待定目标中确定至少一个备选目标。
  23. 根据权利要求14至22中任一项所述的装置，其特征在于，所述处理器用于执行如下步骤：
    在多个所述待定目标中确定与所述预估目标的相似度最大的待定目标为所述备选目标。
  24. 根据权利要求14至22中任一项所述的装置，其特征在于，所述处理器用于执行如下步骤：
    对多个所述待定目标与所述预估目标的相似度进行排序；
    根据所述相似度的预设次序在多个所述待定目标确定所述备选目标。
  25. 根据权利要求14至22中任一项所述的装置，其特征在于，所述处理器用于执行如下步骤：
    将大于预设值的相似度对应的待定目标确定为所述备选目标。
  26. 根据权利要求14至22中任一项所述的装置，其特征在于，所述处理器还用于执行如下步骤：
    以所述第一图像帧中的目标的特征作为输入，以所述实际目标的中心作为输出构成样本，更新所述预设跟踪模型对应的第二训练样本集；
    根据更新后的第二训练样本集，通过机器学习更新所述预设跟踪模型。
  27. 一种无人机，其特征在于，包括权利要求14至26中任一项所述的装置。
PCT/CN2019/089668 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机 Ceased WO2020237674A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/089668 WO2020237674A1 (zh) 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机
CN201980009924.9A CN111684491A (zh) 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/089668 WO2020237674A1 (zh) 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机

Publications (1)

Publication Number Publication Date
WO2020237674A1 true WO2020237674A1 (zh) 2020-12-03

Family

ID=72451448

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/089668 Ceased WO2020237674A1 (zh) 2019-05-31 2019-05-31 目标跟踪方法、目标跟踪装置和无人机

Country Status (2)

Country Link
CN (1) CN111684491A (zh)
WO (1) WO2020237674A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393492A (zh) * 2021-05-27 2021-09-14 浙江大华技术股份有限公司 目标跟踪方法、装置、电子装置和存储介质
CN116430893A (zh) * 2023-03-24 2023-07-14 佛山中科云图智能科技有限公司 一种无人机目标识别与目标跟踪方法、系统及计算机设备

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190635A (zh) * 2018-07-25 2019-01-11 北京飞搜科技有限公司 基于分类cnn的目标追踪方法、装置及电子设备
CN109697727A (zh) * 2018-11-27 2019-04-30 哈尔滨工业大学（深圳） 基于相关滤波和度量学习的目标跟踪方法、系统及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105989367B (zh) * 2015-02-04 2019-06-28 阿里巴巴集团控股有限公司 目标获取方法及设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190635A (zh) * 2018-07-25 2019-01-11 北京飞搜科技有限公司 基于分类cnn的目标追踪方法、装置及电子设备
CN109697727A (zh) * 2018-11-27 2019-04-30 哈尔滨工业大学（深圳） 基于相关滤波和度量学习的目标跟踪方法、系统及存储介质

Also Published As

Publication number Publication date
CN111684491A (zh) 2020-09-18

Similar Documents

Publication Publication Date Title
Kamal et al. Automatic traffic sign detection and recognition using SegU-Net and a modified Tversky loss function with L1-constraint
CN111931764B (zh) 一种目标检测方法、目标检测框架及相关设备
CN108062531B (zh) 一种基于级联回归卷积神经网络的视频目标检测方法
CN109543641B (zh) 一种实时视频的多目标去重方法、终端设备及存储介质
CN113128368B (zh) 一种人物交互关系的检测方法、装置及系统
CN111401293B (zh) 一种基于Head轻量化Mask Scoring R-CNN的手势识别方法
Kim et al. Fast pedestrian detection in surveillance video based on soft target training of shallow random forest
CN104615986B (zh) 利用多检测器对场景变化的视频图像进行行人检测的方法
CN107851318A (zh) 用于对象跟踪的系统和方法
US20130342636A1 (en) Image-Based Real-Time Gesture Recognition
WO2014174932A1 (ja) 画像処理装置、プログラム及び画像処理方法
CN108257158A (zh) 一种基于循环神经网络的目标预测与跟踪方法
CN108846415A (zh) 工业分拣机器人的目标识别装置和方法
CN114399655A (zh) 目标检测方法、系统及存储介质
CN111091101B (zh) 基于一步法的高精度行人检测方法、系统、装置
CN112183649A (zh) 一种用于对金字塔特征图进行预测的算法
CN111815670A (zh) 多视图目标跟踪方法、装置、系统、电子终端、及存储介质
CN110310305B (zh) 一种基于bssd检测与卡尔曼滤波的目标跟踪方法与装置
Yang et al. Lightweight and efficient deep learning models for fruit detection in orchards
CN115661542B (zh) 一种基于特征关系迁移的小样本目标检测方法
Zhou et al. Object detection in low-light conditions based on DBS-YOLOv8
CN116958873A (zh) 行人跟踪方法、装置、电子设备及可读存储介质
CN112734800A (zh) 一种基于联合检测与表征提取的多目标跟踪系统和方法
CN114299128B (zh) 多视角定位检测方法及装置
CN110427915B (zh) 用于输出信息的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19931398

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19931398

Country of ref document: EP

Kind code of ref document: A1