WO2021146952A1 - 跟随方法、可移动平台、装置和存储介质 - Google Patents

跟随方法、可移动平台、装置和存储介质 Download PDF

Info

Publication number
WO2021146952A1
WO2021146952A1 PCT/CN2020/073626 CN2020073626W WO2021146952A1 WO 2021146952 A1 WO2021146952 A1 WO 2021146952A1 CN 2020073626 W CN2020073626 W CN 2020073626W WO 2021146952 A1 WO2021146952 A1 WO 2021146952A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
image frame
current image
following
interest
Prior art date
Application number
PCT/CN2020/073626
Other languages
English (en)
French (fr)
Inventor
聂谷洪
杨龙超
朱高
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/073626 priority Critical patent/WO2021146952A1/zh
Priority to CN202080004152.2A priority patent/CN112585944A/zh
Publication of WO2021146952A1 publication Critical patent/WO2021146952A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects

Definitions

  • the present invention relates to the field of image processing, in particular to a following method, a movable platform, a device and a storage medium.
  • Smart follow is a technology that takes the sensor data collected by the sensor equipped with the smart device as input, and then automatically and continuously locks and follows an object in the sensor field of view after designating it.
  • This kind of follow-up technology is often used in the field of video shooting, and the above-mentioned sensor data is the image captured by the camera.
  • the shooting screen can present a more unique angle, on the other hand, it can also free the user's hands and make shooting easier.
  • the smart device may specifically be a movable platform, such as a drone, an unmanned vehicle, a handheld stabilizer, and so on.
  • the implementation process of target following is usually: the smart device first uses the configured camera to capture an image, and then performs target detection on the captured image to identify the target according to the detection result, thereby completing the target following.
  • devices with intelligent follow-up functions such as drones and handheld gimbals, are limited by factors such as computing power and bandwidth, and there are bottlenecks in the follow-up effect.
  • the invention provides a following method, a movable platform, a device and a storage medium, which are used to reduce the amount of calculation in the following process while ensuring the following effect of the target object.
  • the first aspect of the present invention is to provide a following method, which includes:
  • the historical image frame is an image frame before the current image frame
  • the second aspect of the present invention is to provide a follower device, which includes:
  • Memory used to store computer programs
  • the processor is configured to run a computer program stored in the memory to realize:
  • the historical image frame is an image frame before the current image frame
  • the third aspect of the present invention is to provide a movable platform, the platform at least includes: a body, a power system, a control device, and a camera device;
  • the power system is arranged on the body and used to provide power for the movable platform
  • the camera device is arranged on the body and is used to collect image frames
  • the control device includes a memory and a processor
  • the memory is used to store a computer program
  • the processor is configured to run a computer program stored in the memory to realize:
  • the historical image frame is an image frame before the current image frame
  • the fourth aspect of the present invention is to provide a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used in the first aspect. The following method described.
  • the following method, movable platform, device and storage medium provided by the present invention first acquire a follow frame of a historical image frame, the follow-up result of this historical image frame to the target is successful, and the follow frame of the historical image frame covers at least Part of the target object is then determined according to the following frame to determine the region of interest of the current image frame, where the current image frame is the image frame following the historical image frame. Then, the first similarity between the following frame and the region of interest is calculated, and if the similarity meets the first preset threshold, it is considered that the follow-up result of the current image frame to the target object is successful.
  • the area of interest is smaller than the entire area of the current image frame, and only the area of interest is required for feature matching, and the entire image frame area is not required for feature matching, thereby reducing the amount of calculation and
  • the bandwidth consumption of the transmission to the GPU can also avoid the interference of the target following caused by the interference information contained in the area outside the region of interest in the current image frame.
  • using the region of interest for feature matching is equivalent to zooming in on the region of interest and then performing feature matching, which can improve the accuracy of target following and avoid the problem of track loss due to too small a target.
  • FIG. 1 is a schematic flowchart of a following method provided by an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of another following method provided by an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of adjusting an initial follow frame according to a detection frame according to an embodiment of the present invention
  • FIG. 4 is a schematic flowchart of another follow method provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of the size relationship between the region of interest and the following frame provided by an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a follower device provided by an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of another following device provided by an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention.
  • the above-mentioned movable platform can be unmanned aerial vehicle, unmanned vehicle, unmanned ship, stabilizer and so on. Among them, take the drone as an example to illustrate:
  • UAVs have been used in many fields, such as entertainment, surveillance, security and other fields.
  • these fields there is often a need to follow a target in a sports environment. For example, when a user is moving in an environment, he can hope to record his entire movement process. At this time, the drone needs to perform the operation on the user. Follow to capture the user's movement process.
  • FIG. 1 is a schematic flowchart of a following method provided by an embodiment of the present invention.
  • the main body of execution of the following method is the following device. It can be understood that the following device can be implemented as software or a combination of software and hardware; when the following device executes the following method, it can achieve accurate following of the target.
  • the follower device can be various types of movable platforms. In this embodiment and the following embodiments, drones are used as examples for description. Of course, the execution body can also be other types of movable platforms.
  • the present invention does not Qualify.
  • the method may include:
  • S101 Determine a region of interest of the current image frame according to the following frame of the historical image frame in which the target object is successfully followed.
  • the camera configured by the drone can capture an image frame corresponding to the flying environment of the drone in real time, and for the current moment, the camera configured by the drone can capture an image frame, that is, the current image frame. Due to the limited volume of the target, it is generally not filled in an entire image frame. Therefore, after the current image frame is obtained, part of the image can be selected as the region of interest, and the presence of the region of interest can be determined.
  • the target that is, the following result of the current image frame to the target is determined in the region of interest.
  • the size of the following frame of the historical image frame may be called the first size
  • the size of the region of interest of the current image frame may be called the second size
  • a partial image of the second size may be selected from the current image frame as the region of interest of the current image frame, where the second size is larger than the first size.
  • the ratio of the second size to the first size may be a first preset value, and the first preset value is usually an integer.
  • the first size of the following frame can be doubled to obtain the second size of the region of interest.
  • the area of interest of the current image frame may also have the same area center as the following frame of the historical image frame, that is, the centers of the two overlap.
  • the historical image frame and the follow frame of the historical image frame are involved in the process of selecting the region of interest.
  • the following descriptions can be made for both:
  • an optional method can be an image frame in which any pair of target objects are collected before the current image frame.
  • the follow frame of the historical image frame if the follow-up result of a historical image frame to the target is successful, it indicates that the target is included in the historical image frame, and the target will be marked with a follow frame.
  • the following frame can contain at least part of the target. In an embodiment, all the targets may also be contained in the follower frame.
  • S102 Calculate the first similarity between the following frame and the region of interest.
  • the first similarity between the following frame of the historical image frame and the region of interest of the current image frame is calculated.
  • the image features of the following frame and the region of interest can be extracted first, and then the similarity between the image features can be calculated, using the image The similarity between the features is used as the first similarity between the following frame and the region of interest.
  • feature extraction is performed on the following frame and the region of interest respectively to generate multiple feature vectors respectively. Then calculate the first similarity between the following frame and the region of interest according to the respective feature vectors of the following frame and the region of interest.
  • this historical image frame contains the target object
  • the first similarity meets the first preset threshold, it can be considered that the target object is also contained in the region of interest of the current image frame, and the current image frame is determined to be the target object. Things follow success.
  • that the first similarity degree satisfies the first preset threshold value may be that the first similarity degree is greater than or equal to the first preset threshold value.
  • the following method provided in this embodiment first obtains the following frame of the historical image frame that successfully followed the target, and determines the region of interest of the current image frame according to the following frame, where the current image frame is the image frame following the historical image frame. Then, the first similarity between the following frame and the region of interest is calculated, and if the similarity meets the first preset threshold, it is determined that the current image frame successfully follows the target object.
  • the area of interest is smaller than the entire area of the current image frame, and only the area of interest is required for feature matching, and the entire image frame area is not required for feature matching, thereby reducing the amount of calculation and
  • the bandwidth consumption of the transmission to the GPU can also avoid the interference of the target following caused by the interference information contained in the area outside the region of interest in the current image frame.
  • using the region of interest for feature matching is equivalent to zooming in on the region of interest and then performing feature matching, which can improve the accuracy of target following and avoid the problem of track loss due to too small a target.
  • image frame A and image frame B are respectively collected at the Mth time and the Nth time, where the Nth time is the current time, and both the image frame A and the image frame B follow the target successfully.
  • the target is usually constantly moving. Therefore, the position of the target in the image frame A and the position of the target in the image frame B usually have a certain change.
  • the current image frame interest area and the historical image frame following frame have the same area center, and the sizes of the two are also closely related.
  • the target may not be included.
  • the target In the area of interest of the image frame A, that is, the target is located in the non-interest area of the image frame A.
  • the UAV must not get the correct follow result based on the region of interest that no longer contains the target. This also shows that the method of determining historical image frames from any image frame collected before the current image frame provided in the embodiment shown in FIG. 1 is not very appropriate.
  • the image frame adjacent to the current image frame acquisition moment can be determined as the historical image frame, and this historical image frame should also successfully follow the target object.
  • the historical image frame determined in this way and the current image frame have adjacent acquisition moments. Due to the short time interval, the position of the target in the image frame usually does not change significantly, which can ensure the following performance .
  • the following frame of the historical image frame that has successfully followed is used when determining whether the current frame image successfully follows the target object.
  • the camera configured by the drone continuously captures image frames, when the next image frame appears, the aforementioned current image frame will correspondingly become a historical image frame. At this time, it is necessary to follow the frame of this current image frame. To determine whether the next image frame succeeds in following the target object.
  • FIG. 2 is a schematic flowchart of another following method provided by an embodiment of the present invention. As shown in FIG. 2, after step 103, the following method may further include the following steps:
  • S201 Match the region of interest of the current image frame with the follow frame of the historical image frame to determine the initial follow frame of the current image frame.
  • the region of interest of the current image frame can be determined, and then the region of interest can be matched with the follow frame of the historical image frame, and the matching region can be determined as the initial follow frame of the current image frame.
  • the region of interest of the current image frame is enlarged and then matched with the follow frame of the historical image frame, which can avoid the situation that the target object is too small and causes the tracking to be lost.
  • the region of interest is enlarged and matched with the follow frame of the historical image frame, so that the generated follow frame is more accurate, and the following position of the target object caused by shrinking and expanding the frame is prevented from being inaccurate.
  • the problem of large errors the distance between the movable platform and the target calculated based on the position of the following frame on the image frame is more accurate, which is conducive to flight control operations such as obstacle avoidance.
  • the region of interest can be enlarged to the same dimension as the current image frame and then matched with the following frame.
  • S202 Perform target detection processing on the current image frame according to a preset algorithm to generate at least one detection frame.
  • a preset target detection algorithm it is also possible to perform target detection on the current image frame according to a preset target detection algorithm, so as to detect all objects of the preset category in the current image frame, that is, to detect the location of each object.
  • a preset neural network model may be used to detect the current image frame for target detection. .
  • step 201 and then performing step 202 is just an example. In practical applications, there may be no time sequence between the above two steps, and the two steps may be executed simultaneously or sequentially, which is not limited in the present invention.
  • S203 Adjust the initial follow frame according to the at least one detection frame to generate a follow frame of the current image frame.
  • the two are only obtained in different ways, so they are named differently, but in fact they are the result of labeling the target object.
  • the two frames Can contain the target.
  • the initial follow frame and the detection frame can also be fused to realize the calibration of the initial follow frame using the detection frame, so as to obtain the final follow-up of the current image frame frame.
  • the follow frame of the current image frame obtained here is equivalent to the follow frame of the historical image frame in the embodiment shown in FIG. 1, at this time, you can continue Perform the steps in Figure 1 to determine whether the next frame of image follows the target successfully.
  • the feature matching method can be further used to obtain the initial following frame of the current image frame, on the other hand, You can also use the target detection algorithm to get the detection frame of the current image frame. Then, the detection frame is used to correct the initial follow frame, and the corrected result is determined as the final follow frame of the current image frame. Through the correction processing, it can be ensured that the position and size of the following frame of the current image frame are more accurate.
  • step 203 is an optional implementation method.
  • the implementation can be as follows:
  • S2031 Calculate the second similarity between the at least one detection frame and the initial following frame respectively.
  • S2032 Determine whether the second similarity degree satisfies a second preset threshold, and if it satisfies the second preset threshold, execute step 2033; otherwise, execute step 2034.
  • S2034 Determine that the initial follow frame is the follow frame of the current image frame.
  • the second degree of similarity between at least one detection frame and the initial following frame is calculated first, and each detection frame corresponds to a second degree of similarity. Then, the magnitude relationship between the second similarity degree and the second preset threshold is judged, so as to choose whether to merge the detection frame with the initial follow frame according to the magnitude relationship.
  • the second similarity is greater than or equal to the second preset threshold, it can be considered that the object contained in the detection frame and the object contained in the initial follow frame of the current image frame have a higher degree of similarity, and the initial follow frame can be detected with this target
  • the frames are merged to obtain the final following frame of the current image frame.
  • the second similarity is less than the second preset threshold, it can be considered that the object contained in the detection frame is far from the object contained in the initial follow frame of the current image frame, or even not an object.
  • the fusion process may not be performed, but directly The initial follow frame is determined as the final follow frame of the current image frame.
  • step 2031 the following processing may be performed on at least one detection frame:
  • the detection frame is deleted.
  • the detection frame that obviously contains non-target objects can be eliminated, thereby ensuring the fusion effect of the detection frame and the initial following frame, so as to obtain an accurate following frame of the current image frame.
  • the target corresponds to a smaller area in the image frame captured by the drone, which is prone to mismatching. And lead to a failure to follow. If the distance between the target and the UAV is relatively short, when the target suddenly decelerates, the UAV will easily collide with the target and cause damage.
  • the distance between the target and the drone can be further determined according to the following frame of the current image frame.
  • the distance does not meet the preset distance range, it indicates that the distance between the drone and the target is too large or too small. At this time, adjust the motion state of the drone, specifically, adjust the flying speed of the drone. In order to make the distance between the drone and the target meet the preset distance range. If the distance meets the preset distance range, the flying speed of the drone will not be adjusted. By adjusting the flight speed, it is possible to avoid the following failure of the UAV or damage caused by collision.
  • the prerequisite for the implementation of the foregoing embodiments is that the first similarity meets the first preset threshold, that is, the current image frame successfully follows the target object.
  • the first similarity may not meet the first preset threshold, which indicates that the current image frame fails to follow the target.
  • FIG. 4 is a schematic flowchart of another following method provided by an embodiment of the present invention. As shown in FIG. 4, based on any of the foregoing embodiments, the following method may further include the following steps:
  • S303 Determine whether the number of consecutive cumulative follow-up failures meets the preset number of times, if the preset number of times is met, execute step 304, otherwise execute step 305.
  • the first similarity degree that does not satisfy the first preset threshold may be that the first similarity degree is less than the first preset threshold.
  • the image frame performs the intelligent follow operation, and the UAV also exits the intelligent follow function.
  • the drone can respond to the start operation triggered by the user and re-enable the intelligent follow function.
  • the number of consecutive cumulative failures is less than the preset number, it indicates that the failure of the current image frame to follow the target is an occasional phenomenon. At this time, it can also be determined based on the follow frame of the historical image frame that is closest to the current image frame and successfully followed the target. The region of interest of an image frame.
  • the second size of the corresponding region of interest is larger than the third size of the region of interest of the previous image frame.
  • the ratio between the second size of the region of interest of the current image frame and the first size of the following frame of the successfully following historical image frame may be a second preset value. That is, N+L*t, where N and L are preset values, and t is the number of consecutively accumulated follow-up failures.
  • the preset value N here is also the first preset value mentioned above, and N>0, L>0, t ⁇ 0. In other words, the more consecutively accumulated follow-up failures, the larger the size of the region of interest in the next image frame, so that the target can be found in this larger region.
  • the historical image frame is the image frame that is the closest to the current image frame that has been successfully followed.
  • the drone if it is determined that the current image frame fails to follow the target, it can be further determined whether it is necessary to continue to follow the next image frame according to the number of consecutively accumulated follow failures.
  • the drone fails to follow for a long time, if it continues to follow, it will usually continue to fail, resulting in a waste of the drone's computing resources. At this time, the drone can directly exit the smart follow, avoiding the waste of computing resources.
  • the current image frame is only an accidental follow-up failure, the UAV will continue to follow the target to avoid an accidental follow-up error and completely abandon the target's following. And as the number of occasional follow-up failures increases, the previously selected region of interest is more inappropriate. At this time, the selected region of interest will also be larger, making the target object in a larger region of interest.
  • Fig. 6 is a schematic structural diagram of a following device provided by an embodiment of the present invention. referring to Fig. 6, this embodiment provides a following device, which can execute the above-mentioned following method; specifically, the following device includes :
  • the first determining module 11 is configured to determine the region of interest of the current image frame according to the following frame of the historical image frame that has successfully followed the target, wherein the second size of the region of interest is larger than the first size of the following frame Size, the following frame covers at least part of the target, and the historical image frame is an image frame before the current image frame.
  • the calculation module 12 is configured to calculate the first similarity between the following frame and the region of interest
  • the second determining module 13 is configured to determine that the current image frame successfully follows the target object if the first similarity degree meets a first preset threshold.
  • the device shown in FIG. 6 can also execute the methods of the embodiments shown in FIG. 1 to FIG. 5.
  • parts that are not described in detail in this embodiment reference may be made to the related descriptions of the embodiments shown in FIG. 1 to FIG. 5.
  • the implementation process and technical effects of this technical solution please refer to the description in the embodiment shown in FIG. 1 to FIG. 5, which will not be repeated here.
  • Fig. 7 is a schematic structural diagram of another follower device provided by an embodiment of the present invention
  • the structure of the follower device shown in Fig. 7 can be implemented as an electronic device, which can be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, etc. Wait.
  • the following device may include: one or more processors 21 and one or more memories 22.
  • the memory 22 is used to store a program that supports the following device to execute the following method provided in the embodiments shown in FIGS.
  • the processor 21 is configured to execute a program stored in the memory 22.
  • the program includes one or more computer instructions, where one or more computer instructions can implement the following steps when executed by the processor 21:
  • the historical image frame is an image frame before the current image frame
  • the structure of the device may also include a communication interface 23 for communication between the electronic device and other devices or a communication network.
  • the processor 21 is further configured to: if the first degree of similarity is greater than or equal to the first preset threshold, determine that the current image frame successfully follows the target object.
  • processor 21 is further configured to: match the follow frame of the current image frame and the historical image frame to determine the initial follow frame of the current image frame;
  • the initial following frame is adjusted according to at least one of the detection frames to generate the following frame of the current image frame.
  • the historical image frame is an image frame that is closest to the current image frame and successfully followed by the target.
  • the ratio of the second size of the region of interest of the current image frame to the first size of the following frame of the historical image frame is a first preset value.
  • the area of interest of the current image frame and the following frame of the historical image frame have the same area center.
  • processor 21 is further configured to: respectively calculate a second degree of similarity between the at least one detection frame and the initial following frame;
  • the initial following frame and the detection frame are fused to generate the following frame of the current image frame.
  • the processor 21 is further configured to: if the second degree of similarity does not meet the second preset threshold, determine that the initial following frame is the following frame of the current image frame.
  • the processor 21 is further configured to: if the distance between any one of the detection frames and the initial following frame of the current image frame is greater than the preset distance, delete the detection frame.
  • the processor 21 is further configured to: if the current image frame successfully follows the target object, determine the distance between the target object and the movable platform according to the following frame of the current image frame;
  • the movement state of the movable platform is adjusted so that the distance between the movable platform and the target object meets the preset distance range.
  • the processor 21 is further configured to: if the first similarity does not meet the first preset threshold, determine that the current image frame fails to follow the target;
  • processor 21 is further configured to: if the number of consecutive cumulative following failures does not meet the preset number of times, determine the region of interest of the next image frame according to the number of consecutive cumulative following failures;
  • the following operation is continued to be performed on the next image frame according to the region of interest.
  • the second size of the region of interest of the current image frame is larger than the third size of the region of interest of the previous image frame.
  • the ratio of the second size of the region of interest of the current frame image to the first size of the following frame of the historical image frame satisfies a second preset value
  • the second preset value is N+L*t
  • N and L are preset values
  • t is the number of continuous cumulative follow-up failures, N>0, L>0, t ⁇ 0.
  • the device shown in FIG. 7 can execute the methods of the embodiments shown in FIGS. 1 to 5, and for parts that are not described in detail in this embodiment, please refer to the related descriptions of the embodiments shown in FIGS. 1 to 5.
  • For the implementation process and technical effects of this technical solution please refer to the description in the embodiment shown in FIG. 1 to FIG. 5, which will not be repeated here.
  • FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention. referring to FIG. 8, an embodiment of the present invention provides a movable platform, and the movable platform is at least one of the following: Aircraft, unmanned ships, and unmanned vehicles; specifically, the movable platform includes: a body 31, a power system 32, a control device 33, and a camera 34.
  • the power system 32 is arranged on the body 31 and used to provide power for the movable platform.
  • the camera device 34 is arranged on the body 31 and is used to collect image frames.
  • the control device 33 includes a memory 331 and a processor 332.
  • the memory 331 is used to store computer programs
  • the processor 332 is configured to run a computer program stored in the memory to realize:
  • the historical image frame is an image frame before the current image frame
  • the processor 332 is further configured to: if the first degree of similarity is greater than or equal to the first preset threshold, determine that the current image frame successfully follows the target object.
  • the processor 332 is configured to: match the region of interest of the current image frame with the follow frame of the historical image frame to determine the initial follow frame of the current image frame;
  • the initial following frame is adjusted according to at least one of the detection frames to generate the following frame of the current image frame.
  • the historical image frame is an image frame that is closest to the current image frame and successfully followed by the target.
  • the ratio of the second size of the region of interest of the current image frame to the first size of the following frame of the historical image frame is a first preset value.
  • the area of interest of the current image frame and the following frame of the historical image frame have the same area center.
  • processor 332 is further configured to: respectively calculate a second degree of similarity between the at least one detection frame and the initial following frame;
  • the initial following frame and the detection frame are fused to generate the following frame of the current image frame.
  • the processor 332 is further configured to: if the second degree of similarity does not meet the second preset threshold, determine that the initial following frame is the following frame of the current image frame.
  • the processor 332 is further configured to: if the distance between any one of the detection frames and the initial following frame of the current image frame is greater than the preset distance, delete the detection frame.
  • the processor 332 is further configured to: if the current image frame successfully follows the target object, determine the distance between the target object and the movable platform according to the following frame of the current image frame;
  • the movement state of the movable platform is adjusted so that the distance between the movable platform and the target object meets the preset distance range.
  • processor 332 is further configured to: if the first degree of similarity does not meet the first preset threshold, determine that the current image frame fails to follow the target;
  • processor 332 is further configured to: if the cumulative number of consecutive follow-up failures does not meet the preset number of times, determine the region of interest of the next image frame according to the cumulative number of consecutive follow-up failures;
  • the following operation is continued to be performed on the next image frame according to the region of interest.
  • the second size of the region of interest of the current image frame is larger than the third size of the region of interest of the previous image frame.
  • the ratio of the second size of the region of interest of the current frame image to the first size of the following frame of the historical image frame satisfies a second preset value
  • the second preset value is N+L*t
  • N and L are preset values
  • t is the number of continuous cumulative follow-up failures, N>0, L>0, t ⁇ 0.
  • the movable platform shown in FIG. 8 can execute the methods of the embodiments shown in FIGS. 1 to 5, and for parts that are not described in detail in this embodiment, please refer to the related descriptions of the embodiments shown in FIGS. 1 to 5.
  • an embodiment of the present invention provides a computer-readable storage medium.
  • the storage medium is a computer-readable storage medium.
  • the computer-readable storage medium stores program instructions. method.
  • the related detection device for example: IMU
  • the embodiments of the remote control device described above are merely illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. It can be combined or integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, remote control devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • the aforementioned storage media include: U disk, mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

一种跟随方法、装置、可移动平台和存储介质，先获取对目标物跟随成功的历史图像帧的跟随框，根据此跟随框确定当前图像帧的感兴趣区域（S101）。然后，计算跟随框与感兴趣区域之间的第一相似度（S102），若此相似度满足第一预设阈值，则确定当前图像帧对目标物跟随成功（S103）。一方面，感兴趣区域小于当前图像帧的整个区域，仅需要对感兴趣区域进行特征匹配，以减小跟随过程中的计算量及传输至GPU的带宽消耗，避免感兴趣区域之外的区域包含的干扰信息对目标跟随造成的干扰。另一方面，采用感兴趣区域进行特征匹配，相当于把感兴趣区域放大后再进行特征匹配，可以提高目标跟随的精度，避免由于目标物太小而容易出现跟丢的问题。

Description

跟随方法、可移动平台、装置和存储介质 技术领域
本发明涉及图像处理领域，尤其涉及一种跟随方法、可移动平台、装置和存储介质。
背景技术
智能跟随是一种以智能设备配置有的传感器采集到的传感数据作为输入，在对传感视野中的某个对象进行指定后，自动持续对其进行锁定跟随的技术。这种跟随技术经常应用于视频拍摄领域，则上述的传感数据即为摄像头拍得的图像。通过具有跟随功能的智能设备的使用，一方面能够让拍摄画面呈现出更独特的角度，另一方面也能解放用户的双手，让拍摄更加轻松自如。其中，智能设备具体来说可以是可移动平台，比如无人机、无人车、手持稳定器等等。
现有技术中，目标跟随的实现过程通常是：智能设备先利用配置的摄像头拍得图像，再对拍得图像进行目标检测，以根据检测结果识别出目标物，从而完成目标物的跟随。在实际应用中，具有智能跟随功能的设备，如无人机，手持云台等，受到算力和带宽等因素限制，跟随效果存在瓶颈。
发明内容
本发明提供了一种跟随方法、可移动平台、装置和存储介质，用于减小跟随过程中的计算量，同时保证目标物的跟随效果。
本发明的第一方面是为了提供一种跟随方法，所述方法包括：
根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
计算所述跟随框与所述感兴趣区域之间的第一相似度；
若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目 标物跟随成功。
本发明的第二方面是为了提供一种跟随装置，所述装置包括：
存储器，用于存储计算机程序；
处理器，用于运行所述存储器中存储的计算机程序以实现：
根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
计算所述跟随框与所述感兴趣区域之间的第一相似度；
若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
本发明的第三方面是为了提供一种可移动平台，所述平台至少包括：机体、动力系统、控制装置以及摄像装置；
所述动力系统，设置于所述机体上，用于为所述可移动平台提供动力；
所述摄像装置，设置于所述机体上，用于采集图像帧；
所述控制装置包括存储器和处理器；
所述存储器，用于存储计算机程序；
所述处理器，用于运行所述存储器中存储的计算机程序以实现：
根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
计算所述跟随框与所述感兴趣区域之间的第一相似度；
若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
本发明的第四方面是为了提供一种计算机可读存储介质，所述存储介质为计算机可读存储介质，该计算机可读存储介质中存储有程序指令，所述程序指令用于第一方面所述的跟随方法。
本发明提供的跟随方法、可移动平台、装置和存储介质，先获取一历史图像帧的跟随框，此历史图像帧对目标物的跟随结果为跟随成功，并且此历史图像帧的跟随框覆盖至少部分目标物，再根据此跟随框确定当前图像帧的 感兴趣区域，其中，当前图像帧是历史图像帧之后的图像帧。然后，计算跟随框与感兴趣区域之间的第一相似度，若此相似度满足第一预设阈值，则认为当前图像帧对目标物的跟随结果为跟随成功。
可见，在使用上述方法时，一方面，感兴趣区域小于当前图像帧的整个区域，仅需要感兴趣区域进行特征匹配，无需整个图像帧区域进行特征匹配，从而减小跟随过程中的计算量及传输至GPU的带宽消耗，同时，也能够避免当前图像帧中感兴趣区域之外的区域包含的干扰信息对目标跟随造成的干扰。另一方面，采用感兴趣区域进行特征匹配，相当于把感兴趣区域放大后再进行特征匹配，可以提高目标跟随的精度，避免由于目标物太小而容易出现跟丢的问题。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1为本发明实施例提供的一种跟随方法的流程示意图；
图2为本发明实施例提供的另一种跟随方法的流程示意图；
图3为本发明实施例提供的根据检测框调整初始跟随框的流程示意图；
图4为本发明实施例提供的又一种跟随方法的流程示意图；
图5为本发明实施例提供的感兴趣区域与跟随框之间大小关系的示意图；
图6为本发明实施例提供的一种跟随装置的结构示意图；
图7为本发明实施例提供的另一种跟随装置的结构示意图；
图8为本发明实施例提供的一种可移动平台的结构示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技 术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。
在对本发明实施例提供的跟随方法进行详细介绍之前，先对可移动平台的目标跟随进行简单介绍。并且上述的可移动平台可以是无人机、无人车、无人船、稳定器等等。其中，以无人机为例进行说明：
无人机已经应用于众多领域中，比如娱乐、监控、安防等领域。在这些领域中，往往都具有跟随运动环境中一目标物的需求，比如用户在以一环境中运动时，可以有希望记录自己整个运动过程的需求，此时，无人机就需要对用户进行跟随，以拍摄用户的运动过程。
对用户的跟随，一种原始的方式，可以让飞手控制无人机的飞行以实现跟随。但这种人为遥控的方式自动化程度不高，并且也考验飞手的控制能力。在此种情况下，启动无人机的智能跟随功能便是一种较好的选择，这样无需专业的操作人员，同时也能够解放用户的双手，提高用户的使用效果。
下面结合附图，对本发明的一些实施方式作详细说明。在各实施例之间不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
图1为本发明实施例提供的一种跟随方法的流程示意图。该跟随方法的执行主体是跟随装置，可以理解的是，该跟随装置可以实现为软件、或者软件和硬件的组合；在跟随装置执行该跟随方法时，可以实现对目标物的准确跟随。跟随装置具体来说可以是各种类型的可移动平台，本实施例以及下述各实施例均以无人机为例进行说明，当然执行主体也可是其他类型的可移动平台，本发明并不进行限定。具体的，该方法可以包括：
S101，根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域。
无人机配置的摄像头可以实时拍得对应于无人机飞行环境的图像帧，则对于当前时刻，无人机配置的摄像头可以拍得一图像帧也即是当前图像帧。由于目标物的体积有限，一般不会充斥于一整幅图像帧中，因此，在得到当前图像帧后，还可以在其中选取部分图像作为感兴趣区域，并在此感兴趣区域内确定是否存在目标物，也即是在此感兴趣区域内确定当前图像帧对目标物的跟随结果。
当然，也正是由于只有当前图像帧中的部分图像会参与到后续判断跟随是否成功的过程中，因此，也就能够减小后续处理流程中的计算量。
对于感兴趣区域的选取，可以依据对目标物跟随成功的历史图像帧的跟随框来选取。为了后续描述简洁，可以将历史图像帧的跟随框的尺寸称为第一尺寸，将当前图像帧的感兴趣区域的尺寸称为第二尺寸。可选地，可以从当前图像帧中选取第二尺寸的部分图像作为当前图像帧的感兴趣区域，其中，第二尺寸大于第一尺寸。由于历史图像帧对目标物是跟随成功的，因此，可选地，第二尺寸和第一尺寸的比值可以为第一预设值，且第一预设值通常为整数。比如，可以将跟随框的第一尺寸扩大两倍，以得到感兴趣区域的第二尺寸。并且可选地，当前图像帧的感兴趣区域还可以和历史图像帧的跟随框具有相同的区域中心，即两者的中心重合。
根据上述描述可知，在感兴趣区域选取的过程中涉及到了历史图像帧以及历史图像帧的跟随框，对二者可以进行以下说明：
对于历史图像帧，顾名思义，一种可选地方式，其可以是在当前图像帧之前采集到的任一对目标物跟随成功的图像帧。对于历史图像帧的跟随框，若一历史图像帧对目标物的跟随结果为跟随成功，表明目标物是包含于历史图像帧中的，并且此目标物会用一跟随框标注出来。跟随框内可以包含至少部分的目标物。在一实施例中，目标物也可以全部包含于跟随框内。
S102，计算跟随框与感兴趣区域之间的第一相似度。
接着，计算历史图像帧的跟随框与当前图像帧的感兴趣区域之间的第一相似度。可选地，由于跟随框和感兴趣区域实际上都是图像帧中的部分图像，因此，可以先分别提取跟随框和感兴趣区域的图像特征，再计算图像特征之间的相似度，用图像特征之间的相似度作为跟随框与感兴趣区域之间的第一相似度。
可选地，分别对跟随框和感兴趣区域进行特征提取，以分别生成多个特征向量。再根据跟随框和感兴趣区域各自的特征向量进行计算得到跟随框与感兴趣区域之间的第一相似度。
S103，若第一相似度满足第一预设阈值，则确定当前图像帧对目标物跟随成功。
由于此历史图像帧跟随框中包含目标物，因此，若第一相似度满足第一预设阈值，也就可以认为当前图像帧的感兴趣区域中也包含目标物，则确定当前图像帧对目标物跟随成功。其中，可选地，第一相似度满足第一预设阈值可以是第一相似度大于或等于第一预设阈值。
本实施例提供的跟随方法，先获取对目标物跟随成功的历史图像帧的跟随框，根据此跟随框确定当前图像帧的感兴趣区域，其中，当前图像帧是历史图像帧之后的图像帧。然后，计算跟随框与感兴趣区域之间的第一相似度，若此相似度满足第一预设阈值，则确定当前图像帧对目标物跟随成功。可见，在使用上述方法时，一方面，感兴趣区域小于当前图像帧的整个区域，仅需要感兴趣区域进行特征匹配，无需整个图像帧区域进行特征匹配，从而减小跟随过程中的计算量及传输至GPU的带宽消耗，同时，也能够避免当前图像帧中感兴趣区域之外的区域包含的干扰信息对目标跟随造成的干扰。另一方面，采用感兴趣区域进行特征匹配，相当于把感兴趣区域放大后再进行特征匹配，可以提高目标跟随的精度，避免由于目标物太小而容易出现跟丢的问题。
假设在第M时刻、第N时刻分别采集到图像帧A和图像帧B，其中，第N时刻为当前时刻，且图像帧A和图像帧B均对目标物跟随成功。在无人机的实际飞行过程中目标物通常是不断运动的，因此，目标物在图像帧A中的位置与其在图像帧B中的位置通常具有一定变化。
而一方面，根据图1所示实施例中的描述可知，当前图像帧感兴趣区域和历史图像帧跟随框具有相同的区域中心，并且二者的尺寸大小也密切相关。另一方面，又因为目标物在两图像帧中的位置变化较大，因此，若直接使用图像帧B的跟随框来确定图像帧A的感兴趣区域，则目标物很有可能不能被划入图像帧A的感兴趣区域内，也即是目标物位于图像帧A的非感兴趣区域内。此时，无人机根据此已经不包含目标的感兴趣区域必然不能得到正确的跟随结果。这也就表明，图1所示实施例中提供的将在当前图像帧之前采集到的任一图像帧确定历史图像帧的方式并不十分恰当。
因此，另一种可选地方式，可以将与当前图像帧采集时刻相邻的图像帧确定为历史图像帧，并且此历史图像帧对目标物也要是跟随成功的。利用这种方式确定出的历史图像帧与当前图像帧具有相邻的采集时刻，由于时间间隔较短，因此，目标物在图像帧中的位置通常不会出现较大的变动，可以保证跟随性能。
根据图1所示实施例中的描述可知，在确定当前帧图像对目标物是否跟随成功时使用到的是跟随成功的历史图像帧的跟随框。随着无人机配置的摄像 头不断拍得图像帧，当出现下一图像帧时，上述的当前图像帧也就相应地变为历史图像帧，此时，就需要根据此当前图像帧的跟随框来判断下一图像帧对目标物体是否跟随成功。
基于此，图2为本发明实施例提供的另一种跟随方法的流程示意图。如图2所示，在步骤103之后，该跟随方法还可以包括以下步骤：
S201，对当前图像帧的感兴趣区域和历史图像帧的跟随框进行匹配，以确定当前图像帧的初始跟随框。
在执行步骤101后，便可以确定出当前图像帧的感兴趣区域，之后可以将此感兴趣区域与历史图像帧的跟随框进行匹配处理，并根据匹配区域确定为当前图像帧的初始跟随框。
在一实施例中，将当前图像帧的感兴趣区域放大后再与历史图像帧的跟随框进行匹配处理，可以避免目标物太小而导致跟丢的情况。
另外，感兴趣区域放大后与所述历史图像帧的跟随框进行匹配处理，这样使得生成的跟随框更准确，防止缩框、扩框等造成目标物的跟随位置不准确而导致距离测算存在较大误差的问题。并且根据跟随框在图像帧上的位置计算得到的可移动平台和目标物之间的距离更为准确，有利于避障等飞控操作。
示例性地，可以将感兴趣区域放大至与当前图像帧相同的维度后再与跟随框进行匹配处理。
另外，在完成特征匹配生成初始跟随框后，可缩小回原来的尺度，以使得初始跟随框能用于当前图像帧的目标跟随。
S202，根据预设算法对当前图像帧进行目标检测处理，以生成至少一个检测框。
同时，还可以根据预设的目标检测算法对当前图像帧进行目标检测，以将当前图像帧中预设类别的物体都检测出来，也即是检测出各物体所在的位置。例如，可以采用预设的神经网络模型检测所述当前图像帧进行目标检测。。
需要说明的有，上述先执行步骤201再执行步骤202的方式只是一种示例。在实际应用中，上述两步骤之间可以不具有时序，二者可以同时执行也可以先后执行，本发明并不对此进行限定。
S203，根据至少一个检测框对初始跟随框进行调整，以生成当前图像帧的跟随框。
无论是当前图像帧的初始跟随框还是及当前图像帧对应的检测框，二者只不过是得到的方式不同，所以命名不同，但实际上它们都是对目标物进行标注处理的结果，两框中都可以包含目标物。此时，为了进一步保证目标物能够更加完整、精准地被标注，还可以将初始跟随框与检测框进行融合处理，以实现利用检测框对初始跟随框进行校准，从而得到当前图像帧最终的跟随框。
在无人机在下一时刻拍得下一图像帧后，此处得到的当前图像帧的跟随框也即相当于图1所示实施例中的历史图像帧的跟随框，此时，便可以继续执行图1中的各步骤来判断下一帧图像对目标物是否跟随成功。
本实施例中，在使用图1所示实施例的方式确定出当前图像帧对目标物跟随成功后，一方面，可以进一步使用特征匹配的方式得到当前图像帧的初始跟随框，另一方面，还可以使用目标检测算法得到当前图像帧的检测框。然后，再利用检测框对初始跟随框进行校正，将校正后的结果确定为当前图像帧最终的跟随框。通过校正处理，能够保证当前图像帧的跟随框的位置和大小更加准确。
当无人机拍摄到下一时刻的下一图像帧后，上述的当前帧图像的跟随框已经变为历史图像帧的跟随框，此时，可以继续利用图1所示实施例的方式确定此下一图像帧对目标物是否跟随成功。由于当前图像帧的跟随框已经是准确的，因此，也就能够准确的确定出下一图像帧对目标物是否跟随成功。
图2所示实施例中已经提及了根据检测框对初始跟随框进行调整的步骤，如图3所示，调整过程一种可选地实现方式，也即是步骤203一种可选地可实现方式可以为：
S2031，分别计算至少一个检测框与初始跟随框之间的第二相似度。
S2032，判断第二相似度是否满足第二预设阈值，若满足第二预设阈值，则执行步骤2033，否则执行步骤2034。
S2033，将初始跟随框与检测框进行融合，以生成当前图像帧的跟随框。
S2034，确定初始跟随框为当前图像帧的跟随框。
具体来说，先分别计算至少一个检测框与初始跟随框之间的第二相似度，则每个检测框都对应于一个第二相似度。然后，再判断第二相似度与第二预设阈值之间的大小关系，以根据大小关系选择是否将检测框与初始跟随框进 行融合。
若第二相似度大于或等于第二预设阈值，可以认为检测框包含的物体与当前图像帧的初始跟随框包含的物体具体有较高的相似度，则可以将初始跟随框与此目标检测框进行融合，从而得到当前图像帧最终的跟随框。
若第二相似度小于第二预设阈值，可以认为检测框包含的物体与当前图像帧的初始跟随框包含的物体差距较大甚至不是一个物体，此时则可以不进行融合处理，而是直接将初始跟随框确定为当前图像帧最终的跟随框。
另外，由于相邻的图像帧之间的采集时间间隔非常短，因此，目标物在图像帧中的位置不会发生非常大的变化，则会存在图像帧中非目标物体所处的位置与目标物相距较远的情况。此时，若将这些与目标物相距较远的物体的检测框参与到初始跟随框的调整过程中，会增大计算量。
因此，为了避免上述问题，基于上述各实施例，在步骤2031之前，还可以对至少一个检测框进行以下处理：
若至少一个检测框中存在一检测框与当前图像帧的初始跟随框的距离大于预设距离，则删除此检测框。
具体地，先计算每一检测框与历史图像帧的初始跟随框之间的距离。若检测框与初始跟随框之间的距离大于预设距离，表明此检测框中包含的物体与初始跟随框中包含的物体位置差距较远，检测框中包含的是非目标物，此时便可以将此检测框删除，以进一步利用剩余的至少一个检测框执行图3所示的流程。
本实施例中，通过对至少一个检测框的筛选处理，可以剔除明显包含非目标物的检测框，从而保证检测框与初始跟随框的融合效果，以得到精准的当前图像帧的跟随框。
另外，在跟随目标物的过程中，若目标物与无人机之间的距离较远，则目标物在无人机拍得的图像帧中对应于一个较小的区域，从而容易出现匹配不上，而导致发生跟随失败的情况。若目标物与无人机之间的距离较近，当目标物突然减速运动时，无人机又会很容易与目标物发生冲撞，造成损坏。
因此，当根据图1～图3所示实施例提供的方式确定出当前图像帧对目标物跟随成功后，还可以进一步根据当前图像帧的跟随框确定目标物与无人机之 间的距离。
若距离不满足预设距离范围，表明无人机与目标物之间的距离过大或者过小，此时，则调整无人机的运动状态，具体来说就是调整无人机的飞行速度，以使无人机与目标物之间的距离满足预设距离范围。若距离满足预设距离范围，则不对无人机的飞行速度进行调整。通过飞行速度的调整便能够避免出现无人机的跟随失败或者发生冲撞造成损毁的情况。
上述各实施例执行的前提均是第一相似度满足第一预设阈值也即是当前图像帧对目标物跟随成功。除此之外，容易理解的，在经过步骤102之后，还有可能出现第一相似度不满足第一预设阈值的情况，此时表明当前图像帧对目标物跟随失败。
对于此种情况，图4为本发明实施例提供的又一种跟随方法的流程示意图。如图4所示，在上述任一实施例的基础上，该跟随方法还可以包括以下步骤：
S301，若第一相似度不满足第一预设阈值，则确定当前图像帧对目标物跟随失败。
S302，统计连续累积跟随失败次数。
S303，判断连续累积跟随失败次数是否满足预设次数，若满足预设次数，则执行步骤304，否则执行步骤305。
S304，停止对下一图像帧进行智能跟随操作。
S305，对下一图像帧继续执行跟随操作。
具体地，由于此历史图像帧跟随框中包含目标物，因此，当第一相似度不满足第一预设阈值时，也就可以认为当前图像帧的感兴趣区域中不包含目标物，则确定当前图像帧对目标物跟随失败。其中，可选地，第一相似度不满足第一预设阈值可以是第一相似度小于第一预设阈值。
此时，将连续累积跟随失败次数加1，以得到最新的连续累积跟随失败次数。再进一步判断更新后的失败次数是否大于预设次数。若失败次数大于或等于预设次数，表明对于无人机连续采集到的多张图像帧中都没有跟随到目标物，此时可以认为目标物已跟丢，此时无人机停止对下一图像帧进行智能跟随操作，无人机也即是退出智能跟随功能。无人机可以响应用户触发的启动操作，而重新开启智能跟随功能。
若连续累积失败次数小于预设次数，表明当前图像帧对目标物的跟随失 败是一个偶发现象，此时还可以同时根据与当前图像帧最近的对目标物跟随成功的历史图像帧的跟随框定下一图像帧的感兴趣区域。
对于待进行跟随过程的当前图像帧而言，其对应的感兴趣区域的第二尺寸大于上一图像帧的感兴趣区域的第三尺寸。
具体来说，当前图像帧的感兴趣区域的第二尺寸和跟随成功的历史图像帧的跟随框的第一尺寸之间的比值可以为第二预设值。即N+L*t，其中，N、L为预设值，t为连续连续累积跟随失败次数。且此处的预设值N也即是上述提及的第一预设值，且N>0，L>0，t≥0。也就是说，连续累积跟随失败次数越多，则下一图像帧的感兴趣区域的尺寸越大，以便在此更大的区域内找寻目标物。
在一实施例中，N＝2,L＝1。在对当前图像帧进行跟随操作前，若连续累积次数为0，即上一图像帧跟随成功时，则当前图像帧的感兴趣区域的尺寸大小为2+1X0＝2倍的历史图像帧的跟随框的尺寸，如图5中的(a)所示。若连续累积次数为1时，则当前图像帧的感兴趣区域的尺寸大小为2+1X1＝3倍的历史图像帧的跟随框的尺寸，如图5中的(b)所示。
在一实施例中，该历史图像帧为与当前图像帧距离最近的跟随成功的图像帧。
本实施例中，若确定出当前图像帧对目标物跟随失败后，可以进一步根据连续累积跟随失败次数确定是否需要继续对下一图像帧进行跟随。无人机长时间跟随失败时，若继续跟随的话通常还是会继续跟随失败，从而造成无人机运算资源的浪费，此时，无人机便可以直接退出智能跟随，避免了运算资源的浪费。若当前图像帧只是偶发的一次跟随失败，则无人机还会继续对目标物进行跟随，避免因一次偶发的跟随失误，而完全放弃目标物的跟随。并且随着偶发的跟随失败的次数越多，说明之前选取的感兴趣区域越不恰当，此时选取出的感兴趣区域也会越大，使得在一个较大的感兴趣区域内对目标物进行跟随，保证跟随效果。
图6为本发明实施例提供的一种跟随装置的结构示意图；参考附图6所示，本实施例提供了一种跟随装置，该跟随装置可以执行上述的跟随方法；具体的，跟随装置包括：
第一确定模块11，用于根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述 跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧。
计算模块12，用于计算所述跟随框与所述感兴趣区域之间的第一相似度；
第二确定模块13，用于若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
图6所示装置还可以执行图1～图5所示实施例的方法，本实施例未详细描述的部分，可参考对图1～图5所示实施例的相关说明。该技术方案的执行过程和技术效果参见图1～图5所示实施例中的描述，在此不再赘述。
图7为本发明实施例提供的另一种跟随装置的结构示意图；图7所示跟随装置的结构可实现为一电子设备，该电子设备可以是无人机、无人车、无人船等等。如图7所示，该跟随装置可以包括：一个或多个处理器21和一个或多个存储器22。其中，存储器22用于存储支持跟随装置执行上述图1～图5所示实施例中提供的跟随方法的程序。处理器21被配置为用于执行存储器22中存储的程序。
具体的，程序包括一条或多条计算机指令，其中，一条或多条计算机指令被处理器21执行时能够实现如下步骤：
根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
计算所述跟随框与所述感兴趣区域之间的第一相似度；
若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
其中，该装置的结构中还可以包括通信接口23，用于电子设备与其他设备或通信网络通信。
进一步的，处理器21还用于：若所述第一相似度大于或等于所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
进一步的，处理器21还用于：对所述当前图像帧和所述历史图像帧的跟随框进行匹配，以确定所述当前图像帧的初始跟随框；
根据预设算法对所述当前图像帧的感兴趣区域进行目标检测处理，以生 成至少一个检测框；
根据至少一个所述检测框对所述初始跟随框进行调整，以生成所述当前图像帧的跟随框。
其中，所述历史图像帧为与所述当前图像帧最近的且目标物跟随成功的图像帧。
其中，所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值为第一预设值。
其中，所述当前图像帧的感兴趣区域与所述历史图像帧的跟随框具有相同的区域中心。
进一步的，处理器21还用于：分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度；
若所述第二相似度满足第二预设阈值，将所述初始跟随框与所述检测框进行融合，以生成所述当前图像帧的跟随框。
进一步的，处理器21还用于：若所述第二相似度不满足所述第二预设阈值，则确定所述初始跟随框为当前图像帧的跟随框。
进一步的，处理器21还用于：若任一所述检测框与所述当前图像帧的初始跟随框的距离大于所述预设距离，则删除所述检测框。
进一步的，处理器21还用于：若所述当前图像帧对所述目标物跟随成功，则根据所述当前图像帧的跟随框，确定所述目标物与可移动平台之间的距离；
若所述距离不满足预设距离范围，则调整所述可移动平台的运动状态，以使所述可移动平台与所述目标物之间的距离满足所述预设距离范围。
进一步的，处理器21还用于：若所述第一相似度不满足所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随失败；
统计连续累积跟随失败次数；
若所述连续累积跟随失败次数满足预设次数，则停止对下一图像帧进行智能跟随操作。
进一步的，处理器21还用于：若所述连续累积跟随失败次数不满足所述预设次数，则根据所述连续累积跟随失败次数确定所述下一图像帧的感兴趣区域；
根据所述感兴趣区域对所述下一图像帧继续执行跟随操作。
其中，所述当前图像帧的感兴趣区域的第二尺寸大于上一图像帧的感兴 趣区域的第三尺寸。
其中，所述当前帧图像的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值满足第二预设值，所述第二预设值为N+L*t，其中，N、L为预设值，t为连续累积跟随失败次数，N>0，L>0，t≥0。
图7所示装置可以执行图1～图5所示实施例的方法，本实施例未详细描述的部分，可参考对图1～图5所示实施例的相关说明。该技术方案的执行过程和技术效果参见图1～图5所示实施例中的描述，在此不再赘述。
图8为本发明实施例提供的一种可移动平台的结构示意图；参考附图8所示，本发明实施例的提供了一种可移动平台，该可移动平台为以下至少之一：无人飞行器、无人船、无人车；具体的，该可移动平台包括：机体31、动力系统32、控制装置33以及摄像装置34。
所述动力系统32，设置于所述机体31上，用于为所述可移动平台提供动力。
所述摄像装置34，设置于所述机体31上，用于采集图像帧。
所述控制装置33包括存储器331和处理器332。
所述存储器331，用于存储计算机程序；
所述处理器332，用于运行所述存储器中存储的计算机程序以实现：
根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
计算所述跟随框与所述感兴趣区域之间的第一相似度；
若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
进一步的，处理器332还用于：若所述第一相似度大于或等于所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
进一步的，该处理器332用于：对所述当前图像帧的感兴趣区域和所述历史图像帧的跟随框进行匹配，以确定所述当前图像帧的初始跟随框；
根据预设算法对所述当前图像帧进行目标检测处理，以生成至少一个检测框；
根据至少一个所述检测框对所述初始跟随框进行调整，以生成所述当前图像帧的跟随框。
其中，所述历史图像帧为与所述当前图像帧最近的且目标物跟随成功的图像帧。
其中，所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值为第一预设值。
其中，所述当前图像帧的感兴趣区域与所述历史图像帧的跟随框具有相同的区域中心。
进一步的，处理器332还用于：分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度；
若所述第二相似度满足第二预设阈值，将所述初始跟随框与所述检测框进行融合，以生成所述当前图像帧的跟随框。
进一步的，处理器332还用于：若所述第二相似度不满足所述第二预设阈值，则确定所述初始跟随框为当前图像帧的跟随框。
进一步的，处理器332还用于：若任一所述检测框与所述当前图像帧的初始跟随框的距离大于所述预设距离，则删除所述检测框。
进一步的，处理器332还用于：若所述当前图像帧对所述目标物跟随成功，则根据所述当前图像帧的跟随框，确定所述目标物与可移动平台之间的距离；
若所述距离不满足预设距离范围，则调整所述可移动平台的运动状态，以使所述可移动平台与所述目标物之间的距离满足所述预设距离范围。
进一步的，处理器332还用于：若所述第一相似度不满足所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随失败；
统计连续累积跟随失败次数；
若所述连续累积跟随失败次数满足预设次数，则停止对下一图像帧进行智能跟随操作。
进一步的，处理器332还用于：若所述连续累积跟随失败次数不满足所述预设次数，则根据所述连续累积跟随失败次数确定所述下一图像帧的感兴趣区域；
根据所述感兴趣区域对所述下一图像帧继续执行跟随操作。
其中，所述当前图像帧的感兴趣区域的第二尺寸大于上一图像帧的感兴趣区域的第三尺寸。
其中，所述当前帧图像的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值满足第二预设值，所述第二预设值为N+L*t，其中，N、L为预设值,t为连续累积跟随失败次数，N>0,L>0,t≥0。
图8所示的可移动平台可以执行图1～图5所示实施例的方法，本实施例未详细描述的部分，可参考对图1～图5所示实施例的相关说明。该技术方案的执行过程和技术效果参见图1～图5所示实施例中的描述，在此不再赘述。
另外，本发明实施例提供了一种计算机可读存储介质，存储介质为计算机可读存储介质，该计算机可读存储介质中存储有程序指令，程序指令用于实现上述图1～图5的跟随方法。
以上各个实施例中的技术方案、技术特征在与本相冲突的情况下均可以单独，或者进行组合，只要未超出本领域技术人员的认知范围，均属于本申请保护范围内的等同实施例。
在本发明所提供的几个实施例中，应该理解到，所揭露的相关检测装置(例如：IMU)和方法，可以通过其它的方式实现。例如，以上所描述的遥控装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，遥控装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的 全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得计算机处理器(processor)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁盘或者光盘等各种可以存储程序代码的介质。
以上所述仅为本发明的实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (33)

  1. 一种跟随方法，其特征在于，所述方法包括：
    根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
    计算所述跟随框与所述感兴趣区域之间的第一相似度；
    若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  2. 根据权利要求1所述的方法，其特征在于，所述若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功，包括：
    若所述第一相似度大于或等于所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  3. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    对所述当前图像帧的感兴趣区域和所述历史图像帧的跟随框进行匹配，以确定所述当前图像帧的初始跟随框；
    根据预设算法对所述当前图像帧进行目标检测处理，以生成至少一个检测框；
    根据至少一个所述检测框对所述初始跟随框进行调整，以生成所述当前图像帧的跟随框。
  4. 根据权利要求3所述的方法，其特征在于，所述根据至少一个所述检测框对所述跟随框进行调整，以生成所述当前图像帧的跟随框，包括：
    分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度；
    若所述第二相似度满足第二预设阈值，将所述初始跟随框与所述检测框进行融合，以生成所述当前图像帧的跟随框。
  5. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    若所述第二相似度不满足所述第二预设阈值，则确定所述初始跟随框为当前图像帧的跟随框。
  6. 根据权利要求4所述的方法，其特征在于，所述分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度之前，所述方法还包括：
    若任一所述检测框与所述当前图像帧的初始跟随框的距离大于所述预设 距离，则删除所述检测框。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述历史图像帧为与所述当前图像帧最近的且目标物跟随成功的图像帧。
  8. 根据权利要求7所述的方法，其特征在于，所述当前图像帧的感兴趣区域与所述历史图像帧的跟随框具有相同的区域中心。
  9. 根据权利要求8所述的方法，其特征在于，所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值为第一预设值。
  10. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    若所述当前图像帧对所述目标物跟随成功，则根据所述当前图像帧的跟随框，确定所述目标物与可移动平台之间的距离；
    若所述距离不满足预设距离范围，则调整所述可移动平台的运动状态，以使所述可移动平台与所述目标物之间的距离满足所述预设距离范围。
  11. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    若所述第一相似度不满足所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随失败；
    统计连续累积跟随失败次数；
    若所述连续累积跟随失败次数满足预设次数，则停止对下一图像帧进行智能跟随操作。
  12. 根据权利要求11所述的方法，其特征在于，所述方法还包括：
    若所述连续累积跟随失败次数不满足所述预设次数，则根据所述连续累积跟随失败次数确定所述下一图像帧的感兴趣区域；
    根据所述感兴趣区域对所述下一图像帧继续执行跟随操作。
  13. 根据权利要求1所述的方法，其特征在于，所述当前图像帧的感兴趣区域的第二尺寸大于上一图像帧的感兴趣区域的第三尺寸。
  14. 根据权利要求1所述的方法，其特征在于，所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值满足第二预设值，所述第二预设值为N+L*t，其中，N、L为预设值,t为连续累积跟随失败次数，N>0,L>0,t≥0。
  15. 一种可移动平台，其特征在于，至少包括：机体、动力系统、摄像装置以及控制装置；
    所述动力系统，设置于所述机体上，用于为所述可移动平台提供动力；
    所述摄像装置，设置于所述机体上，用于采集图像帧；
    所述控制装置包括存储器和处理器；
    所述存储器，用于存储计算机程序；
    所述处理器，用于运行所述存储器中存储的计算机程序以实现：
    根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
    计算所述跟随框与所述感兴趣区域之间的第一相似度；
    若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  16. 根据权利要求15所述的平台，其特征在于，所述处理器还用于：
    若所述第一相似度大于或等于所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  17. 根据权利要求16所述的平台，其特征在于，所述处理器还用于：
    对所述当前图像帧的感兴趣区域和所述历史图像帧的跟随框进行匹配，以确定所述当前图像帧的初始跟随框；
    根据预设算法对所述当前图像帧进行目标检测处理，以生成至少一个检测框；
    根据至少一个所述检测框对所述初始跟随框进行调整，以生成所述当前图像帧的跟随框；
    其中，所述历史图像帧为与所述当前图像帧最近的且目标物跟随成功的图像帧；所述当前图像帧的感兴趣区域与所述历史图像帧的跟随框具有相同的区域中心；所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值为第一预设值。
  18. 根据权利要求17所述的平台，其特征在于，所述处理器还用于：
    分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度；
    若所述第二相似度满足第二预设阈值，将所述初始跟随框与所述检测框进行融合，以生成所述当前图像帧的跟随框。
  19. 根据权利要求18所述的平台，其特征在于，所述处理器还用于：
    若所述第二相似度不满足所述第二预设阈值，则确定所述初始跟随框为 当前图像帧的跟随框。
  20. 根据权利要求18所述的平台，其特征在于，所述处理器还用于：
    若任一所述目标检测框与所述当前图像帧的初始跟随框的距离大于所述预设距离，则删除所述检测框。
  21. 根据权利要求19所述的平台，其特征在于，所述处理器还用于：
    若所述当前图像帧对所述目标物跟随成功，则根据所述当前图像帧的跟随框，确定所述目标物与可移动平台之间的距离；
    若所述距离不满足预设距离范围，则调整所述可移动平台的运动状态，以使所述可移动平台与所述目标物之间的距离满足所述预设距离范围。
  22. 根据权利要求15所述的平台，其特征在于，所述处理器还用于：
    若所述第一相似度不满足所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随失败；
    统计连续累积跟随失败次数；
    若所述连续累积跟随失败次数满足预设次数，则停止对下一图像帧进行智能跟随操作。
  23. 根据权利要求22所述的平台，其特征在于，所述处理器还用于：
    若所述连续累积跟随失败次数不满足所述预设次数，则根据所述连续累积跟随失败次数确定所述下一图像帧的感兴趣区域；
    根据所述感兴趣区域对所述下一图像帧继续执行跟随操作；
    其中，所述当前图像帧的感兴趣区域的第二尺寸大于上一帧图像帧的感兴趣区域的第三尺寸；
    其中，所述当前像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值满足第二预设值，所述第二预设值为N+L*t，其中，N、L为预设值,t为连续累积跟随失败次数，N>0,L>0,t≥0。
  24. 一种跟随装置，其特征在于，所述装置包括：
    存储器，用于存储计算机程序；
    处理器，用于运行所述存储器中存储的计算机程序以实现：
    根据对目标物跟随成功的历史图像帧的跟随框，确定当前图像帧的感兴趣区域，其中，所述感兴趣区域的第二尺寸大于所述跟随框的第一尺寸，所述跟随框覆盖至少部分所述目标物，所述历史图像帧为所述当前图像帧之前的图像帧；
    计算所述跟随框与所述感兴趣区域之间的第一相似度；
    若所述第一相似度满足第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  25. 根据权利要求24所述的装置，其特征在于，所述处理器还用于：
    若所述第一相似度大于或等于所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随成功。
  26. 根据权利要求25所述的装置，其特征在于，所述处理器还用于：
    对所述当前图像帧的感兴趣区域和所述历史图像帧的跟随框进行匹配，以确定所述当前图像帧的初始跟随框；
    根据预设算法对所述当前图像帧进行目标检测处理，以生成至少一个检测框；
    根据至少一个所述检测框对所述初始跟随框进行调整，以生成所述当前图像帧的跟随框；
    其中，所述历史图像帧为与所述当前图像帧最近的且目标物跟随成功的图像帧；所述当前图像帧的感兴趣区域与所述历史图像帧的跟随框具有相同的区域中心；所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值为第一预设值。
  27. 根据权利要求26所述的装置，其特征在于，所述处理器还用于：
    分别计算至少一个所述检测框与所述初始跟随框之间的第二相似度；
    若所述第二相似度满足第二预设阈值，将所述初始跟随框与所述检测框进行融合，以生成所述当前图像帧的跟随框。
  28. 根据权利要求27所述的装置，其特征在于，所述处理器还用于：
    若所述第二相似度不满足所述第二预设阈值，则确定所述初始跟随框为当前图像帧的跟随框。
  29. 根据权利要求27所述的装置，其特征在于，所述处理器还用于：
    若任一所述检测框与所述当前图像帧的初始跟随框的距离大于所述预设距离，则删除所述检测框。
  30. 根据权利要求28所述的装置，其特征在于，所述处理器还用于：
    若所述当前图像帧对所述目标物跟随成功，则根据所述当前图像帧的跟随框，确定所述目标物与可移动平台之间的距离；
    若所述距离不满足预设距离范围，则调整所述可移动平台的运动状态， 以使所述可移动平台与所述目标物之间的距离满足所述预设距离范围。
  31. 根据权利要求24所述的装置，其特征在于，所述处理器还用于：
    若所述第一相似度不满足所述第一预设阈值，则确定所述当前图像帧对所述目标物跟随失败；
    统计连续累积跟随失败次数；
    若所述连续累积跟随失败次数满足预设次数，则停止对下一图像帧进行智能跟随操作。
  32. 根据权利要求31所述的装置，其特征在于，所述处理器还用于：
    若所述连续累积跟随失败次数不满足所述预设次数，则根据所述连续累积跟随失败次数确定所述下一图像帧的感兴趣区域；
    根据所述感兴趣区域对所述下一图像帧继续执行跟随操作；
    其中，所述当前像帧的感兴趣区域的第二尺寸大于上一图像帧的感兴趣区域的第三尺寸；
    其中，所述当前图像帧的感兴趣区域的第二尺寸与所述历史图像帧的跟随框的第一尺寸的比值满足第二预设值，所述第二预设值为N+L*t，其中，N、L为预设值,t为连续累积跟随失败次数，N>0,L>0,t≥0。
  33. 一种计算机可读存储介质，其特征在于，所述存储介质为计算机可读存储介质，该计算机可读存储介质中存储有程序指令，所述程序指令用于实现权利要求1至14中任一项所述的跟随方法。
PCT/CN2020/073626 2020-01-21 2020-01-21 跟随方法、可移动平台、装置和存储介质 WO2021146952A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/073626 WO2021146952A1 (zh) 2020-01-21 2020-01-21 跟随方法、可移动平台、装置和存储介质
CN202080004152.2A CN112585944A (zh) 2020-01-21 2020-01-21 跟随方法、可移动平台、装置和存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/073626 WO2021146952A1 (zh) 2020-01-21 2020-01-21 跟随方法、可移动平台、装置和存储介质

Publications (1)

Publication Number Publication Date
WO2021146952A1 true WO2021146952A1 (zh) 2021-07-29

Family

ID=75145415

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/073626 WO2021146952A1 (zh) 2020-01-21 2020-01-21 跟随方法、可移动平台、装置和存储介质

Country Status (2)

Country Link
CN (1) CN112585944A (zh)
WO (1) WO2021146952A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982559A (zh) * 2012-11-28 2013-03-20 大唐移动通信设备有限公司 车辆跟踪方法及系统
US9390506B1 (en) * 2015-05-07 2016-07-12 Aricent Holdings Luxembourg S.A.R.L. Selective object filtering and tracking
CN106096577A (zh) * 2016-06-24 2016-11-09 安徽工业大学 一种摄像头分布地图中的目标追踪系统及追踪方法
CN106682619A (zh) * 2016-12-28 2017-05-17 上海木爷机器人技术有限公司 一种对象跟踪方法及装置
CN109165646A (zh) * 2018-08-16 2019-01-08 北京七鑫易维信息技术有限公司 一种确定图像中用户的感兴趣区域的方法及装置
CN109598234A (zh) * 2018-12-04 2019-04-09 深圳美图创新科技有限公司 关键点检测方法和装置
CN110415208A (zh) * 2019-06-10 2019-11-05 西安电子科技大学 一种自适应目标检测方法及其装置、设备、存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5720275B2 (ja) * 2011-02-03 2015-05-20 株式会社リコー 撮像装置および撮像方法
CN105825524B (zh) * 2016-03-10 2018-07-24 浙江生辉照明有限公司 目标跟踪方法和装置
CN109102522B (zh) * 2018-07-13 2021-08-31 北京航空航天大学 一种目标跟踪方法及装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982559A (zh) * 2012-11-28 2013-03-20 大唐移动通信设备有限公司 车辆跟踪方法及系统
US9390506B1 (en) * 2015-05-07 2016-07-12 Aricent Holdings Luxembourg S.A.R.L. Selective object filtering and tracking
CN106096577A (zh) * 2016-06-24 2016-11-09 安徽工业大学 一种摄像头分布地图中的目标追踪系统及追踪方法
CN106682619A (zh) * 2016-12-28 2017-05-17 上海木爷机器人技术有限公司 一种对象跟踪方法及装置
CN109165646A (zh) * 2018-08-16 2019-01-08 北京七鑫易维信息技术有限公司 一种确定图像中用户的感兴趣区域的方法及装置
CN109598234A (zh) * 2018-12-04 2019-04-09 深圳美图创新科技有限公司 关键点检测方法和装置
CN110415208A (zh) * 2019-06-10 2019-11-05 西安电子科技大学 一种自适应目标检测方法及其装置、设备、存储介质

Also Published As

Publication number Publication date
CN112585944A (zh) 2021-03-30

Similar Documents

Publication Publication Date Title
US11205274B2 (en) High-performance visual object tracking for embedded vision systems
KR102615196B1 (ko) 객체 검출 모델 트레이닝 장치 및 방법
CN111627045B (zh) 单镜头下的多行人在线跟踪方法、装置、设备及存储介质
US10140719B2 (en) System and method for enhancing target tracking via detector and tracker fusion for unmanned aerial vehicles
CN111797657A (zh) 车辆周边障碍检测方法、装置、存储介质及电子设备
CN109543641B (zh) 一种实时视频的多目标去重方法、终端设备及存储介质
CN111291650B (zh) 自动泊车辅助的方法及装置
CN110986969B (zh) 地图融合方法及装置、设备、存储介质
JP2009015827A (ja) 物体追跡方法、物体追跡システム、及び物体追跡プログラム
WO2022227771A1 (zh) 目标跟踪方法、装置、设备和介质
CN112419722A (zh) 交通异常事件检测方法、交通管控方法、设备和介质
US11972634B2 (en) Image processing method and apparatus
WO2020107312A1 (zh) 一种刚体配置方法及光学动作捕捉方法
KR102210404B1 (ko) 위치 정보 추출 장치 및 방법
US11080562B1 (en) Key point recognition with uncertainty measurement
Morales et al. Image generation for efficient neural network training in autonomous drone racing
EP3819815A1 (en) Human body recognition method and device, as well as storage medium
WO2020213099A1 (ja) オブジェクト検出・追跡装置、方法、およびプログラム記録媒体
Al-Muteb et al. An autonomous stereovision-based navigation system (ASNS) for mobile robots
WO2021146952A1 (zh) 跟随方法、可移动平台、装置和存储介质
WO2021203368A1 (zh) 图像处理方法、装置、电子设备和存储介质
Abulwafa et al. A fog based ball tracking (FB 2 T) system using intelligent ball bees
CN110651274A (zh) 可移动平台的控制方法、装置和可移动平台
CN115082690B (zh) 目标识别方法、目标识别模型训练方法及装置
US11314968B2 (en) Information processing apparatus, control method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20915860

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20915860

Country of ref document: EP

Kind code of ref document: A1