WO2021217450A1 - Procédé et dispositif de suivi de cible, et support de stockage - Google Patents

Procédé et dispositif de suivi de cible, et support de stockage Download PDF

Info

Publication number
WO2021217450A1
WO2021217450A1 PCT/CN2020/087591 CN2020087591W WO2021217450A1 WO 2021217450 A1 WO2021217450 A1 WO 2021217450A1 CN 2020087591 W CN2020087591 W CN 2020087591W WO 2021217450 A1 WO2021217450 A1 WO 2021217450A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
tracked
similarity
target objects
motion information
Prior art date
Application number
PCT/CN2020/087591
Other languages
English (en)
Chinese (zh)
Inventor
胡晓翔
封旭阳
张李亮
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/087591 priority Critical patent/WO2021217450A1/fr
Priority to CN202080005654.7A priority patent/CN112955712A/zh
Publication of WO2021217450A1 publication Critical patent/WO2021217450A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Definitions

  • This application relates to the technical field of target tracking, in particular to a target tracking method, a target tracking device, a movable platform, an unmanned aerial vehicle, a control system, and a storage medium.
  • UAV tracking and shooting target objects such as tracking and shooting people, cars, animals, etc.
  • UAV tracking and shooting target objects mainly uses the camera to shoot images including the target object, and achieve target tracking by recognizing the image characteristics of the target object in the image.
  • the phenomenon of target switching will occur. Since similar targets are distinguished from image features, it is often necessary to compromise between generalization performance and discrimination performance, and it is difficult to achieve better results. For example, when tracking a vehicle, because there may be two identical cars, it is difficult to distinguish from the image features, which may cause the tracking target to be lost.
  • the present application provides a target tracking method, a target tracking device, a movable platform, an unmanned aerial vehicle, a control system, and a storage medium, so as to avoid the loss of target tracking, thereby improving the accuracy of target tracking.
  • this application provides a target tracking method, which is applied to an unmanned aerial vehicle, and the method includes:
  • the target to be tracked is determined from the multiple target objects according to the motion information, so as to track the target to be tracked.
  • the present application also provides a target tracking device, which is applied to a movable platform, and the flight instruction device includes a processor and a memory;
  • the memory is used to store a computer program
  • the processor is configured to execute the computer program and, when executing the computer program, implement the steps of any one of the target tracking methods provided in the embodiments of the present application.
  • the present application also provides a movable platform, the movable platform including: a camera, a memory, and a processor;
  • the photographing device is used for photographing
  • the memory is used to store a computer program
  • the processor is configured to execute the computer program and, when executing the computer program, implement the steps of any one of the target tracking methods provided in the embodiments of the present application.
  • the present application also provides a control system that includes a control terminal and the movable platform according to any one of the embodiments of the present application, and the control terminal is used to control the operation of the movable platform .
  • the present application also provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes any of the The target tracking method described in the item.
  • a target tracking method, a target tracking device, a movable platform, a drone, a control system, and a storage medium provided by the embodiments of the present application obtain a captured image containing a target to be tracked, and determine the target to be tracked according to the captured image The image characteristics of the target; and then according to the image characteristics of the target to be tracked, the motion information of the target to be tracked is obtained; if the captured image includes multiple target objects, the motion information is selected from the multiple target objects The target to be tracked is determined to track the target to be tracked, thereby avoiding interference from other types of target objects, thereby improving the accuracy of target tracking.
  • Fig. 1 is a schematic structural diagram of a control system provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of determining the location information of a target provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of steps of a target tracking method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a target tracking scene provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a scene including multiple target objects in a captured image provided by an embodiment of the present application
  • Fig. 6 is a schematic diagram of a multi-target tracking scenario provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a scenario in which a target to be tracked is determined from multiple target objects according to an embodiment of the present application
  • FIG. 8 is a schematic diagram of a scenario where the control terminal displays target tracking according to an embodiment of the present application.
  • FIG. 9 is a schematic block diagram of a target tracking device provided by an embodiment of the present application.
  • Fig. 10 is a schematic block diagram of a movable platform provided by an embodiment of the present application.
  • the drone is tracking and shooting target objects, such as tracking and shooting people, cars, animals, etc., mainly by shooting images including the target object through a camera, and achieving target tracking by recognizing the image characteristics of the target object in the image.
  • embodiments of the present application provide a target tracking method, a target tracking device, a movable platform, an unmanned aerial vehicle, a control system, and a storage medium to solve the above-mentioned problems.
  • FIG. 1 is a schematic structural diagram of a control system provided by an embodiment of the present application.
  • the control system 100 includes a control terminal 10 and a drone 20.
  • the control terminal 10 is in communication with the drone 20 and is used to control the flight of the drone 10, and the drone 10 is used to control the target. Tracking and shooting, and using wireless image transmission technology to send the captured images to the control terminal 20 for display for the user to view.
  • the control terminal 10 includes a display device 11, and the display device 11 is used to display an image taken by a drone.
  • the display device 11 includes a display screen provided on the control terminal 10 or a display independent of the control terminal 10.
  • the display independent of the control terminal 10 may include a mobile phone, a tablet computer, a personal computer, etc., or may be a belt Other electronic devices with display screens.
  • the display includes LED display, OLED display, LCD display and so on.
  • the drone 20 includes a photographing device 21, which is used to photograph a photographed image of a target object, and send the photographed image to the drone, and the drone performs target tracking based on the photographed image.
  • a photographing device 21 which is used to photograph a photographed image of a target object, and send the photographed image to the drone, and the drone performs target tracking based on the photographed image.
  • it may include one camera, that is, a monocular shooting solution; it may also include two cameras, that is, a binocular shooting solution.
  • the image of the target object is captured by the imaging device 21, and the speed information of the target object can also be determined according to multiple frames of images, for example, the optical flow method is used to determine the speed of the target object. It is also possible to determine the position and distance of the target object relative to the UAV according to the captured image, and then obtain the position information of the UAV; and determine the position information of the target object according to the position, distance and the position information of the UAV.
  • the location information of the drone includes the coordinates of the drone in the geodetic coordinate system.
  • can, i.e. the position information of the UAV 20 according to the coordinates of the UAV relative to a pedestrian and a coordinate distance D and the azimuth ⁇ 1 20 may determine a pedestrian in a pedestrian.
  • a measuring device is also installed on the drone 20, and the measuring device includes a laser measuring device or an inertial measuring device.
  • the measuring device can be used to measure the distance and position of the target object relative to the drone 20. According to the distance and position of the target object relative to the UAV 20, combined with the coordinates of the UAV, the position information of the target object can be determined. It is also possible to estimate the speed of the target object according to the change in the distance of the target object relative to the UAV 20 at different times and the flying speed of the UAV.
  • the drone 20 may be a rotorcraft.
  • the drone 20 may be a multi-rotor aircraft that may include multiple rotors.
  • the multiple rotors can rotate to generate lifting force for the drone 20.
  • the rotor may be a propulsion unit, which allows the drone 20 to move freely in the air.
  • the rotor can rotate at the same rate and/or can generate the same amount of lift or thrust.
  • the rotor can rotate at different speeds at will, generate different amounts of lifting force or thrust, and/or allow the drone 20 to rotate.
  • one, two, three, four, five, six, seven, eight, nine, ten or more rotors may be provided on the drone 20.
  • These rotors can be arranged such that their rotation axes are parallel to each other.
  • the rotation axis of the rotors may be at any angle with respect to each other, which may affect the movement of the drone 20.
  • the drone 20 may include multiple rotors.
  • the rotor may be connected to the body of the drone 20, and the body of the drone 20 may include a control unit, an inertial measurement unit (IMU), a processor, a battery, a power supply, and/or other sensors.
  • the rotor may be connected to the body by one or more arms or extensions branching from the central part of the body. For example, one or more arms may extend radially from the central body of the drone 20, and may have rotors at or near the end of the arms.
  • the drone may be, for example, a quad-rotor drone, a hexa-rotor drone, or an eight-rotor drone.
  • it can also be a fixed-wing UAV, or a combination of a rotary-wing type and a fixed-wing UAV, which is not limited here.
  • control system in FIG. 1 is only used to explain the target tracking method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the flight control method provided in the embodiment of the present application.
  • FIG. 3 is a schematic flowchart of steps of a target tracking method provided by an embodiment of the present application.
  • the target tracking method is applied to UAVs. Even if multiple similar targets appear in the captured images during the tracking process, it can ensure that the UAV will not lose tracking, thereby improving the accuracy of target tracking.
  • the target tracking method includes steps S101 to S103.
  • S102 Acquire motion information of the target to be tracked according to the image feature of the target to be tracked;
  • S103 If the captured image includes multiple target objects, determine the target to be tracked from the multiple target objects according to the motion information, so as to track the target to be tracked.
  • the captured image When tracking a target to be tracked, it is necessary to obtain a captured image containing the target to be tracked. Specifically, an image of the area where the target to be tracked is located can be captured by a photographing device to obtain a captured image containing the target to be tracked, and the captured image is determined. The image characteristics of the target to be tracked are described so as to track the target to be tracked according to the image characteristics. It should be noted that the captured image includes multiple frames of images containing the target to be tracked. Specifically, the captured image is a video captured during target tracking in practical applications.
  • the target to be tracked can be tracked based on the target tracking algorithm according to the image characteristics of the target to be tracked in the captured image.
  • the influence of a target object similar to the tracking target often occurs, and there may also be occlusion and interleaving, which will cause the tracking of the target to be tracked to be lost.
  • the target tracking algorithm includes any one of a mean shift algorithm, a Kalman filter algorithm, a particle filter algorithm, and a moving target modeling algorithm. In some other embodiments, other target tracking algorithms can also be used, which is not limited here.
  • the drone 20 uses the imaging device 21 to photograph the target to be tracked (vehicle a) to obtain a photographed image including the vehicle a.
  • the drone 20 obtains the photographed image containing the vehicle a, and tracks and photographs the vehicle a according to the photographed image, and sends the tracked photographed image to the control terminal 10 for display, and displays it on the positioned target to be tracked
  • the tracking mark is the positioning frame of the vehicle displayed on the display device 11 of the control terminal 10 in FIG. 4.
  • the situation where the tracking of the target to be tracked is lost includes: it is impossible to distinguish which vehicle is the target to be tracked or the wrong target object is tracked.
  • the image characteristics of the target to be tracked may be used to obtain the The motion information of the target to be tracked.
  • the captured image includes a plurality of target objects
  • the target to be tracked is determined from the plurality of target objects according to the motion information to track the target to be tracked, which can overcome tracking only based on image information
  • the problem of mis-tracking and tracking loss caused by, for example, avoiding the problem of mis-tracking caused by the indistinguishability of two targets with the same or similar image features, thereby improving the accuracy of target tracking.
  • the motion information includes speed information and/or position information
  • the speed information is the speed and/or direction of the target to be tracked
  • the position information is the coordinates of the target to be tracked, which may specifically be coordinates in a geodetic coordinate system.
  • the final tracking target can be determined from the four vehicles according to the motion information of the target to be tracked (vehicle a) .
  • different target objects may be of similar or identical types, such as pedestrians of almost the same height and fat or thin, or vehicles of similar models, or identical vehicles, the image features of these target objects are relatively similar, so the image features It may be indistinguishable when tracking.
  • the motion information corresponding to different target objects is likely to be different. Therefore, in the target tracking process, when multiple target objects appear, the motion information is used to determine the target to be tracked, thereby improving the accuracy of target tracking.
  • the movement information of the target to be tracked is obtained according to the image feature, and the speed or position of the target to be tracked can be obtained by image processing.
  • the measurement device can also be used for measurement, or other methods can also be used here. Not limited.
  • the tracking of the target to be tracked may be lost. Therefore, before the target to be tracked is not lost, the image features and motion information of multiple target objects can be determined first, and then the motion information of the target to be tracked can be determined according to the image features and motion information of multiple target objects, so that When the target to be tracked is not lost, the target to be tracked is determined from multiple target objects according to the motion information of the target to be tracked. As a result, the accuracy of determining the target to be tracked is improved, thereby improving the accuracy of target tracking.
  • the captured image before the target to be tracked is not lost is the first few frames of the captured image when the target to be tracked is lost. Therefore, the motion information corresponding to the same target object does not change much, so this method is used to determine the target to be tracked. Motion information can improve the accuracy of determining the target to be tracked.
  • the determining the image features and motion information of the multiple target objects may specifically be: identifying the image features corresponding to the multiple target objects in the captured image, and corresponding to the multiple target objects. To determine the position of each of the target objects in the captured image according to the image features of the In the process of multi-target tracking, the motion information of multiple target objects is determined. Using a multi-target tracking method to determine the motion information of the multiple target objects can improve the accuracy of the motion information of the multiple target objects.
  • the image feature corresponding to each vehicle in the captured image is identified, the location of each vehicle in the captured image is determined according to the image feature corresponding to each vehicle, and the location of each vehicle in the captured image is determined for multiple vehicles.
  • the target object is multi-target tracking, specifically as shown in Figure 6, multi-target tracking is performed on four vehicles at the same time, and the motion information of each vehicle is determined in the process of multi-target tracking.
  • the motion information can also be obtained through multi-frame images.
  • the optical flow method is used to determine, of course, other methods can also be used, which are not limited here.
  • the image characteristics of the target to be tracked and the image characteristics of the target object can be selected from multiple
  • the target object that matches the target to be tracked is determined among the target objects; the motion information of the target object that matches the target to be tracked is used as the motion information of the target to be tracked.
  • the target to be tracked is determined from the plurality of target objects according to the motion information, specifically, the motion information of the target to be tracked is used as a distinction to determine the target to be tracked from the plurality of target objects Target.
  • the vehicle 4 is The target to be tracked; or, by comparing image features and motion information, determine the target to be tracked from the four vehicles, specifically determining the target (vehicle a) similar to the target to be tracked (vehicle a) from the four vehicles according to the image characteristics and motion information
  • a vehicle that is, a vehicle whose image features and motion information corresponding to the four vehicles are relatively close to the target to be tracked (vehicle a), such as vehicle 4, is determined to be the target to be tracked, and the vehicle 4 is determined to be the target to be tracked. Perform tracking and no longer track other vehicles (vehicle 1, vehicle 2, and vehicle 3).
  • the determining the target to be tracked from the plurality of target objects according to the motion information is specifically: calculating the target to be tracked according to the motion information of the target to be tracked and the motion information of the plurality of target objects
  • the similarity between the target and the multiple target objects; the target to be tracked is determined from the multiple target objects according to the similarity.
  • the calculation of the similarity between the target to be tracked and the plurality of target objects according to the motion information of the target to be tracked and the motion information of the plurality of target objects is specifically that when target tracking occurs When the target is lost, calculate the similarity between the motion information of the target to be tracked before the target tracking is lost and the motion information of the current multiple target objects, and determine the target object with the highest similarity among the multiple target objects as the The target to be tracked, and the motion information of the target to be tracked is updated according to the motion information of the target object with the highest similarity; Similarity calculation is performed on the motion information of the target object, the target object with the highest similarity among the multiple target objects is determined as the target to be tracked, and the target object to be tracked is updated according to the motion information of the target object with the highest similarity Movement information of the target.
  • the motion information of the target to be tracked is updated, the motion information of the target to be tracked and the image feature of the target to be tracked are fused or correlated to realize the tracking of the target to be tracked.
  • the similarity between the target to be tracked and each target object is calculated to obtain multiple similarities, and each similarity corresponds to a target object, the highest similarity is determined from the multiple similarities, and the highest similarity is determined.
  • the target object corresponding to the degree is used as the target to be tracked.
  • the determining the target to be tracked from the plurality of target objects according to the similarity specifically includes: determining the target to be tracked from the plurality of target objects according to the position similarity; Alternatively, the target to be tracked is determined from the plurality of target objects according to the speed similarity; or, the target to be tracked is determined from the plurality of target objects according to the position similarity and speed similarity. Track the target.
  • the target to be tracked is determined from the plurality of target objects, and the maximum sum corresponding to the maximum sum can be determined according to the sum of the position similarity and the speed similarity.
  • the target object is the target to be tracked.
  • the determining the target to be tracked from the plurality of target objects according to the similarity is specifically: obtaining a preset weight corresponding to the position similarity, and obtaining a preset weight corresponding to the speed similarity;
  • the target to be tracked is determined from a plurality of target objects according to the position similarity, the speed similarity, and the corresponding preset weight.
  • the product of the position similarity and the preset weight corresponding to the position similarity is calculated, and the product of the speed similarity and the preset weight corresponding to the speed similarity is calculated; and then the two products are calculated Sum, taking the sum of the two products as the final similarity between the target to be tracked and the target object, and determining the target object corresponding to the largest final similarity as the target to be tracked.
  • the size of the influence between the position information and the speed information can be adjusted by using the preset weight, so the target to be tracked can be determined more accurately.
  • the preset weight corresponding to the position similarity is less than the preset weight corresponding to the speed similarity. Since the position information has small changes in the images taken before and after the frames, while the speed information is basically unchanged in the images taken in the previous and next frames, by setting different weight ratios to increase the proportion of speed similarity, the determination of the tracking target can be improved. Accuracy.
  • the target to be tracked is determined from a plurality of the target objects according to the motion information, so as to compare the The target to be tracked is tracked. For example, the same vehicles, or the same pedestrians. Because it is easy to lose track in targets of the same type.
  • the target to be tracked may be determined from a plurality of target objects in combination with the image characteristics of the target to be tracked. That is, the determining the target to be tracked from the plurality of target objects according to the similarity is specifically: determining from the plurality of target objects according to the similarity and the image characteristics of the target to be tracked The target to be tracked.
  • the similarity is the similarity corresponding to the motion information, that is, from a plurality of the target objects, the target object with the highest similarity to the target to be tracked and the closest image feature is found as the target to be tracked.
  • the determining the target to be tracked from the plurality of target objects according to the similarity and the image characteristics of the target to be tracked is specifically: determining from the plurality of target objects according to the image characteristics A target object similar to the target to be tracked; the target to be tracked is determined from target objects similar to the target to be tracked according to the similarity.
  • one of the vehicle 3 and the vehicle 4 is the target to be tracked, and the two models are the same. Therefore, first, according to the image characteristics of the target to be tracked and the image characteristics of multiple target objects, the target objects similar to the target to be tracked are determined as the vehicle 3 and the vehicle 4. Then in the slave vehicle 3 and the vehicle 4, according to the vehicle with a higher similarity corresponding to the movement information of the vehicle 3 and the vehicle 4, the vehicle 4 is finally determined as the target to be tracked if the similarity of the vehicle 4 is higher. After excluding vehicle 1 and vehicle 2 according to the image characteristics, the final target to be tracked is determined according to the motion information, so that the target to be tracked can be determined quickly and accurately.
  • the image features involved in the above embodiments are the features of the target to be tracked or the target object extracted from the captured image using image recognition technology, and the image feature includes the corresponding color feature and distribution location feature of the target in the captured image. , Texture feature, contour feature one or more.
  • determining the target to be tracked from the plurality of target objects according to the similarity is specifically: determining the target from the plurality of target objects according to the similarity and the Reid feature of the target to be tracked The target to be tracked; wherein the Reid feature is a feature that uses pedestrian re-recognition technology to identify the target to be tracked from the captured image.
  • the target tracking method before the determining the target to be tracked from the plurality of target objects according to the motion information, the target tracking method further includes: according to the image feature of the target to be tracked and a plurality of The image feature of the target object, determining whether there is a target object similar to the target to be tracked among the plurality of target objects; if there are at least two target objects similar to the target to be tracked among the plurality of target objects , The target to be tracked is determined from a plurality of target objects according to the motion information. If there is only one target object similar to the target to be tracked among the plurality of target objects, even if the target tracking is lost, the target to be tracked can be tracked again from the perspective of image feature recognition.
  • Reid features when performing target detection on the captured image and determining that there are multiple target objects in the captured image, Reid features may also be extracted from the captured image, so that the multiple target objects can be
  • multi-target tracking is performed on a plurality of the target objects according to the Reid feature.
  • the pedestrian re-recognition technology is used to extract the Reid features of the multiple target objects from the captured image, and the Reid feature can compensate for the visual limitation of the shooting device, thereby improving the accuracy of multi-target tracking.
  • the target to be tracked is determined from a plurality of target objects according to the motion information.
  • the target to be tracked may be determined from a plurality of target objects according to the image feature, the Reid feature, and the image feature of the motion information.
  • the similarity can be determined based on the image characteristics of the target to be tracked, the Reid characteristics of the target to be tracked, and the image characteristics and Reid characteristics of multiple target objects, and the motion information of the target to be tracked. , Determine the final target object from multiple target objects as the target to be tracked for tracking. This can improve the accuracy of target tracking.
  • the image feature of the target to be tracked the Reid feature of the target to be tracked, the image features of the multiple target objects, and the Reid feature of the target to be tracked.
  • the similarity is determined according to the motion information from the similar target objects, and the final target object is determined as the target to be tracked for tracking according to the similarity. This can improve the accuracy of target tracking.
  • the captured image containing the target to be tracked may also be sent to the control terminal, so that the control terminal displays the captured image and the target to be tracked in the positioning.
  • the tracking logo is displayed on the display.
  • the captured image displayed by the control terminal may include multiple target objects, one of which is a target to be tracked determined according to the target tracking method, in the displayed captured image
  • a plurality of target objects appear, among which the target object located in the positioning frame is the target to be tracked, and the target to be tracked is determined according to the target tracking method provided in the embodiment of the present application.
  • FIG. 9 is a schematic block diagram of a target tracking device provided by an embodiment of the present application.
  • the target tracking device can be applied to a movable platform for tracking the target.
  • the target tracking device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.
  • a bus such as an I2C (Inter-integrated Circuit) bus.
  • movable platforms include drones, robots or unmanned vehicles.
  • the processor 301 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.
  • MCU micro-controller unit
  • CPU central processing unit
  • DSP Digital Signal Processor
  • the memory 302 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.
  • the processor is configured to run a computer program stored in a memory, and implement any one of the target tracking methods provided in the embodiments of the present application when the computer program is executed.
  • the processor is configured to run a computer program stored in a memory, and implement the following steps when the computer program is executed:
  • the type of the target to be tracked is the same as the type of the target object.
  • the acquiring the motion information of the target to be tracked according to the image characteristics of the target to be tracked includes:
  • the determining image features and motion information of a plurality of the target objects includes:
  • Identify the image features corresponding to multiple target objects in the captured image and determine the position of each target object in the captured image according to the image features corresponding to the multiple target objects; according to each target object Multi-target tracking is performed on a plurality of the target objects at the positions in the captured image; and in the process of performing the multi-target tracking, the motion information of the multiple target objects is determined.
  • the determining the target to be tracked from the plurality of target objects according to the motion information includes:
  • the motion information includes position information and/or speed information; correspondingly, the similarity includes position similarity and/or speed similarity.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of the target objects.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the preset weight corresponding to the position similarity is less than the preset weight corresponding to the speed similarity.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of the target objects.
  • the determining the target to be tracked from a plurality of target objects according to the similarity and the image feature of the target to be tracked includes:
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of target objects; wherein, the Reid feature is to identify all the objects from the captured image using a pedestrian re-recognition technology. Describe the characteristics of the target to be tracked.
  • the processor before the determining the target to be tracked from the plurality of target objects according to the motion information, the processor further implements:
  • the image characteristics of the target to be tracked and the image characteristics of the plurality of target objects determine whether there is a target object similar to the target to be tracked among the plurality of target objects; if there is a plurality of target objects If at least two target objects are similar to the target to be tracked, the target to be tracked is determined from a plurality of target objects according to the motion information.
  • the processor is further configured to: extract Reid features from the captured image;
  • said performing multi-target tracking on a plurality of the target objects further includes: performing multi-target tracking on a plurality of the target objects according to the Reid feature.
  • the determining the target to be tracked from the plurality of target objects according to the motion information includes:
  • the target to be tracked is determined from a plurality of target objects.
  • the processor further implements:
  • the target to be tracked is tracked according to the captured image.
  • the target tracking algorithm includes any one of a mean shift algorithm, a Kalman filter algorithm, a particle filter algorithm, and a moving target modeling algorithm.
  • the movable platform includes at least one camera, and/or the movable platform includes a measuring device;
  • the photographing device is used for photographing images
  • the measuring device is used for measuring the position and distance of the tracking target relative to the movable platform.
  • the photographing device includes one camera or two cameras; the measurement device includes a laser measurement device or an inertial measurement device.
  • the processor is further configured to:
  • the photographed image containing the target to be tracked is sent to the control terminal, so that the control terminal displays the photographed image, and displays a tracking mark on the located target to be tracked.
  • FIG. 10 is a schematic block diagram of a movable platform provided by an embodiment of the present application.
  • the movable platform 400 is used to track the target.
  • the mobile platform 400 includes a processor 401, a memory 402, and a camera 403.
  • the memory 302 and the camera 403 are connected by a bus processor 401, such as an I2C (Inter-integrated Circuit) bus.
  • I2C Inter-integrated Circuit
  • the photographing device 403 is used to photograph an image of the area where the target to be tracked is located, so as to obtain a photographed image containing the target to be tracked.
  • movable platforms include drones, robots or unmanned vehicles.
  • the processor 401 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.
  • MCU micro-controller unit
  • CPU central processing unit
  • DSP Digital Signal Processor
  • the memory 402 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.
  • the processor is configured to run a computer program stored in a memory, and implement any one of the target tracking methods provided in the embodiments of the present application when the computer program is executed.
  • the processor is configured to run a computer program stored in a memory, and implement the following steps when the computer program is executed:
  • the type of the target to be tracked is the same as the type of the target object.
  • the acquiring the motion information of the target to be tracked according to the image characteristics of the target to be tracked includes:
  • the determining image features and motion information of a plurality of the target objects includes:
  • Identify the image features corresponding to multiple target objects in the captured image and determine the position of each target object in the captured image according to the image features corresponding to the multiple target objects; according to each target object Multi-target tracking is performed on a plurality of the target objects at the positions in the captured image; and in the process of performing the multi-target tracking, the motion information of the multiple target objects is determined.
  • the determining the target to be tracked from the plurality of target objects according to the motion information includes:
  • the motion information includes position information and/or speed information; correspondingly, the similarity includes position similarity and/or speed similarity.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of the target objects.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the preset weight corresponding to the position similarity is less than the preset weight corresponding to the speed similarity.
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of the target objects.
  • the determining the target to be tracked from a plurality of target objects according to the similarity and the image feature of the target to be tracked includes:
  • the determining the target to be tracked from the plurality of target objects according to the similarity includes:
  • the target to be tracked is determined from a plurality of target objects; wherein, the Reid feature is the recognition of all the objects from the captured image using pedestrian re-recognition technology. Describe the characteristics of the target to be tracked.
  • the processor before the determining the target to be tracked from the plurality of target objects according to the motion information, the processor further implements:
  • the image characteristics of the target to be tracked and the image characteristics of the plurality of target objects determine whether there is a target object similar to the target to be tracked among the plurality of target objects; if there is a plurality of target objects If at least two target objects are similar to the target to be tracked, the target to be tracked is determined from a plurality of target objects according to the motion information.
  • the processor is further configured to: extract Reid features from the captured image;
  • said performing multi-target tracking on a plurality of the target objects further includes: performing multi-target tracking on a plurality of the target objects according to the Reid feature.
  • the determining the target to be tracked from the plurality of target objects according to the motion information includes:
  • the target to be tracked is determined from a plurality of target objects.
  • the processor further implements:
  • the target to be tracked is tracked according to the captured image.
  • the target tracking algorithm includes any one of a mean shift algorithm, a Kalman filter algorithm, a particle filter algorithm, and a moving target modeling algorithm.
  • the movable platform includes at least one camera, and/or the movable platform includes a measuring device;
  • the photographing device is used for photographing images
  • the measuring device is used for measuring the position and distance of the tracking target relative to the movable platform.
  • the photographing device includes one camera or two cameras; the measurement device includes a laser measurement device or an inertial measurement device.
  • the processor is further configured to:
  • the photographed image containing the target to be tracked is sent to the control terminal, so that the control terminal displays the photographed image, and displays a tracking mark on the located target to be tracked.
  • control system includes the movable platform and the control terminal according to any one of the above-mentioned embodiments of the claims, and the control terminal is used to control the operation of the movable platform .
  • control system 100 includes a drone 20 and a control terminal 10.
  • the control terminal 10 is used to control the flight of the drone 20 for tracking the target.
  • the embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation Examples provide the steps of any of the target tracking methods.
  • the computer-readable storage medium may be the internal storage unit of the removable platform described in any of the foregoing embodiments, for example, the memory or internal memory of the removable platform.
  • the computer-readable storage medium may also be an external storage device of the removable platform, such as a plug-in hard disk equipped on the removable platform, a smart memory card (Smart Media Card, SMC), and Secure Digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Image Analysis (AREA)

Abstract

Dispositif et procédé de suivi de cible, et support de stockage. Le procédé consiste : à obtenir une image capturée contenant une cible à suivre, et à déterminer une caractéristique d'image de ladite cible (S101) ; à obtenir des informations de mouvement de ladite cible en fonction de la caractéristique d'image de ladite cible (S102) ; si l'image capturée comprend de multiples objets cibles, à déterminer la cible à suivre parmi les multiples objets cibles en fonction des informations de mouvement de façon à suivre la cible à suivre (S103).
PCT/CN2020/087591 2020-04-28 2020-04-28 Procédé et dispositif de suivi de cible, et support de stockage WO2021217450A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/087591 WO2021217450A1 (fr) 2020-04-28 2020-04-28 Procédé et dispositif de suivi de cible, et support de stockage
CN202080005654.7A CN112955712A (zh) 2020-04-28 2020-04-28 目标跟踪方法、设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/087591 WO2021217450A1 (fr) 2020-04-28 2020-04-28 Procédé et dispositif de suivi de cible, et support de stockage

Publications (1)

Publication Number Publication Date
WO2021217450A1 true WO2021217450A1 (fr) 2021-11-04

Family

ID=76236219

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/087591 WO2021217450A1 (fr) 2020-04-28 2020-04-28 Procédé et dispositif de suivi de cible, et support de stockage

Country Status (2)

Country Link
CN (1) CN112955712A (fr)
WO (1) WO2021217450A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113766130B (zh) * 2021-09-13 2023-07-28 维沃移动通信有限公司 视频拍摄方法、电子设备及装置
CN115861860B (zh) * 2023-02-24 2023-06-02 杭州靖安科技有限公司 一种无人机的目标跟踪定位方法和系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8971575B2 (en) * 2012-11-23 2015-03-03 Cyberlink Corp. Systems and methods for tracking objects
CN106651908A (zh) * 2016-10-13 2017-05-10 北京科技大学 一种多运动目标跟踪方法
CN108446622A (zh) * 2018-03-14 2018-08-24 海信集团有限公司 目标物体的检测跟踪方法及装置、终端
CN109446942A (zh) * 2018-10-12 2019-03-08 北京旷视科技有限公司 目标跟踪方法、装置和系统
CN110414443A (zh) * 2019-07-31 2019-11-05 苏州市科远软件技术开发有限公司 一种目标跟踪方法、装置及枪球联动跟踪方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH089227A (ja) * 1994-06-17 1996-01-12 Hitachi Ltd 自動追尾機能を有する撮像装置
JP4373840B2 (ja) * 2004-04-21 2009-11-25 日本電信電話株式会社 動物体追跡方法、動物体追跡プログラムおよびその記録媒体、ならびに、動物体追跡装置
CN104794733B (zh) * 2014-01-20 2018-05-08 株式会社理光 对象跟踪方法和装置
CN105120146B (zh) * 2015-08-05 2018-06-26 普宙飞行器科技（深圳）有限公司 一种利用无人机进行运动物体自动锁定拍摄装置及拍摄方法
CN105578034A (zh) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 一种对目标进行跟踪拍摄的控制方法、控制装置及系统
WO2018214093A1 (fr) * 2017-05-25 2018-11-29 深圳市大疆创新科技有限公司 Procédé et appareil de suivi
CN107505951B (zh) * 2017-08-29 2020-08-21 深圳市道通智能航空技术有限公司 一种目标跟踪方法、无人机和计算机可读存储介质
JP6412998B1 (ja) * 2017-09-29 2018-10-24 株式会社Ｑｏｎｃｅｐｔ 動体追跡装置、動体追跡方法、動体追跡プログラム
JP6932098B2 (ja) * 2018-03-19 2021-09-08 三菱電機株式会社 追尾装置
CN109325961B (zh) * 2018-08-27 2021-07-09 北京悦图数据科技发展有限公司 无人机视频多目标跟踪方法及装置
CN109472258A (zh) * 2018-09-27 2019-03-15 易瓦特科技股份公司 跟踪方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8971575B2 (en) * 2012-11-23 2015-03-03 Cyberlink Corp. Systems and methods for tracking objects
CN106651908A (zh) * 2016-10-13 2017-05-10 北京科技大学 一种多运动目标跟踪方法
CN108446622A (zh) * 2018-03-14 2018-08-24 海信集团有限公司 目标物体的检测跟踪方法及装置、终端
CN109446942A (zh) * 2018-10-12 2019-03-08 北京旷视科技有限公司 目标跟踪方法、装置和系统
CN110414443A (zh) * 2019-07-31 2019-11-05 苏州市科远软件技术开发有限公司 一种目标跟踪方法、装置及枪球联动跟踪方法

Also Published As

Publication number Publication date
CN112955712A (zh) 2021-06-11

Similar Documents

Publication Publication Date Title
US20220337798A1 (en) Detecting Optical Discrepancies In Captured Images
WO2020113423A1 (fr) Procédé et système de reconstruction tridimensionnelle de scène cible et véhicule aérien sans pilote
WO2019104571A1 (fr) Procédé et dispositif de traitement d'image
WO2022021027A1 (fr) Procédé et appareil de suivi de cible, véhicule aérien sans pilote, système et support de stockage lisible associés
WO2020014909A1 (fr) Procédé et dispositif de photographie, et véhicule aérien sans pilote
CN112567201A (zh) 距离测量方法以及设备
US11057604B2 (en) Image processing method and device
TWI693959B (zh) 控制裝置、飛行體以及記錄媒體
JP2018526641A (ja) レーザ深度マップサンプリングのためのシステム及び方法
CN108759826B (zh) 一种基于手机和无人机多传感参数融合的无人机运动跟踪方法
Grabe et al. Robust optical-flow based self-motion estimation for a quadrotor UAV
WO2021217450A1 (fr) Procédé et dispositif de suivi de cible, et support de stockage
WO2021081774A1 (fr) Procédé et appareil d'optimisation de paramètres, dispositif de commande et aéronef
Eynard et al. Real time UAV altitude, attitude and motion estimation from hybrid stereovision
CN102538782A (zh) 一种基于计算机视觉的直升机着陆引导装置和方法
CN109754420B (zh) 一种目标距离估计方法、装置及无人机
WO2021081958A1 (fr) Procédé de détection de terrain, plateforme mobile, dispositif de commande, système et support de stockage
CN112116651A (zh) 一种基于无人机单目视觉的地面目标定位方法和系统
US20210097696A1 (en) Motion estimation methods and mobile devices
WO2020024182A1 (fr) Procédé et appareil de traitement de paramètre, dispositif de caméra et aéronef
Zhang et al. The use of optical flow for UAV motion estimation in indoor environment
WO2020019175A1 (fr) Procédé et dispositif de traitement d'image et dispositif photographique et véhicule aérien sans pilote
WO2020113417A1 (fr) Procédé et système de reconstruction tridimensionnelle d'une scène cible, et véhicule aérien sans pilote
WO2022021028A1 (fr) Procédé de détection de cible, dispositif, aéronef sans pilote et support de stockage lisible par ordinateur
WO2021035746A1 (fr) Procédé et dispositif de traitement d'image, et plate-forme mobile

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20934068

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20934068

Country of ref document: EP

Kind code of ref document: A1