WO2021083151A1 - Procédé et appareil de détection de cible, support de stockage et véhicule aérien sans pilote - Google Patents

Procédé et appareil de détection de cible, support de stockage et véhicule aérien sans pilote Download PDF

Info

Publication number
WO2021083151A1
WO2021083151A1 PCT/CN2020/124055 CN2020124055W WO2021083151A1 WO 2021083151 A1 WO2021083151 A1 WO 2021083151A1 CN 2020124055 W CN2020124055 W CN 2020124055W WO 2021083151 A1 WO2021083151 A1 WO 2021083151A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
sub
rectangular frame
boundary
Prior art date
Application number
PCT/CN2020/124055
Other languages
English (en)
Chinese (zh)
Inventor
李亚学
Original Assignee
深圳市道通智能航空技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术股份有限公司 filed Critical 深圳市道通智能航空技术股份有限公司
Publication of WO2021083151A1 publication Critical patent/WO2021083151A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the embodiments of the present invention relate to the technical field of drones, and in particular to target detection methods, devices, storage media, and drones.
  • Unmanned Aerial Vehicles refer to unmanned aircraft operated by radio remote control equipment and independent program control equipment, or fully or intermittently autonomously operated by onboard computers. Compared with manned aircraft, unmanned aerial vehicles have the advantages of small size, low cost, low environmental requirements, and strong survivability, and are often more suitable for tasks in dangerous or harsh environments. With the rapid development of the UAV manufacturing industry, UAV systems are widely used in areas such as smart city management and intelligent traffic monitoring. Among them, target detection is a basic but challenging functional requirement in UAV systems, which is closely related to applications such as infrastructure inspection, city perception, map reconstruction, and traffic control. These applications have promoted the development of online monitoring systems based on drones. These online systems can perform various tasks, such as on-site facility inspections and detection of violations, identification of unhealthy crops, and acquisition of map data.
  • the UAV is generally integrated with an image processing chip, and the image analysis unit in the image processing chip analyzes and processes the images taken by the PTZ camera on the UAV to achieve target detection.
  • the resolution of the input image of the image processing chip is generally low, and the resolution of the pan/tilt camera is very high. It is necessary to resize the image taken by the pan/tilt camera before performing the resize operation. Handed over to the image processing chip for processing, which makes part of the image information lost in the image processing process, making the detection of some small targets very difficult or even impossible. Therefore, the existing UAV target detection scheme needs to be improved.
  • the embodiments of the present invention provide a target detection method, device, storage medium, and equipment, which can optimize the existing target detection scheme.
  • an embodiment of the present invention provides a target detection method, which is applied to an unmanned aerial vehicle, in which at least two image analysis units are integrated, and the method includes:
  • the at least two sub-images are input into the at least two image analysis units, and the target detection result is determined according to the analysis results of the at least two image analysis units.
  • an embodiment of the present invention provides a target detection device applied to an unmanned aerial vehicle. At least two image analysis units are integrated in the unmanned aerial vehicle, and the device includes:
  • the image acquisition module is used to acquire the first image taken by the pan-tilt camera
  • An image segmentation module configured to perform segmentation processing on the first image to obtain at least two segmented images
  • a size changing module configured to perform a size changing operation on the at least two divided images to obtain at least two sub-images, and the resolutions of the at least two sub-images match the resolutions corresponding to the at least two image analysis units;
  • the target detection module is configured to input the at least two sub-images into the at least two image analysis units, and determine the target detection result according to the analysis results of the at least two image analysis units.
  • an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the target detection method as provided in the embodiment of the present invention is implemented.
  • an embodiment of the present invention provides an unmanned aerial vehicle, including a memory, at least two image analysis units, a processor, and a computer program stored in the memory and running on the processor, wherein the When the processor executes the computer program, the target detection method as provided in the embodiment of the present invention is implemented.
  • the target detection solution provided in the embodiment of the present invention is applied to an unmanned aerial vehicle.
  • the unmanned aerial vehicle is integrated with at least two image analysis units to obtain the first image taken by the pan/tilt camera, and perform segmentation processing on the first image to determine at least Perform a size change operation on two divided images, and the resolutions of the obtained at least two sub-images match the resolutions corresponding to the at least two image analysis units.
  • the at least two sub-images are input into the at least two image analysis units, and the The analysis results of the two image analysis units determine the target detection result.
  • the loss of image information can be reduced when the size is changed, and then it can be analyzed by at least two image analysis units, which can improve the target detection Accuracy and success rate.
  • FIG. 1 is a schematic flowchart of a target detection method according to Embodiment 1 of the present invention
  • FIG. 2 is a schematic diagram of an image taken by a pan-tilt camera according to Embodiment 1 of the present invention
  • FIG. 3 is a schematic diagram of an image that has undergone size modification processing according to Embodiment 1 of the present invention.
  • FIG. 4 is a schematic diagram of a left side image after segmentation according to Embodiment 1 of the present invention.
  • FIG. 5 is a schematic diagram of a right side image after segmentation according to Embodiment 1 of the present invention.
  • FIG. 6 is a schematic diagram of a left side image after a size change according to Embodiment 1 of the present invention.
  • FIG. 7 is a schematic diagram of a right side image after a size change according to Embodiment 1 of the present invention.
  • FIG. 8 is a schematic flowchart of a target detection method according to Embodiment 2 of the present invention.
  • FIG. 9 is a schematic flowchart of a target detection method according to Embodiment 3 of the present invention.
  • FIG. 10 is a schematic diagram of a left side image including position information according to Embodiment 3 of the present invention.
  • FIG. 11 is a schematic diagram of a right image including position information according to Embodiment 3 of the present invention.
  • FIG. 12 is a schematic diagram of fusion of analysis results according to Embodiment 3 of the present invention.
  • FIG. 13 is a structural block diagram of a target detection device provided by Embodiment 4 of the present invention.
  • FIG. 14 is a structural block diagram of an unmanned aerial vehicle according to Embodiment 6 of the present invention.
  • Fig. 1 is a schematic flow chart of a target detection method provided by Embodiment 1 of the present invention.
  • the method can be executed by a target detection device, where the device can be implemented by software and/or hardware, and generally can be integrated in a drone.
  • the method includes:
  • Step 101 Obtain a first image taken by a pan-tilt camera.
  • the pan/tilt camera can be integrated inside the drone, or externally placed on the drone, to establish a connection with the drone through wired or wireless methods.
  • the gimbal camera can collect images in real time during the flight of the drone, and can obtain the images taken by the gimbal camera in real time or at a preset frequency.
  • the first image may be an image taken at any time, which is not limited in the embodiment of the present invention.
  • At least two image analysis units are integrated in the drone, and the specific type of the image analysis unit is not limited.
  • it may be a forward inference engine (Neural Network Inference Engine, NNIE) that is accelerated based on a neural network. ).
  • the image analysis unit can be integrated in an image processing chip, such as the HI3559C chip, which carries two forward reasoners specifically aimed at the acceleration of neural networks. The two forward reasoners can handle tasks such as detection and classification independently.
  • Step 102 Perform segmentation processing on the first image to obtain at least two segmented images.
  • the resolution of the current pan/tilt camera can reach very high, such as 8K, and the ratio of the captured image is generally 4:3 or 16:9.
  • the resolution of the input image of the image analysis unit is generally low, such as 512*512. In this way, when the image taken by the pan/tilt camera is resized, it will lose more Image information reduces the accuracy of target detection, and it is especially difficult to detect small targets.
  • targets on the ground are smaller in the image and more difficult to detect.
  • FIG. 2 is a schematic diagram of an image taken by a pan-tilt camera according to Embodiment 1 of the present invention.
  • the image ratio is 16:9 and the resolution is 1920*1080.
  • Fig. 3 is a schematic diagram of an image that has undergone a resizing process according to the first embodiment of the present invention. After resize, the image in Fig. 2 becomes the 1:1 image in Fig. 3, with a resolution of 512*512, so , Most of the information in Figure 2 is lost, resulting in unsatisfactory detection of small targets.
  • the first image may be segmented according to a preset rule to obtain at least two segmented images.
  • the preset rules can include specific number of divisions and division methods, such as equal division or proportional division, etc., such as the division of left and right structures, the division of upper and lower structures, the division of Tian shape, and the division of Jiugong format.
  • the specific number of segmentation and the segmentation method can be determined according to the number of image analysis units and other reference factors, which are not limited in the embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a divided left image according to Embodiment 1 of the present invention
  • FIG. 5 is a schematic diagram of a divided right image according to Embodiment 1 of the present invention. Referring to FIG. 4 and FIG. The image in 2 is divided equally between the left and right structure, and two divided images with an image ratio of 8:9 are obtained.
  • Step 103 Perform a size change operation on the at least two divided images to obtain at least two sub-images, and the resolutions of the at least two sub-images match the resolutions corresponding to the at least two image analysis units.
  • the specific manner of performing the resizing operation on the segmented image is not limited, and the purpose of the resizing operation is to match the resolutions of at least two sub-images with the resolutions corresponding to at least two image analysis units.
  • the resolutions corresponding to at least two image analysis units are equal, so that the resolutions of at least two sub-images are the same as the resolution.
  • the size change operation may include a size change operation performed by an interpolation algorithm.
  • the interpolation algorithm may be, for example, the nearest neighbor method, the bilinear method, the bicubic method, the algorithm based on the pixel region relationship, and the Lanzos interpolation method.
  • FIG. 6 is a schematic diagram of a left side image after a size change provided in the first embodiment of the present invention
  • FIG. 7 is a schematic diagram of a right side image after a size change provided in the first embodiment of the present invention.
  • the size change operation is performed on the images in FIGS. 5 and 6 respectively, and two images with an image ratio of 1:1 and a resolution of 512*512 are obtained.
  • Step 104 Input the at least two sub-images into the at least two image analysis units, and determine a target detection result according to the analysis results of the at least two image analysis units.
  • the total area of the image input to the image analysis unit becomes larger. Referring to the above example, it is equivalent to double the target in the image. If the number of divided images is more, the magnification is also Larger, reducing the difficulty of target detection, can effectively improve the accuracy and success rate of target detection.
  • the method further includes: controlling the at least two image analysis units to analyze and process the received sub-images in parallel.
  • the advantage of this setting is that it improves the accuracy and success rate of target detection while ensuring real-time detection.
  • the content contained in the analysis result of the image analysis unit is related to the specific type, model, and function of the image analysis unit, which is not limited in the embodiment of the present invention.
  • the analysis result may include the location information and type information of the analyzed target.
  • the target can be a target object or a target person, etc. It can be a pre-designated target or an automatically recognized target, which can be set according to actual needs.
  • the analysis results of at least two image analysis units can be integrated to determine the final target detection result.
  • the target detection method provided in the embodiment of the present invention is applied to an unmanned aerial vehicle.
  • the unmanned aerial vehicle is integrated with at least two image analysis units to acquire the first image taken by the pan/tilt camera, and perform segmentation processing on the first image, and at least Perform a size change operation on two divided images, and the resolutions of the obtained at least two sub-images match the resolutions corresponding to the at least two image analysis units.
  • the at least two sub-images are input into the at least two image analysis units, and the The analysis results of the two image analysis units determine the target detection result.
  • the loss of image information can be reduced when the size is changed, and then it can be analyzed by at least two image analysis units, which can improve the target detection Accuracy and success rate.
  • FIG. 8 is a schematic flowchart of a target detection method according to the second embodiment of the present invention. The method is optimized on the basis of the above embodiment, and the specific process of determining the target detection result according to the analysis results of the at least two image analysis units To refine.
  • the determining the target detection result according to the analysis results of the at least two image analysis units includes: acquiring the analysis results of the at least two image analysis units; performing fusion processing on the at least two analysis results to obtain the target Test results.
  • the advantage of this setting is that during the segmentation process, the target may be in the segmentation position, resulting in the target being detected in the adjacent images.
  • the analysis results can be fused to determine the segmentation of the target.
  • the analysis results are merged.
  • the analysis result includes the type information and position information of the analyzed target
  • the fusion processing of at least two analysis results includes: sequentially recording every two adjacent sub-images as a current sub-image pair ,
  • the current sub-image pair includes a first sub-image and a second sub-image, and the following operations are performed on the current sub-image pair: determining the first target in the first sub-image, and determining the first target in the second sub-image According to the first position information and first type information corresponding to the first target, and the second position information and second type information corresponding to the second target, the first target and the Whether the second target corresponds to the same target, and if so, the first target and the second target are merged into the same target.
  • the advantage of this setting is that it can determine whether there is an image corresponding to the segmented target in every two adjacent sub-images one by one according to the position information and the type information, and can accurately identify the segmented target in the image.
  • the method includes the following steps:
  • Step 201 Obtain a first image taken by a pan-tilt camera.
  • Step 202 Perform segmentation processing on the first image to obtain at least two segmented images.
  • Step 203 Perform a size change operation on the at least two divided images to obtain at least two sub-images, and the resolutions of the at least two sub-images match the resolutions corresponding to the at least two image analysis units.
  • Step 204 Input at least two sub-images into the at least two image analysis units, and control the at least two image analysis units to analyze and process the received sub-images in parallel.
  • Step 205 Obtain analysis results of at least two image analysis units.
  • Step 206 Record every two adjacent sub-images as a current sub-image pair in turn, and perform the following operations for the current sub-image pair: determine the first target in the first sub-image and the second target in the second sub-image; According to the first location information and the first type information corresponding to the first target, and the second location information and the second type information corresponding to the second target, it is determined whether the first target and the second target correspond to the same target. A target and the second target merge into the same target.
  • the analysis result includes the type information and position information of the analyzed target, and the current sub-image pair includes the first sub-image and the second sub-image.
  • the first target may be any target analyzed in the first sub-image
  • the second target may be any target analyzed in the second sub-image.
  • a preliminary screening may be performed on the first target and the second target, and the target closer to the segmentation boundary may be determined as the first target or the second target.
  • the first target as an example, record the boundary overlapping with the second sub-image in the first sub-image as the segmentation boundary, obtain the candidate targets analyzed in the first sub-image, and determine for each candidate target The fifth distance between the current candidate target and the segmentation boundary, and if the fifth distance is less than a third preset threshold, the current candidate target is determined as the first target.
  • the third preset threshold may be set according to the size of the sub-image, for example, it may be a preset ratio of the side length of the side perpendicular to the segmentation boundary, and the preset ratio may be, for example, 10%.
  • the position information in the analysis result may include a coordinate range
  • the coordinate range may form a certain shape, such as a circle, an ellipse, or a rectangle, or a shape that matches the shape of the target.
  • the position information includes the coordinates of a rectangular frame, and the rectangular frame contains an image corresponding to the analyzed target.
  • the rectangular frame corresponding to the first target is recorded as a first rectangular frame
  • the rectangular frame corresponding to the second target is recorded as a second rectangular frame.
  • Type information can be determined by the specific capabilities of the image analysis unit. For example, it can analyze whether it is a moving object or a still life, an animal or a person, and it can also analyze specific categories, such as vehicles, houses, etc., and analyze more detailed categories. , Such as cars, buses, fire trucks, ambulances, etc.
  • the same rule is adopted for numbering the four sides of the first rectangle and the second rectangle, and then it is determined whether the two rectangles correspond to the same target based on the distance between the sides and the type information corresponding to the two targets.
  • the first target and the first type information are determined according to the first position information and the first type information corresponding to the first target, and the second position information and the second type information corresponding to the second target. Whether the second target corresponds to the same target can include:
  • the first distance between the first boundary of the first rectangular frame and the third boundary of the second rectangular frame, and the first distance between the first boundary of the first rectangular frame and the third boundary of the second rectangular frame are calculated.
  • the first boundary in each rectangular frame is the left boundary
  • the first sub-image and the second sub-image are up and down
  • adjacent the first boundary in each rectangle is the upper boundary;
  • the first ratio is less than a first preset threshold
  • the second ratio is greater than a second preset threshold
  • the first type information and the second type information are the same
  • the first target and the second type information are determined Whether the second target corresponds to the same target, wherein the first preset threshold is smaller than the second preset threshold.
  • the first preset threshold and the second preset threshold can be set according to actual needs, for example, the first preset threshold is 0.1 and the second preset threshold is 0.6.
  • the fusing the first target and the second target into the same target includes: determining a target rectangular frame according to the coordinates of the first rectangular frame and the coordinates of the second rectangle, the target The rectangular frame includes both the first rectangular frame and the second rectangular frame; the target rectangular frame and the first type information are determined as the analysis result corresponding to the fused target.
  • the advantage of this setting is that when the first target and the second target are determined to be the same target, the target rectangular frame containing both the first rectangular frame and the second rectangular frame can be determined as the final position information of the target, avoiding the final target The number of targets in the test result is wrong.
  • Step 207 Determine the result after the fusion processing as the target detection result.
  • the target detection method In the target detection method provided by the embodiment of the present invention, after the image segmentation and size change are performed, it is handed over to at least two image analysis units for parallel processing. After the preliminary analysis results of each image analysis unit are obtained, it is fully considered that the same target is segmented into The conditions in the two sub-images are merged with the analysis results of the determined segmented targets, and finally an accurate target detection result is obtained.
  • the target detection result may further include performing a splicing operation on at least two sub-images containing the target detection result, and output to the user equipment.
  • the user equipment may be a device such as a mobile terminal or a computer for the user to view the target detection result.
  • Fig. 9 is a schematic flow chart of a target detection method provided in the third embodiment of the present invention. The method is described by taking the example of integrating two image analysis units in a drone to divide the image into left and right equally. Specifically, the method Including the following steps:
  • Step 301 Obtain a first image taken by a pan-tilt camera.
  • the following description is made by taking the image in FIG. 2 as the first image as an example.
  • the image ratio is 16:9 and the resolution is 1920*1080.
  • Step 302 Perform left and right average division processing on the first image to obtain two divided images.
  • Step 303 Perform a size change operation on the two divided images to obtain two sub-images, and the resolutions of the two sub-images match the resolutions corresponding to the two image analysis units.
  • the two image analysis units are the two forward reasoners that are carried on the HI3559C chip specifically for accelerating neural networks, and the resolution supported by each reasoner is 512*512.
  • the resolution supported by each reasoner is 512*512.
  • Resizing the images in Figure 5 and Figure 6 were performed to obtain two images with an image ratio of 1:1 and a resolution of 512*512.
  • Step 304 Input two sub-images into two image analysis units, and control the two image analysis units to analyze and process the received sub-images in parallel.
  • Step 305 Obtain the analysis results of the two image analysis units.
  • the two reasoners will respectively provide location information (BBox) and category (Class) information of the analyzed target in the sub-image to which it belongs.
  • the position information is represented by the coordinates of the rectangular frame.
  • the two reasoners will also respectively give the position information of the car in the sub-image for which they are responsible.
  • FIG. 10 is a schematic diagram of a left side image including location information provided by Embodiment 3 of the present invention
  • FIG. 11 is a schematic diagram of a right side image including location information provided by Embodiment 3 of the present invention, as shown in FIGS. 10 and 11 , Respectively use rectangular boxes to circle the positions of the analyzed targets.
  • Step 306 Determine the first target in the first sub-image and the second target in the second sub-image, according to the first location information and the first type information corresponding to the first target, and the second location information corresponding to the second target With the second type of information, it is determined whether the first target and the second target correspond to the same target, and if so, the first target and the second target are merged into the same target.
  • Fig. 12 is a schematic diagram of a fusion of analysis results provided by Embodiment 3 of the present invention.
  • the left rectangular box Left BBox represents the first rectangular box of the first target
  • the right rectangular box Right BBox represents the second target’s first rectangular box.
  • the rectangular frame in the left picture and the rectangular frame in the right picture are in the same coordinate system, and the coordinate system can be determined according to the spliced image of the first sub-image and the second sub-image, for example, the lower left corner of the image is the coordinate
  • the origin, the lower boundary is the horizontal axis, and the left boundary is the vertical axis.
  • the rectangular box on the left can be represented by coordinates (Ltop, Lbottom, Lleft, Lrigth), and the rectangular box on the right can be represented by coordinates (Rtop, Rbottom, Rleft, Rrigth).
  • W, w, H, and h in the figure are as follows Formula representation.
  • the left and right BBoxes are the same target.
  • the merged BBox is (Ltop, Rbottom, Lleft, Rrigth).
  • 0.1 is the first preset threshold
  • 0.6 is the second preset threshold.
  • the merged BBox is (min(Ltop, Rtop), max(Lbottom, Rbottom) ), Lleft, Rrigth).
  • Step 307 Determine the result after the fusion processing as the target detection result.
  • the original image collected by the pan/tilt camera is divided equally between the left and right structure, and the size is changed, and then the two reasoners are processed in parallel, and the preliminary analysis of the two reasoners is obtained. After the result, the situation that the same target is segmented into two sub-images is fully considered, and the analysis results of the determined segmented targets are merged, and finally an accurate target detection result is obtained.
  • the detection resolution is expanded from 512*512 to 1024*512, which can improve the detection success rate of small targets.
  • the parallel operation of dual reasoners is the same as that of 512*512 with single reasoners.
  • the image resizing of :9 is 1:1 compared to the image resizing of 16:9 to 1:1. The resizing process retains more information of the image and further improves the accuracy and success rate of target detection.
  • Fig. 13 is a structural block diagram of a target detection device provided by the fourth embodiment of the present invention.
  • the device can be implemented by software and/or hardware, and can generally be integrated in an unmanned aerial vehicle.
  • Target detection can be performed by executing a target detection method.
  • At least two image analysis units are integrated in the drone.
  • the device includes:
  • the image acquisition module 401 is used to acquire the first image taken by the pan-tilt camera;
  • the image segmentation module 402 is configured to perform segmentation processing on the first image to obtain at least two segmented images
  • the size changing module 403 is configured to perform a size changing operation on the at least two divided images to obtain at least two sub-images, and the resolutions of the at least two sub-images match the resolutions corresponding to the at least two image analysis units ;
  • the target detection module 404 is configured to input the at least two sub-images into the at least two image analysis units, and determine the target detection result according to the analysis results of the at least two image analysis units.
  • the target detection device provided in the embodiment of the present invention is applied to an unmanned aerial vehicle.
  • the unmanned aerial vehicle is integrated with at least two image analysis units to acquire the first image taken by the pan/tilt camera, and perform segmentation processing on the first image to determine at least Perform a size change operation on two divided images, and the resolutions of the obtained at least two sub-images match the resolutions corresponding to the at least two image analysis units.
  • the at least two sub-images are input into the at least two image analysis units, and the The analysis results of the two image analysis units determine the target detection result.
  • the loss of image information can be reduced when the size is changed, and then it can be analyzed by at least two image analysis units, which can improve the target detection Accuracy and success rate.
  • the determining the target detection result according to the analysis results of the at least two image analysis units includes:
  • the analysis result includes type information and location information of the analyzed target
  • the fusion processing of at least two analysis results includes:
  • each two adjacent sub-images are recorded as a current sub-image pair in turn, the current sub-image pair includes a first sub-image and a second sub-image, and the following operations are performed on the current sub-image pair:
  • the position information includes the coordinates of a rectangular frame, which contains an image corresponding to the analyzed target; the rectangular frame corresponding to the first target is marked as the first rectangular frame, and the second target corresponds to The rectangular frame of is marked as the second rectangular frame;
  • the first target and the second target are determined according to the first location information and the first type information corresponding to the first target, and the second location information and the second type information corresponding to the second target Whether it corresponds to the same target, including:
  • the first distance between the first boundary of the first rectangular frame and the third boundary of the second rectangular frame, and the first distance between the first boundary of the first rectangular frame and the third boundary of the second rectangular frame are calculated.
  • the first boundary in each rectangular frame is the left boundary
  • the first sub-image and the second sub-image are up and down
  • the first boundary in each rectangular box is the upper boundary;
  • the first ratio is less than a first preset threshold
  • the second ratio is greater than a second preset threshold
  • the first type information and the second type information are the same
  • the first target and the second type information are determined Whether the second target corresponds to the same target, wherein the first preset threshold is smaller than the second preset threshold.
  • the fusing the first target and the second target into the same target includes:
  • a target rectangular frame according to the coordinates of the first rectangular frame and the coordinates of the second rectangle, the target rectangular frame including both the first rectangular frame and the second rectangular frame;
  • the target rectangular frame and the first type information are determined as the analysis result corresponding to the fused target.
  • determining the first target in the first sub-image includes:
  • For each candidate target determine the fifth distance between the current candidate target and the segmentation boundary, and if the fifth distance is less than the third preset threshold, determine the current candidate target as the first target.
  • the at least two image analysis units include at least two forward reasoners NNIE that are accelerated based on a neural network.
  • An embodiment of the present invention also provides a storage medium containing computer-executable instructions, which are used to execute a target detection method when executed by a computer processor, and the method includes:
  • the at least two sub-images are input into the at least two image analysis units, and the target detection result is determined according to the analysis results of the at least two image analysis units.
  • Storage medium any of various types of storage devices or storage devices.
  • the term "storage medium” is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; Volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.
  • the storage medium may further include other types of memory or a combination thereof.
  • the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system connected to the first computer system through a network (such as the Internet).
  • the second computer system can provide the program instructions to the first computer for execution.
  • storage media may include two or more storage media that may reside in different locations (for example, in different computer systems connected through a network).
  • the storage medium may store program instructions (for example, embodied as a computer program) executable by one or more processors.
  • a storage medium containing computer-executable instructions provided by an embodiment of the present invention is not limited to the above-mentioned target detection operation, and can also execute the target detection method provided in any embodiment of the present invention. Related operations.
  • FIG. 14 is a structural block diagram of an unmanned aerial vehicle according to Embodiment 6 of the present invention.
  • the drone 500 may include: a memory 501, a processor 502, and at least two image analysis units 503 (only one is shown in the figure).
  • a computer program stored on the memory 501 and running on a processor, the processor 502 executing The computer program implements the target detection method described in the embodiment of the present invention.
  • the method may include:
  • the resolutions of the at least two sub-images match the resolutions corresponding to the at least two image analysis units;
  • the at least two sub-images are input into the at least two image analysis units, and the target detection result is determined according to the analysis results of the at least two image analysis units.
  • the computer device provided by the embodiment of the present invention can reduce the loss of image information during the size change operation after the original image taken by the pan/tilt camera is divided, and then it can be analyzed by at least two image analysis units, which can improve The accuracy and success rate of target detection.
  • the target detection device, storage medium, and computer equipment provided in the foregoing embodiments can execute the target detection method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for executing the method.
  • the target detection method provided in any embodiment of the present invention can execute the target detection method provided in any embodiment of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

Procédé et appareil de détection de cible, support de stockage, et véhicule aérien sans pilote. Le procédé consiste : à acquérir une première image capturée par une caméra à cardan (101) ; à réaliser un traitement par segmentation sur la première image de manière à obtenir au moins deux images segmentées (102) ; à réaliser une opération de changement de taille sur lesdites images segmentées de manière à obtenir au moins deux sous-images, les résolutions desdites sous-images étant adaptées aux résolutions correspondant à au moins deux unités d'analyse d'image (103) ; et à entrer lesdites sous-images dans lesdites unités d'analyse d'image, et à déterminer un résultat de détection de cible en fonction des résultats d'analyse desdites unités d'analyse d'image (104). L'utilisation de la solution technique du présent procédé permet de réduire la perte d'informations d'image et d'améliorer la précision et le taux de réussite de la détection de cible.
PCT/CN2020/124055 2019-11-01 2020-10-27 Procédé et appareil de détection de cible, support de stockage et véhicule aérien sans pilote WO2021083151A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911060737.1A CN110796104A (zh) 2019-11-01 2019-11-01 目标检测方法、装置、存储介质及无人机
CN201911060737.1 2019-11-01

Publications (1)

Publication Number Publication Date
WO2021083151A1 true WO2021083151A1 (fr) 2021-05-06

Family

ID=69442491

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/124055 WO2021083151A1 (fr) 2019-11-01 2020-10-27 Procédé et appareil de détection de cible, support de stockage et véhicule aérien sans pilote

Country Status (2)

Country Link
CN (1) CN110796104A (fr)
WO (1) WO2021083151A1 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110796104A (zh) * 2019-11-01 2020-02-14 深圳市道通智能航空技术有限公司 目标检测方法、装置、存储介质及无人机
CN111951328A (zh) * 2020-08-05 2020-11-17 清华大学苏州汽车研究院（吴江） 一种物体位置检测方法、装置、设备和存储介质
CN112101134B (zh) * 2020-08-24 2024-01-02 深圳市商汤科技有限公司 物体的检测方法及装置、电子设备和存储介质
CN112307976B (zh) * 2020-10-30 2024-05-10 北京百度网讯科技有限公司 目标检测方法、装置、电子设备以及存储介质
CN112348835B (zh) * 2020-11-30 2024-04-16 广联达科技股份有限公司 一种物料数量的检测方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103149939A (zh) * 2013-02-26 2013-06-12 北京航空航天大学 一种基于视觉的无人机动态目标跟踪与定位方法
US20140336848A1 (en) * 2013-05-10 2014-11-13 Palo Alto Research Center Incorporated System and method for detecting, tracking and estimating the speed of vehicles from a mobile platform
CN104966092A (zh) * 2015-06-16 2015-10-07 中国联合网络通信集团有限公司 一种图像处理方法和装置
CN107909600A (zh) * 2017-11-04 2018-04-13 南京奇蛙智能科技有限公司 一种基于视觉的无人机实时运动目标分类与检测方法
CN109765939A (zh) * 2018-12-21 2019-05-17 中国科学院自动化研究所南京人工智能芯片创新研究院 无人机的云台控制方法、装置和存储介质
CN110796104A (zh) * 2019-11-01 2020-02-14 深圳市道通智能航空技术有限公司 目标检测方法、装置、存储介质及无人机

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103149939A (zh) * 2013-02-26 2013-06-12 北京航空航天大学 一种基于视觉的无人机动态目标跟踪与定位方法
US20140336848A1 (en) * 2013-05-10 2014-11-13 Palo Alto Research Center Incorporated System and method for detecting, tracking and estimating the speed of vehicles from a mobile platform
CN104966092A (zh) * 2015-06-16 2015-10-07 中国联合网络通信集团有限公司 一种图像处理方法和装置
CN107909600A (zh) * 2017-11-04 2018-04-13 南京奇蛙智能科技有限公司 一种基于视觉的无人机实时运动目标分类与检测方法
CN109765939A (zh) * 2018-12-21 2019-05-17 中国科学院自动化研究所南京人工智能芯片创新研究院 无人机的云台控制方法、装置和存储介质
CN110796104A (zh) * 2019-11-01 2020-02-14 深圳市道通智能航空技术有限公司 目标检测方法、装置、存储介质及无人机

Also Published As

Publication number Publication date
CN110796104A (zh) 2020-02-14

Similar Documents

Publication Publication Date Title
WO2021083151A1 (fr) Procédé et appareil de détection de cible, support de stockage et véhicule aérien sans pilote
CN110163904B (zh) 对象标注方法、移动控制方法、装置、设备及存储介质
US11120280B2 (en) Geometry-aware instance segmentation in stereo image capture processes
CN110825101A (zh) 一种基于深度卷积神经网络的无人机自主降落方法
WO2021253245A1 (fr) Procédé et dispositif d'identification de tendance au changement de voie d'un véhicule
CN115376109B (zh) 障碍物检测方法、障碍物检测装置以及存储介质
CN111738033B (zh) 基于平面分割的车辆行驶信息确定方法及装置、车载终端
CN113903011A (zh) 一种适用于室内停车场的语义地图构建及定位方法
US20220301099A1 (en) Systems and methods for generating object detection labels using foveated image magnification for autonomous driving
CN110568861A (zh) 一种人机运动障碍物监测方法、可读存储介质和无人机
CN114841910A (zh) 车载镜头遮挡识别方法及装置
CN114627395B (zh) 基于嵌套靶标的多旋翼无人机角度分析方法、系统及终端
CN112069997A (zh) 一种基于DenseHR-Net的无人机自主着陆目标提取方法及装置
CN114596239A (zh) 装卸货事件检测方法、装置、计算机设备和存储介质
CN112286230A (zh) 无人机视觉图像算法、避障步骤及其信息融合处理系统
CN112706159A (zh) 机器人控制方法、装置及机器人
CN114396911B (zh) 一种障碍物测距方法、装置、设备及存储介质
CN113139984B (zh) 检测与跟踪融合的长时间无人机目标跟踪方法及系统
US20240193964A1 (en) Lane line recognition method, electronic device and storage medium
CN114549976A (zh) 一种基于多摄像头的移动机器人的轨迹测量方法及系统
Perevoshchikova et al. Algorithms for Recognizing Railway Infrastructure Objects in Images
WO2020202636A1 (fr) Procédé de traitement d'informations et système de traitement d'informations
CN115239748A (zh) 基于航拍视频的路径综合信息流生成方法及系统
CN117173651A (zh) 用于确定可行驶区域信息的方法、装置和计算机可读存储介质
CN114549978A (zh) 一种基于多摄像头的移动机器人运行方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20882331

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20882331

Country of ref document: EP

Kind code of ref document: A1