WO2022116675A1 - Procédé et appareil de détection d'objet, support de stockage et dispositif électronique - Google Patents

Procédé et appareil de détection d'objet, support de stockage et dispositif électronique Download PDF

Info

Publication number
WO2022116675A1
WO2022116675A1 PCT/CN2021/122451 CN2021122451W WO2022116675A1 WO 2022116675 A1 WO2022116675 A1 WO 2022116675A1 CN 2021122451 W CN2021122451 W CN 2021122451W WO 2022116675 A1 WO2022116675 A1 WO 2022116675A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection area
area
target
current detection
pixel
Prior art date
Application number
PCT/CN2021/122451
Other languages
English (en)
Chinese (zh)
Inventor
骆磊
黄晓庆
Original Assignee
达闼机器人股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 达闼机器人股份有限公司 filed Critical 达闼机器人股份有限公司
Publication of WO2022116675A1 publication Critical patent/WO2022116675A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof

Definitions

  • the present disclosure relates to the technical field of object detection, and in particular, to an object detection method, apparatus, storage medium and electronic device.
  • the robot is usually required to have functions such as navigation, obstacle avoidance, and capture (for example, an intelligent sweeping robot), which requires the robot to be able to detect objects in the surrounding environment.
  • functions such as navigation, obstacle avoidance, and capture (for example, an intelligent sweeping robot), which requires the robot to be able to detect objects in the surrounding environment.
  • ultrasonic, infrared, laser radar and other sensing devices are mainly used as the navigation obstacle avoidance device of the robot, and object detection is realized through the navigation obstacle avoidance device.
  • object detection is realized through the navigation obstacle avoidance device.
  • using these sensing devices as navigation and obstacle avoidance devices has various drawbacks.
  • ultrasonic and infrared sensing devices are relatively cheap, but can only detect objects at close range, and when the situation is complicated, the number of object detection failures will be relatively high.
  • lidar can detect objects with high accuracy and long detection distance, it is expensive, consumes high power for active scanning, and its volume and weight are relatively large, so it is not suitable for most robots.
  • the present disclosure provides an object detection method, apparatus, storage medium and electronic device.
  • an object detection method which is applied to an object detection device, the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of For the second microlens, the field of view of the first microlens is greater than the field of view of the second microlens, and the method includes:
  • each of the preset detection areas is an area composed of target pixels on the image sensor, the target pixels corresponding to each of the preset detection areas are different, and the target pixels include the first Pixel points, and/or second pixel points, the first pixel points are pixels included in the projection areas of the first microlenses on the image sensor, and the second pixel points are a plurality of pixel points included in the projection area of the second microlens on the image sensor, and the first pixel information is the pixel point information collected by the target pixel point corresponding to the current detection area;
  • the first object feature satisfies the area switching condition corresponding to the current detection area, determining a target detection area other than the current detection area from a plurality of the preset detection areas;
  • the first object feature includes a first distance between the object to be detected and the object detection device, and a first size and a first moving speed of the object to be detected;
  • the region switching conditions include the following: Any of the conditions:
  • the first size is smaller than a first size threshold corresponding to the current detection area, or the first size is greater than or equal to a second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold size threshold;
  • the first distance is greater than the first distance threshold corresponding to the current detection area, or the first distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold distance threshold;
  • the first moving speed is less than the first speed threshold corresponding to the current detection area, or the first moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • the method further includes:
  • the first object feature does not satisfy the region switching condition corresponding to the current detection region, the first object feature is used as the second object feature.
  • the preset detection area includes a large field of view detection area, a small field of view detection area and a full-lens detection area
  • the target pixel corresponding to the large field of view detection area includes the first pixel.
  • the target pixel point corresponding to the small field of view detection area includes the second pixel point
  • the all-lens detection area includes the large field of view detection area and the small field of view detection area.
  • the number of the target pixels corresponding to the all-lens detection area is greater than the small field of view detection area, and the number of the target pixels corresponding to the small field of view detection area is greater than the large field of view. detection area.
  • an object detection apparatus which is applied to an object detection device, the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of second microlenses, The angle of view of the first microlens is greater than the angle of view of the second microlens, and the device includes:
  • a processing module configured to determine the first object feature of the object to be detected according to the first pixel information corresponding to the current detection area in the plurality of preset detection areas;
  • each of the preset detection areas is an area composed of target pixels on the image sensor, the target pixels corresponding to each of the preset detection areas are different, and the target pixels include the first Pixel points, and/or second pixel points, the first pixel points are pixels included in the projection areas of the first microlenses on the image sensor, and the second pixel points are a plurality of pixel points included in the projection area of the second microlens on the image sensor, and the first pixel information is the pixel point information collected by the target pixel point corresponding to the current detection area;
  • a determination module configured to determine a target detection area other than the current detection area from a plurality of the preset detection areas when the first object feature satisfies the area switching condition corresponding to the current detection area;
  • the processing module is further configured to determine a second object feature of the object to be detected according to second pixel information corresponding to the target detection area, where the second pixel information is the target corresponding to the target detection area The pixel point information collected by the pixel point.
  • the first object feature includes a first distance between the object to be detected and the object detection device, and a first size and a first moving speed of the object to be detected;
  • the region switching conditions include the following: Any of the conditions:
  • the first size is smaller than a first size threshold corresponding to the current detection area, or the first size is greater than or equal to a second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold size threshold;
  • the first distance is greater than the first distance threshold corresponding to the current detection area, or the first distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold distance threshold;
  • the first moving speed is less than the first speed threshold corresponding to the current detection area, or the first moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • the processing module is further configured to use the first object feature as the second object feature when the first object feature does not satisfy the region switching condition corresponding to the current detection region.
  • the preset detection area includes a large field of view detection area, a small field of view detection area and a full-lens detection area
  • the target pixel corresponding to the large field of view detection area includes the first pixel.
  • the target pixel point corresponding to the small field of view detection area includes the second pixel point
  • the all-lens detection area includes the large field of view detection area and the small field of view detection area.
  • the number of the target pixels corresponding to the all-lens detection area is greater than the small field of view detection area, and the number of the target pixels corresponding to the small field of view detection area is greater than the large field of view. detection area.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the object detection method provided in the first aspect.
  • an electronic device comprising:
  • a processor configured to execute the computer program in the memory, to implement the steps of the object detection method provided by the first aspect.
  • the present disclosure first determines the first object feature of the object to be detected by first pixel information collected according to the target pixel point corresponding to the current detection area in the plurality of preset detection areas, wherein each preset detection area is The area is an area composed of target pixels on the image sensor.
  • the target pixels corresponding to each preset detection area are different.
  • the target pixels include the first pixel and/or the second pixel.
  • a target detection area other than the current detection area is determined from a plurality of preset detection areas, and the second pixel information is collected according to the target pixel points corresponding to the target detection area, A second object characteristic of the object to be detected is determined.
  • the present disclosure divides a plurality of preset detection areas corresponding to different numbers of target pixel points through the first microlens and the second microlens, and determines the second object feature of the object to be detected by using the second pixel information corresponding to the target detection area , can realize the accurate detection of the object to be detected under the condition of low power consumption, and at the same time, the structure of the object detection device is simple, and the cost, weight and volume of the object detection device are reduced.
  • the current detection area can be switched, and an appropriate target detection area can be selected for object detection, so as to reduce the power consumption of object detection while ensuring the object detection accuracy.
  • FIG. 1 is a flowchart of an object detection method according to an exemplary embodiment
  • FIG. 2 is a schematic diagram of the field of view of a first microlens and a second microlens according to an exemplary embodiment
  • FIG. 3 is a flowchart illustrating another object detection method according to an exemplary embodiment
  • FIG. 4 is a schematic diagram showing the distribution of a first microlens and a second microlens according to an exemplary embodiment
  • FIG. 5 is a block diagram of an object detection apparatus according to an exemplary embodiment
  • Fig. 6 is a block diagram of an electronic device provided according to an exemplary embodiment.
  • the application scenario is to use object detection equipment to detect objects to be detected in the surrounding environment.
  • the object detection device may be provided on the terminal device, and the object detection device may include a lens and an image sensor.
  • the image sensor is located on the side corresponding to the image-side surface of the lens, and has an imaging surface facing the image-side surface.
  • the imaging surface is composed of multiple pixels.
  • the lens and the image sensor can be completely fitted together, or they can be separated by a certain interval. the distance.
  • the lens can be a flat lens or a curved lens
  • the image sensor can be a CMOS (English: Complementary Metal-Oxide-Semiconductor, Chinese: Complementary Metal Oxide Semiconductor) sensor, or a CCD (English: Charge-coupled Device, Chinese: Charge Coupled Device) element), or any other photosensitive sensor, which is not specifically limited in the present disclosure.
  • the terminal device may be, for example, a terminal device such as a smart robot, a smart phone, a tablet computer, a smart watch, and a smart bracelet.
  • Fig. 1 is a flow chart of an object detection method according to an exemplary embodiment.
  • the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of second microlenses, and the field angle of the first microlens is larger than that of the second microlens the field of view, the method may include the following steps:
  • Step 101 Determine a first object feature of an object to be detected according to the first pixel information corresponding to the current detection area in the plurality of preset detection areas.
  • each preset detection area is an area composed of target pixels on the image sensor, the target pixels corresponding to each preset detection area are different, and the target pixels include first pixels and/or second pixels , the first pixel is the pixel included in the projection area of the plurality of first microlenses on the image sensor, the second pixel is the pixel included in the projection area of the plurality of second microlenses on the image sensor, and the first pixel
  • the information is the pixel point information collected by the target pixel point corresponding to the current detection area.
  • a new sensing device can be constructed through lenses and image sensors for object detection.
  • the lens may be divided into a plurality of first microlenses and a plurality of second microlenses arranged in a grid format.
  • a plurality of first micro-lenses and a plurality of second micro-lenses in a mesh format can be formed on the lens by photolithography (or other techniques), or a plurality of first micro-lenses in a mesh format can be formed on the lens by a layer of nano-film a microlens and a plurality of second microlenses.
  • each first microlens corresponds to one or more first pixels on the image sensor, and the first pixels corresponding to each first microlens are the pixels included in the projection area of the first microlens on the image sensor point.
  • imaging is performed on the first pixel point corresponding to the first microlens.
  • Each second microlens corresponds to one or more second pixels on the image sensor, and the second pixels corresponding to each second microlens are pixels included in the projection area of the second microlens on the image sensor.
  • a second microlens captures the object to be detected, it will image on the second pixel point corresponding to the second microlens.
  • the FOV (English: Field of View, Chinese: Field of View) of the first microlens is greater than the FOV of the second microlens, as shown in FIG. 2 , the dotted line in FIG. 2 is the FOV of the first microlens, and the The solid line is the FOV of the second microlens.
  • the imaging surface of the image sensor can be divided into a plurality of preset detection areas according to the type of pixel points (first pixel point or second pixel point), and each preset detection area can be an area composed of target pixel points .
  • the target pixels corresponding to each preset detection area are different, and the number of target pixels is different.
  • the target pixels may include the first pixel and/or the second pixel, that is, the target pixel corresponding to the preset detection area may Only the first pixel is included, only the second pixel may be included, or both the first pixel and the second pixel may be included.
  • a preset detection area may be selected from a plurality of preset detection areas as the current detection area, and the first pixel information collected by the target pixel points corresponding to the current detection area, Calculate the first object feature of the object to be detected.
  • the first pixel information may include the pixel value of each target pixel corresponding to the current detection area
  • the first object feature may include the first distance between the object to be detected and the object detection device, and the first size and first movement of the object to be detected. speed.
  • Step 102 in the case that the first object feature satisfies the area switching condition corresponding to the current detection area, determine a target detection area other than the current detection area from a plurality of preset detection areas.
  • the object to be detected is detected by the target pixels corresponding to different preset detection areas, the number of target pixels used is different, and the data amount of pixel information collected by the target pixels is different, which will lead to Target pixels corresponding to different preset detection areas have different resolution and power consumption for detecting the object to be detected.
  • the greater the number of target pixel points corresponding to the preset detection area the higher the resolving power of detecting the object to be detected, and the higher the power consumption of detecting the object to be detected at the same time.
  • the resolving power can reflect the performance of detecting the object to be detected. The higher the resolving power, the better the performance of detecting the object to be detected, that is, the higher the detection accuracy of the object to be detected.
  • the area switching conditions corresponding to each preset detection area can be preset, so that when the object detection device detects the object to be detected through the target pixel point corresponding to each preset detection area, it can determine that the preset detection area is to be detected. Whether the resolving power of the object for detection is insufficient or the resolving power is excessive.
  • the area switching condition corresponding to each preset detection area may be set according to the distance range between the object to be detected and the object detection device that can be detected by the preset detection area, as well as the size range and moving speed range of the object to be detected.
  • the first object feature satisfies the area switching condition corresponding to the current detection area, it means that the resolution power to detect the object to be detected by the target pixel point corresponding to the current detection area is insufficient or excessive, and the current detection area needs to be switched.
  • a preset detection area with a higher resolution may be selected from a plurality of preset detection areas as the target detection area (that is, select the target pixel point more preset detection areas) to detect objects to be detected.
  • a preset detection area with a smaller resolution can be selected from a plurality of preset detection areas as the target detection area to perform the detection of the object to be detected. Detection, that is, selecting a preset detection area with a smaller number of target pixels to detect the object to be detected, thereby reducing the power consumption of detecting the object to be detected. By switching the current detection area to the target detection area, the power consumption of object detection can be reduced while ensuring the object detection performance (detection accuracy), so as to meet the needs of different scenarios.
  • Step 103 Determine the second object feature of the object to be detected according to the second pixel information corresponding to the target detection area, where the second pixel information is pixel point information collected from the target pixel point corresponding to the target detection area.
  • the second object feature of the object to be detected may be calculated according to the second pixel information collected from the target pixel point corresponding to the target detection area.
  • the second object characteristic may include a second distance of the object to be detected from the object detection device, and a second size and a second moving speed of the object to be detected.
  • the present disclosure first determines the first object feature of the object to be detected by first pixel information collected according to the target pixel point corresponding to the current detection area in the plurality of preset detection areas, wherein each preset detection area
  • the area is an area composed of target pixels on the image sensor.
  • the target pixels corresponding to each preset detection area are different.
  • the target pixels include the first pixel and/or the second pixel.
  • a target detection area other than the current detection area is determined from a plurality of preset detection areas, and the second pixel information is collected according to the target pixel points corresponding to the target detection area, A second object characteristic of the object to be detected is determined.
  • the present disclosure divides a plurality of preset detection areas corresponding to different numbers of target pixel points through the first microlens and the second microlens, and determines the second object feature of the object to be detected by using the second pixel information corresponding to the target detection area , can realize the accurate detection of the object to be detected under the condition of low power consumption, and at the same time, the structure of the object detection device is simple, and the cost, weight and volume of the object detection device are reduced.
  • the current detection area can be switched, and an appropriate target detection area can be selected for object detection, so as to reduce the power consumption of object detection while ensuring the object detection accuracy.
  • Fig. 3 is a flowchart showing another object detection method according to an exemplary embodiment. As shown in Figure 3, the method further includes the following steps:
  • Step 104 in the case that the first object feature does not satisfy the region switching condition corresponding to the current detection region, the first object feature is used as the second object feature.
  • the first object feature does not meet the area switching condition corresponding to the current detection area, it means that the resolution power of detecting the object to be detected through the target pixel point corresponding to the current detection area meets the requirements, and the resolution power is not excessive, There is no need to switch the current detection area.
  • the first object feature can be directly used as the second object feature.
  • the first object feature includes a first distance between the object to be detected and the object detection device, and a first size and a first moving speed of the object to be detected
  • the area switching condition may include any of the following conditions:
  • the first size is smaller than the first size threshold corresponding to the current detection area, or the first size is greater than or equal to the second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold.
  • the first distance is greater than the first distance threshold corresponding to the current detection area, or the first distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold.
  • the first moving speed is less than the first speed threshold corresponding to the current detection area, or the first moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • the first size threshold is the minimum size of the object to be detected that can be detected in the current detection area
  • the first distance threshold is the maximum distance between the object to be detected and the object detection device that can be detected in the current detection area
  • the first speed threshold is The minimum speed of the object to be detected that can be detected in the current detection area.
  • the second size threshold is the maximum size of the object to be detected that can be detected in the current detection area
  • the second distance threshold is the minimum distance between the object to be detected and the object detection device that can be detected in the current detection area
  • the second speed threshold is the current detection area. The maximum speed of the object to be detected that can be detected.
  • first microlens and the second microlens may be the same or different.
  • the lens may be divided into a plurality of first microlenses and a plurality of second microlenses which are arranged in a grid format and have the same shape and size according to a preset arrangement.
  • the preset arrangement can be, for example, that the first microlenses and the second microlenses are evenly distributed on the lens according to a certain ratio.
  • the distribution of the micro-lens and the second micro-lens on the lens can be shown in Figure 4.
  • the smallest square in Figure 4 corresponds to one pixel.
  • the nine-square grid filled with shadows in Figure 4 is the first micro-lens.
  • Figure 4 The nine-square grid surrounded by dotted lines is the second microlens.
  • the preset detection area includes a large field of view detection area, a small field of view detection area and a full-lens detection area
  • the target pixel point corresponding to the large field of view detection area includes the first pixel point
  • the small field of view detection area corresponds to
  • the target pixel point includes the second pixel point
  • the full-lens detection area includes a detection area with a large field of view and a detection area with a small field of view.
  • the number of the first microlenses included on the lens is greater than the number of the second microlenses
  • the number of target pixels corresponding to the full-lens detection area is greater than the small field of view detection area
  • the small field of view detects
  • the number of target pixels corresponding to the area is larger than the detection area with a large field of view.
  • the relationship between the resolution power of the detection area with large field of view, the detection area with small field of view and the full-lens detection area is: full-lens detection area > detection area with small field of view > detection area with large field of view
  • the power consumption relationship is: : full-lens detection area ⁇ small field of view detection area ⁇ large field of view detection area.
  • the preset detection area includes a large field of view detection area, a small field of view detection area and a full-lens detection area, and the number of first microlenses on the lens is greater than the number of second microlenses.
  • the user can manually select a preset detection area from the detection area with a large field of view, a detection area with a small field of view, and a full-lens detection area as the current detection area to detect the object to be detected.
  • the method of automatic selection can also be adopted.
  • the detection area with a large field of view can be selected by default, and then the first pixel information of the target pixel corresponding to the detection area with a large field of view can be used to calculate the first object of the object to be detected.
  • the first size threshold corresponding to the detection area with a large field of view If it is determined that the first size is smaller than the first size threshold corresponding to the detection area with a large field of view, the first distance is greater than the first distance threshold corresponding to the detection area with a large field of view, and the first moving speed is smaller than the first distance corresponding to the detection area with a large field of view. Any one of the speed thresholds is satisfied (at this time, the resolution of the target pixel corresponding to the detection area with a large field of view is insufficient to detect the object to be detected, and the appearance is that the pixel value of the adjacent target pixel has a large difference or pixel value. change too slowly), switch the detection area with a large field of view to a detection area with a small field of view.
  • the first object feature of the object to be detected is recalculated, and it is judged whether the recalculated first object feature satisfies the area corresponding to the detection area with small field of view switch condition.
  • the recalculated first size is smaller than the first size threshold corresponding to the small field of view detection area
  • the recalculated first distance is greater than the first distance threshold corresponding to the small field of view detection area
  • the recalculated first moving speed is smaller than the small field of view Any one of the first speed thresholds corresponding to the angle detection area is satisfied (at this time, the target pixel corresponding to the small field of view detection area has insufficient resolving power to detect the object to be detected, and the appearance is that the pixel value difference between adjacent target pixel points larger), switch the detection area with small field of view to the full-lens detection area.
  • the first object feature of the object to be detected is recalculated through the first pixel information collected from the target pixel point corresponding to the full-lens detection area, and it is judged whether the recalculated first object feature satisfies the area switching conditions corresponding to the full-lens detection area .
  • the recalculated first size is greater than or equal to the second size threshold corresponding to the current detection area
  • the recalculated first distance is less than or equal to the second distance threshold corresponding to the current detection area
  • the recalculated first moving speed is greater than or equal to Any one of the second velocity thresholds corresponding to the current detection area satisfies (at this time, the target pixels corresponding to the full-lens detection area have excessive resolving power to detect the object to be detected, and the appearance is that the difference in pixel values of adjacent target pixels is relatively high. small), switch the full-lens detection area to the small field of view detection area.
  • the present disclosure first determines the first object feature of the object to be detected by first pixel information collected according to the target pixel point corresponding to the current detection area in the plurality of preset detection areas, wherein each preset detection area
  • the area is an area composed of target pixels on the image sensor.
  • the target pixels corresponding to each preset detection area are different.
  • the target pixels include the first pixel and/or the second pixel.
  • a target detection area other than the current detection area is determined from a plurality of preset detection areas, and the second pixel information is collected according to the target pixel points corresponding to the target detection area, A second object characteristic of the object to be detected is determined.
  • the present disclosure divides a plurality of preset detection areas corresponding to different numbers of target pixel points through the first microlens and the second microlens, and determines the second object feature of the object to be detected by using the second pixel information corresponding to the target detection area , can realize the accurate detection of the object to be detected under the condition of low power consumption, and at the same time, the structure of the object detection device is simple, and the cost, weight and volume of the object detection device are reduced.
  • the current detection area can be switched, and an appropriate target detection area can be selected for object detection, so as to reduce the power consumption of object detection while ensuring the object detection accuracy.
  • Fig. 5 is a block diagram of an object detection apparatus according to an exemplary embodiment.
  • the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of second microlenses, and the field angle of the first microlens is larger than that of the second microlens
  • the device 200 includes:
  • the processing module 201 is configured to determine the first object feature of the object to be detected according to the first pixel information corresponding to the current detection area in the plurality of preset detection areas.
  • each preset detection area is an area composed of target pixels on the image sensor, the target pixels corresponding to each preset detection area are different, and the target pixels include first pixels and/or second pixels , the first pixel is the pixel included in the projection area of the plurality of first microlenses on the image sensor, the second pixel is the pixel included in the projection area of the plurality of second microlenses on the image sensor, and the first pixel
  • the information is the pixel point information collected by the target pixel point corresponding to the current detection area.
  • the determining module 202 is configured to determine a target detection area other than the current detection area from a plurality of preset detection areas under the condition that the first object feature satisfies the area switching condition corresponding to the current detection area.
  • the processing module 201 is further configured to determine the second object feature of the object to be detected according to the second pixel information corresponding to the target detection area, where the second pixel information is pixel point information collected from the target pixel point corresponding to the target detection area.
  • the first object feature includes a first distance between the object to be detected and the object detection device, and a first size and a first moving speed of the object to be detected.
  • Region switching conditions include any of the following conditions:
  • the second size is smaller than the first size threshold corresponding to the current detection area, or the second size is greater than or equal to the second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold.
  • the second distance is greater than the first distance threshold corresponding to the current detection area, or the second distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold.
  • the second moving speed is less than the first speed threshold corresponding to the current detection area, or the second moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • the processing module 201 is further configured to use the first object feature as the second object feature when the first object feature does not satisfy the region switching condition corresponding to the current detection region.
  • the preset detection area includes a large field of view detection area, a small field of view detection area and a full-lens detection area
  • the target pixel point corresponding to the large field of view detection area includes the first pixel point
  • the small field of view detection area corresponds to
  • the target pixel point includes the second pixel point
  • the full-lens detection area includes a detection area with a large field of view and a detection area with a small field of view.
  • the number of target pixels corresponding to the full-lens detection area is larger than that of the small field of view detection area, and the number of target pixels corresponding to the small field of view detection area is larger than the large field of view detection area.
  • the present disclosure first determines the first object feature of the object to be detected by first pixel information collected according to the target pixel point corresponding to the current detection area in the plurality of preset detection areas, wherein each preset detection area
  • the area is an area composed of target pixels on the image sensor.
  • the target pixels corresponding to each preset detection area are different.
  • the target pixels include the first pixel and/or the second pixel.
  • a target detection area other than the current detection area is determined from a plurality of preset detection areas, and the second pixel information is collected according to the target pixel points corresponding to the target detection area, A second object characteristic of the object to be detected is determined.
  • the present disclosure divides a plurality of preset detection areas corresponding to different numbers of target pixel points through the first microlens and the second microlens, and determines the second object feature of the object to be detected through the second pixel information corresponding to the target detection area , can realize the accurate detection of the object to be detected under the condition of low power consumption, and at the same time, the structure of the object detection device is simple, and the cost, weight and volume of the object detection device are reduced.
  • the current detection area can be switched, and an appropriate target detection area can be selected for object detection, so as to reduce the power consumption of object detection while ensuring the object detection accuracy.
  • FIG. 6 is a block diagram of an electronic device 700 according to an exemplary embodiment.
  • the electronic device 700 may include: a processor 701 and a memory 702 .
  • the electronic device 700 may also include one or more of a multimedia component 703 , an input/output (I/O) interface 704 , and a communication component 705 .
  • I/O input/output
  • the processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned object detection method.
  • the memory 702 is used to store various types of data to support operations on the electronic device 700, such data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data, Such as contact data, messages sent and received, pictures, audio, video, and so on.
  • the memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory ( Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read-Only Memory (EPROM for short), Programmable Read-Only Memory (PROM for short), read-only Memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • Multimedia components 703 may include screen and audio components. Wherein the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals.
  • the audio component may include a microphone for receiving external audio signals.
  • the received audio signal may be further stored in memory 702 or transmitted through communication component 705 .
  • the audio assembly also includes at least one speaker for outputting audio signals.
  • the I/O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, a button, and the like. These buttons can be virtual buttons or physical buttons.
  • the communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices.
  • Wireless communication such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or one or more of them
  • the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module and so on.
  • the electronic device 700 may be implemented by one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), digital signal processors (Digital Signal Processor, DSP for short), digital signal processing devices (Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic components Implementation is used to implement the above object detection method.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSP Digital Signal Processor
  • DSP Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • controller microcontroller, microprocessor or other electronic components Implementation is used to implement the above object detection method.
  • a computer-readable storage medium comprising program instructions, the program instructions implementing the steps of the above-mentioned object detection method when executed by a processor.
  • the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to implement the above-mentioned object detection method.
  • An object detection method applied to an object detection device, the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of second microlenses, the field of view of the first microlens The angle is greater than the field angle of the second microlens, and the method includes:
  • each of the preset detection areas is an area composed of target pixels on the image sensor, the target pixels corresponding to each of the preset detection areas are different, and the target pixels include the first Pixel points, and/or second pixel points, the first pixel points are pixels included in the projection areas of the first microlenses on the image sensor, and the second pixel points are a plurality of pixel points included in the projection area of the second microlens on the image sensor, and the first pixel information is the pixel point information collected by the target pixel point corresponding to the current detection area;
  • the first object feature satisfies the area switching condition corresponding to the current detection area, determining a target detection area other than the current detection area from a plurality of the preset detection areas;
  • the area switching condition includes any one of the following conditions:
  • the first size is smaller than a first size threshold corresponding to the current detection area, or the first size is greater than or equal to a second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold size threshold;
  • the first distance is greater than the first distance threshold corresponding to the current detection area, or the first distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold distance threshold;
  • the first moving speed is less than the first speed threshold corresponding to the current detection area, or the first moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • the first object feature does not satisfy the region switching condition corresponding to the current detection region, the first object feature is used as the second object feature.
  • the preset detection area includes a detection area with a large field of view, a detection area with a small field of view, and a full-lens detection area
  • the large field of view The target pixel point corresponding to the angle detection area includes the first pixel point
  • the target pixel point corresponding to the small field-of-view detection area includes the second pixel point
  • the full-lens detection area includes the large angle detection area.
  • the number of the target pixels corresponding to the all-lens detection area is greater than that of the small field of view detection area, and the number of the target pixels corresponding to the small field of view detection area larger than the detection area of the large field of view.
  • An object detection device applied to an object detection device, the device includes a lens and an image sensor, the lens includes a plurality of first microlenses and a plurality of second microlenses, the field of view of the first microlens The angle is greater than the field angle of the second microlens, and the device includes:
  • a processing module configured to determine the first object feature of the object to be detected according to the first pixel information corresponding to the current detection area in the plurality of preset detection areas;
  • each of the preset detection areas is an area composed of target pixels on the image sensor, the target pixels corresponding to each of the preset detection areas are different, and the target pixels include the first Pixel points, and/or second pixel points, the first pixel points are pixels included in the projection areas of the first microlenses on the image sensor, and the second pixel points are a plurality of pixel points included in the projection area of the second microlens on the image sensor, and the first pixel information is the pixel point information collected by the target pixel point corresponding to the current detection area;
  • a determination module configured to determine a target detection area other than the current detection area from a plurality of the preset detection areas when the first object feature satisfies the area switching condition corresponding to the current detection area;
  • the processing module is further configured to determine a second object feature of the object to be detected according to second pixel information corresponding to the target detection area, where the second pixel information is the target corresponding to the target detection area The pixel point information collected by the pixel point.
  • the apparatus wherein the first object characteristic includes a first distance between the object to be detected and the object detection device, and a first size and a first moving speed of the object to be detected;
  • the area switching condition includes any one of the following conditions:
  • the first size is smaller than a first size threshold corresponding to the current detection area, or the first size is greater than or equal to a second size threshold corresponding to the current detection area, and the second size threshold is greater than the first size threshold size threshold;
  • the first distance is greater than the first distance threshold corresponding to the current detection area, or the first distance is less than or equal to the second distance threshold corresponding to the current detection area, and the second distance threshold is smaller than the first distance threshold distance threshold;
  • the first moving speed is less than the first speed threshold corresponding to the current detection area, or the first moving speed is greater than or equal to the second speed threshold corresponding to the current detection area, and the second speed threshold is greater than the first speed threshold.
  • An electronic device comprising:
  • a processor configured to execute the computer program in the memory, to implement the steps of the method in any one of Embodiments 1 to 5.

Abstract

La présente divulgation concerne un procédé et un appareil de détection d'objet, ainsi qu'un support de stockage et un dispositif électronique, ledit procédé consistant à : d'après les premières informations de pixel correspondant à une zone de détection actuelle parmi une pluralité de zones de détection prédéfinies, déterminer les premières caractéristiques d'un objet à détecter, chaque zone de détection prédéfinie étant une zone composée de points de pixel cibles sur un capteur d'image ; lorsque les premières caractéristiques d'objet remplissent une condition de commutation de zone correspondant à la zone de détection actuelle, déterminer une zone de détection cible autre que la zone de détection actuelle parmi la pluralité de zones de détection prédéfinies ; et d'après les secondes informations de pixel correspondant à la zone de détection cible, déterminer les secondes caractéristiques de l'objet à détecter. Selon la présente divulgation, la consommation d'énergie requise pour effectuer la détection de l'objet à détecter est faible, le coût de l'équipement de détection d'objet est bas, et le poids et le volume de l'équipement de détection d'objet sont petits. La zone de détection actuelle peut être commutée et une zone de détection de cible appropriée peut être sélectionnée pour la détection d'objet afin de réduire la consommation d'énergie de la détection d'objet tout en garantissant la précision de la détection d'objet.
PCT/CN2021/122451 2020-12-02 2021-09-30 Procédé et appareil de détection d'objet, support de stockage et dispositif électronique WO2022116675A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011406181.X 2020-12-02
CN202011406181.XA CN112584015B (zh) 2020-12-02 2020-12-02 物体探测方法、装置、存储介质和电子设备

Publications (1)

Publication Number Publication Date
WO2022116675A1 true WO2022116675A1 (fr) 2022-06-09

Family

ID=75127215

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/122451 WO2022116675A1 (fr) 2020-12-02 2021-09-30 Procédé et appareil de détection d'objet, support de stockage et dispositif électronique

Country Status (2)

Country Link
CN (1) CN112584015B (fr)
WO (1) WO2022116675A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112600994B (zh) * 2020-12-02 2023-04-07 达闼机器人股份有限公司 物体探测装置、方法、存储介质和电子设备
CN112584015B (zh) * 2020-12-02 2022-05-17 达闼机器人股份有限公司 物体探测方法、装置、存储介质和电子设备

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1937236A (zh) * 2005-09-19 2007-03-28 C.R.F.阿西安尼顾问公司 包含耦合至微透镜矩阵的光探测器矩阵的多功能光传感器
CN102710902A (zh) * 2011-02-16 2012-10-03 意法半导体(R&D)有限公司 多通道图像传感器
WO2013031447A1 (fr) * 2011-08-26 2013-03-07 三洋電機株式会社 Dispositif de détection d'objets et dispositif d'acquisition d'informations
CN109829393A (zh) * 2019-01-14 2019-05-31 北京鑫洋泉电子科技有限公司 一种移动物体探测方法、装置及存储介质
EP3660539A1 (fr) * 2018-11-27 2020-06-03 Sick Ag Capteur optoélectronique et procédé de détection d'objets
WO2020114930A1 (fr) * 2018-12-03 2020-06-11 Carl Zeiss Microscopy Gmbh Procédé et dispositif pour déceler des signaux de fluorescence dans une région tridimensionnelle d'un échantillon
CN111458045A (zh) * 2020-04-22 2020-07-28 电子科技大学 一种基于焦面哈特曼波前传感器的大视场波前探测方法
CN111896933A (zh) * 2020-07-16 2020-11-06 深圳市速腾聚创科技有限公司 激光雷达系统及探测方法
CN112584015A (zh) * 2020-12-02 2021-03-30 达闼机器人有限公司 物体探测方法、装置、存储介质和电子设备
CN112600994A (zh) * 2020-12-02 2021-04-02 达闼机器人有限公司 物体探测装置、方法、存储介质和电子设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103531596B (zh) * 2013-09-22 2015-09-23 华中科技大学 一种基于单眼套叠的全色复眼成像探测芯片
CN105509894B (zh) * 2015-12-07 2019-11-12 华中科技大学 一种液晶基图像与波前双模电调成像探测芯片
CN110441769A (zh) * 2018-05-03 2019-11-12 北京航空航天大学 基于sar序贯图像的目标定位方法、装置及存储介质
CN109443196B (zh) * 2018-09-28 2020-07-24 顺丰科技有限公司 一种体积测量方法及系统
CN110244314B (zh) * 2019-06-24 2022-04-19 北京机械设备研究所 一种“低慢小”目标探测识别系统与方法
CN110458888A (zh) * 2019-07-23 2019-11-15 深圳前海达闼云端智能科技有限公司 基于图像的测距方法、装置、存储介质和电子设备

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1937236A (zh) * 2005-09-19 2007-03-28 C.R.F.阿西安尼顾问公司 包含耦合至微透镜矩阵的光探测器矩阵的多功能光传感器
CN102710902A (zh) * 2011-02-16 2012-10-03 意法半导体(R&D)有限公司 多通道图像传感器
WO2013031447A1 (fr) * 2011-08-26 2013-03-07 三洋電機株式会社 Dispositif de détection d'objets et dispositif d'acquisition d'informations
EP3660539A1 (fr) * 2018-11-27 2020-06-03 Sick Ag Capteur optoélectronique et procédé de détection d'objets
WO2020114930A1 (fr) * 2018-12-03 2020-06-11 Carl Zeiss Microscopy Gmbh Procédé et dispositif pour déceler des signaux de fluorescence dans une région tridimensionnelle d'un échantillon
CN109829393A (zh) * 2019-01-14 2019-05-31 北京鑫洋泉电子科技有限公司 一种移动物体探测方法、装置及存储介质
CN111458045A (zh) * 2020-04-22 2020-07-28 电子科技大学 一种基于焦面哈特曼波前传感器的大视场波前探测方法
CN111896933A (zh) * 2020-07-16 2020-11-06 深圳市速腾聚创科技有限公司 激光雷达系统及探测方法
CN112584015A (zh) * 2020-12-02 2021-03-30 达闼机器人有限公司 物体探测方法、装置、存储介质和电子设备
CN112600994A (zh) * 2020-12-02 2021-04-02 达闼机器人有限公司 物体探测装置、方法、存储介质和电子设备

Also Published As

Publication number Publication date
CN112584015B (zh) 2022-05-17
CN112584015A (zh) 2021-03-30

Similar Documents

Publication Publication Date Title
WO2022116676A1 (fr) Appareil et procédé de détection d'objet, support de stockage et dispositif électronique
WO2022116675A1 (fr) Procédé et appareil de détection d'objet, support de stockage et dispositif électronique
US9230306B2 (en) System for reducing depth of field with digital image processing
US8724013B2 (en) Method and apparatus with fast camera auto focus
US8233077B2 (en) Method and apparatus with depth map generation
US8345986B2 (en) Image processing apparatus, image processing method and computer readable-medium
US10917586B2 (en) Electronic device providing switching between signals output from concurrently operable image sensors
US20140354874A1 (en) Method and apparatus for auto-focusing of an photographing device
US10564390B2 (en) Focusing control device, focusing control method, focusing control program, lens device, and imaging device
JP2011508268A5 (fr)
CN104215216B (zh) 一种测距装置及终端
US10078198B2 (en) Photographing apparatus for automatically determining a focus area and a control method thereof
US11343422B2 (en) Focusing control device, focusing control method, focusing control program, lens device, and imaging device
US20200412967A1 (en) Imaging element and imaging apparatus
US9407845B1 (en) Self powering camera
CN104215215B (zh) 一种测距方法
US20200221050A1 (en) Imaging control device, imaging apparatus, imaging control method, and imaging control program
WO2020003862A1 (fr) Dispositif d'imagerie, procédé d'imagerie, et programme
CN114286011B (zh) 对焦方法和装置
US10939056B2 (en) Imaging apparatus, imaging method, imaging program
US10520793B2 (en) Focusing control device, focusing control method, focusing control program, lens device, and imaging device
CN114827438B (zh) 协处理芯片、电子设备以及触控响应方法
US20230134771A1 (en) Image sensor, image acquisition apparatus, and electronic apparatus including the image acquisition apparatus
US11861874B2 (en) Imaging apparatus, imaging control method, and imaging control program
US11871106B2 (en) Imaging apparatus, imaging method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21899709

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10-11-2023)