WO2013035612A1 - Dispositif de détection d'obstacle, procédé de détection d'obstacle, et programme de détection d'obstacle - Google Patents

Dispositif de détection d'obstacle, procédé de détection d'obstacle, et programme de détection d'obstacle Download PDF

Info

Publication number
WO2013035612A1
WO2013035612A1 PCT/JP2012/071951 JP2012071951W WO2013035612A1 WO 2013035612 A1 WO2013035612 A1 WO 2013035612A1 JP 2012071951 W JP2012071951 W JP 2012071951W WO 2013035612 A1 WO2013035612 A1 WO 2013035612A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
road surface
obstacle
obstacle detection
determined
Prior art date
Application number
PCT/JP2012/071951
Other languages
English (en)
Japanese (ja)
Inventor
高橋　勝彦
Original Assignee
日本電気株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電気株式会社 filed Critical 日本電気株式会社
Publication of WO2013035612A1 publication Critical patent/WO2013035612A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Definitions

  • the present invention relates to obstacle detection based on a distance image acquired by a camera or a distance sensor.
  • sensors that can acquire distance images such as cameras that project pattern light and measure distances based on the principle of triangulation, and time-of-flight (TOF) cameras, have appeared.
  • the obstacle refers to an object that exists closer than a predetermined distance.
  • Patent Document 1 and Patent Document 2 As a general technique for such obstacle detection, for example, the techniques described in Patent Document 1 and Patent Document 2 can be cited.
  • Patent Document 1 a three-dimensional coordinate value of a point in the real world corresponding to each pixel included in a distance image is obtained, a road surface plane is detected based on this, and a point having a predetermined height or more is detected from the road surface plane.
  • An object detection apparatus that detects a corresponding pixel as a target object is disclosed.
  • Patent Document 2 discloses a technique for reducing memory consumption when an obstacle is obtained by Hough transformation. Specifically, it is assumed that the surface corresponding to the obstacle is a plane perpendicular to the road surface. An image processing apparatus that performs Hough conversion while ignoring the Z component is disclosed. Further, Patent Document 2 discloses an image processing apparatus that, when detecting a horizontal plane, determines a candidate whose horizontal vector angle is within a predetermined range as a horizontal plane with respect to a plane candidate to be detected. ing.
  • obstacles can be detected by using a general object detection device or image processing device.
  • the general object detection device and the image processing device as described above are mounted on a vehicle and used for detecting an obstacle around the vehicle in a road environment, the following problems arise. This is a problem.
  • Patent Document 1 The biggest problem is that when the object detection device disclosed in Patent Document 1 is applied to a distance image obtained by imaging the road environment, an obstacle with a height within a certain range from the road surface is accurately detected. It is impossible.
  • FIGS. 2-1 and 2-2 examples of specific processing for detecting an obstacle based on the principle of object detection disclosed in Patent Document 1 are shown in FIGS. 2-1 and 2-2.
  • FIG. 2-1 is a diagram using captured images
  • FIG. 2-2 is a diagram that schematically illustrates the image of FIG. 2-1 for convenience of explanation.
  • FIGS. 2-1 and 2-2 (b) are based on the principle of object detection disclosed in Patent Document 1 from distance images taken in the road environment of FIGS. 2-1 and 2-2 (a). An example of the result of detecting an obstacle is shown.
  • FIGS. 2-1 and 2-2 in this example, as a pedestrian, a first person 11 is present in the foreground, and a second person 11 is present behind the left side. Yes.
  • the range of the pedestrian's calf to the toe approximately 30 cm from the road surface.
  • the road surface is mistakenly determined.
  • the threshold value for the height from the road surface which is used for the determination of the road surface and the obstacle, is lowered, the vicinity of the ankle can be detected.
  • the threshold value is lowered, there is a problem that the ratio of erroneously detecting a pixel corresponding to the road surface as an obstacle increases as described above.
  • Patent Document 2 On the other hand, according to the principle of obstacle detection disclosed in Patent Document 2, it is assumed that the surface of the obstacle is a plane perpendicular to the road surface. For this reason, it is not possible to detect a surface facing obliquely. In Patent Document 2, the horizontal plane is detected based on the angle of the vertical vector of the surface. As described above, in the outdoor road environment, noise is easily applied to the distance value of the pixel corresponding to the road surface. It is difficult to accurately determine the line direction. For this reason, it is difficult to correctly determine the road surface.
  • the present invention provides an obstacle detection device, an obstacle detection method, and an obstacle detection program capable of more accurately detecting an obstacle even in a range as close to the road surface as possible from a distance image capturing a road environment.
  • the purpose is to provide.
  • a distance image acquisition unit that acquires a distance image, and a point in the real world corresponding to each of the pixels based on a pixel value of each pixel included in the distance image
  • a three-dimensional coordinate conversion unit that calculates a three-dimensional coordinate, a road surface equation estimation unit that estimates a parameter of an equation representing a road surface based on the three-dimensional coordinate, and an equation that represents a road surface plane estimated by the road surface equation estimation unit
  • a road surface height estimation unit that estimates the height from the road surface plane for each pixel based on the parameters and the three-dimensional coordinates calculated for each pixel, and the height estimated by the road surface height estimation unit
  • a road surface pixel selecting unit that selects pixels having a value less than a predetermined first threshold as a road surface pixel, and for pixels that are not selected as the road surface pixels, an obstacle based on the pixel value of the pixel It is determined that the road An obstacle detection unit that determines whether a pixel is selected as a nearby pixel based on not
  • a distance image is acquired, and based on a pixel value of each pixel included in the distance image, three-dimensional coordinates of a point on the real world corresponding to each pixel are obtained.
  • Calculating, estimating a parameter of an equation representing a road plane based on the three-dimensional coordinates, and calculating each parameter based on the parameter of the equation representing the estimated road plane and the three-dimensional coordinates calculated for each pixel Estimate the height from the road surface every time, select the pixels whose estimated height is less than a predetermined first threshold as pixels near the road surface, and for pixels not selected as the road surface pixels Is determined to be an obstacle based on the pixel value of the pixel, and for pixels selected as pixels near the road surface, not only the pixel value of the pixel but also the pixel determined to be the obstacle Whether it is an obstacle based on the positional relationship Determining, obstacle detection and wherein the is provided.
  • a distance image acquisition unit that acquires a distance image, and a point on the real world corresponding to each of the pixels based on a pixel value of each pixel included in the distance image
  • a three-dimensional coordinate conversion unit that calculates a three-dimensional coordinate, a road surface equation estimation unit that estimates a parameter of an equation representing a road surface based on the three-dimensional coordinate, and an equation that represents a road surface plane estimated by the road surface equation estimation unit
  • a road surface height estimation unit that estimates the height from the road surface plane for each pixel based on the parameters and the three-dimensional coordinates calculated for each pixel, and the height estimated by the road surface height estimation unit
  • a road surface pixel selecting unit that selects pixels having a value less than a predetermined first threshold as a road surface pixel, and for pixels that are not selected as the road surface pixels, an obstacle based on the pixel value of the pixel It is determined that the road An obstacle detection unit that determines whether a pixel is selected as a nearby pixel based on not
  • the distance value of the point corresponding to the road surface and the estimated road surface equation are determined in consideration of the influence of noise, and the range as close to the road surface as possible from the distance image captured in the road environment is determined. It is also possible to detect obstacles more accurately.
  • FIG. 2 is a simplified schematic diagram of FIG. 2-1. It is a flowchart showing an example of operation
  • FIG. 7 is a simplified schematic diagram of FIG. It is a figure showing the example of an obstacle detection result by embodiment of this invention.
  • FIG. 8 is a simplified schematic diagram of FIG. 8-1. It is a figure showing an example of the composition for realizing the embodiment of the present invention.
  • an obstacle detection apparatus 100 includes a distance image acquisition unit 101, a three-dimensional coordinate conversion unit 102, a road surface equation estimation unit 103, a road surface height estimation unit 104, and a local surface direction.
  • An estimation unit 105, a road surface vicinity pixel selection unit 106, and an obstacle detection unit 107 are included.
  • the obstacle detection device 100 relates to an obstacle detection device that detects an obstacle present in a scene including a road surface (plane) based on a distance image.
  • the pixels near the road surface should be determined based on the height from the road surface.
  • the determination is made based on the characteristics of the obstacle itself in which noise is difficult to be superimposed. That is, the obstacle detection unit 107 determines an obstacle that is estimated to exist above a certain level from the road surface as an obstacle based only on the point, and is estimated to be present below a certain distance value from the road surface. Is determined to be an obstacle when the upper pixel has been determined to be an obstacle and the angle formed by the local normal direction of the surface and the normal direction of the road surface is equal to or greater than a threshold value.
  • the distance image acquisition unit 101 acquires a distance image obtained by converting the distance from the camera that has acquired the distance image to the object in the captured scene into a pixel value. Then, the distance image acquisition unit 101 outputs the acquired distance image to the three-dimensional coordinate conversion unit 102.
  • the distance image acquisition unit 101 can be realized by an arbitrary module.
  • the distance image acquisition unit 101 can be realized by a TOF camera or a distance sensor that measures a distance according to the principle of triangulation by projecting a near infrared pattern.
  • the separated image acquisition unit 101 itself does not have a function as a camera or a distance sensor, and may receive a distance image from a camera or a distance sensor connected to the outside.
  • the three-dimensional coordinate conversion unit 102 converts the distance value represented by the pixel value of the distance image output from the distance image acquisition unit 101 into 3D coordinates of a point on the corresponding object in the scene and outputs the converted 3D coordinate.
  • the focal length of the distance image acquisition unit 101 is f (cm), the number of horizontal pixels of the distance image is a (pixel), the number of pixels in the height direction of the distance image is b (pixel), and the horizontal width of the image plane is ⁇ (cm). ), And the vertical length of the image plane is h (cm).
  • the camera center is the origin
  • the optical axis direction of the distance image acquisition unit 101 is Z (axis)
  • the axis parallel to the vertical axis of the image plane is Y (axis)
  • the axis is parallel to the horizontal axis of the image plane.
  • ⁇ represents a constant representing a scale.
  • the pixel value of the distance image can be converted into a three-dimensional position (X, Y, Z) in the camera center coordinate system.
  • the road surface equation estimation unit 103 acquires the three-dimensional coordinates output from the three-dimensional coordinate conversion unit 102, and estimates and outputs an equation representing a road surface plane existing near the lower part of the distance image. It is easy to express the road surface equation in the same coordinate system as the three-dimensional coordinate output from the three-dimensional coordinate conversion unit 102, that is, the camera center coordinate system.
  • FIG. 3 is a flowchart showing the operation when calculating the road surface equation.
  • an area in the distance image used for calculating the road surface plane is determined (step S301). Any method can be used as the method of determining the region. For example, when the approximate elevation angle and depression angle of the distance image acquisition unit 101 are known, the position of the horizontal line in the distance image is roughly determined. Therefore, the pixel below the horizontal line may be determined as an area used for calculating the road surface plane. Further, it may be determined more simply, for example, a fixed area of the distance image may be used.
  • step S302 three points are randomly selected from the pixels included in the region determined to be used for calculating the road surface in step S301 (step S302).
  • a coefficient of an equation representing a plane passing through the three points is calculated based on the three-dimensional coordinate values corresponding to the three points selected in step S302 (step S303).
  • the number of pixels whose distance from the plane represented by the equation calculated in step S303 is equal to or less than a predetermined road surface equation estimation threshold (corresponding to the “third threshold” of the present invention) is counted. Then, the number of pixels as the counting result is stored in association with the equation calculated in step S303 (step S304).
  • a predetermined road surface equation estimation threshold corresponding to the “third threshold” of the present invention
  • N is an arbitrary integer of 1 or more. Setting a larger number of N increases the possibility of obtaining a more appropriate result, but the number of processing increases accordingly, so it is preferable to set an appropriate number according to the situation.
  • step S302 if the number of repetitions of the processing from step S302 to step S304 is less than N (No in step S305), three new random points are selected, and the processing is repeated from step S302 again.
  • step S305 when N times of processing have been completed (Yes in step S305), the equation when the number of pixels is the largest in step S304 is determined and output as an equation representing the true road surface plane (step S306).
  • step S306 the equation when the number of pixels is the largest is output based on the idea that the equation that correctly represents the road surface plane should have a larger number of pixels whose distance from the road surface plane is less than or equal to the threshold value.
  • the random sampling method such as steps S302 to S305 is called RANSAC (RANdom SAmple Consensus).
  • step S304 may be skipped. As a result, the processing time can be slightly reduced.
  • the road surface height estimation unit 104 calculates, for each point in the distance image, the distance between the corresponding point in the scene and the road surface plane output in step S306. Specifically, the road plane equation is
  • the local surface direction estimation unit 105 calculates, for each point in the distance image, a local surface direction (normal direction) near the corresponding point in the scene.
  • the normal direction of the surface is calculated by the following procedure. First, three points are selected from points existing in the vicinity of the point of interest in the distance image. And the equation of the plane passing through 3 points,
  • ⁇ ⁇ There are various ways to select the three points. For example, as shown in FIG. 4A, a pixel 402 that is in a fixed relative positional relationship around the pixel of interest 401 may be selected, or may be selected randomly. Instead of selecting three points, four pixels 402 around the pixel of interest 401 are selected as shown in FIG. 4B, and coefficients (L, M, N) are calculated using the least square method. You may make it ask.
  • the road surface vicinity pixel selecting unit 106 selects pixels whose distance between the road surface plane output in step S306 and a point on the real world is less than a predetermined first threshold. Sort as a pixel near the road surface. Then, the road surface vicinity pixel selection unit 106 outputs to the obstacle detection unit 107 each of the road surface vicinity pixel and the pixel that has not been selected as the road surface vicinity pixel.
  • the obstacle detection unit 107 detects an obstacle based on the outputs of the local surface direction estimation unit 105 and the road surface vicinity pixel selection unit 106. This will be described with reference to FIG. 5 which is a flowchart showing the obstacle detection process.
  • step S501 all the pixels that are not selected as road surface pixels by the road surface pixel selection unit 106 (that is, pixels whose distance between the road surface plane and a point in the real world is equal to or greater than a predetermined first threshold) are used as obstacles. It determines with corresponding (step S501).
  • other pixels in the vicinity of the road surface that is, pixels in which the distance between the road surface plane and a point in the real world is less than a predetermined first threshold
  • step S502 it is confirmed whether or not it is determined that a pixel adjacent immediately above the focused pixel corresponds to an obstacle (step S502). If the immediately adjacent pixel has already been determined to be an obstacle, it is verified whether the angle formed by the normal direction of the local surface of the pixel and the normal direction of the road surface is equal to or greater than the second threshold (step S503). . If it is equal to or greater than the threshold (Yes in step S503), it is determined that the pixel corresponds to an obstacle (step S504).
  • step S503 it is determined that the pixel is not an obstacle, that is, a pixel corresponding to the road surface (step S505).
  • FIG. 6 is a flowchart showing the overall operation of the obstacle detection apparatus 100.
  • the distance image acquisition unit 101 acquires a distance image (step S601).
  • the three-dimensional coordinate conversion unit 102 converts the distance information obtained for each pixel into three-dimensional coordinates on the object surface (step S602).
  • the road surface equation estimating unit 103 obtains and outputs an equation of a plane estimated to best describe the road surface plane (step S603).
  • the road surface height estimation unit 104 collates the road surface plane output from the road surface equation estimation unit 103 with the 3D coordinates of the point and measures the distance between the point and the road surface plane (step S604).
  • the local surface direction estimation unit 105 calculates the local surface orientation of each point on the image (step S605).
  • the road surface vicinity pixel selection unit 106 extracts pixels whose height estimated by the road surface height estimation unit is less than a predetermined first threshold as road surface vicinity pixels (step S606).
  • the obstacle detection unit 107 detects an obstacle based on the outputs of the local surface direction estimation unit 105 and the road surface vicinity pixel selection unit 106 (step S607).
  • the obstacle detection device 100 can detect an obstacle more accurately by the above operation, and can particularly detect an obstacle in a range close to the road surface with high accuracy. This is because, for pixels that can be determined as an obstacle relatively accurately based on the height from the road surface, a determination is made based on only the pixel value of the pixel, and pixels near the road surface are determined as an obstacle. This is because it is comprehensively determined whether or not the object is an obstacle based on the positional relationship with the pixel and the local surface direction.
  • pixels that are present near the road surface are not determined based on the height from the road surface, but are determined based on the characteristics of the obstacle itself that is difficult to superimpose noise, that is, the direction of the surface. is there. Also, based on the direction of the surface, only the pixels that are adjacent to the pixels that can be clearly determined to be obstacles based on the height from the road surface, or the pixels that are adjacent to the pixels that are determined to be obstacles based on the height from the road surface. This is because the use of pixels reduces the risk of erroneously determining an obstacle even if the surface direction cannot be obtained correctly with respect to the pixel corresponding to the road surface.
  • FIGS. 7-1 and 7-2, Fig. 8-1 and Fig. 8-2 show examples of processing results.
  • FIGS. 7-1 and 7-2 and FIGS. 8-1 and 8-2 are also in the same relationship as FIGS. 2-1 and 2-2, and the images of FIGS. FIGS. 7-2 and 8-2 are schematic diagrams more simply.
  • FIGS. 7-1 and 7-2 show the surface direction of each pixel obtained by the local surface direction estimation unit 105 in the situation of FIGS. 2-1 and 2-2 and the method of the road surface plane output by the road surface equation estimation unit 103.
  • An example of a result of extracting pixels whose angle with the line direction is equal to or greater than a second threshold is shown.
  • FIGS. 8-1 and 8-2 The obstacle detection results according to the present invention are shown in FIGS. 8-1 and 8-2. As can be seen from comparison with FIGS. 2-1 and 2-2 (b), it can be seen that the leg of the person in front and the leg of the person in the back of the left hand can be accurately detected to a point closer to the road surface.
  • the processing after the three-dimensional coordinate conversion unit 102 is expressed by the camera center coordinate system.
  • this is only an example, and it may be expressed by a coordinate system other than the camera center coordinate system.
  • a coordinate system other than the camera center coordinate system.
  • a world coordinate system is defined with respect to the horizontal plane and expressed in the world coordinate system. Since the transformation of the coordinate system can be performed by matrix operations well known to those skilled in the art, detailed description thereof is omitted.
  • the road surface equation estimation unit 103 calculates various plane equation candidates by selecting various three points by RANSAC, and calculates the candidate road surface with the maximum number of pixels whose distance from the plane is equal to or less than a predetermined threshold. The method of finally adopting the equation was explained. In this case, instead of selecting three points, various sets of three points may be selected to obtain an equation representing a plane by three-dimensional Hough transform.
  • step S502 it is determined in step S502 whether it has been determined that the pixel immediately above the pixel of interest corresponds to the obstacle. This may be confirmed as to whether or not it is determined that any one or more of the pixels adjacent to the pixel just above, the upper left, or the upper right is corresponding to the obstacle. By doing so, it becomes possible to better detect the vicinity of the lower part of the obliquely inclined obstacle.
  • the road surface equation estimation unit 103 estimates the parameters of the equation representing the road surface plane based on the three-dimensional coordinates.
  • the optical system internal parameters and external parameters of the distance image acquisition unit 101 are known and the road surface can be assumed to be substantially horizontal, only the parameters of the equation representing the road surface plane may be stored. That is, the road surface equation estimation unit 103 may be omitted from the obstacle detection apparatus 100 illustrated in FIG.
  • FIG. 9 is a block diagram showing a configuration of a computer 1000 for realizing the obstacle detection apparatus 100.
  • the computer 1000 includes an ECU (Electronic Control Unit) 1001, a camera 1002 and a hard disk 1003. By cooperating these units included in the computer 1000, each functional block included in the obstacle detection device 100 can be realized.
  • ECU Electronic Control Unit
  • the ECU 1001 controls the entire computer 1000 and includes, for example, a CPU, a RAM, a ROM, a signal processing circuit, a power supply circuit, and the like.
  • Functional blocks from the three-dimensional coordinate conversion unit 102 to the obstacle detection unit 107 are realized by the ECU 1001.
  • the distance image acquisition unit 101 is realized by the camera 1002.
  • the hard disk 1003 is a device for storing programs and data, and there is no problem even if it is configured by a storage medium other than the hard disk, such as SSD (Solid State Drive) realized by a flash memory.
  • SSD Solid State Drive
  • the hard disk 1003 stores a computer program for realizing each process and determination logic of the flowchart referred to in the description of the obstacle detection unit 107 from the three-dimensional coordinate conversion unit 102.
  • the ECU 1001 reads out the computer program from the hard disk 1003 and performs arithmetic processing, thereby realizing the above-described processes and determination logic.
  • microcomputer by making the functions executed by the ECU 1001 into hardware. Furthermore, some functions may be realized by hardware, and similar functions may be realized by cooperative operation of the hardware and the software program.
  • the ECU 1001, the camera 1002, and the hard disk 1003 are described as being realized by a single computer.
  • the ECU 1001, the camera 1002, and the hard disk 1003 may be realized by another computer and connected by means such as a bus or a cable compliant with the USB standard.
  • the connection may be a wired connection, but part or all of the connection may be a wireless connection.
  • the computer program has been described as being stored in the hard disk 1003 in advance.
  • a program for causing the computer 1000 to operate as all or part of the obstacle detection device 100 or to execute the above-described processing is a flexible disk, a CD-ROM (Compact Disc-Read-Only Memory), a DVD ( Stored in a computer-readable recording medium such as Digital Versatile Disc), MO (Magneto Optical Disk (Disc)) BD (Blu-ray Disc), etc., installed on another computer, and operates as the above-mentioned means Alternatively, the above-described steps may be executed.
  • the program may be stored in a disk device or the like of a server device on the Internet, and for example, the program may be superimposed on a carrier wave and downloaded to a computer to execute the program.
  • a distance image acquisition unit that acquires a distance image; Based on the pixel value of each pixel included in the distance image, a three-dimensional coordinate conversion unit that calculates three-dimensional coordinates of a point on the real world corresponding to each of the pixels;
  • a road surface equation estimating unit for estimating parameters of an equation representing a road surface based on the three-dimensional coordinates;
  • Road surface height estimation for estimating the height from the road surface plane for each pixel based on the parameters of the equation representing the road surface plane estimated by the road surface equation estimation unit and the three-dimensional coordinates calculated for each pixel
  • a road surface vicinity pixel selection unit that selects pixels whose estimated height of the road surface height estimation unit is less than a predetermined first threshold as road surface pixels;
  • a pixel that is not selected as a pixel near the road surface is determined to be an obstacle based on the pixel value of the pixel, and a pixel that is selected as a pixel near the road surface is determined only by the pixel value of the pixel.
  • Additional remark 2 It is an obstacle detection apparatus of Additional remark 1, Comprising: A local surface direction estimation unit that estimates a normal direction of a local surface in the real world from the three-dimensional coordinates calculated for each pixel; The obstacle detection unit determines whether or not the pixel selected as the road surface vicinity pixel is an obstacle in consideration of the estimation result of the local surface direction estimation unit. Detection device.
  • An obstacle detection apparatus of Additional remark 2, Comprising: The angle formed between the normal direction of the local surface estimated by the local surface direction estimation unit and the normal direction of the road surface plane indicated by the parameter of the equation representing the road surface plane is equal to or greater than a predetermined second threshold.
  • An obstacle detection device characterized in that if it is an obstacle, it is determined that the object is not an obstacle if it is less than a second threshold value.
  • the obstacle detection device (Supplementary note 4) The obstacle detection device according to any one of supplementary notes 1 to 3, The obstacle detection unit further satisfies the condition for the pixel selected as the pixel in the vicinity of the road surface on the condition that the pixel adjacent to the upper side of the pixel has been determined as an obstacle. In this case, the obstacle detection apparatus determines that the pixel is an obstacle.
  • the obstacle detection device is any one of a pixel adjacent to the upper side, a pixel adjacent to the upper right side, and a pixel adjacent to the upper left side of the pixel.
  • An obstacle detection device characterized in that a further condition is that one or more pixels have been determined as an obstacle, and the pixel is determined as an obstacle when the condition is further satisfied.
  • the obstacle detection device is An area used for calculation of a road surface plane is determined, a coefficient of an equation representing a plane passing through the plurality of points is calculated using a plurality of points randomly extracted from the determined area, and the calculated equation represents The process of counting the number of pixels whose distance from the plane is equal to or less than a predetermined third threshold is repeated N (an integer greater than or equal to 1) times, and the equation when the number of counted pixels is the largest is An obstacle detection apparatus characterized by determining an equation representing a true road surface plane.
  • a further condition is that a pixel adjacent to the upper side of the pixel has been determined as an obstacle, and the condition An obstacle detection method characterized in that the pixel is determined as an obstacle when the above is further satisfied.
  • the obstacle detection method according to any one of supplementary notes 7 to 9, In determining whether or not the obstacle is detected, for the pixels selected as the pixels near the road surface, the pixel adjacent to the upper side, the pixel adjacent to the upper right side, and the pixel adjacent to the upper left side of the pixel An obstacle detection method characterized by further determining that any one or more of the pixels have been determined as an obstacle and determining the pixel as an obstacle when the condition is further satisfied.
  • a distance image acquisition unit that acquires a distance image; Based on the pixel value of each pixel included in the distance image, a three-dimensional coordinate conversion unit that calculates three-dimensional coordinates of a point on the real world corresponding to each of the pixels;
  • a road surface equation estimating unit for estimating parameters of an equation representing a road surface based on the three-dimensional coordinates;
  • Road surface height estimation for estimating the height from the road surface plane for each pixel based on the parameters of the equation representing the road surface plane estimated by the road surface equation estimation unit and the three-dimensional coordinates calculated for each pixel
  • a road surface vicinity pixel selection unit that selects pixels whose estimated height of the road surface height estimation unit is less than a predetermined first threshold as road surface pixels;
  • a pixel that is not selected as a pixel near the road surface is determined to be an obstacle based on the pixel value of the pixel, and a pixel that is selected as a pixel near the road surface is determined only by the pixel value of the pixel.
  • the obstacle detection program according to supplementary note 13, wherein The computer, A local surface direction estimation unit that estimates a normal direction of a local surface in the real world from the three-dimensional coordinates calculated for each pixel; The obstacle detection unit further functions as an obstacle detection device that determines whether the pixel selected as the road vicinity pixel is an obstacle in consideration of the estimation result of the local surface direction estimation unit. Obstacle detection program characterized by causing
  • the obstacle detection program according to supplementary note 14, The angle formed between the normal direction of the local surface estimated by the local surface direction estimation unit and the normal direction of the road surface plane indicated by the parameter of the equation representing the road surface plane is equal to or greater than a predetermined second threshold.
  • An obstacle detection program characterized in that it is determined as an obstacle if it is less than the second threshold and is not an obstacle.
  • the obstacle detection program according to any one of supplementary notes 13 to 15, The obstacle detection unit further satisfies the condition for the pixel selected as the pixel in the vicinity of the road surface on the condition that the pixel adjacent to the upper side of the pixel has been determined as an obstacle.
  • An obstacle detection program characterized by determining the pixel as an obstacle.
  • the obstacle detection program according to any one of supplementary notes 13 to 15,
  • the obstacle detection unit is any one of a pixel adjacent to the upper side, a pixel adjacent to the upper right side, and a pixel adjacent to the upper left side of the pixel.
  • An obstacle detection program characterized in that a further condition is that one or more pixels have been determined as an obstacle, and the pixel is determined as an obstacle when the condition is further satisfied.
  • the obstacle detection program according to any one of supplementary notes 13 to 17,
  • the road surface equation estimator is An area used for calculation of a road surface plane is determined, a coefficient of an equation representing a plane passing through the plurality of points is calculated using a plurality of points randomly extracted from the determined area, and the calculated equation represents The process of counting the number of pixels whose distance from the plane is equal to or less than a predetermined third threshold is repeated N (an integer greater than or equal to 1) times, and the equation when the number of counted pixels is the largest is An obstacle detection program characterized by determining an equation representing a true road surface plane.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

La présente invention a pour objet de détecter plus précisément, même dans une plage de valeurs se rapprochant au maximum de la surface d'une route, un obstacle à partir d'une image de distance qui capture un environnement de route. Une image de distance est acquise. Des coordonnées tridimensionnelles de points dans le monde réel correspondant à chaque pixel inclus dans l'image de distance sont calculées sur la base des valeurs de pixel de chaque pixel. Un paramètre d'une équation qui représente un plan de la surface d'une route est estimé en fonction des coordonnées tridimensionnelles. La hauteur pour chaque pixel à partir du plan de la surface d'une route est évaluée sur la base du paramètre estimé de l'équation qui représente le plan de la surface d'une route et des coordonnées tridimensionnelles qui sont calculées pour chaque pixel. Un pixel dans lequel la hauteur est inférieure à un premier seuil prédéterminé est sélectionné comme un pixel proche de la surface d'une route. Une détermination est effectuée pour un pixel qui n'est pas sélectionné comme le pixel proche de la surface d'une route étant un obstacle en fonction de la valeur de pixel du pixel. Une détermination est effectuée pour le pixel qui est sélectionné comme le pixel proche de la surface d'une route afin de savoir si ledit pixel est un obstacle en fonction également de la relation d'emplacement entre ledit pixel et le pixel qui est déterminé comme étant l'obstacle, ainsi que de la valeur de pixel dudit pixel.
PCT/JP2012/071951 2011-09-09 2012-08-30 Dispositif de détection d'obstacle, procédé de détection d'obstacle, et programme de détection d'obstacle WO2013035612A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2011197256 2011-09-09
JP2011-197256 2011-09-09

Publications (1)

Publication Number Publication Date
WO2013035612A1 true WO2013035612A1 (fr) 2013-03-14

Family

ID=47832059

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2012/071951 WO2013035612A1 (fr) 2011-09-09 2012-08-30 Dispositif de détection d'obstacle, procédé de détection d'obstacle, et programme de détection d'obstacle

Country Status (1)

Country Link
WO (1) WO2013035612A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017036995A (ja) * 2015-08-10 2017-02-16 朝日航洋株式会社 レーザ計測システムおよびレーザ計測方法
JP6164546B1 (ja) * 2016-11-07 2017-07-19 クモノスコーポレーション株式会社 測量方法及び測量装置
CN108885791A (zh) * 2018-07-06 2018-11-23 深圳前海达闼云端智能科技有限公司 地面检测方法、相关装置及计算机可读存储介质
JP2020098188A (ja) * 2018-09-27 2020-06-25 バイドゥ オンライン ネットワーク テクノロジー （ベイジン） カンパニー リミテッド 障害物検出方法、障害物検出装置、電子機器、車両及び記憶媒体
WO2021005659A1 (fr) * 2019-07-05 2021-01-14 パナソニックセミコンダクターソリューションズ株式会社 Système de traitement d'informations, système de capteurs, procédé de traitement d'informations et programme
CN112561874A (zh) * 2020-12-11 2021-03-26 杭州海康威视数字技术股份有限公司 一种遮挡物检测方法、装置及监控摄像机
JP2021085828A (ja) * 2019-11-29 2021-06-03 株式会社豊田自動織機 障害物検出装置
JPWO2021199609A1 (fr) * 2020-04-03 2021-10-07

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10143659A (ja) * 1996-11-06 1998-05-29 Komatsu Ltd 物体検出装置
JP2002352249A (ja) * 2001-05-23 2002-12-06 Toshiba Corp 画像処理装置及びその方法
JP2005258941A (ja) * 2004-03-12 2005-09-22 Toyota Central Res & Dev Lab Inc 障害物検出装置
JP2006349607A (ja) * 2005-06-20 2006-12-28 Toyota Central Res & Dev Lab Inc 距離計測装置
JP2008026998A (ja) * 2006-07-18 2008-02-07 Sumitomo Electric Ind Ltd 障害物位置算出システム、及び障害物位置算出方法
JP2009140023A (ja) * 2007-12-03 2009-06-25 Mazda Motor Corp 車両用障害物検出装置
JP2010181919A (ja) * 2009-02-03 2010-08-19 Toyohashi Univ Of Technology 三次元形状特定装置、三次元形状特定方法、三次元形状特定プログラム

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10143659A (ja) * 1996-11-06 1998-05-29 Komatsu Ltd 物体検出装置
JP2002352249A (ja) * 2001-05-23 2002-12-06 Toshiba Corp 画像処理装置及びその方法
JP2005258941A (ja) * 2004-03-12 2005-09-22 Toyota Central Res & Dev Lab Inc 障害物検出装置
JP2006349607A (ja) * 2005-06-20 2006-12-28 Toyota Central Res & Dev Lab Inc 距離計測装置
JP2008026998A (ja) * 2006-07-18 2008-02-07 Sumitomo Electric Ind Ltd 障害物位置算出システム、及び障害物位置算出方法
JP2009140023A (ja) * 2007-12-03 2009-06-25 Mazda Motor Corp 車両用障害物検出装置
JP2010181919A (ja) * 2009-02-03 2010-08-19 Toyohashi Univ Of Technology 三次元形状特定装置、三次元形状特定方法、三次元形状特定プログラム

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017036995A (ja) * 2015-08-10 2017-02-16 朝日航洋株式会社 レーザ計測システムおよびレーザ計測方法
JP6164546B1 (ja) * 2016-11-07 2017-07-19 クモノスコーポレーション株式会社 測量方法及び測量装置
JP2018077065A (ja) * 2016-11-07 2018-05-17 クモノスコーポレーション株式会社 測量方法及び測量装置
CN108072353A (zh) * 2016-11-07 2018-05-25 蛛巢株式会社 测量方法以及测量装置
CN108072353B (zh) * 2016-11-07 2020-06-02 蛛巢株式会社 测量方法以及测量装置
US10690478B2 (en) 2016-11-07 2020-06-23 Kumonos Corporation Survey method and survey apparatus
CN108885791A (zh) * 2018-07-06 2018-11-23 深圳前海达闼云端智能科技有限公司 地面检测方法、相关装置及计算机可读存储介质
US11393219B2 (en) 2018-09-27 2022-07-19 Apollo Intelligent Driving Technology (Beijing) Co., Ltd. Method and apparatus for detecting obstacle, electronic device, vehicle and storage medium
JP2020098188A (ja) * 2018-09-27 2020-06-25 バイドゥ オンライン ネットワーク テクノロジー （ベイジン） カンパニー リミテッド 障害物検出方法、障害物検出装置、電子機器、車両及び記憶媒体
JP7395301B2 (ja) 2018-09-27 2023-12-11 アポロ インテリジェント ドライビング テクノロジー（ペキン）カンパニー リミテッド 障害物検出方法、障害物検出装置、電子機器、車両及び記憶媒体
WO2021005659A1 (fr) * 2019-07-05 2021-01-14 パナソニックセミコンダクターソリューションズ株式会社 Système de traitement d'informations, système de capteurs, procédé de traitement d'informations et programme
JP7372970B2 (ja) 2019-07-05 2023-11-01 ヌヴォトンテクノロジージャパン株式会社 情報処理システム、センサシステム、情報処理方法及びプログラム
JPWO2021005659A1 (fr) * 2019-07-05 2021-01-14
JP2021085828A (ja) * 2019-11-29 2021-06-03 株式会社豊田自動織機 障害物検出装置
JP7230787B2 (ja) 2019-11-29 2023-03-01 株式会社豊田自動織機 障害物検出装置
JPWO2021199609A1 (fr) * 2020-04-03 2021-10-07
WO2021199609A1 (fr) * 2020-04-03 2021-10-07 日立Astemo株式会社 Dispositif de détection d'objet et procédé de détection d'objet
CN112561874A (zh) * 2020-12-11 2021-03-26 杭州海康威视数字技术股份有限公司 一种遮挡物检测方法、装置及监控摄像机

Similar Documents

Publication Publication Date Title
WO2013035612A1 (fr) Dispositif de détection d'obstacle, procédé de détection d'obstacle, et programme de détection d'obstacle
US8976999B2 (en) Vehicle detection apparatus
WO2017057058A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
RU2016150826A (ru) Оценка глубины с использованием многоракурсного стереоизображения и откалиброванного проектора
US10650535B2 (en) Measurement device and measurement method
JP5834933B2 (ja) 車両位置算出装置
JP6638723B2 (ja) 画像解析装置、画像解析方法、及び、画像解析プログラム
EP3716210A1 (fr) Procédé de génération de données de groupe de points tridimensionnels, procédé d'estimation de position, dispositif de génération de données de groupe de points tridimensionnels et dispositif d'estimation de position
US20150310296A1 (en) Foreground region extraction device
JP2015184929A (ja) 立体物検出装置、立体物検出方法、および立体物検出プログラム
US20200193184A1 (en) Image processing device and image processing method
JP2012252501A (ja) 走行路認識装置及び走行路認識用プログラム
JP2018073275A (ja) 画像認識装置
JP6577595B2 (ja) 車両用外界認識装置
JP2005156199A (ja) 車両検知方法及び車両検知装置
JP2010020404A (ja) 画像処理装置及びその方法
JP5903901B2 (ja) 車両位置算出装置
JP5785515B2 (ja) 歩行者検出装置及び方法、並びに車両用衝突判定装置
JP7064400B2 (ja) 物体検知装置
JP5891751B2 (ja) 画像間差分装置および画像間差分方法
JP5272818B2 (ja) 画像処理装置および方法、並びに、プログラム
JP6677142B2 (ja) 駐車枠認識装置
JP2021043141A (ja) 物体距離推定装置及び物体距離推定方法
US20180268228A1 (en) Obstacle detection device
JP3931885B2 (ja) 障害物検知装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12830638

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12830638

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP