JP2011209896A - Obstacle detecting apparatus, obstacle detecting method, and obstacle detecting program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an obstacle detecting apparatus, wherein shadows of various peripheral objects that exist on road surface are prevented from being determined to be real obstacles, thereby properly detecting existence of obstacles, when detecting obstacles from images captured by onboard camera installed on a moving object moving on a road.SOLUTION: The obstacle detecting apparatus includes: an imager which is installed on a moving object moving on the road and used for photographing its periphery; an obstacle candidate region extractor which extracts partial regions that may correspond to obstacles from photographed images as obstacle candidate regions; an imaging direction extractor which outputs a direction of an optical axis of the imager; a light source direction extractor which outputs a direction to the location of the light source; a light source direction edge extractor which extracts edge pixels directed to the same direction as a direction to the location of the light source from the obstacle candidate region, assuming that pixels in the obstacle candidate region correspond to points on the road plane in real world; and an obstacle determination unit which determines an obstacle candidate region to be an obstacle region if a ratio of the number of edge pixels extracted by the light source direction edge extractor to the total number of edge pixels is not more than a predetermined value.

Description

  The present invention relates to an obstacle detection device, an obstacle detection method, and an obstacle detection program for detecting an obstacle present on a road surface using a single camera installed on a moving body moving on a road surface. The present invention relates to an obstacle detection device, an obstacle detection method, and an obstacle detection program capable of accurately discriminating shadows and obstacles of an object formed on a road surface.

  In an obstacle detection device that detects obstacles that exist in the vicinity by a single camera installed on a moving body, it is an important issue to distinguish between the shadow of an object on the road surface and the obstacle. There are Patent Documents 1 to 3 and the like as related techniques that can be applied to solve the problem.

  In Patent Document 1, a shadow area is extracted using color information, and a vehicle periphery that integrates a shadow area and a non-shadow area belonging to the same object based on color information of pixels near the boundary between the shadow area and the sunny area An image providing apparatus is disclosed.

  The related art described in Patent Document 1 is effective when the surface color of the road surface is a single color, that is, when the reflectance of the road surface is constant, but when the road surface is composed of regions of various colors, Since it becomes difficult to distinguish whether the color information is due to the amount of light source (ie, shadow area or sunny area) or due to changes in road surface reflectance, There are limits. In addition, in outdoor sunlight, the spectral distribution of incident light differs between the shadow area and the sunny area due to the scattering of short-wavelength light in the atmosphere and the reflection of light from objects around the road surface. In the method of this patent document that has not been made, it becomes more difficult to distinguish between a shadow formed on the road surface and an obstacle.

  On the other hand, related techniques described in Patent Documents 2 and 3 exist as methods for determining a shadow region using a principle different from that of Patent Document 1.

  Patent Document 2 provides a vehicle peripheral image that specifies a shadow region of the host vehicle included in the in-vehicle camera image based on the current position of the sun and the position and shape information of the host vehicle, and displays an image in which the shadow is corrected. An apparatus is disclosed. Further, Patent Document 3 discloses a graphic recognition device that performs graphic recognition while estimating shadow areas caused by the host vehicle and other vehicles using shape information of surrounding vehicles.

  All of the methods described in these patent documents are based on the principle of accurately predicting the shape of the shadow region generated by the vehicle based on the current position of the sun and the shape information of the own vehicle or other vehicle.

JP 2007-272292 A (FIG. 1) JP 2007-300559 A (FIG. 3) Japanese Patent Application No. 2007-554840 (FIG. 1)

  However, when the related techniques disclosed in Patent Documents 2 and 3 are used for the obstacle detection device, the following problems occur.

  First, in these related techniques, a shadow region caused by an object other than a vehicle cannot be distinguished from an obstacle. This is because, since these related technologies use the vehicle shape information expressed in the 3D model to estimate the position where the shadow appears, it is not possible to predict the shadow region due to an object other than the vehicle that has not been 3D modeled. This is because it cannot be done. If the related technology is applied to other than the vehicle, all the surrounding buildings, road structures, etc. must be modeled in 3D. There is a problem that processing and cost occur.

  In addition, as a more essential problem, it is extremely difficult to apply these related techniques to objects such as roadside trees and pedestrians whose shape and position change every moment. This is because, in order to accurately predict a shadow area using these techniques, it is assumed that the shape and position of an object can be accurately grasped.

(Object of invention)
In order to solve the above-described problem, the object of the present invention is to distinguish the shadows and obstacles of various peripheral objects formed on the road surface from the camera image mounted on the moving body without performing 3D modeling of the peripheral objects. Another object is to provide an obstacle detection device, an obstacle detection method, and an obstacle detection program that can detect an obstacle with high accuracy.

  The first obstacle detection apparatus according to the present invention includes an imaging unit mounted on a moving body that moves on a road surface, and a partial area that may correspond to an obstacle in the captured image. Obstacle candidate area extracting means for extracting as, imaging orientation extracting means for outputting the orientation of the optical axis of the imaging means, light source orientation extracting means for outputting the orientation in which the light source exists, and pixels in the obstacle candidate area Is assumed to correspond to a point on the road surface in the real world, and a light source direction edge extracting means for extracting edge pixels facing the direction equal to the direction in which the light source exists from the obstacle candidate region, and the light source Obstacle determining means for determining that the obstacle candidate area is an obstacle area when the ratio of the number of edge pixels extracted by the direction edge extracting means to the total number of edge pixels is equal to or less than a predetermined value.

  The second obstacle detection device of the present invention is mounted on a moving body that moves on the road surface, and there is an image pickup unit that captures a surrounding color image including a plurality of wavelengths, and may correspond to an obstacle from the color image. Obstacle candidate area extraction means for extracting a partial area as an obstacle candidate area, imaging orientation extraction means for outputting the orientation of the optical axis of the imaging means, light source orientation extraction means for outputting an orientation in which a light source exists, Assuming that the pixels in the obstacle candidate area correspond to points on the road surface in the real world, the light source direction for extracting edge pixels facing the same direction as the direction in which the light source exists from the obstacle candidate area An edge extraction unit, a light source spectral characteristic acquisition unit that acquires and stores the intensity of direct sunlight and scattered light from the atmosphere for a plurality of wavelengths, and an edge pixel extracted by the light source direction edge extraction unit The difference between the intensity ratio of the light of the wavelength included in the color image of the two pixels extracted from the regions on both sides of the edge pixel and the intensity ratio of the light in the sun and shade defined by the intensity of the direct light and scattered light is Obstacle determining means for obtaining the number of edge pixels below a certain value and determining the obstacle candidate area as an obstacle area when the ratio of the number of edge pixels to the total number of edge pixels is equal to or less than a predetermined value Including.

  In the first obstacle detection method of the present invention, a partial area that may correspond to an obstacle from surrounding captured images taken by an imaging unit mounted on a moving body that moves on a road surface is set as an obstacle candidate area. An obstacle candidate region extracting step to extract, an imaging orientation extracting step for outputting the orientation of the optical axis of the imaging means, a light source orientation extracting step for outputting an orientation in which a light source exists, and pixels in the obstacle candidate region are A light source direction edge extraction step that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate region, assuming that it corresponds to a point on the road surface in the real world, and the light source direction An obstacle determination step for determining the obstacle candidate area as an obstacle area when the ratio of the number of edge pixels extracted in the edge extraction step to the total number of edge pixels is equal to or less than a predetermined value. With the door.

  According to the second obstacle detection method of the present invention, a partial area that may correspond to an obstacle is detected from surrounding color images including a plurality of wavelengths captured by an imaging unit mounted on a moving body moving on a road surface. An obstacle candidate area extracting step for extracting as an object candidate area; an imaging azimuth extracting step for outputting the azimuth of the optical axis of the imaging means; a light source azimuth extracting step for outputting an azimuth in which a light source exists; and the obstacle candidate area A light source direction edge extraction step for extracting edge pixels facing in the same direction as the direction in which the light source exists from the obstacle candidate region, assuming that the pixels in the road correspond to points on the road surface in the real world; The light source spectral characteristic acquisition step for acquiring and storing the intensity of the direct light of sunlight and the scattered light from the atmosphere for a plurality of wavelengths, and the edge extracted in the light source direction edge extraction step Among the elements, the intensity ratio of light of a wavelength included in the color image of two pixels extracted from the regions on both sides of the edge pixel and the intensity ratio of light in the sun and shade defined by the intensity of the direct light and scattered light The number of edge pixels whose difference from the predetermined value is less than a certain value is determined, and the obstacle candidate area is determined as an obstacle area when the ratio of the number of edge pixels to the total number of edge pixels is equal to or less than a predetermined value. An obstacle determination step.

  The first obstacle detection program of the present invention uses, as an obstacle candidate area, a partial area that may correspond to an obstacle from surrounding captured images taken by an imaging unit mounted on a moving body that moves on a road surface. Obstacle candidate region extraction processing to extract, imaging orientation extraction processing to output the orientation of the optical axis of the imaging means, light source orientation extraction processing to output the orientation in which the light source exists, and pixels in the obstacle candidate region Assuming that it corresponds to a point on the road surface in the real world, a light source direction edge extraction process for extracting edge pixels facing the same direction as the direction in which the light source exists from the obstacle candidate region, and the light source direction An obstacle determination process for determining the obstacle candidate area as an obstacle area when the ratio of the number of edge pixels extracted in the edge extraction process to a predetermined value or less is equal to or less than a predetermined value; To be executed in.

  According to the second obstacle detection program of the present invention, a partial area that may correspond to an obstacle in a surrounding color image including a plurality of wavelengths captured by an imaging unit mounted on a moving body moving on a road surface is obstructed. Obstacle candidate region extraction processing to extract as an object candidate region, imaging orientation extraction processing to output the orientation of the optical axis of the imaging means, light source orientation extraction processing to output the orientation in which a light source exists, and the obstacle candidate region Assuming that the pixels inside correspond to points on the road surface plane in the real world, light source direction edge extraction processing for extracting edge pixels facing in the same direction as the direction in which the light source exists from the obstacle candidate area; Among the edge pixels extracted by the light source spectral characteristic acquisition process for acquiring and storing the intensity of the direct light of sunlight and the scattered light from the atmosphere for a plurality of wavelengths, and the edge pixel extracted by the light source direction edge extraction process, an edge The difference between the light intensity ratio of the wavelengths included in the color image of the two pixels extracted from the regions on both sides of the element and the light intensity ratio in the sun and shade defined by the intensity of the direct light and scattered light is constant. An obstacle determination step of determining the number of edge pixels equal to or less than a value and determining the obstacle candidate area as an obstacle area when a ratio of the number of edge pixels to the total number of edge pixels is equal to or less than a predetermined value; , Execute on the computer.

  According to the present invention, it is possible to distinguish an obstacle from a shadow formed on a road surface by various objects from one camera image mounted on a moving body without performing 3D modeling of surrounding buildings and road structures. However, it can be detected with high accuracy.

It is a block diagram which shows the structure of the obstruction detection apparatus by the 1st Embodiment of this invention. It is a figure for demonstrating the method of matching the edge point on an image with the point on a road plane. It is a figure for demonstrating how to obtain | require the direction of the edge on the road plane with respect to the edge point on an image. It is a figure which shows the shape of the silhouette of the typical object seen from the position of the light source. FIG. 5 is a diagram illustrating an example of a shape of a shadow formed on a road surface plane with respect to an object having the silhouette of FIG. 4. It is a flowchart explaining operation | movement of the obstruction detection apparatus by the 1st Embodiment of this invention. It is a figure which shows the hardware structural example of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the obstruction detection apparatus by the 2nd Embodiment of this invention. It is a figure for demonstrating the setting method of the pixel which refers color information. It is a flowchart explaining operation | movement of the obstruction detection apparatus by the 2nd Embodiment of this invention. It is a flowchart explaining the processing content of the obstacle determination module in the 2nd Embodiment of this invention. It is a figure which shows the example of the local edge direction of the shadow on a road surface.

  Embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
Referring to FIG. 1, the obstacle detection device according to the first embodiment includes an imaging module 101 mounted on a moving body, and a partial area that may correspond to an obstacle in an image output by the imaging module 101. An obstacle candidate region extraction module 102 that extracts as an obstacle candidate region, an imaging orientation extraction module 103 that outputs the orientation imaged by the imaging module 101, a light source orientation extraction module 104 that extracts a light source orientation, and an obstacle A light source direction edge extraction module 105 that extracts edge pixels having an edge direction facing the direction in which the light source exists from the candidate region, and whether the obstacle candidate region is an obstacle region based on the ratio of the number of edges facing the light source direction An obstacle determination module 106 for determining whether or not is included is included.

  The moving body assumed in this embodiment is an object that moves on a road surface such as a vehicle or a robot.

  The imaging module 101 is composed of one camera mounted on a moving body that moves on the road surface. The imaging module 101 is installed, for example, rearward behind the moving body, and images the visual field range including the rear road surface. The captured image is used to detect an obstacle behind when the moving body moves backward. Needless to say, in the case of detecting an obstacle in front of the moving body, an imaging module is installed so as to photograph the forward direction in the front portion of the moving body. The image captured by the imaging module 101 is output to the obstacle candidate area extraction module 102.

  The obstacle candidate area extraction module 102 extracts position information of an obstacle candidate area that is a partial area that may correspond to the obstacle from the image input from the imaging module 101. There may be only one partial area or a plurality of partial areas.

  Further, the shape of the partial area may be rectangular, circular, or indefinite. In the case of a rectangle, for example, coordinate values of an upper left corner and a lower right corner of the rectangle are used as position information. In the case of a circle, the center information of the circle and its radius are used as position information. In the case of the indefinite form, for example, the coordinate values of the pixels at the boundary of the partial area may be expressed in a list form, or a chain code may be used.

  As a specific method of extracting a partial area that may be an obstacle, a pixel having a large difference in pixel value between adjacent pixels (hereinafter referred to as an edge pixel) is differentiated by a differential filter such as a Sobel filter or a Roberts filter. Thus, it is possible to use a method of extracting a small region in which many of them exist. In addition, classifiers that have been learned to discriminate obstacles from road surfaces using statistical learning techniques are applied to various small areas of the image to extract small areas that are likely to be obstacles from the output of the classifier. It is possible to use the method to do.

  As another method, a point that is estimated to correspond to a position above the road plane in the real world by applying a monocular stereoscopic technique to the video input from the imaging module 101 every moment. It is conceivable to extract a small region including many.

  It is highly likely that the obstacle candidate area extraction module 102 will extract an area including the shadow of the object as an obstacle candidate area. This is because, for example, in a method for obtaining a pixel having a large difference in pixel value between adjacent pixels by the differential filter calculation, a pixel corresponding to a shadow boundary is also extracted. Even when using a statistical learning method, it is difficult to learn to distinguish shadow areas and obstacles that can have a wide variety of shapes. There is a high possibility of extraction as a candidate region. When using the monocular stereoscopic technique, the shadow of a stationary object is not determined as an obstacle candidate area, but in principle, 3D measurement cannot be performed on a shadow generated by an animal body. Is likely to extract as.

  The imaging azimuth extraction module 103 is composed of an azimuth sensor installed on a moving body, and outputs the azimuth of the optical axis of the imaging module 101. Here, the azimuth unit and the reference direction are design problems. For example, assuming that the north direction is 0, a negative angle with respect to the east direction from the north direction (for example, − 90 degrees), and a positive angle with respect to the direction from the west (for example, 90 degrees with respect to the true west) may be output as the direction.

  Originally, the azimuth is 360 degrees, but here, the azimuth opposite 180 degrees may be expressed as the same value. That is, for example, the same value may be output for the northeast azimuth and the southwest azimuth. The reason is that the light source direction edge extraction module 105, which will be described later, treats the direction opposite to 180 degrees without distinguishing.

  The light source orientation extraction module 104 is a module that outputs the orientation of the light source. The main light source in the daytime outdoors is the sun. Therefore, the light source direction extraction module 104 acquires the current date and time by the built-in clock function, acquires the current position by the built-in GPS function, and applies the parameters obtained from these to the calculation formula for calculating the position of the sun. The sun's direction is calculated and output.

  Various formulas for calculating the direction of the sun have been proposed and published on the Internet and the like (for example, http://www.es.ris.ac.jp/˜nakagawa/met_cal/solar.html). . Also, if it is known that there is intense lighting fixed to the building as a light source, based on the position information of the light source installed in the building and the current position of the moving body itself, It is possible to calculate the orientation of the light source.

  The light source direction edge extraction module 105 assumes that the edge pixels in the region correspond to points on the road surface in the real world with respect to the obstacle candidate region in the input image extracted by the obstacle candidate region extraction module 102. Then, an edge pixel having an edge direction that matches the orientation of the light source output by the light source orientation extraction module 104 is extracted.

  Hereinafter, a specific method for realizing the light source direction edge extraction module 105 will be described in detail with reference to the drawings.

  FIG. 2 is a diagram for explaining a method of making edge pixels in the obstacle candidate area correspond to points on the road surface in the real world.

  When an edge pixel 203 is extracted from the obstacle candidate area 202, a line segment connecting the viewpoint (camera center) 201 and the edge pixel 203 is used in order to make the edge pixel 203 correspond to a point on the road surface in the real world. Is extended to the road surface plane 204, and the intersection point may be set as the edge point 205. That is, the edge point 205 is a corresponding point when the edge point 203 in the obstacle candidate region is made to correspond to a point on the road surface.

  Here, the edge pixel refers to a pixel at a location where the pixel value of an adjacent pixel is greatly changed. As a specific edge pixel extraction method, for example, a differential filter such as a Sobel filter or a Prewitt filter is convoluted with an image in an obstacle candidate region, and the absolute value of the filter output is determined in advance. There is a method of extracting pixels that are equal to or greater than the threshold value. Further, the position of the edge point 205 can be easily realized by using a widely known technique called homography. The coordinate system representing the position on the road surface may be set arbitrarily. The homography matrix representing the coefficient of geometric transformation can be determined and stored in advance if the positional relationship between the imaging module 101 and the road surface plane is determined.

  Next, the edge direction with respect to the edge point 205 is obtained.

  First, the edge direction of the edge pixel 203 on the image is obtained from the output of a differential filter such as a Sobel filter or a Prewitt filter as follows.

  Here, lx (x, y) and ly (x, y) indicate the gradient amount of the pixel value in the x-axis and y-axis directions of the pixels around the edge pixel. That is, the direction orthogonal to the pixel gradient direction is the edge direction on the image. According to the above equation, for example, θ = 0 degrees when the direction orthogonal to the maximum direction of the gradient of the pixel value is directed to the x-axis direction, and θ when the direction is the direction of the straight line y = x. = -45 degrees.

  Next, the edge direction on the image is converted into a direction on the road surface plane. An example of conversion will be described with reference to FIG.

  The points 301 and 302 existing on both sides of the edge direction 308 around the edge pixel 307 on the image are virtually obtained. Next, the points 303 and 304 are obtained at the corresponding points when the points 301 and 302 in the obstacle candidate area of the input image correspond to the points on the road surface. Here, points 303 and 304 on the road surface plane corresponding to these points 301 and 302 on the road surface coordinate system 305 in which the vertical axis or the horizontal axis is aligned with the optical axis direction of the imaging module 101 are referred to as the homography described above. Obtained from the matrix, the direction of the line segment connecting the points 303 and 304 is obtained. At this time, like the imaging orientation extraction module 103, it is not necessary to distinguish from the opposite direction of 180 degrees, and the definition area may be limited to 180 degrees.

  Then, in order to facilitate the comparison between the direction in which the light source exists and the direction of the edge, the position and direction of the edge are changed from east to west in consideration of the deviation between the azimuth of the optical axis of the imaging module 101 and the azimuth direction The north-south direction is expressed by a road surface coordinate system 306 in which the respective directions are aligned with the axial direction. At this time, the unit for expressing the direction of the edge, the reference orientation, and the like may be freely set. Further, the resolution of the direction does not need to be in units of 1 degree, and may be expressed by a more coarsely quantized direction, for example, an amount quantized every 45 degrees.

  In the above description, the procedure for obtaining the edge direction on the input image and then converting the edge direction to the position / direction on the road surface is shown. However, after the input image is first converted to the bird's-eye image, the bird's-eye view is converted. Edge extraction processing may be performed on the image to determine the edge position and direction.

  Then, the direction of the edge on the road surface plane obtained as described above is compared with the azimuth in which the light source exists, and an edge directed in the same direction as the azimuth in which the light source exists is extracted. At the time of comparison, it is possible to determine in advance a difference (allowable amount) between directions that are allowed to be equal directions.

  A specific method of determining the acceptance criteria will be described with reference to FIG. FIG. 4 shows the silhouette shape of an object when an object that generates a shadow on the road surface is viewed from the direction in which the light source exists. 4A shows an example in which the silhouette boundary is oriented in the vertical direction, and FIG. 4B shows an example in which the silhouette boundary is tilted inward by about 15 degrees from the vertical direction and the silhouette shape is a triangle. Examples of the object having a silhouette shape as shown in FIG. 4A include a utility pole and a pedestrian. Examples of objects having a silhouette shape as shown in FIG. 4B include trees and pylons.

  When sunlight hits an object having a silhouette shape as shown in FIG. 4 (a), the shadow shape is almost a rod shape as shown in FIG. 5 (a) regardless of the altitude of the sun, and is locally on the road plane. The edge orientation is the same as the orientation of the sun. Therefore, when distinguishing a shadow of an object having a silhouette shape as shown in FIG. 5A from an obstacle, it is only necessary to extract an edge in which the direction of the light source and the direction of the edge on the road surface plane are very equal.

  On the other hand, in order to deal with a shadow caused by an object having a silhouette shape as shown in FIG. 4B, a difference between the direction of the edge and the direction of the light source is allowed to some extent. For example, when the object having the silhouette shown in FIG. 4B is exposed to sunlight from a position at an altitude of 30 degrees and an altitude of 60 degrees, the shapes of the shadows that can be formed on the road surface are shadows 505 and 506, respectively. The difference between the direction of the edges of the shadow boundary portions 503 and 504 and the sun direction 502 is about 9 degrees and about 35 degrees, respectively.

  In this way, if the inclination of the boundary of the silhouette of the object and the altitude of the sun for which false detection of shadows is to be suppressed are determined, the direction of the edge on the road plane to be allowed by the light source direction edge extraction module 105 and the direction in which the sun exists And the difference can be determined.

  This allowable amount may always be a constant value, or may be changed according to the solar altitude at that time. If the allowable amount is changed in accordance with the solar altitude, the inclination of the boundary of the silhouette of the object for which the erroneous detection of the shadow is to be suppressed can be always kept constant.

  4 and 5, for the sake of simplicity, the description has been made on the object whose inclination of the silhouette boundary is constant when viewed from the light source. However, the silhouette shape is a more complicated object such as a person. However, since both the inclination of the silhouette boundary and the direction of the edge on the road surface have the same characteristics locally, it is effective to extract the edge directed in the direction in which the light source exists.

  Further, when the altitude of the sun is higher, the difference between the edge direction on the road surface and the light source direction tends to be larger than the inclination angle of the silhouette boundary with respect to the horizontal, and it tends to be difficult to recognize as a shadow. However, in this case, the length of the shadow becomes short, and it becomes difficult to occur when only the shadow is included in the input image and no obstacle is included. Therefore, in this case, since the obstacle can be detected, there is almost no problem in the function of the obstacle detection device.

  The obstacle determination module 106 determines that the obstacle candidate region is a true obstacle only when the number of edges extracted by the light source direction edge extraction module 105 is a predetermined number or less with respect to the total number of edges. In other cases, it is determined that the obstacle candidate area is a shadow area (not determined as an obstacle).

  Next, the operation of the first embodiment will be described with reference to the drawings. FIG. 6 is a flowchart for explaining the operation of the obstacle detection apparatus according to this embodiment.

  First, the imaging module 101 acquires an image (step S601). Next, from the images acquired by the imaging module 101, the obstacle candidate area extraction module 102 extracts an area that is highly likely to be an obstacle (step S602).

  The imaging orientation extraction module 103 outputs the orientation of the optical axis of the camera (step S603). The light source orientation extraction module 104 outputs the orientation of the light source with respect to the imaging module (step S604).

  The light source direction edge extraction module 105 assumes that the edge pixel in the obstacle candidate region corresponds to a point on the road surface in the real world, and determines the local orientation of the edge on the road surface and the orientation of the light source. An edge whose difference is equal to or smaller than a certain value is extracted (step S605).

  Finally, when the number of edges extracted by the light source direction edge extraction module 105 is less than or equal to a certain number with respect to the total number of edges, the obstacle determination module 106 determines that the obstacle candidate area is a true obstacle. Judge that there is. In other cases, the region is considered to be a shadow region, and is determined not to be an obstacle region (step S606).

  The above-described obstacle detection device has been described as a configuration in which the above-described module is mounted and operated in a configuration including an ECU (Electronic Control Unit) 701 and an in-vehicle camera 702 as illustrated in FIG.

  The ECU 701 controls the entire apparatus, and includes, for example, a CPU, a RAM, a ROM, a signal processing circuit, a power supply circuit, and the like. That is, the obstacle detection device according to the above-described embodiment reads the ECU program 701 and executes a computer program that can realize the function and the determination logic described with reference to the flowchart in the description, except for the function of the imaging module. Is realized. It is also possible to configure a microcomputer by implementing the function executed by the ECU 701 as hardware. Furthermore, it is possible to realize a configuration in which some functions are realized by hardware, and similar functions are realized by cooperative operation of the hardware and the software program.

(Effects of the first embodiment)
The effect by 1st Embodiment comprised as mentioned above is demonstrated below.

  According to the present embodiment, without performing 3D modeling of surrounding buildings and road structures, obstacles are distinguished from shadows on the road surface caused by various objects from a single camera image mounted on a moving body. Can be detected.

  The reason will be described with reference to FIG. 12. A local edge direction 1103 of a boundary line of a shadow generated on the road plane by the object 1102 standing in the vertical direction from the road plane is the presence of the light source 1101 with respect to the object 1102. Based on the fact that the pixels in the obstacle candidate area correspond to points on the road surface in the real world, based on the fact that they are likely to coincide with the direction (here, the opposite direction is also identified as 180 degrees), the obstacle This is because an edge pixel facing in the same direction as the direction in which the light source exists is extracted from the object candidate area, and the shadow on the road surface and the obstacle are distinguished based on the ratio.

  As the background for this effect, the main objects that cause shadows in the environment where this device functions are pedestrians, street trees, utility poles, vehicles, etc., many of which are facing the vertical direction, and the road surface. Since the obstacle on the plane is the detection target, there is a special feature of the present apparatus that it is only necessary to distinguish the obstacle from the shadow on the road surface plane.

(Second Embodiment)

  A second embodiment of the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 8, the obstacle detection apparatus according to the second embodiment includes a color image capturing module 801 capable of measuring the intensity of light having a plurality of wavelengths mounted on a moving body moving on a road surface, and a color image. An obstacle candidate area extraction module 102 that extracts a partial area that may correspond to an obstacle from the color image output by the imaging module 801 as an obstacle candidate area, and the direction in which the color image imaging module 801 is imaging An imaging orientation extraction module 103 for outputting, a light source orientation extraction module 104 for extracting the orientation of the light source, and a light source direction edge extraction module 105 for extracting edge pixels having an edge direction facing the orientation in which the light source exists from the obstacle candidate area. A light source spectral characteristic acquisition module 802 that stores light intensities of a plurality of wavelengths included in the light source, and a light source direction edge The pixel values corresponding to the plurality of wavelengths of the pixels around the edge extracted by the output module 105 and the direct light of sunlight and the intensity of scattered light stored in the light source spectral characteristic acquisition module 802 are obtained for each wavelength in the sun and shade. And an obstacle determination unit 803 that determines whether the obstacle candidate area is an obstacle area based on the intensity of light.

  Of the modules shown in FIG. 8, the modules other than the color image capturing module 801, the light source spectral characteristic acquisition module 802, and the obstacle determination unit 803 are the same as the components of the first embodiment described above. The obstacle candidate area extraction module 102 according to the present embodiment uses a color image as input information, but the “possibility of an obstacle” described in the item regarding the obstacle candidate area extraction module shown in FIG. The “specific extraction method of partial area” and the like can be applied to a color image, and there is no particular difference.

  Therefore, in the following, the three modules that are different from the first embodiment, that is, the color image capturing module 801, the light source spectral characteristic acquisition module 802, and the obstacle determination unit 803 will be described in detail.

  The color image capturing module 801 acquires and outputs a color image including intensity information regarding light having a plurality of wavelengths. Here, the color image is not limited to wavelengths corresponding to the so-called three primary colors of light, blue, green, and red, but may include light intensity information relating to other wavelengths. Further, not only the wavelength in the visible light region but also the wavelength in the near infrared light region may be included.

  The light source spectral characteristic acquisition module 802 acquires and stores the intensity of light of a plurality of wavelengths of direct light of sunlight and scattered light from the atmosphere, that is, the intensity of light of a plurality of wavelengths included in the color image. It is. Since the light intensity also changes depending on the date and the atmospheric condition, the intensity of the light of each wavelength is separately acquired and stored according to the date and the atmospheric condition at that time. That is, the intensity of light of a plurality of wavelengths is stored in association with date / time and atmospheric state information. Therefore, the light source spectral characteristic acquisition module 802 may include a date / time calendar function and a clock function. Further, the atmospheric state information may be set according to the average state of the city where the apparatus is used, or may be set by taking in from outside by communication.

  The obstacle determination unit 803 determines whether the obstacle candidate area is a true obstacle based on the color information value of the pixels around the edge extracted by the light source direction edge extraction module 105. In other words, it is determined whether or not the light source direction edge is an edge caused by a shadow, and the obstacle candidate region is determined to be a true obstacle when it does not have the feature of the edge caused by the shadow.

  Hereinafter, a specific determination method of the obstacle determination unit 803 will be described with reference to FIGS. 9 and 11. FIG. 9 is a diagram for explaining the positions of the pixels used by the obstacle determination means 803 for determination. FIG. 11 is a flowchart for explaining the contents of the determination process.

  When the light source direction edge extraction module 105 extracts the edge pixel 901 as the light source direction edge pixel, and the local edge direction is the direction of the direction 902, the obstacle determination unit 803 divides the edge pixel 901 by the edge. One point 903 and one point 904 are extracted from each of the two regions (step S1101). Extraction rules for points 903 and 904 are set in advance. For example, as an extraction rule, a rule for extracting, as a point, a pixel at a certain distance from the point 901 in a direction orthogonal to the edge direction with the edge pixel 901 as a base point.

  Then, the obstacle determination unit 803 obtains the ratio of the light intensity of the plurality of wavelengths of the image corresponding to the points 903 and 904 for each wavelength (step S1102). Two light intensity ratios are obtained: the light intensity at point 904 based on the light intensity at point 903 and the light intensity at point 903 based on the light intensity at point 904. This is because if the obstacle candidate region is a shadow region, it can be dealt with whether the point 903 or the point 904 is a shaded region. If the number of wavelengths is three, the intensity ratio of the three types of light is obtained.

  Separately from this, the obstacle determination means 803 refers to the intensity of light of a plurality of wavelengths stored in the light source spectral characteristic acquisition module 802, and the intensity of light in the sun and shade for the plurality of wavelengths constituting the color image. The ratio is obtained (step S1103). In this case, for example, the ratio of the intensity of shaded light may be obtained with reference to the intensity of sunlight.

  Then, the obstacle determination unit 803 compares one of the two light intensity ratios obtained from the input image with respect to the edge point with the light intensity ratio in the sun and the shade for a plurality of wavelengths, and relates to all wavelengths. It is determined whether or not the difference in light intensity ratio is equal to or less than a certain value (step S1104). For other edge pixels, the same determination is performed for other two points (pixels) extracted from the two regions. This determination is made for all edge pixels in the light source direction.

  Finally, the obstacle determination means 803 obtains the number of edge pixels whose light intensity ratio difference is not more than a certain value (step S1105), and the ratio of the number of edge pixels to the total number of edge pixels is a predetermined value. It is determined whether or not the following is true (steps S1106 and S1107). Finally, the obstacle determination unit 803 determines that the obstacle candidate area is a true obstacle area when it is equal to or smaller than a predetermined value (step S1108). It is determined that the area is not an obstacle area (step S1109).

  This is based on the premise that the pixel value has a value proportional to the product of the intensity of incident light and the reflectance of the road surface. That is, when the difference in the light intensity ratio for all wavelengths is less than or equal to a certain value, the light intensity for each wavelength at points 903 and 904 is exactly the sun, regardless of the road surface reflectance value. It is similar to the light intensity in the shade, and it can be determined that the obstacle candidate region is a shadow region.

  As described above, according to the present embodiment, the obstacles are finally determined more accurately by comparing the light intensities for a plurality of wavelengths in the pixels around the edge pixels with the light intensities in the sun and shade. Obstacles can be detected while distinguishing obstacles from shadows caused by objects. In particular, even when the longitudinal direction of the line-shaped pattern drawn on the road surface is similar to the direction in which the light source exists, it is possible to suppress erroneous determination of this pattern as an obstacle.

  When the obstacle candidate region is a region corresponding to a shadow, the property that the light intensity at each wavelength at the points 903 and 904 is similar to the light intensity in the sun and in the shade is the road surface reflectance and Since it does not depend on the color, an accurate determination can be made even on a road surface having a line-shaped road surface paint. In addition, by limiting the pixels that verify the light intensity ratio for each wavelength to pixels whose edge direction points to the direction of the light source, it is possible to focus on the image features around the pixel that seems to be the boundary between the sun and the shade. Judgment is easier.

  Next, the operation of the second embodiment will be described with reference to the drawings. FIG. 10 is a flowchart for explaining the operation of the obstacle detection apparatus according to the second embodiment.

  First, the color image capturing module 801 acquires an image (step S1001). Next, from the images acquired by the color image capturing module 801, the obstacle candidate area extraction module 102 extracts an area having a high possibility of an obstacle (step S1002).

  Next, the imaging orientation extraction module 103 outputs the orientation of the optical axis of the color image imaging module 801 (camera) (step S1003). The light source orientation extraction module 104 outputs the orientation of the light source with respect to the color image capturing module 801 (step S1004). The light source direction edge extraction module 105 extracts an edge whose difference between the local orientation of the edge on the road plane and the orientation of the light source is equal to or less than a certain value (step S1005).

  The obstacle determination module 803 then compares the intensity ratio of the light of the wavelengths included in the color image of two pixels extracted one by one from the regions on both sides of the edge among the edge pixels extracted by the light source direction edge extraction unit 105. Are compared with the light intensity ratio defined by the intensity of direct light and scattered light and the light intensity ratio in the shade, and the number of edge pixels where the difference in the light intensity ratio is equal to or less than a certain value is obtained (step S1006).

  Finally, the obstacle candidate area is determined as an obstacle area when the ratio of the number of edge pixels where the difference in intensity ratio is equal to or less than a certain value to the total number of edge pixels is equal to or less than a predetermined value (step S1007).

  In the above description, the points 903 and 904 are directly extracted from the input image, but the positions of two similar points on the road surface plane are obtained based on the positions and directions of the edges on the road surface, and corresponding to this. The pixel position in the input image to be obtained may be obtained to obtain the light intensity ratio for each wavelength.

(Effects of the second embodiment)
The effect by 2nd Embodiment comprised as mentioned above is demonstrated below.
According to the present embodiment, since the intensity of light for a plurality of wavelengths in the pixels around the edge pixel is compared with the intensity of light in the sun and shade, the obstacle is finally determined. Obstacles can be detected while distinguishing them from shadows. In particular, even when the longitudinal direction of the line-shaped pattern drawn on the road surface is similar to the direction in which the light source exists, it is possible to suppress erroneous determination of this pattern as an obstacle.

  Although the present invention has been described with reference to the preferred embodiments, the present invention is not necessarily limited to the above embodiments, and various modifications can be made within the scope of the technical idea. .

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  Moreover, although the several procedure is described in order in the method and computer program of this invention, the order of the description does not limit the order which performs a several procedure. For this reason, when implementing the method and computer program of this invention, the order of the several procedure can be changed in the range which does not interfere in content.

  The plurality of procedures of the method and the computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  Further, a part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
An imaging means mounted on a moving body moving on the road surface and photographing the periphery;
Obstacle candidate area extraction means for extracting a partial area that may correspond to an obstacle from the captured image as an obstacle candidate area;
Imaging orientation extraction means for outputting the orientation of the optical axis of the imaging means;
A light source direction extracting means for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction means;
Obstacle determination means for determining the obstacle candidate area as an obstacle area when a ratio of the number of edge pixels extracted by the light source direction edge extraction means to a total edge pixel number is equal to or less than a predetermined value;
An obstacle detection device comprising:

(Appendix 2)
An image pickup means mounted on a moving body moving on a road surface and shooting a peripheral color image including a plurality of wavelengths;
An obstacle candidate area extraction means for extracting a partial area that may correspond to an obstacle from the color image as an obstacle candidate area;
Imaging orientation extraction means for outputting the orientation of the optical axis of the imaging means;
A light source direction extracting means for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction means;
Light source spectral characteristic acquisition means for acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for a plurality of wavelengths;
Of the edge pixels extracted by the light source direction edge extracting means, defined by the intensity ratio of light of wavelengths included in the color image of two pixels extracted from the regions on both sides of the edge pixel, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; Obstacle determination means for determining an object candidate area as an obstacle area;
An obstacle detection device comprising:

(Appendix 3)
The light source orientation extracting means includes
Supplementary note 1 or Supplementary note 2, wherein the current date and time are obtained by a built-in clock function, and the current position is obtained by a built-in GPS function, whereby the sun direction is calculated and output as a light source direction Obstacle detection device.

(Appendix 4)
The light source direction edge extracting means includes:
In the obstacle candidate area, a pixel at a position where the pixel value of an adjacent pixel greatly changes is extracted as an edge pixel, and a line segment connecting the center of the imaging unit and the extracted edge pixel is extended to the road surface plane. Then, by obtaining the intersection as an edge point, the edge pixel corresponds to a point on the road surface plane,
Obtaining the edge direction on the captured image of the edge pixel based on the gradient of the pixel value around the edge pixel,
Converting the edge direction on the captured image of the edge pixel into a direction on a road surface plane;
Supplementary note 1 to supplementary note 1, wherein edge pixels facing a direction equal to the azimuth of the light source are extracted by comparing the direction of the edge pixel on the road surface plane with the azimuth of the light source. 4. The obstacle detection device according to any one of 3.

(Appendix 5)
The obstacle determination means includes
Extracting two pixels from two regions divided by the edge pixel for an edge pixel facing in the same direction as the direction in which the light source exists, obtaining an intensity ratio of light of a plurality of wavelengths in a color image of the two pixels,
Referring to the light source spectral characteristic acquisition means, obtain the intensity ratio of light in the sun and shade for the plurality of wavelengths,
Compares the light intensity ratio of the two pixels with respect to the edge pixel and the light intensity ratio in the sun and shade with respect to the plurality of wavelengths, and determines whether the difference in the light intensity ratio is a certain value or less. And
Any one of appendix 2 to appendix 4, wherein it is determined whether or not a ratio of the number of edge pixels whose light intensity ratio difference is equal to or less than a predetermined value to a total number of edge pixels is equal to or less than a predetermined value. Obstacle detection device according to the above.

(Appendix 6)
An obstacle candidate region extraction step for extracting a partial region that may correspond to an obstacle as an obstacle candidate region from surrounding captured images taken by an imaging means mounted on a moving body moving on a road surface;
An imaging orientation extraction step of outputting the orientation of the optical axis of the imaging means;
A light source orientation extraction step for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area A direction edge extraction step;
An obstacle determination step of determining the obstacle candidate region as an obstacle region when a ratio of the number of edge pixels extracted in the light source direction edge extraction step to a total edge pixel number is equal to or less than a predetermined value;
An obstacle detection method characterized by comprising:

(Appendix 7)
An obstacle candidate region extraction step for extracting a partial region that may correspond to an obstacle from surrounding color images including a plurality of wavelengths captured by an imaging unit mounted on a moving body moving on a road surface as an obstacle candidate region. When,
An imaging orientation extraction step of outputting the orientation of the optical axis of the imaging means;
A light source orientation extraction step for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area A direction edge extraction step;
A light source spectral characteristic acquisition step of acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for a plurality of wavelengths;
Of the edge pixels extracted in the light source direction edge extraction step, defined by the intensity ratio of light of wavelengths included in the color image of two pixels extracted from the regions on both sides of the edge pixel, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; An obstacle determination step for determining an object candidate area as an obstacle area;
An obstacle detection method characterized by comprising:

(Appendix 8)
In the light source direction extraction step,
Appendix 6 or appendix 7, wherein the current date and time are obtained by a built-in clock function, and the current position is obtained by a built-in GPS function, whereby the azimuth of the sun is calculated and output as a light source azimuth. Obstacle detection method.

(Appendix 9)
In the light source direction edge extraction step,
In the obstacle candidate area, a pixel at a position where the pixel value of an adjacent pixel greatly changes is extracted as an edge pixel, and a line segment connecting the center of the imaging unit and the extracted edge pixel is extended to the road surface plane. Then, by obtaining the intersection as an edge point, the edge pixel corresponds to a point on the road surface plane,
Obtaining the edge direction on the captured image of the edge pixel based on the gradient of the pixel value around the edge pixel,
Converting the edge direction on the captured image of the edge pixel into a direction on a road surface plane;
Supplementary note 6 to supplementary note 6, wherein edge pixels pointing in the same direction as the azimuth of the light source are extracted by comparing the direction of the edge pixel on the road surface plane with the azimuth of the light source. The obstacle detection method according to claim 8.

(Appendix 10)
In the obstacle determination step,
Extracting two pixels from two regions divided by the edge pixel for an edge pixel facing in the same direction as the direction in which the light source exists, obtaining an intensity ratio of light of a plurality of wavelengths in a color image of the two pixels,
Find the intensity ratio of light in the sun and shade for the multiple wavelengths acquired in the light source spectral characteristics acquisition step,
Compares the light intensity ratio of the two pixels with respect to the edge pixel and the light intensity ratio in the sun and shade with respect to the plurality of wavelengths, and determines whether the difference in the light intensity ratio is a certain value or less. And
Any one of appendix 7 to appendix 9, wherein it is determined whether or not a ratio of the number of edge pixels whose light intensity ratio difference is not more than a certain value to the total number of edge pixels is not more than a predetermined value. The obstacle detection method according to the above.

(Appendix 11)
An obstacle candidate area extraction process for extracting a partial area that may correspond to an obstacle as an obstacle candidate area from surrounding captured images taken by an imaging means mounted on a moving body moving on a road surface;
Imaging orientation extraction processing for outputting the orientation of the optical axis of the imaging means;
Light source direction extraction processing for outputting the direction in which the light source exists,
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction processing,
An obstacle determination process for determining the obstacle candidate area as an obstacle area when the ratio of the number of edge pixels extracted in the light source direction edge extraction process to the total number of edge pixels is equal to or less than a predetermined value; Obstacle detection program characterized by being executed.

(Appendix 12)
Obstacle candidate area extraction processing for extracting a partial area that may correspond to an obstacle from surrounding color images including a plurality of wavelengths taken by an imaging unit mounted on a moving body moving on a road surface as an obstacle candidate area When,
Imaging orientation extraction processing for outputting the orientation of the optical axis of the imaging means;
Light source direction extraction processing for outputting the direction in which the light source exists,
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction processing,
Light source spectral characteristic acquisition processing for acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for multiple wavelengths,
Defined by the intensity ratio of the light of the wavelength included in the color image of the two pixels extracted from the regions on both sides of the edge pixel among the edge pixels extracted by the light source direction edge extraction processing, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; An obstacle detection program that causes a computer to execute an obstacle determination step of determining an object candidate area as an obstacle area.

(Appendix 13)
In the light source orientation extraction process,
Supplementary note 11 or Supplementary note 12, wherein the current date and time are obtained by a built-in clock function, and the current position is obtained by a built-in GPS function, whereby the azimuth of the sun is calculated and output as a light source azimuth. Obstacle detection program.

(Appendix 14)
In the light source direction edge extraction process,
In the obstacle candidate area, a pixel at a position where the pixel value of an adjacent pixel greatly changes is extracted as an edge pixel, and a line segment connecting the center of the imaging unit and the extracted edge pixel is extended to the road surface plane. Then, by obtaining the intersection as an edge point, the edge pixel corresponds to a point on the road surface plane,
Obtaining the edge direction on the captured image of the edge pixel based on the gradient of the pixel value around the edge pixel,
Converting the edge direction on the captured image of the edge pixel into a direction on a road surface plane;
Supplementary note 11 to supplementary note 11, wherein edge pixels pointing in the same direction as the azimuth of the light source are extracted by comparing the direction of the edge pixel on the road surface plane with the azimuth of the light source. The obstacle detection program according to any one of claims 13 to 13.

(Appendix 15)
In the obstacle determination process,
Extracting two pixels from two regions divided by the edge pixel for an edge pixel facing in the same direction as the direction in which the light source exists, obtaining an intensity ratio of light of a plurality of wavelengths in a color image of the two pixels,
Find the intensity ratio of light in the sun and shade for the multiple wavelengths acquired in the light source spectral characteristics acquisition process,
Compares the light intensity ratio of the two pixels with respect to the edge pixel and the light intensity ratio in the sun and shade with respect to the plurality of wavelengths, and determines whether the difference in the light intensity ratio is a certain value or less. And
Any one of appendix 12 to appendix 14, wherein it is determined whether the ratio of the number of edge pixels where the difference in light intensity ratio is less than a certain value to the total number of edge pixels is less than a predetermined value. Obstacle detection program described in Crab.

101: Imaging module 102: Obstacle candidate area extraction module 103: Imaging direction extraction module 104: Light source direction extraction module 105: Light source direction edge extraction module 106: Obstacle determination module 201: View point 202: Obstacle candidate area 203: Obstacle Edge point in candidate area 204: road surface 701: ECU
702: Imaging device 801: Color image imaging module 802: Light source spectral characteristic acquisition module 803: Obstacle determination module 901: Edge pixel 902: Edge direction 903, 904: Pixel referring to color information 1101: Light source 1102: Object (tree)
1103: Local edge direction of shadow on road surface

Claims (10)

An imaging means mounted on a moving body moving on the road surface and photographing the periphery;
Obstacle candidate area extraction means for extracting a partial area that may correspond to an obstacle from the captured image as an obstacle candidate area;
Imaging orientation extraction means for outputting the orientation of the optical axis of the imaging means;
A light source direction extracting means for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction means;
Obstacle determination means for determining the obstacle candidate area as an obstacle area when a ratio of the number of edge pixels extracted by the light source direction edge extraction means to a total edge pixel number is equal to or less than a predetermined value;
An obstacle detection device comprising: An image pickup means mounted on a moving body moving on a road surface and shooting a peripheral color image including a plurality of wavelengths;
An obstacle candidate area extraction means for extracting a partial area that may correspond to an obstacle from the color image as an obstacle candidate area;
Imaging orientation extraction means for outputting the orientation of the optical axis of the imaging means;
A light source direction extracting means for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction means;
Light source spectral characteristic acquisition means for acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for a plurality of wavelengths;
Of the edge pixels extracted by the light source direction edge extracting means, defined by the intensity ratio of light of wavelengths included in the color image of two pixels extracted from the regions on both sides of the edge pixel, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; Obstacle determination means for determining an object candidate area as an obstacle area;
An obstacle detection device comprising: The light source direction edge extracting means includes:
In the obstacle candidate area, a pixel at a position where the pixel value of an adjacent pixel greatly changes is extracted as an edge pixel, and a line segment connecting the center of the imaging unit and the extracted edge pixel is extended to the road surface plane. Then, by obtaining the intersection as an edge point, the edge pixel corresponds to a point on the road surface plane,
Obtaining the edge direction on the captured image of the edge pixel based on the gradient of the pixel value around the edge pixel,
Converting the edge direction on the captured image of the edge pixel into a direction on a road surface plane;
The edge pixel that faces the same direction as the direction in which the light source exists is extracted by comparing the direction of the edge pixel on the road surface plane with the direction in which the light source exists. The obstacle detection device according to claim 2. The obstacle determination means includes
Extracting two pixels from two regions divided by the edge pixel for an edge pixel facing in the same direction as the direction in which the light source exists, obtaining an intensity ratio of light of a plurality of wavelengths in a color image of the two pixels,
Referring to the light source spectral characteristic acquisition means, obtain the intensity ratio of light in the sun and shade for the plurality of wavelengths,
Compares the light intensity ratio of the two pixels with respect to the edge pixel and the light intensity ratio in the sun and shade with respect to the plurality of wavelengths, and determines whether the difference in the light intensity ratio is a certain value or less. And
4. The method according to claim 2, wherein it is determined whether or not a ratio of the number of edge pixels where the difference in light intensity ratio is equal to or less than a predetermined value to a total number of edge pixels is equal to or less than a predetermined value. Obstacle detection device according to. An obstacle candidate region extraction step for extracting a partial region that may correspond to an obstacle as an obstacle candidate region from surrounding captured images taken by an imaging means mounted on a moving body moving on a road surface;
An imaging orientation extraction step of outputting the orientation of the optical axis of the imaging means;
A light source orientation extraction step for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area A direction edge extraction step;
An obstacle determination step of determining the obstacle candidate region as an obstacle region when a ratio of the number of edge pixels extracted in the light source direction edge extraction step to a total edge pixel number is equal to or less than a predetermined value;
An obstacle detection method characterized by comprising: An obstacle candidate region extraction step for extracting a partial region that may correspond to an obstacle from surrounding color images including a plurality of wavelengths captured by an imaging unit mounted on a moving body moving on a road surface as an obstacle candidate region. When,
An imaging orientation extraction step of outputting the orientation of the optical axis of the imaging means;
A light source orientation extraction step for outputting a direction in which the light source exists;
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area A direction edge extraction step;
A light source spectral characteristic acquisition step of acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for a plurality of wavelengths;
Of the edge pixels extracted in the light source direction edge extraction step, defined by the intensity ratio of light of wavelengths included in the color image of two pixels extracted from the regions on both sides of the edge pixel, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; An obstacle determination step for determining an object candidate area as an obstacle area;
An obstacle detection method characterized by comprising: In the light source direction edge extraction step,
In the obstacle candidate area, a pixel at a position where the pixel value of an adjacent pixel greatly changes is extracted as an edge pixel, and a line segment connecting the center of the imaging unit and the extracted edge pixel is extended to the road surface plane. Then, by obtaining the intersection as an edge point, the edge pixel corresponds to a point on the road surface plane,
Obtaining the edge direction on the captured image of the edge pixel based on the gradient of the pixel value around the edge pixel,
Converting the edge direction on the captured image of the edge pixel into a direction on a road surface plane;
6. The edge pixel pointing in a direction equal to the direction in which the light source exists is extracted by comparing the direction of the edge pixel on the road surface plane with the direction in which the light source exists. The obstacle detection method according to claim 6. In the obstacle determination step,
Extracting two pixels from two regions divided by the edge pixel for an edge pixel facing in the same direction as the direction in which the light source exists, obtaining an intensity ratio of light of a plurality of wavelengths in a color image of the two pixels,
Find the intensity ratio of light in the sun and shade for the multiple wavelengths acquired in the light source spectral characteristics acquisition step,
Compares the light intensity ratio of the two pixels with respect to the edge pixel and the light intensity ratio in the sun and shade with respect to the plurality of wavelengths, and determines whether the difference in the light intensity ratio is a certain value or less. And
8. The method according to claim 6, wherein it is determined whether or not a ratio of the number of edge pixels where the difference in light intensity ratio is equal to or less than a predetermined value to a total number of edge pixels is equal to or less than a predetermined value. Obstacle detection method described in 1. An obstacle candidate area extraction process for extracting a partial area that may correspond to an obstacle as an obstacle candidate area from surrounding captured images taken by an imaging means mounted on a moving body moving on a road surface;
Imaging orientation extraction processing for outputting the orientation of the optical axis of the imaging means;
Light source direction extraction processing for outputting the direction in which the light source exists,
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction processing,
An obstacle determination process for determining the obstacle candidate area as an obstacle area when the ratio of the number of edge pixels extracted in the light source direction edge extraction process to the total number of edge pixels is equal to or less than a predetermined value; Obstacle detection program characterized by being executed. Obstacle candidate area extraction processing for extracting a partial area that may correspond to an obstacle from surrounding color images including a plurality of wavelengths taken by an imaging unit mounted on a moving body moving on a road surface as an obstacle candidate area When,
Imaging orientation extraction processing for outputting the orientation of the optical axis of the imaging means;
Light source direction extraction processing for outputting the direction in which the light source exists,
Assuming that pixels in the obstacle candidate area correspond to points on the road surface in the real world, a light source that extracts edge pixels that point in the same direction as the direction in which the light source exists from the obstacle candidate area Direction edge extraction processing,
Light source spectral characteristic acquisition processing for acquiring and storing the intensity of direct light of sunlight and scattered light from the atmosphere for multiple wavelengths,
Defined by the intensity ratio of the light of the wavelength included in the color image of the two pixels extracted from the regions on both sides of the edge pixel among the edge pixels extracted by the light source direction edge extraction processing, and the intensity of the direct light and scattered light Determining the number of edge pixels whose difference between the light intensity ratio in the sun and the shade is equal to or less than a certain value, and the ratio of the number of the edge pixels to the total number of edge pixels is equal to or less than a predetermined value; An obstacle detection program that causes a computer to execute an obstacle determination step of determining an object candidate area as an obstacle area.

Images