JP2008257399A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor for detecting a part wherein a person may dash out without using a special device. <P>SOLUTION: This image processor 1 has a camera 10 picking up an image in front of a vehicle, and an image processing ECU (Electronic Control Unit) 20. The image processing ECU 20 is configured with: a target point setting part 21 setting a target point in a photographic image imaged by the camera 10; a peripheral point setting part 22 setting a peripheral point positioned around the target point, and on a straight line connecting the target point and a vanishing point; and a discontinuity decision part 24 detecting a gap wherein a person may dash out by comparing a characteristic amount of the target point and a characteristic amount of the peripheral point to decide discontinuity of the depth between the target point and the peripheral point in a real space. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus.

  In order to improve safety by supporting the recognition and judgment behavior of a vehicle driver, an apparatus for detecting an object jumping out in front of the vehicle has been researched and developed. As such an apparatus, in Patent Document 1, a captured image of a travel destination of the host vehicle is acquired, a detection region virtually fixed in real space is set in the captured image, and the set detection is performed. A pop-out object detection device is disclosed that calculates an optical flow in a region and detects a pop-out object based on the optical flow.

On the other hand, the movement of the host vehicle is obtained by acquiring the detection information from a pedestrian or other vehicle equipped with a device capable of detecting a movement state such as a movement direction and a movement speed and exchanging the detection information by wireless communication. A technique for predicting the jump of a pedestrian or other vehicle existing in a direction is also known.
JP 2006-72439 A

  Here, according to the pop-out detection device described above, it is possible to detect an object that has actually popped out. However, even if an object that has already popped out can be detected, the countermeasures that can be taken from that point of time may be limited. On the other hand, in order to realize the above-described technology for predicting the pedestrians and other vehicles jumping out, the pedestrians and other vehicles existing in the moving direction of the own vehicle have special sensors such as various sensors and communication terminals for detecting the moving state. It is necessary to have a proper device.

  The present invention has been made to solve the above-described problems, and provides an image processing apparatus capable of detecting a place where a jump may occur without using a special apparatus. Objective.

  An image processing apparatus according to the present invention includes an imaging unit that captures the front of a vehicle, an attention point setting unit that sets an attention point in a captured image captured by the imaging unit, and an attention point set by the attention point setting unit. The discontinuity in the real space between the attention point and the surrounding point is determined by comparing the feature amount of the attention point and the feature amount of the surrounding point with the surrounding point setting means for setting the surrounding points located around And determining means.

  According to the image processing device of the present invention, the feature amount is compared between the attention point in the captured image and the surrounding points located around the attention point, so that in the real space corresponding to the attention point and the surrounding points respectively. A discontinuity between the two points is determined. As a result, without using a special device, based on the determination result regarding the discontinuity between the two points, it faces a road where an object that is not visible to the driver may pop out (the depth is discontinuous). It becomes possible to detect the gap.

  In the image processing apparatus according to the present invention, it is preferable that the surrounding point setting unit obtains the vanishing point in the captured image and sets the surrounding point on a straight line connecting the attention point and the vanishing point.

  Since pedestrians and other vehicles move in parallel to the road surface, there is usually a gap along which the object pops out along the road surface. According to the image processing apparatus of the present invention, the surrounding points to be compared with the attention point are set on a straight line connecting the attention point and the vanishing point. This means that a point in a positional relationship parallel to the road surface with respect to the point of interest is set as a surrounding point. For this reason, it is possible to appropriately detect a gap where an object is likely to jump out in real space, that is, a gap existing along the road surface.

  In the image processing apparatus according to the present invention, the attention point setting unit sets a plurality of attention points in a substantially vertical direction of the captured image, and the surrounding point setting unit includes a plurality of surrounding points corresponding to each of the plurality of attention points. It is preferable that the continuity determination unit determines discontinuity in the real space by comparing the feature quantities of the plurality of attention points with the feature quantities of the plurality of surrounding points.

  As described above, the gap where the object pops out exists along the road surface. Further, the walls and the like forming the gap are often formed in a substantially vertical direction from the road surface. In the image processing apparatus according to the present invention, a plurality of attention points are set in a substantially vertical direction of a captured image, that is, a substantially vertical direction in real space, and a plurality of surrounding points corresponding to each of the plurality of attention points are set. A discontinuity in the real space is determined by comparing feature quantities between these attention points and surrounding points. Therefore, the discontinuity in the real space can be determined along the edge of the wall surface or the like that forms the gap, and it is possible to improve the accuracy of detecting the gap where there is a high possibility that the object will pop out.

  In the image processing apparatus according to the present invention, it is preferable to use luminance, optical flow, or three-dimensional coordinates as the feature amount of the attention point and the feature amount of the surrounding points.

  When the depth is discontinuous between the point of interest and the surrounding points, the brightness, optical flow, and three-dimensional coordinates on the captured image of the point of interest and the surrounding points are compared to when the depth is continuous. The value of changes significantly. According to the image processing apparatus of the present invention, it is possible to appropriately determine whether or not the depths of the attention point and the surrounding points are discontinuous by using luminance, optical flow, or three-dimensional coordinates as feature quantities to be compared. It becomes possible to judge.

  According to the present invention, since the feature amount of the attention point and the feature amount of the surrounding point are compared to determine the discontinuity of the depth in the real space between the attention point and the surrounding point, a special device is provided. It is possible to detect a place where a pop-out may occur without the need for.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the figure, the same reference numerals are used for the same or corresponding parts.

  First, the configuration of the image processing apparatus 1 according to the embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the image processing apparatus 1.

  The image processing device 1 is mounted on a vehicle, images the front of the vehicle, and determines discontinuity in the depth direction between two points (or between two or more points) included in the captured image, It detects a gap that faces the road in front of the vehicle where an object may pop out. For this purpose, the image processing apparatus 1 includes a camera 10 that captures a scene in front of the vehicle and an image processing ECU (Electronic Control Unit) 20.

  The camera 10 is attached to a vehicle on which the image processing device 1 is mounted, and images a scene in front of the vehicle. For example, the camera 10 is attached to the rear surface of a rearview mirror or the like toward the front of the vehicle. The camera 10 captures an image at regular time intervals (for example, 1/30 seconds), and outputs image information of each frame to the image processing ECU 20 as an image signal. That is, the camera 10 functions as an imaging unit described in the claims.

  The image processing ECU 20 includes a microprocessor that performs calculations, a ROM that stores programs for causing the microprocessor to execute each process, a RAM that stores various data such as calculation results, and a backup RAM in which the stored contents are held by a battery. Etc.

  In the image processing ECU 20, the attention point setting unit 21, the surrounding point setting unit 22, the feature amount extraction unit 23, and the discontinuity determination unit 24 are functionally constructed by executing a program stored in the ROM or the like. The Here, the attention point setting unit 21 corresponds to attention point setting means described in the claims, and the surrounding point setting unit 22 corresponds to surrounding point setting means. The feature quantity extraction unit 23 and the discontinuity determination unit 24 correspond to a determination unit.

  The attention point setting unit 21 sets the attention point in the captured image captured by the camera 10. Here, the attention point is a point to check whether or not there is a pop-up in the captured image. When a region that is likely to jump out in the real space is previously determined from the prior information, and a region in the captured image corresponding to the real space is known, a point included in the region is set as a point of interest. On the other hand, when there is no prior information, all pixels in the captured image are set as attention points. Also, a point included in a region where the color in the captured image changes abruptly, for example, a point included in a region where the luminance changes abruptly if the captured image is a monochrome image, or a color itself if it is a color image A point included in a region in which the value of is greatly changed may be set as a point of interest. Note that the number of attention points to be set may be one or plural. However, when a plurality of attention points are set, it is preferable to set a plurality of attention points in a substantially vertical direction of the captured image.

  The surrounding point setting unit 22 obtains a vanishing point in the captured image, and finds a point located on the straight line connecting the attention point and the vanishing point around the attention point set by the attention point setting unit 21. Set as a surrounding point. Therefore, when a plurality of attention points are set, a plurality of surrounding points are also set correspondingly.

  The feature amount extraction unit 23 acquires feature amounts of the attention point and the surrounding points, for example, luminance, optical flow, and / or three-dimensional coordinates in the captured image of the attention point and the surrounding points.

  The discontinuity determination unit 24 compares the feature amount of the attention point acquired by the feature amount extraction unit 23 with the feature amount of the surrounding point, thereby discontinuity in the depth direction in the real space between the attention point and the surrounding point. Determine sex. That is, when the feature amount of the attention point and the feature amount of the surrounding points are different from each other by a predetermined value or more, it is determined that there is a gap where the depth is discontinuous. On the other hand, when the deviation between the feature amount of the attention point and the feature amount of the surrounding points is less than the predetermined value, it is determined that the attention point and the surrounding points are continuous in the depth direction (that is, there is no gap). . Then, when a gap along the road surface is detected, the discontinuity determination unit 24 identifies the gap as a pop-out area and outputs the result.

  Next, the operation of the image processing apparatus 1 will be described with reference to FIG. FIG. 2 is a flowchart showing a processing procedure of the pop-out area determination process by the image processing apparatus 1. This process is performed by the image processing ECU 20, and is repeatedly executed at a predetermined timing from when the power of the image processing ECU 20 is turned on to when it is turned off.

  In step S100, a captured image in front of the vehicle captured by the camera 10 is read. Subsequently, in step S102, a point of interest in the captured image read in step S100 is set. Since the attention point setting method is as described above, a duplicate description is omitted here.

  In subsequent step S104, surrounding points corresponding to the attention point set in step S102 are set. Here, the vanishing point in the captured image is obtained, and a point around the attention point and located on a straight line connecting the attention point and the vanishing point is set as the surrounding point. Here, a method for setting the surrounding points will be described more specifically with reference to FIGS. FIG. 3 is an example of a captured image for explaining a surrounding point setting method. FIG. 4 is a bird's eye view of the real space corresponding to the captured image shown in FIG. 3 as viewed from above.

  In the captured image shown in FIG. 3, the region where the pop-out can occur is a region surrounded by an ellipse, that is, a region where the wall is interrupted. Here, as shown in FIG. 4, the region where the wall is interrupted is that the wall is discontinuous when the continuity of the wall is evaluated in the direction along the road boundary line (depth direction). It is a place. Therefore, in order to determine the discontinuity of the wall from the captured image shown in FIG. 3, the discontinuity between the pixels on the straight line connecting each point in the captured image and the road vanishing point DP formed by the road boundary line. It can be realized by determining. Therefore, here, a point located on the straight line L connecting the attention point AP and the vanishing point DP is set as a peripheral point CP for evaluating whether or not the attention point AP is a discontinuous point.

  Subsequently, in step S106, feature quantities of the attention point and the surrounding points, for example, luminance, optical flow, and / or three-dimensional coordinates in the captured image of the attention point and the surrounding points are acquired. Here, when the luminance value is used as the feature amount, the luminance value at each point is extracted from the pixels in the captured image. When the optical flow is used as the feature amount, the optical flow at each point is extracted from the captured image. A known technique can be used for extraction of the optical flow (see, for example, “B. Horn and B. Schunck. Determining optical flow. Artificial Intelligence, 17: 185-203, 1981”). In addition, when three-dimensional coordinates are used as feature quantities, the three-dimensional coordinates of each point are acquired. In addition, a well-known technique can be used for acquisition of a three-dimensional coordinate (for example, refer to "Xugang three-dimensional vision Kyoritsu publication P95").

  Next, in step S108, the continuity between the attention point and the surrounding points is determined according to the comparison result between the feature amount (for example, luminance) of the attention point acquired in step S106 and the feature amount (for example, luminance) of the surrounding points. Sex / discontinuity is determined. Here, when the absolute value of the difference between the feature value of the attention point and the feature value of the surrounding points is less than a predetermined threshold value, it can be determined that the attention point and the surrounding points are continuous in the depth direction. it can. On the other hand, when the absolute value of the difference between the feature value of the attention point and the feature value of the surrounding points is equal to or greater than a predetermined threshold value, the attention point and the surrounding points are not continuous in the depth direction (that is, discontinuous). Can be determined.

  Also, based on the feature values of the surrounding points, the population distribution that generates the feature values of the surrounding points is estimated, and the probability that the feature amount of the attention point is generated from the population distribution is calculated. When the threshold value is less than the threshold value, it is determined that the attention point and the surrounding points are discontinuous, and when the probability is equal to or higher than a predetermined threshold value, it is determined that the attention point and the surrounding points are continuous. Also good. In addition, the estimation of the population distribution and the calculation of the probability that the feature quantity of the attention point is generated from the population distribution can use known techniques (for example, “Kenichiro Ishii, Nobuyoshi Ueda, Hidesaku Maeda, Hiroshi Murase, Easy-to-understand pattern recognition, see Ohm, P49-55).

  In the subsequent step S110, the pop-out area is specified based on the determination result determined in step S108. Here, depending on whether or not the point of interest (or surrounding points) determined to be discontinuous in step S108 exists continuously over a predetermined threshold, it is a discontinuous line, that is, a pop-up area. It is determined whether or not there is. Here, when the attention points determined to be discontinuous exist continuously for a predetermined threshold value or more, it is determined to be a pop-out area. On the other hand, if the point of interest determined to be discontinuous does not exist continuously for a predetermined threshold value or more, it is not determined to be a pop-out area. This is for removing the detection noise and improving the reliability of the determination result.

  As described above, according to the present embodiment, the feature amount is compared between the attention point in the captured image and the surrounding points located around the attention point, so that in the real space corresponding to each of the attention point and the surrounding points. The depth discontinuity between the two points is determined. As a result, without using a special device, based on the determination result regarding the discontinuity of the depth between two points, the gap facing the road where the object may jump out (the depth becomes discontinuous) It becomes possible to detect.

  Since pedestrians and other vehicles move in parallel to the road surface, there is usually a gap along which the object pops out along the road surface. According to the present embodiment, the surrounding points to be compared with the attention point are set on a straight line connecting the attention point and the vanishing point. For this reason, it is possible to appropriately detect a gap where an object is likely to jump out in real space, that is, a gap existing along the road surface.

  As described above, a wall or the like that forms a gap from which an object pops out is often formed in a substantially vertical direction from the road surface. Here, in this embodiment, when the attention point determined to be discontinuous exists continuously for a predetermined threshold value or more, it is determined to be a pop-out area. That is, since the discontinuity in the real space is determined along the edge of the wall surface or the like that forms the gap, it is possible to improve the accuracy of detecting the gap where the object is likely to jump out.

  When the depth is discontinuous between the point of interest and the surrounding points, the brightness, optical flow, and three-dimensional coordinates on the captured image of the point of interest and the surrounding points are compared to when the depth is continuous. The value of changes significantly. According to the present embodiment, brightness, optical flow, and / or three-dimensional coordinates are used as feature quantities to be compared, thereby appropriately determining whether or not the depths of the attention point and the surrounding points are discontinuous. It becomes possible.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made.

It is a block diagram which shows the structure of the image processing apparatus which concerns on embodiment. It is a flowchart which shows the process sequence of the protrusion area | region determination process by the image processing apparatus which concerns on embodiment. It is an example of the captured image for demonstrating the setting method of a surrounding point. It is the bird's-eye view which looked at the real space corresponding to the captured image shown by FIG. 3 from upper direction.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 10 ... Camera, 20 ... Image processing ECU, 21 ... Attention point setting part, 22 ... Peripheral point setting part, 23 ... Feature-value extraction part, 24 ... Discontinuity determination part.

Claims (4)

Imaging means for imaging the front of the vehicle;
An attention point setting means for setting an attention point in the captured image captured by the imaging means;
Surrounding point setting means for setting surrounding points located around the attention point set by the attention point setting means;
An image comprising: determination means for determining discontinuity in the real space between the attention point and the surrounding point by comparing the feature amount of the attention point and the feature amount of the surrounding point. Processing equipment.   The image processing according to claim 1, wherein the surrounding point setting unit obtains a vanishing point in the captured image and sets the surrounding point on a straight line connecting the attention point and the vanishing point. apparatus. The attention point setting means sets a plurality of attention points in a substantially vertical direction of the captured image,
The surrounding point setting means sets a plurality of surrounding points corresponding to the plurality of attention points,
3. The determination unit according to claim 2, wherein the determination unit determines discontinuity in the real space by comparing the feature quantities of the plurality of attention points with the feature quantities of the plurality of surrounding points. The image processing apparatus described.   The image processing apparatus according to claim 1, wherein the feature amount of the attention point and the feature amount of the surrounding points are luminance, optical flow, or three-dimensional coordinates.

Images