JP2010067053A - Vehicle environment recognition apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for matching the recognition result of the color of each pixel in a color image with the color perception of human being in response to difference in environment around a vehicle. <P>SOLUTION: The vehicle environment recognition apparatus 100 calculates a color position by coordinate-conversion of the original position in the RGB space showing the color of each pixel of the color image into L<SP>*</SP>a<SP>*</SP>b<SP>*</SP>space. Further, illuminance (or brightness) is measured, the color position of the pixel is corrected so as to be displaced in parallel to the L<SP>*</SP>axis based on the measured value, and each pixel is determined to be a pixel having a color corresponding to the region including the corrected color position. The correction of the color position reduces or eliminates the influence of the difference in illuminance or brightness around the vehicle 1 to the color recognition manner. One color is determined as a color of each pixel correspondingly to one region defined according to the difference of conspicuity in the color space, difference of warm color and cold color, difference of expansive color and contracting color, or the like. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus for recognizing a surrounding environment of a vehicle.

A technique for recognizing a road sign such as a white line or a yellow line provided on a road on which a vehicle is traveling has been proposed based on a color image obtained through an in-vehicle camera (see Patent Document 1). Specifically, first, the RGB value of each pixel of the color image is projected onto a three-dimensional coordinate system different from the RGB coordinate system. Each axis of this three-dimensional coordinate system is defined by, for example, hue, brightness, and saturation. Each pixel in the color image is recognized as a pixel having a representative value of the divided area in the three-dimensional coordinate system including the projection position.
Japanese Patent Application Laid-Open No. 06-338991

  However, depending on the light intensity around the vehicle, etc., the white line provided on the road on which the vehicle is traveling is recognized as a reddish advance color or as a blued reverse color, etc. Colors may be recognized in a form that does not match human color perception.

  Therefore, an object of the present invention is to provide an apparatus capable of matching the color recognition result of each pixel in a color image with human color perception according to the difference in environment around the vehicle.

  According to a first aspect of the present invention, there is provided a vehicle environment recognition device that captures an image of the periphery of a vehicle, a measuring device that measures a value of a designated variable representing a color element in the periphery of the vehicle, and a color image captured by the image capture device. The color position is calculated by transforming the original position in the RGB space representing the color of the pixels constituting the color space into a color space having a designated axis defined by the designated variable, and the measurement is performed by the measuring device. A first processing unit that calculates the color position of the pixel by correcting the coordinates of the specified axis of the color position according to a value of a specified variable, and each of a plurality of color regions in the color space and a color A color area storage unit that associates and stores, and the color area that includes the color position of the pixel calculated by the first processing unit in the color space is associated with the color area storage unit. The colors are to have stored, characterized in that a second processing unit for determining a color of the pixel.

  According to the vehicle environment recognition device of the first aspect of the invention, the color position is calculated by coordinate-transforming the original position in the RGB space representing the color of each pixel of the color image into the color space. Furthermore, the designated axis of the color space is defined and the value of the designated variable representing the color element is measured. Then, based on the measured value of the designated variable, the color position of the pixel is corrected so as to be displaced parallel to the designated axis, and each pixel is determined to be a pixel having a color corresponding to the area including the corrected color position. Is done. By correcting the color position, the influence of the difference in the value of the designated variable around the vehicle on the color recognition mode can be reduced or eliminated. In addition, since one color is determined as the color of each pixel corresponding to one area defined according to the difference in attractiveness, the difference between warm and cold colors, the difference between expanded and contracted colors, etc. in the color space. The color of each pixel can be recognized in such a manner that the sense of incongruity in consideration of human color perception is eliminated. Accordingly, the color recognition result of each pixel in the color image can be matched with the human color perception according to the difference in the measured value of the designated variable and the difference in the environment around the vehicle.

  In the present invention, categorical colors in human color perception are used as the plurality of color regions. The categorical basic colors are based on categorical color judgment of color stimulation, and are 11 colors of red, brown, orange, yellow, green, blue, purple, pink, black, gray, and white. In addition, due to the characteristics of a running image in which the area ratio of the blue or gray sky area and the gray road area is often high in the entire image, blue is divided into two by lightness, and gray is divided into three by lightness. A method of dividing and painting an image into 14 categorical color regions may be used.

The vehicle environment recognition device according to a second aspect of the present invention is the vehicle environment recognition device according to the first aspect, wherein the measuring device measures illuminance around the vehicle as the designated variable, and the first processing unit is measured by the measuring device. The calculation is performed by correcting the L ^* coordinate of the color position in the L ^* a ^* b ^* space as the color space based on illuminance.

  According to the vehicle environment recognition device of the second aspect of the present invention, the color recognition result of each pixel in the color image and the human color perception match according to the environmental difference around the vehicle, specifically, the illuminance difference. Can be achieved.

  An embodiment of a vehicle environment recognition device of the present invention will be described. First, the configuration of the vehicle environment recognition device will be described. A vehicle environment recognition apparatus 10 shown in FIGS. 1 and 2 is an electronic control unit (CPU, ROM, RAM, A / D conversion circuit and a bus line connecting them) mounted on a vehicle (four-wheeled vehicle) 1. It is comprised by.) It is comprised. A real space coordinate system is defined in which the center of the front end of the vehicle 1 is the origin O, and the right direction, the downward direction, and the forward direction of the vehicle 1 are the + X direction, the + Y direction, and the + Z direction, respectively. The vehicle environment recognition device 10 includes a camera (imaging device) C such as a CCD camera or a CMOS color image sensor, and an illuminometer (measuring instrument) S. The camera C captures an image including the road ahead of the vehicle 1 with the forward direction of the vehicle 1 as the imaging range. The camera C is disposed so as to be positioned on the center line of the vehicle 1 in the width direction. The illuminometer S measures the value of illuminance (designated variable) around the vehicle 1.

The vehicle environment recognition device 10 includes a first processing unit 11, a second processing unit 12, and a color area storage unit 13. The first processing unit 11 and the second processing unit 12 include a memory and an arithmetic processing unit (CPU) that reads a program from the memory and executes assigned arithmetic processing. The first processing unit 11 to the original position in the RGB space representing the color of pixels constituting the color image captured by the camera C, illuminance L having L ^* axis defined by (specified variable) (specified axis) ^* a ^* b ^* Calculate the color position by transforming the coordinates into space (color space). The first processing unit 11 calculates the color position of each pixel by correcting the coordinates of the L * axis of the color position according to the illuminance measured by the illuminometer. The second processing unit 12 is stored in correspondence with the color region in the L ^* a ^* b ^* space including the color position of the pixel calculated by the first processing unit 11 in the color space by the color region storage unit 13. Is determined as the color of this pixel.

  As shown in FIG. 2, the vehicle 1 further includes sensors such as a vehicle speed sensor 101, an acceleration sensor 102, and a yaw rate sensor 103, a steering device 110, and a braking device 120. Each of the vehicle speed sensor 101, the acceleration sensor 102, and the yaw rate sensor 103 outputs signals corresponding to the speed, acceleration, and yaw rate of the vehicle 1, respectively. The steering device 110 is configured to drive a front wheel steering mechanism driven by operation of a steering wheel by an actuator. The steering device 110 may drive the rear wheel steering mechanism instead of or in addition to the front wheel driving mechanism. The vehicle environment recognition device 10 controls the behavior of the vehicle 1 by controlling the operation of one or both of the steering device 110 and the braking device 120 based on the recognition result of the color of each pixel constituting the color image.

The function of the vehicle environment recognition device 10 having the above configuration will be described. First, when an image signal is input from the front camera C1 to the vehicle environment recognition device 10, a color image is acquired by the first processing unit 11 (FIG. 3 / S002). Thereby, for example, as shown in FIG. 4A, a color image showing the state of the road ahead of the vehicle 1 is acquired. Next, the first processing unit 11 calculates an RGB value representing the color of each pixel of the color image, that is, a position in the RGB space (FIG. 3 / S004). Subsequently, the first processing unit 11 performs coordinate conversion of the position in the RGB space representing the color of each pixel into the L ^* a ^* b ^* space according to the function f, whereby the color of each pixel in the L ^* a ^* b ^* space. The position is calculated (FIG. 3 / S006). Thus, for example, a point (R ₀ , G ₀ , B ₀ ) in the RGB space shown on the upper side of FIG. 5 becomes a point (L ₀ ^** ) in the L ^* a ^* b ^* space shown on the lower side of FIG. , A ₀ ^* , b ₀ ^* ) = f ((R ₀ , G ₀ , B ₀ )). Moreover, the illuminance around the vehicle 1 is measured by the illuminometer S (FIG. 3 / S008). Based on the illuminance measured by the illuminometer S, the second processing unit 12 corrects the color position so that the color position is displaced parallel to the L ^* axis (FIG. 3 / S010). For example, the color position is corrected such that the higher the measured illuminance, the greater the displacement in the L ^* -axis negative direction (−L ^* direction), while the lower the measured illuminance, the greater the displacement in the L ^* -axis positive direction (+ L ^* direction). Thus, the color position is corrected. As a result, for example, as indicated by an arrow on the lower side of FIG. 5, the point (L ₀ ^* , a ₀ ^* , b ₀ ^* ) in the L ^* a ^* b ^* space is displaced in parallel to the L * axis. It is corrected. Then, the second processing unit 12 includes the corrected color position of the pixel among the plurality of color regions defined in the L ^* a ^* b ^* space stored by the color region storage unit 13. After determining one color region, the color associated with the one color region and stored in the color region storage unit 13 is determined as the color of this pixel (FIG. 3 / S012). Each color region can be determined by experiments or the like. Categorical basic colors i (i = red, brown, orange, yellow, green, blue, purple, pink, black, gray, white) for L ^* a ^* b ^* color space (closed area), which is defined in the space Vi Is included in the corrected color position, the color of the pixel is determined as “i”. For example, when the corrected color position is included in the color region (closed region) Vwhite (i = white) defined in the L ^* a ^* b ^* space, the color of the pixel is determined to be “white”. . Thus, for example, as shown in FIG. 4B, even if the road and the lane mark are bluish due to illuminance or light irradiation conditions in the original color image, the pixel portion corresponding to the road is human. A color image in which the pixel portion corresponding to lane mark (white line) LM is corrected to white perceived as the color of the lane mark is obtained.

  Then, a pair of left and right lane marks LM are extracted from the corrected color image, a traveling zone in which the left and right are demarcated by the lane mark LM is determined, and as necessary so that the vehicle 1 does not protrude from this traveling zone. Thus, the operation of one or both of the steering device 110 and the braking device 120 is controlled.

According to the vehicle environment recognition apparatus 100 that exhibits the above function, the color position is calculated by coordinate-transforming the original position in the RGB space representing the color of each pixel of the color image into the L ^* a ^* b ^* space ( (See FIG. 3 / S006, FIG. 5). Further, the illuminance (or brightness) is measured, and based on this measurement value, the color position of the pixel is corrected so as to be displaced parallel to the L ^* axis, and each pixel corresponds to a region including the corrected color position. It is determined that the pixel has a color (see arrows in FIGS. 3 / S010, S012, and FIG. 5). By correcting the color position, the influence of the difference in illuminance or lightness around the vehicle 1 on the color recognition mode can be reduced or eliminated. In addition, since one color is determined as the color of each pixel corresponding to one area defined according to differences in attractiveness, differences between warm and cold colors, differences between expanded and contracted colors, etc. in the color space. Then, the color of each pixel can be recognized in such a manner that the sense of incongruity in view of human color perception is eliminated (see FIG. 4B). Therefore, the color recognition result of each pixel in the color image and the human color perception can be matched according to the difference in illuminance and the difference in environment around the vehicle 1.

Incidentally, L ^* a ^* b ^* in place of the corrected is that of or in addition to displaced parallel to the L ^* axis direction on the basis of the color position illuminance in space, one of the chromaticity a ^* and b ^* or The color position may be corrected so as to be displaced parallel to both axes. In this case, the color position can be corrected after a hue or saturation reference is determined from a color image obtained by the camera C as a measuring instrument or a camera mounted on the vehicle 1 separately from the camera C.

Further, the position of the RGB space representing the color of each pixel constituting the color image is changed to a color in a color space such as CMY space, CMYK space, CMK space, HSV space, HLS space, etc., instead of the L ^* a ^* b ^* space. The coordinates may be converted into a position. The color position coordinate-converted into the HSV space or HLS space can be corrected based on the illuminance measurement result by the illuminometer S so as to be displaced in a direction parallel to the V axis (brightness axis) and the L axis (luminance axis). . The color position coordinate-converted into CMY space, CMYK space, or CMK space is measured based on a color image obtained by a camera C as a measuring device or a camera mounted on the vehicle 1 separately from this. Correction can be made based on the reference value of (C).

Structure explanatory drawing of the vehicle environment recognition apparatus of this invention Structure explanatory drawing of the vehicle environment recognition apparatus of this invention Flow chart showing the procedure of lane mark recognition processing Explanatory drawing about correction of each pixel in a color image Explanatory drawing about coordinate conversion from RGB space to color space

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Vehicle environment recognition apparatus, 11 ... 1st process part, 12 ... 2nd process part, 13 ... Color area memory | storage part, C ... Camera (imaging device), S ... Illuminance meter (measuring instrument)

Claims (2)

An imaging device for imaging the periphery of the vehicle;
A measuring device for measuring a value of a designated variable representing a color element in the vicinity of the vehicle, and an original position in RGB space representing a color of a pixel constituting a color image taken by the imaging device is defined by the designated variable. A color position is calculated by converting the coordinates into a color space having a designated axis, and the coordinates of the designated axis of the color position are corrected according to the value of the designated variable measured by the measuring device. A first processing unit that calculates the color position of the pixel by:
A color area storage unit that stores each of the plurality of color areas in the color space in association with each other; and
In the color space, the stored color associated with the color area including the color position of the pixel calculated by the first processing unit by the color area storage unit is used as the color of the pixel. A vehicle environment recognition device comprising: a second processing unit for determining. The vehicle environment recognition device according to claim 1,
The measuring device measures the illuminance around the vehicle as the designated variable,
Vehicle, characterized in that calculating by the first processing unit on the basis of the illuminance measured by the measuring device, to correct the L ^* coordinate of the color position in the L ^* a ^* b ^* space as the color space Environment recognition device.

Images