JP3782322B2 - In-vehicle image processing camera device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an in-vehicle image processing camera apparatus, and more particularly to preprocessing for improving image recognition performance using a color imaging device.
[0002]
[Prior art]
In recent years, many systems have been developed in which an image processing camera having an image recognition processing function is applied to a vehicle. For example, as shown in FIG. 2, an in-vehicle image processing camera device that recognizes a white line installs an image processing camera 1 inside a windshield of a vehicle 2 and photographs a road surface 3 in front of the vehicle.
[0003]
FIG. 3 is an example of a road surface image taken by the image processing camera 1 and displays a white line 31 indicating a road boundary, a center line 32 indicating a center separation of lanes, and a road surface 30. 33 is the horizon.
[0004]
The lane recognition device recognizes the white lines 31 and 32 and calculates the inclination of the line segments, and measures the direction of the vehicle with respect to the lane and the distance to the white lines 31 and 32, thereby determining the side of the vehicle. Detect direction position. Using this white line recognition result, it is possible to construct a departure warning from a lane, an automobile line maintenance system, and the like. An example of such an apparatus is detailed in JP-A-11-242735.
[0005]
Such a white line recognition apparatus operates extremely stably when white lines are clearly imaged, and white line recognition is performed. FIG. 4 shows a configuration example of white line recognition. The image processing apparatus 1 stores a video signal captured by the camera 11 in a memory (RAM) 14 that can be written and read by an image capturing circuit 12 and a direct memory access circuit (DMA) 160 built in the microcomputer 16. For example, white lines are recognized in accordance with a program stored in a read-only memory (ROM) 15. The recognition result is sent to an alarm device or the like via the communication port 161.
[0006]
In the in-vehicle environment, the brightness of the road surface to be imaged changes greatly, so it is necessary to dynamically change the sensitivity of the camera 11. For this reason, a command for controlling the brightness is sent from the microcomputer 16 to the camera 11 via the built-in communication circuit 162.
[0007]
[Problems to be solved by the invention]
Conventionally, a black and white camera is often used as an imaging device used for white line recognition. Even when a color device is used, it is usual to use only a luminance signal. In other words, there is no example of efficiently recognizing a lane marker such as a white line using the characteristics of color information.
[0008]
In the Japanese road environment, yellow lines are used as lane markers in addition to white lines. For this reason, considering white line recognition as a center, there is a problem that the yellow line recognition rate decreases. As a method for solving this problem, there is a technique disclosed in, for example, Mitsubishi Electric Technical Bulletin (Vol. 74, No. 9, 2000, page 43). According to the technical report, the yellow line recognition rate is improved by using a special optical filter. However, since an optical filter is employed, the optical system becomes complicated, and it is difficult to change the filter characteristics flexibly according to the characteristics of the recognition object.
[0009]
The object of the present invention has been made in view of the above-mentioned problems of the prior art, and does not use a special optical filter, can flexibly cope with a recognition target, and has high recognition performance for in-vehicle image processing. It is to provide a camera device.
[0010]
[Means for Solving the Problems]
In the present invention, the above-mentioned object is achieved by individually extracting color signals from a color imaging device and dynamically changing the optimum combination of these signals.
[0011]
For example, when a lane marker drawn on a road is recognized, if a yellow line is mixed, a yellow screen is synthesized from the image information of the red component and the green component so that the contrast of the lane marker is increased, and the synthesis is performed. Perform recognition processing on the screen.
[0012]
More generally, a combination of three primary colors, that is, only R, only G, only B, the sum of R and G, the sum of R and B, the sum of B and G, and the sum of R and G and B For each, the contrast between the background (for example, the road surface) and the recognition target (for example, lane marker) is evaluated, and the recognition process is performed on the combined screen that provides the maximum contrast.
[0013]
The present invention is not limited to recognition of lane markers. For example, it is possible to recognize a traffic light or the like. In this way, a better recognition result can be obtained by synthesizing an image having the maximum contrast in various environments.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of an in-vehicle image processing camera device according to an embodiment. This block is basically the same as the conventional example of FIG. 4, and the same symbols are the same elements. The difference is that the camera unit is set to color output, and the three primary colors of red R, green G, and blue B are captured, and the capture circuit 12 performs the same correspondence accordingly.
[0015]
FIG. 5 shows a detailed circuit example of the camera 11. The image sensor 110 includes a CMOS, a CDS / AGC circuit 111 that performs camera signal processing, an ADC 112 that converts video into a digital signal, and a control circuit 113. The CDS / AGC circuit 111 has a function of receiving a command by communication from the outside and changing the gain of the video amplifier. The ADC 112 has a function of AD converting and outputting the three primary colors of R, G, and B. Further, the control circuit 113 can receive so-called electronic shutter control by receiving commands from the outside and controlling the CMOS sensor 110. Recently, a product in which all such circuits are configured by one chip has begun to be provided, and the circuit can be easily realized.
[0016]
In this embodiment, the three primary colors RGB are captured. However, since the signal format expressed by the combination of the luminance signal and the color difference signal can be equivalently converted to the three primary colors RGB, it goes without saying that this signal format may be used.
[0017]
FIG. 6 shows a flowchart of processing operations performed by the microcomputer 16. The program of the microcomputer 16 is stored in the ROM 15, and when the power is turned on, the program stored in the ROM 15 is activated.
[0018]
First, initialization is performed in S10. In this step, register setting in the microcomputer, communication circuit setting, and clear processing of a predetermined area of the RAM 14 are performed as necessary.
[0019]
Next, in S11, the sensitivity of the camera 11 is set. Normally, feedback processing is performed by processing the imaging result, but immediately after the initialization processing, the default sensitivity is set.
[0020]
FIG. 7 shows a detailed flow of camera sensitivity setting. First, in S110, it is determined whether or not it is immediately after initialization S10. If it is immediately after initialization, the video necessary for camera sensitivity adjustment has not been taken, so the default brightness is set in S112, and the process proceeds to S113. If it is not immediately after initialization, measure the brightness using the previously captured video.
[0021]
FIG. 8 is an explanatory diagram of brightness measurement. In the brightness measurement, the average brightness is calculated by taking in data of a predetermined area of the screen. For example, if a clear sky is included in the calculation target, the sensitivity will be reduced more than necessary, and as a result, the lane markers 31 and 32 and the road surface 30 that are recognition targets cannot be captured correctly.
[0022]
For example, if the imaging device is a CCD (Charge Coupled Device), when the sun 34 enters the field of view, when the sensitivity is adjusted to the road surface 30, a so-called smear phenomenon occurs due to the brightness of the sun 34, and a large vertical stripe 35 is formed. There appears a problem that appears on the screen and in some cases, the recognition object is hidden by smear. However, in this embodiment, a CMOS sensor is used. Unlike a CCD, a CMOS sensor has a different video signal transfer method, so that it is difficult for a smear phenomenon to occur, and a good recognition result can be obtained.
[0023]
Returning to FIG. 7 again, the description will be continued. In S113, the shutter speed is calculated based on the measured brightness. In S114, the gain value of AGC is determined. In S115, the shutter speed value and the AGC value obtained above are transmitted to the camera 11 by communication.
[0024]
The camera sensitivity setting method is performed by combining the so-called electronic shutter speed and the gain adjustment of the video signal amplifier. This can be handled by a known technique.
[0025]
Next, returning to FIG. 6, the description will be continued. In S12, video data is captured. This is an operation of taking video data for one screen into the SRAM 14 for each color of red (R), green (G), and blue (B).
[0026]
In S <b> 13, the captured red (R) green (G) data for each color is added for each corresponding pixel and stored in a predetermined area of the SRAM 14. Thereafter, the contrast between the road surface 30 and the lane markers 31 and 32 is calculated. Details will be described later. In S14, the sum of red (R), green (G), and blue (B) is similarly obtained, and the contrast between the road surface 30 and the lane markers 31 and 32 is similarly calculated.
[0027]
The processes of S13 and S14 are efficient if they are calculated as shown in FIG. That is, R and G stored in the memory are added and stored in the memory (R + G). Further, B data is added to R + G and stored in the SRAM 14 in the same manner.
[0028]
Next, a contrast evaluation method in S13 and S14 will be described. FIG. 10 shows an image of a road viewed from a vehicle. Here, for the sake of explanation, the color of the roadside lane marker 31 is white and the center line 32 is yellow. This image is directly taken into the memory for each color, and as described above, R + G and R + G + B are calculated and stored in the memory in the same manner.
[0029]
Here, an inspection line for evaluating contrast is provided on the memory. This corresponds to a case where the vehicle is illuminated by a headlamp of a car during night driving, and takes into consideration that the brightness does not change much in the lateral direction.
[0030]
FIG. 11 shows an example in which a luminance change is taken with the inspection line as the horizontal axis. A solid line is a luminance value of R + G + B, and a broken line is a luminance value of R + G. The A portion has the luminance corresponding to the white line 31 in the A portion of FIG. 10, and the B portion has the luminance corresponding to the center line 32 in the same manner.
[0031]
In the case of R + G + B, the brightness is high in the white line portion (A portion), and the value of Wrgb is taken. Further, since the luminance is slightly reduced in the yellow line portion (B portion), a slightly lower luminance value Yrgb is taken. On the road surface portions other than the white line and the yellow line, since black is mainly, the luminance value is low and the average value Brgb is taken.
[0032]
On the other hand, the luminance value distribution by R + G is as shown by a broken line. That is, in the white line portion (A portion), the luminance value of Wrg that is slightly lower than Wrgb is taken as much as the blue component is removed. On the other hand, in the yellow line part (B part), since yellow is originally mainly, the drop in the detected luminance value with respect to Yrgb is small and takes the value of Yrg. Similarly, the road surface portion takes a luminance value Brg lower than Brgb.
[0033]
The ratio of the luminance value between the lane marker portion (A, B portion) and the other road surface portion is the contrast. Judging from this characteristic, the contrast of R + G + B and R + G does not change much in the white line portion. On the other hand, in the yellow line portion, the contrast of R + G is maximized. That is, in the case of having the characteristics as shown in FIG. 11, R + G can be recognized more easily because the contrast is higher.
[0034]
Returning to FIG. 6, the description will be continued. In S15, as described above, the contrast characteristics are compared. In S16, recognition is performed on the composite image that provides a higher contrast in S15. In S17, the recognition result is transmitted to the host system by communication. For example, as described in Japanese Patent Laid-Open No. 6-333192, white line recognition means uses a means such as image differentiation from a video to obtain a plurality of candidate points for a white line portion, and a means such as linear approximation. Thus, it can be realized by estimating the white line portion.
[0035]
In general, in the case of daytime, the characteristics shown in FIG. 11 can be obtained. However, since the brightness decreases at night, there is no guarantee that a high R + G contrast can be obtained. In such a case, a composite image with a higher contrast can be used by evaluating each combination in S15. According to this, flexibility becomes high compared with the system using a fixed optical filter. In addition, since the present embodiment requires less computation processing, it also has a feature that the target function can be realized even if the computation processing is somewhat slow.
[0036]
The above explanation is based on the assumption that the lane marker is a white line and a yellow line, but the color of the road surface that the lane marker is the background of, or how the light hits, for example, the spectrum of the light changes at sunset, sunrise, etc. A flexible response is required.
[0037]
FIG. 12 shows each luminance by the lane marker and the three primary colors of the road surface. To describe the output value of red (R) in FIG. 10, W1r is a detected luminance value of the lane marker 31, and W2r is a detected luminance value of the lane marker 32. Br is a detected luminance value of the road surface 30. Similarly, the luminance values of green (G) and blue (B) are represented by W1g, W2g, Bg, W1b, W2b, and Bb.
[0038]
As described before, in order to achieve the best recognition of the lane marker, it is necessary to maximize the contrast of not only one lane marker but also both lane markers. A method for this will be described.
[0039]
FIG. 13 shows values obtained by combining the values of the three primary colors shown in FIG. For example, the screen R + B is a case where the screen is synthesized using information on red (R) and blue (B), and W1r, W2r, W1b, and W2b are added as S. On the other hand, Br and Bb are added as N for the road surface as the background. The contrast is a value obtained by dividing S by N (S / N). Since other combinations are the same, details are omitted. As the best combination, the combination that takes the maximum value from the S / N column in the figure may be selected. About the above algorithm, it is implement | achieved by the arithmetic processing of the microcomputer 16 in FIG.
[0040]
Next, as another embodiment of the present invention, an example applied to traffic signal recognition will be described. FIG. 14 is an image captured by the camera 11 of FIG. A traffic light 36 is installed beside the road. In order to recognize the lamp color of the traffic light 36, it is only necessary to specify a lamp having high brightness from red, yellow, and green. In Japan, since the lamps of traffic lights are arranged in the order of green, yellow, and red from the left, there are cases where it is possible to estimate the lighting color based on the luminance alone without necessarily recognizing the color itself.
[0041]
In the case of traffic signal recognition, unlike the white line written on the road surface described above, the location where the traffic signal itself is installed is not constant. For this reason, there is a possibility that the position of the traffic light becomes unspecified on the screen viewed by the camera, and the recognition efficiency is deteriorated. In order to improve this, the presence area of the traffic light may be limited by some means and the recognition area 37 may be provided.
[0042]
FIG. 15 shows an example of obtaining information on the installation location of the traffic light. Information on the location of the traffic light can be obtained from a navigation device or the like. That is, information on the position of the traffic light from the in-vehicle navigation device 4 (for example, the direction from the center of the intersection, the two-dimensional distance of east, west, south, and north), the height, shape (horizontal and vertical) of the traffic signal, and the vehicle position information from the intersection. The recognition area 37 can be acquired by taking the image processing camera 1 and estimating the position of the traffic light. If the recognition area 37 can be acquired, an image that maximizes the contrast ratio between the lamp 36 and the area (background) may be combined with the result obtained by changing the combination of the three primary colors to perform recognition processing.
[0043]
【The invention's effect】
According to the present invention, a special optical filter is not used, and an image with high contrast can be selected and synthesized according to the conditions of the outside world, and the recognition process can be performed on the result. An image processing camera device with better recognition performance can be provided.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an image processing camera device according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing a visual sense of an in-vehicle image processing camera device.
FIG. 3 is an explanatory diagram showing an image in front of the vehicle photographed by the image processing camera device.
FIG. 4 is a configuration diagram of a conventional image processing camera.
FIG. 5 is a block diagram showing a detailed configuration of a color camera.
FIG. 6 is a flowchart showing processing of the image processing camera device according to the embodiment of the present invention.
FIG. 7 is a flowchart for explaining camera sensitivity adjustment;
FIG. 8 is an explanatory diagram showing a region for inspecting brightness for camera sensitivity adjustment;
FIG. 9 is a configuration diagram for synthesizing an image with three primary colors.
FIG. 10 is an explanatory diagram showing an inspection line on a video screen for sampling a luminance value.
FIG. 11 is an explanatory diagram showing a composite lane marker and road surface luminance values based on RGB and RG signals on the inspection line;
FIG. 12 is an explanatory diagram showing lane markers and road surface brightness values in RGB three primary colors.
FIG. 13 is an explanatory diagram illustrating a method for calculating contrast on a composite screen using RGB three primary colors.
FIG. 14 is an explanatory diagram showing a traffic signal recognition area.
FIG. 15 is a conceptual diagram for obtaining a traffic signal recognition area using transmission information of a navigation device.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image processing camera apparatus, 2 ... Vehicle, 3 ... Road, 4 ... Navigation apparatus, 11 ... Camera, 12 ... Image capture circuit, 14 ... RAM, 15 ... ROM, 16 ... Microcomputer, 30 ... Road surface, 31, 32 ... Lane marker, 36 ... traffic light, 37 ... recognition area.

Claims (6)

An imaging device capable of taking out a three-color signals of the individual is mounted on a vehicle by photographing a road surface on which the vehicle is traveling, performs recognition processing of the lane on the road surface for the resulting Film image In an in-vehicle image processing camera device,
The three color signals obtained by the imaging device are independently captured, the color signal combinations are changed and added, and the contrast between the recognition object and the background is calculated for the result, and the road surface and the road surface An in-vehicle image processing camera device that obtains a combination having a maximum contrast with the upper lane marker and performs the recognition processing calculation of the lane . In claim 1 ,
The three color signal combinations include all the combinations of the three primary colors R, G, and B, or all combinations of luminance signals and color difference signals that can be equivalently converted to the three primary colors. An in-vehicle image processing camera device obtained by evaluating In claim 1 ,
When said recognition target includes lane marker yellow line in-vehicle image processing camera apparatus characterized by pre-Symbol obtain combinations obtained by adding the red and green output signal of the imaging device. In claim 1, 2 or 3 ,
An in-vehicle image processing camera apparatus characterized in that after a color signal of an imaging device is once taken into a memory, a composite image is obtained by adding image information of each color. In claim 1, 2 or 3 ,
When evaluating the contrast of the recognition object, an inspection line is provided in the horizontal direction within the irradiation range of the headlamp of the car with respect to the imaging screen, and the contrast is evaluated on the inspection line. In-vehicle image processing camera device. In any one of Claims 1 thru | or 5 ,
It said imaging device is a CMOS sensor,
By using the captured image area excluding the empty part of the image, the image processing camera for use in vehicles and lane markers on the road surface and the road surface and selects the combination of the color signal having the maximum contrast apparatus.

Images