JP3413745B2 - Road surface monitoring device by image processing - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention is mounted on a vehicle,
Based the travel path of the vehicle to the image data captured by the television camera for the imaging field of view, are those concerning the device for recognizing the lane the vehicle should travel.

[0002]

2. Description of the Related Art As an on-vehicle camera, that is, a conventional technique for recognizing a traveling lane based on image data captured by a television camera mounted on a vehicle and having a traveling path of the vehicle as an imaging field of view, For example, JP-A-62
There is one disclosed in Japanese Patent No. 221800.

In this prior art, for example, the front of the vehicle is photographed by a color image pickup device, color image data is formed based on the video signal, image data of a color expressing a lane is extracted from this color image data, and this extraction is performed. The presence of a lane is judged based on the image data thus obtained.

[0004]

By the way, in many cases, the image data taken by the vehicle-mounted camera shows the state of the data even in the same image in the short distance area and the long distance area from the own vehicle. It's very different. For example, in the daytime, in a long distance area, the road surface reflects sunlight and is often whitish, that is, the brightness is higher than in a short distance area. Also, at night,
In the short distance area, the headlight of the vehicle is bright, while in the long distance area, the headlight does not reach,
It is often dark.

Therefore, in the method of extracting an image signal of a certain color from the image data as in the prior art, in order to simultaneously extract the lanes in both the short distance and the long distance, the conditions of the color to be extracted are set. , The short-distance area and the long-distance area must be set separately.

However, according to the above-mentioned conventional technique, only one color condition for extracting a lane can be set. Therefore, if a color condition for extracting a lane in a short distance region is set, a long distance is set. Lane cannot be extracted in the area
Also, if the color condition is set so that the lane can be extracted in the long distance area, the lane cannot be extracted in the short distance area. In other words, there is a problem that the lane cannot be accurately extracted at the same time in both the short distance and long distance areas.

In view of the above-mentioned problems of the prior art, the present invention recognizes the lane of the vehicle with respect to the driving lane based on the image information taken by the vehicle-mounted camera. An object of the present invention is to provide a device that can simultaneously recognize a lane in a region.

[0008]

In order to achieve the above object, according to the present invention, a road surface in front of a vehicle is photographed by an image pickup device, and extraction is performed to extract a lane from a short-distance area of the photographed image information. Set the conditions and the extraction conditions for extracting the lane from the long-distance area, extract the pixels that satisfy the extraction color conditions from the short-distance area and the long-distance area, and extract the lane of the lane based on the extracted pixels. It is designed for recognition.

According to the present invention, the conditions for extracting the lanes are set independently for the short-distance region and the long-distance region. Therefore, the lanes are defined for the short-distance region and the long-distance region. It becomes possible to recognize at the same time.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described below with reference to the illustrated embodiments. FIG. 1 is a block diagram of an embodiment of the present invention. In this embodiment, a photographing unit 1, a processing unit 5, and a memory 2 are provided.
3, the display unit 14, and the processing unit 5 further includes an image processing unit 6 and a determination unit 7. Then, the vehicle state sensor 20, the vehicle motion control device 22, and the alarm device 2
A signal is transmitted and received between 1 and the determination unit 7.

Next, each component of this embodiment will be described. First, the photographing unit 1 will be described. The CCD camera 2 is a means for capturing an image in front of the vehicle and obtaining image information, and has a function of converting image information of a subject in front of the vehicle into a color analog signal. For this reason, when mounted on a vehicle, for example, it may be installed near a grill on the front of the vehicle, in a place where there is little dirt, or in a place with a wide front field of view such as a room mirror or a sun visor in the vehicle interior.

The A / D converter 3 performs a process of converting an analog image signal output from the CCD camera 2 into a digital image signal. The color difference conversion circuit 4 uses the digital signal output from the A / D converter 3, that is, the three primary colors of the subject.
Digital signals corresponding to (R (red), G (green), B (blue))
A process of converting the luminance signal Y and the color difference signals RY and BY defined by the following equations is performed.

Y = 0.3R + 0.6G + 0.1B R−Y = 0.7R−0.6G−0.1B B−Y = −0.3R−0.6G + 0.9B The luminance signal obtained by conversion by the above equation The Y and the color difference signals RY and BY are input to the delay circuit 16 and stored in the buffer 25.

Next, the processing unit 5 will be described, but since the operation content thereof will be described later, the function of each component will be outlined here. As shown in FIG. 1, the processing unit 5 includes an image processing unit 6 and a determination unit 7. Therefore, first, the components of the image processing unit 6 will be described.

The buffer 25 is a means for storing the image information sent from the color difference conversion circuit 4, but does not necessarily have a capacity for one frame. For example, if the unit of image processing is three lines (three scanning lines), it may be configured to have a capacity for three lines.

The short-distance area color sampling circuit 27 receives the image information from the buffer 25, and extracts the color data (Y, RY, B from the pixel at the sampling position designated by the short-distance area extraction condition determining unit 29 described later). -Y) is sampled, and the sampled data is transferred to the long-distance region extraction condition determining unit 30 and the short-distance region extraction condition determining unit 29.

The long-distance area color sampling circuit 28 receives the image information from the buffer 25, and extracts the color data (Y, RY, B from the pixel at the sampling position designated by the long-distance area extraction condition determining unit 30 described later). -Y) is sampled and the sampled data is transferred to the short distance area extraction condition determining unit 29 and the long distance area extraction condition determining unit 30.

The long-distance area extraction condition determining unit 30 uses color data corresponding to road colors stored in the memory 23 in advance.
(Y, RY, BY) and the color data (Y, RY, BY) sampled by the long-distance area color sampling circuit 28 and the short-distance area color sampling circuit 27. The area extraction color condition is determined, and the processing is transferred to the long-distance area edge extraction processing circuit 10.

The long-distance area extraction condition determining unit 30
Receives the lane information recognized in the process of the previous cycle from the lane recognizing unit 12, determines an area to be subjected to the long-distance area edge extraction processing described later, and sets the long-distance area edge extraction processing area data to the long-distance area. Processing for transferring to the area edge extraction processing circuit 10 is performed.

Further, the long-distance area extraction condition determining unit 3
0 receives the lane information recognized in the process of the previous cycle from the lane recognition unit 12, determines the position of the long-distance area sampling pixel for sampling the color data, and sets the sampling position data to the long-distance area color sampling circuit. 28
Process to transfer to.

The long-distance area extraction color condition, the long-distance area sampling position, and the long-distance area edge extraction processing area, which are long-distance area extraction conditions, may be set in advance.

The short-distance area extraction condition determining unit 29 uses color data corresponding to road colors stored in the memory 23 in advance.
(Y, RY, BY) and the color data (Y, RY, BY) sampled by the long-distance area color sampling circuit 28 and the short-distance area color sampling circuit 27. The distance area extraction color condition is determined and transferred to the short distance area edge extraction processing circuit 26.

Further, this short distance area extraction condition determining unit 29
Receives the lane information recognized in the process of the previous cycle from the lane recognizing unit 12, determines an area to be subjected to the short-distance area edge extraction processing described later, and sets the short-distance area edge extraction processing area data to the short-distance area. Processing for transferring to the area edge extraction processing circuit 26 is performed.

Further, the short-distance area extraction condition determining unit 2
9 receives the lane information recognized in the process of the previous cycle from the lane recognition unit 12, determines the position of the short-distance area sampling pixel for sampling the color data, and outputs the sampling position data to the short-distance area color sampling circuit. 27
Process to transfer to.

The short-distance area extraction condition, the short-distance area extraction color condition, the short-distance area sampling position, and the short-distance area edge extraction processing area may be set in advance.

The electronic zoom circuit 8 receives the image information from the buffer 25, and in accordance with the zoom condition (zoom center coordinates and enlargement ratio) determined for each horizontal line by the zoom condition determining unit 9 which will be described later, the image is displayed. After performing the process of enlarging the information only in the horizontal direction (for example, every one horizontal line), the process of transferring the enlarged image information to the long-distance region edge extraction processing circuit 10 is performed.

Here, the zoom center coordinate is a parameter indicating the position of a reference point for performing the enlargement process when enlarging the image information, and the enlargement ratio is a parameter indicating the magnification for enlarging the image information.

The long-distance area edge extraction processing circuit 10 includes a long-distance area extraction processing area transferred from the long-distance area extraction condition determining unit 30 in the enlarged image information transferred from the electronic zoom circuit 8. From the pixel, the pixel corresponding to the long-distance area extraction color condition determined by the long-distance area extraction condition determination unit 30 is extracted, image processing for removing noise and the like is performed, and then the extraction is performed on one horizontal line. The position coordinate data (edge coordinate data) for the pixel is converted into edge coordinate conversion unit 11
Works to transfer to.

The short distance area edge extraction processing circuit 26 determines the short distance area extraction condition from the short distance area extraction processing area transferred from the short distance area extraction condition determination unit 29 in the image signal received from the buffer 25. Pixels corresponding to the short-distance region extraction color condition determined by the unit 29 are extracted, image processing is performed to remove noise and the like, and then position coordinate data (edge coordinate data) for the extracted pixels is displayed on one horizontal line. ) Is transferred to the lane recognition unit 12.

When one horizontal line is examined, when a change point from a pixel which does not satisfy the extraction color condition to a pixel which satisfies the extraction color condition is assumed, the pixel which satisfies the extraction color condition is assumed. Is called a rising edge pixel, and its coordinate data is called edge coordinate data of the rising edge pixel. Similarly, when one horizontal line is examined, if a change point from a pixel that satisfies the extraction color condition to a pixel that does not satisfy the extraction color condition is assumed, the pixel that satisfies the extraction color condition falls. It is called an edge, and its coordinate data is
It is called edge coordinate data of the falling edge pixel. Here, it is assumed that the edge coordinate data includes, in addition to the coordinate information, information indicating whether the edge coordinate is a rising edge or a falling edge.

Next, the edge coordinate conversion unit 11 enlarges the edge coordinate data transferred from the long-distance region edge extraction processing circuit 10 based on the zoom center coordinates and the enlargement ratio transferred from the zoom condition determination unit 9. It functions to convert it to the previous coordinate data and transfer it to the lane recognition unit 12. The zoom center coordinates and the enlargement ratio will be described in detail later in the image zoom processing.

Next, the lane recognition unit 12 uses the edge coordinate data transferred from the edge coordinate conversion unit 11 and the short-distance area edge extraction processing circuit 26 to determine the left lane and the right lane for the traveling lane in which the vehicle is traveling. The edge coordinate data for the lane is extracted, the left and right lanes are recognized based on the extracted edge coordinate data, and the lane recognition result is used as the short distance area extraction condition determination unit 29, the long distance area extraction condition determination unit 30, and the zoom condition determination unit. 9, transferred to the risk determination unit 13 and the replacement circuit 15.

Next, the zoom condition determination unit 9 receives the recognized lane information from the lane recognition unit 12, calculates the zoom center coordinates and the enlargement ratio, and the edge coordinate conversion unit 11 and the image zoom. Transfer to circuit 8. The zoom center coordinates and the enlargement ratio, which are the zoom conditions, may be set in advance for each horizontal line, and the image enlargement processing may be performed.

Next, the structure of the display unit 14 will be described. The delay circuit 16 has a function of delaying and outputting the input image information by a time corresponding to various processing times in the processing unit 5, and outputs the signal output from the color difference conversion circuit 4 and the lane recognition unit. This is a circuit for synchronizing the processing result output from 12 when inputting it to the replacement circuit 15.

The replacement circuit 15 is a circuit for superimposing the processing result from the processing section 5 on the image delayed by the delay circuit 16. Specifically, the lane recognition unit 12 as shown in FIG. 3 (2) recognizes the captured image as shown in FIG. 3 (1) output from the delay circuit 16 without any image processing. The image information of the lane is superimposed to obtain the image information of the recognized lane drawn on the captured image as shown in FIG. 3 (3).

In addition, the danger level judgment unit 13 uses the alarm device 21.
At the same time, when driving the
The superimposing information to be displayed on the display 8 may be provided to the replacement circuit 15. A state in which this alarm information is displayed on the monitor 18 is shown in FIG.

The encoder circuit 17 is a circuit for converting the image information from the replacing circuit 15 into an NTSC signal. Then, the monitor 18 receives the NTSC signal and displays image information corresponding to the NTSC signal on the display screen.

Next, the components of the judgment unit 7 will be described. The vehicle state determination unit 19 determines the traveling state of the own vehicle based on the signal sent from the vehicle state sensor 20, and transfers the determination result to the risk degree determination unit 13. Here, the vehicle state sensor 20 is means for detecting the momentum of the own vehicle, the driver's intention to operate, and the like. For example, a vehicle speed sensor for measuring the speed of the own vehicle, other direction indicators, steer angle sensors, etc. Is mentioned.

The risk determining unit 13 recognizes the traveling risk of the own vehicle based on the left and right lane information for the traveling lane of the own vehicle sent from the recognizing unit 12 and the data transmitted from the vehicle state determining unit 19. For example, when the speed of the host vehicle detected by the vehicle speed sensor is equal to or higher than the initial fixed value and the presence of a forward vehicle or the like is recognized at a specific position in the front image information, this is determined to be a dangerous state, and Motion control device 22 and alarm device 21
Drive.

The vehicle motion control device 22 is a device for controlling a drive system, a control system, a steering system and the like, and a specific example thereof is an automatic brake or the like. Further, as the alarm device 21, any means may be used as long as it is a means for appealing to the driver by appealing to the driver's hearing or vision, and for example, a driving operation of a chime or an LED display may be considered. . It should be noted that the determination unit 7 is not limited to the above configuration, and various modes are conceivable.

The memory 23 is used not only for the above-described processing but also for storing the zoom condition determined by the zoom condition determining unit 9 and also as a work area when the lane recognizing unit 12 and the like perform the process. It is supposed to do.

Next, the operation of this embodiment will be described. First, in this embodiment, roughly, two types of processing, that is, a first processing and a second processing are executed in parallel in different cycles.

First, in the first process, the image information obtained by the CCD camera 2 is analog-digital converted by the A / D converter 3, and further converted by the color difference conversion circuit 4 into luminance and color difference signals and delayed. The circuit 16 gives a delay of a predetermined time, the image processing is performed by the function of the replacement circuit 15, the image processing result is converted into an NTSC signal by the encoder circuit 17, and the processing result is displayed on the monitor 18. Yes, first, regarding this first processing, FIG. 1 and FIG.
Will be explained.

First, in FIG. 2, when the power of the apparatus is turned on (S2), initialization is performed (S4). here,
Examples of the initial setting include processing such as clearing the work area area of the memory 23. Further, at this time, as operations performed by a human, initial adjustment of the image capturing unit 1 and the display unit 14, zoom conditions and sampling positions used in a second process described later, short range region edge extraction, and long range region edge extraction are performed. There is an operation of setting an initial value of a processing area or the like for performing the operation using an input unit (not shown).

Next, a process (S6) of outputting the image information signal in front of the vehicle taken by the CCD camera 2 as an analog RGB signal is performed, and then the A / D converter 3 converts the analog RGB signal into a digital RGB signal. Shi
(S8), the digital RGB signal is processed by the color difference conversion circuit 4 according to the conversion formula described above, and the luminance (Y) and the color difference signal are obtained.
Conversion to (RY, BY) (S10).

Next, the delay circuit 16 receives the image signal output from the color difference conversion circuit 4 as an input signal, delays the input signal by the processing time required by the processing section 5 (S12), and the processing section 5 receives it. Synchronize with the processed signal.

Next, the replacing circuit 15 performs a process of superimposing the processing result of the processing unit 5 on the unprocessed original image information (S14), and outputs the output signal of the replacing circuit 15 to the encoder circuit 17 Converted to NTSC signal by (S1
6), the display screen corresponding to the NTSC signal is displayed on the monitor 18 (S18).

Then, in order to obtain image information of the subject in the next cycle, the process branches to step (S6). In addition,
The above processing is sequentially performed with a frame rate of 16.7 ms as a cycle, for example.

Next, the second processing will be described with reference to FIG. In this second processing, the positions of the pixels to be sampled and the processing area for the edge extraction processing are determined (S1
9), the color data of the pixel at the determined sampling position is sampled (S21), the extraction color conditions for the short distance area and the long distance area are determined, and then the luminance from the color difference conversion circuit 4 is determined.
(Y) and the color difference signals (RY, BY), the edge pixels are extracted (S23, S25), the left and right lanes of the traveling lane in which the vehicle is traveling are recognized (S28), and the vehicle becomes dangerous. It is determined whether or not the vehicle is in the state (S30), and the driving (alarm) process of the alarm device 21 (S32) to the driving of the vehicle motion control device 22
(Avoidance) process (S34) consists of a series of processes.

First, the flow of the sampling position and edge extraction processing area determination processing (S19) is shown in FIG. In the processing (S19) of FIG. 4, the coordinate data of the recognized lane obtained by processing in the previous cycle is referred to, and the short-distance area extraction condition determining unit 29 sets the short-distance area edge extraction processing area setting (S19).
1902) and short range area sampling position setting (S19)
04), and the long-distance area extraction condition determining unit 3 is executed.
0 sets the long-distance area edge extraction processing area (S19
06) and the setting of the near and far area sampling position (S190
Perform 8).

Next, the short distance area edge extraction processing area setting (S1902) processing by the short distance area extraction condition determining unit 29 and the long distance area edge extraction processing area setting (S1906) by the long distance area extraction condition determining unit 30 are performed. The processing will be described with reference to FIG. FIG. 5 shows a scene in which the edge extraction processing area is set based on an example of the result of the lane recognition obtained by processing in the previous cycle. The left end, the right end, and the lower end position of the extraction processing area are fixed, and are preset as shown by the solid gray line.

Next, the upper end of the long-distance area edge extraction processing area and the boundary position of the short-distance area edge extraction processing area and the long-distance area edge extraction processing area are the lane recognition values obtained by processing in the previous cycle. Set based on the result.

In the setting method at this time, for example, at the upper end position of the long-distance area edge extraction processing area, the width of the vehicle traveling lane sandwiched between the recognized left and right lanes is preset as shown in the figure. It may be set to a position having a certain fixed value Lw. Regarding the boundary position between the short-distance area edge extraction processing area and the long-distance area edge extraction processing area, the upper end position of the long-distance area edge extraction processing area and the short-distance area edge extraction The lower end position of the processing area may be set to a position divided into, for example, 1 to 2, as illustrated.

Next, the short-distance region sampling position setting process (S1904) by the short-distance region extraction condition determining unit 29 and the long-distance region sampling position setting (S1908) process by the long-distance region extraction condition determining unit 30 will be described with reference to FIG.
Will be explained. FIG. 6 is an example in which five sampling positions are set for the screen in which the edge extraction processing area is set as described in FIG. 5, and among the sampling positions, the sampling position 3 at the bottom of the screen is set. 4 is fixed and set in advance. On the other hand, the sampling positions 0, 1, and 2 are set based on the result of the lane recognition obtained by processing in the previous cycle.

As a method of setting these positions 0, 1 and 2, for example, for the sampling pixel position 0, as shown in the figure, the upper end position of the long distance area edge extraction processing area and the short distance area edge extraction processing are shown. It may be set at a position where the region and the boundary position of the long-distance region edge extraction processing region are divided into 1: 2, and at a position which is the center of the own vehicle traveling lane sandwiched between the recognized left and right lanes. Further, as shown in the figure, the sampling positions 1 and 2 are, for example, on the boundary position between the short distance area edge extraction processing area and the long distance area edge extraction processing area, and sandwiched between the recognized left and right lanes. It is sufficient to set the vehicle traveling lane at a position that divides it into three equal parts.

Next, color data sampling processing (S21)
Will be described. This color data sampling process (S
In 21), the pixels at the five sampling positions set in the sampling position and edge extraction processing region setting processing (S19) are selected from the captured image signal to which the color data of all the pixels of the captured image are sequentially transferred, and Color data (Y, B-
Y, RY) are sampled. At this time, the pixels at the sampling positions 1, 2, 3, and 4 are sampled by the short-range area color sampling circuit 29, and the pixels at the sampling pixel position 0 are sampled. The long-distance area color sampling circuit 30 performs sampling.

Next, in the extraction color condition determination and edge extraction processing (S23, S25) for the short-distance area and the long-distance area respectively, edge pixels are extracted from the image information obtained by photographing, and the edge coordinate data is extracted as lane lines. The process of transferring to the recognition unit 12 is performed. FIG. 7 (1) shows a captured original image, and FIG. 7 (2) shows a state in which edge pixels are extracted from the original image.
Here, the short range area extraction color condition determination and edge extraction processing
(S23) and the long-distance area extraction color condition determination and edge extraction processing (S25) will be described in detail.

First, FIG. 8 is a flowchart showing the processing contents of the short-distance area extraction color condition determination and edge extraction processing (S23). In FIG. 8, in the short distance area extraction color condition determination processing (S2302), color data sampling processing is performed.
The short-distance region extraction condition determining unit 29 determines the short-distance region extraction color condition from the color data obtained by sampling in (S21). Note that this short-distance area extraction color condition determination processing
(S2302) will be described in detail later.

Next, a short-distance area edge extraction process (S23
In 04), the short distance area edge extraction processing circuit 26 extracts the edge pixels existing in the short distance area edge extraction processing area set by the short distance area extraction condition determining unit 29.

FIG. 9 shows this short distance area edge extraction processing.
The processing content of (S2304) is shown in a flowchart. Here, first, the short-distance area pixel determination processing (S23040
2) compares the short-distance area extraction color condition set by the short-distance area extraction condition determination unit 29 with the color data of all the pixels in the short-distance area edge extraction processing area to obtain the short-distance area extraction color. This is the process of extracting the pixels that satisfy the conditions.
For example, as shown in FIG. 10 (a1), the image signal in the short distance area edge extraction processing area of the original image shown in FIG.
Binarization processing is performed on pixels that satisfy the short-distance region extraction color condition and pixels that do not.

Next, the short distance area edge point discrimination processing (S2
30404) is a process for removing noise. For example, a smoothing process using spatial filtering of 3 pixels × 3 pixels is performed. Of these 9 pixels, more than half of the pixels satisfy the extraction color condition. If it ’s a gathering, these 9
It is assumed that the individual pixels are pixels that satisfy the extraction color condition. Then, after filtering the binarized image information as shown in FIG. 10 (a1), the edge point is determined.

That is, when the binarized image information shown in FIG. 10 (a1) is subjected to the filtering process and then examined for each horizontal line, from the pixels which do not satisfy the near-distance area extraction color condition data. When a pixel that satisfies the short-distance region extraction color condition data is changed, a pixel that satisfies the extraction color condition data is set as a rising edge pixel, and its coordinate data is set as edge coordinate data of the rising edge pixel.

Similarly, when each horizontal line is checked, if the pixel that satisfies the short-distance region extraction color condition data changes from the pixel that does not satisfy the short-distance region extraction color condition data, the extraction color condition A pixel that satisfies the data is defined as a falling edge pixel, and its coordinate data is defined as edge coordinate data of the falling edge pixel. This state is shown in FIG.
It is shown in (a2).

Then, when the edge coordinate data shown in FIG. 10 (a2) is obtained in this manner, the edge coordinate data shown in FIG. 10 (1) is obtained.
As shown in FIG. Note that FIG.
0 (a2) is the case of image information consisting of only the road surface and the lane, but it goes without saying that edge pixels that do not form the lane will be extracted when another vehicle or the like exists on the road surface.

Next, FIG. 11 is a flowchart showing the processing contents of the long-distance area extraction color condition determination and edge extraction processing (S25). In FIG. 11, first, in the long-distance area extraction color condition determination processing (S2502), the long-distance area extraction condition determination unit 30 extracts the long-distance area extraction from the color data obtained by sampling in the color data sampling processing (S21). Determine the color conditions. The long-distance area extraction color condition determination processing (S2502) will be described in detail later.

Next, in the image zoom and long distance area edge extraction processing (S2504), the long distance area extraction condition determining unit 30
The long-distance area edge extraction processing circuit 10 extracts the edge pixels existing in the long-distance area edge extraction processing area set in step 1.

FIG. 12 is a flowchart showing the processing contents of the image zoom and long distance area edge extraction processing (S2504). First, the image zoom processing (S250402) will be described in detail later. The image zoom circuit 8 enlarges the image information in the long-distance area edge extraction processing area set by the condition determining unit 30 only in the horizontal direction according to the zoom condition set by the zoom condition determining unit 9 to obtain a road surface or a white line. In the process of enlarging the image such as, for example, the process of enlarging the image signal in the long-distance region edge extraction processing region of the original image shown in FIG. 10 (0) is performed as shown in FIG. 10 (b1).

Next, a long-distance area edge extraction process (S250
In 404), the long-distance area edge extraction processing circuit 10 extracts edge pixels existing in the long-distance area edge extraction processing area set by the long-distance area extraction condition determining unit 30.
Here, in FIG. 13, a long-distance area edge extraction process (S25
The processing contents of 0404) are shown in a flowchart.

First, the long-distance area pixel discrimination processing (S2504
0402) compares the long-distance area extraction color condition set by the long-distance area extraction condition determination unit 30 with the color data of all pixels in the long-distance area edge extraction processing area of the enlarged image. In the process of extracting the pixels that satisfy the long-distance region extraction color condition, for example, the image signal in the long-distance region edge extraction processing region of the enlarged image shown in FIG. As shown in, the binarization processing is performed on pixels that satisfy the long-distance region extraction color condition and pixels that do not.

Next, the long-distance area edge point discrimination processing (S25
040404), in order to remove noise, for example, 3
Smoothing processing using spatial filtering of pixels × 3 pixels is performed, and if more than half of these 9 pixels are a group of pixels that satisfy the extraction color condition, the extraction color is extracted for these 9 pixels. The pixels satisfying the conditions.

Then, after filtering the binarized image information as shown in FIG. 10 (b2), the edge point is discriminated. That is, when the binarized image information of FIG. 10 (b2) is filtered and then examined for each horizontal line, from the pixels that do not satisfy the far-distance region extraction color condition data, the long-distance region extraction color When changing to a pixel that satisfies the condition data, a pixel that satisfies the extracted color condition data is set as a rising edge pixel, and its coordinate data is
It is the edge coordinate data of the rising edge pixel.

Similarly, when checking for each horizontal line, from the pixel satisfying the long-distance region extraction color condition data,
When a pixel that does not satisfy the long-distance region extraction color condition data is changed, a pixel that satisfies the extraction color condition data is set as a falling edge pixel, and its coordinate data is set as edge coordinate data of the falling edge pixel. This state is shown in FIG. 10 (b3).

Then, the edge coordinate data is transferred to the edge coordinate conversion section 11. In the example shown in FIG. 10 (b3), the image information includes the road surface and the lane. However, when another vehicle or the like exists on the road surface, edge pixels that do not form the lane are extracted.

Next, an edge coordinate conversion process (S25040
In 6), the edge coordinate data is referred to in FIG. 1 by referring to the enlargement ratio of the zoom center coordinate transferred from the zoom condition determining unit 9.
As indicated by 0 (1), the coordinate data is converted to the unenlarged coordinate data and transferred to the lane recognition unit 12. Here, a method of converting the edge coordinates will be described with reference to FIG.

FIGS. 14 (a) and 14 (b) respectively show image information before and after the enlargement process. When the zoom center coordinate of the y-th horizontal line from the upper end of the screen is C and the enlargement ratio is R, FIG. 1
The edge coordinate data P ′ of 4 (b) can be converted into the edge coordinate data P before expansion by the following equation. Here, W is the width in the horizontal direction of the set of pixels forming the image information. P = (P'-W / 2) / R + C Next, in the lane recognition process (S28), a plurality of approximate straight lines are obtained from the transferred edge coordinate data, and as shown in FIG. Estimate the lane for the traveling lane. Therefore, the lane recognition unit 12 uses the edge coordinate conversion unit 1
1 and the edge coordinate data sent from the short-distance area edge extraction processing circuit 26, any of the edge pixels considered to be for the left lane, the edge pixels considered for the right lane, and any other edge pixel. It is determined whether or not it is, and the process of classification is performed.

As described above, the edge coordinate data is composed of the coordinate information of each edge pixel and the rising or falling information. When the road surface color is used as the extraction color condition data, in many cases, the rising edge pixel indicates the left lane and the falling edge pixel indicates the right lane.

However, when there is a vehicle ahead, an obstacle, etc., edge pixels other than the lane may be recognized as edge pixels of the lane. Therefore, FIG.
It is assumed that the rising edge pixels existing on the left side of the screen center line in (3) are the edge pixels of the left lane, and the falling edge pixels existing on the right side are the right lane edge pixels, and the other edge pixels are the forward lane pixels. It may be determined as an edge pixel such as.

Next, based on the edge coordinate data of the edge pixels for the left and right lanes, an approximate straight line is obtained by, for example, the method of least squares, and this is set as the recognition lane. At this time,
As described above, the edge coordinate data other than the edge pixels for the left and right lanes is determined to be a pixel indicating the front vehicle, and the edge coordinate data that is closest to the position of the own vehicle is the presence position of the front vehicle. To do.

Returning to FIG. 2, next, in the risk determination processing (S30), the running state (vehicle speed, etc.) of the host vehicle is determined from the signal from the vehicle state sensor 20 such as the vehicle speed sensor, and the determination result is obtained. , It is estimated whether or not the host vehicle is in a dangerous state from the relationship between the travelable road surface defined by the lane and the existing position of the vehicle ahead. For example, when the speed of the host vehicle detected by the vehicle speed sensor is equal to or higher than a predetermined value and the position of the preceding vehicle is within a predetermined range, this is determined to be a dangerous state.

In the alarm process (S32), when the risk determining unit 13 determines that the state is dangerous, the alarm device 21 is driven to notify the driver of the own vehicle. Also, the avoidance process (S34)
Then, when the risk determination unit 13 determines that the vehicle is in a dangerous state, the vehicle motion control device 22 such as a brake device is driven.
It should be noted that the vehicle motion control device 22 may be driven when it is determined that the driver's operation is insufficient for the warning of the dangerous state.

Therefore, by repeating these series of processes, the operation as the monitoring device can be obtained, and as a result, according to this embodiment, the safe running of the vehicle can be obtained. Become.

Next, the image zoom processing (S250402), which was briefly described above, will be described in detail. Figure 15
Is a flowchart showing the processing contents of this image zoom processing (S250402), and will be described below with reference to this flowchart.

First, the zoom condition determining unit 9 processes the long-distance region edge extraction processing region set by the long-distance region extracting condition determining unit 30 in the previous cycle to obtain the coordinate data of the recognized lane. Etc., the zoom center coordinates and the enlargement ratio are determined for each horizontal line (S2504020).
2).

Here, the zoom center coordinate is a parameter indicating the position of a reference point for performing the enlargement process when enlarging the image information, and the enlargement ratio is a parameter indicating the magnification for enlarging the image information. The zoom center coordinate is, for example, the center position of the vehicle driving lane sandwiched between the recognized left and right lanes, and the enlargement lane does not extend beyond the screen (for example, 80% of the screen width). It should be decided).

The zoom condition determining unit 9 makes such a determination.
Is performed by referring to the coordinate data of the recognized lane sent from the lane recognition unit 12 and the like to grasp the center position and the traveling lane width of the own driving lane sandwiched between the recognized lanes. When it is not necessary to sequentially update the coordinates and the enlargement ratio, the coordinates and the enlargement ratio may be determined in advance for each horizontal line and stored in the zoom condition determining unit 9.

Next, description will be made assuming an actual scene (landscape image). FIG. 16 (a) shows an example of a lane recognition result obtained by processing the image of a straight road in the previous cycle.
When the screen width is W, the left lane coordinate is xl, and the right lane coordinate is xr on the y-th horizontal line from the upper end of the screen, the zoom center coordinate C and the enlargement ratio R are expressed by the following equations. Zoom center coordinates: C = (xr + xl) / 2 Enlargement ratio: R = α × W / (xr−xl) where (xr−xl) / W <α <1 where α is the lane after the enlargement process. It is a coefficient to prevent the image from protruding from the screen.

Therefore, the coefficient α is lower limit value ((xr-x
If it is smaller than l) / W), the enlargement ratio R becomes less than 1 and the image is reduced. In this case, α = (xr−xl) / W, that is, the enlargement ratio R = 1. FIG. 16 shows an example of the enlarged image information when α = 0.8.

Next, FIG. 17 shows a lane recognition result when a scene of a curved road is assumed, and the recognized left lane partially exists in the screen because the road is a left curve. An example is shown. In this case, in the y-th horizontal line from the upper end of the screen, the left lane coordinate does not exist in the screen, and the right lane coordinate is xr, the zoom center coordinate C and the enlargement ratio R are expressed by the following equations. Zoom center coordinates: C = xr / 2 Magnification ratio: R = α × W / xr This is because when xl = 0 in the formula for obtaining the zoom coordinate center C and the magnification ratio R in the case of the straight road in FIG. Equivalent to.

FIG. 18 also shows the case of a curved road scene. However, since the road is a right curve, the lane in which the right lane recognized this time is only partially present in the screen. It is an example of a recognition result. In this case, in the y-th horizontal pixel line from the top of the screen, the left lane coordinate is xl, but the right lane coordinate does not exist in the screen. Therefore, the zoom center coordinate C and the enlargement ratio R are expressed by the following equations. To be done. Zoom center coordinate: C = (W + xl) / 2 Magnification ratio: R = α × W / (W−xl) This is the formula in which xr is the zoom center coordinate C and the magnification ratio R in the case of the straight road in FIG. This corresponds to the case where = W.

Returning to FIG. 1, the image zoom circuit 8 enlarges the image in accordance with the determined zoom center coordinates and enlargement ratio. Here, referring to FIG. 19, the enlargement process for one horizontal line will be described. explain. As shown in FIG. 19, 8-pixel data (assuming that one horizontal line is composed of 8 pixels) is magnified twice by setting the zoom center coordinate C at the position indicated by the arrow and magnifying rate R = 2. The case of processing will be described.

First, the number of pixels to be enlarged is calculated. At this time, the number of pixels after the enlargement process must also be eight, so if the enlargement ratio is 2, the number of pixels to be enlarged is four. Next, the pixel data to be enlarged is selected. Here, four pixels are selected with the zoom center coordinate as the center, and the size of the image information with respect to the selected pixels is doubled and assigned to eight pixels forming one horizontal line. By such processing, the image information of one horizontal line shown in FIG. 19A is enlarged as shown in FIG. 18B.

By performing the above-described processing on all horizontal lines in the long-distance area edge extraction processing area set by the long-distance area extraction condition determining unit 30, the image in the long-distance area edge extraction processing area is obtained. The information can be expanded horizontally.

Next, the edge extraction processing by the image enlargement processing will be described in more detail with reference to FIGS. First, assuming that the original image is as shown in FIG. 20 (a), FIG. 20 (b) is a diagram obtained by enlarging the image of FIG. 20 (a). The enlargement ratio at this time is shown in FIG.
As shown in (c), the maximum value is 2
It turns out that It should be noted that FIG. 20 (c) is a diagram showing a state in which the enlargement ratio is set for each horizontal line. Further, FIG. 21 (A) is a portion A in FIG. 20 (a).
It is an enlarged view of (a part of the right lane) and is shown in Fig. 20 (B).
Is a diagram obtained by performing image enlargement processing (enlargement ratio: 2 times) on FIG.

As is clear from the long-distance area edge point discrimination processing (S25040404) in FIG. 13 described above, in this embodiment, spatial filtering is performed by 9 pixels of 3 pixels × 3 pixels for noise removal. That is, if more than half of these 9 pixels satisfy the extraction color condition, it is determined that these 9 pixels satisfy the extraction color condition.

Therefore, as shown by the thick lines in FIGS. 21A and 21B, if it is divided into a set of 3 pixels × 3 pixels, first, in the case of FIG. , There is only one set of 3 pixels × 3 pixels having a white line color, and as a result, in the edge point determination processing, it is considered that the pixels are configured as shown in FIG. . On the other hand, in the case of FIG. 21B, the set of 3 pixels × 3 pixels having a white line color is 4, and in this case, the pixels are configured as shown in FIG. 21B ′ in the edge point determination processing. Is considered to have been done.

As is apparent from the above, in the enlarged image, the ratio of the white line in the area A portion (see FIG. 20) becomes 2/3, which is 1/100 of that in the case where the enlargement processing is not performed.
The ratio of the white line becomes larger than that of No. 3, and as a result, the white line that cannot be recognized without the enlargement processing can be surely recognized by increasing the ratio of the white line in the image information by performing the enlargement processing. Can be obtained.

Therefore, according to this embodiment, by performing the enlargement process, the lane with respect to the traveling lane in which the host vehicle travels can be surely recognized to a farther distance, and safer traveling becomes possible.

Next, another embodiment of the present invention will be described. In the above embodiment, as an example of the image enlargement processing, the method of calculating and obtaining the enlargement ratio for each horizontal line has been described. However, in the image enlargement processing, as shown in FIG. 22, all the horizontal lines to which the enlargement processing is performed may be enlarged at the same enlargement ratio.

As the enlargement ratio in this case, for example, the enlargement ratio calculated at the lower end of the long-distance region edge extraction processing region can be used. Therefore, according to this method, the enlargement ratio is calculated for each horizontal line. Compared with the method, the amount of calculation can be significantly reduced.

Next, the short-distance area extraction color condition determination processing (S2302), which is the main processing in the present invention described above, is executed.
<Long-distance area extraction color condition determination processing (S2502) will be described in more detail. This short-distance area extraction color condition determination processing is performed by the short-distance area extraction condition determination unit 29 shown in FIG.
23. As shown in the flowchart of FIG. 23, the short-distance area color difference (RY, BY) condition determination processing (S230202) and the short-distance area luminance (Y) condition determination processing (S230204). It is composed of.

First, the near-distance area color difference condition determination processing (S2
30202), sampling points 3 and 4 shown in FIG.
It is verified whether the color difference data (RY, BY) of No. 3 is the road surface color difference data, and that the color difference data at sampling points 3 and 4 may not be the road surface color difference data. If it is determined, the color difference data at the sampling points 3 and 4 are changed.

Therefore, for example, first, sampling point 3
If the absolute value of the difference between the color difference data (RY, BY) of 4 and 4 is obtained and the absolute value is larger than the preset value,
As shown in FIG. 24, the sampling color difference data at the sampling points 3 and 4 accurately indicate the color difference data on the road surface for some reason such as the presence of a yellow line at either of the sampling points 3 and 4. It is determined that there is a possibility that there is no such difference, and the sampling color difference data at the sampling points 3 and 4 is changed to the color difference data at the sampling points 3 and 4 obtained by the sampling in the processing of the previous cycle.

On the other hand, color difference data of sampling points 3 and 4
If the absolute value of the difference between (RY, BY) is smaller than a preset value, it is determined that the sampling color difference data of sampling points 3 and 4 accurately represent the color difference data of the road surface, and the sampling point The sampling color difference data of 3 and 4 are not changed.

Next, a preset correction amount is added to the color difference data at the sampling points 3 and 4, and a range including this value is set as a color difference condition. However, if the range set by adding a preset correction amount to the color difference data of the sampling points 3 and 4 does not include the achromatic region, the color difference condition is set so as to include the preset achromatic region. Set again.

The color difference condition is set so as to include an achromatic region for the following reason. That is, since the road surface is generally paved with asphalt or concrete, its color is black or gray and has no color. Therefore, if the area with no color, that is, the "achromatic" area is set to be included in the color difference condition, there is a high possibility that the road surface can be accurately extracted even if incorrect color difference data is sampled. Because.

Next, a method for setting the color difference will be described with reference to FIG. 25 (1) as an example. In this FIG. 25 (1),
Sampling color difference data SAMP3 _{BY of} sampling point 3
SAMP3 _RY and sampling color difference data SAMP4 _BY and SAMP4 _RY at the sampling point 4 are indicated by black circles.
Then, first, a preset correction amount is added to these sampling data so as to have a certain width. This can reduce the influence of variations in sampling data.

Here, at these sampling points 3 and 4,
Correction amount (α, β) for color difference BY and color difference R
The range in which the correction amount (γ, δ) is added to −Y is as shown by the broken line, but these are the ranges shown by the following equation. Sampling point 3: SAMP3 _BY- β <SAMP3 _BY <SAMP3 _BY + α SAMP3 _{RY −} γ <SAMP3 _RY <SAMP3 _RY + δ Sampling point 4: SAMP4 _{BY −} β <SAMP4 _BY <SAMP4 _BY + α SAMP4 _{RY −} γ <SAMP4 _RY <SAMP4 _RY + δ Here, the correction amounts α, β, γ, δ are set in advance and stored in the memory.

Then, the area A including the range surrounded by the broken line is set as the color difference condition. However, if the area A does not include a preset "achromatic color" area as indicated by hatching, the area B indicated by a thick line is included so as to include the "achromatic color" area. The color difference condition is set again.

Next, in the short-distance area luminance (Y) condition determining process, the maximum value and the minimum value of the sampling luminance data from the sampling points 1 to 4 are obtained, and a predetermined luminance correction amount is added to these values. Is the process of setting the brightness condition by
This process will be described by way of example with reference to FIG. Now, assuming that the luminance data from sampling points 1 to 4 is as shown in FIG. 25 (2),
At this time, since the sampling point 1 has the maximum luminance, the luminance of the sampling point 1 has a predetermined correction amount d.
The value to which 1 is added is set as the upper limit value of the brightness condition.

On the other hand, since the minimum value is the sampling point 2, a value obtained by adding a predetermined correction amount d2 to the brightness data at this sampling point 2 is set as the lower limit value of the brightness condition. Therefore, in this case, the range set as the brightness condition is the range indicated by the double arrow on the upper side of the drawing.

Next, the long-distance area extraction color condition determination process (S
2502) will be described. This long-distance area extraction color condition determination processing is executed by the long-distance area extraction condition determination unit 30, and as shown in the flowchart of FIG. 26, long-distance area color difference (RY, BY) condition determination processing (S2502).
02) and the long-distance area luminance (Y) condition determination processing (S2502
04) and.

First, the distance area color difference condition determination processing (S25
0202), the short-distance area color difference condition determination processing (S230
The color difference condition determined in 202) is set as the long-distance area color difference condition. Next, long-distance area brightness condition determination processing
In (S250204), the following processing is executed. First,
Day and night discrimination processing is performed on the luminance data sampled at five points. This day / night determination process is performed as follows.

As shown in FIG. 27, the average value of the data of the sampled luminance data on the same horizontal line is obtained, and the average value is calculated from the bottom of the screen by M ₁ (average luminance of sampling points 3 and 4). , M ₂ (average luminance of sampling points 1 and 2).

Then, if the relationship between these average values and the luminance data Y _{0 at the} sampling point 0 satisfy the following expression, it is determined that it is night. Night judgment conditional expression: M ₁ −M ₂ 2> ε, Y ₀ <ζ where ε and ζ are determination reference values, which are data set in advance and stored in the memory.

As a result, when it is determined that it is night,
The nighttime brightness condition determination process is executed, and when it is determined that it is not nighttime, the daytime brightness condition determination process is executed. First, in the nighttime brightness condition determination process, similar to the short-distance region brightness condition determination process, the maximum value and the minimum value of the sampling brightness data from the sampling points 0 to 2 are obtained, and the brightness determined in advance to the maximum value and the minimum value. A correction amount is added to set the long-distance area luminance condition.

Next, the daytime brightness condition determining process is performed as follows. First, as shown in FIG. 27, in the luminance data obtained by the day / night discrimination processing and sampled, the difference DEF between the average values M ₁ and M ₂ of the data on the same horizontal line.
The difference DEF ₂ between ₁ and the average value M ₂ and the luminance data Y ₀ at the sampling point 0 is obtained. Therefore, these differences DEF ₁ , D
EF ₂ is represented by the following equation. DEF ₁ = M ₂ −M ₁ DEF ₂ = Y ₀ −M ₂ Then, it is judged whether or not the value of DEF ₂ obtained is within the normal range with respect to the obtained DEF ₁ , and outside the normal range. If it is determined that there is, the value of DEF ₂ is changed.

This judgment is based on a certain value of DEF ₁ .
The range normally taken by DEF ₂ is set in advance as shown in FIG. For example, in FIG. 28, when DEF ₁ is D and DEF ₂ is E1, DE
It is determined that F ₂ is within the normal range. Further, when DEF ₁ is D and DEF ₂ is E2, DEF ₂ is larger than the normal value. At this time, therefore, the value of DEF ₂ is changed to E2 ′. Further, when DEF ₁ is D and DEF ₂ is E3, DEF2 is smaller than the normal value, so the value of DEF ₂ is changed to E3 '.

Next, the daytime brightness condition is set using the value of DEF ₂ thus obtained. The lower limit value of the daytime brightness condition at this time is the short range area brightness condition determining process. The same as the lower limit of the area brightness condition.
On the other hand, the upper limit value of the daytime brightness condition is set to the upper limit value of the near field brightness condition obtained by the near field brightness condition determination processing,
It is set by adding F ₂ .

As described above, according to the embodiment of the present invention, different extraction conditions can be set for the short-distance area and the long-distance area. The left and right lanes can be recognized accurately.

[0120]

According to the present invention, in an apparatus and method for recognizing a lane based on image information taken by an image taking apparatus, the lane is extracted for each of the short distance area and the long distance area. , Lanes can always be detected reliably regardless of changes in weather and lighting conditions, and lanes can be recognized simultaneously in a short-distance area and a long-distance area. As a result, it can greatly contribute to ensuring safe driving.

[Brief description of drawings]

FIG. 1 is a block diagram showing a system configuration of an embodiment of the present invention.

FIG. 2 is a flowchart for explaining an overall data processing operation according to an embodiment of the present invention.

FIG. 3 is an explanatory diagram showing an example of an image display surface according to an embodiment of the present invention.

FIG. 4 is a flowchart for explaining a sampling position and edge extraction processing area determination processing according to an embodiment of the present invention.

FIG. 5 is an explanatory diagram of a long distance and short distance area edge extraction processing window according to an embodiment of the present invention.

FIG. 6 is an explanatory diagram showing sampling positions in a long distance area and a short distance area according to an embodiment of the present invention.

FIG. 7 is an explanatory diagram of lane recognition processing according to an embodiment of the present invention.

FIG. 8 is a flowchart illustrating a short-distance area extraction color condition and edge extraction processing according to an embodiment of the present invention.

FIG. 9 is a flowchart illustrating a short-distance area edge extraction process according to an embodiment of the present invention.

FIG. 10 is an explanatory diagram showing lane recognition processing according to an embodiment of the present invention.

FIG. 11 is a flow chart for explaining long-distance area extraction color condition determination and edge extraction processing according to an embodiment of the present invention.

FIG. 12 is a flowchart for explaining image zoom and long-distance area edge extraction processing according to an embodiment of the present invention.

FIG. 13 is a flowchart illustrating a long-distance area edge extraction process according to an embodiment of the present invention.

FIG. 14 is an explanatory diagram showing the principle of edge coordinate conversion processing according to an embodiment of the present invention.

FIG. 15 is a flowchart illustrating an image zoom process according to an embodiment of the present invention.

FIG. 16 is an explanatory diagram for obtaining zoom center coordinates in a scene according to an embodiment of the present invention.

FIG. 17 is an explanatory diagram for obtaining zoom center coordinates in another scene according to an embodiment of the present invention.

FIG. 18 is an explanatory diagram for obtaining zoom center coordinates in still another scene according to an embodiment of the present invention.

FIG. 19 is an explanatory diagram showing the principle of enlargement processing according to an embodiment of the present invention.

FIG. 20 is an explanatory diagram of screens before and after an enlargement process according to an embodiment of the present invention.

FIG. 21 is an explanatory diagram showing the principle of edge determination processing according to an embodiment of the present invention.

FIG. 22 is an explanatory diagram showing screen enlargement ratios before and after enlargement processing when the enlargement ratio changes discontinuously in the embodiment of the present invention.

FIG. 23 is a flowchart illustrating a short-distance area extraction color condition determination process according to an embodiment of the present invention.

FIG. 24 is an explanatory diagram showing a state of a sampling screen in a certain scene according to an embodiment of the present invention.

FIG. 25 is an explanatory diagram of a process of obtaining a short-distance area extraction color condition in the embodiment of the present invention.

FIG. 26 is a flowchart illustrating a long-distance area extraction color condition determination process according to an embodiment of the present invention.

FIG. 27 is an explanatory diagram of a process of obtaining a long-distance area extraction color condition according to the embodiment of the present invention.

FIG. 28 is an explanatory diagram showing a long-distance area luminance condition settable area according to an embodiment of the present invention.

[Explanation of symbols]

1 shooting department 5 processing section 6 Image processing unit 7 Judgment section 10 Long-distance area extraction processing circuit 12 lane recognition 14 Display 26 Short-distance Area Extraction Processing Circuit 29 Short-distance region extraction condition determination unit 30 Long-distance area extraction condition determination unit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI G08G 1/16 G08G 1/16 C (72) Inventor Tatsuhiko Moji 2520 Takaba, Hitachinaka City, Ibaraki Hitachi, Ltd. Automotive Equipment Business Inside (72) Inventor Kazuhiko Hanawa 2520 Takaba, Hitachinaka City, Ibaraki Prefecture Hitachi, Ltd. Automotive Equipment Division (72) Inventor Shin Shioya 1099, Ozenji, Aso-ku, Kawasaki-shi, Kanagawa Hitachi Systems Development Laboratory Co., Ltd. (72) Inventor Ryushi Nishimura, 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa, Hitachi, Ltd., Multimedia System Development Division (56) References JP-A-5-20593 (JP, A) JP-A-7-28975 (JP, A) JP-A 2-90381 (JP, A) JP-A 6-14230 (JP, A) JP 2-284268 (JP, A) JP-A-6-229760 (JP, A) JP-A-7-302346 (JP, A) JP-A-2-206883 (JP, A) JP-A-6-215290 (JP, A) A) JP 3-52081 (JP, A) JP 7-319541 (JP, A) JP 62-221800 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) ) G60T 7/00-7/60 B60R 21/00 G06T 1/00 G08G 1/00-1/16

Claims (7)

(57) [Claims] 1. A photographing means for photographing the front or rear of a vehicle, a first area for an image photographed by the photographing means, and a second area that is farther from the first area to the vehicle. Dividing means for dividing into a region and a binarization process of pixels from the image of the first region that satisfy extraction color conditions and pixels that do not meet the conditions, and spatially filtering the binarized image Extracting means for extracting the boundary between the lane and the road surface, and binarizing the pixel of the image of the second region that satisfies the extraction color condition and the pixel that does not satisfy the condition, and performing the binarization process. Second extraction means for extracting the boundary between the lane and the road surface by spatially filtering the obtained image, and the vehicle front direction or the rear direction based on the result of the first extraction means and the result of the second extraction means. Lane recognition lane recognition Means, first color sampling means for sampling color data from the first region of the image photographed by the photographing means, and sampling color data from the second region of the image photographed by the photographing means Second color sampling means, the first extracting means based on the color data sampled by the first color sampling means and the color data sampled by the second color sampling means.
The image of the first area is divided into
Used when binarizing pixels that do not meet the conditions
That the first threshold value determining means for determining a first threshold value, based on the first color data sampled by sampled color data and said second color sampling means in the color sampling means, said second In the extraction means
The image of the second area is divided into
Used when binarizing pixels that do not meet the conditions
That the second and the second threshold determination means for determining a threshold value, the monitoring device of the road surface by image processing, characterized in that it comprises a. 2. The traveling road surface monitoring device by image processing according to claim 1, further comprising a risk determining unit that determines a risk of a traveling state of the vehicle based on a result of the lane recognizing unit. 3. The invention according to claim 1, wherein the first area determination means determines the first area based on the recognized lane obtained by processing in the previous cycle, and the processing is performed in the previous cycle. And a second area determining means for determining the second area based on the recognized lane obtained as described above. 4. The invention according to claim 2, wherein the first area determination means determines the first area based on the recognized lane obtained by processing in the previous cycle, and the processing is performed in the previous cycle. And a second area determining means for determining the second area on the basis of the recognized lane obtained as described above. 5. The first sampling position determining means for determining the position for sampling color data from the first region based on the recognized lane obtained by processing in the previous cycle. And second sampling position determining means for determining a position for sampling color data from the second region based on the recognized lane obtained by processing in the previous cycle. Monitoring device for traveling road surface by image processing. 6. The first sampling position determining means for determining the position for sampling color data from the first region based on the recognized lane obtained by processing in the previous cycle. And second sampling position determining means for determining a position for sampling color data from the second region based on the recognized lane obtained by processing in the previous cycle. Monitoring device for traveling road surface by image processing. 7. The first sampling position determining means for determining the position for sampling color data from the first region based on the recognized lane obtained by processing in the previous cycle. If, based on the recognition lane obtained by processing in the previous cycle, characterized by comprising a second sampling position determining means for determining a location for sampling the color data from said second region Monitoring device for traveling road surface by image processing.