JP3331095B2 - In-vehicle image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an in-vehicle image processing apparatus, and more particularly to an in-vehicle image processing apparatus for performing white line recognition.

[0002]

2. Description of the Related Art Conventionally, a white line recognition device for a vehicle, for example, which is known as an on-vehicle image processing device, includes an A / D converter based on image information input from an image photographing means 1 which is a CCD camera, as shown in FIG. It is known that an image signal is converted into a digital signal by a conversion unit 2 and a white line recognition unit 3 performs white line recognition from the digitally converted signal.

In the prior art, various white line recognition means for finding a white line position by recognizing a white line from a digital signal have been proposed. For example, a predetermined image is determined using a luminance difference between the white line and a road surface. A method of obtaining a white line candidate by binarizing with a threshold value of
As disclosed in Japanese Patent Publication No. 498, a method is known in which a white line candidate point is obtained by executing a process such as spatially differentiating a rate of change of a luminance difference between a white line and a road surface, that is, a luminance signal.

[0004] Further, a method has been considered in which a signal obtained by subjecting an image signal to existing preprocessing such as binarization as in the previous example or the image signal itself is time-integrated to obtain a candidate point. In the following, a description will be given of a white line position detecting means for obtaining a candidate point from such a time integration.

FIG. 10 shows an example of an image when the front of the vehicle is photographed by the image photographing means 1, wherein 4 is a white line on the road, 5 is the road, 6 is characters on the road, and the like. Here, the detection of the white line 4 is performed for each scanning line, and the detection is performed by outputting the white line position for each of the right and left points.

Hereinafter, a configuration and operation for performing white line recognition from an input screen on one scanning line will be described. FIG. 11 shows a flow (a) when a certain scanning line (a line in FIG. 10) is extracted and processed, and the data (b) at that time, with the horizontal axis representing the horizontal position on the screen. It is.

[0008] Reference numeral 8 in FIG. 11B shows input image data when the luminance value is plotted on the vertical axis. As described above, the white line 4, the character 6 on the road, and the like have a large luminance value, and the road 5 and the like have a small luminance value. First, the input screen (input image data) is detected by the contour detection unit 9. The contour line is obtained by the contour line detecting means 9 by the absolute value of the change in the input image data. The output data (contour detection data) of the contour detection means 9 to which the input image data 8 has been input is shown by a graph such as 10.

Next, binarization is performed by the binarizing means 11 using a specific threshold value. When the contour detection data 10 is binarized by a specific threshold, binarized data as indicated by 12 is obtained. Next, the binary data 12 is time-integrated by the time integration means 13.

The time integration focuses on one pixel on the screen, for example, and is constituted by one screen delay means 23 and internal division means 24 as shown in FIG. When a time constant τ25 representing a certain internal division ratio is given, the internal dividing means 24 receives two input τ−1: 1.
It outputs the internal division value. The luminance value of the pixel at a certain time is P (n), and the time integration value of the same pixel at the same time is I
(N), the time integration value of the same pixel in the previous frame is represented by I
(N-1), assuming that a predetermined time constant is τ, the time integration value is given by the following equation.

I (n) = (1-1 / τ) × I (n−1) + 1 / τ × P (n) (Equation 1)

Here, if the time constant τ is made large, the integration time becomes long, and if it is made small, the integration time becomes short. According to such integration processing, a contour line which appears temporally stably at the same position strongly appears, so that a temporally unstable contour line such as a character on a road has a small value.

A graph obtained by integrating the binary data 12 with respect to time is as shown in FIG. As described above, the data corresponding to the position of the white line 4 has a large value, while the data corresponding to the position of the character 6 on the road has a small value.

Next, the data 1 obtained by the integration means 13
For 4, change data 16 is obtained by the change detection means 15. The variation detecting means 15 obtains the variation as an absolute value of the variation, similarly to the contour detection means 9.

Next, the scanning start point setting means 17 sets the center point on the line as the specific reference point 18 and sets this as the scanning start point. The scanning unit 19 scans the variation data 16 from the specific reference point 18 in the left-right direction. Further, in parallel with the scanning unit 19, the change detection unit 20 detects a change in the change data 16 to detect a white line candidate point 21. The change detecting means 20 calculates a change in the data, for example, and detects a data whose change data is equal to or larger than a predetermined threshold.

The white line candidate 21 detected by the change detecting means 20 is determined by the judging means 22 as a white line position if the white line candidate is stable in time series, otherwise the previous white line position is determined. Determined as the white line.

The white line position detecting means according to one prior art of the present invention has been described above. The white line position detecting means according to another prior art of the present invention is disclosed in Japanese Patent Publication No. 6-24035. There is known a straight line group extracting unit that extracts a contour line from a captured image, performs a Hough transform, and extracts a straight line group approximate on the image.

[0017]

However, in the case of the former prior art, when the above-described processing is performed on an actual road, the white line position is recognized inside the actual white line position, and the accurate white line position is detected. There is a problem that cannot be recognized. This is because the position of the white line on the screen moves slightly to the left and right due to subtle shaking of the vehicle on the front screen actually taken by the camera mounted on the vehicle. Is wider than the white line, so the detection scan is inside (specific reference point)
This is because the white line is recognized inside from the actual. In FIG. 13, (a) is an image taken by a CCD camera, (b) is contour data,
(C) shows time-integrated data.

On the other hand, the latter conventional technique has a problem in that the processing time becomes longer when the number of contour extraction points is large.

The present invention has been made to solve such a problem, and an object of the present invention is to provide an in-vehicle image processing apparatus capable of performing accurate image recognition processing such as accurate recognition of a white line position. It is. Another object of the present invention is to provide an in-vehicle image processing apparatus capable of reducing processing time and performing accurate image recognition processing.

[0020]

According to a first aspect of the present invention, there is provided an in-vehicle image processing apparatus including a photographing means for photographing a periphery of a vehicle, and performing an image recognition process based on an obtained image signal. The processing device includes a time integrating means for time-integrating data based on the image signal obtained by the photographing means, and a dividing means for dividing the data into a plurality of regions based on the data obtained by the time integrating means. The image recognition processing is performed based on the divided area obtained by the dividing means.

According to a second aspect of the present invention, in the in-vehicle image processing apparatus of the first aspect, the image recognition processing is performed by limiting at least one of the plurality of regions. Is provided.

According to a third aspect of the present invention, there is provided the on-vehicle image processing apparatus according to the first or second aspect, wherein the time constant of the time integration by the time integration means is set at the position of the image. It is designed to be changed accordingly.

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. FIG. 1 is a diagram showing the overall configuration of one embodiment of the present invention. In FIG. 1, 1 is an image photographing means (CCD camera), 2 is A / D conversion means, 28 is preprocessing means such as contour detection, 29 is time integrating means, 30 is area dividing means, 3 is white line recognition means. , 45 are stabilization processing means, and 46 is a lane width learning means.

FIG. 2 shows an example of an image when the front of the vehicle is photographed by the CCD camera 1, which is obtained by performing preprocessing such as contour detection and binarization processing on FIG. is there. FIG. 3 is a diagram when time integration is performed on the image of FIG. 2 to divide the region. In FIG. 3, 26 is a region O,
Reference numeral 27 indicates the area I. Here, FIG. 2 described above is an output diagram of the preprocessing unit 28, and FIG.
0 is shown for each output.

FIG. 4 shows an example of the preprocessing means. 31 is one scanning line delay means, 32 is one pixel delay means,
33 and 34 are difference means, 35 and 36 are absolute value means, 37
Is a specific threshold, 38 and 39 are comparison means, and 40 is an OR operator. Here, the difference in the horizontal direction with respect to the camera is obtained by the difference means 33, the absolute value thereof is obtained by the absolute value means 35, and the absolute value thereof is binarized by the comparison means 38 to obtain a contour in the horizontal direction with respect to the camera. On the other hand, the difference in the vertical direction of the camera is similarly obtained by the difference means 34, the absolute value thereof is obtained by the absolute value means 36, and binarization is performed by the comparison means 39, thereby obtaining the vertical contour of the camera. Further, a logical sum of the horizontal contour and the vertical contour is obtained by an OR operator 40 to detect the contour.

The image subjected to contour detection and binarization is subjected to time integration using the integrator shown in FIG. In this integration, a long time integration can be obtained by increasing the time constant τ that determines the internal division ratio shown in the above (Equation 1), while a short time can be obtained by reducing the time constant τ. The time integral of can be obtained.
Therefore, for example, a portion corresponding to a short distance in front of the road is relatively stable, so a large time constant τ is taken, and a portion corresponding to a long distance in front of the road is relatively unstable, so that the time constant τ is set small. By properly using the time constant depending on the location, the processing time can be efficiently reduced.

Next, the output signal from the time integrating means 29 is input to the area dividing means 30. This is performed by using the comparing means 42 as shown in FIG. 5. When the signal is larger than the specific threshold 41 by the comparing means 42, the screen is displayed in the area I (27), and when the signal is smaller than the specific threshold 41, the screen is displayed in the area O (26). It is divided into two regions.

Here, as the specific threshold value 41, for example, a value obtained by adding a specific value based on the average of the entire screen or one scanning line of the time-integrated signal is used. For example, when the value of the time constant τ is changed between the near (far distance) portion and the far (far distance) portion of the road, the threshold value is determined based on the average of each scanning line.

The region I in the region divided by the previous stage
Is a region where a contour line with a small movement on the screen exists, and a region O is a region with a large movement line on the screen or a region without the contour line itself.

For example, an area I such as a white line or a vehicle which exists at the same position on the screen to some extent in time is an area I, and other areas such as characters drawn on a road, pedestrian crossings, etc. When the vehicle is moving, such as near the thing that does not exist in the same place on the screen in time,
An area O is obtained.

As described above, by limiting the detection processing such as white line recognition or vehicle recognition to the region I, the processing speed can be increased as a whole, and the stability of the apparatus can be further improved. Next, white line recognition is performed by the white line recognition means 3 using this area. In the white line recognition, a white line outline is detected from a raw image signal in the obtained region I, and white line recognition is performed based on the white line outline.

FIG. 6 is a flowchart for white line recognition.
The white line recognition processing is performed for each scanning line, and is further performed twice separately from the center of the screen (specific reference point) twice on each side. First, for example, the flowchart of FIG. 6 is executed on the right side of the reference point. First, it is checked whether or not the region I exists (step S1). If the region I does not exist, the previous white line recognition value is set as one of the candidate points (step S2). If the area I exists, scanning is performed in order from the area I closer to the center (step S3), and the existence of a contour line is determined (step S4). If it is determined that an outline exists, up to two from the inside are set as candidate points of the white line recognition value (step S5), and the process proceeds to step S7. Step S7 will be described later. After the processing in step S2 or when there is no white line in the region I in step S5, the position of the previous white line (the maximum value of the integral value) is set as a candidate for the white line recognition position (step S6), and step S6 is performed.
Go to 7. After the processing in step S7, the same processing is performed on the left side of the reference point.

Next, the processing in step S7 will be described. In step S7, for example, if there is a contour line candidate point in step S5 for the processing result on the right side, regions (1) and (3) are designated (energized) as shown in the figure. On the other hand, steps S2 and S6
When the white line position was previously determined as a candidate, the area (2)
And (4) are specified. Next, as for the processing result on the left side, if there is a contour line candidate point in step S5, regions (1) and (2) are designated, while step S5 is performed.
2. If the white line position was previously selected as a candidate in step S6, areas (3) and (4) are designated.

Thus, in step S7,
When the specification on the right side is completed, the area specified on both the left and right sides is selected as the left and right white line candidate points, and the white line is determined as follows according to each of the areas (1) to (4).

The region (1) is a case where a region I exists on both the left and right sides and a contour line exists in the region I. In this case, among the candidate points, the candidate point whose distance between the left and right candidate points is closest to the lane width is determined as the white line recognition point.

In the area (2), the area I exists on the left side, and further, the outline exists in the area I, and the area I does not exist or the outline exists in the area I on the right side. If not. In this case, the left candidate point is selected so that the distance from the previous right candidate point is closest to the lane width, and the right candidate point is determined so that the distance from the selected left recognition point is the lane width.

Right recognition point = left recognition point + lane width (when the right coordinate is larger than the left coordinate)

In the area (3), the area I exists on the right side, and the outline exists in the area I. The area I does not exist on the left side, or the outline exists in the area I even if it exists. It does not exist. In this case, the right candidate point is selected so that the distance from the previous left candidate point is closest to the lane width, and the left candidate point is determined so that the distance from the selected right recognition point is the lane width.

Left recognition point = right recognition point−lane width (when the right coordinate is larger than the left coordinate)

The region (4) corresponds to a case where the region I does not exist on both the left and right sides, or a case where there is no contour line in the region I. In this case, the left and right candidate points which are closest to the lane width are determined among the left and right candidate points. Here, the learned lane width is used as the lane width. (Lane width learning will be described later).

Next, stabilization processing is performed on the recognition points by the stabilization processing means 45. The stabilization process is performed on the recognition point by using the integrating means and the change amount limiting means. Here, the integration means is obtained by the same method as the integration means (FIG. 12) of the conventional device. The change amount limiting means sets a previous value of a certain value a (n) to a
(N-1) and the maximum change amount is δ (> 0),

When a (n) −a (n−1)> δ a (n) = a (n−1) + δ

When -δ <a (n) -a (n-1) <δ a (n) is left as it is.

When -δ <a (n) -a (n-1) a (n) = a (n-1) -δ

The above processing is performed.

FIG. 7 shows a graph of the integration process and the change amount limiting process. In the figure, (a) shows an example when the integration process is performed on the input data, (b) shows an example when the change amount limiting process is performed on the input data, and (c) shows an example when the change amount limiting process is performed on the input data. The time integration process has a function of stabilizing minute noise, but has little effect on noise having a large change amount.
On the other hand, the change amount limitation processing has a function of stabilizing noise of a large change amount, but has little effect on small noise.

The lane width learning means 46 will be described below. The lane width learning is performed after the stabilization process, and is performed based on the left and right white line recognition point distances only when all of the following four conditions (a) to (d) are satisfied. Here, learning refers to a process of recognizing and storing a lane width.

(A) Case of region (1). (B) Left and right recognition point distance is standard lane width (on screen) ± 10%
Within. (C) The distance between the left and right recognition points is within ± 10% of the previous lane width. (D) The distance between the left and right recognition points is equal to or less than the lane width (the actual positional relationship is nearer) detected on the lower scanning line in the screen.

As a result, only the vehicle that meets the above conditions is reflected in the lane width, so that the correct lane width can always be obtained. The learning operation is performed by time integration of the left and right recognition points. Thus, the lane width can be stabilized over time. Further, as the stabilizing operation, the above-described change amount limitation can be performed.

Embodiment 2 In the first embodiment of the present invention, an example has been described in which the outline of a white line is detected and white line recognition is performed in a region limited by the limiting unit after division by the dividing unit. An example in which a white line is extracted by a Hough transform in a limited area will be described. FIG. 8 shows an example in which a white line is extracted as a straight line using Hough transform instead of white line recognition in the above-described embodiment. First, in step S11, the region is divided as described above, and then, in step S12, Hough transform is performed.

Here, the Hough transform will be briefly described. The Hough transform is one of the techniques for extracting a straight line from a binary image, for example.

Y = ax + b

In other words, the coefficients (a, b) corresponding to the point (x, y) on the screen are:

B = y−xa (Equation 1)

The image of the point (x, y) on the ab plane is a straight line of (1). Where all points (x, y)
, A straight line image on the ab plane is obtained, and a point (a, b) where the straight line group is most overlapped, that is, a point (a, b) having the highest weight on the ab plane is extracted, thereby extracting a straight line. First, Hough transform is performed only on the divided region I. Since the area is limited only to the area I, the data amount can be reduced, the processing speed can be increased, and the influence of noise other than the white line can be reduced. Next, a left-right approximate straight line is extracted. The extraction area is limited based on the vehicle width condition with the previously extracted straight line.

The lane width condition in step S13 is, for example, that the approximate lines of the left and right white lines are (a _L , b _L ) and (a _L , b _L ), respectively.
_R, putting a b _R), the straight line _{(a L, b L),} (a R, a distance b _R) is represented by a straight line _{(a R -a L, b R} -b L), which is on the screen It may be a condition that the vehicle must be close to the lane width of the vehicle.

First, the (a, b) plane is limited to a straight line group satisfying the lane width condition from the straight line recognized last time. Next, in step S14, within the limited area,
(A, b) having the maximum weight is extracted. Then, in step S15, if the weight is equal to or less than a certain threshold value, the previous extraction value is used (step S16), otherwise, the current extraction line is set in step S17.
By such an operation, a white line approximate straight line is extracted.

As described above, in the embodiment of the present invention, white line recognition for extracting a contour line from a limited area after area division or white line recognition for extracting a straight line by Hough transform has been described. The present invention is not limited to the image recognition processing related to white line recognition described above, but can be applied to, for example, binarization processing, template matching processing, and the like. Further, in the embodiment, an example in which the image is divided into two regions (a fast moving region and a slow moving region) has been described. Is also applicable.

According to the vehicle-mounted image processing apparatus according to the first aspect of the present invention, it is possible to perform appropriate image processing according to the recognition target such as a white line. For example, when this apparatus is used for white line recognition, This has the effect that accurate white line recognition (position detection) can be performed.

Further, according to the vehicle-mounted image processing apparatus according to the second aspect of the present invention, in addition to the effect of the first aspect, it is possible to reduce the processing time.

Further, the vehicle-mounted image processing apparatus according to the third aspect of the present invention has the effect that the effects of the first or second aspect can be further enhanced and the processing time can be shortened. .

[Brief description of the drawings]

FIG. 1 is a block diagram according to one embodiment of the present invention.

FIG. 2 is a diagram showing an example when preprocessing such as contour detection is performed on an image photographed in front of a vehicle according to the embodiment;

FIG. 3 is a diagram showing an example when the image of FIG. 2 is subjected to time integration processing and divided into regions.

FIG. 4 is a diagram illustrating an example of preprocessing such as contour detection in the embodiment;

FIG. 5 is a block diagram of area division according to the embodiment.

FIG. 6 is a flowchart illustrating a flow of a white line recognition process according to the embodiment.

FIG. 7 is a graph when a time integration process and a change amount limiting process are performed.

FIG. 8 is a flowchart showing a flow of white line approximate straight line extraction in another embodiment.

FIG. 9 is a block diagram of a conventional technique.

FIG. 10 is a diagram showing an example of an image obtained by imaging the front of a vehicle in the related art.

FIG. 11 is a diagram showing a flow of a recognition process of a conventional example.

FIG. 12 is a block diagram illustrating a configuration of a time integration process.

FIG. 13 is an explanatory diagram of a problem in the related art.

[Explanation of symbols]

1 image photographing means (CCD camera), 2 A / D conversion means, 3 white line recognition means, 4 white lines, 5 roads, 6 characters on roads, 7 background, 23 one screen delay means, 24 internal division means, 25 time constant τ, 26 area O, 27 area I, 28 preprocessing means, 29 time integration means, 30 area division means, 31 one scanning line delay means, 32 one pixel delay means, 33, 34 difference means, 35, 36 absolute value means , 37 specifying threshold value, 38, 39 comparing means, 40 OR
Operator, 41 specific threshold, 42 comparison means.

Claims (3)

(57) [Claims] 1. An in-vehicle image processing apparatus comprising a photographing means for photographing a periphery of a vehicle and performing an image recognition process based on an obtained image signal, wherein a time based integration of data based on the image signal obtained by the photographing means is performed. And a dividing unit for dividing the data into a plurality of regions based on the data obtained by the time integrating unit. The image recognition processing is performed based on the divided regions obtained by the dividing unit. A vehicle-mounted image processing apparatus. 2. The on-vehicle image processing apparatus according to claim 1, further comprising a limiting unit configured to perform the image recognition processing by limiting at least one of the plurality of regions. 3. The on-vehicle image processing apparatus according to claim 1, wherein a time constant of time integration by said time integration means is changed according to a position of an image.