JPH11203458A - Road shape recognizing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To unnecessitate scanning such as white line detection and to be hardly affected by an environment by comparing an image position at the time of flat road surface with the image position of an on-road symbol detected from a picked-up image and detecting a road shape from the compared result. SOLUTION: A video signal image picked-up by a camera 1, is stored in an image memory 2 as a digital image, that image memory 2 is connected with a position detecting part 3, and that position detecting part 3 sets a radius showing the size of a circle so as to detect a delineartor preceding for a prescribed distance from the image. The circle of that radius is detected within a detection range set by a detection range setting part 4, the central position of that detected circle is compared with the image position at the time of flat road surface set by an image pickup position setting part 6 by a road shape detecting part 6, and a road gradient is detected. Thus, since the operation of a white line or the like is not required, processing is made simple and programming is facilitated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road shape recognizing device capable of recognizing a road shape of a car or the like at high speed with a small calculation load.

[0002]

2. Description of the Related Art As a conventional road recognition device, for example,
JP-A-7-198349 or JP-A-7-275
There is one disclosed in Japanese Patent Application Publication No. 41 (JP-A) 41. In these conventional devices, a road surface image is obtained by a camera, and a white line drawn on the road surface is detected by image processing. After recognizing the white line by an approximate line, a road gradient or the like is detected from the approximate line.

[0003]

However, in order to detect a white line as a road shape, if there are a plurality of white lines, it is necessary to always draw the white lines on the road at certain fixed intervals. Cannot be detected. In addition, even on a road where the intervals between the white lines are constant, a gradient cannot be detected because a white line cannot be detected on a road with a faint white line or a road with snow. Furthermore, in order to detect a gradient from a white line shape, a high-precision white line shape approximation is required, which increases the amount of calculation and requires a high-performance image processing device capable of high-speed processing.

A gyrocompass conventionally used as an in-vehicle gradient measuring device can measure a gradient angle of a road surface directly below a vehicle, but cannot determine a gradient of a road ahead from a vehicle. SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned conventional problems, and an object of the present invention is to provide a road shape recognizing device which is simple in operation and is less affected by the environment.

[0005]

According to the present invention, a camera for capturing an image of a road ahead of a vehicle, an image memory for storing an image captured by the camera, and A position detecting means for detecting a road sign having a predetermined height with a predetermined size and specifying a position on an image; an imaging position setting means for storing an image position of the sign when the road surface is flat; and an image when the road surface is flat. Road shape detecting means for detecting a road shape by comparing the position with the image position of the detected road sign is provided.

According to a second aspect of the present invention, after the position detecting means detects a road sign of a predetermined size, the position detecting means tracks and detects a traffic sign approaching the road by sequentially increasing the size according to the vehicle speed. The road shape detecting means detects the road shape based on the trajectory.

According to a third aspect of the present invention, there is provided a camera for capturing an image of a road ahead of a vehicle, an image memory for storing an image captured by the camera, and a predetermined distance from the captured image at a predetermined height with respect to a road surface. A plurality of road signs to be installed are detected at a predetermined size, and a position detection unit for specifying the position of the road sign on the image and a method of arranging the image position of the road sign and its size are used to confirm a detection target. Detection target confirmation means, imaging position setting means for storing the image position of the sign when the road surface is flat, and detecting the road gradient by comparing the image position of the road sign with the position when the road is flat And road shape detecting means for detecting the road shape at the image position of each road sign.

According to a fourth aspect of the present invention, the road sign is detected within a predetermined image range, and the range is set so as to cover the fluctuation of the sign image by the road gradient.

According to a fifth aspect of the present invention, it is confirmed that the on-road sign is a delineator, the position detecting means detects a circle having a predetermined radius, and a bar below the circle is a delineator. And a confirmation function.

According to a sixth aspect of the present invention, the road sign is a traffic light, the position detecting means detects a circle having a predetermined radius, and three detected circles are arranged in a predetermined relationship. And a confirmation function for confirming. According to a seventh aspect of the present invention, the position detecting means is provided with a function of detecting a color from three circles and confirming whether or not the color matches the traffic light.

[0011] According to an eighth aspect of the present invention, the road sign is a speed limit sign and the position detecting means detects two circles having different radii, and the detected two circles having different radii are concentric. Therefore, it has a function of confirming that the ratio is a certain value. According to a ninth aspect of the present invention, the confirmation means detects a color between the concentric great circle and the small circle, confirms that the color is red, and detects the color.

According to a tenth aspect of the present invention, the position detecting means determines a horizontal and vertical differential image from the captured image, and a gradient angle for determining an edge gradient angle based on the horizontal and vertical differential values. A calculation unit, a circle center calculation unit that calculates the center position of the circle when the edge is part of a circle having a certain radius r based on the gradient angle, and a circle center position calculated for each edge is voted on the Hough plane. And a circle center position detection unit for detecting the position of a point voted a predetermined number or more times on the Hough plane as the center position of the circle, and the gradient angle calculation unit to calculate the horizontal and vertical differential values. A gradient angle table for storing a corresponding gradient angle is connected, and the gradient angle table stores a flag indicating that there is no edge corresponding to a value where the horizontal and vertical differential values are smaller than the threshold value. It was as.

According to an eleventh aspect of the present invention, the gradient angle calculation unit creates a list indicating gradient angles only for edge positions equal to or greater than a threshold value and their coordinate positions from the gradient angle table,
The circle center calculation unit calculates the circle center position based on the listed gradient angles and coordinate positions. According to a twelfth aspect of the present invention, the circle center calculation unit is connected to a circle center table storing a calculated value of a center position of a circle having a radius r corresponding to an edge gradient angle.

[0014]

According to the first aspect of the present invention, when a sign of a predetermined height is imaged from a predetermined distance ahead, the size on the image is constant,
Since the road gradient is detected by using the fact that the vertical position on the image changes according to the road gradient, a sign image of a predetermined size is detected from the captured image of the camera, and the image position when the road is flat is determined. The road gradient can be detected only by the operation of comparison. Since an operation such as white line detection is not required, the processing is simple and an effect of easy programming can be obtained.

According to the second aspect of the present invention, the size of the sign detected in the first aspect is sequentially expanded in accordance with the vehicle speed, and the sign is detected. The tracking detection can be performed only by changing the size of the sign to be performed, and comprehensive road information can be provided without complicated operation.

According to the third aspect of the present invention, a plurality of road signs installed at predetermined intervals are detected at a predetermined size from the image, and the gradient and shape of the road are detected.
The gradient can be detected only by the operation of comparing with the image position when the road is flat. As for the road shape, a road shape such as a straight line or a curve can be detected only by comparing the vertical and horizontal positions of each sign. Since all of these operations are performed only by comparison, an effect that processing is simple and programming is easy can be obtained.

According to the fourth aspect of the present invention, the road sign is detected within a predetermined image range, and this range is set so as to cover the fluctuation of the sign image by the road gradient.
The amount of data to be processed is reduced, and road signs can be detected at high speed, and the effect of reducing erroneous detection can be obtained.

According to the fifth aspect of the present invention, the road sign is a delineator, and the position detecting means detects a circle having a predetermined radius.
Since a bar is detected under the circle to confirm that the bar is a delineator, the detection result is highly reliable.

According to the sixth aspect of the present invention, the road sign is a traffic light, and the position detecting means detects a circle having a predetermined radius and confirms that the three detected circles are arranged in a horizontal line, for example. Therefore, it is possible to reliably detect the traffic light. According to the seventh aspect of the present invention, the 3
Since the color is further detected from the three circles to check whether the color matches the traffic light, it is possible to prevent erroneous detection of the three circles by other objects.

In the invention according to claim 8, the road sign is a speed limit sign, the position detecting means detects two circles having different radii, and the two circles having different detected diameters are concentric and have a constant ratio. Since the value is confirmed, the reliability is improved as compared with the detection using only the concentricity or the ratio. According to the ninth aspect of the present invention, the color between the detected concentric great circle and small circle is detected to confirm that the color is red, so that the reliability of the detection result is further improved.

According to the tenth aspect of the present invention, the position detecting means obtains a horizontal and vertical differential image from the captured image, and obtains a gradient angle of an edge from a gradient angle table based on the horizontal and vertical differential images. At this time, if the horizontal and vertical differential values are smaller than the threshold, a flag indicating that there is no edge is obtained, so that threshold processing can be performed simultaneously with one operation, and circle detection can be performed at high speed.

According to the eleventh aspect of the present invention, the gradient angle calculation unit creates a list indicating the gradient angles of only edge positions equal to or greater than a threshold value and their coordinate positions from the gradient angle table, and the circle center calculation unit Since the center of the circle is calculated based on the listed gradient angles and coordinate positions, the amount of information is reduced, and information can be stored with a small amount of memory. Speed is improved. According to the twelfth aspect of the invention, since the circle center calculation unit is connected to the circle center table storing the calculated value of the center position of the circle having the radius r corresponding to the gradient angle of the edge, the same calculation is repeated. And the processing efficiency is improved.

[0023]

Embodiments of the present invention will be described below with reference to embodiments. First, the detection principle of the first embodiment will be described with reference to FIGS. 1 to 6, and the configuration of the embodiment will be described with reference to FIGS. FIG. 1 is a diagram showing a relationship between an imaging distance Z and a vertical position yp on an image when a circular sign D with a radius R on a road is imaged on a circle d with a radius r on the image, and FIG. It is an image. Here, the camera is a CCD camera including a lens L and a CCD having a focal length f, and the lens is located at a height Hc from the road surface and is an imaging surface separated from the lens on the optical axis of the lens by a focal length f. Is arranged with a CCD.

As can be seen from FIG. 1, the relationship between the radius r of the sign picked up on the image and the distance Z to the sign is given by the following equation (1).
Therefore, when a marker having the same radius R is imaged at the same radius r, the distance to the marker is always the same. r = R · f / Z Equation (1)

If the road is flat, the position yp (y-axis) where the sign is imaged and the distance Z to the sign are given by equation (2), so that a sign having the same height Hd is the same distance ahead. The height position on the image is always the same when the image is picked up from. yp = (Hc−Hd) · f / Z Equation (2) Here, the unit of yp and f is a pixel of the CCD of the camera, H
The unit of c, Hd, and Z is m. That is, if the road is flat, the signs that are imaged in the same size on the image have the same imaging distance, and are always imaged at the same height position on the image.

On the other hand, when the road ahead of the camera is a downhill or an uphill, the sign is imaged at a position lower or higher than the flat road, so that the sign is lower or higher than the flat road on the image. Become. FIG. 3 and FIG. 4 show a state of image pickup and a picked-up image of a downhill slope and an uphill slope, respectively, of the road ahead. (A) shows a state where an image is captured, and (b) shows a captured image. In FIG. 3, since the road ahead of the camera is downhill, the sign D is imaged lower by the gradient angle α, and the image pickup position yd is lower than yp on the flat road in FIG. In addition, on the uphill slope in FIG. 4, the sign D is imaged higher by the gradient angle α, and the imaging position yu on the image is higher than yp on the flat road in FIG.

As described above, since the vertical position of the sign at the predetermined height on the image taken from the predetermined distance differs depending on the road conditions, the sign is detected from the image and the gradient of the road is determined at the vertical position on the image. Can be determined. In addition, since the sign at a predetermined distance is imaged at a predetermined size on the image, it is necessary to detect a sign having the same size as a detection target, detect the sign at the same size from the captured image, and detect a road gradient at a position on the image. Is possible. In addition, at the time of detection, since only a simple process of detecting a sign image of a fixed size is sufficient, blogger programming is easy, and high-speed processing can be performed even with an ordinary image processing apparatus.

Next, a description will be given of an embodiment in which the gradient of the road ahead is obtained by using the above principle. Here, a delineator is used as a road sign. The delineator is a sign installed at equal intervals beside the road and has a predetermined height from the road surface. Each delineator has a shape in which a rod supports a circular reflector in common. The circular reflector faces the traveling direction of the road, and a reflector having a radius R and a rod B supporting the reflector as shown in FIG. When an image of the road ahead is taken while the traveling vehicle is traveling, an image as shown in (b) is obtained.

As described above, the fact that the circle is supported by the rod is a feature common to each delineator, and since the size of the reflector is known, the circle d whose size is known from the image is detected as an object to be detected. The delineator can be detected later by confirming that the bar is attached to the circle. The gradient of the road is detected by comparing the position of the detected delineator circle with the position on a flat road.

Next, detection of a delineator on an image will be described. Since the size of the delineator (reflector) is fixed, if the imaging distance is determined, the expression (〓
According to 1), the radius r of the circle detected from the image is determined. The detection of the delineator is performed on the circle having the radius r, but the detection is performed by limiting the detection range to an area where the delineator can appear due to the road gradient. For example, assuming that the maximum gradient angle is α (rad) for both downhills and uphills, the upper and lower limits (yu, yd) of the detection position of the delineator are determined from FIGS. Therefore, the detection range of the circle can be limited to the range of yd to yu on the y-axis as shown in FIG. yd and yu are calculated by equation (3). When the detection area of the circle is limited as described above, the detection processing speeds up, and an effect of reducing erroneous detection is obtained.

(Equation 1)

Next, a first embodiment will be described with reference to FIGS. FIG. 7 is a block diagram showing the configuration of this embodiment, and FIG. 8 is a flowchart showing the flow of the processing. In FIG. 7, a camera 1 includes a lens L having a focal length f.
And a CCD arranged so that the distance from the lens to the imaging surface is equal to the focal length f. The CCD camera is attached to an appropriate portion on the vehicle so as to be able to image the road ahead. The distance between the lens center of the camera and the road surface is Hc.

The video signal picked up by the camera 1 is output to the image memory 2 as a digital image and stored. The image memory 2 is connected to a position detection unit 3, and the position detection unit 3 is also connected to a road shape detection unit 5. A detection range setting unit 4 and an imaging position setting unit 6 are connected to the position detection unit 3 and the road shape detection unit 5, respectively.

A radius r indicating the size of a circle is set in the position detecting section 3 so that a delineator at a predetermined distance Z from the image can be detected from the image, and the video signal input from the image memory 2 is set to a detection range setting. Image processing is performed within the detection range set by the unit 4 to detect a circle having a radius r, and the center position of the circle is extracted. The road shape detection unit 5 detects the road gradient by comparing the center position of the circle detected by the position detection unit 3 with the image position when the road set by the imaging position setting unit 6 is flat.

Next, the processing flow in the above configuration is shown in FIG.
This will be described according to the flowchart of FIG. First step 10
In step 1, the imaging distance Z of the delineator is determined, and the detection range setting unit 4, the imaging position setting unit 6, and the position detection unit 3 perform initial setting based on the determined imaging distance Z. As a result, an area surrounded by yd and yu calculated based on the above equation (3) is set as a detection area in the detection range setting unit 4 as shown in FIG. Further, the radius r of the circle to be detected is set in the position detection unit 3. The imaging position setting unit 6 stores the center position yp of the circle of the delineator when the road is flat.

In step 102, an image signal is input from the image memory 2 to the position detector 3. Step 103
In, the position detection unit 3 reads the set value (yd to yu) of the detection range from the detection range setting unit 4, performs image processing on the image signal within the set range, and detects a circle having a radius r. Do. For circle detection, for example, a known technique such as template matching or Hough transform can be used. The high-speed circle detection using the Hough transform will be described later in detail.

When a circle having a radius r is detected, a vertical edge is further detected from the captured image, and it is confirmed that there is a vertically long edge below the detected circle. If there is a vertical edge,
The detection circle extracts the vertical position (y-axis) on the image as a circle of the delineator. To detect a vertical edge, for example, an operator for detecting a vertical edge such as a Sobel can be used. By the above processing, the circle of the delineator located a predetermined distance ahead is detected from the image, and the height of the position, that is, the vertical position (y-axis) is specified.

In step 104, the coordinate value yc of the vertical position of the extracted circle is compared with the position yp set in the image pickup position setting section 6 to detect a forward road gradient.
That is, from the magnitude relationship between the vertical position yc of the circle detected from the image and the height position yp at which the center of the circle is imaged when the road is flat, if yc = yp, the front is a flat road, if yc is yp, The front is determined to be an uphill, and if yc> yp, the front is determined to be a downhill.

As described above, a delineator whose shape, size, and height are known is detected as a detection target, and the road gradient is determined based on the vertical position on the image. It is possible to detect the gradient of the road ahead before the host vehicle enters the gradient only by performing circle detection using monocular image processing without performing complicated processing.

In addition, since this method does not use white line detection, it is possible to detect a gradient that is not affected by the shape or presence or absence of a white line. Furthermore, with this method, image processing is performed only by simple processing of detecting circles with fixed sizes and shapes, so programming is simple and high-speed processing is possible even with image processing equipment with general performance Is obtained. In addition, as for the detection of the circle of the delineator, it is possible to detect the delineator with high quality by simple processing by checking whether or not the bar which is the characteristic of the circle is detected. It will be highly reliable.

In this embodiment, the inclination of an uphill or a downhill is detected. However, the integration with the inclination sensor of the own vehicle such as a gyro compass allows not only the change in the gradient of the road ahead but also the actual inclination. The gradient can also be predicted. In addition, if it is determined from the gyro compass that the vehicle is downhill, if the image indicates that the road ahead is flat, the vehicle is traveling uphill, indicating that the downhill continues ahead. If it is determined from the image that the road ahead is flat, it can be determined that the vehicle is going uphill in front.

Next, a second embodiment will be described. In this embodiment, the delineator detected in the first embodiment is detected by tracking to detect not only the gradient but also the road shape. When the vehicle travels, the positional relationship between the delineator and the camera changes, and the delineator is imaged while drawing the shape of the road on the image. Therefore, the road shape can be detected by tracking and detecting the delineator. That is, after detecting the delineator from a predetermined distance and detecting the gradient of the road ahead at the height position, the shape of the road including the lateral direction can be detected by tracking and detecting the change in the position of the delineator. .

In order to track and detect the delineator, it is necessary to change the detection radius r of the circle on the image according to the change in the imaging distance. Thus, for example, if one image processing time is Δt, the vehicle speed is V, the radius of the delineator is R, and the distance to the first delineator to be detected is Z, the circle to be detected in the processing after i times is determined. The radius ri can be calculated based on equation (4). FIG. 9 shows the change in the distance and the change in the radius r of the circle. (A) shows a case where an image is taken at a distance from the predetermined distance Z, and (b) shows an image pickup state after updating the image. ri = f · R / (Z−iΔt · V) Equation (4) where i is the number of times of image update and is composed of integers of 0, 1,. The detection range of the circle at that time is the imaging distance Z
−i · Δt · V (m), as in FIG.
, And is a range surrounded by yui and ydi.

(Equation 2)

As described above, by detecting the circle by changing the detection range and the radius of the circle in synchronization with the update of the image, and comparing the positions of the circles at each time, the road shape can be detected. . For example, when the position change is non-linear as in FIGS. 10 and 11, it can be understood that the road is a curve. At this time, whether the vertical position of the circle d detected first is above or below a flat road can be determined as an uphill slope curve or a downhill slope curve. 10 shows an uphill curve, and FIG. 11 shows a downhill curve. When the position change of the circle of the delineator changes linearly as shown in FIG. 12 and FIG. 13, if the detection position of the first circle d is above a flat road, it is straight uphill, and if it is below it, it is straight uphill. We can judge that it is downhill. 12 shows a straight uphill slope, and FIG. 13 shows a straight downhill slope.

FIG. 14 is a block diagram showing the configuration of the second embodiment. The camera 1 is installed in a vehicle as in the first embodiment, and the lens L has a height Hc above the road surface. The video signal from the camera 1 is output to the image memory 2 and stored. The image memory is connected to the position detector 31. The vehicle-side vehicle speed signal V is input to the position detection unit 31 and the detection range setting unit 41 connected thereto. The position detecting unit 31 is connected to the position storing unit 80, and the position storing unit 80 is connected to the road shape detecting unit 51 together with the imaging position setting unit 61.

The radius r of the circle to be detected is set as an initial setting in the position detector 31 so that the circle of the delineator at a predetermined distance ahead is detected. After detecting a circle with radius r,
The radius of the detected circle is calculated from the updated image based on the equation (4) using the vehicle speed V and the number of updates i in synchronization with the update of the image memory. Then, a circle whose radius is calculated at each time is detected by image processing on the video signal within the range set by the detection range setting unit 41.

The detection range setting unit 41 sets the detection range of the circle from the updated image based on the equation (5) with the vehicle speed V and the update count i in synchronization with the update of the image memory. The circle detection range is calculated according to the change in the imaging distance. The position storage unit 80 stores the position of the circle at each time detected by the position detection unit 31. The imaging position setting unit 61 stores the imaging position of the delineator at a predetermined distance Z ahead as in the first embodiment.

The road shape detecting section 51 compares the position of the circle detected first among the detected positions of the circle with the position yp of the flat road set by the image pickup position setting section 61 to determine the gradient of the road ahead. Is detected. Then, based on the position of each time,
The road shape is determined based on the shape of the trajectory as shown in FIG. As described above, this embodiment adds a function of tracking and detecting the circle detected in the first embodiment to determine the road shape, so that not only the road gradient but also the road shape can be detected. It is possible to provide comprehensive road conditions.

Next, a third embodiment will be described. In this embodiment, a plurality of delineators set at equal intervals are detected, and the road shape is detected based on the arrangement on the image. FIG. 15 is a diagram illustrating a state where three delineators arranged at equal intervals are imaged. Assuming that the distance to the foremost detection target delineator D is Z (m) and the distance between the delineators is e, the delineator distances are Z, Z + e, and Z + 2e, respectively. By substituting these into Equations (1) and (2), the radius and position of the circle on the image can be calculated.

That is, the radius of each circle can be obtained by equation (6). r0 = R ・ f / Z, r1 = R ・ f / (Z + e), r2 = R ・ f / (Z + 2e), (6) where r0 is the radius of the circle of the delineator at a distance Z away. r1 is the radius of the circle of the delineator at the distance Z + e. r3 is the radius of the circle of the delineator at a distance Z + 2e.

In the case where the detection target is three consecutive delinators, all of them can be imaged in the ranges yu and y.
Since d is the same concept as in FIG. 6, yd is the height at which the foremost delineator is imaged at the bottom, and yu is the highest height at which the delineator two images behind it can be imaged. 7). FIG. 16 is an image in which a detection range is set.

(Equation 3)

Within this range, three types of radii r0,
A circle having a size of r1 and r2 is detected, and based on this, a delineator can be detected in the same manner as in the first embodiment. Also, at this time, the positional relationship between three circles of this size,
For example, it is possible to reconfirm that the positional relationship, such as the order from the largest to the smallest from the bottom up, is a delineator. Further, based on these positional relationships, it is possible to recognize the road shape ahead including the road gradient and the lateral displacement.

That is, the positions y0, y1, and y2 at which the three circles are imaged when the road ahead is flat can be obtained by substituting the respective distances into equation (2). The actually detected y as shown in FIG.
From the relationship between 0, y1, y2 and the y-coordinate value calculated on a flat road, for example, if y0-y2 is larger than the calculated value, it can be determined that the uphill starts from behind y0. If y0-y2 is smaller than the calculated value, it can be determined that the downhill starts from behind y0. FIG. 17 shows an uphill slope, and FIG. 18 shows a downhill slope.

FIG. 19 is a diagram showing the relationship between the imaging position in the x-axis direction and the distance of equally spaced delineators when the displacement from the road side is W (m) when the road ahead is straight. . At this time, the positions in the x-axis directions x0, x1, and x2 at which the three delineators are imaged can be obtained by Expression (8). x0 = W · f / Z, x1 = W · f / (Z + e), x2 = W · f / (Z + 2e), (8)

Here, even if the actual displacement W is unknown, the ratio between (x0-x1) and (x1-x2) when the front is a straight line is calculated from the distance Z to the three delineators and the distance e from the distance e. A = (x0−x1) / (x1−
If x2) is smaller than the reference value, it can be determined that the curve is a right curve.

FIG. 20 is a block diagram showing the configuration of the third embodiment. As in the first embodiment shown in FIG. 7, a camera 1 is installed in a vehicle, and an image signal is output and stored in an image memory 2. The image memory 2 is connected to a position detection unit 30 having a detection range setting unit 40. The position detection unit 30 is connected to the road shape detection unit 50 via the detection target confirmation unit 70. An imaging position setting unit 60 is connected to the road shape detection unit 50.

As the initial setting, the detection range setting section 40 stores yu and yd calculated based on the above equation (7) to set the detection range of the circle. The position detector 30 has three types of radii r0 calculated based on the calculation of the above equation (6),
r1 and r2 are stored. In the imaging position setting unit 60, an imaging position at the time of a road where the circle having the largest diameter r0 among the circles is flat is set. When the road is straight, (x0
-X1) and (x1-x2) are calculated and stored as a reference value.

When a road shape is detected, a video signal of a captured image is output from the image memory to the position detection unit 30. The position detection unit 30 has the radii r0, r1, r2 set from the captured image within the range set by the detection range setting unit 40.
To detect three types of circles, the vertical direction (y-axis), and the horizontal direction (x
Extract the position of (axis). From the bottom, the detection target confirmation unit 70
Check to see if the diameter is progressively smaller to make sure the circle is a delineator.

The road shape detecting section 50 first judges the road gradient by comparing the vertical position of the circle (r0) having a large radius with the position of a flat road set in the image pickup position setting section 60. A is determined by the horizontal position x0, x1, x2 of each circle.
= (X0-x1) / (x1-x2) and compare with a reference value. If A and the reference value match, it is determined that the road is a straight road as shown in FIG. If A is smaller than the reference value, FIG.
It is determined to be a right curve road as shown in 2. If A is larger than the reference value, it is determined that the curve is a left curve as shown in FIG. This embodiment is configured as described above, and by using the delineators arranged at equal intervals, it is possible to recognize a road shape including a curve in a lateral direction in addition to a gradient of a road ahead. In addition, since the detection of the delineator is checked according to the arrangement of the circles, the detection result is highly reliable.

Next, detection of the circle of the delineator will be described as a fourth embodiment. This embodiment is mainly designed to detect a circle at high speed even when there is noise. FIG. 24 shows the position detector 3 (30,
It is a functional block diagram of circle detection in 31). The captured image from the image memory 2 is input to the differential image creation unit 33. Here, the image is differentiated within the detection range set by the detection range setting unit to create differential images in the x and y directions.

The gradient angle calculator 34 calculates the gradient angle θ of each edge from the gradient angle table 38 based on the horizontal and vertical differential values (dx, dy). The circle center calculation unit 35 calculates the center position (xc, yc) from the circle center table 39 using the edge as a part of a circle having a radius r based on the gradient angle θ. Hou is the circle center coordinate calculated for each edge point.
The gh plane creation unit 36 votes on the Hough plane. The circle center position detecting unit 37 extracts points voted a predetermined number of times or more on the Hough plane, and detects the points as circle centers.

FIG. 25 is a flowchart showing the flow of the circle detecting process. First, in step 201, when a captured image is input from the image memory 2 to the differential image creation unit 33, differentiation processing is performed on the captured image in the detection range set by the detection range setting unit, and horizontal and vertical A derivative image is created. The differentiation process is performed by, for example, convolution of the sobel operator and the captured image in FIG. Here, FIG. 26 (a) is a horizontal differential processing single-level operator, and FIG. 26 (b) is a vertical differential processing single-level operator. By the differentiation process, a horizontal differential image (b) and a vertical differential image (c) are created from the original image (a) as shown in FIG. 27, for example.

In step 202, a gradient angle image in which the gradient angle of the edge has been calculated by the gradient angle calculator 34 based on the horizontal differential value and the vertical differential value of each edge is created. The gradient angle θ can be obtained by Expression (9). θ = arctan (dx / dy) (9) However, the gradient angle may be obtained only at a position having an edge equal to or larger than the threshold value. In addition, since the calculation by equation (9) requires processing time to calculate each time an image is updated, considering the speedup, the following method is used to calculate the gradient including the gradient angle of the entire screen. Create a corner image.

Since the luminance range of an image is generally from 0 to 255, the values of the horizontal differential dx and the vertical differential dy are values between -128 and 127 as a result of using the operator in FIG. Expressing it as an 8-bit two's complement, 0
x80 to 0x7F. From the values of dx and dy, a two-dimensional table in which the vertical axis is dy and the horizontal axis is dx is created so that the value of equation (9) can be obtained by table lookup. FIG. 28 shows the structure of the table.

The table shows that A, B,
Four regions C and D are formed. dx is 0 to 12
7, dy 0 to 127 constitutes the C region, where π to 3
A gradient angle of π / 2 is calculated. dx is -128 to -1,
dy of 0 to 127 constitutes the D region, where 3π / 2 to 2
A gradient angle of 2π is calculated. Also. dx is 0 to 127,
dy = −128 to −1 configures the B region, where π / 2
A gradient angle of ~ π is calculated. dx is -128 to -1, d
y is -128 to -1 to form the A region, where 0 to π /
2 are calculated. Since the circle detection is not performed on the portion where the edge is equal to or smaller than the threshold, a flag indicating that the edge is equal to or smaller than the threshold is set in the portion (hatching) where the absolute values of dx and dy are equal to or smaller than the threshold (TH). Keep it in. This table is used as a gradient angle table. Slope angle calculator 3
4 calculates the gradient angle by subtracting the above table from the gradient angle table based on the values of dx and dy.

As a result, the gradient angle θ at each edge is obtained from the gradient angle image at the position where each edge exists. At the position where there is no edge, a flag indicating that there is no edge is obtained. As described above, since the calculation of the gradient angle of each edge is performed by table look-up, the calculation by Expression (9) is omitted, and high-speed processing becomes possible. Also, when the edge strength is equal to or less than the threshold value, a flag indicating that there is no edge is obtained by the above lookup, so that the operation for determining the threshold value is omitted and the processing speed is further improved.

The circle center calculator 35 determines the center position (xc, yc) of the circle based on the gradient angle of each edge, assuming that the pixel (x, y) having the edge is a part of a circle having a radius r. Ask. FIG.
9 is a diagram showing the positional relationship between the gradient angle of an edge and the center position of a circle including the edge.

If the gradient angle of the edge at a certain position (x, y) is known, geometric calculations show that if the edge is part of a circle with a radius r, the center (xc, yc) of the circle is It can be obtained from Equation 10. xc = x + r · cos θ yc = y + r · sin θ (10)

The detection of xc and yc is performed based on the principle of Hough transform. The Hough transform means that if the center (x
If a circle having a radius r from c, yc) exists in the image within the processing range, if the above calculation is repeated at the gradient angles θ of all edges in the entire image, the center position of the circle will be found at all edges on the circumference. Since it is obtained as (xc, yc), the number of times that the calculation result of Expression (10) is obtained as (xc, yc) increases. From this, if (xc, yc) and the number of times obtained are equal to or greater than a certain threshold value, the position can be determined to be the center of the radius r circle.

Next, based on the above principle, the center position is obtained at high speed by using the above-described gradient angle image including the flag indicating that there is no edge. First, the Hough plane creating unit 36 creates a Hough plane corresponding to a captured image. FIG. 30 shows a Hough plane in a formal manner, and each grid corresponds to a pixel position of an image. The circle center table 39 has rc corresponding to the gradient angle θ.
The data of osθ and r-sinθ are stored.

When the gradient angle θ is input to the circle center table, the corresponding r-cos θ and r-sin as shown in FIG.
θ is read. (A) is a table for calculating r-cos θ, and (b) is a circle center table for calculating r-sin θ. In all the pixels of the gradient angle image, first, it is checked whether or not the content is a flag indicating that there is no edge, and only at the position where the gradient angle is stored, a value is obtained from the circle center table to obtain the edge. The center of the circle is calculated by substituting into the equation (10) together with the pixel position.

In step 203, the result of the calculation of the center of the circle is cast to the corresponding position on the Hough brain. This makes it possible to judge the presence or absence of an edge only by judging the flag, determine the gradient angle at the same time as determining whether or not there is an edge, and look up and add the circle based only on that angle to obtain the circle. Since the center can be calculated, the center of the circle can be voted on the Hough plane only by high-speed and simple calculation.

In the case where a higher speed is required in creating the Hough plane, the following method is also available. That is, when the gradient angles are looked up from the table in FIG. 28, a list as shown in FIG. 32 is created instead of creating a gradient angle image having a flag and a gradient angle. This list is created in the following way. In the list, the vertical axis is y
Create two types of lists, a list that stores the gradient angle and a list that stores the x coordinate.
A counter is provided for storing the number of edges on the line for each coordinate. (A) is a gradient angle list, and (b) is an x coordinate list. Here, the counter is provided in the x coordinate list.

Differential values dx, dy for each y coordinate of the captured image
The gradient angle is obtained by looking up the gradient angle table. Only when there is an edge, the gradient angle is stored in the list of gradient angles at that position, the x coordinate value is stored in the list of x coordinates, and the counter is incremented. When this operation is performed on all y lines, the number of edges on the y line is entered in the counter at the head of each y line, and only the gradient angle of each edge on the line and the x coordinate value of the edge are stored. The list is completed.

The Hough plane can be created by using this list and calculating the equation (10) for each line by the number of times stored in the counter. Is omitted, and in particular, processing of an image having few edges such as a short list can be performed at higher speed. Another advantage is that information on the gradient angle can be stored in a smaller area than the gradient angle image.

Next, in step 204, Houg
The center position of the circle from within the h plane is detected. The detection of the center position of the circle can be detected as the position of the maximum value in the plane if the image to be processed has no noise and includes only the circle. If the voting result does not become a peak point due to a large amount of noise in the image or the like, the score increases as a region having an area near the center of the circle as shown in FIG. 33. For example, labeling the Hough plane with a certain threshold or more is performed. The center of the circle can be obtained by setting the center of gravity of the label image as the center of the circle.

In addition, this method can cope with not only an image having a lot of noise but also an image in which an edge is hard to appear such that only a semicircle is detected by lowering the setting of the labeling threshold. By the above method, the center position of the circle having the radius r in the image can be detected, and the circle is detected based on the center position and the radius r.

Next, a modified example will be described in which the detection target is a road sign other than the delineator using the above-described circle detection principle. First, the signal detection method is explained. In the case of a traffic light, the positional relationship between the three circles is constant and the height of the traffic light is also within a certain height range. Therefore, first, three circles of a predetermined size are detected. As the method of circle detection, the above-described method is used based on the radius of the circle of the traffic light. If the center positions of the three detected circles are aligned on one straight line, it is determined that the signal is a signal. In addition, when performing more accurately, the color inside the circle is blue, as shown in FIG.
The determination can also be made by determining whether the order is yellow and red and whether there are horizontal edges above and below the circle.

Next, a method of detecting a speed limit sign will be described. Here, a speed limit sign as shown in FIG. 35 will be described. The sign indicating the speed limit also has a known size and shape, similarly to the traffic light, and in most cases, the height is often constant. It is also known that the inside of a large circle is red and the inside of a small circle is white, and that the ratio (Rs: Rb) between the large and small circles is also constant. Using this known information, two circles having different ratios of radius Rs: Rb are detected, and a marker whose center is located at the same position is detected to detect a marker. In addition, by recognizing that the area between the large circle and the small circle of the pair of two circles is red, it is possible to reconfirm that the area is a sign.

Next, a description will be given of road gradient detection using detection results of traffic signals and signs. Traffic lights and signs are located throughout the general road, unlike delinators,
Although it can be used as gradient detection on general roads, the height and the existence interval are not always constant. However, on general roads, there are some steep roads that do not exist on expressways, and when such a slope is ahead, as shown in FIG.
When a traffic light that is only captured near the center yc of the image is captured at a very low or very high position in the image, it is known that the road ahead is steep. FIG. 37 and FIG. 38 are captured images when a steep uphill and a steep downhill are respectively imaged.

[0080]

As described above, according to the first aspect of the present invention, when a sign having a predetermined height is imaged from a predetermined distance ahead, the size on the image is constant, and the vertical position on the image is the road gradient. The road gradient is detected using the change in the road gradient, so the sign image of a predetermined size is detected from the image captured by the camera, and the road gradient is detected only by the operation of comparing with the image position when the road is flat. Can be detected.
Since an operation such as white line detection is not required, the processing is simple and an effect of easy programming can be obtained.

According to the second aspect of the present invention, the size of the detected sign is further sequentially enlarged in accordance with the vehicle speed, and the detected sign is tracked, and the road shape is detected based on the track. Tracking detection can be performed only by changing the size, and comprehensive road information can be provided without complicated operations.

According to the third aspect of the present invention, a plurality of road signs installed at predetermined intervals are detected at a predetermined size from the image, and the gradient and shape of the road are detected.
The gradient can be detected only by the operation of comparing with the image position when the road is flat. As for the road shape, a road shape such as a straight line or a curve can be detected only by comparing the vertical and horizontal positions of each sign. Since all of these operations are performed only by comparison, an effect that processing is simple and programming is easy can be obtained.

According to the fourth aspect of the present invention, the road sign is detected within a predetermined image range, and this range is set so as to cover the fluctuation of the sign image by the road gradient. Can be detected at high speed, and the effect of reducing erroneous detection can be obtained.

According to the fifth aspect of the present invention, the road sign is used as a delineator, and the position detecting means detects a circle having a predetermined radius, and detects a bar below the circle to confirm that it is a delineator. The detection result becomes highly reliable.

According to the sixth aspect of the present invention, the road sign is used as a traffic light, and the position detecting means detects a circle having a predetermined radius, and confirms that three detected circles are arranged in a horizontal line, for example. Therefore, the traffic signal can be detected more reliably. According to the seventh aspect of the present invention, the color is further detected from the detected three circles to check whether the color matches the traffic light. Therefore, the three circles are erroneously detected by other objects. Can be prevented.

According to the invention of claim 8, the road sign is a speed limit sign, the position detecting means is a circle detecting means for detecting two circles having different radii, and the two detected circles having different diameters are concentric. Since the ratio is confirmed to be a constant value, highly reliable detection is achieved. According to the ninth aspect of the invention, the color between the detected concentric great circle and small circle is detected to confirm that the color is red, so that the reliability is further improved.

According to the tenth aspect, the position detecting means obtains a horizontal and vertical differential image from the captured image, and obtains a gradient angle of an edge from a gradient angle table based on the horizontal and vertical differential images. At this time, if the horizontal and vertical differential values are smaller than the threshold, a flag indicating that there is no edge is obtained. Therefore, threshold processing can be performed by one operation, and circle detection can be performed at high speed.

According to the eleventh aspect of the present invention, when detecting a circle, the gradient angle calculation unit creates a list indicating the gradient angles and the coordinate positions of only the edge positions above the threshold value from the gradient angle table, Since the center calculation unit calculates the center of the circle based on the listed gradient angles and coordinate positions, the amount of information is reduced and the information can be stored with a small amount of memory as compared with performing the entire screen. Possible and processing speed is improved. According to the twelfth aspect of the present invention, since the circle center table in which the calculated value of the center position of the circle having the radius r corresponding to the gradient angle of the edge is connected to the circle center calculation unit, the same calculation is repeated. And the processing efficiency is improved.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a principle of detecting a road gradient.

FIG. 2 is an image captured by a camera.

FIG. 3 shows a state when a downhill is imaged and an image thereof.

FIG. 4 shows a state when an image of an uphill is taken and its image.

FIG. 5 shows a shape of a delineator and a captured image thereof.

FIG. 6 is a diagram illustrating a detection range of a delineator.

FIG. 7 is a block diagram showing the configuration of the first embodiment.

FIG. 8 is a flowchart illustrating a flow of a process according to the embodiment.

FIG. 9 is an explanatory diagram of a change in a distance to a detection target and a change in a position and a size on a captured image.

FIG. 10 is a time trajectory of a detected position of a circle on an uphill slope curve.

FIG. 11 is a time trajectory of a detection position of a circle on a downward slope curve.

FIG. 12 is a time trajectory of a circle detection position on an uphill slope on a straight road.

FIG. 13 is a time trajectory of a circle detection position on a downhill slope of a straight road.

FIG. 14 is a block diagram showing a configuration of a second example.

FIG. 15 is a captured image when three delineators are continuously imaged.

FIG. 16 is a diagram showing detection ranges of three delineators.

FIG. 17 is a captured image when the road ahead is uphill.

FIG. 18 is a captured image when a forward road is downhill.

FIG. 19 is an explanatory diagram of positions in the x-axis direction at which delineators arranged at equal intervals are imaged;

FIG. 20 is a block diagram showing a configuration of a third embodiment.

FIG. 21 is a captured image when the road ahead is straight.

FIG. 22 is a captured image when the front road is curved right and left.

FIG. 23 is a captured image when the road ahead is a left curve.

FIG. 24 is a functional block diagram showing a configuration of circle detection.

FIG. 25 is a flowchart illustrating a flow of a circle detection process.

FIG. 26: Sobel operator for obtaining horizontal and vertical differential images

FIG. 27 is an image showing the effect of the differentiation processing.

FIG. 28 is a gradient angle calculation table.

FIG. 29 is an explanatory diagram of a method of obtaining a center of a circle from an edge position and a gradient angle of the position.

FIG. 30 is a diagram formally showing a Hung plane.

FIG. 31 is a table for subtracting r-cos θ and r-sin θ from θ.

FIG. 32 is a list of gradient angles and x coordinate positions.

FIG. 33 is a diagram showing a labeling effect.

FIG. 34 is a diagram showing the shape of a traffic light.

FIG. 35 is a diagram showing the shape of a speed-limited road sign.

FIG. 36 is a diagram showing an imaging position of a traffic light on a flat road.

FIG. 37 is a diagram illustrating an imaging position of a traffic light on a steep ascending slope.

FIG. 38 is a diagram illustrating an imaging position of a traffic light on a steep downhill.

[Explanation of symbols]

Reference Signs List 1 CCD camera (camera) 2 Image memory 3, 30, 31 Position detecting unit (position detecting means) 4, 40, 41 Detection range setting unit 5, 50, 51 Road shape detecting unit (road shape detecting means) 6, 60 61 imaging position setting unit (imaging position setting means) 70 detection target confirmation unit 80 position storage unit

Claims (12)

[Claims] 1. A camera for imaging a road ahead of a vehicle,
An image memory for storing a captured image of a camera, a position detecting means for detecting a road sign having a predetermined height with respect to a road surface with a predetermined size from the captured image and specifying a position on the image; Image position setting means for storing the image position of the sign, and road shape detecting means for detecting the road shape by comparing the image position when the road surface is flat with the detected image position of the road sign. A road shape recognition device characterized by the above-mentioned. 2. The method according to claim 1, further comprising: detecting a road sign of a predetermined size, and further detecting the approaching road sign by sequentially increasing the size according to the vehicle speed, and detecting the approaching road sign. The road shape recognition device according to claim 1, wherein the road shape is detected based on the trajectory. 3. A camera for imaging a road ahead of a vehicle,
An image memory that stores a captured image of a camera, and from the captured image, a plurality of road signs installed at a predetermined height with respect to a road surface at a predetermined interval are detected at a predetermined size, and an image of the road sign is detected. A position detection unit that specifies the position of the road sign, a detection target confirmation unit that confirms a detection target by arranging the image position of the road sign and its size, and an imaging position setting that stores the image position of the sign when the road surface is flat. Means, and a road shape detecting means for detecting a road gradient by comparing an image position of the road sign with a position when the road is flat, and detecting a road shape at an image position of each road sign. A road shape recognition device characterized by the following. 4. The traffic sign according to claim 1, wherein the road sign is detected within a predetermined image range, and the range is set so as to cover a change in the sign image depending on a road gradient.
4. The road shape recognition device according to 2 or 3. 5. The road sign is a delineator, and the position detecting means has a circle detecting function for detecting a circle having a predetermined radius and a confirming function for detecting a bar below the circle to confirm that it is a delineator. 4. The road shape recognition device according to claim 1, wherein the road shape is recognized. 6. The road sign is a traffic light, and the position detecting means has a circle detecting function for detecting a circle having a predetermined radius, and a confirmation function for confirming that three detected circles are arranged in a predetermined relationship. Claim 1, characterized by having
4. The road shape recognition device according to 2 or 3. 7. The road according to claim 6, wherein the position detecting means is further provided with a function of detecting a color from three circles and confirming whether the color matches the traffic light. Shape recognition device. 8. The road sign is a speed limit sign, the position detecting means detects a circle having two different diameters, and the detected two circles having different diameters are concentric and have a constant ratio. The road shape recognition device according to claim 1, further comprising a confirmation function for confirming the value. 9. The road shape recognition apparatus according to claim 8, wherein said confirmation means detects a color between a concentric great circle and a small circle, confirms that the color is red, and detects the color. 10. A position detecting unit comprising: a differential image generating unit for obtaining a horizontal and vertical differential image from a captured image; a gradient angle calculating unit for obtaining an edge gradient angle based on horizontal and vertical differential values; A circle center calculating unit for calculating the center position of the circle when the edge is a part of a circle having a certain radius r, and a Hough plane creating unit for voting the circle center position obtained for each edge to a Hough plane. , A circle center position detecting unit for detecting a position of a point voted a predetermined number or more times on the Hough plane as a center position of the circle, and storing a gradient angle corresponding to a horizontal or vertical differential value in the gradient angle calculating unit. And a flag indicating that there is no edge corresponding to a value whose horizontal and vertical differential values are smaller than a threshold value. The road shape recognition device according to 5, 6, 7, 8 or 9. 11. The gradient angle calculation unit creates a list indicating gradient angles of only edge positions equal to or larger than a threshold value and their coordinate positions from the gradient angle table, and the circle center calculation unit calculates the listed gradients. The road shape recognition device according to claim 10, wherein a circle center position is calculated based on the corner and the coordinate position. 12. A circle center table storing a calculated value of a center position of a circle having a radius r corresponding to a gradient angle of an edge is connected to the circle center calculation unit. Road shape recognition device.