JP4209105B2 - Vehicle periphery visual recognition device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a vehicle periphery visual recognition device that captures an image of a field of view in the left-right direction of a vehicle and presents it to a driver.
[0002]
[Related background]
For example, at an intersection with poor visibility, the driver pushes the front part of the vehicle to the intersecting road for safety confirmation, which causes problems such as hindering traffic. As a countermeasure, for example, in Japanese Patent No. 3033389, a pair of CCD cameras are installed on the front part of the vehicle toward the left and right, and the left and right fields of view captured by these CCD cameras are presented to the driver on a display. Has been proposed. Japanese Patent Laid-Open No. 2000-71875 proposes a technique in which a left and right field of view is imaged by a single CCD camera via a mirror.
[0003]
And in any peripheral visual recognition device, assuming the case where the vehicle stops diagonally with respect to the intersecting road, it is equipped with a mechanism that changes the angle of the CCD camera in the horizontal direction according to the angle of the road, Thus, consideration is given to obtaining an appropriate image representing the left and right road conditions even when the vehicle stops diagonally.
[0004]
[Problems to be solved by the invention]
However, any of the above-described vehicle periphery visual recognition devices has not been proposed with respect to specific control details of the camera angle. The driver at the time of safety confirmation grasps the road condition based on the image displayed on the display. Therefore, the driver needs to display an easy-to-understand image according to the actual road condition. When the car stops at an angle, if the camera angle is changed inappropriately, left and right images that are completely different from the reality are displayed, which is not helpful to the driver.
[0005]
Moreover, since the above-described vehicle periphery visual recognition device requires a mechanism for changing the camera angle, various adverse effects caused by this, such as an increase in the number of parts and an increase in manufacturing cost, Alternatively, there is another problem of causing adverse effects such as a decrease in reliability due to a failure of the movable part.
The present invention has been made to solve such problems, and the object of the present invention is to eliminate the mechanism for changing the camera angle and to prevent roadblocks from occurring, and to express road conditions. An object of the present invention is to provide a vehicle periphery visual recognition device that automatically displays an appropriate image and allows a driver to check road conditions very easily and quickly.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the invention of claim 1 is arranged so as to capture the field of view of the vehicle in the vehicle width direction, and has a wide image with a margin in the horizontal direction with respect to the road area representing the situation of the intersecting road. An imaging unit in which a corner is set, a reference position specifying unit that specifies a reference position having a predetermined positional relationship with respect to a road area from an image captured by the imaging unit, and a reference specified by the reference position specifying unit Road area extraction means for extracting a road area image from the captured image based on the position, image display means for enlarging and displaying the road area image extracted by the road area extraction means, and road area extraction means when the road area extracted Te is protruding from the captured image, and shift amount determining means determines the amount of deviation between the captured image whether less than a predetermined threshold value, the shift amount using the shift amount determining means When judged to be less than the threshold A road area correcting means kept within the captured image by moving the road area, when the shift amount is determined to be above the threshold due to a shift amount determining means, which has a warning means for issuing a warning to the driver is there.
[0007]
Accordingly, the field of view of the vehicle in the vehicle width direction is imaged by the imaging means, a reference position having a predetermined positional relationship with respect to the road area is specified by the reference position specifying means from the captured image, and based on the specified reference position, An image of the road area is extracted from the captured image by the road area extraction means, and is enlarged and displayed by the image display means.
Since the road area is extracted from the captured image and enlarged and displayed in this way, the angle of view of the imaging means can be set wider than the angle of view for the driver to check the road condition, in other words, the intersection A wide field angle with a margin in the horizontal direction can be set with respect to the road area representing the situation of the road to be intersected, and the road area that intersects in the captured image even when the vehicle stops diagonally with respect to the intersecting road As a result, a mechanism for changing the camera angle can be eliminated, and an appropriate image representing a crossing road condition can be displayed. Therefore, for example, it is possible to capture an intersecting road on a modified T-shaped road, or an oncoming lane when turning right at an intersection, and the intersecting road or oncoming lane is extracted from the captured image as a road area to check the driver. It can be useful for.
In addition, when the extracted road area image protrudes from the captured image, the deviation amount is determined by the deviation amount determination means. When the deviation amount is less than a predetermined threshold, the road area image is moved. It is stored in the captured image. Therefore, even in such a case, the road area can be used for confirmation of the driver. Further, when the deviation amount is equal to or larger than the threshold value, a warning is issued to the driver by the warning means, and visual safety confirmation is prompted.
[0008]
In the invention of claim 2, the reference position specifying means extracts a plurality of linear components from the captured image, specifies a vanishing point where the extracted linear components intersect as a reference position, and the road area extracting means determines the vanishing point. A predetermined range of at least the lower, left, and right sides at the center is extracted from the captured image as a road region image .
A plurality of parallel linear component of the center line and road or the like constituting the road area intersect at one point in the image, certain correlation is established between the position and the road region of the vanishing point is a point that the concentration The road area is located slightly below the vanishing point . Therefore, if this vanishing point is regarded as a reference position and a predetermined range of at least the lower, left, and right sides centered on the vanishing point is extracted from the captured image as an image of the road region, an appropriate road region is automatically displayed. It becomes possible.
[0010]
In the invention of claim 3 , the imaging means images the left and right fields of view of the vehicle, specifies the reference position by the reference position specifying means, and the road area by the road area extraction means, based on the left and right captured images. Extraction and image enlargement display by the image display means are performed individually. Therefore, since each process is performed separately for the left and right fields of view, for example, even when the left and right road conditions are completely different, it is possible to extract and display an appropriate road area.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of a vehicle periphery visualizing device embodying the present invention will be described.
FIG. 1 is an overall configuration diagram showing a vehicle periphery visual recognition apparatus according to the present embodiment. An ECU (electronic control unit) 1 that performs overall control of the vehicle periphery visual recognition device is installed in a vehicle interior, and includes an input / output device (not shown), a storage device (ROM, RAM, etc.), a central processing unit (CPU), A timer counter and the like are provided. The input side of the ECU 1 is connected to a power switch 2 of the vehicle periphery visual recognition device and sensors / switches such as a pair of CCD cameras 3a and 3b (imaging means), and the output side of the ECU 1 is connected to an instrument panel of the vehicle. Devices such as a liquid crystal display 4 (image display means) and a speaker 5 installed on the top are connected.
[0012]
The CCD cameras 3a and 3b are built in the left and right front fenders of the vehicle, and respectively capture the left and right fields of view of the vehicle. The CCD cameras 3a and 3b of the present embodiment are provided with a wide-angle lens in the light receiving portion, and the angle of view is greatly enlarged compared to the angle of view of the CCD camera applied to the above-described conventional technology, for example.
[0013]
Hereinafter, the setting state of the angle of view of the CCD cameras 3a and 3b will be described. FIG. 2 is an explanatory diagram comparing the angle of view of the CCD camera and the angle of view of the road area extracted from the captured image as described later. . This type of peripheral visual recognition device displays images captured by the CCD cameras 3a and 3b on the display 4, but displays the situation of the road where the driver crosses (for example, the presence or absence of a traveling vehicle on the road). 14, it is necessary to display a corresponding area (hereinafter referred to as a road area) with a certain size. For this reason, in the prior art, the angle of view of the CCD camera is set to about 30 °, for example. This angle of view is substantially the same as the angle of view β of the extracted image described later, and is therefore quite narrow. If the car stops at a slight angle, the intersecting road area will be removed from the angle of view. Needless to say, such a case is assumed that the prior art includes a mechanism for changing the camera angle.
[0014]
On the other hand, in the present embodiment, the angle of view of the CCD cameras 3a and 3b is set to an extremely wide α, for example, about 90 °. Therefore, even when the vehicle stops obliquely on the modified T-shaped road shown in FIG. 2, a road area that intersects the angle of view can be captured without changing the camera angle. In this embodiment, a road area that intersects is extracted from the images picked up by the CCD cameras 3a and 3b in this way and is enlarged and displayed on the display 4. Hereinafter, details of the processing will be described.
[0015]
Here, FIG. 3 is an explanatory diagram illustrating an image captured by the right CCD camera, and FIG. 4 is an explanatory diagram illustrating an image of a road region extracted from the captured image. As shown in these drawings, in the captured image E1 on the right side of the own vehicle, a plurality of parallel straight lines (that is, a straight line component y described later) such as the center line L1 and the road side L2 of the intersecting road are included in the image. It has the characteristic of intersecting at one point. Hereinafter, this point is referred to as a vanishing point VP, and a certain correlation is established between the position of the vanishing point VP and the position of a traveling vehicle or the like on the road that requires confirmation (that is, the road region E2). It has been found that a traveling vehicle or the like is always located slightly below the vanishing point VP.
[0016]
When the vehicle reaches a T-junction or an intersection and the power switch 2 is turned on by the driver, the ECU 1 executes an image extraction / display routine shown in FIG. 5 at a predetermined control interval. This process is performed individually for the captured images E1 of the right CCD camera 3a and the left CCD camera 3b, and the contents are exactly the same, so only the right process will be described below. First, in step S2, a captured image E1 of the right CCD camera 3a is captured, and in step S4, a plurality of points whose luminance change is equal to or greater than a predetermined threshold (hereinafter referred to as luminance change points) are extracted from the captured image E1. In subsequent step S6, two or more linear components y = Aix + Bi and weight wi are obtained by Hough transform based on each extracted luminance change point. The straight line component y is the center line L1, the road side L2, etc. in the image, and the weight w i represents the credibility as a straight line. This Hough transform is a well-known method for specifying a linear component y in an image. A schematic procedure will be described below with reference to FIG.
[0017]
FIG. 6 is an explanatory diagram showing the principle when a linear component y is extracted by Hough transform on an image. In this figure, the image is represented as an xy plane, and the luminance change at three points on the linear component y indicated by a broken line. A point P is extracted (upper part of the figure). First, the Hough transform is applied to these luminance change points P. That is, assuming a straight line of all angles passing through each luminance change point P (one example in the figure), the orthogonal distance ρ and the angle θ from the origin (0, 0) are calculated for each straight line. The orthogonal distance ρ and the angle θ are collected for each luminance change point, and each luminance change point P is represented as a curve on the ρ-θ plane (middle of the figure).
[0018]
Next, the ρ-θ plane is divided into a large number of cells specified by ρ and θ, and the number of the curves located in each cell is voted to obtain a vote frequency distribution (lower part of the figure). Then, when the local maximum point (ρ, θ) is determined from the voting frequency distribution and a procedure (inverse Hough conversion) opposite to the above Hough transform is applied to the local maximum point (ρ, θ), it is indicated by a broken line in the upper part of the figure. The straight line component y is specified, and the number of votes at this time is set as the weight wi. In the figure, the coordinates (3, 7) at which the number of votes 3 is obtained are obtained as maximum points.
[0019]
In other words, the obtained local maximum point (ρ, θ) represents the coordinates at which the curves intersect in the middle part of the figure, and these coordinates are a common straight line having the same slope and intercept passing through the luminance change points P in the upper part of the figure. Represents a component (that is, a broken line component y). Therefore, if the local maximum value (ρ, θ) is subjected to inverse Hough transform, the linear component y that causes each luminance change point P can be obtained. The weight wi is set to a larger value based on the larger number of votes for the linear component y specified based on the large number of luminance change points P (in other words, the linear component with clear contrast as an image).
[0020]
After specifying two or more linear components y and weights wi in this way, the ECU 11 proceeds to step S8. In step S8, the intersection point pi (xi, yi) of the two linear components y and the weight wpi of each intersection point pi are obtained. In step S6, since two or more straight line components y are specified, the intersection points pi are obtained for all combinations of these straight line components y, while the weights w i of the two straight lines y that form the basis of each intersection point pi are obtained. By addition or multiplication, a weight wpi is obtained as a correlated value.
[0021]
The straight line component y obtained in step S6 may include a noise component, but such a straight line component y has a small weight wi, and the weight wpi of the intersection point pi based on the straight line component y is also the same. Inevitably small. FIG. 7 is an explanatory diagram showing the situation of specifying the intersection point pi based on the image shown in FIG. 3. In FIG. 7, three line components y are extracted and three intersection points pi are specified from each line component y. Shows the case.
[0022]
In the subsequent step S10, the center of gravity position is obtained based on each intersection pi and the weight wpi, and this center of gravity position is determined as the vanishing point VP (reference position specifying means). FIG. 8 is an explanatory diagram showing the specification of the vanishing point VP based on the intersection point pi shown in FIG. 7, and the vanishing point VP is specified from the three intersection points pi. In addition, since not only the position of each intersection point pi but also the weight wpi is considered, for example, if the intersection point pi having a large weight wpi exists at a position separated from other intersection points pi, the vanishing point VP inevitably has the weight wpi. Will approach the large intersection point pi.
[0023]
In a succeeding step S12, it is determined whether or not the intersecting road area E2 can be extracted from the captured image E1 of the CCD camera 3a. The road area E2 to be extracted is set in advance with reference to the vanishing point VP as shown in FIG. That is, as described with reference to FIG. 3, the traveling vehicles etc. on the intersecting road are always located slightly below the vanishing point VP. Therefore, the traveling vehicles etc. (classified as passenger cars, trucks, etc.) in the road area E2. 8), for example, as shown in FIG. 8, the range of L1 on the left and right, L2 on the upper side, and L3 (> L2) on the lower side is centered on the vanishing point VP. Set as E2. The actual L1 to L3 are set as the number of pixels, and the extraction process is also performed based on the number of pixels.
[0024]
When the determination in step S12 is YES (positive), the process proceeds to step S14 to extract the road area E2 from the captured image E1 (road area extraction means), and the road area E2 extracted in the subsequent step S16 is displayed on the display 4. (Image display means) and the routine is terminated. The ECU 1 repeats the above processing, and each time the road area E2 is sequentially extracted based on the new captured image E1, the intersecting road condition is displayed on the display 4 as a pseudo moving image in real time.
[0025]
As a result, the road region E2 is extracted at an angle of view corresponding to β from the image E1 captured at the angle of view α of the right CCD camera 3a shown in FIG. 2 (the same applies to the left side), as shown in FIG. Compared with the captured image E1, the road area E2 is enlarged and displayed as shown in FIG. 4, and the road situation where the driver intersects can be easily grasped.
On the other hand, FIG. 9 is an explanatory diagram showing a case where a road region protrudes from a captured image and the amount of shift is small. When the determination in step S12 is NO (No), that is, when the road region E2 is set at the normal position with the vanishing point VP as a reference, when the image protrudes from the captured image E1 as indicated by a hatched frame in the figure. The process proceeds to step S18. In step S18, it is determined whether or not the shift amount of the road area E2 with respect to the captured image E1 is large (deviation amount determination means) . For example, since the position of the road region E2 correlates with the position of the vanishing point VP, the distance Lvp of the vanishing point VP with respect to the captured image E1 is calculated, and when the distance Lvp is less than a predetermined threshold, NO is determined in step S18. And go to step S20.
[0026]
In step S20, the road area E2 is corrected, and the road area E2 is moved from the normal position by the shortest distance and is stored in the captured image E1 as indicated by a two-dot chain line (road area correction means) . Thereafter, the corrected road area E2 is extracted in step S14, enlarged and displayed on the display 4 in step S16, and the routine is terminated.
When the amount of deviation of the road area E2 with respect to the captured image E1 is small, the travel distance of the road area E2 at the time of correction is short, so that a traveling vehicle or the like located below the vanishing point VP is within the road area E2 even after correction. It can be captured and used for grasping the road condition of the driver.
[0027]
On the other hand, when the determination in step S18 is YES, that is, when the deviation amount of the road area E2 is large, it is estimated that the important traveling vehicle or the like cannot be captured in the road area E2 even if the correction is performed. In step S22, a warning is issued to the driver through the display 4 and the speaker 5 (warning means) . This warning alerts the driver and performs a visual safety check.
[0028]
In parallel with the above processing for the right CCD camera 3a, the same processing is performed for the left CCD camera 3b in step S24.
As described above, in the vehicle periphery visual recognition device according to the present embodiment, by expanding the angle of view of the CCD cameras 3a and 3b, even if the own vehicle stops diagonally with respect to the intersecting road, it intersects in the captured image E1. The road area E2 to be captured can be captured, and the intersecting road area E2 is extracted from the captured image E1 based on the vanishing point VP, and then enlarged to a size that can be grasped by the driver. . Accordingly, the mechanism for changing the camera angle provided in the prior art can be eliminated to prevent various adverse effects such as weight increase, and an appropriate image (road region E2) representing the road condition intersecting can be obtained. It can be displayed automatically, allowing the driver to check the road conditions very easily and quickly.
[0029]
Further, the use of the wide-angle lens greatly expands the angle of view of the CCD cameras 3a and 3b so as to capture a wide area, so that it can be used not only for the modified T-shaped road but also for various driving situations. When turning right at an intersection, a straight oncoming vehicle that is blocked by a right turn oncoming vehicle and difficult to see can be captured in the image and displayed on the display 4 in an enlarged manner. Therefore, the available driving situation can be greatly expanded.
[0030]
Furthermore, since the vanishing point VP is specified, the road region E2 is extracted, and the enlarged image is displayed separately based on the captured images E1 of the left and right CCD cameras 3a and 3b, the left and right road conditions are completely different. Even in such a case, it is possible to always extract and display an accurate road region E2 respectively, thereby prompting the driver to make an appropriate determination.
This is the end of the description of the embodiment, but the aspect of the present invention is not limited to this embodiment. For example, in the above-described embodiment, the CCD cameras 3a and 3b are provided in the front part (front fender) of the vehicle and used for checking road conditions when the vehicle reaches an intersection or the like during normal traveling. For example, the CCD camera 3a , 3b may be provided in the rear part of the vehicle and used for confirmation when going back from the garage or parking lot.
[0031]
In the above embodiment, the CCD cameras 3a and 3b are built in the front fender. However, the installation location is not limited to this. For example, the CCD cameras 3a and 3b may be built in a front bumper, a side under mirror, or the like. Instead, for example, as shown in FIG. 10, the left and right images may be taken into a common CCD camera 12 (imaging means) via a mirror 11 arranged in a V shape. Needless to say, the CCD camera 12 in this case also has a wide angle of view by adopting a wide-angle lens, and can capture a wide area. On the other hand, when taking an image through the mirror 11 as described above, the substantial angle of view of the CCD camera 12 may be enlarged by forming the mirror 11 in a convex shape instead of the wide-angle lens.
[0032]
Furthermore, in the above embodiment, the left and right CCD cameras 3a and 3b are individually subjected to image analysis to extract the road area E2. For example, when the intersecting road is a straight road, the left and right road areas are extracted. Since E2 is located on the opposite side of 180 °, the road area E2 in the other captured image E1 can be specified geometrically based on the position of the road area E2 extracted in one captured image E1. Therefore, the road area E2 may be extracted in this way for the other road area E2.
[0033]
On the other hand, in the above embodiment, image display on the display 4 is started in response to the driver's operation of the power switch 2, but for example, the vehicle position is determined based on GPS information obtained from the navigation device, and from the map database. Image display may be started automatically when it is determined that the vehicle has reached a T-junction or an intersection after obtaining the road conditions around the vehicle position.
[0034]
Moreover, in the said embodiment, although the road area | region E2 was extracted based on the vanishing point VP, it is also possible to perform based on the element correlated with the vanishing point VP. For example, if the position of the vanishing point VP changes, the road direction, that is, the slope or intercept of the straight line component y changes, so that instead of the process based on the vanishing point VP, the straight line component y is based on the slope or intercept. The road area E2 may be extracted.
[0035]
【The invention's effect】
As described above, according to the vehicle periphery visual recognition device of the first and second aspects of the invention, since the angle of view of the image pickup means is set wide, the road area is extracted from the picked-up image, and is enlarged and displayed. The mechanism to change the angle is abolished to prevent harmful effects, and the right and left images showing the road conditions are automatically displayed to make the driver check the road conditions very easily and quickly. Can do.
[0036]
According to the vehicle periphery visual recognition device of the invention of claim 3 , in addition to the invention of claim 1 or 2 , since each process is separately performed for the left and right fields of view, an accurate road area is always displayed. Thus, it is possible to prompt the driver to make an appropriate judgment.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram showing a vehicle periphery visual recognition apparatus according to an embodiment.
FIG. 2 is an explanatory diagram comparing the angle of view of a CCD camera on a modified T-shaped road and the angle of view of a road area extracted from a captured image.
FIG. 3 is an explanatory diagram showing an image captured by the right CCD camera.
FIG. 4 is an explanatory diagram showing an image of a road area extracted from a captured image.
FIG. 5 is a flowchart showing an image extraction / display routine executed by the ECU.
FIG. 6 is an explanatory diagram showing the principle when a linear component y is extracted by Hough transform on an image.
7 is an explanatory diagram showing a specific situation of an intersection point pi based on the image of FIG. 3. FIG.
FIG. 8 is an explanatory diagram showing a road area extraction status for a vanishing point VP;
FIG. 9 is an explanatory diagram illustrating a correction situation when a road region protrudes from a captured image.
FIG. 10 is a diagram showing another example of a CCD camera using a mirror.
[Explanation of symbols]
1 ECU (reference position specifying means, road area extracting means, image display means)
3a, 3b, 12 CCD camera 4 Display (image display means)
E1 Road area E2 Imaging means VP Vanishing point (reference position)

Claims (3)

An imaging means that is arranged to image the field of view of the vehicle in the vehicle width direction and has a wide angle of view with a margin in the horizontal direction with respect to a road area that represents the condition of the intersecting road;
Reference position specifying means for specifying a reference position having a predetermined positional relationship with respect to a road area from an image picked up by the image pickup means;
Road area extracting means for extracting an image of the road area from the captured image based on the reference position specified by the reference position specifying means;
Image display means for enlarging and displaying an image of the road area extracted by the road area extraction means;
When the road area extracted by the road region extracting means is protruding from the captured image, and shift amount determining means determines the amount of deviation between the captured image whether less than a predetermined threshold,
When the shift amount is determined to be less than the threshold value by the shift amount determining means, and the road area correcting means moves the road area kept within the captured image,
A vehicle periphery visualizing device comprising: warning means for issuing a warning to a driver when the deviation amount is determined to be greater than or equal to a threshold value by the deviation amount determination means.   The reference position specifying unit extracts a plurality of linear components from the captured image, specifies a vanishing point where the extracted linear components intersect as the reference position, and the road region extracting unit centers on the vanishing point. The vehicle periphery visual recognition device according to claim 1, wherein at least a predetermined range of the lower side, the left side, and the right side is extracted from the captured image as an image of the road region. The imaging means images the left and right fields of view of the vehicle, and based on the left and right captured images, specifies the reference position by the reference position specifying means, extracts the road area by the road area extracting means, 3. The vehicle periphery visualizing device according to claim 1, wherein the enlarged display of the image by the image display means is performed individually.

Images