JPH10341430A - Detection of front vehicle and system for monitoring front of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize high real time processing performance and to reduce calculation quantity by photographing a left picture and a right picture at the same time by means of synchronizing right and left cameras and generating a composite stereo video signal of both pictures. SOLUTION: A stereo moving picture acquirement device 40 collects an even scanning line signal in a left video signal 41La from the left camera 41L and an odd scanning line signal in a right video signal 41Ra from the right camera 41R in synchronizing with a switch signal 43b. The odd scanning line signal in the left video signal 41La is set to be the even scanning line signal in a synthesis video signal 44a and the odd scanning line signal in the right video signal 41Ra is set to be the odd scanning line signal in the synthesis video signal 44a so as to generate the composite video signal 44a. When all the pictures obtained from the left camera 41L are in the same height as a road, the picture equal to the picture taken by the right camera 41R can be generated if all picture elements in the picture are moved to a scanning direction by the value of a visual difference.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a front vehicle detection method and a vehicle front monitoring system to which a stereo image processing technique is applied, and more particularly, to a left image collected using a left camera and a right image collected using a right camera. The present invention relates to a front vehicle detection method and a front vehicle monitoring system that performs stereo image processing using a right image to measure a distance between a vehicle traveling in front of the host vehicle and an own vehicle, and generates an alarm when there is a danger.

[0002]

2. Description of the Related Art The number of people who have obtained a driver's license has increased year by year, and more than half of the entire population now holds a driver's license. Also, the performance of automobiles has been dramatically improved, and the road traffic network has been greatly developed.

[0003] For these reasons, the frequency of use of automobiles has become extremely high, and their applications are daily commuting, leisure on holidays, and driving of automobiles themselves are leisure activities. For this reason, it has become indispensable for Japanese people to hinder daily life without a car. Furthermore, in the present age, the range of drivers is expanding, and as seen in the increase of women and elderly drivers, everyone is driving the car casually and frequently.

However, driving a car requires
Just because it's familiar to everyone doesn't mean you're happy with it. As the number of drivers increases, an unfortunate by-product of increasing traffic accidents has emerged.

In the last few years, about 10,000 people have been lost due to traffic accidents. This is because although the performance of recent automobiles has been improved, safety is still largely dependent on the skill of the driver. A novice driver is often distracted by the fact that he or she drives the car normally, causing distractions around him and causing accidents. It is common to hear accidents caused by drowsy driving.

[0006] However, such an accident is unlikely to occur if a passenger is present and pays close attention to the surroundings like the driver. However, it is difficult to always take care of the passengers, even if they are present, and the instructions of the passengers are not always accurate. At times, it may lead to dangerous directions. Therefore, there is an increasing expectation that a driving assistance system that monitors the vicinity of the vehicle besides the driver and warns the driver of a dangerous situation or the like will be put to practical use.

In recent years, research and development of driving assistance systems have been conducted in various fields from such a background. There are a wide variety of types, including a front vehicle detection method that recognizes the environment ahead of the vehicle in order to prevent rear-end collisions and collisions, and a vehicle detection method that recognizes the road environment so that an automobile normally travels in its own lane. More recently, the development of unmanned vehicles has become more of a reality than a dream story like in the past.

[0008] Among them, a system put into practical use is a method for detecting a forward vehicle using a laser radar.
This forward vehicle detection method irradiates a laser from a running vehicle toward a forward vehicle, bounces off an obstacle, and returns a distance D from the time until the vehicle returns to the own vehicle to the forward vehicle.
Is to measure.

However, the laser radar has a drawback that the range that can be monitored is narrow and the information that can be obtained is very small due to the relationship between the thickness of the laser and the like. Furthermore, when traveling on a curve, the irradiated laser does not return to the host vehicle even when it hits the preceding vehicle, so that it has a disadvantage that it functions effectively only when traveling straight.

[0010] For this reason, recently, a forward vehicle detecting method and a forward vehicle monitoring system for capturing an image ahead of the own vehicle by a camera installed in the own vehicle and performing forward monitoring using the image, that is, a road image, A method for detecting a forward vehicle and a vehicle forward monitoring system by processing are disclosed.

The method for detecting a forward vehicle and the system for monitoring a vehicle ahead by such road image processing have the advantage that information in multiple directions can be obtained and the monitoring area can be widened.

[0012] However, while what is desired to be recognized, such as the driving environment, is a three-dimensional environment, when only one camera is used, the obtained image is generally two-dimensional, and the depth information is lost. There was a technical problem of doing it.

In order to solve such a technical problem,
As a forward vehicle detection method and a forward vehicle monitoring system for acquiring three-dimensional information by applying a stereo image processing technology, a binocular stereoscopic technology using a plurality of images from different viewpoints has been disclosed. (In the following description, the first
Let's call it prior art).

The first prior art vehicle detection method and the vehicle front monitoring system can obtain three-dimensional information by calculating the parallax of the pixel of interest between the left and right paired images. It is widely known as a forward vehicle detection method and a forward vehicle monitoring system for distance measurement in image processing.

However, in the first prior art vehicle detection method and vehicle front monitoring system of the prior art for performing binocular stereopsis using a plurality of images from different viewpoints, the parallax of the pixel of interest is obtained by the current image processing technology. However, there is a technical problem that complicated processing is required. Further, in the case of road images, there is a technical problem that a great deal of processing time is required to calculate parallax for information such as road surface display that does not need to be avoided. Therefore, in general, the amount of calculation is large and real-time processing (processing that enables instantaneous and instantaneous output of a detection result) or vehicle periphery monitoring that requires a high-speed processing time close thereto (particularly vehicle front monitoring). There was also a technical problem that people tended to be shunned.

On the other hand, as a conventional technique for solving such a problem that the amount of calculation is large, for example, Japanese Patent Laid-Open No. 7-2
No. 50319 (Title of Invention: Vehicle Perimeter Monitoring Apparatus, Filing Date: March 14, 1994, Applicant: Yazaki Sogyo Co., Ltd.) discloses a forward vehicle detecting method and a vehicle forward monitoring system. In the following description, it will be referred to as a second prior art).

That is, the vehicle surroundings monitoring device of the second prior art is similar to the first prior art in that the periphery of the vehicle is monitored based on image data obtained by two photographing means. One memory for recording a projected image created assuming a height of zero based on the one image signal output from the other memory, the other memory for recording the projected image, and the image data of one memory and the other memory On the other hand, a road surface image removing means for removing an image on the road surface due to a difference from the image data,
Object edge detecting means for detecting the edge of the object from the horizontal differential value of the other memory and the image data output from the road surface image removing means, and alarm means for outputting an alarm based on the calculated position. Was.

The left video signal and the right video signal from the two photographing means (cameras) are respectively recorded in the two frame memories, and the contents recorded in the frame memories are taken into the road surface image removing means, and the collision and the like are performed. When the danger is detected, the driver is warned of the danger by the warning means. Divide the imaging range into about 512 pixels x 512 pixels,
A frame memory that records the luminance of one pixel in about 8 bits has been generally used.

[0019]

However, according to the first or second prior art vehicle detection method and the front vehicle monitoring system, the left and right video signals from two CCD cameras are recorded. It is also necessary to prepare two frame memories. When two frame memories are used, there is a technical problem that it is difficult to claim superiority in terms of manufacturing cost, installation space, and the like.

Although a solution to the problem of real-time processing in the first prior art is disclosed in the second prior art, higher real-time processing performance and a reduction in calculation amount are required, and many research results have been obtained. It has been considered that there is still room for research, and the development of a forward vehicle detection method and a vehicle forward monitoring system equipped with an algorithm that can easily perform real-time processing according to the purpose has been an issue.

An object of the present invention is to solve such a conventional problem. First, a stereo image is obtained by using a left image collected by using a left camera and a right image collected by using a right camera. In the forward vehicle detection method that measures the distance between the vehicle running in front and the own vehicle by performing image processing of the formula and issues an alarm when dangerous, the left and right cameras are synchronized at the same time with the left image and the left image. Capturing a right image and creating a composite video signal of the left image and the right image, and a stereo video acquisition step including a step of alternately scanning the left video signal and the right video signal to form one frame. Recording the composed image signal into a single frame memory; reading out the composite image signal recorded in the frame memory; and reading out the odd-numbered scanning line signals constituting one frame of the read composite image signal. Running A left / right image separation acquisition step of separating line signals to create right image information and separating even-numbered scanning line signals in a scanning line signal to create left image information, and forming a horizontal edge and a vertical edge in the left image information. Following the step of extracting and extracting and separating and extracting the horizontal and vertical edges in the right image information and the preceding step, the thinning of the edges is performed on the horizontal and vertical edges extracted in the left image information. An edge image creating step of creating a left edge image and thinning edges on horizontal and vertical edges extracted in the right image information to create a right edge image, and estimating the left edge image from the right edge image Estimated edge image creation process to be performed,
A step of performing a difference operation on a horizontal edge between the estimated left edge image and the left edge image to generate a difference horizontal edge, and a step of performing a difference operation on a vertical edge between the estimated left edge image and the left edge image to generate a difference vertical edge Generating a differential edge image by synthesizing the differential horizontal edge and the differential vertical edge, and performing a differential operation between the estimated left edge image and the left edge image. A front vehicle extraction step including a step of calculating the center of gravity of the edge point, and a step of creating a window including a predetermined ratio of all edges centered on the calculated center of gravity,
Setting a window in the left current image, which is an image captured by the left camera, setting a corresponding area on the right current image, which is an image captured by the right camera; A distance calculating step including a step of calculating a parallax between the window in the current image and a step of calculating a distance between the vehicle traveling ahead and the host vehicle based on the calculated parallax; A step of generating an alarm signal when it is determined that a dangerous state is determined based on the distance, and a step of generating an alarm by receiving the alarm signal based on the calculated distance. Two CCD cameras are detected by a method of detecting a preceding vehicle using a binocular road image in front of the own vehicle obtained by using two cameras and calculating a distance and a relative speed to the preceding vehicle at a high speed. Left video signal from & right Can record image signals on a common frame memory, as a result, has an object to provide a preceding vehicle detection method can claim a point in superiority of manufacturing cost and installation space.

Further, it is an object of the present invention to provide a forward vehicle detecting method capable of realizing high real-time processing performance and reducing the amount of calculation, and claiming an advantage in having an algorithm which can easily perform real-time processing according to the purpose of use. It is an object.

Second, stereo image processing is performed using the left image collected using the left camera and the right image collected using the right camera, and the distance between the vehicle traveling ahead and the host vehicle is determined. A left and right camera and a left camera that are installed at the same height from the road surface on the vehicle with the optical axis installed parallel to the traveling direction vector of the own vehicle in a vehicle front monitoring system that measures And synchronizing the left camera and the right camera based on the synchronizing signal and a synchronizing signal generator for generating a synchronizing signal for instructing a timing of photographing in the right camera and generating a switching signal synchronized with at least 1/2 frame time. Means for photographing the left image and the right image at the same time to create a composite image signal of the left image and the right image. Even scan line signal in the video signal /
An odd scan line signal / even scan line signal and an odd scan line signal / even scan line signal in a right video signal from the right camera are collected, and an even scan line signal / odd scan line signal / even scan line signal of a left video signal is collected. Even scan line signal /
An odd scanning line signal / even scanning line signal and an odd scanning line signal / even scanning line signal of the right video signal are combined as an odd scanning line signal / even scanning line signal in the composite video signal to generate a composite video signal. A single frame memory for recording a synthesized image signal in which one frame is composed by synthesizing a left video signal and a right video signal alternately with scanning lines, and a frame memory. The recorded composite image signal is read out, odd-numbered scanning line signals in the scanning line signals constituting one frame of the read composite image signal are separated to create right image information, and even-numbered scanning line signals are formed. Left and right image separation and acquisition means for generating left image information by separating scanning line signals, and monitoring areas for front vehicle detection for the right image information and the left image information are respectively provided. A monitoring area setting means for extracting and extracting horizontal and vertical edges in the left image information and separating and extracting the horizontal and vertical edges in the right image information and extracting the horizontal and vertical edges in the left image information The left edge image is created by thinning the edge with respect to the horizontal edge and the vertical edge extracted in the right image information, and the right edge image is created by thinning the edge. A front vehicle extracting means for creating a difference image between the left edge image and the estimated edge image independently from the horizontal edge and the vertical edge, and combining these to create a difference edge image; A window is set on the left current image, which is a shot image to be shot, and the window is set to correspond to the right current image, which is a shot image shot by the right camera. Distance calculation means for calculating a distance between a vehicle running ahead and the host vehicle based on the calculated parallax between the window in the left current image and the window in the right current image by setting an area; Determining means for determining whether or not the vehicle is in a dangerous state based on the calculated distance and generating an alarm signal when the vehicle is determined to be in a dangerous state; and an alarm for receiving a warning signal and generating an alarm based on the calculated distance. Means for detecting a preceding vehicle using a binocular road image in front of the own vehicle obtained by using two cameras and calculating a distance to the preceding vehicle and a relative speed at a high speed. The left video signal and the right video signal from the two CCD cameras can be recorded in a common frame memory, and as a result, the superiority in terms of manufacturing cost and installation space can be claimed. Monitoring system The purpose is to provide a system.

Further, it is an object of the present invention to provide a vehicle forward monitoring system capable of realizing high real-time processing performance and reducing the amount of calculation and claiming an advantage in that it has an algorithm which can easily perform real-time processing according to the purpose of use. It is an object.

[0025]

According to the first aspect of the present invention, stereo image processing is performed by using a left image collected by using the left camera 41L and a right image collected by using the right camera 41R. In a forward vehicle detection method that measures a distance between a vehicle traveling ahead and the own vehicle 20 and generates an alarm when there is a danger, the left and right cameras are synchronized to photograph a left image and a right image at the same time. And a step of generating a combined video signal 44a of the left image and the right image.

According to the first aspect of the present invention, a stereo moving image acquiring step is provided, the left and right cameras are synchronized to photograph the left image and the right image at the same time, and the left image and the right image are combined. The video signal 44a is created. As a result, two CCs
Since the left video signal 41La and the right video signal 41Ra output from the D camera are synchronized with each other, they can be synthesized by a synthesized video creation process. The synchronizing signal 43a allows the left camera 41L and the right camera 41R to perform an imaging operation at the same time, and a large error in the detection distance can occur even in a situation where a falling object is detected while traveling at a speed of 100 km / h or more. This makes it possible to achieve high detection accuracy by synchronizing the imaging timing. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a second aspect of the present invention, in the method for detecting a forward vehicle according to the first aspect, each of the left camera 41L and the right camera 41R includes an odd scan line signal / even scan line signal and an even scan line signal / odd number. When a scanning line signal / even scanning line signal is output, the stereo moving image acquiring step includes a predetermined scanning line signal in the left video signal 41La from the left camera 41L and a predetermined scanning line in the right video signal 41Ra from the right camera 41R. A method for detecting a preceding vehicle, comprising the step of collecting signals and generating the composite video signal 44a.

According to the invention described in claim 2, according to claim 1
In addition to the effects described in the above, a stereo moving image acquisition step is provided to collect and combine a predetermined scanning line signal in the left video signal 41La from the left camera 41L and a predetermined scanning line signal in the right video signal 41Ra from the right camera 41R. The video signal 44a is created. As a result, the amount of memory for recording captured images can be set smaller than in the past, and the vehicle forward monitoring system 10 using stereo image processing technology having high detection accuracy and high-speed processing performance can be realized. As a result, it is possible to claim advantages in terms of manufacturing cost and installation space. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a third aspect of the present invention, in the method for detecting a front vehicle according to the second aspect, the stereo moving image acquiring step includes the step of: obtaining an even-numbered scanning line signal / odd-numbered scanning line in the left video signal 41La from the left camera 41L. Signals / even scanning line signals and odd scanning line signals / even scanning line signals in the right video signal 41Ra from the right camera 41R are collected, and the even scanning line signal / odd scanning line signal / even scanning line of the left video signal 41La are collected. The signals are used as the even scan line signal / odd scan line signal / even scan line signal in the composite video signal 44a, and the odd scan line signal / even scan line signal of the right video signal 41Ra is used as the odd scan line signal in the composite video signal 44a. This is a method for detecting a preceding vehicle, which includes a step of creating a combined video signal 44a by combining the signals as a signal / even scanning line signal.

According to the invention described in claim 3, according to claim 2
In addition to the effects described in the above, a stereo moving image acquisition step is provided to provide an even scanning line signal / odd scanning line signal / even scanning line signal in the left video signal 41La from the left camera 41L and a right video signal 41Ra from the right camera 41R. An odd scan line signal / even scan line signal is collected, and an even scan line signal / odd scan line signal / even scan line signal of the left video signal 41La is converted into an even scan line signal / odd scan line signal / An even-numbered scan line signal is used, and the odd-numbered scan line signal / even-numbered scan line signal of the right video signal 41Ra is further combined into a composite video signal 44a.
Are combined as an odd scanning line signal / even scanning line signal to create a composite video signal 44a. As a result, the amount of memory of the frame memory 45 for recording a captured image can be set to be less than the sum of the number of even-numbered scanning lines and the number of odd-numbered scanning lines. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle.
This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory 45.

According to a fourth aspect of the present invention, in the method for detecting a forward vehicle according to the second aspect, the odd scan line signal / even scan line signal and the even scan line signal / odd scan line signal are set every 1/2 frame time. When the left camera 41L and the right camera 41R operate in the interlaced mode in which the signal is output separately from the / even scan line signal, the stereo moving image acquiring step includes a left video signal 41La from the left camera 41L and a right video signal from the right camera 41R. This is a method for detecting a preceding vehicle, which comprises a step of alternately collecting and collecting a video signal 41Ra every フ レ ー ム frame time to generate the composite video signal 44a.

According to the invention described in claim 4, according to claim 2,
In addition to the effects described in the above, a stereo moving image acquisition step is provided to provide an even scanning line signal / odd scanning line signal / even scanning line signal in the left video signal 41La from the left camera 41L and a right video signal 41Ra from the right camera 41R. An odd scan line signal / even scan line signal is collected, and an even scan line signal / odd scan line signal / even scan line signal of the left video signal 41La is converted into an even scan line signal / odd scan line signal / An even-numbered scan line signal is used, and the odd-numbered scan line signal / even-numbered scan line signal of the right video signal 41Ra is further combined into a composite video signal 44a.
Are combined as an odd scanning line signal / even scanning line signal to create a composite video signal 44a. As a result, the amount of memory of the frame memory 45 for recording a captured image can be set to be smaller than the number of even-numbered scanning lines for one-half frame and the number of odd-numbered scanning lines for one-half frame. 45 can be reduced to about half of the conventional capacity. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory 45.

According to a fifth aspect of the present invention, in the method for detecting a forward vehicle according to the fourth aspect, the composite video signal 44a is provided.
Uses the odd-numbered scanning line signal / even-numbered scanning line signal of the right video signal 41Ra for 1/2 frame of the composite video signal 44a, and the even-numbered scanning signal of the left video signal 41La. 1/2 which is a scanning line signal / odd scanning line signal / even scanning line signal
The remaining one frame in the composite video signal 44a is
This is a method for detecting a preceding vehicle, which comprises a step of combining the combined video signal 44a using image data for / 2 frames.

According to the invention described in claim 5, according to claim 4,
In addition to the effects described in (1), a stereo moving image acquisition step is provided, and a half frame (odd scanning line signal / even scanning line signal) in the right video signal 41Ra is converted into half frame image data in the composite video signal 44a. And the 1/2 video frame signal of the even-numbered scan line signal / odd-numbered scan line signal / even-numbered scan line signal in the left video signal 41La is used as image data for the remaining 1/2 frame in the composite video signal 44a. The video signal 44a is created. As a result, the amount of memory of the frame memory 45 for recording a captured image can be set to be smaller than the number of even-numbered scanning lines for one-half frame and the number of odd-numbered scanning lines for one-half frame. 45 can be reduced to about half of the conventional capacity. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory 45.

The invention according to claim 6 is the invention according to claims 3 to 5
In the forward vehicle detection method according to any one of the above, the left video signal 41La and the right video signal 4
This is a method for detecting a preceding vehicle, comprising the step of recording the combined image signal composed of one frame by combining with 1Ra in a single frame memory 45.

According to the invention of claim 6, according to claim 3,
In addition to the effects described in any one of (1) to (5), by providing such a step, the composite video signal 44a for a frame is recorded in the single frame memory 45. As a result, the frame memory 45 can be shared, and the amount of memory of the frame memory 45 for recording a captured image can be reduced to the number of even-numbered scanning lines for 1/2 frame and the number of odd-numbered scanning lines for 1/2 frame. , And the capacity of the frame memory 45 can be reduced to about half that of the related art. Further, even if the frame memory 45 is shared, the parallax required for calculating the distance is in the horizontal direction (scanning line direction), so that the detection resolution is not inferior to the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory 45 due to the sharing of the frame memory 45.

According to a seventh aspect of the present invention, in the method for detecting a forward vehicle according to the sixth aspect, the frame memory 45 is provided.
And reads out the combined image signal recorded in the same, separates the odd-numbered scanning line signals in the scanning line signals constituting one frame of the read combined image signal to create right image information 31a, and This is a front vehicle detection method including a left / right image separation / acquisition step of generating left image information 31b by separating even-numbered scanning line signals in a scanning line signal.

According to the invention of claim 7, according to claim 6,
In addition to the effects described in the above, by separating the signals into the even-numbered scanning line signals and the odd-numbered scanning line signals recorded in the shared frame memory 45, There is an effect that a pair of binocular images can be obtained as a moving image by using two pieces of the right image information 31a and the left image information 31b having a resolution twice as large as the direction. Further, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the right image information 31a and the left image information 3
The detection resolution of 1b is not inferior to the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory 45 due to the sharing of the frame memory 45.

According to an eighth aspect of the present invention, in the method for detecting a forward vehicle according to the seventh aspect, a monitoring area 32a for detecting the forward vehicle 21 is set for each of the right image information 31a and the left image information 31b. Is a method for detecting a preceding vehicle.

According to the invention described in claim 8, according to claim 7,
In addition to the effects described in (1), by providing the monitoring area setting step, these video signals can be eliminated in advance, and the monitoring area 32a including the preceding vehicle 21 can be set. When the right camera 41R or the left camera 41L captures an image of the front of the host vehicle 20 using the right camera 41R or the left camera 41L, a video signal (image information) of a sky, a tree, a building, or the like, which is a video signal that does not need to be avoided and does not require distance measurement, is left video signal 41La. And the right video signal 41Ra. As a result, the processing time can be reduced by eliminating video signals that do not need to be avoided and do not need to be measured, and leaving only video signals (image information) that need to be avoided and need to be measured. It works.

Following the previous step, a step of creating an estimated left edge image / estimated right edge image from the right edge image / left edge image, and following the previous step, the left and right edges are respectively independent of the horizontal and vertical edges. A method for detecting a front vehicle, comprising: a step of creating a difference image between an edge image / right edge image and an estimated edge image, and combining these to create a difference edge image. According to the ninth aspect of the present invention, stereo image processing is performed using left image information 31b collected using the left camera 41L and right image information 31a collected using the right camera 41R, and the vehicle travels forward. In a forward vehicle detection method for measuring a distance between a vehicle and the own vehicle 20 and generating an alarm in a dangerous case, a horizontal edge and a vertical edge are separated and extracted from the left image information 31b, and the right image information 31a is extracted. Separating and extracting horizontal and vertical edges at
Subsequent to the previous process, the horizontal edge and the vertical edge extracted in the left image information 31b are thinned to create a left edge image 33AL, and the horizontal edge extracted in the right image information 31a. A step of creating a right edge image 33AR by thinning the edge with respect to the vertical edge, and following the previous step, an estimated left edge image 33EL / estimated right edge image from the right edge image 33AR / left edge image 33AL. Following the step of creating the 33ER and the preceding step, the left edge image 33AL / the right edge image 33AR independently of the horizontal edge and the vertical edge.
And a difference image 33D of the estimated edge image 33E, and combining these to create a difference edge image 334c.

According to the ninth aspect of the present invention, by providing a forward vehicle extracting step, an estimated edge image 33E artificially created by using image processing and a left edge image 33AL actually captured are obtained. Is performed, only the road surface 34 on the same plane is deleted, and the preceding vehicle 21 at a higher place can be left. Further, more edges to be measured having a height from the road surface 34 can be left more than before. Furthermore, when the present forward vehicle detection method is applied to the entire screen, a considerable difference appears in the amount of extracted edges in the forward vehicle 21 area. As a result, it is possible to calculate the center of gravity of the edge in the post-processing, create the window 336a including the 80% of all edges centered on the center of gravity, and further improve the estimation accuracy when estimating it as the front vehicle 21 area. It has the effect of being able to do so. As a result, it is possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a tenth aspect of the present invention, in the forward vehicle detecting method according to the ninth aspect, the forward vehicle extracting step separates and extracts a horizontal edge and a vertical edge from the left image information 31b. The right image information 31
a, the horizontal edge and the vertical edge are separated and extracted, and, following the previous step, the horizontal edge and the vertical edge extracted in the left image information 31b are thinned to obtain a left edge image 33AL. Front vehicle detection comprising an edge image creation step of creating a right edge image 33AR by performing edge thinning on the horizontal and vertical edges extracted in the right image information 31a. Is the way.

According to the tenth aspect of the present invention, in addition to the effect of the ninth aspect, by providing an edge image creating step, the object to be measured having a height from the road surface 34 (the forward vehicle) 21) is left as much as possible by processing the difference image 33D, and the edge on the road surface 34 can be deleted as much as possible by processing the difference image 33D. That is, the road surface 34
, And the overlap portion between the horizontal edge and the vertical edge, and the overlap portion between the vertical edge and the horizontal edge, which are originally desired to be left, can be processed with the difference image 33D and left with higher accuracy than before. As a result, a considerable difference appears in the extracted edge amount of the front vehicle 21 region. As a result, the center of gravity of the edge can be calculated in the post-processing, and the window 336a including 80% of all edges centered on the center of gravity can be created to improve the estimation accuracy when estimating it as the area of the front vehicle 21. become. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to an eleventh aspect of the present invention, in the method for detecting a forward vehicle according to the tenth aspect, the step of extracting the forward vehicle includes the step of extracting the right edge image 33AR from the left edge image 33AR.
This is a forward vehicle detection method including an estimated edge image creation step of estimating AL.

According to the eleventh aspect of the present invention, in addition to the effect of the tenth aspect, by providing an estimated edge image creating step, the left image information 31b and the right image information 31 are provided.
a, an edge image is extracted by separating a horizontal edge and a vertical edge, and an edge image with a thinned edge is created, and then the estimated edge image 3 is obtained from the right edge image 33AR.
3E, the estimated edge image 33E and the left edge image 3
3AL difference horizontal edge 334a, difference vertical edge 334b
Are created separately for the horizontal edge and the vertical edge, and thereafter, the difference horizontal edge 334a and the difference vertical edge 334b are combined to generate the difference horizontal edge 334a,
The difference vertical edge 334b can be obtained.
In the estimated edge image 33E, only the road surface 34 at the same height from the road surface 34 is actually the same image as the left edge image 33AL captured by the left camera 41L, and is located at a position higher than the road surface 34 such as the preceding vehicle 21. An object can be prevented from being an image equal to the actual left edge image 33AL. Left edge image 33 actually captured
When the common part with the estimated edge image 33E is deleted from the AL, the road surface 34 (the road surface 34 of the traveling lane) and the road surface 34 sign (for example, white line or speed limit display) at the same height are deleted from the left edge image 33AL. Only the pixels of the front vehicle 21 existing at a position higher than the road surface 34 can be left in the left edge image 33AL, and finally, the pixels remaining in the left edge image 33AL are
Has an effect that the distance can be calculated as a candidate pixel of.

According to a twelfth aspect of the present invention, in the front vehicle detecting method according to the eleventh aspect, the front vehicle extracting step calculates a difference between a horizontal edge of the estimated left edge image 33EL and a horizontal edge of the left edge image 33AL. To create a difference horizontal edge 334a, and perform a difference operation on the vertical edge between the estimated left edge image 33EL and the left edge image 33AL to create a difference vertical edge 334b. Composing the difference vertical edge 334b to create a difference edge image 334c.

According to the twelfth aspect of the invention, in addition to the effect of the eleventh aspect, by providing a step of creating the differential edge image 334c, both the left image information 31b and the right image information 31a are provided. After creating an edge image by separating and extracting the horizontal edge and the vertical edge, creating an edge image with a thinned edge, creating an estimated edge image 33E from the right edge image 33AR, the estimated edge image 33E and the left edge image The difference horizontal edge 334a and the difference vertical edge 334b of the 33AL are created separately for the horizontal edge and the vertical edge, and then the difference horizontal edge 334a and the difference vertical edge 334b are combined to create the difference horizontal edge 334a and the difference vertical edge 334c of the difference edge image 334c. The edge 334b can be used. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a thirteenth aspect of the present invention, in the method for detecting a forward vehicle according to the twelfth aspect, the forward vehicle extracting step includes the step of calculating the difference between the estimated left edge image 33EL and the left edge image 33AL. Horizontal edge 3
34a and a step of calculating a center of gravity of the edge point of the residual image in which the difference vertical edge 334b has been created, and a step of creating a window 336a including a predetermined ratio of all edges around the calculated center of gravity. This is a method for detecting a vehicle ahead.

According to the thirteenth aspect of the present invention, in addition to the effect of the twelfth aspect, by providing a step of forming the window 336a, the distance to be measured having a height from the road surface is increased. It is possible to leave more edges than before.
Further, the extracted edge amount in the area of the front vehicle 21 can be made significantly different. According to this, the center of gravity of the edge is calculated in the post-processing, and a window 336a including 80% of all edges is created around the center of gravity, and the window 336a is created.
This has the effect of improving the estimation accuracy when estimating a region. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a fourteenth aspect of the present invention, when the estimated edge image 33E has all the image information of one camera
4. An image obtained by estimating an edge image in a case where the edge image according to the image information is photographed by the other camera, assuming the pattern on the image information is the upper pattern.
3. A method for detecting a forward vehicle according to claim 3.

According to the fourteenth aspect of the present invention, the same effect as any one of the ninth to thirteenth aspects can be obtained.

According to a fifteenth aspect of the present invention, in the method for detecting a forward vehicle according to any one of the eleventh to fourteenth aspects, the window 336a is added to a left current image which is an image captured by the left camera 41L. Setting process and right camera 4
Setting a region corresponding to the right current image, which is an image captured by 1R, calculating a disparity between the window 336a in the left current image and the window 336a in the right current image; A forward vehicle detection method configured to include a distance calculation step including a step of calculating a distance between a vehicle traveling ahead and the host vehicle 20 based on the calculated parallax.

According to the fifteenth aspect of the present invention, in addition to the effect of any one of the eleventh to fourteenth aspects, by providing a distance calculation step, the window 3 in the left current image is provided.
Find the parallax between 36a and the window 336a in the right current image,
From the value of the parallax, the absolute distance to the object to be measured can be obtained with high accuracy and at high speed. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a sixteenth aspect of the present invention, in the method for detecting a forward vehicle according to the fifteenth aspect, the distance calculating step uses a relational expression calculated in advance based on a proportional relationship between the disparity and the reciprocal of the distance. Window 33 in the left current image
It is configured to calculate the parallax between the window 6a and the window 336a in the right current image, and calculate the distance based on the calculated parallax.

According to the sixteenth aspect of the present invention, in addition to the effect of the fifteenth aspect, by providing a distance calculating step, the window 336a in the left current image is formed by using a relational expression calculated in advance. Since the distance can be calculated by calculating the parallax between the window and the window 336a in the right current image, the absolute distance from the parallax value to the object to be measured can be obtained with high accuracy and high speed. . This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a seventeenth aspect of the present invention, in the method for detecting a forward vehicle according to the fifteenth or sixteenth aspect, it is determined whether or not the vehicle is in a dangerous state based on the calculated distance. This is a forward vehicle detection method configured to include a step of generating an alarm signal 36a.

According to the seventeenth aspect of the present invention, in addition to the effect of the fifteenth or sixteenth aspect, the alarm signal 36a
By providing the step of creating a danger, it is possible to determine at high speed and with high accuracy whether or not there is a dangerous state based on the distance calculated with high speed and with high accuracy. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to an eighteenth aspect of the present invention, in the method for detecting a forward vehicle according to the seventeenth aspect, a step of receiving the warning signal 36a and generating a warning based on the calculated distance is provided. This is a method for detecting a forward vehicle.

According to the eighteenth aspect of the invention, in addition to the effect of the seventeenth aspect, by providing a step of generating an alarm, a high-speed and high-precision alarm can be generated. become. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the nineteenth aspect, the left camera 41
When the distance between the vehicle traveling ahead and the host vehicle 20 is measured by performing stereo image processing using the left image collected using L and the right image collected using the right camera 41R, and is dangerous. In the vehicle front monitoring system that generates an alarm, the optical axis is installed in parallel with the traveling direction vector of the own vehicle 20, and the left and right cameras provided at the same height from the road surface 34 on the vehicle, the left camera 41L and Synchronization signal 43a for instructing the timing of photographing in right camera 41R
Signal generator 4 for generating a switching signal 43b synchronized with at least 1/2 frame time.
3 and the left camera 41L and the right camera 41R are synchronized based on the synchronization signal 43a to capture the left image and the right image at the same time to create a composite video signal 44a of the left image and the right image. The vehicle front monitoring system 10 includes a stereo moving image acquisition device 40 including a creation unit 44.

According to the nineteenth aspect, in addition to the effect of the first aspect, the left camera 41L (the right camera 4
When all the images obtained from the camera 1R) are at the same height as the road surface 34, when all the pixels in the image are moved in the scanning line direction by the parallax value, the image is captured by the right camera 41R (the left camera 41L). An image equivalent to an image can be created.

According to a twentieth aspect of the present invention, in the vehicle front monitoring system 10 according to the nineteenth aspect, the left camera 4
When each of the 1L and the right camera 41R outputs an odd scan line signal / even scan line signal and an even scan line signal / odd scan line signal / even scan line signal, the stereo moving image acquisition device 40 outputs the left camera 41L. The predetermined scanning line signal in the left video signal 41La from the camera and the right video signal 4 from the right camera 41R
This is a vehicle front monitoring system 10 configured to collect a predetermined scanning line signal at 1Ra and create the composite video signal 44a.

According to the twentieth aspect, the same effect as the second aspect can be obtained.

According to a twenty-first aspect of the present invention, in the vehicle front monitoring system 10 according to the twentieth aspect, the stereo moving image acquisition device 40 synchronizes with the switching signal 43b to output a left image from the left camera 41L. An even scan line signal / odd scan line signal / even scan line signal in the signal 41La and an odd scan line signal / even scan line signal in the right video signal 41Ra from the right camera 41R are collected, and the left video signal 41La is even scanned. The line signal / odd scan line signal / even scan line signal is used as the even scan line signal / odd scan line signal / even scan line signal in the composite video signal 44a, and the odd scan line signal / even scan line of the right video signal 41Ra. The signal is composed as an odd-numbered scanning line signal / even-numbered scanning line signal in the composite video signal 44a to generate the composite video signal 44a. This is a vehicle front monitoring system 10.

According to the twenty-first aspect, the same effect as the third aspect is obtained.

According to a twenty-second aspect of the present invention, in the vehicle front monitoring system according to the twenty-first aspect, the stereo moving image acquisition device 40 comprises an odd scanning line signal / even scanning line signal every 1/2 frame time. When the left camera 41L and the right camera 41R operate in the interlace mode in which the scan signal and the even scan line signal / odd scan line signal / even scan line signal are separately output, the switching signal 4 related to the interlace is output.
3b, in synchronization with the left video signal 41L from the left camera 41L.
a and the right video signal 41Ra from the right camera 41R,
The vehicle front monitoring system 10 is configured to alternately collect and generate the composite video signal 44a every two frame times.

According to the twenty-second aspect, the same effect as the fourth aspect can be obtained.

According to a twenty-third aspect of the present invention, in the vehicle forward monitoring system 10 according to the twenty-second aspect, the stereo moving image acquisition device 40 transmits the switching signal 43b.
In synchronism with the above, a half frame of the odd scanning line signal / even scanning line signal in the right video signal 41Ra is used as image data of a half frame in the composite video signal 44a, and the left video signal The composite video signal 44a is synthesized using 1/2 frame of the even scan line signal / odd scan line signal / even scan line signal in 41La as image data of the remaining 1/2 frame in the composite video signal 44a. 1 is a vehicle front monitoring system 10 configured to perform the following.

According to the twenty-third aspect of the present invention, the same effect as that of the fifth aspect is obtained.

According to a twenty-fourth aspect of the present invention, a vehicle front monitoring system 1 according to any one of the twenty-first to twenty-third aspects.
0, a single frame memory 4 for recording the synthesized image signal in which one frame is composed by synthesizing the left video signal 41La and the right video signal 41Ra alternately with scanning lines.
5 is a vehicle front monitoring system 10 configured to include:

According to the twenty-fourth aspect, the same effect as the sixth aspect can be obtained.

According to a twenty-fifth aspect of the present invention, in the vehicle front monitoring system according to the twenty-fourth aspect, the composite image signal recorded in the frame memory 45 is read, and the read composite image signal of the read composite image signal is read. The right image information 31a is created by separating the odd-numbered scanning line signals in the scanning line signals forming one frame, and the left image information 31b is created by separating the even-numbered scanning line signals in the scanning line signal. The left and right image separation acquisition means 31 and the right vehicle information 21a and the left vehicle information 31b
The vehicle front monitoring system 10 includes a data processing device 30 including a monitoring area setting unit 32 that sets a monitoring area 32a for detection.

According to the twenty-fifth aspect, the same effect as the seventh aspect can be obtained.

According to a twenty-sixth aspect of the present invention, in the vehicle front monitoring system 10 according to the twenty-fifth aspect, the data processing device 30 separates and extracts a horizontal edge and a vertical edge from the left image information 31b. The horizontal and vertical edges are separated and extracted in the right image information 31a, and the horizontal and vertical edges extracted in the left image information 31b are thinned to form a left edge image 33AL. At the same time, the horizontal edge and the vertical edge extracted in the right image information 31a are thinned to form the right edge image 3
3AR, and an estimated left edge image 33EL as an edge image 33E from the right edge image 33AR.
It is provided with a front vehicle extracting means 33 which creates a difference image 33D of the left edge image 33AL and the estimated edge image 33E independently of the horizontal edge and the vertical edge, and combines them to create a difference edge image 334c. Is a vehicle forward monitoring system 10 that is operating.

According to the twenty-sixth aspect, the same effect as the ninth aspect can be obtained.

According to a twenty-seventh aspect of the present invention, in the vehicle front monitoring system 10 according to the twenty-fifth aspect, the front vehicle extracting means 33 separates and extracts a horizontal edge and a vertical edge from the left image information 31b. At the same time, a horizontal edge and a vertical edge are separated and extracted in the right image information 31a, and the horizontal and vertical edges extracted in the left image information 31b are thinned to create a left edge image 33AL. At the same time, the horizontal image and the vertical edge extracted in the right image information 31a are thinned to create the right edge image 33AR to generate the estimated left edge image 33EL / estimated right edge image. The creation unit 332, the difference between the estimated left edge image 33EL and the horizontal edge of the left edge image 33AL with respect to the horizontal edge The difference horizontal edge 3 carries out an operation
34a, and the estimated left edge image 33EL.
/ Estimated right edge image 33ER and the left edge image 33AL
A difference edge creating unit that creates a difference vertical edge 334b by performing a difference operation on a vertical edge with the right edge image 33AR, and combines the difference horizontal edge 334a and the difference vertical edge 334b to create a difference edge image 334c. 3
34, the estimated left edge image 33EL / estimated right edge image 33ER, and the left edge image 33AL / right edge image 3
In the difference calculation with 3AR, the difference horizontal edge 334a
A window creation unit 33 that calculates the center of gravity of the edge point of the residual image for which the difference vertical edge 334b has been created, and creates a window 336a that includes a predetermined ratio of all edges centered on the calculated center of gravity.
6 and a vehicle front monitoring system 10 having
It is.

According to the twenty-seventh aspect, the same effects as those of any one of the eleventh to thirteenth aspects can be obtained.

According to a twenty-eighth aspect of the present invention, in the vehicle front monitoring system 10 according to the twenty-sixth or twenty-seventh aspect,
The data processing device 30 sets the window 336a in the left current image, which is an image captured by the left camera 41L, and sets an area corresponding to the right current image, which is an image captured by the right camera 41R. Then, a parallax between the window 336a in the left current image and the window 336a in the right current image is calculated, and based on the calculated parallax, the distance between the vehicle traveling ahead and the host vehicle 20 is calculated. It is a vehicle front monitoring system 10 configured to have a distance calculating means 35 for calculating.

According to the twenty-eighth aspect of the invention, the same effects as those of the fifteenth or sixteenth aspects are obtained.

According to a twenty-ninth aspect of the present invention, in the vehicle front monitoring system according to the twenty-eighth aspect, the data processing device 30 determines whether or not the vehicle is in a dangerous state based on the calculated distance. The vehicle front monitoring system 10 is configured to include a determination unit 36 that generates an alarm signal 36a when it is determined that the vehicle is in the state.

According to the twenty-ninth aspect, the same effect as that of the seventeenth aspect can be obtained.

According to a thirtieth aspect of the present invention, in the vehicle front monitoring system according to the twenty-ninth aspect, the data processing device receives an alarm signal 36a based on the calculated distance and issues an alarm. This is a vehicle front monitoring system 10 configured to include an alarm unit 37 that generates an alarm.

According to the thirtieth aspect, the same effect as the twentieth aspect can be obtained.

[0085]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

The vehicle front monitoring system 10 of the present embodiment
Performs stereo image processing using the left image collected using the left camera 41L and the right image collected using the right camera 41R to measure the distance between the vehicle traveling ahead and the host vehicle 20. It has a function of generating an alarm when there is a danger, and is configured mainly by the data processing device 30 shown in FIG. 1, the stereo moving image acquisition device 40 shown in FIG. 2, and a single frame memory 45.

FIG. 1 shows a vehicle front monitoring system 1 according to the present invention.
FIG. 2 is a functional block diagram for explaining an embodiment of the data processing device 30 constituting 0.

The data processing device 30 shown in FIG. 1 is composed mainly of left / right image separation / acquisition means 31, monitoring area setting means 32, front vehicle extraction means 33, distance calculation means 35, determination means 36, and alarm means 37. Have been.

The left and right image separation and acquisition means 31 shown in FIG.
A left / right image separation / acquisition step, which will be described later, is executed. A combined image signal recorded in the frame memory 45 is read, and an odd-numbered scan in a scanning line signal constituting one frame of the read combined image signal is performed. At the same time as generating the right image information 31a by separating the line signals, it has a function of separating the even-numbered scanning line signals in the scanning line signals to create the left image information 31b.

The monitoring area setting means 32 shown in FIG. 1 executes a monitoring area setting step which will be described later. The monitoring area setting means 32 performs the following processing on the right image information 31a and the left image information 31b.
It is desirable to have a function of setting each of the monitoring areas 32a related to detection, and to be configured mainly by a microcomputer.

The forward vehicle extracting means 33 shown in FIG. 1 executes a forward vehicle extracting step to be described later, and separates and extracts horizontal and vertical edges from the left image information 31b and simultaneously extracts the right image information 31a. , The horizontal and vertical edges are separated and extracted, and the left image information 31
b, the left edge image 33AL is created by thinning the horizontal and vertical edges extracted at the same time, and at the same time, the thinning of the edges is performed on the horizontal and vertical edges extracted from the right image information 31a. To create the right edge image 33AR,
From the estimated left edge image 33E as the edge image 33E
L, a difference image 33D of the left edge image 33AL and the estimated edge image 33E is generated independently of the horizontal edge and the vertical edge, and these are combined to obtain a difference edge image 3
It has a function of creating 34c, and is mainly configured by an edge image creating unit 332, a difference edge creating unit 334, and a window creating unit 336, and is executed by the aforementioned microprocessor.

The edge image creating section 332 shown in FIG. 1 executes an edge image creating step, which will be described later, and separates and extracts horizontal edges and vertical edges from the left image information 31b and simultaneously extracts the right image information 31a. , The horizontal and vertical edges are separated and extracted, and the horizontal and vertical edges extracted in the left image information 31b are thinned to form a left edge image 33AL. Edge thinning is performed on the extracted horizontal edge and vertical edge to create a right edge image 33AR and an edge image 33AR.
It has a function of creating an estimated left edge image 33EL as E, and is executed by the aforementioned microprocessor.

The difference edge creation section 334 shown in FIG. 1 executes a step of creating a difference edge image 334c, which will be described later, and performs difference calculation on the horizontal edge between the estimated left edge image 33EL and the left edge image 33AL. To create a difference horizontal edge 334a, and at the same time, perform a difference operation on the vertical edge between the estimated left edge image 33EL and the left edge image 33AL to create a difference vertical edge 334b.
It has a function of creating a difference edge image 334c by combining the difference horizontal edge 334a and the difference vertical edge 334b, and is executed by the aforementioned microprocessor.

The window creation unit 336 shown in FIG. 1 executes a process of creating a window 336a, which will be described later. In the difference calculation between the estimated left edge image 33EL and the left edge image 33AL, the difference horizontal edge 334a and the difference horizontal edge 334a are calculated. Vertical edge 33
It has the function of calculating the center of gravity of the edge point of the residual image for which 4b has been created, and creating a window 336a containing a predetermined percentage of all edges centered on the calculated center of gravity, and is executed by the aforementioned microprocessor.

The distance calculating means 35 shown in FIG. 1 executes a distance calculating step which will be described later.
Window 336a on the left current image
Is set, a corresponding area is set on the right current image, which is an image captured by the right camera 41R, and the disparity between the window 336a in the left current image and the window 336a in the right current image is calculated. It has a function of calculating the distance between the vehicle traveling ahead and the host vehicle 20 based on the parallax thus performed, and is executed by the above-described microprocessor.

The judging means 36 shown in FIG. 1 has a function of judging whether or not the vehicle is in a dangerous state based on the calculated distance, and generating an alarm signal 36a when it is determined that the vehicle is in a dangerous state. Running on

The alarm means 37 shown in FIG. 1 has a function of receiving an alarm signal 36a based on the calculated distance and generating an alarm, and is preferably configured around an alarm buzzer.

FIG. 2 shows a vehicle front monitoring system 1 according to the present invention.
FIG. 2 is a functional block diagram for explaining an embodiment of a stereo moving image acquisition device 40 that constitutes a stereo moving image acquisition device 0.

The stereo moving picture acquisition device 40 shown in FIG.
Performs a stereo moving image acquisition step described later. As described later with reference to FIG. 5, in synchronization with the switching signal 43b, the odd scanning line signal 1/1 in the right video signal 41Ra (video signal) is used. Synthesized video signal 4 for 2 frames
4a (video signal) is used as image data for 1/2 frame, and at the same time, 1/2 frame, which is an even-numbered scanning line signal in left video signal 41La (video signal), is used as the remaining 1/2 in composite video signal 44a. It has a function of synthesizing a composite video signal 44a using image data for a frame, and has a left camera 41L and a right camera 41R, a synchronization signal generator 43, and a composite video creating means 44.

The left camera 41L and the right camera 41R have the function of setting the optical axis of the camera in parallel with the traveling direction vector of the vehicle 20 and at the same height from the road surface 34 on the vehicle. Specifically, it is desirable to use a CCD camera.

The synchronizing signal generator 43 generates a synchronizing signal 43a for instructing the left camera 41L and the right camera 41R to shoot a scene centered on the vehicle in front, and at the same time, generates a synchronizing signal 43a in the interlaced scanning system.
It has a function of generating a switching signal 43b synchronized with two frame times (1/60 second).

The composite video creating means 44 is provided with a synchronizing signal 43a.
Has the function of synchronizing the left camera 41L and the right camera 41R on the basis of the above, photographing the left image and the right image at the same time, and creating a composite video signal 44a of the left image and the right image, Running on

[0103] Such a stereo moving image acquisition apparatus 40
Collects an even-numbered scanning line signal in the left video signal 41La from the left camera 41L and an odd-numbered scanning line signal in the right video signal 41Ra from the right camera 41R in synchronization with the switching signal 43b, and The scanning line signal is used as the even-numbered scanning line signal in the combined video signal 44a, and at the same time, the odd-numbered scanning line signal of the right video signal 41Ra is combined with the combined video signal 4a.
The composite video signal 44a is formed by combining the odd-numbered scan line signals in 4a.

As described above, according to the stereo moving image acquisition apparatus 40, if all the images obtained from the left camera 41L (right camera 41R) are at the same height as the road surface 34, Is moved in the scanning line direction by the value of parallax, the right camera 41R (the left camera 41L)
Thus, it is possible to create an image equivalent to the captured image.

A single frame memory 45 shown in FIGS.
Are the left video signal 41La and the right video signal 41
It has a function of recording a composite image signal composed of one frame by being combined with Ra.

Next, an embodiment of a forward vehicle detection method executed by using the above-described forward vehicle monitoring system 10 will be described with reference to the drawings.

When the optical axes of the two cameras are set parallel to the traveling direction vector, the road surface 34 is always at the same height from each camera. Therefore, in the left and right images captured by the two cameras, the parallax P of the left and right road surfaces 34 is calculated for each scanning line. When the images obtained from one camera are all at the same height as the road surface 34, if all the pixels in the image are moved by the value of the parallax P for each scanning line, the image is picked up by the other camera. It is possible to create an image that is equivalent to the image that has been set.

However, if pixels of the front vehicle 21 and the like located higher than the road surface 34 are moved by the same front vehicle detection method, the image will not be equivalent to an image captured by another camera.

In this manner, by comparing the artificially created estimated image with the actually picked-up image, only the road surface 34 on the same plane is deleted, and the forward vehicle located at a higher place than the road surface 34 is deleted. 21 can be left.

According to this principle, using the binocular road image of a running vehicle obtained from two cameras, a road surface 34 and a road surface 34 such as a road surface display that need not be avoided among the pixels in the image are used in advance. The processing time required for calculating the distance D is reduced by erasing pixels at the same height.

The purpose of the present embodiment is to detect the preceding vehicle 21 by erasing the road surface 34 from the binocular road image ahead of the own vehicle 20 and to calculate the distance D and the relative speed v at high speed. The purpose is to develop a method for detecting the vehicle ahead.

The feature of the present forward vehicle detection method is that by eliminating the road surface 34 which does not need to be avoided in advance, the processing time of the forward vehicle detection method conventionally using the binocular road image is problematic. There is a place to shorten. For this purpose, the camera uses the condition that the traveling direction vector of the host vehicle 20 and the optical axis are parallel to each other and that the camera is always located at the same height from the road surface 34.

FIG. 3 is a flowchart for explaining an embodiment of a forward vehicle detecting method executed by the forward vehicle monitoring system 10.

The front vehicle detection method shown in FIG. 3 performs stereo image processing using the left image collected by using the left camera 41L and the right image collected by using the right camera 41R, and travels forward. An algorithm that measures the distance between the vehicle and the vehicle 20 and generates an alarm when there is a danger,
It mainly comprises a stereo moving image acquisition step, a left and right image separation acquisition step, an estimated edge image creation step, a front vehicle extraction step, a distance calculation step, a determination step, and an alarm generation step.

[0115] A method for detecting a forward vehicle that performs binocular stereoscopic vision using images obtained from two cameras and calculates the distance D will be described below. In addition, the front vehicle detection method of the binocular image data in the present embodiment, and the front vehicle that calculates the distance D to the front vehicle 21 and the relative speed v between the front vehicle 21 and the host vehicle 20 using the binocular stereoscopic method The detection method will be described.

First, the outline of the binocular stereoscopic method will be described.

An image picked up by a camera generally has only two-dimensional information, and distance information that is three-dimensional information from the camera to the object to be measured has been lost. However, the environment in which a car travels has a three-dimensional spread, and the distance D from the preceding vehicle 21 is very important information in order to avoid a rear-end collision.

Therefore, as a method for detecting a preceding vehicle from distance information by image processing, there is a binocular stereoscopic method using two images acquired from different directions.

The present vehicle detection method uses such a binocular stereoscopic method, and calculates the distance D from the value of the parallax P by obtaining the parallax P of the point of interest between the two images. This is a method for detecting a preceding vehicle, and it is possible to obtain an absolute distance D to a target to be measured.

FIG. 4 is a principle diagram for explaining a binocular stereoscopic method used in the forward vehicle detection method and the forward vehicle monitoring system 10.

First, the principle of binocular stereopsis will be described. As shown in FIG. 4, two cameras having a focal length f set apart by a distance r capture an image of a point P located at a distance D from a straight line L connecting the two cameras. Here, the optical axes of the cameras are both perpendicular to the straight line L.

At this time, the distance from the position of the image of the point P in the right camera to the position of the image in the left camera is r + x, and the parallax P is x. In the conventional method for detecting a forward vehicle,
As can be seen from the figure, the distance D to the point P is obtained from x as D = r · f / x (1) (FIG. 4).

However, in the binocular stereopsis,
The actual distance to the preceding vehicle 21 and the calculated distance greatly differ due to a slight displacement of the camera. Therefore, in the present forward vehicle detection method, an image of the distance measurement target that is actually placed every 10 m from 10 m to 50 m ahead is taken, the parallax P of each distance measurement target is detected, and the parallax P and the distance are calculated. The relational expression is calculated.

That is, if the object to be measured is an image captured by the same system, the actual distance can be accurately calculated from the relational expression and the parallax P.

As described above, it is possible to obtain distance information of a point of interest in an image from two images having only two-dimensional image information.

This forward vehicle detection method is similar to a method in which a human perceives the depth of the object to be measured by the difference in how the left and right eyes look.

However, in this forward vehicle detection method, there is a problem that the point of interest on each of the two images may not actually correspond to the same point P.

That is, the projection of the point of interest onto the arbitrary image plane from the three-dimensional information is uniquely determined, but the reverse is not unique.

That is, when any feature of the preceding vehicle 21 is not extracted, the left image information 41L is displayed over the entire image.
And the right image information 41R will be matched,
Processing becomes very complicated. Therefore, 3
It is common to see that the preceding vehicle detection method for acquiring dimensional information requires a great deal of processing time.

However, it is ideal for the driving assistance system to acquire driving information such as distance in real time. Therefore, it is required that processing such as distance calculation is always performed at a high speed according to the driving speed. Therefore, in order to monitor the surroundings of the vehicle using the binocular stereoscopic method, it is necessary to shorten the processing time by speeding up a corresponding point search or the like.

In this embodiment, the binocular stereoscopic method is used from the viewpoint of calculating the distance D to the vehicle 21 ahead.
In addition, the road surface display that does not need to be avoided is deleted, and only the image of the preceding vehicle 21 is left, thereby speeding up the corresponding point search and shortening the processing time, which is a conventional problem. Subsequently, a stereo moving image acquiring step will be described.

When monitoring the vicinity of a vehicle by image processing, it is desirable that the acquired image is a moving image. This is because the driving environment of a car changes over time, and this must be recognized.
Further, in order to perform binocular stereoscopic vision, two images at the same time are required. However, at present, it is very difficult to capture two images at the same time using the home frame memory 45.

Therefore, in the stereo moving image acquisition step, the left and right cameras are synchronized and the left and right images are photographed at the same time to create a composite video signal 44a of the left and right images (step S1). )have.

According to such a program code, half of the odd-numbered scanning line signal in the right video signal 41Ra is used as image data for one-half frame in the composite video signal 44a, and at the same time, the left video signal is used. 41La
Are used as image data for the remaining 1/2 frame of the combined video signal 44a to create the combined video signal 44a. As a result, the amount of memory of the frame memory 45 for recording a captured image can be set to be smaller than the number of even-numbered scanning lines for one-half frame and the number of odd-numbered scanning lines for one-half frame. 45 can be reduced to about half of the conventional capacity. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory 45.

Further, the stereo moving image acquisition step is performed when the left camera 41L and the right camera 41R operate in the interlaced mode in which the odd scan line signal and the even scan line signal are output separately every half frame time. Left video signal 41La from 41L and right video signal 4 from right camera 41R
1Ra and a program code (Step S2) for alternately collecting and collecting 1Ra every フ レ ー ム frame time to create a composite video signal 44a.

In the stereo moving image acquisition step, a synchronizing signal 43a is generated from the synchronizing signal
The data is input to the CD cameras 41L and 41R, and the two CCD cameras 41L and 41R are synchronized (step S1). In addition, a switching signal 43b is provided to the composite image creating means 44 every 1/60 second (step S2).

At this time, the two CCD cameras 41L, 4
Left video signal 41La and right video signal 41R output from 1R
Since “a” is synchronized with each other, they can be combined by the combined video creating means 44 (step S2).

According to such a program code, an even-numbered scanning line signal in the left video signal 41La from the left camera 41L and an odd-numbered scanning line signal in the right video signal 41Ra from the right camera 41R are collected. The even scanning line signal of 41La is used as the even scanning line signal in the combined video signal 44a, and the odd scanning line signal of the right video signal 41Ra is combined as the odd scanning line signal in the combined video signal 44a to create the combined video signal 44a. ing. As a result, the amount of memory of the frame memory 45 for recording a captured image can be set to be smaller than the number of even-numbered scanning lines for one-half frame and the number of odd-numbered scanning lines for one-half frame. 45 can be reduced to about half of the conventional capacity. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory 45.

[0139] The stereo moving image acquiring step is a program code for recording a combined image signal in which one frame is composed by combining the left image signal 41La and the right image signal 41Ra alternately in a scanning line in a single frame memory 45. (Step S
3).

The synthesized image creating means 4 obtained in this way
4 is output to the frame memory 45 (step S
3).

According to such a program code, the composite video signal 44a for a frame is stored in a single frame memory 4
Recorded at 5. As a result, the frame memory 45 can be shared, and the amount of memory of the frame memory 45 for recording a captured image can be reduced to the number of even-numbered scanning lines for 1/2 frame and the number of odd-numbered scanning lines for 1/2 frame. , And the capacity of the frame memory 45 can be reduced to about half that of the related art. Further, even if the frame memory 45 is shared, the parallax required for calculating the distance is in the horizontal direction (scanning line direction), so that the detection resolution is not inferior to the conventional method for detecting a forward vehicle.
This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory 45 due to the sharing of the frame memory 45.

Following the reading of the composite image signal recorded in the frame memory 45 (step S4), the left and right image separation / acquisition step is performed in the odd-numbered scanning line signals constituting one frame of the read composite image signal. At the same time as generating the right image information 31a by separating the scanning line signals, a program code for generating the left image information 31b by separating the even-numbered scanning line signals in the scanning line signals (step S
5).

According to such a program code, by separating the signals into even-numbered scanning line signals and odd-numbered scanning line signals recorded in the shared frame memory 45, There is an effect that a set of binocular images can be obtained as a moving image by using two pieces of right image information 31a and left image information 31b having a resolution twice as high in the horizontal and vertical directions. Further, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution of the right image information 31a and the left image information 31b is not inferior to that of the conventional method of detecting a forward vehicle. According to this,
It is possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory 45 due to the sharing of the frame memory 45.

FIG. 5 shows a composite video signal 44a composed of a right video signal 41Ra and a left video signal 41La used in the forward vehicle detection method and the vehicle forward monitoring system 10.
FIG. 3 is a diagram for explaining the image form of FIG.

The switching signal 4 in the composite image creating means 44
Each time 3b switches, two CCD cameras 41L, 4
Since the left video signal 41La and the right video signal 41Ra from 1R are alternately output to the frame memory 45 (step S
4), as shown in FIG. 5, the output image becomes an image in which the left image information 41L and the right image information 41R appear alternately on the scanning lines (step S5).

By separating this image into those of even-numbered scanning lines and those of odd-numbered scanning lines in the processing in the data processing device 30, a right image having twice the resolution in the horizontal and vertical directions is obtained. 2 of information 31a and left image information 31b
One set of binocular images can be obtained as a moving image in one sheet (step S2).

FIG. 6 shows an image pickup state of the left camera 41L and the right camera 41R (specifically, a CCD camera).
FIG. 6A is a diagram for explaining an interlace mode in which an odd-numbered scanning line signal and an even-numbered scanning line signal are separately output every 1/2 frame time, and FIG. 6B is a diagram for explaining the interlacing mode.
FIG. 6 is a diagram for describing a frame structure of a right video signal 41Ra and a left video signal 41La in the interlace mode of FIG.

As shown in FIG. 6A, the captured image is output to the outside as a left video signal 41La and a right video signal 41Ra for every 525 scanning lines.

As a method of outputting to the outside, an interlace mode is often used. In the interlaced mode, as shown in FIG.
Every 60 seconds), the odd scanning line signal and the even scanning line signal are separately output. Therefore, the left video signal 41La and the right video signal 4La of one screen (one frame) in one frame time (1/30 second)
1Ra will be output.

Next, a stereo moving image acquisition step (steps S1, S2, S3) and a left and right image separation acquisition step (step S1)
A description will be given using the acquisition of the binocular road image in 5).

FIG. 7 is a block diagram of the stereo moving image acquisition device 4 shown in FIG.
At 0, a left camera 41L and a right camera 41R having an optical axis that coincides with the traveling direction vector when the vehicle 20 travels straight and are parallel to the road surface 34 are attached to the ceiling of the vehicle, and the binocular road is attached. FIG. 9 is a diagram for explaining how to acquire an image.

As shown in FIG. 7, a binocular road image is obtained by using the stereo moving image obtaining apparatus 40 described in the previous section.

First, two CCD cameras 41L and 41R are mounted on the ceiling of the vehicle 20. The optical axes of the two CCD cameras 41L and 41R are set so as to be parallel to the road surface 34. The optical axis is set so as to coincide with the traveling direction vector when the vehicle 20 goes straight.
The vehicle travels on the highway and the image of the road ahead traveling is recorded in the frame memory 45 as a moving image (that is, the left video signal 41La and the right video signal 41Ra) (step S1 → 2 →).
3).

The composite video signal 44a (moving image) obtained by the above-described preceding vehicle detection method is continuously taken into the data processing device 30 via the frame memory 45 (step S4). The captured composite video signal 44a is converted into an even-numbered image of a scanning line and an odd-numbered composite video signal 44a.
(Step S)
5). Thus, the right image information 31a and the left image information 31b are obtained (step S5).

Finally, the present preceding vehicle detection method is applied to the obtained right image information 31a and left image information 31b. 2.3 Setting of the monitoring area 32a in the monitoring area setting step (step S6) will be described.

When the right camera 41R or the left camera 41L captures an image of the front of the host vehicle 20, the captured left video signal 4
In general, 1La and the right video signal 41Ra include many video signals (image information) that do not need to be avoided, such as the sky, trees, and buildings, and do not require distance measurement.

In the conventional method for detecting a forward vehicle, the parallax P is calculated even for a video signal which does not need to be avoided and does not require distance measurement.
, A great deal of processing time was required to calculate these parallaxes P.

[0158] Such video signals that do not need to be measured and do not need to be measured are considered to be present at almost the same place in any image when they are captured by the same system. Therefore, in this embodiment, as shown in FIG. The monitoring area 32a that excludes these video signals and includes the preceding vehicle 21 is set in advance (step S6).

The step of setting the monitoring area 32a has a program code (step S6) for detecting the front vehicle 21 for the right image information 31a and the left image information 31b.

According to such a program code, it becomes possible to eliminate these video signals in advance and set a monitoring area 32a including the preceding vehicle 21. When the right camera 41R or the left camera 41L captures an image of the front of the host vehicle 20 using the right camera 41R or the left camera 41L, a video signal (image information) of a sky, a tree, a building, or the like, which is a video signal that does not need to be avoided and does not require distance measurement, is left video signal 41La. And the right video signal 41Ra. As a result, the processing time can be reduced by eliminating video signals that do not need to be avoided and do not need to be measured, and leaving only video signals (image information) that need to be avoided and need to be measured. It works.

As a result, the processing time is reduced by excluding video signals that do not need to be avoided and do not require distance measurement, and leaving only video signals (image information) that need to be avoided and require distance measurement.

In this embodiment, it is desirable to manually set the monitoring area 32a as an initial setting, and to use this monitoring area 32a for all images (step S).
6).

FIG. 8 is a diagram for explaining the monitoring area setting step.

Therefore, as shown in FIG. 8, from 0 to 60 pixels (pixels) are removed in order to eliminate the sky, and 60 pixels are removed from the left white line in order to exclude objects to be measured other than the left and right lanes. The area within the area is defined as the monitoring area 32a, and the remaining area is removed, and the right white line is similarly 60 pixels from the white line.
The area within is defined as the monitoring area 32a, and the remaining area is removed. Since the front vehicle 21 is not shown in the lower part, 15
The portion of 0 pixel or more is removed and the monitoring area 3
2a is set (step S6).

Next, creation of the edge image 33A will be described using an edge image creation step (steps S7 and S8).

[0166] The edge image creation step includes the left image information 31b.
At the same time, the horizontal and vertical edges are separated and extracted. At the same time, the horizontal and vertical edges are separated and extracted in the right image information 31a, and the program code (step S7) is extracted in the left image information 31b following the previous process. The left edge image 33AL is created by performing edge thinning on the extracted horizontal edge and vertical edge, and at the same time, the edge thinning is performed on the horizontal edge and vertical edge extracted in the right image information 31a. The program code for creating the edge image 33AR is included (step S8).

According to such a program code, the edge of the object to be measured (the front vehicle 21 or the like) having a height from the road surface 34 is left as much as possible by processing the difference image 33D.
4 The upper edge can be erased as much as possible by processing the difference image 33D. That is, the overlap portion between the horizontal edge and the vertical edge, which is higher than the road surface 34 and which should be originally left, and the overlap portion between the vertical edge and the horizontal edge are represented by the difference image 33.
It becomes possible to perform D processing and leave the data with higher accuracy than before. As a result, a considerable difference appears in the extracted edge amount of the front vehicle 21 region. As a result, the center of gravity of the edge can be calculated in the post-processing, and the window 336a including 80% of all edges centered on the center of gravity can be created to improve the estimation accuracy when estimating it as the area of the front vehicle 21. become. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

It is desirable that the host vehicle 20 recognizes the preceding vehicle 21 at a higher speed. Therefore, the right camera 41
The processing is performed only on the feature points in the right image information 31a and the left image information 31b obtained from the R and the left camera 41L.

When calculating the parallax P between the right image information 31a and the left image information 31b for each scanning line, it is considered that the calculation can be easily performed on the edge image 33A.

FIG. 9 is a diagram for explaining a step of employing an edge point in the edge image creation step.

Therefore, in the present forward vehicle detection method, operators (1),
An edge point extracted by using (2) as shown in FIG. 9 is adopted (step S7).

FIG. 10 is a diagram for explaining the thinning processing of the edge image in the edge image creation step. FIG.
FIG. 1 is a diagram for explaining a process of distinguishing a vertical edge from a horizontal edge in an edge image creation step.

Subsequently, when erasing the road surface 34, in order to leave as many edges of the preceding vehicle 21 as possible and to further speed up the processing, an edge image 33A as shown in FIG.
(Step S8), and then the process of FIG.
As shown in (5), a process of distinguishing between a vertical edge and a horizontal edge was performed (step S8).

[0174]

(Equation 1) … Operator (1)

[0175]

(Equation 2) .. Operator (2) Next, the elimination of the road surface 34 in the estimated edge image creation step (step S9) and the front vehicle extraction step (steps S10, S11, S12) will be described.

The estimated edge image creating step includes a program code (step S9) for estimating the left edge image 33AL from the right edge image 33AR.

According to such a program code, both the left image information 31b and the right image information 31a are separated into horizontal and vertical edges to create an extracted edge image, and after creating an edge image with thinned edges, , An estimated edge image 33E is created from the right edge image 33AR, and a difference horizontal edge 334a and a difference vertical edge 334b between the estimated edge image 33E and the left edge image 33AL are created separately for the horizontal edge and the vertical edge. 33
4a and difference vertical edge 334b are combined to obtain difference edge image 3
The difference horizontal edge 334a and the difference vertical edge 334b of 34c can be obtained. In addition, the estimated edge image 33
In E, only the marking on the road surface 34 at the same height from the road surface 34 is actually the same image as the left edge image 33AL captured by the left camera 41L, and an object at a position higher than the road surface 34 such as the preceding vehicle 21 is actually Left edge image 3
It is possible to prevent the image from being equal to 3AL. Estimated edge image 3 from left edge image 33AL actually captured
3E, the road surface 3 at the same height
4 (the road surface 34 of the traveling lane), the road surface 34 sign (for example, white line or speed limit display) are deleted from the left edge image 33AL, and the front vehicle 2 existing at a position higher than the road surface 34
Only one pixel can be left in the left edge image 33AL, and finally, there is an effect that the distance can be calculated with the remaining pixels in the left edge image 33AL as candidate pixels of the front vehicle 21.

According to a twelfth aspect of the present invention, in the method for detecting a forward vehicle according to the eleventh aspect, the difference edge image 33
The step of creating 4c (step S10) includes performing a difference operation on the horizontal edge between the estimated left edge image 33EL and the left edge image 33AL to create a difference horizontal edge 334a, and at the same time, the estimated left edge image 33EL and the left edge image 3
It includes a program code for creating a difference vertical edge 334b by performing a difference operation on a vertical edge with 3AL, and a program code for creating a difference edge image 334c by combining the difference horizontal edge 334a and the difference vertical edge 334b. .

FIGS. 12A and 12B show the estimated edge image 3 from the left edge image 33AL in the front vehicle extraction step.
This is a difference image 33D obtained by subtracting 3E (step S1).
0).

FIGS. 13A and 13B show an estimated edge image 33E and a left edge image 33E without considering the directionality of the edge.
AL is shown by overlapping.

The purpose of this processing is to leave the edge of the object to be measured (the front vehicle 21 and the like) having a height from the road surface 34 as much as possible and to delete the edge on the road surface 34 as shown in FIG.
From (a) and (b), the overlap portion (A portion) of the horizontal edge and the vertical edge, which has a height from the road surface 34 and which should be originally left, and the overlap portion (B portion) of the vertical edge and the horizontal edge are differentiated. Has disappeared.

On the other hand, FIGS. 12 (a) and 12 (b) show the same situation, where the horizontal edge takes a difference between horizontal edges, and the vertical edge takes a difference between vertical edges.

In this case, the edge disappearance as shown by the portions A and B in FIGS. 13A and 13B does not occur. Therefore, the present method can leave more edges to be measured having a height from the road surface 34 than before.

When the present front vehicle detection method is applied to the entire screen, a considerable difference appears in the amount of extracted edges in the front vehicle 21 area. Thereby, the center of gravity of the edge is calculated in the post-processing, and the window 3 including 80% of all edges centered on the center of gravity
The accuracy of estimation when creating 36a and estimating it as the area of the front vehicle 21 is further improved. According to the above algorithm, it is possible to more reliably extract the front vehicle 21 area.

According to such a program code, both the left image information 31b and the right image information 31a are separated into horizontal and vertical edges to create an extracted edge image, and after creating an edge image with thinned edges, , An estimated edge image 33E is created from the right edge image 33AR, and a difference horizontal edge 334a and a difference vertical edge 334b between the estimated edge image 33E and the left edge image 33AL are created separately for the horizontal edge and the vertical edge. 33
4a and difference vertical edge 334b are combined to obtain difference edge image 3
The difference horizontal edge 334a and the difference vertical edge 334b of 34c can be obtained. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

In the forward vehicle extraction step, the program code for calculating the center of gravity of the edge point of the residual image in which the difference horizontal edge 334a and the difference vertical edge 334b are created in the difference calculation between the estimated left edge image 33EL and the left edge image 33AL. Step S11) is included.

Step of Creating Window 336a (Step S1)
2) is a program code including a predetermined ratio of all edges around the calculated center of gravity.

Here, the estimated edge image 33E is an image obtained by assuming that all the image information of one camera is a pattern on the road surface 34 and estimating the edge image when the edge image related to the image information is photographed by the other camera. Means information.

Such a program code (step S1)
According to (1, 12), it is possible to leave more edges to be measured having a height from the road surface 34 than before. Further, the extracted edge amount in the area of the front vehicle 21 can be made significantly different. According to this, calculation of the center of gravity of the edge in the post-processing, creation of a window 336a including 80% of all edges centered on the center of gravity, and improvement in estimation accuracy when estimating the window 331 as the area in front of the vehicle 21 This has the effect of being able to perform. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

In the present forward vehicle detection method, the processing time is reduced by erasing the road surface 34 which does not need to be avoided and which does not need to be measured, and executes image processing to reduce the number of feature points.

In order to erase the road surface 34, a left edge image 33A (left edge image 33AL) is estimated from a right edge image 33A (= right edge image 33AR). At this time, in this forward vehicle detection method, the CCD camera 4
It is assumed that the left video signal 41La and the right video signal 41Ra (image) obtained from 1L and 41R are all at the same height from the road surface 34 and the road surface 34.

FIG. 14 is a diagram for explaining the parallax between the left image information 31b and the right image information 31a.

Generally, a parallax P occurs between the captured left image information 41L and right image information 41R (see FIG. 14). Such a value of the parallax P indicates that the object to be measured is three-dimensionally
If it exists farther than R and the left camera 41L, it will be smaller, and if it exists closer, it will be larger.

When considering a two-dimensional image, FIG.
If an xy coordinate system such as 4 is applied, the road surface 34
Of the parallax P increases as the y-coordinate increases.
That is, since the value of the parallax P linearly changes in accordance with the value of the y coordinate, the parallax P and the value of the y coordinate can be represented by a linear function.

Thus, in a certain image, the parallax P
By calculating in advance the relational expression between y and the y coordinate, if the image of the road surface 34 is captured by the same system,
From this relational expression and the left image information 41L, the parallax P of each pixel is calculated for each scanning line, and image processing for moving the pixel by the value of the parallax P is performed, whereby the image is actually captured with the right image information 41R. The same image as the right image information 41R can be created.

FIG. 15 is a diagram for explaining a parallax calculation error with respect to a distance-measuring object existing at a position higher than the road surface.

However, in the image of the object to be measured located at a position higher than the road surface 34, that is, the image of the preceding vehicle 21, this relational expression does not hold between the parallax P and the y coordinate as shown in FIG. . Therefore, assuming that the front vehicle 21 is at the same height from the road surface 34 and the road surface 34, the parallax P of each pixel of the left image information 41L is calculated for each scanning line using this relational expression, and Even if it is moved by the value of P, accurate creation of the right image information 41R cannot be performed.

In the forward vehicle detection method, the processing time is reduced by eliminating the road surface 34 which does not need to be avoided by using the above principle and reducing the number of feature points.

In the present embodiment, it is assumed that all captured images are at the same height from the road surface 34 and the road surface 34,
Left edge image 33 estimated from right edge image 33AR
AL is calculated as the estimated edge image 33E (step S9).

FIG. 16 is a diagram for explaining an error between the object to be measured at a position higher than the road surface 34 and the left edge image 33AL.

As described above, in the estimated edge image 33E, only the road surface 34 and the road surface mark at the same height from the road surface 34 become the same image as the left edge image 33AL actually picked up by the left camera 41L. As shown in FIG. 16, an object such as the preceding vehicle 21 located at a position higher than the road surface 34 is not an image equal to the actual left edge image 33AL.

FIG. 17 is a diagram for explaining step S10 in the front vehicle extraction process.

Therefore, as shown in FIG. 17, the estimated edge image 33E is obtained from the actually captured left edge image 33AL.
Is deleted (step S10), the road surface 3
The road surface 34 (the road surface of the traveling lane) and the road marking (for example, the white line and the speed limit display) at the same height from 4 are deleted from the left edge image 33AL, and are located at a position higher than the road surface 34 as shown in FIG. Only the pixels of the front vehicle 21 existing in the left edge image 33AL remain in the left edge image 33AL (step S10). Finally, the distance D is calculated using the remaining pixels in the left edge image 33AL as candidate pixels of the preceding vehicle 21.

Next, the measurement of the distance D to the front vehicle 21 will be described using a front vehicle extraction step (steps S11, S12) and a distance calculation step (steps S13, S14, S15, S16).

In the distance calculating step, the parallax between the window 336a in the left current image and the window 336a in the right current image is calculated using a relational expression calculated in advance based on the proportional relationship between the parallax and the reciprocal of the distance. A program code for calculating a distance based on the calculated parallax, a program code for setting a window 336a in a left current image, which is a captured image captured by the left camera 41L (step S13), and an image captured by the right camera 41R A program code (step S14) for setting a region corresponding to the right current image, which is a photographed image to be obtained, and a program code (step S14) for calculating the disparity between the window 336a in the left current image and the window 336a in the right current image. S15) and based on the calculated parallax,
The program code (step S16) for calculating the distance between the vehicle traveling ahead and the host vehicle 20 is included.

Such a program code (step S1)
According to 3, 14, 15, 16), the parallax between the window 336a in the left current image and the window 336a in the right current image is obtained, and the absolute distance to the distance measurement target is calculated from the parallax value. It can be obtained with high accuracy and at high speed. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use. Further, since the distance can be calculated by calculating the disparity between the window 336a in the left current image and the window 336a in the right current image using the relational expression calculated in advance, the absolute distance from the value of the disparity to the distance measurement target can be calculated. The distance can be determined with high accuracy and high speed. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

First, the preceding vehicle extracting step (step S1)
The calculation of the parallax P of the preceding vehicle 21 in (0, 11, 12) will be described.

Most of the pixels of the left edge image 33AL left after the erasing of the road surface 34 are pixels of the preceding vehicle 21. Therefore, the center of gravity of all the remaining pixels becomes substantially equal to the center of gravity of the preceding vehicle 21 (step S11).
A rectangle centering on the center of gravity is created, and image processing is performed so that pixels included in the rectangle are 80% of all pixels (step S12).

[0209] The rectangle thus created includes substantially the entire front vehicle 21. The square created in this way is considered as the window 336a, and image processing for providing the same size window 336a on the right image information 41R is executed (step S13).

FIG. 18 is a diagram for explaining step S14 in the distance calculation process.

Subsequently, as shown in FIG. 18, while moving the window 336a on the right image information 41R to the left, the luminance difference between the two windows 336a of the right image information 41R and the left image information 41L (left A difference that can be calculated using the signal levels of the video signal 41La and the right video signal 41Ra) is calculated, and the point where the value is minimum is detected (step S14). Subsequently, the parallax P of the left and right windows 336a at this time is calculated for each scanning line. The parallax P calculated in this manner becomes the parallax P of the preceding vehicle 21 (step S15).

Subsequently, a distance calculating step (step S16)
The calculation of the relationship between the parallax P and the distance D in will be described.

The conventional method for calculating the distance D based on the binocular image is a method for detecting the front vehicle by calculating the distance D to the front vehicle 21 based on the principle of triangulation.

However, this forward vehicle detection method can calculate the forward vehicle 2 even if the camera setting value is slightly deviated.
A significant error has occurred in the distance D up to 1 and accurate distance measurement has been difficult. Therefore, in the present embodiment, an accurate distance measurement is performed by calculating a relational expression between the parallax P and the distance D in advance.

FIG. 19 is a diagram for explaining a process of calculating a relational expression between parallax and distance in advance.

In this embodiment, as shown in FIG. 19, every 10 m from a camera (a camera having the same performance as the left camera 41 L and the right camera 41 R), a distance measurement target (for example, FIG. Is placed, and the object to be measured is imaged. By calculating the parallax P of these objects to be measured for each scanning line, the parallax P and the distance D are calculated.
Is calculated.

Since the reciprocal of the parallax P and the distance D are proportional, the reciprocal of the parallax P and the distance D can be represented by a linear function. This function is used to calculate the forward vehicle 2 calculated in the previous section.
By substituting the parallax P of 1, the distance D to the preceding vehicle 21 is calculated. In the vehicle front monitoring system 10 used this time, the relationship between the parallax P and the distance D is represented by Expression (2).

D = α / (P−9) [m] (2) α: proportionality constant, d: distance, P: parallax In the present embodiment, α is set to 680. It is necessary to be aware of the optimum value depending on the installation position of the camera 41L and the right camera 41R.

Next, the calculation process of the relative speed v of the preceding vehicle 21 will be described.

When monitoring the front of the vehicle 20, it is preferable that information on the relative speed v of the vehicle 21 can be obtained at the same time as the distance D to the vehicle 21.

Even if the preceding vehicle 21 is very close, if the preceding vehicle 21 is far from the host vehicle 20, the degree of danger is low.
Conversely, even if there is a forward vehicle 21 to a certain distance, if the forward vehicle 21 is rapidly approaching the own vehicle 20, the risk is high.

In this embodiment, the relative speed v of the preceding vehicle 21 is calculated simultaneously with the distance D to the preceding vehicle 21.

The frame interval of the image data taken into the data processing device 30 is, for example, 0.5 second. Further, by the processing up to the previous section, the distance D to the preceding vehicle 21 is calculated. Therefore, information such as the distance D of the front vehicle 21 and the image data acquisition interval for each image data acquisition can be obtained. From these values, if the capture interval of the image data is 0.5 seconds, the forward vehicle
It can be seen how close the vehicle 1 is to the vehicle 20. Therefore, the relative speed v of the preceding vehicle 21 is calculated by equation (3).

V = (d2-d1) /0.5 [m / s] (3) where, v: relative speed d1: distance to the preceding vehicle 21 obtained from the image data captured one immediately before d2 : Distance to preceding vehicle 21 In the determination step following the distance calculation step, it is determined whether or not the vehicle is in a danger state based on the calculated distance, and if it is determined that the vehicle is in the danger state, a program code for generating an alarm signal 36a (step S17) ).

With such a program code (step S1)
According to 7), there is an effect that it is possible to determine with high speed and high accuracy whether or not there is a dangerous state based on the distance calculated with high speed and high accuracy. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

The alarm step following the judgment step is a step program code (step S18) for generating an alarm in response to the alarm signal 36a based on the calculated distance.

With such a program code (step S1)
According to 8), a high-speed and high-precision alarm can be generated. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

[0228]

According to the first aspect of the present invention, it is possible to avoid the possibility that a large error in the detection distance occurs, and to achieve high detection accuracy by synchronizing the imaging timing. It has the effect of being able to. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the second aspect of the present invention, the first aspect
In addition to the effects described in (1), the amount of memory for recording captured images can be set smaller than in the past, and further, a vehicle forward monitoring system using a stereo image processing technology having high detection accuracy and high-speed processing performance can be realized. Become like As a result, it is possible to claim advantages in terms of manufacturing cost and installation space. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the invention described in claim 3, according to claim 2
In addition to the effects described in (1), the amount of memory of the frame memory for recording the captured image can be set to be smaller than the number of even-numbered scanning lines and the number of odd-numbered scanning lines as compared with the related art.
The parallax required for distance calculation is horizontal (scanning line direction)
Therefore, the detection resolution does not deteriorate as compared with the conventional method for detecting the forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory.

According to the invention described in claim 4, according to claim 2,
In addition to the effects described in (1), the amount of memory of the frame memory for recording the captured image is set to be smaller than the conventional one by the sum of the number of even-numbered scanning lines for 1/2 frame and the number of odd-numbered scanning lines for 1/2 frame. As a result, the capacity of the frame memory can be reduced to about half that of the related art. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory.

According to the invention set forth in claim 5, according to claim 4,
In addition to the effects described in (1), the amount of memory of the frame memory for recording the captured image is set to be smaller than the conventional one by the sum of the number of even-numbered scanning lines for 1/2 frame and the number of odd-numbered scanning lines for 1/2 frame. As a result, the capacity of the frame memory can be reduced to about half that of the related art. In addition, since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution is not inferior to that of the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of manufacturing cost and installation space related to the amount of memory of the frame memory.

According to the invention described in claim 6, according to claim 3,
5. In addition to the effects described in any one of (5) to (5), the frame memory can be shared, and the memory amount of the frame memory for recording a captured image can be reduced by one-half the number of even-numbered scanning lines and It can be set to be as small as the sum of the number of odd-numbered scanning lines for / 2 frames, and the capacity of the frame memory can be reduced to about half that of the related art. Even if the frame memory is shared, the parallax required for calculating the distance is in the horizontal direction (scanning line direction), so that the detection resolution is not inferior to the conventional method for detecting a forward vehicle. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory due to the sharing of the frame memory.

According to the invention of claim 7, according to claim 6,
In addition to the effects described in (1), a set of binocular images can be obtained as a moving image by using two pieces of right image information and left image information having twice the resolution in the horizontal and vertical directions. Also,
Since the parallax required for calculating the distance is in the horizontal direction (scanning line direction), the detection resolution of the right image information and the left image information is not inferior to that of the conventional forward vehicle detection method. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use. As a result, it is possible to claim superiority in terms of the manufacturing cost and the installation space related to the memory amount of the frame memory due to the sharing of the frame memory.

According to the invention of claim 8, according to claim 7,
In addition to the effects described in (1), the provision of the monitoring area setting step makes it possible to eliminate these video signals in advance and to set a monitoring area including the preceding vehicle.
When an image of the front of the vehicle is taken by the right camera or the left camera, video signals (image information) of the sky, trees, buildings, etc., which do not need to be avoided and do not need to be measured, are left video signals or right video signals. Can be removed from within. As a result, it is possible to reduce the processing time by eliminating video signals that do not need to be avoided and do not require distance measurement, and leave only video signals (image information) that need to be avoided and require distance measurement.

According to the ninth aspect of the present invention, it is possible to delete only the road surface on the same plane and leave the preceding vehicle at a higher place. Further, more edges to be measured having a height from the road surface can be left more than before. Further, when the present forward vehicle detection method is applied to the entire screen, a considerable difference appears in the amount of extracted edges in the forward vehicle region. As a result, it is possible to calculate the center of gravity of the edge in the post-processing, create a window including 80% of all edges centered on the center of gravity, and further improve the estimation accuracy when estimating the window as the front vehicle area. become. As a result, it is possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the tenth aspect of the invention, in addition to the effect of the ninth aspect, the edge of the object to be measured (vehicle ahead, etc.) having a height from the road surface can be subjected to differential image processing to minimize the difference. The edge on the road surface can be erased as much as possible by differential image processing. In other words, it is possible to perform the difference image processing on the overlapping portion between the horizontal edge and the vertical edge which is higher than the road surface and which is desired to be left, and the overlapping portion between the vertical edge and the horizontal edge, and to leave it with higher accuracy than before. . As a result, a considerable difference appears in the amount of extracted edges in the front vehicle region. This makes it possible to calculate the center of gravity of the edge in the post-processing, create a window including 80% of all edges centered on the center of gravity, and further improve the estimation accuracy when estimating the window as the front vehicle area. . This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the eleventh aspect of the present invention, in addition to the effect of the tenth aspect, by providing an estimated edge image creating step, both the left image information and the right image information have a horizontal edge and a vertical edge. After creating an edge image that is extracted by extracting the edge image, creating an edge image with a thinned edge, creating an estimated edge image from the right edge image, a difference horizontal edge and a difference vertical edge between the estimated edge image and the left edge image Is created separately for the horizontal edge and the vertical edge, and thereafter, the difference horizontal edge and the difference vertical edge are combined to be a difference horizontal edge and a difference vertical edge of the difference edge image. In the estimated edge image, only the road marking at the same height from the road surface is the same image as the left edge image actually captured by the left camera, and objects at a position higher than the road surface, such as a preceding vehicle, are actually left edges. It is possible to prevent the image from being equal to the image. When the common part with the estimated edge image is deleted from the actually captured left edge image, the road surface at the same height (road surface of the traveling lane)
And road markings (for example, white line and speed limit display) are deleted from the left edge image, and only the pixels of the preceding vehicle existing at a position higher than the road surface can be left in the left edge image. , The remaining pixels in the left edge image can be calculated as candidate pixels for the preceding vehicle.

According to the twelfth aspect of the present invention, in addition to the effect of the eleventh aspect, an advantage is provided in that an algorithm for easily performing forward detection processing with high detection accuracy suitable for the purpose of use is provided. Be able to argue.

According to the thirteenth aspect of the present invention, in addition to the effect of the twelfth aspect, by providing a step of creating a window, an edge of a distance measurement target having a height from a road surface is reduced. It is possible to leave more than before. Further, it is possible to make a considerable difference between the amounts of extracted edges in the front vehicle region. Based on this, it is possible to calculate the center of gravity of the edge in the post-processing, create a window 336a including 80% of all edges centered on the center of gravity, and improve the estimation accuracy when estimating it as the front vehicle area. become able to. This makes it possible to claim superiority in providing an algorithm that facilitates forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the fourteenth aspect of the present invention, the same effects as the effects of any one of the ninth to thirteenth aspects can be obtained.

According to the invention described in claim 15, in addition to the effects described in any one of claims 11 to 14, the parallax between the window in the left current image and the window in the right current image is reduced. Then, the absolute distance from the parallax value to the object to be measured can be obtained with high accuracy and high speed.
This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the sixteenth aspect of the present invention, in addition to the effect of the fifteenth aspect, the relationship between the window in the left current image and the window in the right current image is calculated using a relational expression calculated in advance. Since the distance can be calculated by calculating the parallax, the absolute distance from the parallax value to the object to be measured can be obtained with high accuracy and high speed. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the seventeenth aspect of the present invention, in addition to the effects of the fifteenth or sixteenth aspect, it is also possible to determine whether or not the vehicle is in a dangerous state based on the distance calculated at high speed and with high accuracy. Can be determined. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to the eighteenth aspect, in addition to the effect of the seventeenth aspect, a high-speed and high-precision alarm can be generated. This makes it possible to claim superiority in that it has an algorithm that can easily perform high-speed forward monitoring processing with high detection accuracy suitable for the purpose of use.

According to a nineteenth aspect of the present invention, a vehicle traveling forward by performing stereo image processing using a left image collected by using a left camera and a right image collected by using a right camera. In a vehicle front monitoring system that measures the distance to the own vehicle and issues an alarm in case of danger, the optical axis was installed parallel to the traveling direction vector of the own vehicle and provided at the same height from the road surface on the vehicle Left and right cameras, a synchronization signal that generates a synchronization signal that instructs the timing of shooting with the left camera and the right camera, and a synchronization signal generator that generates a switching signal synchronized with the frame time. A stereo moving image acquiring apparatus comprising: a synthesized video creating unit that shoots a left image and a right image at the same time in synchronization with a right camera and creates a synthesized video signal of the left image and the right image Is a vehicle front monitoring system is configured to have.

According to the nineteenth aspect of the invention, in addition to the effect of the first aspect, when all the images obtained from the left camera (right camera) are at the same height as the road surface, By moving all the pixels in the scanning line direction by the value of the parallax, it is possible to create an image equivalent to an image captured by the right camera (left camera).

According to the twentieth aspect, the same effect as that of the second aspect can be obtained.

According to the twenty-first aspect of the invention, the same effect as the third aspect can be obtained.

According to the twenty-second aspect, the same effect as that of the fourth aspect can be obtained.

According to the twenty-third aspect of the present invention, the same effect as the fifth aspect is obtained.

According to the twenty-fourth aspect, the same effect as that of the sixth aspect can be obtained.

According to the twenty-fifth aspect, the same effect as the seventh aspect can be obtained.

According to the twenty-sixth aspect, the same effect as the ninth aspect can be obtained.

According to the twenty-seventh aspect, the same effects as those of any one of the eleventh to thirteenth aspects can be obtained.

According to the twenty-eighth aspect of the present invention, the same effect as that of the fifteenth or sixteenth aspect is obtained.

According to the twenty-ninth aspect, the same effect as that of the seventeenth aspect can be obtained.

According to the thirtieth aspect, the same effect as the twentieth aspect can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram for explaining an embodiment of a data processing device constituting a vehicle front monitoring system of the present invention.

FIG. 2 is a functional block diagram for explaining an embodiment of a stereo moving image acquisition device constituting the vehicle front monitoring system of the present invention.

FIG. 3 is a flowchart illustrating an embodiment of a forward vehicle detection method performed by the forward vehicle monitoring system.

FIG. 4 is a principle diagram for explaining a binocular stereoscopic method used in a forward vehicle detection method and a forward vehicle monitoring system.

FIG. 5 is a diagram for explaining an image form of a composite video signal composed of a right video signal and a left video signal used in a forward vehicle detection method and a forward vehicle monitoring system.

FIG. 6A is a diagram for explaining an interlace mode in which an odd-numbered scanning line signal and an even-numbered scanning line signal are separately output every 1/2 frame time, and FIG.
FIG. 6 is a diagram for explaining a frame structure of a right video signal and a left video signal in the interlace mode of FIG.

FIG. 7 shows the stereo moving image acquisition apparatus of FIG. 2 in which a left camera and a right camera having an optical axis that coincides with a traveling direction vector when the own vehicle goes straight and are parallel to a road surface are provided on the ceiling of the automobile. It is a figure for explaining a mode that it attaches to a part and acquires a binocular road image.

FIG. 8 is a diagram for explaining a monitoring area setting step.

FIG. 9 is a diagram for explaining a step of employing an edge point in an edge image creation step.

FIG. 10 is a diagram illustrating a thinning process of an edge image in an edge image creation step.

FIG. 11 is a diagram illustrating a process of distinguishing a vertical edge from a horizontal edge in an edge image creation step.

FIGS. 12A and 12B are diagrams for explaining a step of subtracting an estimated edge image from a left edge image to create a difference image in a front vehicle extraction step.

FIGS. 13A and 13B are diagrams showing an estimated edge image and a left edge image superimposed without considering the directionality of the edge.

FIG. 14 is a diagram for describing parallax between left image information and right image information.

FIG. 15 is a diagram for describing a parallax calculation error for a distance measurement target existing at a position higher than a road surface.

FIG. 16 is a diagram for explaining an error between a subject to be measured at a position higher than a road surface and a left edge image.

FIG. 17 is a diagram for explaining step S10 in a front vehicle extraction process.

FIG. 18 is a diagram illustrating step S14 in the distance calculation step.

FIG. 19 is a diagram for explaining a process of calculating a relational expression between parallax and distance in advance.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Vehicle front monitoring system 20 ... Own vehicle 21 ... Front vehicle 30 ... Data processing apparatus 31 ... Left / right image separation acquisition means 31a ... Right image information 31b ... Left image information 32 ... Monitoring area setting means 32a ... Monitoring area 33 ... Front vehicle Extraction means 332... Edge image creating section 33A... Edge image 33AL... Left edge image 33AR... Right edge image 33E... Estimated edge image 33D... Difference image 34... Road surface 334. 334c: Difference edge image 336: Window creation unit 336a: Window 35: Distance calculation means 36: Judgment means 36a ... Warning signal 37: Warning means 40: Stereo moving image acquisition device 41L: Left camera 41La: Left video signal 41R: Right camera 41Ra right video signal 43 synchronization signal generator 43a synchronization signal 43b switching signal 44 ... Synthetic image creating unit 44a ... composite video signal 45 ... frame memory

Claims (30)

[Claims] 1. A stereo image processing is performed using a left image collected by using a left camera and a right image collected by using a right camera to measure a distance between a vehicle traveling ahead and the host vehicle. A method for detecting a forward vehicle that generates an alarm in a dangerous case, comprising: synchronizing the left and right cameras, photographing the left image and the right image at the same time, and creating a composite video signal of the left image and the right image; and A method for detecting a preceding vehicle, comprising a stereo moving image acquiring step including: 2. When each of the left camera and the right camera outputs an odd scan line signal / even scan line signal and an even scan line signal / odd scan line signal / even scan line signal, the stereo moving image acquiring step is performed. Collecting a predetermined scanning line signal in a left video signal from a left camera and a predetermined scanning line signal in a right video signal from a right camera to generate the composite video signal. The method for detecting a forward vehicle according to any one of the preceding claims. 3. The stereo moving image acquiring step includes: an even scan line signal / odd scan line signal / even scan line signal in a left video signal from a left camera and an odd scan line signal / even in a right video signal from a right camera. And a scanning line signal is collected, and an even scanning line signal / odd scanning line signal / even scanning line signal of the left video signal is set as an even scanning line signal / odd scanning line signal / even scanning line signal in the composite video signal. A step of combining the odd scanning line signal / even scanning line signal of the right video signal as an odd scanning line signal / even scanning line signal in the composite video signal to generate the composite video signal. 3. The method for detecting a forward vehicle according to claim 2. 4. An odd scan line signal / even scan line signal and an even scan line signal / odd scan line signal /
When the left camera and the right camera operate in the interlace mode in which the even-number scan line signal is output separately, the stereo moving image acquiring step includes the step of converting the left video signal from the left camera and the right video signal from the right camera into the 1st video signal. The method according to claim 2, further comprising a step of generating the composite video signal by alternately switching and collecting the composite video signal every / 2 frame time. 5. The composite video signal uses a half frame of an odd scan line signal / even scan line signal in the right video signal as image data of a half frame in the composite video signal. The composite video signal is obtained by using 1/2 frame of the even-numbered scan line signal / odd-number scan line signal / even-number scan line signal in the left video signal as image data of the remaining 1/2 frame in the composite video signal. The method according to claim 4, further comprising the step of combining. 6. The method according to claim 1, further comprising a step of recording the synthesized image signal, which is composed of one frame by combining the left video signal and the right video signal alternately with scanning lines, in a single frame memory. A method for detecting a forward vehicle according to claim 3. 7. A right image by reading out the composite image signal recorded in the frame memory and separating an odd-numbered scan line signal in a scan line signal constituting one frame of the read composite image signal. 7. The front vehicle detection method according to claim 6, further comprising a left / right image separation / acquisition step of generating information and separating left / right image information by separating even-numbered scanning line signals in the scanning line signal. 8. The front vehicle detection method according to claim 7, further comprising a step of setting a monitoring area for front vehicle detection with respect to the right image information and the left image information. 9. A distance between a vehicle traveling ahead and a host vehicle by performing stereo image processing using left image information collected using a left camera and right image information collected using a right camera. In the method for detecting a forward vehicle, which generates a warning in a dangerous case by measuring the horizontal image and the vertical edge in the left image information, the horizontal and vertical edges are separated and extracted in the right image information. And a preceding step, thinning the edges of the horizontal edge and the vertical edge extracted in the left image information to create a left edge image, and the horizontal edge extracted in the right image information. A step of creating a right edge image by thinning the edge with respect to the edge and the vertical edge, and following the previous step, estimating the left edge from the right edge image / left edge image. Following the previous step, a difference image between the left edge image / right edge image and the estimated edge image is created independently of the horizontal edge and the vertical edge. A front vehicle detection method, comprising: a front vehicle extraction step including a step of creating a difference edge image by combining. 10. The front vehicle extracting step includes a step of separating and extracting a horizontal edge and a vertical edge in the left image information and a step of separating and extracting a horizontal edge and a vertical edge in the right image information. Following the process,
The horizontal edge and the vertical edge extracted in the left image information are thinned to form a left edge image, and the horizontal and vertical edges extracted in the right image information are thinned. The method according to claim 9, further comprising an edge image creating step of creating a right edge image by performing the conversion. 11. The front vehicle detection method according to claim 10, wherein the front vehicle extraction step includes an estimated edge image creation step of estimating a left edge image from the right edge image. 12. The front vehicle extracting step performs a difference operation on a horizontal edge between the estimated left edge image and the left edge image to create a difference horizontal edge, and further includes the estimated left edge image and the left edge image. Generating a difference vertical edge by performing a difference operation on the vertical edge of the difference, and a step of combining the difference horizontal edge and the difference vertical edge to create a difference edge image, The front vehicle detection method according to claim 11, wherein 13. The front vehicle extracting step is a step of calculating a center of gravity of an edge point of a residual image in which the difference horizontal edge and the difference vertical edge are created in a difference operation between the estimated left edge image and the left edge image. The method according to claim 12, further comprising: creating a window including a predetermined ratio of all edges around the calculated center of gravity. 14. The estimated edge image is an image obtained by assuming that all image information of one camera is a pattern on a road surface and estimating an edge image when the edge image related to the image information is photographed by the other camera. The front vehicle detection method according to any one of claims 9 to 13, wherein the method is information. 15. A step of setting the window in a left current image, which is an image captured by the left camera, and a step of setting a corresponding area on the right current image, which is an image captured by the right camera. Calculating the disparity between the window in the left current image and the window in the right current image, and calculating the distance between the vehicle traveling ahead and the host vehicle based on the calculated disparity. The method for detecting a forward vehicle according to any one of claims 11 to 14, further comprising a distance calculating step including the steps of: 16. The distance calculating step includes calculating a parallax between a window in the left current image and a window in the right current image using a relational expression calculated in advance based on a proportional relationship between the parallax and the reciprocal of the distance. The method according to claim 15, wherein the distance is calculated based on the calculated parallax. 17. The method according to claim 15, further comprising the step of determining whether or not the vehicle is in a dangerous state based on the calculated distance, and generating an alarm signal when the vehicle is determined to be in a dangerous state. Forward vehicle detection method. 18. The method according to claim 17, further comprising the step of receiving the warning signal and generating a warning based on the calculated distance. 19. A stereo image processing is performed using a left image collected using a left camera and a right image collected using a right camera to measure a distance between a vehicle traveling ahead and the host vehicle. In a vehicle front monitoring system that issues an alarm in the event of a danger, the left and right cameras, with the optical axis installed in parallel with the traveling direction vector of the vehicle and at the same height from the road surface on the vehicle, and the left camera And a synchronizing signal for instructing the timing of photographing with the right camera,
A synchronization signal generator that generates a switching signal synchronized with a frame time, and synchronizes a left camera and a right camera based on the synchronization signal to shoot a left image and a right image at the same time, and a left image and a right image. 1. A vehicle forward monitoring system, comprising: a stereo moving image acquisition device including a composite video creating unit that creates a composite video signal. 20. When each of the left camera and the right camera outputs an odd scan line signal / even scan line signal and an even scan line signal / odd scan line signal / even scan line signal, Wherein a predetermined scanning line signal in the left video signal from the left camera and a predetermined scanning line signal in the right video signal from the right camera are collected to generate the composite video signal. Item 20. A vehicle forward monitoring system according to item 19. 21. The stereo moving image acquisition device, in synchronization with the switching signal, an even scan line signal / odd scan line signal / even scan line signal in a left video signal from a left camera and a right video from a right camera. An odd scan line signal / even scan line signal in the signal is collected, and an even scan line signal / odd scan line signal / even scan line signal of the left video signal is converted into an even scan line signal / odd scan line signal in the composite video signal. / Even scanning line signal, and combine the odd scanning line signal / even scanning line signal of the right video signal as an odd scanning line signal / even scanning line signal in the composite video signal to generate the composite video signal. 21. The vehicle front monitoring system according to claim 20, wherein the system is configured. 22. The stereo moving image acquiring apparatus according to claim 1, wherein
When the left camera and the right camera operate in the interlace mode in which the odd scan line signal / even scan line signal and the even scan line signal / odd scan line signal / even scan line signal are output separately every two frame times, the interlacing is performed. In synchronism with the switching signal, a left video signal from a left camera and a right video signal from a right camera are alternately switched and collected every フ レ ー ム frame time to generate the composite video signal. The vehicle front monitoring system according to claim 21, wherein the system is configured. 23. The stereo moving image acquiring apparatus, in synchronization with the switching signal, divides a half frame of an odd scanning line signal / even scanning line signal in the right video signal into one in the composite video signal. / 2 frames of image data, and the 1/2 frame of the even-numbered scan line signal / odd-numbered scan line signal / even-numbered scan line signal in the left video signal is used as the remaining 1 in the composite video signal.
23. The vehicle forward monitoring system according to claim 22, wherein the combined video signal is combined using the image data for / 2 frames. 24. A single frame memory for recording the synthesized image signal in which one frame is formed by synthesizing the left video signal and the right video signal alternately with scanning lines. The vehicle front monitoring system according to any one of claims 21 to 23. 25. The combined image signal recorded in the frame memory is read out, and odd-numbered scan line signals in the scan line signals constituting one frame of the read combined image signal are separated to form a right image. Left and right image separation obtaining means for generating information and separating even-numbered scanning line signals in the scanning line signal to create left image information; and monitoring the front image detection for the right image information and the left image information. 25. The vehicle forward monitoring system according to claim 24, further comprising a data processing device including a monitoring area setting unit that sets each area. 26. The data processing device, wherein a horizontal edge and a vertical edge are separated and extracted from the left image information, and a horizontal edge and a vertical edge are separated and extracted from the right image information. The left edge image is created by performing edge thinning on the horizontal edge and the vertical edge extracted in, and the edge thinning is performed on the horizontal edge and vertical edge extracted in the right image information. A right edge image is created, an estimated left edge image is created from the right edge image, a difference image between the left edge image and the estimated edge image is created independently from the horizontal edge and the vertical edge, and these are combined to obtain a difference image. 26. The vehicle forward monitoring system according to claim 25, further comprising a forward vehicle extracting unit that creates an edge image. 27. The front vehicle extracting means separates and extracts a horizontal edge and a vertical edge from the left image information, and separates and extracts a horizontal edge and a vertical edge from the right image information. The left edge image is created by thinning the horizontal and vertical edges extracted in the information, and the thinning of the edges is performed on the horizontal and vertical edges extracted in the right image information. An edge image creating unit for creating a right edge image to create an estimated left edge image / estimated right edge image, and an edge image creation unit for creating an estimated left edge image / estimated right edge image, A difference horizontal edge is created by performing a difference operation, and the estimated left edge image / estimated right edge image and the left edge image / right edge image are calculated. A difference edge creating unit that creates a difference vertical edge by performing a difference operation on a vertical edge with the image, and combines the difference horizontal edge and the difference vertical edge to create a difference edge image; and the estimated left edge image In the difference calculation between the left edge image and the left edge image, the center of gravity of the edge point of the residual image in which the difference horizontal edge and the difference vertical edge are created is calculated, and a window including a predetermined ratio of all edges around the calculated center of gravity is created. 26. The vehicle front monitoring system according to claim 25, further comprising a window creating unit that performs the operation. 28. The data processing device, wherein the window is set in a left current image which is a photographed image taken by a left camera, and an area corresponding to a right current image which is a photographed image taken by a right camera is set. Set, calculate the disparity between the window in the left current image and the window in the right current image, and calculate the distance between the vehicle running ahead and the host vehicle based on the calculated disparity. 28. The vehicle front monitoring system according to claim 26, further comprising a distance calculating unit. 29. The data processing apparatus according to claim 29, further comprising: a determination unit configured to determine whether or not the vehicle is in a dangerous state based on the calculated distance, and to generate an alarm signal when determining that the vehicle is in a dangerous state. Item 30. A vehicle forward monitoring system according to item 28. 30. The vehicle front monitoring system according to claim 29, wherein the data processing device has an alarm unit that receives the alarm signal and generates an alarm based on the calculated distance.