JPH0935197A - Vehicle recognizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to recognize a preceding vehicle by detecting the height of tail light by the distribution of featured points in the vertical direction of a picture and checking whether a vehicle can be recognized or not based upon vehicle width detected by the distribution of horizontal featured points and an estimated distance. SOLUTION: A telecamera 6b arranged on the upper part of the center of a fron window in a vehicle room photographs a two-dimensional (2D) picture including a traveling road surface and an A/D converter 6c converts the analog video signal into a digital signal and stores the digital signal in an image memory 5a. A CPU 1 synthetically photographs a white line on the end part of its own lane, an inter- vehicle distance between its own vehicle and the preceding vehicle, etc., displays the updated contents of these synthesized results and transfers these information to a host CPU 8 through a communication controller 7. The picture is scanned in the vertical direction, featured points of which brightness is changed are extracted, the height of tail light is detected based upon the distribution, and whether the vehicle can be recognized based upon the 2D picture or not is discriminated based upon the vehicle width detected based upon the distribution of horizontal featured points and an estimated distance from the preceding vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle recognition method, and based on two-dimensional image information obtained by photographing a scene including a road surface such as the front of the vehicle with a photographing device installed on the vehicle, It can be used to recognize a preceding vehicle or an oncoming vehicle traveling.

[0002]

2. Description of the Related Art An example of this type of detection device is disclosed in Japanese Patent Laid-Open No. 64-1
It is disclosed in Japanese Patent No. 5605. In this case, a vehicle including a road surface in front of the vehicle obtained by an image pickup device mounted on the vehicle is used.
Position on the X, Y Cartesian coordinates above the set value by calculating the differential value of the image data showing the image on the X, Y Cartesian coordinate system screen in the direction forming an angle of 45 degrees with respect to the X axis. Is set as a feature point, a series of feature points is processed into a thin line, a thin line of a predetermined length or more is extracted, and it is set based on the feature of the white line on the screen when the front is photographed from the TV camera on the vehicle. A thin line that satisfies the conditions is recognized as a white line on the road surface. That is, two straight lines representing the left and right white lines of the lane on which the vehicle is traveling are obtained. The intersection of the two straight lines is obtained, and the presence or absence of an object is detected in the area between the two straight lines (triangle area) from the intersection to the own vehicle. When the object is detected, the distance from the own vehicle to the object (the object is the preceding vehicle Then the inter-vehicle distance) is calculated. This kind of technology can be used during the daytime when the entire preceding vehicle and the like can be recognized from the captured image. However, at night, since the surroundings are dark, the preceding vehicle can hardly recognize it except for the tail light portion that is emitting light, and therefore the preceding vehicle cannot be recognized.

On the other hand, in the technique disclosed in Japanese Patent Application Laid-Open No. 2-190978, a scene in front of the vehicle is photographed through a red light transmission filter to obtain an image in which only the red component is extracted, and the red color is extracted. -It facilitates the extraction of lulump. Further, the contour of each red region is detected by image processing, and each of the pair of left and right tail lamps is recognized from the contour information. Then, after recognizing a pair of tail lamps, the left tail lamp from the left end to the right tail lamp.
Detects the horizontal distance X on the image to the right edge of the lamp,
The inter-vehicle distance between the vehicle (own vehicle) and the preceding vehicle is detected based on the distance X. It is considered that this technology can basically recognize the preceding vehicle both during the day and at night.

[0004]

However, since the size and shape of the tail lamp vary depending on the type of vehicle,
It is very difficult to correctly identify each tail lamp. To improve the recognition accuracy, if you reduce the noise image other than the tail lamp by using a red light transmission filter,
It may be difficult to detect necessary information such as white lines on the road and headlights of oncoming vehicles.

Generally, the preceding vehicle recognition algorithm focusing on the tail lamp can be applied at night when the tail lamp is lit, but it is difficult to extract from the image of the tail lamp that is not lit at daytime, so it is difficult to extract it from the daytime. Is not available.

If an image is photographed using a special photographing device such as a night-vision camera or an infrared camera, the contour line of the entire preceding vehicle can be detected even at night. Although it can be used, a special photographing device is extremely expensive and cannot be mounted on a general automobile in practical use.

Further, if a preceding vehicle recognition algorithm that is completely different between daytime and nighttime is used, it is inevitable that the structure of the device becomes complicated. In particular, in order to process a large amount of image information in a short time, it is necessary to prepare various dedicated hardware devices suitable for processing in addition to the microcomputer, so two types of algorithms are selected. Execution requires more hardware devices to be installed, and inevitably becomes expensive.

Further, when the preceding vehicle recognition algorithm is changed between daytime and nighttime, it is necessary to correctly recognize daytime and nighttime and select an appropriate one, but automatic recognition of daytime and nighttime is difficult. . When the algorithm is switched by the driver's switch operation, an inappropriate algorithm may be selected due to an operation error or the like.

The present invention aims to solve the above problems.

[0010]

In order to solve the above problems, according to claim 1 of the present application, a scene including a road surface outside the vehicle is photographed by a photographing device (6b) mounted on the vehicle. A region determined by processing the obtained two-dimensional image information, identifying a plurality of white lines on the road surface detected as a series of feature points in the image in a predetermined direction, and determining the positions of a plurality of straight lines representing the white lines as a reference. Regarding the vehicle recognition method for recognizing the vehicle in the two-dimensional image information, the position of the point at infinity ((XV, YV)) on the lane on which the vehicle travels is set to the position of the intersection of the plurality of straight lines. Based on the identification (D4), the two-dimensional image information is scanned at a plurality of horizontal positions in a substantially vertical direction to detect a plurality of pieces of information on the first feature point whose brightness changes ( 132), information on the detected first feature points Frequency identifies the position in the vertical direction is equal to or greater than the predetermined threshold value as a vertical reference position (1
34, 135), and based on the distribution state of the plurality of first feature points in the vicinity of the vertical reference position, identify the horizontal position of the center of gravity as the horizontal center of gravity position (Mx (NR)) (136). , Based on information of second feature points obtained by scanning the two-dimensional image information in the horizontal direction for a predetermined range near the left end position (XVLP) of the object predicted based on the horizontal center of gravity position The actual left end position of the target object is identified (201, 202L to 207L), and the right end position of the target object (X is predicted based on the horizontal center of gravity position).
VRP), the actual right end position of the object is identified based on the information of the third feature point obtained by scanning the two-dimensional image information in the horizontal direction (201, 20).
2R to 207R), the position of the point at infinity (YV), the reference position in the vertical direction (My (NR)), the height (Hc) of the position where the image capturing device is installed, and the ground of a predetermined light emitting portion on the vehicle. Based on the height (TL) and the focal length (Sy) of the photographing device, an estimated distance (L1) corresponding to the distance from the photographing device to the vehicle in the two-dimensional image information is obtained and identified. Based on the distance (WV) between the actual left end position of the object and the right end position of the actual object and the estimated distance (L1), at least the presence or absence of vehicle recognition based on the two-dimensional image information is identified. (H4).

Further, according to a second aspect of the present invention, the two-dimensional image information obtained by photographing the scene including the road surface outside the vehicle with the photographing device (6b) mounted on the vehicle is processed, and the feature in the image is obtained. A vehicle that identifies a plurality of white lines on a road surface detected as a series of points in a predetermined direction and recognizes a vehicle in the two-dimensional image information with respect to a region determined based on the positions of a plurality of straight lines representing the white lines. In the recognition method: the position of the point at infinity ((XV, YV)) on the lane on which the vehicle is traveling is identified based on the position of the intersection of the plurality of straight lines (D4), and the two-dimensional image information is obtained. , At a plurality of positions in the horizontal direction, each of which is scanned substantially in the vertical direction to detect a plurality of pieces of information of the first feature points whose brightness changes (132), and the plurality of detected first feature points are detected. Vertical direction in which the occurrence frequency of the information of the Is identified as the vertical reference position (134, 135), and the horizontal position of the center of gravity is determined based on the distribution state of the plurality of first feature points in the vicinity of the vertical reference position. Mx (NR)) (136), and the two-dimensional image information is obtained by scanning the two-dimensional image information in the horizontal direction for a predetermined range near the left end position (XVLP) of the object predicted based on the horizontal center of gravity position. The actual left end position of the object is identified based on the information of the second feature point (201, 202L to 207L), and the position of the right end position (XVRP) of the object predicted based on the horizontal center of gravity position is determined. For the predetermined range, the actual right end position of the object is identified based on the information of the third feature point obtained by scanning the two-dimensional image information in the horizontal direction (201,
202R to 207R), at least day and night are identified (208A), and when it is identified as day, the position of the point at infinity (YV), the vertical reference position (My (NR)),
Based on the height (Hc) of the position where the photographing device is installed and the focal length (Sy) of the photographing device, and when it is identified as night, the position of the infinity point (YV), the vertical position. Direction reference position (My (NR)), height (Hc) of the position where the photographing device is installed, height (TL) of a predetermined light emitting part on the vehicle from the ground, and focal length (S) of the photographing device.
y), the estimated distance (L) corresponding to the distance from the imaging device to the vehicle in the two-dimensional image information.
1) is obtained (208B, 208C), and based on the distance (WV) between the identified left end position of the actual object and the right end position of the actual object and the estimated distance (L1), at least the two The presence or absence of vehicle recognition based on the three-dimensional image information is identified (H4).

Further, in claim 3, when the actual left end position of the object is identified (206L) based on the information of the second feature points, information of the plurality of detected second feature points is included. The appearance frequency (H (x)) is set to the second threshold value (Th
Compared to 4), the actual left end position of the object is determined based on the horizontal position having a high appearance frequency, and the actual right end position of the object is identified based on the information of the third feature point ( 206R), the appearance frequency (H (x)) of the detected information of the plurality of third feature points is compared with the second threshold value (Th4), and the detected frequency is set to the horizontal position where the appearance frequency is large. Based on the actual right end position of the object based on the
At least day and night are identified, and the magnitude of the second threshold value (Th4) is switched depending on whether it is identified as day or night (229A, 229B, 229C).

Further, according to a fourth aspect of the present invention, at the time of determining the area in the two-dimensional image information to be processed, at least day and night are identified (154A), and the case identified as day and the night are identified. Depending on the case, the position of the region is vertically shifted by an amount corresponding to the height (TL) of the light emitting portion on the vehicle from the ground (154B).

Further, in claim 5, a plurality of small areas (A1 to A16) are defined (S1) in a specific area (RGS) expected to correspond to the sky in the two-dimensional image information, and each of them is defined. Average brightness is calculated for each small area (S
2) Compare the calculated average brightness with a threshold (S
3) The primary identification result (S_f
After obtaining it as lag (i)), day and night are identified based on the plurality of primary identification results (S4).

The symbols shown in the above parentheses are the reference numerals of corresponding elements, coordinates, processing steps, etc. in the embodiments to be described later. It is not limited to the specific elements inside.

[0016]

[Operation] By the photographing device (6b) mounted on the vehicle,
For example, when shooting a scene including the road surface in front,
Two-dimensional image information as shown in FIGS. 64 and 65 is obtained. 63, 64, and 65 are images at night, daytime, and evening, respectively. As can be seen from FIG. 63, at night, in addition to the tail lights emitted by the preceding vehicle and the white line on the road surface (mark for lane identification) illuminated by the headlights of the own vehicle, It hardly appears in the image.
Therefore, the body of the preceding vehicle cannot be recognized from the nighttime image.

According to a first aspect of the present invention, the two-dimensional image information is provided for an area determined based on the positions of a plurality of straight lines representing the white line, for example, an area indicated by a vertical line in the window 2-1 shown in FIG. Is processed as follows. Of course, the order in which the processes are executed can be slightly changed. First, the position of the point at infinity ((XV, YV)) on the lane on which the vehicle travels is identified based on the position of the intersection of the plurality of straight lines (D4: see FIG. 22).

Next, the two-dimensional image information is scanned at a plurality of horizontal positions in a substantially vertical direction to detect a plurality of pieces of information of the first characteristic point where the brightness changes (1
32: see FIG. 31). Thereby, for example, the horizontal component of the contour of the bright image portion in the lane (inside the white line) in which the vehicle is traveling is detected as the first feature point.

Next, the position in the vertical direction in which the appearance frequency of the detected information of the plurality of first characteristic points is equal to or higher than a predetermined threshold value is identified as the vertical direction reference position (134: FIGS. 34 and 135). : FIG. 35). When a histogram is created for the position in the vertical direction (y direction) where each of the first feature points is present, the distribution state of the first feature points can be known. If there is a horizontally extending region (bright portion) in the image being processed, the frequency of appearance increases at that position (y coordinate). This position is identified as the vertical reference position. For example, in the case of the image shown in FIG. 63, many first characteristic points are detected in the taillight portion of the preceding vehicle traveling in front of the own vehicle lane (or the headlight portion of the oncoming vehicle). , The position where it exists (y coordinate) is detected as the vertical reference position (My (NR)).

Next, the horizontal position of the center of gravity is identified as the horizontal center of gravity position (Mx (NR)) based on the distribution state of the plurality of first feature points in the vicinity of the vertical reference position (136). : See FIG. 37). This position is the center position of the entire bright area near the preceding vehicle, that is, the intermediate position between the normally emitting left tail light and the right tail light.

Next, with respect to a predetermined range near the left end position of the object (XVLP: normally, the left end position of the tail light on the left of the preceding vehicle: the predicted position) predicted based on the horizontal center of gravity position, The actual left end position of the object is identified based on the information of the second feature point obtained by scanning the three-dimensional image information in the horizontal direction (201, 202L to 207L).
The second feature point detected here is the vertical component of the contour of the bright image portion existing near the left end position of the object. Therefore, the actual left end position of the object (usually, the left end position of the left tail light of the preceding vehicle: detected position) can be identified based on the information of the second feature point.

Similarly, with respect to a predetermined range in the vicinity of the right end position of the object (XVRP: normally, the right end position of the right tail light of the preceding vehicle: the predicted position) predicted based on the horizontal center of gravity position, The actual right end position of the object is identified based on the information of the third feature point obtained by scanning the two-dimensional image information in the horizontal direction (201, 202R to 207).
R). The third feature point detected here is the vertical component of the contour of the bright image portion existing near the right end position of the object. Therefore, the actual right end position of the object (normally, the right end position of the right tail light of the preceding vehicle: detected position) can be identified based on the information of the third feature point.

The position of the point at infinity (YV), the reference position in the vertical direction (My (NR)), the height (Hc) of the position where the photographing device is installed, and the predetermined light emitting portion on the vehicle from the ground. Height (TL) and focal length (Sy) of the imaging device
Is known, the estimated distance (L1) corresponding to the distance from the imaging device to the vehicle in the two-dimensional image information can be obtained based on them (see FIG. 23).

L1 = Sy · (Hc−TL) / (My (NR) −YV) On the other hand, a typical vehicle width is about 1.7 m, and the actual vehicle width does not change. , This actual vehicle width WV3D
Based on the estimated distance L1 and a predetermined scale factor Sx, the estimated vehicle width WV2D in the two-dimensional image can be obtained.

WV2D = WV3D.times.Sx / L1 The vehicle width WV detected in the two-dimensional image is usually substantially equal to the distance between the actual left end position of the object and the right end position of the object identified by the above processing. Therefore, depending on the error between the detected vehicle width WV and the estimated vehicle width WV2D, if the error is small, it can be considered that the vehicle such as the preceding vehicle has been recognized successfully, and if the error is large, It can be considered that the vehicle is not recognized.

Although the estimated distance L1 may include a relatively large error, after successful recognition of the vehicle,
Based on the actual vehicle width WV3D and the detected vehicle width WV, the inter-vehicle distance L2 with a relatively small error can be obtained.

L2 = WV3D × Sx / WV By the way, as shown in FIG. 64, in the daytime, the entire body of the preceding vehicle appears in the image. In addition, in FIG. 64, since the gradation is not expressed, it is not shown, but in the actual image,
A shadow of it appears on the road surface below the car body. The vertical position of this shadow is identified as the vertical reference position (My (NR)) during the day by applying the above processing. Further, when the above process is executed in the daytime, the left end and the right end of the vehicle body of the preceding vehicle are respectively detected, and therefore the vehicle width can be obtained based on them. In other words, it is possible to recognize the preceding vehicle at night and day by a very similar process, and most of the process can be shared between night and day. However, if the processing is exactly the same, a large error will occur.

Therefore, in claim 2, the processing is partially switched by distinguishing between nighttime and daytime. That is, the detected vertical reference position is TL in the vertical direction in the daytime and the nighttime.
Since it shifts by (the height of the tail light), when obtaining the estimated distance L1, L1 = Sy · Hc / (My (NR) −YV) is applied during the day and L1 = Sy · (Hc− at night. TL) / (My (NR) -YV) is applied. As a result, the preceding vehicle or the like can be recognized both at night and during the day.

In claim 3, the actual left end position of the object is identified based on the information of the second feature point (206).
In L), the appearance frequency (H (x)) of the detected information of the plurality of second feature points is compared with a second threshold value (Th4), and based on the horizontal position where the appearance frequency is large. To determine the actual left end position of the object. Further, when identifying the actual right end position of the object based on the information of the third feature point (206R), the appearance frequency (H (x)) of the information of the plurality of detected third feature points. Is compared with the second threshold value (Th4), and the actual right end position of the object is determined based on the horizontal position having a high appearance frequency.

Here, at night, since the object to be detected is a tail light or the like, the height in the vertical direction is relatively small, whereas in daytime, the object to be detected is the entire vehicle body. Is relatively large. Therefore, the appearance frequency (H (x)) of the information of the plurality of second feature points and the third feature points
Is small at night and large at daytime. Therefore, in the third aspect, day and night are identified, and the magnitude of the second threshold value (Th4) is switched depending on whether it is identified as day or night.

The vertical reference position (My (NR))
Is the position of the tail light at night, while it is the position of the shadow on the road surface below the vehicle body in the daytime, so the position on the image of the detection target is It is displaced by a height equivalent. Therefore, in claim 4, at the time of determining the region in the two-dimensional image information to be processed, at least day and night are identified (154A), and the case of identifying day and the case of identifying night. Then, the position of the region is vertically shifted by an amount corresponding to the height (TL) of the light emitting portion on the vehicle from the ground (154B).

By the way, it is difficult to accurately distinguish between day and night even when the determination is made based on the image taken on the vehicle. In general, it is possible to distinguish between day and night based on the brightness of the sky region in the image. However, for example, in a nighttime image, when street lights, advertising lights, etc. are reflected in the sky region, the brightness of the sky region may be erroneously recognized as daytime.

Therefore, in a fifth aspect, a plurality of small areas (A1 to A16) are defined (S1) in a specific area (RGS) expected to correspond to the sky in the two-dimensional image information, and each small area is defined. Average brightness is calculated for each area (S
2) Compare the calculated average brightness with a threshold (S
3) The primary identification result (S_f
After obtaining it as lag (i)), day and night are identified based on the plurality of primary identification results (S4). As a result, day and night can be discriminated with high reliability.

[0034]

FIG. 1 shows the configuration of a first embodiment of the present invention. This system is a microcomputer (hereinafter C
PU) 1 as a main component, and its bus line has a read-only memory (ROM) 2 in which a control program is stored, a read / write memory (RAM) 3 in which a parameter being processed is stored, and Input / output ports (I / O) 4, 5, 6 and the like to which various components are connected are connected.

The television camera 6b, as shown in FIG.
It is installed near the upper center of the front window in the car, and the front scene, that is, the road surface on which the car runs, the sky,
An image including surrounding objects is photographed as a two-dimensional image and an analog video signal is output. This analog video signal is A
It is given to the / D converter 6c and the CRT driver 4a. In the A / D converter 6c, the analog video signal from the television camera 6b is sampled at a timing of each predetermined pixel forming an image, and the signal level of each pixel is set to 256 gradations (gradation 0 is set). It is converted into digital data (8-bit gradation data) of black level and gradation 255 of white level and supplied to the image memory 5a. In this example, each frame of the image is 512 × 51.
It is composed of two pixels. The image memory 5a has a gradation data storage area for several pages when the area for storing the gradation data of one screen (512 × 512 pixels) is one page.
It has a binary data storage area for storing 1-bit information (binary data) for several screens.

The CPU 1 is a television camera controller 6
The aperture of the TV camera 6b is controlled via a, and the A / D
The input / output of the converter 6c and the writing process of the image memory 5a are synchronously controlled. The TV camera 6b is AG
Since the C (auto gain control) circuit is built in, the level (voltage) of the analog video signal input to the A / D converter 6c is the timing at which the brightest pixel in the image appears. Predetermined maximum level, ie FIG. 66
It is automatically adjusted to Vf shown in. Also, A
In the / D converter 6c, the full scale level is adjusted in advance so that the maximum level Vf of the input signal becomes the maximum gradation 255 of the digital data.

The CRT driver 4a is a television camera 6b.
A digital image (digital output image) provided under the control of the CPU 1 to the image represented by the analog video signal given by the above (for example, FIG. 16); And an image that combines the character and numerical information indicating the distance to the vehicle ahead,
The information is displayed on the monitor television CRT 4b installed near the center of the inner panel on the vehicle.

FIG. 3 shows an outline of the processing operation of the CPU 1. Generally, the CPU 1 initializes an input / output port, initializes an internal register and an internal counter (initialization A) when the power is turned on, and then determines an analog image signal for one screen. Image memory 5 with digital conversion in cycles
The data is written in the input data memory area of a (image input B). Every time this is performed, the CPU 1 performs “screen calibration” C, “day / night determination” S, and “vehicle detection in window 2-1” D.
When ET1 is executed and the preceding vehicle is not detected on the vehicle traveling lane by this DET1, in addition to DET1, "detection of vehicle in window 2-2" DET2 is executed. If the preceding vehicle is not detected on the own vehicle traveling lane even in this DET2, in addition to DET2, "Window 2
-3 vehicle detection ”DET3 is executed. In DET1, "own vehicle lane detection" D, own vehicle lane detection success check E, and own vehicle lane detection are established (success).
In case of "neighboring lane estimation" F, "own vehicle lane preceding vehicle recognition and ranging" H, "right adjacent lane vehicle recognition and ranging" I, "left adjacent lane vehicle recognition" And "distance measurement" J and "output" K are executed in this order. Do the same for DET2 and DET3. However, the processing target area on the screen represented by the image data is different.

In the "output" K, a straight line representing the white line at the end of the own vehicle lane, character information indicating whether or not a preceding vehicle of the own vehicle lane is detected, a square mark surrounding the preceding vehicle and the Distance between the preceding vehicle, character information indicating whether or not a right adjacent lane vehicle is detected, a square mark surrounding the preceding vehicle and a distance between the preceding vehicle and a left adjacent lane vehicle Character information indicating presence / absence, a square mark surrounding the preceding vehicle, and a distance between the preceding vehicle and the preceding vehicle are combined into a photographed image, which is updated and displayed on the CRT 4b, and a white line at the end of the own vehicle lane is displayed. The data representing a straight line and the data representing each inter-vehicle distance are transferred to the host CPU 8 via the communication controller 7 (FIG. 1). The host CPU 8 applies these information to the following distance control (accelerator control corresponding to the following distance, overdrive cutoff control and / or wheel brake pressure control), obstacle detection & wheel brake pressure control, lane. Deviation warning control, lane copying traveling control (steering control and / or wheel blurring for each wheel)
Used for automatic travel control such as pressure control). Below, FIG.
The contents of each item of "Screen calibration" C and "Left adjacent lane vehicle recognition and distance measurement" J will be described in detail.

C. "Screen calibration" C (Fig. 4) The contents of this process are shown in Fig. 4. First, the "feature point detection window set" C1 is executed.

C1. "One set of feature point detection windows"
C1 (FIG. 5) This content is shown in FIG. As shown in FIG. 5 (b), the upper left of the shooting screen of the television camera 6b (the image data is gradation data in the input data memory) is the origin (0,
0), one point (0,350) and another point (51
1, 511) as diagonal corners is set as a feature point detection region (window 1). That is, the X coordinate value 0 of the upper left corner of the area to be determined is set in the register XU.
Write the Y coordinate value 511 to the register YUL, the X coordinate value 511 of the lower right corner of the area to the register XLR, and the Y coordinate value 350 to the register YLR (step 1 in FIG. 5A). ; In the following, in parentheses, the word step is omitted, and only the step number is described). The value indicated by the data of each of these registers will be represented below by the register symbol itself.

"Feature point detection (UP)" C2 described next
5 (b) of FIG. 5 by the above-mentioned "feature point detection window set" C1 for the processing of FIG.
The block (feature point detection window 1) shown by the dotted line is
It has been designated as the processing target area. As shown in FIG. 5B, this region sufficiently includes the leading edge image of the hood of the vehicle.

Next, "feature point detection (UP)" C2 is executed. This content is shown in FIG.

C2. “Feature point detection (UP)” C2 (FIG. 6) Here, the lower left corner (0,
511) The pixel (0, 509) two pixels above in the Y direction is scanned in the Y direction from the bottom to the top in which the Y direction coordinate value is sequentially reduced to 350. The pixels to be designated are sequentially designated as the target pixel (2 to 11 in FIG. 6A), and for each target pixel (xs, ys), the pixel (xs, ys + 1) one pixel below and the pixel two pixels below Of the brightness represented by the gradation data of the pixel (xs, ys + 2) of the pixel, and the pixel (xs, ys-1) one pixel above and the pixel (xs, ys) two pixels above the pixel of interest (xs, ys). −
The absolute value of the difference between 2) and the sum of the brightness represented by the gradation data in the input data memory, that is, the absolute value of the gradation differential value in the vertical direction (Y direction) is calculated (8). This is defined as the gradation differential value D (xs, ys) of the target pixel point (xs, ys).
This gradation differential value D (xs, ys), as shown in (b) of FIG. 6, is obtained by comparing the image density sum of the upper two pixels and the image density sum of the lower two pixels of the target point (xs, ys). Since it is the absolute value of the difference, it represents the change in density in the vertical direction with the point of interest as a boundary, and becomes a large value at the contour line extending horizontally in each image on the screen. It is checked whether or not this gradation differential value D (xs, ys) is equal to or more than a threshold value Th1 (set value) (9). If it is Th1 or more, it is assigned to an area of the image memory 5a. At the address E (xs, ys) corresponding to the pixel of interest (xs, ys) of the value data table,
Write "1" (10). This information "1" means that it is a contour line extending in the horizontal direction of the image. Such processing is performed in the Y direction for one line (X = 0, Y = 509 to 35).
0), the scanning line is moved to the right (6), and the same process is performed for the next one line (X = 1, Y = 509 to 350). Similarly, the last line (X = 51
1, Y = 509 to 350) are similarly processed.

By the above processing, the binary data table E (xs, ys), xs = 0 to 51 of the image memory 5a.
1, ys = 509 to 350), information indicating the contour line extending in the horizontal direction of the image in the feature point detection window 1 (b in FIG. 5) is written. Feature point detection window 1
Is set in a region including the leading edge image of the bonnet, so the binary data table E (xs, ys) contains information representing the leading edge of the bonnet. In this embodiment, the leading edge of the bonnet is regarded as a straight line, and the straight line is detected by the next "bonnet detection" C3.
This content is shown in FIG.

C3. "Bonnet detection" C3 (Fig. 7) First, the outline is explained here by the known "Houg (Houg
h) Transform ”to detect a straight line. The Hough transform is performed in a rectangular coordinate system (XY coordinate system; for example, the feature point detection window 1
Information (for example, the binary data table E (xs, ys)) represented by a pixel distribution in the polar coordinate system (ρ-
This is a method of expressing in the (θ coordinate system), and the conversion of ρ = Xcos (θ) + Ysin (θ) is performed. A certain point (pixel) in the XY coordinate system becomes a certain curve in the ρ-θ polar coordinate system (orthogonal coordinate plane that allocates ρ and θ to one and the other of the two orthogonal axes), and X-
When each point on the straight line in the Y coordinate system is represented by each curve in the ρ-θ coordinate system, these curves intersect at one point.
By substituting this intersection point into the conversion formula to the XY coordinate system, the linear formula in the XY coordinate system can be obtained.

Therefore, the binary data table E
The coordinates having the feature point information “1” on (xs, ys) are ρ
A straight line connecting the characteristic points can be obtained by converting into −θ polar coordinate values, obtaining the intersection of the curves in the ρ−θ polar coordinate system, and substituting this intersection into the conversion formula to the XY coordinate system. However, the straight line connecting the two points can be directly obtained in the XY coordinate system, and there is no point in converting to the ρ-θ polar coordinate system. However, the bonnet outline on the screen is
The contour of the tip of the actual bonnet is curved,
When the bonnet contour is approximated by a straight line, the binary data table E (xs,
If the straight line connecting the two points having the feature point information “1” on ys) is obtained, the approximation error is too large, and the contour line of the non-bonnet may be extracted. FIG.
As shown in FIG. 16B and FIG. 16, the bonnet outline is distributed over a wide area, so that the binary data table E
A lot of feature point information “1” exists on (xs, ys). Therefore, a large number of straight lines connecting the two points (pixels) having the "1" are obtained. When a straight line descending to the right that represents a large number of straight lines is determined, it is the one that best approximates the right half of the bonnet tip contour line (b in FIG. 7). Similarly, when a straight line to the left that represents a large number of straight lines is determined, it is the one that best approximates the left half of the bonnet tip contour line (b in FIG. 7). However, if such statistical processing is performed in the XY coordinate system, the processing becomes extremely complicated. However, when the coordinates (feature points) having the feature point information “1” on the binary data table E (xs, ys) are transformed into the ρ-θ polar coordinate system using the Hough transform, each feature point is represented. The intersections of the curves are concentrated at two points (corresponding to the right half contour line and the left half contour line of the bonnet). Therefore, the characteristic point of the binary data table E (xs, ys) is Hough-transformed, and the frequency (the number of curves passing through one point) of the curve is counted at each point on the ρ-θ polar coordinate system. Select the two points, the maximum point and the next largest point, and substitute each selected point into the conversion formula to the XY coordinate system to express the right half and the left half of the bonnet outline. Is obtained. In this embodiment, the tip contour line of the bonnet is detected according to such logic, that is, by counting the number of times the curve passes through each point on the ρ-θ polar coordinate system.

By the way, the count of the curve crossing frequency at each point on the ρ-θ polar coordinate system is represented by the binary data table E (xs,
If it is performed at each point in the entire region of the ρ-θ polar coordinate system having a size corresponding to the size of (ys), the count processing of the number of times the curve passes becomes enormous, and the straight line detection processing takes a long time. Therefore, in this embodiment, the right and left intermediate points (X = 255, Y = 511) at the lowermost end of the binary data table E (xs, ys) are set to ρ-θ.
It is set at the origin (0,0) of the polar coordinate system, and the binary data table E (xs, ys) is divided into left and right, and the right half is converted into binary data by the first Hough transform. -Bull E (xs,
ys) characteristic points are converted into a low-density (large sampling pitch) ρ-θ polar coordinate system and the number of times a curve passes through each point of the polar coordinate system (the number of curves passing through one point) is counted, and the maximum frequency is obtained. Point (first prediction point) is obtained. Next, the second Hough transform is used to set a small range ρ-θ region centered on the first prediction point, and the size of the region is set to the medium density ρ-θ polar coordinate system.
The characteristic point of the table E (xs, ys) is converted to
The number of times the curve passes through each point of the −θ polar coordinate system is counted, and the point with the highest frequency (second predicted point) is obtained. Then, in the third Hough transform, a smaller range ρ-θ region centered around the second prediction point is set, and the binary data table E (is set in the high density ρ-θ polar coordinate system of the size of the region. (xs, ys) are converted to count the number of times the curve passes through each point of this ρ-θ polar coordinate system, and the point with the highest frequency (third prediction point) is obtained. Then, the straight line represented by the third predicted point is determined as the right half of the bonnet tip contour line (12R to 14R in FIG. 7). Two
The left half of the value data table E (xs, ys) is similarly subjected to the first Hough transform, the second Hough transform and the third Hough transform to obtain a straight line representing the left half of the bonnet tip contour line. It is determined (12L to 14L in FIG. 7).

Next, the contents of the "bonnet detection" C3 will be described in detail with reference to FIG. First, parameters are set for detecting a straight line that approximates the right half of the bonnet tip contour line (12R). Then, the point (Rm3, Tm3) corresponding to the approximate straight line is detected (13R). Rm3
Is the ρ value in the ρ-θ polar coordinate system, and Tm3 is the θ value. The calculated coordinate value of the point is stored in the registers RmR and TmR (14R).
Similar processing is performed to detect a straight line that approximates the left half of the bonnet tip contour line (12L to 14L). "Line fitting" 13R, which detects straight lines by Hough transform,
The contents of 13L are the same as the contents of the "straight line fitting" 63 shown in FIGS. Note that the “straight line fitting” 63 is in contact with the feature point detection window 1 and the upper feature point detection window 2
-1 (FIG. 16) detects an approximate straight line of a white line image on the road, but the "straight line fitting" 13R detects a straight line in the right half region of the window 1, and therefore is to be processed. X-Y coordinate areas (start point coordinate value and end point coordinate value) are different. Similarly, in the "straight line fitting" 13L, the XY coordinate area to be processed is different from that of the "straight line fitting" 63. Also, in the "straight line fitting" 63, the window 2
Since the entire area of -1 is the processing target area and the right end white line and the left end white line are present in that area, two points (the two straight lines) of the point with the highest frequency and the point with the next highest frequency are obtained.
Since R and "straight line fitting" 13L are left and right half regions, only the point with the highest frequency is obtained. The other processing is the same.

"Line Fitting" The straight lines obtained by 13R and 13L are represented by points (RmR, TmR) and (RmL, TmL) in the polar coordinate system. These straight lines are represented by the XY coordinate system. RmR = Xcos (TmR) + Ysin (TmR) and RmL = Xcos (TmL) + Ysin (TmL). These straight lines are as shown in FIG. 7 (b).

C4. "Straight line intersection calculation" C4 (FIG. 4) Referring again to FIG. 4, when the above "bonnet detection" C3 is finished, the CPU 1 executes the calculation shown in the block in contact with the "straight line intersection calculation" C4 block. The intersection (xc, yc) of the straight line is calculated. In this calculation, X
ch is the X of the area of the binary data table E (xs, ys).
The center position in the direction (Xch = 255), Ych is the lowermost position (Ych = 509), and the point (Xch, Ych) is the polar coordinate origin determined by the above-described straight line detection.

C5. "Calculation of roll angle and pan movement amount" C5
(FIG. 8) Here, as shown in (a) of FIG. 8, the roll angle is a straight line (vertical axis of the vehicle) that divides the angle at which the two detected straight lines intersect each other and the Y-axis of the photographing screen. Is an angle θr formed by the vertical axis of the field of view of the television camera 6b and the vertical axis of the vehicle when they are projected on a horizontal plane, that is, the center of the field of view of the television camera 6b with respect to the vertical axis of the vehicle. This is the angle of deviation in the vertical direction. The pan movement amount Xp is a lateral shift amount of the center of the bonnet tip (intersection point of the two straight lines) with respect to a vertical axis that bisects the screen on the shooting screen. When a vehicle ahead of the own vehicle in the traveling direction (preceding vehicle) is calculated and the inter-vehicle distance to this vehicle is calculated, if the roll angle θr or the pan movement amount Xp is large, the vehicle is deviated from the traveling direction of the own vehicle. There is a high probability that a certain vehicle or object will be mistaken for a vehicle or the like on the vehicle lane (the lane on which the vehicle is traveling).
In order to reduce this probability, the mounting angle of the camera 6b may be adjusted so that the roll angle θr and the pan movement amount Xp are both zero, but this adjustment is difficult and troublesome.
Therefore, to some extent, the roll angle θr and the pan movement amount X
p is inevitable. Therefore, in this embodiment, the "angle of screen" C is used to adjust the roll angle θr and the pan movement amount Xp to zero on the screen. However, in order to calculate the adjustment margin, the above-described processing of C1 to C4 is performed. And "calculate roll angle and pan movement amount" C5 described here. The calculation formulas for the roll angle θr and the pan movement amount Xp are shown in the block in contact with the block C5 in FIG. The calculated roll angle θr is stored in the roll angle register θr, and the calculated pan movement amount Xp is stored in the pan movement amount register Xp.

C6-8. "Image correction" C6 to 8 (Fig. 4) and the next "corrected image memory initialization" C
6. By "image correction" including "image rotation parallel movement" C7 and "interpolation" C8, the floor of each pixel of the input data memory (one screen of gradation data storage area) of the image memory 5a. The address of the key data (x, y in (b) of FIG. 8) is
The address that has been rotated and moved in parallel corresponding to the roll angle θr and the pan movement amount Xp (the address of the screen coordinate system representing the photographed image at θr = 0, Xp = 0; FIG. 8B).
X ', y') of the input data memory and write it in the corrected image memory corresponding to the input data memory (one screen of the gradation data storage area of the image memory 5a) (C6, C7).

C6. "Corrected image memory initialization"
C6 (FIG. 9) When such rotation and parallel movement are performed, some pixels in the corrected screen do not have gradation data. In order to be able to recognize such pixels later, in the “correction image memory initialization” C6, all addresses of the correction image memory are
"-1" that does not exist in photographed image data (gradation data)
Write the data representing the. The details of the processing are shown in FIG.

C7. "Image rotation parallel movement" C7 (Fig. 1
0) The contents of this processing are shown in FIG. In this case, the input data memory address (x, y); x = 0 to 511, y
= 0 to 511, the address (xn, yn) of the screen coordinate system (correction image memory) representing the captured image; xn =
0-511, yn = 0-511 (22-2
8, 30 to 33), the address (x,
The gradation data of y) is written in the corrected image memory at the address (x ', y') obtained by converting the address (x, y) (29). As a result, when the gradation data of the corrected image memory is displayed on the CRT 4b, an image as shown in FIG. 16 in which both the roll angle θr and the pan movement amount Xp shown in FIG. 8A are substantially zero is displayed. To be done. However, due to this "image rotation parallel movement", an address (pixel = blank pixel) where gradation data does not exist may occur in the corrected image memory, and in that case, there is the above-mentioned "corrected image". By the memory initialization "C6", the data indicating "-1" remains.

C8. "Interpolation" C8 ((a) of Fig. 11 & Fig. 1
2) In the "interpolation" C8, the average value of the gradation data of the four pixels surrounding the gradation data is assigned to the pixel having no gradation data. If any of the surrounding four pixels is a blank pixel, this allocation is not performed. The contents of this "interpolation" C8 are shown in FIG. 11 (a) and FIG. In this case, the addresses (pixels) of the corrected image memory are sequentially designated, and it is checked whether or not the designated pixel (pixel of interest) is a blank pixel (memory data indicates "-1") ( 36), if the pixel of interest is a blank pixel, as shown in (b) of FIG. -Check for data in this order,
If even one of these four pixels is a blank pixel,
Processing for the pixel of interest ends (leave blank pixel as is)
Then, the pixel of interest is moved to the next (39-42, 42-46, 4
6-50, 50-54). When it is checked whether each of the 4 pixels has gradation data, if there is gradation data, it is added to the contents of the accumulation register Is, and the obtained sum is updated in the accumulation register Is. Shi (40,44,48,5
2) The content of the frequency register Ns is incremented by 1 (41, 45, 49, 53). When this is completed for all of the four pixels, the average value Ic (x, y) of the gradation data of these pixels is calculated and stored in the target pixel (54, 55).

As described above, the "screen calibration" C (FIG. 3) is completed, and the CPU 1 next executes "day / night judgment" S. Briefly, this processing S is to process the image data existing on the image memory 5a to check the ambient brightness and automatically discriminate between day and night. Is reflected in the flag S_night. Since this process is somewhat complicated, its detailed description will be given later for convenience.

Subsequent to the "day / night determination" S, the CPU 1 executes "vehicle detection in window 2-1" DET1. In this DET1, first, "own vehicle lane detection" D is executed.

D. "Own vehicle lane detection" D (Fig. 13) This content is shown in Fig. 13. First, the "feature point detection window 2-1 set" D1 is executed.

D1. "Feature point detection window 2-1 set" D1 (Fig. 14)
Similar to C1 (FIG. 5), but the size and position of window 2-1 is different from window 1, and the memory to which window 2-1 is applied is a corrected image memory. As shown in FIGS. 14 and 16, the window 2-1 is
It is set on the upper side of the window 1.

D2. “Feature point detection (UP)” D2 When the window 2-1 is set, the feature points in the window 2-1 on the corrected image memory are detected. This content is similar to the above-mentioned "feature point detection (UP)" C2, and in "feature point detection (UP)" D2, the "1" information representing the feature point is the binary data table E (xs, ys). However, the detection area is the window 2-1 and
That the processing target data is the data on the corrected image memory,
Is different. D3. "Left and Right White Line Detection" D3 (FIG. 15) Here, similar processing to the above-mentioned "bonnet detection" C3 is performed to detect a straight line approximating the right end white line and a straight line approximating the left end white line of the own vehicle lane. However, window 2-
1 differs from window 1 in size and position, so the polar origin is different (FIG. 16). Also, window 2-1
Is not divided into two parts, left and right. Therefore, the first Hough transform 67 that roughly detects left and right white lines is different in that it detects two points, the maximum frequency point and the next largest point. In the above “bonnet detection” C3, the detailed description of the contents of the “straight line fitting” 13R and 13L is omitted, and therefore, the contents of the “straight line fitting” 63 will be described in detail.

63. "Straight line fit" 63 (Figs. 17 & 18) This content is shown in Figs. 17 and 18. Note that the “straight line fit” 63 is applied to the entire window 2-1. Here, first, the characteristic points on the binary data table E (xs, ys) are converted into the ρ-θ polar coordinate system, and the frequency count value that the curve of each point (r, t) of this coordinate system passes The data of the data table HGn (r, t) assigned to a certain area of the image memory 5a to be written, where n = 1 in this case, is cleared (65 in FIG. 17; details are shown in FIG. 19). Next, the Hough transform parameters (transformation area and Hough transform density) are set (66 in FIG. 17), and the first Hough transform (HG1) is performed (67 in FIG. 17; details are shown in FIG. 2).
0).

"Hough Transform (HG1)" 67 (FIG. 20) Referring to FIG. In this case, the area corresponding to the window 2-1 in the one screen corresponding to the corrected image memory of the binary data table E (xs, ys), that is, the starting point is (51
It is checked whether there is a feature point (“1” information) in the area whose end point is (0,250) (1,350) (87 to 89,
95-98). Every time there is a characteristic point, the origin is (Xch = 255, Ych = 350) on the XY coordinates, θ is in the range of 0 to π, and the unit of θ is (π / 2) × ( 1/3
Position (r, t) on the polar coordinate ρ-θ of 2), t = 0 to 3
20, r is converted into a value calculated by the arithmetic expression shown in the block of step 91 of FIG.
The data of the address (n = 1 in this case) corresponding to the position (r, t) is incremented by 1 (90 to 94). As described above, since one point on the XY coordinates becomes a curve on the polar coordinate ρ-θ when converted to the polar coordinate ρ-θ, the converted position (r, t) is t = 0 to 31. 32 (corresponding to a curve) corresponding to each of the above. Since the data of the addresses assigned to these positions (r, t) in the data table HGn is incremented by 1,
The data at each address represents the frequency at which the characteristic point on the XY coordinates passes through the curve on the ρ-θ coordinates.

In step 91 of FIG. 20, r = {(xs-Xch) .cos [(t. (Te-Ts) / Td) + Ts] + (Ych-ys) .sin [(t. (Te-Ts ) / Td) + Ts] −Rs} × Rd / (Re−Rs) (1) is calculated and the calculated value r is stored in the register r. When this equation (1) is modified, it becomes the following equation (1a).

R / Rd / (Re-Rs) + Rs = (xs-Xch) .cos [(t. (Te-Ts) / Td) + Ts] + (Ych-ys) .sin [(t. (Te- Ts) / Td) + Ts] (1a) This can be expressed by the following equation (1b).

R / d + c = (xs−Xch) · cos (a · t + b) + (Ych−ys) · sin (a · t + b) (1b) where a = (Te−Ts) / Td , B = Ts, c = Rs, d = Rd /
(Re-Rs).

In the equation (1), (Xch, Ych) is the position on the XY coordinate of the polar coordinate origin, and here Xch = 255 and Ych = 350. xs is the X coordinate value of the feature point on the XY coordinate, y
s is a Y coordinate value.

Here, if θ = [(t (Te-Ts) / Td) + Ts] is expressed, (Te-Ts) / Td = by the setting in step 62 of FIG. 15 and step 66 of FIG. Since π / 32 and Ts = 0, θ = t · π / 32 = (π / 32) ta = π / 32, b = 0, c = 0, d = 1/8, and t = 0 .About.31, θ = 0, θ = π / 3
2, θ = 2π / 32, θ = 3π / 32, ..., 31π
/ 32 and π / 32 are the minimum units, and the ρ value r of 32 points is calculated. That is, in the first Hough transform, each of the feature points in the area of the window 2-1 is located at a position (r, t) on the polar coordinate plane in which θ has a minimum unit of π / 32 and is 0 or more and less than π. To be converted.

Rd / (Re-Rs) in the above equation (1) is Rd / (Re-Rs) = 32/256 by the setting in step 62 of FIG. 15 and step 66 of FIG. This is because θ = π / 32 is one unit of t, that is, π is divided into 32, and correspondingly, the range 0 to 256 of ρ is also divided into 32. That is, one unit of r is 256/32.

In summary, "Hough transform (HG1)" 67
Then, the expression (1) is specifically the following expression (1-1).

R = {(xs−255) · cos [(t · (π / 32)] + (350−ys) · sin [(t · (π / 32)]} × 1/8 ( 1-1) By transforming this into the form of equation (1b), r / d ₁ + c ₁ = (xs−Xch) ・ cos (a ₁・ t + b ₁ ) + (Ych−ys) ・ sin (a ₁・t + b ₁ ) ... (1-1b), a ₁ = π / 32, b ₁ = 0, c ₁ = 0, d ₁ =
It is 1/8.

As described above, in the first Hough transform, the unit of θ in the ρ-θ polar coordinate is coarse as π / 32, and the unit of ρ is coarse as 256/32, so that one feature point (xs , Ys) to ρ-θ polar coordinates (the number of times t is changed 32, that is, the number of times the calculation of the equation (1) is performed)
Is extremely small, the number of r data bits is small, and each feature point is ρ
-The conversion speed to polar coordinates is fast, and the frequency count processing speed is fast.

Referring to FIG. 17 again, when the CPU 1 completes the first Hough transform "Hough transform (HG1)" 67, it executes "Maximum point search (HG1)" 68.
The contents are shown in FIG.

"Maximum point search (HG1)" 68 (FIG. 21) The data (frequency) of each address (r, t) of the above-mentioned data table HGn (here, n = 1) is sequentially read (see FIG. 21). 99, 100, 105-108), the read data is compared with the data in the register Gm, and if the read data is larger, the read data is updated and stored in the register Gm, and at this time, the read data address (r, t )
Of r is updated in the register rm11 and t is updated in the register tm11 (101 to 104). If the read data is smaller than Gm, the read data is compared with the data in the register Gmn, and if the read data is larger, the read data is updated in the register Gmn and the address of the read data is read at this time. R of (r, t) is updated and stored in the register rm12 and t in the register tm12 (102A to 102C). When such processing is completed for all the addresses of the data table HGn, the register Gm has the maximum value of the frequency in the data table HGn, and the registers rm11 and tm11 have the maximum value (rm11, tm11). Will be stored. The data table H is stored in the register Gmn.
The next largest value of the frequency in Gn is the register r.
Addresses (rm12, tm1) that have the maximum value in m12, tm12
2) is stored.

Referring again to FIG. 17, the CPU 1
When the “maximum point search (HG1)” 68 is completed, the address (rm11, tm11) expressed in the rt polar coordinate system is changed to
It is converted to ρ-θ polar coordinate system address (Rm11, Tm11) corresponding to the X-Y coordinate system screen (69R). Here, Rm11 = rm11. (Re1-Rs1) / Rd1 (2) Tm11 = tm11. (Te1-Ts1) / Td1 (3) is calculated, and the calculated Rm11 is stored in the register Rm11. The calculated Tm11 is stored in the register Tm11. Similarly, the address (rm12, tm12) is set to ρ− corresponding to the screen of the XY coordinate system.
Convert to θ polar coordinate system address (Rm12, Tm12) (69
L). Step 62 of FIG. 15 and Step 6 of FIG.
According to the setting in 6, here, (Re1−Rs1) = 256 Rd1 = 32 (Te1−Ts1) = π Td1 = 32, and the expression (2) means Rm11 = 8 · rm11, and (3) The formula means Tm11 = (π / 32) · tm11. Address (Rm
Substituting (11, Tm11), (Rm12, Tm12) into the conversion formula to the XY coordinate system, the two straight lines (low line) in the area of the window 2-1 on the screen displaying the image data of the corrected image memory are displayed. A straight line detected by the density Hough transform 67: hereinafter referred to as a first detection straight line) is obtained.

Next, the CPU 1 causes the data table HGn (r, t) assigned to an area of the image memory 5a,
Here, the data of n = 2 is cleared (70 in FIG. 17).
R). Next, the second Hough transform, "Hough transform (HG
2) ”72R conversion parameter is set (7 in FIG. 17).
1R). In the above-mentioned first Hough transform, the range of θ from Ts to T
where e is 0 to π, the unit (Te-Ts) / Td is π / 32, the range Rs to Re of ρ is 0 to 256, and the unit Rd / (R
Although e−Rs) is set to 1/8, here, the range of θ and ρ is set to a small range centered on the above (Rm11, Tm11), and the units of θ and ρ are set small. Specifically, Rs = 8 (rm11-2) Re = 8 (rm11 + 2) Rd = 32 Ts = (π / 32) (tm11-2) Te = (π / 32) (tm11 + 2) Td = 32 are set. . Note that rm11 = Rm11 / 8, tm1 = Tm11 /
(π / 32). And "Hough conversion (HG2)" 72R
Perform

"Hough Transform (HG2)" 72R The content of this "Hough Transform (HG2)" 72R is the same as that of the "Hough Transform (HG1)" 67 described with reference to FIG. , N = 2, and block 91
The contents of the arithmetic expression ((1) above) are different. Step 7
Based on the parameters set in 1R, the arithmetic expression of step 91 of FIG. 20, that is, the above equation (1) is
It becomes the formula 2).

R = {(xs−255) · cos [t · (π / 256) + π (tm11-2) / 32] + (350−ys) · sin [t · (π / 256) + π (tm11− 2) / 32] -8 (rm11-2)} × (1/1) ・ ・ ・ (1
-2) which (1b) was modified into the form of _{equation, r / d 2 + c 2} = (xs-Xch) · cos (a 2 · t + b 2) + (Ych-ys) · sin (a 2 · t + b ₂ ) When represented by (1-2b), in the first conversion described above, a ₁ = (π / 32), b ₁ = 0, c ₁ = 0,
For d ₁ = 1/8, a ₂ = (π / 256), b ₂ = π (tm11-2) / 32, c ₂ = 8 (rm11
−2), d ₂ = 1.

Since t = 0 to 31, θ = π (tm11-2) / 32, θ = (π / 256) + π (tm11-2) / 32, θ = 2 (π / 256) + π (tm11 −2) / 32, θ = 3 (π / 256) + π (tm11-2) / 32, ··· θ = 31 (π / 256) + π (tm11-2) / 32, and (π / 256) Ρ corresponding to 32 points θ with the minimum unit
The value r is calculated. That is, in the second Hough transform, each of the feature points in the area of the window 2-1 has θ of (π / 256) as the minimum unit (one unit of t is π / 256), and θ = π (tm
It is converted to a position (r, t) on the polar coordinate plane within the range of 11-2) / 32 or more and θ = 31 (π / 256) + π (tm11-2) / 32 or less.
The range of ρ is 8 (rm11-2) or more and 8 (rm11 + 2) or less, and the unit of ρ is to divide this range into 32 parts. That is, one unit of r is 32/32 = (1/1) = 1. Therefore, the second Hough transform (72R) is the window 2-1.
Is converted into polar coordinates having a narrower range and higher density than the first Hough transform (67). Feature point 1
In one polar coordinate conversion, the second Hough transform (72R) also performs t = 0 to 31, that is, 32 times of calculation, so that the processing time is approximately the same as that of the first Hough transform (67). The number of operations for converting one feature point (xs, ys) into ρ-θ polar coordinates is extremely small, the number of r data bits is small, the speed of converting each feature point into ρ-θ polar coordinates is high, and the frequency is The count processing speed is fast.

Referring to FIG. 17 again, when the CPU 1 completes the second Hough transform "Hough transform (HG2)" 72R, it executes "Maximum point search (HG2)" 73R. The contents are the above-mentioned “maximum point search (HG1)” 68.
Is the same as However, n = 2. When this process ends, the maximum value of the frequency in the data table HGn is stored in the register Gm, and the address (rm2, tm2) having the maximum value is stored in the registers rm2 and tm2.

Referring again to FIG. 17, the CPU 1
When the "maximum point search (HG2)" 73R ends, r-t
The address (rm2, tm2) expressed in polar coordinate system,
It is converted into ρ-θ polar coordinate system address (Rm2, Tm2) corresponding to the XY coordinate system screen (74R). The calculation formula is Rm2 = rm2 (Re2-Rs2) / Rd2 + Rs2 (4) Tm2 = tm2 (Te2-Ts2) / Td2 + Ts2 (5) The calculated Rm2 is stored in the register Rm2 and the calculated Tm2 is stored.
Is stored in the register Tm2 (74R). According to the setting in step 71R of FIG. 17, here, (Re2-Rs2) = 32 Rd2 = 32 (Te2-Ts2) = π / 8 Td2 = 32, and the equation (4) is, specifically, Rm2 = Rm2 + 8rm11-16 (4-1), and the equation (5) is specifically expressed as follows: Tm2 = tm2 · (π / 256) + (π / 32) · (tm11-2) Means (5-1). By substituting the address (Rm2, Tm2) into the conversion formula to the XY coordinate system, one straight line in the area of the window 2 on the screen displaying the image data of the corrected image memory (with the medium density Hough transform 72R The detected straight line: hereinafter referred to as the second detected straight line) is obtained.

Next, the CPU 1 causes the data table HGn (r, t) assigned to a certain area of the image memory 5a,
Here, the data of n = 3 is cleared (75 in FIG. 17).
R). Next, the third Hough transform, "Hough Transform (HG
3) ”77R conversion parameter is set (7 in FIG. 18).
6R). Here, the range of θ and ρ is set to the above (Rm2, Tm
2) centered on a smaller range, and θ and ρ
The unit of is also set small (76R). Specifically, Rs = rm2 + 8rm11-18 Re = rm2 + 8rm11-14 Rd = 32 Ts = (π / 256) tm2 + (π / 32) tm1-9π / 128 Te = (π / 256) tm2 + (π / 32) tm1 Set -7π / 128 Td = 32. It should be noted that rm2 and tm2 are defined by the above equations (4-1) and (5-1) with respect to Rm2 and Tm2. Then, "Hough conversion (HG3)" 77R is performed.

"Hough Transform (HG3)" 77R This "Hough Transform (HG3)" 77R has the same contents as the above-mentioned "Hough Transform (HG2)" 72R, but FIG.
In the Hough transform process shown therein, the point that n = 3 is different from the content of the arithmetic expression (the above-mentioned expression (1)) in the block 91 in FIG. Based on the parameters set in step 76R,
The arithmetic expression of step 91 of FIG. 20, that is, the expression (1) is
In this “Hough transform (HG3)” 77R, specifically, the following equation (1-3) is obtained.

R = {(xs−255) · cos [t · (π / 2048) + (π / 256) tm2 + (π / 32) tm11−9π / 128] + (350−ys) · sin [t · (π / 2048) ＋ (π / 256) tm2 ＋ (π / 32) tm11−9π / 128] −rm2−8rm11 + 18} × 8 ・ ・ ・ (1-3) Change this into the form of equation (1b) , expressed in _{= r / d 3 + c 3} (xs-Xch) · cos (a 3 · t + b 3) + (Ych-ys) · sin (a 3 · t + b 3) ··· (1-3b), above In the first and second conversions of a ₁ = (π / 32), b ₁ = 0, c ₁ = 0,
_{d 1 = 1/8 a 2} = (π / 256), b 2 = π (tm11-2) / 32, c 2 = 8 (rm11
−2), d ₂ = 1 and a ₃ = (π / 2048), b ₃ = (π / 256) tm 2 + (π / 32) tm 11-9
_{π / 128, c 3 = rm2} + 8rm11-18, a d ₃ = 8.

Since t = 0 to 31, θ = (π / 256) tm2 + (π / 32) tm11−9π / 128, θ = (π / 2048) + (π / 256) tm2 + (π / 32) tm11-9π / 128, θ = 2 (π / 2048) + (π / 256) tm2 + (π / 32) tm11-9π / 128, θ = 3 (π / 2048) + (π / 256) tm2 + (π / 32) tm11-9π / 128, ··· θ = 31 (π / 2048) + (π / 256) tm2 + (π / 32) tm11-9π / 128, and (π / 2048) as the minimum unit, 32 points Θ (t = 0-
31) is calculated corresponding to ρ value r. That is, in the third Hough transform, each feature point of the window 2-1 is
Is the minimum unit (1 unit of t is π / 2048),
And θ = (π / 256) tm2 + (π / 32) tm11-9π / 128 or more θ
= 31 (π / 2048) + (π / 256) tm2 + (π / 32) tm11-9π / 128
It is converted to the position (r, t) on the polar coordinate plane in the following range. The range of ρ is Rs = rm2 + 8rm11-18 or more Re = r
m2 + 8rm11-14 or less, and the unit of ρ divides this range into 32 parts. That is, one unit of r is 4/32 =
(1/8). Therefore, the third Hough transform (77
R) uses the second Hough transform (7
2R), which is converted into polar coordinates with a narrower range and higher density. In the polar coordinate transformation of one feature point, since t = 0 to 31 in this third Hough transformation (77R), 32 times of calculations are performed. Therefore, the second Hough transformation (72R) is performed.
R) and processing time are comparable. One feature point (xs,
The number of calculations for converting (ys) into ρ-θ polar coordinates is extremely small,
The number of r data bits is small, the speed of converting each feature point into ρ-θ polar coordinates is high, and the frequency count processing speed is high.

Referring to FIG. 18, when the CPU 1 completes the third Hough transform "Hough transform (HG3)" 77R, it executes "Maximum point search (HG3)" 78R.
The content is the same as that of the above-mentioned "maximum point search (HG1)" 68. However, n = 3. When this process ends, the maximum value of the frequency in the data table HGn is stored in the register Gm, and the address (rm3, tm3) having the maximum value is stored in the registers rm3 and tm3.

Referring to FIG. 18, when the "maximum point search (HG3)" 78R is completed, the CPU 1 changes the address (rm3, tm3) expressed in the rt polar coordinate system to XY.
Ρ-θ polar coordinate system address (Rm3
1, Tm31) (79R). The calculation formula is Rm31 = rm3 (Re3-Rs3) / Rd3 + Rs3 (6) Tm31 = Tm3 (Te3-Ts3) / Td3 + Ts3 (7) The calculated Rm31 is stored in the register Rm31, and the calculated T
The m31 is stored in the register Tm31 (79R). By the setting of step 76R of FIG. 18, here, (Re3−Rs3) = 4 Rd3 = 32 (Te3−Ts3) = π / 64 Td3 = 32, and the equation (6) is, specifically, Rm31 = (1/8) ・ rm3 + rm2 + 8rm11-18 ・ ・ ・
(6-1), and the equation (7) is specifically expressed by Tm31 = (π / 64) · tm3 + (π / 256) tm2 + (π / 32) tm11-9π / 128 (7- 1) means By substituting the address (Rm31, Tm31) into the conversion formula to the XY coordinate system, one straight line (detected by the high-density Hough transform 77R) of the window 2-1 on the screen displaying the image data of the corrected image memory is detected. Straight line: third below
An equation showing a straight line of detection) is obtained.

As described above, the two straight lines (rm11, tm11) and (rm12, T) detected by the rough Hough transform (67) in the window 2-1.
It means that the first straight line (rm11, tm11) of (m12) is detected,
This straight line is the right end white line or the left end white line. In the same manner as the above steps 70R to 79L, the second straight line (rm12, Tm1
2) to detect the second straight line finely (7 in FIG. 18).
0L-79L).

As described above, the "straight line fitting" 63 shown in FIG.
(Contents: Fig. 17 & Fig. 18) has been completed, and we have obtained a linear equation that expresses two straight lines (the most representative straight line and the next representative straight line) in the image in the window 2-1 area. become. The window 2-1 is set as the most suitable area for detecting the left and right white lines of the area immediately adjacent to the own vehicle in the own vehicle lane, and there are images of the right end white line and the left end white line in the window 2-1 area. Then, the straight line of the third detection has a high probability of being a straight line approximating this white line. Therefore, the "straight line fitting" 63 is the detection of the left and right white lines of the vehicle lane.

Referring again to FIG. 15, the CPU 1 then outputs data (Rm31, Tm31) representing the detected two straight lines,
The inclination angles of (Rm32, Tm32) are compared (64A), the upper left straight line is stored in the right straight line register (RmR, TmR), and the right upper left straight line is stored in the left straight line register (RmL, TmL) (6
4B, 64C). The "left and right white line detection" D3 is thus completed. The CPU 1 then performs "infinity point calculation" D4.
This content is shown in FIG.

D4. "Point calculation at infinity" D4 (Fig. 22) First, in "Line intersection calculation" 109, register R
An intersection (xc,) between a straight line (estimated to be the right end white line of the own vehicle lane) represented by mR and TmR data and a straight line (estimated to be the left end white line of the own vehicle lane) represented by data of registers RmL and TmL
yc) is calculated. Next, the calculated intersection (xc, yc)
Exists in an area of 60 pixels in the horizontal direction and 60 pixels in the vertical direction centered on the point at infinity (Xv, Yv) obtained by weighting smoothing (averaging) the intersection data calculated in the past in time series. Is checked (110, 111). Within this area, the intersection point (xc, yc) obtained this time is highly reliable because it is the point at infinity, so the infinity point data (Xv, Yv) is set to 1 at the intersection point (xc, yc) obtained this time. / 8 weighting,
The infinite point data (Xv, Yv) so far is weighted by 7/8 and updated to a value added (112). Then, the register Nve for counting the number of times the tracking of the point at infinity has failed is cleared (113).

If the intersection point (xc, yc) calculated this time is not within the area of 60 pixels in the horizontal direction and 60 pixels in the vertical direction, tracking of the point at infinity is unsuccessful. Assuming that (Xv, Yv) is an error, the content of the register Nve is incremented by 1 (11
4) It is checked whether the content of the register Nve has reached 5 (115). When it is 5, this means that the intersection calculation was in error five times in total, this time and the past four times. This is because the infinity point data (Xv, Yv) currently on hold is an error. Infinite point data (Xv, Y
v) is updated to the intersection (xc, yc) calculated this time (1
16).

D5. "White line spacing (WL) calculation" D5
(FIGS. 13 & 23) Referring again to FIG. 13, the CPU 1 next calculates the interval (lane width) WL between the right white line and the left white line of the vehicle lane (D5). In this case, the rightmost white line (RmR, TmR) on the screen at the position where the center line of the field of view of the camera 6b (two-dot chain line in FIG. 23) intersects the road surface (the center point of the screen on the screen)
The X position of is converted to the road position XR, and the X position of the leftmost white line (RmL, TmL) on the screen is converted to the road position XL.
The interval WL = XR-XL is calculated. Note that Sx and Sy in the operation block D5 of FIG.
Horizontal and vertical scale factors of Hc
Is the height of the center of the lens 9 of the camera 6b from the road surface, as shown in FIG.

Next, the CPU 1 checks whether the calculated lane interval WL on the road surface is correct (success in detecting the own vehicle lane) (D6). That is, it is checked whether the deviation of WL from the reference value WL3D is within the allowable range DWL. In this example, the highway lane width of Japan is 3.5 ± 0.2.
m, the reference value WL3D = 3.5m, and the allowable value DWL
= 0.3m.

If the deviation of WL with respect to the reference value WL3D is within the allowable range DWL, it is determined that the own vehicle lane has been successfully detected, and information "1" indicating this is written in the register FL (61) to the reference value WL3D. The deviation of WL is the allowable range D
If it is out of WL, it is considered that the own vehicle lane detection has failed, and information "0" indicating this is written in the register FL (62). As described above, the "vehicle lane detection" D shown in FIG. 3 is completed, and when the vehicle lane detection is successful, the data in the register FL is "1".

Referring again to FIG. "Own vehicle lane detection"
When D is finished, the CPU 1 checks the data in the register FL and if it is “1”, “adjacent lane estimation”
Execute F. This content is shown in FIG.

F. "Adjacent lane estimation" F (Fig. 24 & 2
5) Referring to FIG. 24, here, the downward angle α of the camera 6b is first calculated (117). In the calculation formula of step 117, Yo is the Y coordinate value of the screen center, and Ych is “screen calibration”.
It is the Y coordinate value of the lower edge of the screen defined in “Calculation of straight line intersection” C4. Next, the angle φR formed by the right end white line and the angle φL formed by the left end white line of the vehicle lane with respect to the traveling direction of the vehicle are calculated, and the average value φ thereof is calculated. On the screen (image in the corrected image memory) that has completed the “screen calibration” C, the previous “screen calibration” C indicates that the vehicle traveling direction is aligned with the center line (Y = 255) that bisects the screen left and right. Therefore, φR is the angle at which the right end white line intersects with this line, and φL is the angle at which the left end white line intersects. As shown in FIG. 25, the straight line of the angles of the average values φ is the center line of the own vehicle lane. In the next step 119, formulas for calculating the distance XR of the right end white line and the distance XL of the left end white line to the point where the center line of the field of view of the camera 6b intersects the road surface are shown. XR and XL
Is calculated in "calculation of the space between the left and right white lines (WL)" D5, and the calculated value is used. Next, regardless of the presence or absence of the right end white line of the right side lane (right adjacent lane) of the own vehicle lane right end white line, it is assumed that this exists, and the point of XR is laneed to the right in the horizontal direction. Assuming that the straight line connecting the point having the width WL (calculated in D5) and the point at infinity is the right end white line of the right lane, the value Aex1 of the proportional term coefficient A of the formula Y = AX + B representing the straight line (Ta of the angle TmRR formed with the horizontal line (X axis)
Then, the value Bex1 of the constant term B is calculated (n value) (12)
0). Similarly, the value Aex representing the leftmost white line of the left lane
Calculate 2 and Bex2 (121). Then, these values Aex1, Bex1 and Aex2, Bex2 are converted into ρ-θ polar coordinates (122, 123). As described above, two straight lines defining the left adjacent lane and the right adjacent lane are additionally determined in addition to the white lines (two straight lines) at both ends of the own vehicle traveling lane.

Referring again to FIG. "Adjacent lane estimation"
When F is completed, the CPU 1 executes "own vehicle lane preceding vehicle recognition and distance measurement" H. This content is shown in Figure 2.
8 shows.

H. "Self-vehicle lane preceding vehicle recognition and distance measurement"
H (FIGS. 28 & 29 to 46) First, "detection of candidate vehicle position 1" H1 is executed. This content is shown in FIG.

H1. "Detection 1 of vehicle candidate position" H1 (Fig. 29) First, in "feature point memory clear" 131, a multi-gradation data of the number of pixels corresponding to the number of pixels (addresses) in the corrected image memory.
Data memory D (x, y) and binary data memory E (x,
y) is cleared. Details of the processing are shown in FIG. Next, “horizontal feature point detection 1” 132 is executed. This content is shown in FIG.

"Detection of Horizontal Feature Point 1" 132 Referring to FIGS. 31 and 26, here, a horizontal line of Y = 350 (X passing Y = 350 on the screen shown in FIG. 26 is displayed.
The axis parallel line is the lower side, and the white lines at the left and right ends of the vehicle lane are Y
= 230 (in window 2-2: refer to FIG. 26), a trapezoidal window area having a horizontal line connecting two points intersecting with it as a top side (area shown by vertical lines in FIG. 26 = own vehicle lane area)
Of the left half region defined by the Y-axis parallel line of X = Xvc, the inside of the left half region is scanned from the bottom to the top in the Y direction, and this scan is repeated while updating the X position. (Fig. 31
146L to 150L, 152L to 153L), the Y-direction differential value of the image data on the corrected image memory is calculated, and a point having a differential value equal to or larger than the threshold value is detected as a feature point (151).
L). In other words, "1 line horizontal feature point detection" 151L is executed. This content is shown in FIG. This processing 151L
Is very similar to the “feature point detection (UP)” C2 shown in FIG. 6, but steps 154A and 15
4B has been added.

At step 154A, the flag S_night is set.
By referring to, it is discriminated whether it is day or night, and S_night
If = 1, it means that it is nighttime, and therefore the following step 154B is executed. Step 154B is not executed during the daytime. In step 154B, the position of the window (trapezoidal region) to be referred to is shifted in the y direction by correcting each of the parameters ys and ye indicating the y coordinate. Actually, the following processing including calculation is executed.

Ys ← ys- (ys-YV) TL / Hc (8-1) ye ← ye- (ye-YV) TL / Hc (8-2) However, YV : Y-coordinate of the point at infinity TL: Height correction amount (45 cm: equivalent to the ground height of the tail lamp) Hc: Ground height at the position where the TV camera 6b is installed The arrow means substitution of calculation results in that direction. As shown in FIG. 16, when at least a part of the vehicle (preceding vehicle) is present in the window 2-1, a rear image of the preceding vehicle appears in the window 2-1 and black on the road surface below the rear image. A shadow appears. Therefore, in the daytime, the shadow of the preceding vehicle on the road surface is first detected. On the other hand, at night, as shown in FIG. 63, for example, the contour of the vehicle body of the preceding vehicle does not appear in the image,
Only the part that emits light like a tail lamp can be detected. Therefore, at night, the position of the tail lamp of the preceding vehicle is first detected. As will be described later, the shadow of the preceding vehicle on the road surface that appears in the daytime and the trail of the preceding vehicle that appears at night.
Objects such as lulump can be detected with a very similar algorithm. However, since the position (height) of the target object differs between daytime and nighttime, the position of the target object in the image also changes (see FIG. 23). Therefore, in this embodiment, the position in the height direction of the window to be processed is switched between daytime and nighttime by the processing of steps 154A and 154B.

In the processing shown in FIG. 31, next, the right half region of the vehicle lane region divided by the Y-axis parallel line X = Xvc is defined, and the inside of the right half region (X = Xvc is included) is scanned from bottom to top in the Y direction, and this scanning is repeated while updating the X position (steps 146R to 150R, 1).
52R to 153R), the Y-direction differential value of the gradation of the image data on the corrected image memory is calculated, and a point whose differential value is equal to or greater than the threshold value is detected as a feature point (151R).

As described above, the characteristic points in the own vehicle lane area are detected, and the binary data memory E (x,
"1" indicating that the point is a feature point is written in the address corresponding to the feature point in y). In addition, in the above-mentioned "feature point memory clear" 131, the binary data memory E
Clear (x, y) in advance (“0” for all addresses)
Writing), so on the binary data memory E (x, y),
"1" (which is a feature point) is written only in the address corresponding to the feature point in the own vehicle lane area.

Next, the CPU 1 clears the histogram memory (133 in FIG. 29). This content is shown in FIG. The CPU 1 then performs “y-direction histogram creation” 13
Do 4. This content is shown in FIG.

“Y-direction histogram creation” 134 (FIG. 3)
4) Here, Y = 0 on the binary data memory E (x, y).
Up to Y = 511 X-axis parallel lines (X scanning lines; a series of pixels in the X direction), the number of "1" (characteristic points) on the line is counted, and the histogram memory H
The count value is written in (y) in correspondence with the scanning line (Y address).

[Detection of maximum point in y-direction] 135 (FIG. 35) From the address of Y = 511 (X-axis scanning line of Y = 511) of the histogram memory H (y) to Y = 1 in sequence, each address is the target line. (Y), the characteristic point count value H (y) of the attention line (y), the characteristic point count value H (y-1) of the scanning line (y-1) whose Y coordinate value is one smaller than that, Also, the feature point count values H (y + 1) of the scanning line (y + 1) whose Y coordinate value is one larger than the line of interest (y) are compared (174 and 175 in FIG. 35), and H (y) becomes H (y -1)
And H (y + 1), that is, if the attention line (y) is the peak position of the number of characteristic points, the left end point of the own vehicle lane at the position (Y address) of the attention line (y). (X
1) and the right end point (X2) are calculated (176, 177; the calculation formula is shown in FIG. 36). Then, the threshold Th2 is set to (X2
If -X1) / 8 is set (178) and the feature point count value H (y) of the attention line (y) is greater than or equal to the threshold value Th2, the image contour extending in the horizontal direction on the attention line (y). Assuming that there is a line, the address (Y coordinate value) of the line of interest (y) is written to the address N of the memory My (N) (180). Then, the register N is incremented by 1. When the value of the register N becomes 5 or more (the image contour line is detected by 6 different scanning lines), or when the above-mentioned processing with the last scanning line Y = 1 as the line of interest is completed, the "maximum inspection in the y direction" is performed. Out "13
Finish 5

In the daytime, as shown in FIG. 16, when there is at least a part of the vehicle (preceding vehicle) in the window 2-1, a rear image of the preceding vehicle appears in the window 2-1.
A shadow (black) appears on the road surface below the rear image, and a bumper (the lower edge appears as a horizontal line with a particularly high contrast), number plate, and blur on the rear image of the preceding vehicle.
Since there are many image contour lines extending in the lateral (X) direction, such as the lamp panel, the rear window, and the roof, etc., they are close to the own vehicle (the Y coordinate value is large, at addresses 0 to 5 of the memory My (N). ) 6
The position (Y coordinate) of each image contour line is written. Further, if there is a preceding vehicle or an oncoming vehicle in the window at night, the image contour line of the tail lamp or the head lamp portion which is lit is detected, and the Y of that position is detected.
The coordinates are written in the memory My (N).

In step 178, the threshold value Th2
Is set to (X2-X1) / 8, the closer to the camera 6b, the higher the contrast of the image contour line and the more feature points are extracted. However, the contrast of the farther image contour line is low, and the feature point is extracted. This is because the distance from the camera 6b of the image contour on the line of interest (y) is estimated from (X2-X1) and the threshold value is changed according to the distance.

[Calculation of Center of Gravity in X Direction] 136 (FIG. 37) Next, for each position (Y coordinate value) of the image contour line written in the memory My (N), W1 (set value) smaller than the position. W2 (setting value) The X coordinate value of the characteristic point on the binary data memory E (x, y) (the characteristic point in the own vehicle lane area filled with the vertical line in FIG. 27) in the large area is set. The value is added to the accumulation register Sumx, the number of additions is counted by the register Nx (187 to 197), the X-coordinate value of the center of gravity (average coordinate value) is calculated, and written in the memory Mx (N) (198).
As a result, the “y-direction maximum point detection” 135 detects
Center-of-gravity position in the X direction of each image contour line extending in the lateral direction (X
The coordinate value) exists in the memory Mx (N). The Y-direction center-of-gravity position exists in the memory My (N).

With the above, the "detection of candidate vehicle position" H1 shown in FIG. 28 (details are shown in FIG. 29) is finished, and the CPU 1 next clears a register NR to be used later (H2),
"Detection of left and right ends of vehicle" H3 is executed. This content is shown in Figure 3.
8 shows.

H3. "Detection of left and right ends of vehicle" H3 (Fig. 3
8) Here, first, "vehicle left and right end position estimation" 201 is executed. The contents are shown in FIG.

"Vehicle left and right edge position estimation" 201 (FIG. 39) First, in step 208A, day / night is identified with reference to the flag S_night, and step 2 is performed during daytime according to the result.
08B is executed, and step 208C is executed at night.
In step 208B or 208C, the memory My (N)
With the largest Y coordinate value of the image contour line in
(NR) (data of address N = NR = 0), that is, the Y position of the image contour line (horizontal line) of the front object closest to the own vehicle,
Is read out and the horizontal distance (distance on the road surface) L1 from the camera 6b to the camera 6b is calculated by the following equation.

Daytime: L1 = Sy.Hc / (My (NR) -YV) (9-1) Nighttime: L1 = Sy. (Hc-TL) / (My (NR) -YV) (9-2) However, Sy: focal length of TV camera Hc: ground height of TV camera TL: ground height of tail light (45 cm) YV: y-coordinate of infinity point My (NR): horizontal Y Coordinate of Contour Line In this embodiment, the installation state of the television camera 6b and the image plane 10 thereof have the positional relationship shown in FIG. Therefore, detecting the shadow appearing below the preceding vehicle in the daytime is equivalent to detecting the position of the point P shown in FIG.
The y coordinate of the horizontal contour line corresponding to the point P is stored in the memory My (NR).
Is held in. As shown in FIG. 23, assuming that the center of the optical axis of the television camera 6b is inclined by an angle α with respect to the horizontal line and the inclination between the optical axis center and the axis toward the point P is θ, The relationship is established. tan (α-θ) = Hc / L1 (9-3) tan (θ) = (256-My (NR)) / Sy (9-4) Therefore, (9- From equations 3) and (9-4), L1 = Hc · (Sy + B · tan (α)) / (Sy · tan (α) −B) (9−
5) However, B: (256-My (NR)) tan (α) = (256-YV) / Sy (9-6) where α-θ and θ are very small The following relationship holds.

Α-θ = Hc / L1 (9-7) θ = (256-My (NR)) / Sy (9-8) α = (256-YV) / Sy (9-9) Therefore, in the daytime, the distance L1 can be obtained by the above equation (9-1). Further, when the object to be detected is a tail lamp section (at night), if the ground clearance of the tail lamp section is TL, Hc in the above equation (9-1) is replaced with Hc-TL to obtain the above-mentioned value. Since the same relational expression as the case is established, the distance L1 can be obtained by using the expression (9-2) at night.

In the processing shown in FIG. 39, next, assuming that the front object is a vehicle (the correctness of this assumption is verified by "vehicle verification" H4 described later), the vehicle width W on the screen is displayed.
V2Dp and vehicle height HV2Dp are calculated (209). WV3D used for this calculation is a register WV that stores the vehicle width learning value.
3D (described later with reference to FIG. 46) data (a constant when the preceding vehicle has not been detected in the past: the most representative vehicle width of 1.7 m), and HV3D is a minicar. Also cover
The vehicle height is 1.2m. Next, the vehicle width WV2Dp on the screen
And the barycentric position Mx of the image contour line in the memory Mx (N)
From (NR), the left and right end positions XVLP, X on the screen of the front object
The VRP is estimated and calculated (210). Then, the upper and lower edges yVHP and yVLP on the screen are estimated and calculated (211).

Next, the multi-gradation data memory D (x, y) and the binary data memory E (x, y) are cleared (202).
L), "Left edge vertical feature point detection" 203L is executed. This content is shown in FIG.

"Left edge vertical feature point detection" 203L (see FIG. 4)
0) -WV2Dp / 4 to + WV2Dp / centered on the left end position XVLP on the screen between the upper and lower ends yVHP and yVLP on the screen.
The multi-gradation data is calculated by calculating the X-direction differential value of the image data on the corrected image memory in the rectangular window area in the range up to 4.
Data is written in the data memory D (x, y) and the multi-gradation data is compared with the threshold value Th3. If it is Th3 or more, binary data is written.
Write "1" to the memory E (x, y). This feature point detection processing is performed by the "feature point detection (UP)" C2 described above.
The processing is the same as that described above. Next, the CPU 1 clears the histogram memory (H (y) used previously) (204
L), “Create x-direction histogram” 205L is executed. This content is shown in FIG.

"Creation of x-direction histogram" 205L (FIG. 41) The characteristic points on the X-axis parallel line (consecutive of pixels) in the area where the characteristic point is detected by the "left-end vertical characteristic point detection" 203L are expressed in line units. Are counted and written in the histogram memory (herein denoted as H (x)) in correspondence with the line address (X coordinate value). Then, "detection of maximum point in x direction" 206L is executed. The contents are shown in FIG.

"Detection of maximum point in x direction" 206L (FIG. 42) This process is basically the same as the process of "detection of maximum point in y direction" 135 described above, except that the Y axis is replaced with the X axis.
This is performed in the area where the feature point is detected in "left edge vertical feature point detection" 203L. However, since the size of the detection target region is different, the threshold value Th4 for extracting the maximum point is determined as follows. Daytime, that is, the flag S_night is 0
In the case of, the constant HV2Dp / 4 is used as the threshold value Th4, and at night, that is, when S_night is 1, HV2Dp
A constant p / 24 is used as the threshold Th4. In other words, in the daytime, the entire body of the preceding vehicle appears in the image, so a relatively large value is obtained as the feature amount of the contour line of the vehicle body in the vertical direction (vertical direction). Since only the vertical contour line of the tail lamp portion or the head lamp portion that is lit is detected, a value that is relatively large in the daytime and smaller than that in the nighttime is used as the threshold value Th4.

Further, every time the maximum point is detected, the feature point count number of the maximum point is compared with the data of the maximum value memory Hm,
If the maximum point detected this time is larger, this is stored in the maximum value memory H
m is updated and written, and the X coordinate value is stored in the maximum point register Xm.
Write to. When this process is completed for all the regions, the X coordinate value of the Y-axis parallel line (the left edge of the preceding vehicle) having the most feature points in the region is stored in the register Xm. . This X coordinate value is written in the vehicle left end position register XVL (207L).

[Feature point memory clear] 202L-
The same process as "Maximum point detection in x direction" 206L is performed for a rectangular window area in the range from -WV2Dp / 4 to + WV2Dp / 4 with the right end position XVRP between the upper and lower ends yVHP and yVLP on the screen as the center. Go (202R-2 in FIG. 38)
06R), the X coordinate value of the X-axis parallel line (right edge of the preceding vehicle) having the most feature points in the area is written in the vehicle right end position register XVR (207R). FIG. 43 shows the contents of “Detect right vertical feature point” 203R.

The "detection of the left and right ends of the vehicle" H3 is thus completed. The CPU 1 then executes "vehicle verification" H3.
The contents are shown in FIG.

H4. "Vehicle verification" H4 (Fig. 44) Here, the register WV3D (Fig. 4) that stores the vehicle width learning value is stored.
The vehicle width WV3D of the preceding vehicle held by
Is converted to a vehicle width WV2D on the screen by using the distance L1 of the front object from the own vehicle calculated by the "vehicle left and right end position estimation" 201 and the scale factor Sx of the camera 6b (247),
Vehicle width W on the screen from the above-mentioned vehicle left and right end positions XVL, XVR
V is calculated (248), and the difference WV-WV2D between them is ± WV
It is checked whether it is within the range of 2D / 4 (249). That is, the vehicle width calculation value WV on the screen is the vehicle width (actual width) WV2D on the screen of the register WV3D storing the vehicle width learning value.
And check if it substantially matches. Information "1" / "0" indicating detection / non-detection of a preceding vehicle when the vehicle substantially matches
Write "1" to the register FVM that stores
a). If they do not match, the register FVM is cleared (250
b). That is, when the preceding vehicle can be recognized, the register F
“1” is set in VM.

Next, when the CPU 1 writes "1" in the register FVM, the CPU 1 proceeds to "inter-vehicle distance calculation" H6 described later, but when "0" is written in the register FVM, the register NR is incremented by 1. (H5-H8-H9) or "Detect left and right ends of vehicle" H3. The "detection of the left and right ends of the vehicle" H3 is executed from the lateral contour line of NR = 0 (closest to the own vehicle). , 2, ..., The second, third, ..., And the target lateral contour lines are sequentially changed to those farther from the vehicle. Since the maximum value of NR is 5 (only 5 horizontal contour lines are detected at N = 0 to 4 in FIG. 35), a maximum of 5 times "detection of left and right ends of vehicle" H3 is executed and still FVM
If the value does not become "1", the process proceeds to "vehicle width learning calculation" H7 described later, and "vehicle distance calculation" H6 described below is not executed.

H6. "Vehicle distance calculation" H6 (Fig. 45) When "1" is written in the register FVM, the vehicle distance L
2 (0) is calculated (251). The vehicle width (actual width) WV3D of the register WV3D that stores the vehicle width learning value, the vehicle width calculation value WV on the screen, the horizontal distance L1 of the front object calculated from the “vehicle left and right end position estimation” 201 from the vehicle (on the road) Using the actual distance) and the scale factor Sx of the camera 6b, the inter-vehicle distance L2 (0) is calculated (251). This time calculated value L2
(0), the previously calculated value L2 (1), the two-previously calculated value L2
(2) and the two-previously calculated value L2 (3) is calculated and the average value is written to the register L3 (252), and the previous calculated value L2 is calculated.
(1), the two-preceding calculation value L2 (2) and the two-preceding calculation value L2 (3) are updated (253). As a result, the inter-vehicle distance (latest time-series average value) from the preceding vehicle in the own vehicle lane is stored in the register L3.

In the calculation of step 208B or 208C in FIG. 39, the inter-vehicle distance L1 is obtained only based on the difference in the vertical position of the object corresponding to the preceding vehicle photographed by the television camera. The obtained inter-vehicle distance L1 may include a relatively large error. For example, even if the actual inter-vehicle distance is constant, the required inter-vehicle distance L1 may change if the pitching attitude of the vehicle changes. In order to increase the measurement accuracy of the detected inter-vehicle distance, "inter-vehicle distance calculation" H6
The inter-vehicle distance L2 is calculated based on the detected vehicle width of the preceding vehicle. Further, in order to increase the reliability of the inter-vehicle distance L2,
The "vehicle width learning calculation" H7 described below is carried out.

H7. "Vehicle width learning calculation" H7 (Fig. 46) CP is updated when the inter-vehicle distance in register L3 is updated as described above.
U1 converts the vehicle width WV on the screen into the actual vehicle width Wva, and the data of the register WV3D (the vehicle width learning value WV3D held on the time axis until the current inter-vehicle distance calculation) and the current vehicle width The calculated value Wva is added with weighting of 47 to 1, and the value obtained by dividing the sum by 48, that is, the weighted average value is updated and stored in the register WV3D (254, 255a, 255b). When the vehicle preceding the preceding vehicle lane is not detected (FVM = "0") and the "vehicle width learning calculation" H7 is entered, the register WV3
In this embodiment, the vehicle width value 1.7 m of the most typical vehicle (passenger vehicle) is written in D (254, 256).

[0130] The "own vehicle lane preceding vehicle recognition and distance measurement" H is thus completed. Next, the CPU 1 executes "right adjacent lane preceding vehicle recognition and distance measurement" I. This content is shown in FIG.
Shown in In the above-mentioned "left and right white line detection" D3, the data (TmR, RmR) representing the right end white line of the own vehicle lane and the data (TmL, RmL) representing the left end white line are stored in the register, and , "Adjacent lane estimation" F, data (TmRR, RmRR) representing the right end white line (estimation) of the right adjacent lane and data (TmRR, RmRR) representing the left end white line (estimation) of the left adjacent lane. Note that TmLL and RmLL) are stored in the register.

I. "Right adjacent lane preceding vehicle recognition and distance measurement" I (Fig. 47, Fig. 48, Fig. 49) Here, the lower side of the window 2 (Y = 350), the vehicle lane right end white line (TmR, RmR) and In the area (right adjacent lane area) surrounded by the white line (TmRR, RmRR) on the right end of the right adjacent lane, the above-mentioned "recognition and distance measurement of preceding vehicle lane preceding vehicle" is performed.
The process of H is similarly applied to detect an object in the right adjacent lane area in front of the own vehicle, and calculate the distance from the own vehicle. Further, as in the case of "recognition and distance measurement of preceding vehicle lane preceding vehicle" H, the position of the window 2 to be referred to in the y direction is automatically switched between day and night.

J. "Left adjacent lane preceding vehicle recognition and distance measurement" J (Figs. 50, 51, 52) Also in this case, the lower side of the window 2 (Y = 350),
Vehicle lane Left edge white line (TmL, RmL) and adjacent left rail
In the same way, the above-mentioned "own vehicle lane preceding vehicle recognition and distance measurement" process H is applied to the area (left adjacent lane area) surrounded by the leftmost white line (TmLL, RmLL) of the vehicle. An object in front of the own vehicle in the lane area is detected, and the distance from the own vehicle is calculated. Further, as in the case of "recognition and distance measurement of preceding vehicle by preceding vehicle lane" H, the position of the window 2 to be referred to in the y direction is automatically switched between day and night.

Thus, the "vehicle detection in window 2-1" DET1 shown in FIG. 3 is completed. In this DET1, the left and right white lines of the own vehicle lane in the window 2-1 (FIGS. 14, 16, and 26) are detected, the white line of the right edge of the right adjacent lane is detected based on this, and the left adjacent line is detected. Detection of the leftmost white line of the vehicle, detection of the vehicle within the own vehicle lane (the area filled with the vertical line in FIG. 26) within the area extending from the window 2-1 to the center of the window 2-2, and the distance calculation , Vehicle detection and distance calculation in the right adjacent lane in the area (right side horizontal line filled area in FIG. 26), and vehicle in the left adjacent lane in the area (left side horizontal line filled area in FIG. 26) This means that detection and distance calculation have been performed. In this case, when the preceding vehicle in the own vehicle lane is detected, "1" is written in the register FVM. When the vehicle in the right adjacent lane is detected, "1" is written in the register FVR. If a vehicle in the left adjacent lane is detected, the register FVL will show "1".
Is written.

Next, the CPU 1 checks whether the content of the register FVM is "1" (the preceding vehicle is in the own vehicle lane), and if it is "1", it proceeds to "output" K and the image is displayed. Output the recognition result. Then, "image input" B is executed again, and the same processing is performed. That is, substantially the window 2-1
In the processing targeting the above, while the preceding vehicle is detected on the own vehicle lane, the distance between the preceding vehicle and the preceding vehicle is relatively short, so "image input" B to "output" shown in FIG. K is repeatedly executed, and the processes DET2 and DET2 for windows 2-2 and 2-2 described later are not executed.

When the preceding vehicle is not detected on the own vehicle lane in DET1 (FVM = 0), the CPU1 executes the DET.
1 followed by "Vehicle detection in window 2-2" DET2
To execute. The process of DET2 is to execute the same process as the process of DET1 described above in the window 2-2 area shown in FIGS. 14 and 16, and the own vehicle lane area ((a) of FIG. A preceding vehicle detection process is performed for a vertical line filled area), a right adjacent lane area (right horizontal line area), and a left adjacent lane area (left horizontal line area). When the preceding vehicle (on-road object) is detected in the own vehicle lane area, the register FVM
When a vehicle is detected in the right adjacent lane area, "1" is written in the register FVR, and when a vehicle is detected in the left adjacent lane area, "1" is written in the register FVL. And
Writing "1" to register FVM advances to "output" K and D
Do not execute ET3.

When the preceding vehicle is not detected on the own vehicle lane in DET2 (FVM = 0), the CPU 1 executes the DET.
2 followed by "Detection of vehicle in window 2-3" DET3
To execute. The process of DET3 is to execute the same process as the process of DET1 described above in the window 2-3 area shown in FIGS. 14 and 16, and the own vehicle lane area ((b) of FIG. A preceding vehicle detection process is performed for a vertical line filled area), a right adjacent lane area (right horizontal line area), and a left adjacent lane area (left horizontal line area). Since this window 2-3 is the last (farthest) end of the object detection area on the road ahead, the farthest point of the road surface area is the intersection point of the two detected straight lines (LR3, LL3 in FIG. 27) ( At the point of infinity) or when this point of intersection deviates from the screen, the road surface area is defined as the screen edge (Y = 0). When a preceding vehicle (on-road object) is detected in the own vehicle lane area, "1" is written in the register FVM, when a vehicle is detected in the right adjacent lane area, "1" is written in the register FVR, and the left adjacent area is detected. -When a vehicle is detected in the zone, write "1" to the register FVL. And
Go to “Output” K.

The "day / night determination" S shown in FIG. 3 will be described. This processing is roughly divided into four steps S1, S2, S3 and S4 shown in FIG. The video imaged by the TV camera is, for example, a two-dimensional image as shown in FIG. 64, and this image usually includes a sky region, a road region, and other regions. Therefore, in this embodiment, as shown in FIG. 55, the area in the image is divided into an empty area RGS, a road surface area and other areas based on the coordinates (Xv, Yv) of the point at infinity. There is. Then, in order to carry out the day / night determination, in this embodiment, only the image information of the sky region RGS is referred to. That is, in the sky region RGS, the change in the brightness of the day and night is sufficiently large, and the opportunity for the noise image to appear is relatively small. Therefore, it is possible to determine the day and night by identifying the brightness of the sky region RGS.

For example, FIG. 63, which is an image obtained by photographing at night, and FIG. 64, which is an image obtained by photographing during daytime.
65, which are images obtained by shooting in the evening and in the evening, when the video signal of one scanning line SL corresponding to the sky region is extracted, they are shown in FIG.
(B) and (c). 66 (a) and (b) of FIG. 66, a difference in signal level is clearly recognized, and thus day and night can be identified based on this video signal level.

However, even in the sky region RGS, a noise image that adversely affects the determination of day and night often appears in the reference region in the image due to the influence of street lights, advertising lights, and the like. Therefore, in this embodiment, a large number of small areas are defined in the empty area RGS, day and night are identified from the average brightness of each small area, and the final result is determined based on the identification result of the plurality of small areas. It distinguishes day and night.

Briefly, step S1 in FIG.
In step S2, the average density (brightness) of each small area is calculated, and in step S3, each small area is calculated. The average density is binaryly identified (in day / night), and in the final step S4, the final day / night is identified based on the identification result of the plurality of small areas. As a final identification result, the flag S_night is set.

Details of each process will be described below. The equal sign (=) appearing in each process indicated by a rectangular frame in the flowcharts shown in FIG. 56 and subsequent figures indicates the contents of the right side of the expression or the calculation result in the memory (or register, flag) on the left side. It means to substitute (ie store).

Details of step S1 in FIG. 54 are shown in FIG. In this process, 16 small areas A1, A2, A3, ... A16 are provided in the empty area RGS as shown in FIG.
Determine. It should be noted that the numerical values in FIG. 57 indicate the coordinates or size in units of the number of pixels.

Description will be made with reference to FIG. When this process is executed for the first time, since the initialization flag S_init is 0, the process proceeds to step S1_2 after step S1_1, and step S1_
In step 2, the initialization flag S_init is set.

In the next step S1_3, the flag S_night is set.
Is cleared and the constant 16 is stored in the register No_max. In the subsequent step S1_4, the counters i, j and k are each cleared. The following steps S1_5 to S1_12 are repeatedly executed as a loop process while j <No_max.

In step S1_5, memories S_xs (j + 1), S_
ys (j + 1), S_av (j + 1) and S_k (j + 1) are respectively (32
Xj + 16), 20, 255 and 1.0 are stored, and the counter j is incremented (+1) in step S1_6.
Then, in step S1_7, the value of j is stored in the memory S_pos (i, k), and in step S1_8, the counter k is incremented. Also, every time the counter k reaches 4, step S
Go from 1_9 to S1_10, clear the memory S_pos (i, k),
In the next step S1_11, the counter k is cleared and the counter i is incremented.

Note that, for example, the memory S_xs (j + 1) is a part of a predetermined memory array S_xs (0 or 1) to S_xs (maximum value) and is specified by the parameter values in parentheses. Memory. For other memories, the same notation all indicates a part of the same memory array.
The asterisk (*) in the flowchart means multiplication.

By executing the processing shown in FIG. 56, the memories S_xs (1), S_xs (2), S_xs (3), S_xs (4),
..., to S_xs (16), 16, 16 + 32, 16 + 32, respectively
× 2, 16 + 32 × 3, ..., 16 + 32 × 15 are stored.
That is, the small areas A1, A2, A3, A4 shown in FIG.
.... The x coordinate of the left end of each A16 is the memory S_xs (1), S_xs (2), S_xs (3), S_xs (4) ,.
.., held in S_xs (16). In addition, small areas A1 and A
2, A3, A4, ...
The coordinates (all 20) are the memory S_ys (1) and S_ys, respectively.
(2), S_ys (3), S_ys (4), ..., S_ys (16). Furthermore, the contents of the memories S_av (1) to S_av (16) are all initialized to the maximum value of 255, and the memories S_k (1) to S_av (16)
The contents of S_k (16) are all initialized to 1.0. Also, 2
In S_pos (0,0) to S_pos (3,3), which are three-dimensional array memories,
Values are stored as shown in FIG. 60 (a).

Details of step S2 in FIG. 54 are shown in FIG. This will be described with reference to FIG. In this process,
Basically, the average gradation of the image information in each of the small areas A1 to A16 shown in FIG. 57 is calculated. However, FIG.
If the processing of 8 is executed only once, 16 areas A1 to A
By calculating the gradations of only 4 areas out of 16, and executing the process of FIG. 58 four times, the average gradations of all 16 areas are calculated. Which of the 16 areas A1 to A16 is to be processed is determined by referring to the contents of the two-dimensional array memory S_pos (S_cy, j) shown as (a) in FIG. For example, the counter S_cy is 1
Then, if the counter j is 2, the memory S_pos (S_cy, j)
Is 7, the average gradation of the area A7 is calculated.

In the first step S2_1, the counter j is cleared, and in the next step S2_2, the contents of the memory S_pos (S_cy, j) specified by the counter S_cy, j are referred to and the contents are stored in the area number register no. To do. n
If the value of o is not 0, the process proceeds to step S2_4 through step S2_3.

In step S2_4, the memory S_av (no) is cleared, the counter cnt is cleared, the x coordinate of the small area indicated by the value of the register no is obtained from the memory S_xs (no), and stored in the registers xs and x. Then, the y coordinate of the small area indicated by the value of the register no is obtained from the memory S_ys (no) and stored in the registers ys and y.

In step S2_7, the gradation information I (x, of the pixel existing at the coordinates indicated by the registers x and y in the image data.
y) is added (accumulated) to the memory S_av (no), and the counters x and cnt are respectively incremented. x <(xs
+16), the process returns from the next step S2_9 to S2_7. Therefore, until x = (xs + 16), step S
2_7 is repeated. This will convert the coordinates (xs, y) to (xs
For 16 pixels up to +15, y), their gradation information I
The accumulated value of (x, y) is held in the memory S_av (no).

In step S2_10, the value of xs (the left end position of the small area) is preset in the counter x, and 3 is added to the counter y. In the following step S2_11, y and (ys +
24) are compared. Then, if y <(ys + 24), the process returns to step S2_7 again. Therefore, every 3 lines of scanning lines, that is, the y coordinate is ys, ys + 3, ys +.
Steps S2_7 and S2_9 are executed for each of the scanning lines 6, ys + 9, ..., Ys + 21. That is, y
= (Ys + 24), and when the process proceeds from step S2_11 to S2_12, in one small region composed of 16 × 24 pixels, the cumulative value of the gradation of all the pixels shown by hatching in FIG. 59. Are held in the memory S_av (no). Also,
The counter cnt holds the number of pixels referred to in the small area, that is, the number of pixels hatched in FIG. In step S2_12, the result of dividing the content of the memory S_av (no) by the value of the counter cnt is stored in the memory S_av (no). As a result, the average gradation of all the pixels referred to in the small area is obtained and stored in the memory S_av (no). Also, the counter j is incremented.

The process returns from step S2_12 to S2_2 and the above processing is repeated. For example, when the parameter S_cy is 0, the value read from the memory S_pos (S_cy, j) and stored in the register no is changed according to the value of the counter j.
1, 2, 3, 4, 0 (see FIG. 60), the average gradations for the sub-regions A1, A2, A3, and A4 are 1st, 2nd, 3rd, and 4th, respectively. Required and the results are stored in memory S_av (no)
Will be stored in. Then, when the value read from the memory S_pos (S_cy, j) becomes 0, that is, when the processing of the four small areas is completed, the process proceeds from step S2_3 to S2_5.

In step S2_5, the counter S_cy is incremented. As a result, when the value of the counter S_cy reaches 4, the process proceeds from step S2_6 to S2_8,
Clear the counter S_cy. Since the counter S_cy is updated when step S2_5 is executed, the value read from the memory S_pos (S_cy, j), that is, the small area to be processed is different from this time when the processing of FIG. 58 is executed next time. It becomes a thing. Then, when the process of FIG. 58 is repeatedly executed four times, all the processes of the 16 small areas A1 to A16 are completed.

However, the process of FIG. 58 is repeatedly executed every time an image is input, and the result of the process is held in the memory S_av (S_cy, j) until it is updated.
Even if the process of 8 is not repeated 4 times, the average gradation can be obtained for all 16 small regions. As a result, the cycle for identifying day / night can be shortened. Further, in this embodiment, the scanning lines are thinned out by referring to each pixel at a ratio of one line to three lines of the horizontal scanning line without referring to all the pixels in the small area. 5
The time required to execute the process shown in 8 can be shortened.

The detailed contents of step S3 of FIG. 54 are shown in FIG.
It is shown in FIG. This will be described with reference to FIG. Step S3_1
Then, 1 is preset in the counter i. Next step S
In 3_2, the memory S_av (i) specified by the value of the counter i
The average gradation value of one small area held at is compared with a predetermined threshold value THa. And S_av (i) <TH
If it is a, in the next step S3_4, the flag S_flag (i) specified by the value of the counter i is set, and S_av (i) ≧ T
If it is Ha, the flag S_flag (i) specified by the value of the counter i is cleared in step S3_3. That is, when the average gradation of each small area is smaller than the threshold value (dark), the small area is considered as a candidate for "night", and when the average gradation is equal to or higher than the threshold value (bright), The small area is considered as a candidate for "daytime".

In the next step S3_5, the counter i is incremented, and in the following step S3_6, the value of the counter i is compared with the maximum number No_max of small areas. If i ≦ No_max, the process returns to step S3_2 to repeat the above process. That is, for each of the 16 small areas, the memory S_a
Each average gradation value held in v (i) is binaryly identified by the threshold value THa, and the result is stored in the flag S_flag (i). Therefore, by referring to the flag S_flag (i), it is possible to identify whether the i-th small area is a “night” candidate or a “day” candidate.

The detailed contents of step S4 of FIG. 54 are shown in FIG.
It is shown in FIG. This will be described with reference to FIG. In the first step S4_1, the counter i is preset to 1 and the counters cnt and THN are cleared. In the next step S4_2, the counter THN is incremented.
In the following step S4_3, the flag S_flag (i) specified by the value of the counter i is referred to. And S_flag (i) = 1
If so, that is, if the i-th small area is a candidate for "night", the counter cnt is incremented in the next step S4_4. If S_flag (i) = 0, that is, the i-th small area is "daytime"
, The counter cnt is not updated.

In step S4_6, the counter i is compared with the maximum number No_max of small areas. Then, if i ≦ No_max, the process returns to step S4_2 to repeat the above process.
As a result, all of the flags S_flag (1) to S_flag (16) are checked in step S4_3. In the case of proceeding from step S4_6 to S4_7, the number of small areas that are candidates for "night" among the 16 small areas A1 to A16 is the counter c.
held in nt. The counter THN holds the number of all the small areas to be processed (actually No_max).

At step S4_7, THN × (3/4)
And the result is stored in THN. Then, in the next step S4_8, the value of the counter cnt is compared with the value of THN. In other words, the number of small areas that are candidates for "night" is compared with a threshold value (content of THN) determined based on the number of all small areas (16 in this case) that are the processing targets. If cnt ≧ THN, the day / night identification flag S_night is set in the next step S4_10, and cn is set.
If t <THN, the flag S_night is determined in step S4_9.
To clear. Therefore, when the number of small areas of the “night” candidate is 3/4 or more of all the small areas, it is finally identified as “night”, and in other cases, it is identified as “daytime”.

In the first embodiment described above, the vehicle lane left and right end white lines LR1 and LL1 (FIG. 26) are detected in the area of the window 2-1 that is closest to the vehicle and based on this. Then, the left and right white lines of the left and right adjacent lanes in the window 2-1 are estimated, and the region between LR1 and LL1 from the window 2-1 to the middle of the window 2-2 (see FIG. 26).
Of the vertical line), and whether or not an object exists in the left and right adjacent lane regions (horizontal line filled region of FIG. 26) are similarly determined. When the object is detected and the object is detected, the distance from the vehicle to the object is calculated. When an object is detected in this area on the own vehicle lane, the processing regarding windows 2-2 and 2-3 is not executed. Window 2-2 when the object is not detected
The same processing (a in FIG. 27) is also executed for the window 2-3 if the object in the own vehicle lane area is not detected.
The same process (b in FIG. 27) is also executed for the above.

Also, at night, the y of Windu to be processed
The position in the direction is corrected according to the height (TL) of the tail lamp of the preceding vehicle (154 in FIG. 32).
B). In this way, the position of the window to be processed is corrected according to the day / night difference, and further, step 208 in FIG.
B, 208C, the calculation formula for obtaining the distance is switched (or corrected), and steps 229B, 2 in FIG.
By switching the threshold value for discrimination like 29C, most of the processing for recognizing the preceding vehicle can be commonly used in the daytime and the nighttime. Therefore, the preceding vehicle can be recognized both during the day and at night without complicating the hardware structure.

Next, the second embodiment will be described.

In this embodiment, the CPU 1 executes the processing shown in FIG. That is, next to the “day / night judgment” S,
The "self-vehicle lane detection (2-1)" D1 (same as the content of the above-mentioned D) detects two straight lines LR1 and LL1 (c in FIG. 14) in the window 2-1 and the detected straight line is detected. Based on "Adjacent lane estimation (2-1)" F1 (same as the content of the above-mentioned F)
Then, the right end white line LRR1 of the right adjacent lane and the left end white line LLL1 of the left adjacent lane are estimated. And the window 2-
2 is performed, and similar straight line detection and straight line estimation are executed by "own vehicle lane detection (2-2)" D2 and "adjacent lane estimation (2-2)" F2, and window 2- Two straight lines LR2 and LL2 (c in FIG. 14) representing the own vehicle lane end, and the right end white line LRR2 and the left adjacent lane of the right adjacent lane.
Estimate the leftmost white line LLL2 of the computer. In addition, 2 windows
-3, "Self-vehicle lane detection (2-3)" D3 and "Adjacent lane estimation (2-3)" F3 perform similar straight line detection and straight line estimation, and the window 2 Two straight lines LR3 and LL3 representing the vehicle lane end within -3 (c in FIG. 14)
Also, the right end white line LRR3 of the right adjacent lane and the left end white line LLL3 of the left adjacent lane are estimated. As for the position of the window to be processed here in the y direction, the height (TL) of the tail lamp portion of the preceding vehicle at night as in the above embodiment.
The correction according to is added.

As described above, the three straight lines LR1 to LR3 representing the white line at the right end of the own vehicle lane, the three straight lines LL1 to LL3 representing the white line at the left end (c in FIG. 14), and the outside of them, the right line Three straight lines LRR1 to LRR3 representing the rightmost white line (estimated) of the adjacent lane and the leftmost white line (estimated) of the left adjacent lane
That is, three straight lines LLL1 to LLL3 representing the above are obtained.

Next, the CPU 1 sets "own vehicle lane area setting".
h, the straight line LR1 and the right or lower side of the window 2-2 (X = 5
11 vertical line or Y = 350 horizontal line), intersection of straight lines LR1 and LR2 (circle mark in FIG. 14c), intersection of LR2 and LR3, straight line LL1
And the left or bottom side of window 2-1 (vertical line at X = 0 or Y = 35
Intersection of 0), intersection of straight lines LL1 and LL2, LL2 and LL3
And the intersection of the straight lines LR3 and LL3,
An area surrounded by a straight line connecting two points adjacent to each other in the Y direction at these intersections and the lower side of the window 2-1 is set as the vehicle traveling lane area. Then, in the next "recognition and distance measurement of preceding vehicle lane preceding vehicle" H, the object detection and the distance within the set area are performed by the same processing as the object detection and the distance calculation (H) in the first embodiment. Calculate. The position of the window to be processed here in the y direction is also corrected at night according to the height (TL) of the tail lamp portion of the preceding vehicle, as in the above-described embodiment.

Next, the CPU 1 executes the "right adjacent lane area setting" i, and the straight line LR1 and the right side or the lower side of the window 2-1.
Intersection with (vertical line of X = 511 or horizontal line of Y = 350), straight lines LR1 and L
R2 intersection, LR2 and LR3 intersection, straight line LRR1 and window 2
-1 right side or lower side (vertical line of X = 511 or horizontal line of Y = 350), intersection of straight lines LRR1 and LRR2, intersection of LRR2 and LRR3,
Further, the intersection between the straight lines LR3 and LRR3 is calculated, and the area surrounded by the straight line connecting two points adjacent to each other in the Y direction of these intersections and the lower side of the window 2-1 is set as the right adjacent lane area. Then, in the next "right adjacent lane preceding vehicle recognition and distance measurement" I, within the set area by the same processing as the object detection and distance calculation (I) in the right adjacent lane in the first embodiment described above. Object detection and distance calculation are performed. The position of the window to be processed here in the y direction is also corrected at night according to the height (TL) of the tail lamp portion of the preceding vehicle, as in the above-described embodiment.

Next, the CPU 1 executes the "left adjacent lane area setting" j, and the straight line LL1 and the left or lower side of the window 2-1.
Intersection with (vertical line at X = 0 or horizontal line at Y = 350), straight lines LL1 and LL2
Intersection, intersection of LL2 and LL3, straight line LLL1 and window 2-
Calculate the intersection with the left side or the lower side of 1 (vertical line at X = 0 or horizontal line at Y = 350), the intersection between straight lines LLL1 and LLL2, the intersection between LLL2 and LLL3, and the intersection between straight lines LL3 and LLL3, and these A window and a straight line connecting two points adjacent to each other in the Y direction of the intersection of
The area surrounded by the lower side of -1 is set as the left adjacent lane area. And the next "Left adjacent lane preceding vehicle recognition and distance measurement"
In J, the object detection and the distance calculation in the set area are performed by the same processing as the object detection and the distance calculation (J) in the left adjacent lane in the first embodiment. The position of the window to be processed here in the y direction is also corrected at night according to the height (TL) of the tail lamp portion of the preceding vehicle, as in the above-described embodiment.

Also according to the second embodiment, the entire road surface area (own vehicle lane, right adjacent lane) from immediately before the own vehicle to the far point where the left and right white lines of the own vehicle lane intersect. And the left adjacent lane), the object search and the distance of the detected object are calculated.

In both the first and second embodiments described above, windows 2-1, 2-2 and 2-3 are provided.
Then, a search area is set gradually from the immediate vicinity of the own vehicle to a distant area, and a white line that separates the lanes is detected in each area, and the search area is set based on the detected white line. Even if the search area is far away from the road surface, the probability of being off the road surface is low, and detection of road objects, especially detection of distant road objects,
Will be more certain.

Various modifications of the above-mentioned "day / night determination" (step S in FIG. 3) will be described below.

First, FIGS. 67, 68 and 69 show various modifications regarding the shape and arrangement of a large number of small regions positioned in the empty region RGS in the image. First, FIG. 67.
Refer to.

(A) Each of the small areas is a horizontal line, and the small areas are continuously arranged vertically in the empty area RGS.

(B) Each of the small areas is a horizontal line, and the small areas are arranged in the empty area RGS at intervals in the vertical direction.

(C) The small area is a rectangle elongated in the horizontal direction, and the small areas are continuously arranged in the vertical direction in the empty area RGS.

(D) The small regions are horizontally elongated rectangles, and the small regions are arranged in the empty region RGS with a space therebetween in the vertical direction.

(E) The small areas are formed in a rectangular shape elongated in the horizontal direction, and the small areas are arranged in the empty area RGS in the vertical direction, and those adjacent in the vertical direction are partially overlapped with each other.

(F) The small areas are formed in a horizontally elongated rectangular shape, and the small areas are arranged vertically in the empty area RGS so that some adjacent small areas partially overlap each other. In particular, the small areas on the upper side are arranged so as to overlap, and the small areas on the lower side near the point at infinity are arranged so as not to overlap. In this way, each small area is weighted according to its position.

In the example shown in FIG. 67, the shape of each small area is made long in the horizontal direction, but in the example shown in FIG. 68, the shape of each small area is changed to be long in the vertical direction. is there. In a general television camera, main scanning is performed in the horizontal direction and sub-scanning is performed in the vertical direction (vertical direction), so a small area is set so that the horizontal direction that matches the main scanning direction becomes longer. When arranged, the information in the small area can be referred to in a short time. Therefore, in general, the arrangement shown in FIG. 67 is preferable. However, for example, when the TV camera is tilted by 90 degrees and is attached so that the horizontal direction and the vertical direction are opposite to each other, the vertical direction of the drawing is the main scanning direction,
Since the horizontal direction is the sub-scanning direction, it is possible to refer to the image information in a short time by arranging the small areas as shown in FIG.

Next, description will be made with reference to FIG.

(A) All the small areas are rectangular (square) having the same shape, and the small areas are continuously arranged over the entire empty area RGS.

(B) All the small areas have the same shape of rectangle (square). In the empty region RGS, small regions are continuously arranged in the horizontal direction and are spaced apart from each other in the vertical direction.

(C) All the small areas have the same shape of rectangle (square). In the empty region RGS, small regions are continuously arranged in the vertical direction and are spaced apart from each other in the horizontal direction.

(D) All the small areas have the same shape of rectangle (square). Small regions are arranged in the empty region RGS in the vertical direction and in the horizontal direction at regular intervals and at intervals.

(E) All the small areas are rectangular (square) having the same shape. Small regions are arranged in the empty region RGS in the vertical direction and in the horizontal direction at intervals from each other. Also, the small areas are arranged in order in the horizontal direction, but are arranged in a staggered manner in the vertical direction.

(F) Square small areas and small areas each consisting of one horizontal line are arranged in combination. The former small area is located above, and the latter small area is located below (closer to the point at infinity).

In the above-described embodiment, in the process of FIG. 56, each element of the memory array S_pos (i, k) is assigned to FIG.
Although the values shown in (a) of 0 are stored respectively, the arrangement of the values may be changed as shown in (b) of FIG. 60, for example.
In this way, A1, A5, A9, A13, A2
The order can be changed so that each small area is processed in the order of A6, .... Also, the memory array S_pos (i, k)
The number of elements (maximum number of parameters i and k) may be changed.

Next, the processing shown in FIG. 61 (S in FIG. 54) is executed.
A modified example of 3) will be described. FIG. 70 shows one modified example.
Shown in A modification of the process S3x in FIG. 61 is shown in FIG.
0 processing. The changed portion will be described with reference to FIG. 70. In this embodiment, two thresholds THa and THb are used as threshold values for binaryly identifying the contents of the memory S_av (i) holding the average gradation value of each small area.
In this example, the THa value is 60 and the THb value is 80. Then, before binarizing the average gradation value of each small area,
The previous identification result held in the flag S_flag (i) is referred to in step S3_7, and if S_flag (i) = 1 (night candidate), step S3_8 is performed, otherwise step S3_9.
Proceed to. In step S3_8, the contents of the memory S_av (i) are compared with the threshold value THa, and the flag S_f is determined according to the result.
Update the status of lag (i). Further, in step S3_9, the content of the memory S_av (i) is compared with the threshold value THb, and the state of the flag S_flag (i) is updated according to the result. As a result, it is possible to impart a hysteresis characteristic to the binarization of the average gradation of each small area. Therefore, for example, even when the average gradation value is close to the threshold value as in the evening image,
It is possible to prevent frequent switching of the identification result “night / day”.

Another modification is shown in FIG. FIG.
71 is a modification of the processing S3x of FIG. In this embodiment, when the same state is continuously maintained several times in the process of binaryly identifying the average gradation of each small area,
The identification result is switched.

Description will be made with reference to FIG. Step S3_
In 14, the memory S_av that holds the average gradation value of each small area
The content of (i) is compared with the threshold value THa, and depending on the result, the process proceeds to step S3_15 or S3_16. In steps S3_15 and S3_16, the flag S assigned to the i-th small area
Update the state of _flag_N (i). In the next step S3_17, the state of the flag S_flag_N (i) holding the latest identification result is compared with the flag S_flag_O (i) holding the past identification result, and if the two match, the next step is S3_19
If not, the process proceeds to step S3_18.

In step S3_18, the counter S_cnt (i) assigned to the i-th small area is cleared, and in the next step S3_20, the state of the flag S_flag_N (i) is set to the flag S_f.
Save it in lag_O (i).

In step S3_19, the counter S_cnt (i) assigned to the i-th small area is incremented.
The sign “+ =” means that the content on the right side is added to the content on the left side and the result is stored in the memory on the left side (for example, it is generally defined in the programming language “C”). Exist). In the next step S3_21, the value of the counter S_cnt (i) is compared with a threshold value of 3.
If the value of the counter S_cnt (i) is 0, 1 or 2, the process proceeds from step S3_21 to S3_5, but if the value of S_cnt (i) is 3 or more, the process proceeds to step S3_22.

At step S3_22, the counter S_cnt (i)
Store 3 in. In the next step S3_23, the flag S_flag (i) is updated according to the state of the flag S_flag_N (i) holding the latest identification result. Therefore, in this example, only when the result of step S3_14 is the same three times consecutively,
The content of the flag S_flag (i) is changed. So, for example,
Even when the average gradation value is close to the threshold value THa as in the image in the evening, it is possible to prevent frequent switching of the identification result “night / day”.

It is also possible to combine the processing of FIG. 70 and the processing of FIG. 71 described above. For example, the process S3y shown in FIG.
May be replaced with the processing shown in FIG.

Next, the processing shown in FIG. 62 (S in FIG. 54) is executed.
A modified example of 4) will be described. FIG. 73 shows one modification.
Shown in FIG. 7 shows a modification of the process S4x in FIG.
It is the process of 3. The changed portion will be described with reference to FIG. 73. In this embodiment, in the final "night / day" discrimination process, the discrimination result is switched when the same state is continuously maintained several times.

In step S4_11, the content of the counter cnt holding the number of small areas of "night candidate" is compared with the value of the counter THN holding the threshold value, and the process proceeds to step S4_12 or S4_13 according to the result. Steps S4_12 and S
In 4_13, the state of the flag S_night_N is updated. In the next step S4_14, a flag S_nig for holding the latest state is held.
ht_N is compared with the flag S_night_O holding the past state, and the process proceeds to step S4_15 or S4_16 according to the result. In step S4_15, the counter S_cnt_n is cleared, and in the next step S4_17, the state of the flag S_night_N is stored in the flag S_night_O. In step S4_16,
The counter S_cnt_n is incremented and the next step S
In 4_18, the value of the counter S_cnt_n is a threshold value of 10
Compare with Then, only when S_cnt_n is 10 or more, the following steps S4_19 and S4_20 are executed. In step S4_19, 10 is stored in the counter S_cnt_n, and in step S4_20, the flag S_night_N that holds the latest state is stored.
Accordingly, the state of the flag S_night, which is the final identification result, is updated. Therefore, in this example, the state of the flag S_night changes only when the state in which the flag S_night_N and the flag S_night_O coincide with each other occurs 10 times in a row. Thereby, even when the average gradation value of each small area is close to the threshold value (THa) as in the evening image, it is possible to prevent the final identification result (S_night) from being frequently switched. .

Another modification is shown in FIG. FIG. 62
FIG. 74 shows a modified version of steps S4_2 to S4_4 of FIG.
This is the process. The changed portion will be described with reference to FIG. In this embodiment, independent weighting factors are assigned to each small area in advance. These weighting factors are held in the memory array S_k (), and the weighting factor of the i-th small area can be obtained from the memory S_k (i) specified by the parameter i. In step S4_22, the memory S_k
The weighting coefficient of the i-th small area is read from (i) and stored in the register n. In the next step S4_23, the content of register n (weighting coefficient) is added to the content of register THN, and the result is stored in register THN. In the next step S4_24, the flag S_flag (i) of the i-th small area is referred to. When S_flag (i) = 1, the content of the register n is added to the counter cnt in the next step S4_25, and the result is stored in the counter cnt. Therefore,
When the processing of all the small areas is completed and the process proceeds to step S4_7, the content of the register THN is the sum of the weighting factors of all the small areas, and the content of the counter cnt is
It is the sum of the weighting factors of the small areas regarded as “night candidates”. Therefore, in step S4_7, the threshold value is determined based on the content of the register THN, and in step S4_8, the content of the counter cnt is compared with the threshold value (value of THN) to finally determine whether it is night or day. are doing.

In the image (for example, FIG. 64) taken by the television camera, even in the normally empty region RGS, other images (that is, noise) such as trees and street lights may be included in the portion. . Further, the frequency of appearance of noise varies depending on the position on the image, and generally, the frequency decreases as the position approaches the top of the screen. Therefore, for example, when a large number of small areas are arranged in the empty area RGS as shown in FIG. 69 (a), a larger weighting coefficient is assigned as the position of the small area in the screen is increased. As a result, the influence of noise can be reduced.

Another modification is shown in FIG. Figure 75
The above process is a combination of the processes of FIGS. 73 and 74 already described, and the same process steps are indicated by the same numbers.

In the above embodiment, night and day are binary-identified on the basis of the brightness of the sky region in the image. It is also possible to identify three or more types of brightness such as / evening / daytime. If there are three or more types of algorithms for recognizing the preceding vehicle, it is desirable to identify three or more types of brightness such as night / evening / day and select the most preferable algorithm.

In the above embodiment, the preceding vehicle,
Although an image of a subject existing in front of the own vehicle is taken to recognize a preceding vehicle or an oncoming vehicle, if the optical axis of the TV camera is directed rearward, the following vehicle can be recognized by the same algorithm. .

[0202]

In any of the claims of the present invention, the preceding vehicle or the like can be recognized based on the image taken at night. Moreover, since the vehicle can be recognized in the daytime by almost the same algorithm as in the nighttime, even if the vehicle is recognized in both the nighttime and the daytime, the processing in the nighttime and the daytime can be made common. A complicated structure of the device is avoided.

Further, in any of claims 2, 3 and 4, since nighttime and daytime are automatically discriminated and the recognition processing is partially switched, both nighttime and daytime can be reliably performed. The vehicle can be recognized.

Further, according to the present invention, even if the influence of noise such as a street lamp appears in the sky region of the image, correct identification can be finally performed, so that high reliability can be obtained. .

[Brief description of drawings]

FIG. 1 is a block diagram illustrating a front vehicle recognition system on a vehicle embodying the present invention in one aspect.

FIG. 2 is a schematic side view showing an arrangement position on the vehicle of the television camera 6b shown in FIG.

FIG. 3 shows an outline of processing for detecting a white line on the road surface in front of the host vehicle and a preceding vehicle in the first embodiment of the microprocessor 1 shown in FIG. 1 based on the imaged data of the television camera 6b. It is a main routine flow chart.

FIG. 4 is a flow chart showing the contents of “Screen Calibration” C in FIG.
It is a chart.

5 (a) is a flow chart showing the contents of "1 set of feature point detection windows" C1 shown in FIG. 4, and FIG. 5 (b) is a plan view showing a shooting screen of the television camera 6b shown in FIG. is there.

6A is a “feature point detection (UP)” shown in FIG.
A flow chart showing the contents of C2, (b) is a target point (calculation point pixel: X mark) and a reference point (reference point) when calculating the differential value of the image data in the Y direction of the photographing screen. Pixel: White square)
FIG.

7 (a) is a "bonnet detection" C3 shown in FIG.
2B is a plan view showing a straight line approximating the bonnet edge on the photographing screen.

8A is a plan view showing the roll angle and the pan movement amount calculated by “calculation of roll angle and pan movement amount” C5 shown in FIG. 4 on a photographing screen, and FIG. FIG. 9 is a plan view showing a conversion formula for converting a pixel address on a photographing screen by a “image rotation parallel movement” C7 shown in FIG. 3 into a pixel address on a correction screen which is rotated by a roll angle and translated by a pan movement amount. .

FIG. 9: “Corrected image memory initialization” C in FIG.
6 is a flowchart showing the contents of 6.

FIG. 10 is a flowchart showing the contents of “image rotation parallel movement” C7 in FIG.

11 is a flowchart showing a part of "interpolation" C8 in FIG.

FIG. 12 is a flowchart showing the remaining portion of the “interpolation” C8 in FIG.

FIG. 13 is a flow chart showing “own vehicle lane detection” D in FIG.
It is a chart.

14 (a) is a flowchart showing the contents of "feature point detection window 2-1 set" D1 shown in FIG.
FIG. 6B is a plan view showing an area of the window 2 set on the correction screen.

FIG. 15 is a flowchart showing “left and right white line detection” D3 in FIG.

16 is a plan view showing an image of the correction screen and a window 2. FIG.

FIG. 17 is a flowchart showing a part of the content of “straight line fitting” 63 shown in FIG.

FIG. 18 is a flowchart showing the remaining part of the contents of “straight line fitting” 63 shown in FIG.

FIG. 19 shows “Hough grid clear (HG1)” 65 and “Hough grid clear (HG) shown in FIGS. 17 and 18.
2) ”70R, 70L and“ Huff Grid Clear (H
G3) ”75R, 75L is a flowchart showing the contents in a general-purpose format, and n in FIG. 19 is 1, 2, or 3.

FIG. 20 shows the “Hough transform (H
G1) "67," Hough conversion (HG2) "72R, 72L
And "Hough transform (HG3)" 77R, 77L are flow charts showing a general-purpose format, and n in FIG. 20 is 1, 2 or 3.

21 is a "maximum point search (HG1)" 68, "maximum point search (HG2)" 73R shown in FIG. 17 and FIG.
73L and "maximum point search (HG3)" 78R, 78L
2 is a flow chart showing the contents of the general purpose format in FIG.
N in 1 is 1, 2, or 3.

22 is a flowchart showing "infinity point calculation" D4 in FIG.

FIG. 23 is a schematic side view showing the geometrical relationship between the lens and the image pickup element in the television camera 6b shown in FIG. 1 and a preceding vehicle ahead of the vehicle.

FIG. 24 is a flow chart showing “adjacent lane estimation” F in FIG.
It is a chart.

FIG. 25 is a plan view showing the adjacent lanes estimated by the “adjacent lane estimation” F on the correction screen, which are filled with diagonal lines.

FIG. 26 is a search area in the “vehicle detection in window 2-1” DET1 shown in FIG. 3 for performing vehicle recognition and distance detection H, I and J shown in FIGS. 28, 47 and 50; FIG. 5 is a plan view showing a vertical line and a horizontal line with a black line.

FIG. 27 (a) is a plan view showing a search area for performing object detection by “vehicle detection in window 2-2” DET2 shown in FIG. 3 by filling vertical and horizontal lines, and FIG.
FIG. 8 is a plan view showing a search area in which an object is detected by “vehicle detection in window 2-2” DET3 shown in FIG.

28 is a flowchart showing the contents of "recognition and distance measurement of preceding vehicle by own vehicle lane" H shown in FIG.

FIG. 29 shows “Detection of candidate vehicle position 1” H shown in FIG. 28.
It is a flowchart showing the contents of 1.

FIG. 30 is a “feature point memory clear” 13 shown in FIG. 29.
It is a flowchart showing the contents of 1.

[FIG. 31] “Horizontal feature point detection 1” 13 shown in FIG. 29
2 is a flowchart showing the contents of 2.

32 (a) is a flow chart showing the contents of "detection of 1-line horizontal feature point" 151L shown in FIG.
(B) is a target point (calculation point pixel: X mark) and a reference point (reference pixel: white) when calculating the differential value of the image data in the Y direction of the correction screen for detecting the horizontal feature point. It is a top view showing a square.

FIG. 33: “Histogram memory clear” 1 in FIG. 29
33 is a flowchart showing 33.

FIG. 34: “y-direction histogram creation” 13 in FIG.
4 is a flowchart showing No. 4.

FIG. 35 is a flowchart showing “y-direction maximum point detection” 135 in FIG. 29.

FIG. 36 is a flowchart showing “lane left end point calculation” 176 and “lane right end point calculation” 177 of FIG. 35.

FIG. 37 is a flowchart showing “calculation of center of gravity in x direction” 136 of FIG. 29.

FIG. 38 is a flowchart showing “detection of left and right ends of vehicle” H3 in FIG. 28.

FIG. 39 is a flowchart showing “Vehicle left and right end position estimation” 201 in FIG. 38.

FIG. 40 is a “left end vertical feature point detection” 203L in FIG.
Is a flowchart showing.

FIG. 41: “Histogram creation in x direction” 20 of FIG.
It is a flowchart showing 5L.

42 is a flowchart showing the "maximum point detection in x direction" 206L in FIG.

[FIG. 43] “Right edge vertical feature point detection” 203R in FIG.
Is a flowchart showing.

FIG. 44 is a flow chart showing “Vehicle verification” H4 in FIG. 28.
It is a chart.

FIG. 45 is a flowchart showing the “vehicle distance calculation” H6 of FIG. 28.

FIG. 46 is a flowchart showing “vehicle width learning calculation” H7 in FIG. 28.

47] "Right adjacent lane vehicle recognition and distance measurement" in FIG.
It is a flowchart showing I.

FIG. 48: “Detection of position 2 of vehicle candidate 2” I1 in FIG.
Is a flowchart showing.

FIG. 49 is a flowchart showing “horizontal feature point detection 2” 258 in FIG. 48.

50] "Left adjacent lane vehicle recognition and distance measurement" in FIG.
It is a flowchart showing J.

51. "Detection of position of vehicle candidate 3" J1 in FIG.
Is a flowchart showing.

52 is a flowchart showing the "horizontal feature point detection 3" 277 in FIG. 51. FIG.

FIG. 53 shows an outline of processing for detecting a white line on the road surface in front of the host vehicle and a preceding vehicle in the second embodiment of the microprocessor 1 shown in FIG. 1 based on the image data taken by the television camera 6b. It is a flow chart.

FIG. 54 is a flowchart showing “day / night judgment” S in FIG.

FIG. 55 is a plan view showing the positional relationship of each region in a two-dimensional image obtained by photographing.

56 is a flowchart showing the details of step S1 in FIG. 54.
It is a chart.

FIG. 57 shows small areas A1 to A16 defined in the area RGS.
It is a plan view showing the positional relationship of.

FIG. 58 is a flowchart showing the details of step S2 in FIG. 54.
It is a chart.

FIG. 59 is a plan view showing a pixel configuration of one small area.

FIG. 60 is a memory map showing numerical values assigned to respective elements on the two-dimensional memory array S_pos (S_cy, j).

FIG. 61 is a flowchart showing the details of step S3 in FIG. 54.
It is a chart.

FIG. 62 is a flowchart showing the details of step S4 in FIG.
It is a chart.

FIG. 63 is a plan view showing an example of an image taken at night.

FIG. 64 is a plan view showing an example of an image taken in the daytime.

FIG. 65 is a plan view showing an example of an image photographed in the evening.

FIG. 66 is a waveform diagram of a video signal in the period of one horizontal scanning line SL, and (a), (b) and (c) correspond to nighttime, daytime and evening signals, respectively.

FIG. 67 is a plan view showing six kinds of modified examples of the arrangement of the small area group on the empty area RGS.

FIG. 68 is a plan view showing six kinds of modified examples of the arrangement of the small region group on the empty region RGS.

FIG. 69 is a plan view showing six kinds of modified examples of the arrangement of the small area groups on the empty area RGS.

FIG. 70 is a flowchart showing a modification of the processing shown in FIG. 61.

71 is a flowchart showing another modification of the processing shown in FIG. 61.

72 is a flowchart showing a part of the process of another modification of the process shown in FIG. 61. FIG.

73 is a flowchart showing a modification of the processing shown in FIG. 62.

FIG. 74 is a flowchart showing another modification of the processing shown in FIG. 62.

FIG. 75 is a flowchart showing another modification of the processing shown in FIG. 62.

[Explanation of symbols]

1: Microprocessor (CPU) 2: ROM 3: RAM 4 to 6: Input / output port
4a: CRT driver 4b: CRT 5a: image memory 6a: camera controller 6b: television camera 6c: A / D converter 7: communication controller 8: host microprocessor (CPU) patent applicant Aisin Seiki Counsel attorney
Nobuoki Sugi

Claims (5)

[Claims] 1. A two-dimensional image information obtained by photographing a scene including a road surface outside a vehicle with a photographing device mounted on the vehicle is detected as a series of feature points in the image in a predetermined direction. In a vehicle recognition method for recognizing a vehicle in the two-dimensional image information with respect to a region determined by identifying a plurality of white lines on a road surface to be determined based on positions of a plurality of straight lines representing the white line, the vehicle travels. The position of the point at infinity on the lane is identified based on the position of the intersection of the plurality of straight lines, and the two-dimensional image information is scanned at a plurality of positions in the horizontal direction in a substantially vertical direction. , The first with changing brightness
Detecting a plurality of pieces of feature point information, identifying a vertical position where the appearance frequency of the detected plurality of pieces of first feature point information is equal to or greater than a predetermined threshold value as a vertical direction reference position, The horizontal position of the center of gravity is identified as the horizontal center of gravity position based on the distribution state of the plurality of first feature points in the vicinity of the direction reference position, and the object left end position predicted based on the horizontal direction center of gravity position. For a predetermined range in the vicinity of, the actual left end position of the object is identified based on the information of the second feature point obtained by scanning the two-dimensional image information in the horizontal direction, and based on the horizontal position of the center of gravity. For a predetermined range in the vicinity of the predicted right end position of the object, the actual right end position of the object is identified based on the information of the third feature point obtained by scanning the two-dimensional image information in the horizontal direction, The position of the point at infinity, The two-dimensional image from the photographing device based on the vertical reference position, the height of the position where the photographing device is installed, the height of the predetermined light emitting part on the vehicle from the ground, and the focal length of the photographing device. An estimated distance corresponding to the distance to the vehicle in the information is obtained, and at least the two-dimensional based on the distance between the identified left end position of the actual object and the right end position of the actual object, and the estimated distance. A vehicle recognition method, characterized in that the presence or absence of vehicle recognition based on image information is identified. 2. A two-dimensional image information obtained by photographing a scene including a road surface outside the vehicle with a photographing device mounted on the vehicle, and detecting it as a series of characteristic points in a predetermined direction in the image. In a vehicle recognition method for recognizing a vehicle in the two-dimensional image information with respect to a region determined by identifying a plurality of white lines on a road surface to be determined based on positions of a plurality of straight lines representing the white line, the vehicle travels. The position of the point at infinity on the lane is identified based on the position of the intersection of the plurality of straight lines, and the two-dimensional image information is scanned at a plurality of positions in the horizontal direction in a substantially vertical direction. , The first with changing brightness
Detecting a plurality of pieces of feature point information, identifying a vertical position where the appearance frequency of the detected plurality of pieces of first feature point information is equal to or greater than a predetermined threshold value as a vertical direction reference position, The horizontal position of the center of gravity is identified as the horizontal center of gravity position based on the distribution state of the plurality of first feature points in the vicinity of the direction reference position, and the object left end position predicted based on the horizontal direction center of gravity position. For a predetermined range in the vicinity of, the actual left end position of the object is identified based on the information of the second feature point obtained by scanning the two-dimensional image information in the horizontal direction, and based on the horizontal position of the center of gravity. The actual right end position of the object is identified based on the information of the third feature point obtained by scanning the two-dimensional image information in the horizontal direction in a predetermined range near the predicted right end position of the object, at least Day and night On the other hand, when it is identified as daytime, it is determined based on the position of the point at infinity, the vertical reference position, the height of the position where the image capturing device is installed, and the focal length of the image capturing device, and at night. If they are identified, the position of the point at infinity, the vertical reference position, the height of the position where the photographing device is installed, the height of a predetermined light emitting part on the vehicle from the ground, and the photographing device Based on the focal length, the estimated distance corresponding to the distance from the imaging device to the vehicle in the two-dimensional image information, respectively, is obtained, and the actual left end position of the actual object and the right end position of the actual object are identified. A vehicle recognition method, characterized in that the presence or absence of vehicle recognition based on at least the two-dimensional image information is identified based on the distance and the estimated distance. 3. When the actual left end position of the object is identified based on the information of the second feature points, the appearance frequency of the detected information of the plurality of second feature points is determined as a second threshold. In comparison with the value, the actual object left end position is determined based on the horizontal position where the appearance frequency is large, and when identifying the actual object right end position based on the information of the third feature point, The detected appearance frequency of the information of the plurality of third feature points is compared with the second threshold value, and the actual object right end position is determined based on the horizontal position where the appearance frequency is large, 3. The vehicle recognition method according to claim 2, further comprising discriminating at least day and night, and switching the magnitude of the second threshold value depending on whether it is identified as day or night. 4. When determining a region in the two-dimensional image information to be processed, at least day and night are identified, and on the vehicle depending on whether it is identified as day or night. The vehicle recognition method according to claim 2, wherein the position of the region is vertically shifted by an amount corresponding to the height of the light emitting portion from the ground. 5. A plurality of small areas are defined in a specific area expected to correspond to the sky in the two-dimensional image information, average brightness is obtained for each small area, and the obtained average brightness is calculated. The daytime and the night are discriminated based on the plurality of primary discrimination results after obtaining the discrimination result of the brightness of each small area as a primary discrimination result by comparing the different brightness with a threshold value. The vehicle recognition method according to claim 3 or claim 4.