JP2003174642A - Smear detecting method and image processor employing the smear detecting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a smear detecting method capable of detecting a smear with a small arithmetic amount at high speed even from an image adjusted at an automatic gain without missing image information imaged by a CCD camera and the need for two cameras. <P>SOLUTION: When the method adopts a CCD camera, the smear appears in a particular direction. Then the method selects a smear supervisory area 504 in a direction orthogonal to the smear 503 and a spot with high luminance in the supervisory area to be a smear supervisory object. When the luminance on the line in a direction in parallel with the smear 503 including the smear supervisory object is all high, the method discriminates the line to be a smear area. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus using a CCD camera as an image pickup means and a smear detecting method in this image processing apparatus.

[0002]

2. Description of the Related Art Cameras (CCD cameras, etc.
It is widely practiced to process the image data obtained by taking the image), extract the target image from the screen, and use it in various ways. By the way, when a high-intensity light source, such as the sun or a headlight of a car, appears in the captured image, there is a phenomenon called a so-called smear phenomenon in which a bright line is emitted from this light source. , It becomes an obstacle in performing the above-mentioned image processing. The smear phenomenon is a phenomenon in which the signal level of pixels of the CCD is saturated due to the incidence of intense light, and the saturation phenomenon is propagated in the transfer direction of pixel signals (generally, the vertical direction of the image). This is a phenomenon in which a linear high-intensity region is formed like a line (hereinafter, this high-intensity region is referred to as smear). In image processing of camera images, generally, there are many cases in which an object is detected by performing a calculation based on the brightness value of the image, and therefore smear is a cause of being erroneously detected as an object.

Here, the adverse effects of the occurrence of smear will be described with reference to an example. For example, by mounting a camera on the vehicle facing forward or backward, photographing the road including the preceding vehicle or the following vehicle, detecting an abnormality in the inter-vehicle distance between the front and rear vehicles, and issuing an alarm. There is a device. FIG. 9 shows a block diagram of such an inter-vehicle distance monitoring alarm device (hereinafter, abbreviated as monitoring alarm device). Note that, for convenience of explanation, the case of monitoring the rear side is described here, but the principle is the same for the front side. This device is a CCD
An approaching vehicle is detected from the rear road image captured by the camera, and an alarm is sounded if necessary. In this monitoring and warning device, the target object is detected by focusing on the movement amount of the target object on the screen. That is, the image captured to the current time T _n, based on the luminance difference of the captured images to the current time T _n-1, the differential value of the captured image to the current time T _n, existing in the image The moving amount of the moving body is detected, and the detected moving amount is used as a risk level indicating the approach of the following vehicle, and an alarm buzzer is sounded according to the risk level.

In the figure, 1 is a main body of a monitoring / warning device, 2 is an input interface for relaying signals of various external input devices (described later) in the main body, and 3 is image processing described below. Microprocessor to execute, 4
Is an output interface for relaying between the microprocessor 3 and various external output devices (described later), and 5 is a memory (hereinafter referred to as RAM) for exchanging information with the microprocessor 3, such as image processing information described later. Is read and written. The microprocessor 3 is connected to a system memory (hereinafter, ROM) not shown. RO
A program for executing various means described later is stored in M. 20 is C installed at the rear of the car
It is a rear monitoring camera (hereinafter referred to as a camera) such as a CD camera,
The camera 20 transmits an image signal according to the NTSC standard to the microprocessor 3. The microprocessor 3 converts the image signal of the camera 20 by the virtual screen creating means 30.
For example, the chromaticity information and the luminance information are written as, for example, 8-bit digital information (in short, the image is stored) in each image memory corresponding to each pixel on the virtual screen of 256 dots × 240 dots. ing. Two
Reference numeral 1 is a steering angle sensor that generates a signal in response to a rotation angle of a steering wheel angle of the automobile, and reference numeral 22 is a vehicle speed sensor of the automobile. The signal from the steering angle sensor 21 and the signal from the vehicle speed sensor 22 are used when the search line setting means 31 corrects the curve of the search line (described later).

Reference numeral 23 is a deceleration switch that is closed when the brake pedal of the vehicle is depressed or when the accelerator pedal is returned. The signal from the deceleration switch 23 is used by the comparison / determination means 33. Camera 20, steering angle sensor 21,
The respective signals from the vehicle speed sensor 22 and the deceleration switch 23 are supplied to the microprocessor 3 via the input interface 2. 40 is an alarm buzzer, 41 is L
It is an alarm indicator such as an ED. The alarm buzzer 40 and the alarm indicator 41 are driven by the microprocessor 3 via the output interface 4.

The virtual screen creating means 30 is means for writing the image signal input from the camera 20 into the memory 5 as a digital signal as described above. The written screen information is called a virtual screen, and is, for example, as shown at 300 in FIG. The search line setting means 31 displays a search line (3 in FIG. 11) for detecting a following vehicle on the virtual screen 300.
02L, 302R, etc.) and is necessary for understanding the present invention, and therefore will be described in some detail below as a precaution. The processing procedure of the search line setting means 31 is shown in the flowchart of FIG. In step S201 of FIG. 12, search lines 302L and 302R shown in FIG. 11 are initialized based on the installation height of the camera 20, the depression angle, the vehicle width, and the like. This initial setting is premised on traveling on a straight road. In step S202, it is determined whether the lane markings 301L and 301R for lane markings provided on the road surface can be detected. If it is determined that the lane markings 301L and 301R are detected in step S202, the lane markings 301L and 301R are detected and recognized, and the positions of the search lines 302L and 302R are corrected in step S204.

In step S204, step S203
The widths of the lane markings 301L and 301R detected in step S1 are shifted by a predetermined value, and the search lines 302R and 302L are corrected so as to be gradually reduced from the closest distance to the farther position. As shown in. When it is determined in step S202 that the detection is impossible, the input signals are read from the steering angle sensor 21 and the vehicle speed sensor 22 in step S205, and the curves of the search lines 302R and 302L are corrected based on the input signals in step S206. In step S205, the steering angle is read from the steering angle sensor 21 and the vehicle speed is read from the vehicle speed sensor 22. In step S206, the search line 302R, 3 is corrected so that the larger the vehicle speed is and the larger the steering angle is, the more quickly the curve is corrected, and the steering angle is returned to 0.
Perform on 02L. In step S207, the search lines 302R and 302L determined in the above step are divided into a plurality of line segment areas, and a threshold value for determining the degree of risk is set for each line segment area. It is preset as a value that gradually decreases as the position moves away from the position.

The risk calculating means 32 is a means for calculating the risk due to the approach of a following vehicle, and the processing procedure is shown in the flowchart of FIG. 15 and 16 are examples of a virtual screen of horizontal 256 × vertical 240 pixels written in the image memory, FIG. 15 is a virtual screen created in a certain step of the flow of FIG. 14, and FIG. 16 is repeated. It shows a virtual screen created in the next process performed. Figure 1
7 (a) to 7 (d) are diagrams for explaining the results of each processing performed on the search line 302R in FIG. In step S401, the search line 3 is calculated from the brightness data of the virtual screen written in the image memory by the virtual screen creating means 30.
Data of each pixel corresponding to 02R, 302L is extracted,
Create a brightness distribution. The brightness distribution is, for example, as shown in FIG. Reference numeral 601 denotes the luminance distribution of FIG. 15 created previously (referred to as old luminance distribution), and 602 is the luminance distribution of FIG. 16 created this time (referred to as new luminance distribution). At this time, a portion that appears brighter than the road surface has a mountain-shaped luminance distribution, so that the smear portion brighter than the road surface and the image of the following vehicle both have a mountain-shaped luminance distribution.

In step S402, the brightness difference distribution is created by subtracting the brightness extracted in step S401 of the previous process from the brightness extracted in step S401 of the current process. The brightness difference distribution is, for example, as shown by 603 in FIG. If the brightness value of the new brightness distribution 602 at the position of the pixel Z _{(m1, n1)} on the search line is A _{(m1, n1)} and the brightness value of the old brightness distribution 601 is B _{(m1, n1)} , the brightness difference is A
_{It is calculated by (m1, n1)} −B _{(m1 , n1)} .

In step S403, step S401
Create a differential value distribution of the brightness distribution created in. This is performed, for example, by obtaining the brightness difference between adjacent pixels along the search line of the brightness distribution 602. The differential value distribution is as shown by 604 in FIG. 17 (c), for example. If the brightness value of the new brightness distribution 602 at the position of the pixel Z _{(m2, n2 ) on} the right side (far direction) of the pixel Z _{(m1, n1)} on the search line is A _{(m2, n2)} , the differential value is A _{( It is calculated by m1, n1)} −A _{(m2, n2)} .

In step S404, step S402
The product of the luminance difference distribution 603 created in step S403 and the differential value distribution 604 created in step S403 is calculated and the search line 30 is calculated.
Create a synergistic product distribution along 2R, 302L. The synergistic product distribution is, for example, as shown by 605 in FIG.
The cross product is the brightness difference (A _{(m1, n1)} −B _{(m1, n1)} ) × differential value (A
_{It is calculated by (m1, n1)} −A _{(m2, n2)} ).

[0012] In the former brightness distribution 601, the pixel positions of the luminance A _{(m1, n1)} becomes equal to the luminance B _{(mx, nx)} and Z _{(m x, nx),} the phase difference (Z _{(m1, n1 )} −Z _{(mx, nx)} ) is a value that represents the approaching degree (relative speed) of the following vehicle. Brightness difference (A _{(m1, n1)} −
B _{(m1, n1)} ) has almost the same meaning as this phase difference (Z _{(m1, n1)} −Z _{(mx, nx)} ), and is regarded as a value representing the approaching degree (relative speed) of the following vehicle for convenience. obtain. However, when the approaching degree is represented by a brightness difference, it is necessary to use the positive / negative of the differential value (A _{(m1, n1)} −A _{(m2, n2)} ) as a judgment factor of whether or not the following vehicle is approaching. . That is, the product ((A _{(m1, n1)} −
When B _{(m1, n1)} ) × (A _{(m1, n1)} − A _{(m2, n2)} )) is negative, it is considered to be approaching, and when it is positive, the inter-vehicle distance is expanding.

Next, the comparison / determination means 33 shown in FIG. 9 will be described. The comparison / determination unit 33 is a unit that determines the warning output based on the degree of danger due to the approach of the following vehicle, and the processing procedure is shown in the flowchart of FIG. 1823. Step S1
In 01, the synergistic product distribution 6 obtained by the risk calculating means 32
Only the absolute value of the negative number is extracted from 05 and the search line 30
The maximum absolute value for each line segment obtained by dividing 2R and 302L is extracted. In step S102, the maximum absolute value extracted in step S101 is compared with a threshold value set in advance for each line segment. Step S103 is a repetitive process that acts to return to step S102 if the comparison for each line segment is not completed.

If there is a portion where the absolute value of the maximum product is larger than the threshold value in step S102, step S10
4 and it is determined whether the deceleration switch 23 is ON. If it is turned on, an alarm output is generated in step S105, and thereafter, at least for a certain period of time,
The alarm buzzer 40 is sounded and the alarm display 41 is blinked. If it is determined in step S102 that the maximum absolute values of all locations are equal to or less than the threshold value, or if it is determined in step S104 that the deceleration switch is OFF, the alarm output is stopped in step S106.

As described above, since the monitoring / warning apparatus calculates the amount of movement of the object by the optical flow and performs the image processing calculation for detecting the object, when the smear is present in the image, the smear is detected. Has a sufficiently high luminance, there is a problem that when the position of the smear changes due to the movement of the light source or the camera, the smear is erroneously detected as a moving object.

Of course, various types of smear detection methods and image processing techniques for avoiding the influence of the detected smear have been proposed in the past. For example, as a first method, Japanese Patent Laid-Open No. 2000-242793 discloses a white line recognition device that detects white line information in a road image captured by a CCD camera. The smear detection method of the white line recognition device will be described with reference to the captured road image in FIG.

In the figure, reference numeral 300 designates the entire photographing screen, and 301.
R is the left lane marking of the road taken, 301
L is also the right lane marking, 303 is the bumper of the vehicle shown in the screen, 502 is the sun (high brightness light source) shown in the screen, 503 is a smear, 60 is this screen 30
It is a light-shielding area (area covered so that nothing is imaged) provided in the field of view (here, the upper end) of a camera (not shown) that captures 0. Shields the upper end area of the camera's visual field 6
Since it was set to 0, nothing should originally appear in the light-shielded area 60 of the obtained road image, but smear also occurs in this area. Therefore, the high-brightness portion 61 in the light-shielded area 60 is detected, and the area extending straight downward from the high-brightness portion 61 is detected as the smear area 503. Since the original purpose of this image processing is to detect a white line on the road, a process of recognizing a high-intensity part in the screen as a white line is performed. However, when the smear 503 is detected, the smear area 503 is detected.
Is excluded from the candidates for the white line, thereby preventing the smear 503 from causing an erroneous detection. Here, the light shielding region 60 needs to have a certain width (dimension in the vertical direction in the drawing). The reason is that the high-luminance portion reaching from the upper end to the lower end of the light-shielded area is determined to be smear, and if the width is, for example, one pixel, it is determined to be smear due to the presence of noise, resulting in increased false detection.

As a second method, FIG. 20 is a configuration diagram of a traveling road detecting device for detecting a traveling road in a road image disclosed in Japanese Patent Laid-Open No. 10-141921. In this apparatus, two cameras, an ordinary sensitivity camera 251 that captures a travel path detection calculation image and a low sensitivity camera 252 that captures a smear detection calculation image, are installed so as to have the same imaging area. . In principle, the smear does not occur in the low-sensitivity camera 252, and a difference calculation is performed on the brightness-corrected calculation image for smear detection from the calculation image for road detection to obtain a high brightness of the obtained difference image. The area is detected as a smear area. The process after the smear detection is described in the above-mentioned Japanese Patent Laid-Open No.
The description is omitted because it is the same as that of Japanese Patent Publication No. 00-242793.

As a third method, FIG. 21 is a block diagram of a peripheral object recognition device disclosed in Japanese Patent Laid-Open No. 09-142208. This device generates an edge image of an image image (original image) captured by a CCD camera, and if the brightness value of a linear area including the coordinates of the edge point is equal to or greater than a threshold value, this edge is a smear area. Is detected as. When a smear is detected, an image obtained by interpolating an image in the smear area is used for processing to prevent erroneous detection due to smear.

[0020]

Since the conventional smear detection method is constructed as described above, the method of setting the light-shielding area in a part of the image cannot use a part of the image information. There was a point. Therefore, there is a problem that it is suitable for an image processing apparatus that requires only a limited range of image information, but is not suitable for an image processing apparatus that requires image information of the entire imaging region.

Further, in the method of installing a low-sensitivity camera for obtaining an image for smear detection in addition to the camera for obtaining an image used for the original image processing, there is a problem that the number of cameras becomes two and the cost becomes high. there were. Also, brightness correction,
Since a process with a large amount of calculation such as a difference process is required, the calculation time becomes long, and there is a problem that it is not suitable for an image processing apparatus that requires high-speed processing.

Further, in the method of performing edge image processing,
Since it requires a large amount of image processing, it is not suitable for applications requiring high-speed processing. Further, since the smear detection condition is the brightness of the brightness value, there is a problem that the smear cannot be detected in the image adjusted by the auto gain.

Further, in the conventional image processing apparatus for calculating the movement amount of the object by the optical flow or the like, in the image processing calculation for detecting the object, when the position of the smear changes due to the movement of the light source or the camera, the smear There has been a problem that a change in position is detected as a movement amount and is falsely detected as an object.

The present invention has been made in order to solve the above-mentioned problems, and does not lose image information picked up by a CCD camera, does not require two cameras, and is adjusted by auto gain. It is an object of the present invention to provide a smear detection method capable of detecting smear at high speed with a small amount of calculation even for an image. Another object of the present invention is to provide an image processing device using this smear detection method. Another object of the present invention is to provide an image processing apparatus provided with a moving body detection unit that does not cause false detection due to smear.

[0025]

A smear detecting method according to the present invention comprises a smear monitoring area setting procedure for setting a smear monitoring area on a screen in a direction orthogonal to a direction of a smear expected to occur. For all of the first pixels forming the monitoring area, the brightness of each first pixel and the brightness of each second pixel existing at a position apart from each first pixel by a predetermined number of pixels in the direction orthogonal to the smear A smear candidate pixel detection procedure of obtaining a brightness difference and detecting the first pixel having a brightness difference equal to or larger than a predetermined threshold as a smear candidate pixel, and a smear candidate pixel existing on a straight line parallel to the smear including the smear candidate pixel. The brightness difference between the brightness of three pixels and the brightness of the fourth pixel existing at a position that is a predetermined number of pixels away from the third pixel in the direction orthogonal to the smear is obtained, and the brightness difference is equal to or more than a predetermined threshold value. If there is, An area including the serial smear candidate pixel and said third pixel is intended to include the and determining the smear determining step is a smear generation area.

Further, it includes a smear determining procedure for monitoring the change of the determination result by the smear determining procedure in time series and determining the smear when the determination of the smear is continued for a predetermined number of times.

In the image processing apparatus according to the present invention, an image pickup means for producing a smear in a specific direction with respect to a strong light source, and a linear smear monitoring area orthogonal to the smear on the screen obtained by the image pickup means are set. Smear monitoring area setting means for all of the first pixels forming the monitoring area,
The brightness difference between the brightness of each first pixel and the brightness of a second pixel existing at a position that is a predetermined number of pixels away from each first pixel in the direction orthogonal to the smear is obtained, and this brightness difference is a predetermined threshold value. The smear candidate detection means for detecting the first pixel as a smear candidate pixel, the brightness of the third pixel existing on a straight line parallel to the smear including the smear candidate pixel, and the smear from the third pixel On the other hand, the brightness difference from the brightness of the fourth pixel existing at a position separated by a predetermined number of pixels in the orthogonal direction is obtained, and when the brightness difference is equal to or more than a predetermined threshold value, the smear candidate pixel and the third pixel The smear determining means is provided for determining that the area including is a smear occurrence area.

Further, the smear monitoring area setting means sets, on the screen obtained by the image pickup means, for each of a plurality of lines set in a direction orthogonal to the smear, an average luminance value of pixels on the line. After the determination, the line having the lowest time-series average value of the average luminance values of the pixels in the line is the smear monitoring region.

Further, on an image pickup means for producing smear in a specific direction with respect to a strong light source, on a screen obtained by the image pickup means,
Smear monitoring area setting means for setting a linear smear monitoring area orthogonal to the smear, the brightness of each first pixel, and the smear from each first pixel for all of the first pixels forming the monitoring area. A smear for detecting a brightness difference from the brightness of a second pixel existing at a position separated by a predetermined number of pixels in the orthogonal direction with respect to the first pixel whose brightness difference is a predetermined threshold value or more as a smear candidate pixel. Candidate detecting means, luminance of all third pixels including the smear candidate pixel and existing on a line parallel to the smear, and a predetermined number of pixels away from each of all the third pixels in the direction orthogonal to the smear. The brightness difference from the brightness of the fourth pixel existing at the position is found, and if the ratio of the pixels having the calculated brightness difference is equal to or more than a predetermined threshold value is equal to or more than a predetermined ratio, the smear is included in the smear candidate pixel. Parallel Are those with a smear determining means for determining that the smear generation area.

Further, on an image pickup means for producing smear in a specific direction with respect to a strong light source, on a screen obtained by the image pickup means,
Smear monitoring area setting means for setting a linear smear monitoring area orthogonal to the smear, the brightness of each first pixel, and the smear from each first pixel for all of the first pixels forming the monitoring area. A smear for detecting a brightness difference from the brightness of a second pixel existing at a position separated by a predetermined number of pixels in the orthogonal direction with respect to the first pixel whose brightness difference is a predetermined threshold value or more as a smear candidate pixel. Candidate detection means, luminance of a plurality of pixels existing on a line parallel to the smear including the smear candidate pixel, and present at a position separated from each of the plurality of pixels by a predetermined number of pixels in a direction orthogonal to the smear occurrence. If the ratio of the pixels whose brightness difference is greater than or equal to a predetermined threshold value is greater than or equal to a predetermined ratio, the line parallel to the smear including the smear candidate pixel is smeared. Those having a smear determining means for determining that the generation region.

Further, the image processing means for image-processing the image processing object in the screen obtained by the image pickup means and extracting the image, the area determined as the smear occurrence area is excluded from the image processing objects of the screen. It is provided with an excluding means to do so.

Further, the image pickup means is mounted toward the front or rear of the vehicle, and the smear monitoring area is near the bumper image of the vehicle photographed on the screen obtained by the image pickup means. It has been set.

Further, the image extracting means includes an extracting means for calculating a moving amount of the image within the screen and extracting an image for which the moving amount exceeds a predetermined level as a target. It is provided with an erroneous detection prevention unit that excludes the movement amount of the image in the smear occurrence region determined by the smear determination unit from the extraction target.

The smear monitoring area has a width of one pixel. The image pickup means uses a CCD camera.

[0035]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1. 1 is a block diagram of an image processing apparatus using the smear detection method according to the first embodiment of the present invention. In order to explain in a specific case, here, a monitoring / warning device will be described as an example, which is mounted on a vehicle similar to that described in the conventional example, monitors the rear, and issues a warning about an inter-vehicle distance. In FIG. 1, reference numeral 1 is a main body portion of a vehicle rear monitoring / warning device, 2 is an input interface for relaying signals of various external input devices (described later) inside the main body portion, and 3 is image processing described below. 4 is an output interface for relaying between the microprocessor 3 and various external output devices (described later), and 5 is a memory (hereinafter RAM) for exchanging information with the microprocessor 3. Image processing information, which will be described later, is read and written in the RAM. Further, the microprocessor 3 is connected to a system memory (hereinafter referred to as ROM) not shown, and the ROM stores programs for executing various means described later.

Reference numeral 20 denotes a rear monitoring camera (hereinafter referred to as a camera) such as a CCD camera installed in the rear portion of the automobile, and the camera 20 sends an image signal according to the NTSC standard to the microprocessor 3. Microprocessor 3 is camera 2
The image signal of 0 is output by the virtual screen creating means 30 to each image memory corresponding to each pixel on the virtual screen of, for example, 256 dots × 240 dots, and the chromaticity information and the brightness information are, for example, 8-bit digital information. Is written (in short, the image is stored). Reference numeral 21 is a steering angle sensor that generates a signal in response to the rotation angle of the steering wheel angle of this vehicle, and 22 is a vehicle speed sensor of this vehicle. The signal from the steering angle sensor 21 and the vehicle speed sensor 2
The signal from 2 is used when the search line setting unit 31 corrects the curved search line (described later). Reference numeral 23 denotes a deceleration switch that is closed when the brake pedal of the vehicle is depressed or when the accelerator pedal is returned. The signal from the deceleration switch 23 is used by the comparison / determination means 33. Camera 2
Signals of 0, the steering angle sensor 21, the vehicle speed sensor 22, and the deceleration switch 23 are supplied to the microprocessor 3 via the input interface 2. Reference numeral 40 is an alarm buzzer, and 41 is an alarm indicator such as an LED. The alarm buzzer 40 and the alarm indicator 41 are driven by the microprocessor 3 via the output interface 4.

The virtual screen creating means 30 is means for writing the image signal input from the rear monitoring camera 20 into the memory 5 as a digital signal as described in the conventional example. The written screen information is called a virtual screen. The search line setting means 31 is a means for setting a search line for detecting a following vehicle on the virtual screen and has the same function as that described in the conventional example, and therefore its description is omitted. The risk calculating means 32 is a means for calculating the risk due to the approach of the following vehicle, but the description thereof is omitted because it has the same function as described in the conventional example. Comparison judgment means
Reference numeral 33 is a means for making a warning output judgment based on the degree of danger due to the approach of the following vehicle, but the description thereof is omitted because it has the same function as described in the conventional example.

Reference numeral 50 is a smear detecting means according to the present invention, and inside thereof, a smear monitoring area setting means 50a described later.
, Smear candidate detecting means 50b, and smear determining means 50
Including c and. Here, the smear candidate detecting means 50b is also referred to as a lateral direction detecting means. The smear determination means 50c is also referred to as vertical direction detection means. The processing procedure is shown in the flowcharts of FIGS. 3 and 5, respectively. Further, in order to facilitate understanding, a road image for explanation is shown in FIG. 2, and an example of processed data is shown in FIGS. 4 and 5. First, the operation of the smear monitoring area setting means 50a will be described with reference to FIG. In step S700a, as shown in FIG. 2, the smear monitoring area 504 is set by selecting an area in the screen 300 where only the road is considered to be normally reflected in the area and the brightness value should be low. To do. In the example of FIG. 2, the smear monitoring area 504 is set as one line having a one-pixel width immediately above the vehicle bumper image 303 in a direction (horizontal on the screen) orthogonal to a smear that may occur. is doing. This procedure is called a smear monitoring area setting procedure and is performed by the smear monitoring area setting means 50a.

Next, the smear candidate detecting means 50b will be described. In step S701, the virtual screen creating means 3
At 0, data of each pixel corresponding to the smear monitoring area 504 (pixels forming the smear monitoring area is referred to as a first pixel) is extracted from the brightness data of the virtual screen written in the memory 5 at 0, for example, as shown in FIG. ) As shown in the figure, the luminance distribution 801
To create. The horizontal axis of the position on the virtual screen 300 in FIG.
0-255, vertical axis j = 0-239, each first pixel is C
_{(i, j)} (i = 0 to 255, j = 0 to 239), and the brightness value of C _{(i, j)} is P
_{Expressed as} 1 _{(i, j)} . This way, for example, from the bottom of the screen
It is assumed that all pixels in the l-th row are represented by C _{(i, l)} (i = 0 to 255), and the luminance value of C _{(i, l)} is represented by P1 _{(i, l)} (i = 0 to 255). For convenience of description, the coordinates of the smear monitoring area 504 are i = 0 to 2
55, j = y, the coordinates of each first pixel on the monitoring area 504 are C _{(i, y)} (i = 0 to 255), and the brightness value of C _{(i, y)} is P1 _{(i, y) . y)} (i = 0 to 255). Step S70
In FIG. 2, for each first pixel on the monitoring area 504, the brightness difference between the pixels existing three pixels on the left and right of the first pixel (both are referred to as the second pixel) is obtained, and the smaller brightness difference is the processed value. And perform the process. That is, the smear monitoring area 50
P2 _{(i, y)} (i = 0-255)
Then, P2 _{(i, y)} = Min (P1 _{(i, y)} -P1 _{(i-3, y)} ,
It is represented by P1 _{(i, y)} -P1 _{(i + 3, y)} ). Although it has been described here that the bright area is detected by using the pixels existing next to the three pixels on the left and right (maximum width is 5 pixels), the pixels (second pixels) used for comparison are selected according to the width of the area to be detected. The position may be determined. In the diagram of FIG. 4B, the luminance difference graph P obtained by performing the process of step S702 on the luminance distribution 801 is shown.
It is 2.

In step S703, step S702
Value P2 obtained in_{(i, y)}And a predetermined threshold value are compared.
Value P2_{(i, y)}Is above the threshold, that is, pixel C
_{(i, y)} If you can determine that is brighter than the left and right pixels,
1 is added to the pixel of
Perform value conversion processing. FIG. 4C shows a brightness difference graph P2.
This is the binarized data. In step S704, the step
Pixel C whose value is 1 after the binarization processing in step S703 _{(i, y)}
Is a smear candidate pixel (indicating that a smear is
Supplement) to extract. In this embodiment, the maximum width is 5
Bright areas up to pixels are detected, so it is better than surrounding pixels
Pixels that are judged to be bright exist in a row in the horizontal direction.
There is a case. In such cases, the true
Pixels that exist inside or have the brightest brightness
Only the smear candidate pixel may be used as the smear candidate pixel. Step S701
From step S704 is a smear candidate pixel detection procedure.
Smear candidate detection means 50b.

Next, the operation of the smear determining means 50c (vertical direction detecting means) of FIG. 1 will be described with reference to the flowchart of FIG. 5 and the processing data explanatory diagram of FIG. Step S70
In 5a, the luminance distribution P _{(k, j)} , (j = 0 to 239) is first obtained for the pixels existing in the vertical direction of the smear candidate pixels extracted in step S704 of FIG. 3 (referred to as third pixels). . This is shown in FIG. Next, with respect to each of the third pixels, similarly to step S702, a brightness difference comparison process is performed with respect to pixels at positions three pixels left and right (both are referred to as fourth pixels). That is, if the smear candidate pixel is C _{(k, y)} , its luminance value is P1 _{(k, y)} , and the third pixel existing in the vertical direction is C _{(k, y)} .
_{(k, j)} (j = 0 to 239) and its brightness value is P1 _{(k, j)} (j = 0
~ 239), and the value after the brightness difference comparison process is P2 _{(k, j)} , P2 _{(k, j)} = Min (P1 _{(k, j)} -P1 _{(k-3, j)} , P
1 _{(k, j)} -P1 _{(k + 3, j)} ). FIG. 6B shows a pixel C existing in the vertical direction of the smear candidate pixel C _{(k, y).
For (k, j)} (j = 0 to 239), step S705
11 is a vertical luminance difference graph P2 that has been subjected to the processing of FIG. In step S706, the brightness difference value P2 _{(k, j)} obtained in step S705 is compared with a predetermined threshold value, and when the value is equal to or more than the threshold value, that is, the pixel C _{(k, j)} is in the left and right surroundings. If it can be determined that the pixel is brighter than the pixel of 1, the value after processing is given 1 and if it is less than the threshold value, 0 is given.
The threshold value used here is the value P obtained in step S702.
2 _{(i, y)} −α (α is a possible error). FIG. 6C is data 806 obtained by binarizing the brightness difference graph P2.

In step S707, the smear candidate pixel (C _{(k, y)} ) is determined to be a smear only when all the pixels existing in the vertical direction of the smear candidate pixel are brighter than the surrounding pixels.
Otherwise, it is determined that the smear candidate pixel is excluded from the candidates. Assuming that the smear determination flag representing the result after the determination is S1 _(k) , for example, if the results of step S706 are all 1, S1 _{(k) is set} to 1 and the process proceeds to the next step S708. In this case, S1 _{(k) is set} to 0 and the process proceeds to step S709. Of course, 1 means smear and 0 means not smear. Steps S705a to S707 are called a smear determination procedure according to the present invention, and are executed by the smear determination means 50c.

Next, a smear determination procedure based on time series processing will be described. Since the flow of FIGS. 3 and 5 is repeated, data can be obtained in time series. In step S708, changes in the smear determination flag S1 _(k) are monitored in time series, as shown in FIG. 7A, and temporary noise such as road stains and displays appearing on the monitoring area 504. Erroneous detection due to (identification by disappearance of data immediately) is removed, and the final determination and determination of whether or not the pixel C _{(k, y)} determined as smear in step S707 is smear is performed. The smear determination flag indicating the result after this determination is set to S2 _(k) ,
The pixel C _{(k, y)} is set to 1 when it is a smear, and is set to 0 when it is not a smear. As shown in FIG. 7B, the value of S2 _(k) is changed to 1 when S1 _(k) is 1 in a time series, for example, three times or more consecutively, and is changed three times or more. If it is 0 continuously, it is changed to 0. Step S708 is called a smear confirmation procedure.

In this embodiment, smear with a maximum width of 5 pixels can be detected. Therefore, when S2 _{(k )} is 1, the smear is orthogonal to the pixel C _{(k, y).} Pixel C with a maximum width of 5 pixels in the direction (horizontal direction in FIG. 2)
Pixel region C _{(k-2, j)} to C _{(k + 2, j)} (j that exists in the vertical direction including _{(k-2, y)} to C _{(k + 2, y)} (Z = 3) = 0 to 239), it is considered that smear has occurred, so this area is determined to be a smear area. In the present embodiment, the number of continuous determination criteria for time-series determination is three times, but the determination is not limited to this and may be determined according to the device and environmental conditions.

Step S709 is a smear determination completion confirmation step of determining whether or not smear determination has been completed for all smear candidates and returning to step S705 if not, and smear determination is performed for all smear candidates. If it is completed, the process proceeds to the operation repeating step S710, and the smear detecting means 50 is terminated (repeated as necessary).

The smear deleting means 51 for deleting the smear area 503 in FIG. 2 so as not to be erroneously recognized as an object is a synergistic product distribution (605 in FIG. 17 (d) of the conventional example, which represents the moving amount of the image). It can be said that it is represented), and means for deleting a portion determined to be a smear area (referred to as erroneous detection prevention means).
Specifically, for the synergistic product distribution 605 on the search line obtained by the risk calculating means 32, the synergistic product of the pixels in the smear area 503 detected by the smear detecting means 50 is forcibly replaced with 0. Then, the process of excluding it from the detection target is performed.
The means for obtaining the synergistic product distribution is the moving image extracting means referred to in the present invention. FIG. 8A is a diagram showing the synergistic product distribution 605 on the search line obtained by the risk calculating means 32 and the smear areas C _{(k-2, j)} to C _{(k + 2 )} detected by the smear detecting means 50. _{, j)} (j
= 0) to 239), and FIG. 8B shows the synergistic product distribution 901 after the deletion processing. Further, not only may be excluded from the extraction target of the moving image, but it may be excluded from all the image extraction processing targets that are generally performed. In the above description, the width of the smear monitoring area 504 has been described as one pixel, but of course, the width may be a plurality of pixels. The narrower the width, the greater the number of erroneous determinations due to the mixing of noise.However, in the present invention, since vertical and horizontal smear determinations are performed, the probability that noise will enter both determinations is extremely low. Even if noise is mixed in the smear determination and the erroneous detection is performed, the smear determination is eliminated in the next vertical detection.
Therefore, the width of the smear monitoring area may be narrow. The narrower the width, the shorter the processing time. Also, smear monitoring area 5
Since 04 is not shielded as in the conventional case, general image processing can be performed, and the screen is not reduced.

Embodiment 2. In the above description, the condition at the time of smear determination is that all the pixels existing in the vertical direction of the smear candidate are brighter than the surrounding pixels, but the determination condition may be changed according to the specifications. For example, among the pixels existing in the vertical direction of the smear candidate, if the ratio of those brighter than the surrounding pixels is several tens% or more, smear can be detected even in an image containing noise. Step S705 for smear determination
It is described that the smear is performed for all the pixels existing in the vertical direction of the smear candidate. However, since the smear is stable in the vertical direction, the pixels are thinned out and processed (for example, every several pixels are processed). If only one processing is performed), the amount of calculation can be reduced, so that smear can be detected at higher speed.

Embodiment 3. In the above description, the smear monitoring area 504 is located right above the bumper image 303 of the vehicle, which is considered to have low brightness, in a direction orthogonal to the smear expected to occur (horizontally toward the screen). )
The line having a width of 1 pixel is set so that as a more generalized method, the virtual screen creating means 30 searches the brightness data written in the image memory 5 for an area having a low brightness value, and this is used. The smear monitoring area 504 may be automatically set. Specifically, first, a plurality of lines that are orthogonal to the generated smear (horizontal direction) are set, and the process of obtaining the average luminance value of the pixels existing on each line is repeated for a predetermined time. . Next, the average value of the average luminance values for the predetermined time obtained for each line is calculated, and one line having the smallest value may be selected as the smear monitoring area 504. In essence, this is extracting the darkest of the resulting screens.

In the above description, a camera for monitoring the rear of the vehicle is used, but the invention is not limited to this. For example, image processing using a front monitoring camera, a factory monitoring camera, or the like. It goes without saying that it can be applied.

[0050]

As described above, according to the smear detection method of the present invention, smear candidate pixels are detected from the obtained screen, and whether smear is detected based on the luminance difference in the direction orthogonal to the smear candidate pixels. Since it determines whether or not it is possible to use the entire screen as an image processing target, it does not require a camera for capturing an image dedicated to smear detection, only one camera is required, and the amount of calculation is small. The effect is obtained.

Further, the judgment result is monitored in time series, and when the smear judgment result continues for a predetermined number of times, it is judged as smear, so that erroneous judgment due to temporary noise can be avoided. it can.

In the image processing apparatus of the present invention, the smear monitoring area setting means for setting the smear monitoring area on the obtained screen, and the smear candidate pixel detecting means for detecting the high brightness portion in this area as the smear candidate pixel. And a smear determination means for determining that the smear area is a smear area if the brightness of the pixels on the line including the smear candidate pixel and parallel to the smear is higher than a predetermined level, the entire screen including the smear monitoring area is imaged. It can be used as a processing target, and only one camera is required, and the calculation amount can be reduced. In addition, since smear detection is performed using the brightness difference from surrounding pixels, smear can be detected even from an image whose brightness is adjusted by auto gain or the like.

Further, the brightness of all the pixels existing on the straight line including the smear candidate pixel and parallel to the smear, and the positions separated from each of the pixels by a predetermined number of pixels in the direction orthogonal to the smear. The brightness difference with the brightness of the pixels existing in the pixel is found, and if the ratio of the pixels whose calculated brightness difference is greater than or equal to a predetermined threshold is greater than or equal to the predetermined ratio, it is determined that all pixels are smear occurrence areas. Since the determination means is provided, the reliability of smear determination is high.

Further, the brightness of a plurality of pixels existing on a straight line including the smear candidate pixel and parallel to the smear, and the position separated from each of the plurality of pixels by a predetermined number of pixels in the direction orthogonal to the smear. The brightness difference with the brightness of the pixel existing in the, the brightness difference is a predetermined threshold value or more, if the ratio of pixels is a predetermined ratio or more, it is determined that the plurality of pixels is a smear occurrence region Since the smear determination means is provided, the processing speed of smear determination is high.

Since the smear monitoring area setting means selects the darkest line from the plurality of lines as the smear monitoring area, the smear monitoring area setting means can be used for unspecified purposes.

Further, since the area determined as the smear occurrence area is excluded from the object of image processing, erroneous image extraction can be avoided.

Further, since there is an image pickup means mounted toward the front or the rear of the vehicle, and the smear monitoring area setting means is set in the vicinity of the bumper of the own vehicle photographed on the screen, the other vehicle's Light does not enter easily and it does not easily malfunction.

Further, since the moving image extracting means for extracting the moving image and the erroneous detection preventing means for excluding the image in the smear from the moving extraction target are provided, the erroneous detection of the extraction of the moving image in the screen is prevented. be able to.

Further, since the smear monitoring area has a one-pixel width, the amount of calculation for smear monitoring is small, and the processing speed can be increased. Further, since the image pickup means is a CCD camera, smear detection is performed more effectively.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of a screen for explaining the flows of FIGS. 2 and 3.

FIG. 3 is a flowchart illustrating an operation of a lateral direction detecting unit of the smear detecting unit of FIG.

FIG. 4 is a diagram illustrating a processing result of the flow of FIG.

5 is a flow chart for explaining the operation of the vertical direction detection means of the smear detection means of FIG.

FIG. 6 is a diagram illustrating a processing result of the flow of FIG.

FIG. 7 is a diagram illustrating a processing result of the flow of FIG.

FIG. 8 is a diagram for explaining the operation of the smear deleting means of FIG.

FIG. 9 is a configuration diagram of a conventional image processing apparatus.

10 is an explanatory diagram of a virtual screen for explaining the operation of FIG.

FIG. 11 is an explanatory diagram of search lines on the virtual screen of FIG.

12 is a flowchart illustrating the operation of the search line setting means in FIG.

FIG. 13 is an explanatory diagram of a situation where the search line in FIG. 12 is corrected.

FIG. 14 is a flowchart for explaining the operation of the risk degree calculating means of FIG.

FIG. 15 is a diagram of a screen explaining an object extracting operation.

FIG. 16 is a diagram of a screen explaining an object detection operation.

FIG. 17 is an explanatory diagram of an operation of calculating a risk of an inter-vehicle distance.

FIG. 18 is a flowchart illustrating the operation of a conventional comparison / determination unit.

FIG. 19 is a screen diagram illustrating a first conventional smear detection method.

FIG. 20 is a configuration diagram of an image processing apparatus for explaining a second conventional smear detection method.

FIG. 21 is a configuration diagram of an image processing apparatus for explaining a third conventional smear detection method.

[Explanation of symbols]

1 monitoring and alarm device body, 2 input interface, 3 CPU, 4 output interface,
5 memory, 20 camera, 21 steering angle sensor,
22 vehicle speed sensor, 23 deceleration switch, 30
Virtual screen creating means, 31 search line setting means, 32
Risk calculation means, 33 comparison determination means, 40 alarm buzzer, 41 alarm display, 50 smear detection means, 50a smear monitoring area setting means, 50b smear candidate detection means, 50c smear determination means, 51 smear portion deletion means, 300 Virtual screen, 30
1R lane marking (right), 301L lane marking (left), 302R search line (right), 302
L search line (left), 303 own vehicle bumper, 501
Following vehicle, 502 smear, 503 strong light source (sun), 504 smear monitoring area, 505 road display.

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C024 BX04 CX13 GY01 HX29 JX09                 5C054 AA01 EA01 FC05 FC12 HA30

Claims (11)

[Claims] 1. A smear monitoring area setting procedure for setting a smear monitoring area on a screen in a direction orthogonal to a direction of smear expected to occur, and all of the first pixels forming the monitoring area, The brightness difference between the brightness of each first pixel and the brightness of a second pixel existing at a position that is a predetermined number of pixels away from each first pixel in the direction orthogonal to the smear is obtained, and this brightness difference is a predetermined threshold value. The smear candidate pixel detection procedure for detecting the first pixel as a smear candidate pixel, the brightness of the third pixel existing on a straight line parallel to the smear and including the smear candidate pixel, and the third pixel A brightness difference from the brightness of the fourth pixel existing at a position that is a predetermined number of pixels away from the smear in the orthogonal direction is obtained, and when the brightness difference is equal to or larger than a predetermined threshold value, the smear candidate pixel and the third pixel The area containing the pixel and Smear detection method characterized by including the and determining the smear determining step is Mia generation region. 2. A smear determining procedure for observing a change in the determination result by the smear determining procedure in time series, and determining the smear when the smear determination continues for a predetermined number of times. The smear determination method according to claim 1. 3. An image pickup means for producing a smear in a specific direction with respect to a strong light source, and a smear monitoring area setting means for setting a linear smear monitoring area orthogonal to the smear on a screen obtained by the image pickup means. , Luminance of each first pixel and luminance of each second pixel existing at a position distant from the first pixel by a predetermined number of pixels in the direction orthogonal to the smear for all of the first pixels constituting the monitoring area And a smear candidate detecting means for detecting the first pixel having a brightness difference of not less than a predetermined threshold value as a smear candidate pixel, a smear candidate pixel existing on a straight line parallel to the smear including the smear candidate pixel. The brightness difference between the brightness of three pixels and the brightness of the fourth pixel existing at a position that is a predetermined number of pixels away from the third pixel in the direction orthogonal to the smear is obtained, and the brightness difference is equal to or more than a predetermined threshold value. If there is, An image processing apparatus comprising: a smear determining unit that determines an area including the smear candidate pixel and the third pixel as a smear occurrence area. 4. An image pickup means for causing smear in a specific direction with respect to a strong light source, and a smear monitoring area setting means for setting a linear smear monitoring area orthogonal to the smear on a screen obtained by the image pickup means. , Luminance of each first pixel and luminance of each second pixel existing at a position distant from the first pixel by a predetermined number of pixels in the direction orthogonal to the smear for all of the first pixels constituting the monitoring area And a smear candidate detection means for detecting the first pixel whose brightness difference is equal to or more than a predetermined threshold value as a smear candidate pixel, and all of the smear candidate pixels existing on a line parallel to the smear. The luminance difference between the luminance of the third pixel and the luminance of the fourth pixel existing at a position distant by a predetermined number of pixels in the direction orthogonal to the smear from each of all the third pixels, and the obtained luminance difference Is predetermined When the ratio of pixels that are equal to or greater than a threshold value is equal to or higher than a predetermined ratio, a line including the smear candidate pixel and parallel to the smear is provided with a smear determination unit that determines that the line is a smear occurrence region. Processing equipment. 5. An image pickup means for producing a smear in a specific direction with respect to a strong light source, and a smear monitoring area setting means for setting a linear smear monitoring area orthogonal to the smear on a screen obtained by the image pickup means. , Luminance of each first pixel and luminance of each second pixel existing at a position distant from the first pixel by a predetermined number of pixels in the direction orthogonal to the smear for all of the first pixels constituting the monitoring area And a smear candidate detecting means for detecting the first pixel having a brightness difference of not less than a predetermined threshold value as a smear candidate pixel, a plurality of smear candidate pixels existing on a line parallel to the smear. The brightness difference between the brightness of the pixel and the brightness of the pixel existing at a position that is a predetermined number of pixels away from each of the plurality of pixels in the direction orthogonal to the occurrence of smear is found. Ah When the ratio of the pixels to be smeared is a predetermined ratio or more, a line including the smear candidate pixel and parallel to the smear is provided with a smear judging means for judging that the line is a smear occurrence area. . 6. The smear monitoring area setting means, for each of a plurality of lines set in a direction orthogonal to the smear on the screen obtained by the image pickup means, sets an average luminance value of pixels on the line. 6. The smear monitoring area is defined as the line having the lowest time-series average value of the average luminance values of the pixels after the determination. The image processing device according to any one of claims. 7. An image extracting means for performing image processing on an image processing target in a screen obtained by the image capturing means to extract the image, and excluding an area determined to be the smear occurrence area from the image processing target. The image processing apparatus according to claim 3, further comprising a unit. 8. The image pickup means is mounted toward the front or the rear of the vehicle, and the smear monitoring area is in the vicinity of a bumper image of the vehicle photographed on the screen obtained by the image pickup means. It is set, The claim 3 characterized by the above-mentioned.
The image processing apparatus according to any one of items 1 to 5. 9. The image extracting means includes moving image extracting means for calculating a moving amount of the image within the screen and extracting an image in which the moving amount exceeds a predetermined level. The image processing apparatus according to claim 7, further comprising: an erroneous detection prevention unit that excludes a movement amount of the image in the smear occurrence region determined by the smear determination unit from a determination target of the level. 10. The image processing apparatus according to claim 3, wherein the smear monitoring area has a width of one pixel. 11. The image processing apparatus according to claim 3, wherein the image pickup unit is a CCD camera.