JP2002123820A - Detecting method and device for obstacle being stationary on road obstacle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detecting method and a device for obstacle being stationary on a road capable of detecting an obstract being stationary on a road in a camera image, preventing a false background update liable to occur in a small region having no clear pattern, and reducing the probability of false detection. SOLUTION: The stationary obstacle not existing in normal state and hindering traffic such as a dropped object in the environment including a moving object on a road or a passage is detected by processing the image of a stationary single camera in this method and device. The image sampled from a dynamic image is split into small regions. The calculation of a normalized correlation value with the region corresponding to the background image having no moving object prepared in advance and no obstacle in each small region and an average luminance value in the small region is repeated for the sampled image at every fixed time. The change quantity of the correlation value and the change quantity of the average luminance value are judged with thresholds, the background image is properly updated based on the judgment and evaluation of them, thereby the stationary obstacle on the road is detected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a method and an apparatus for detecting a stationary obstacle on a road.

[0002]

2. Description of the Related Art There is an image processing technique as a technique for detecting abnormalities in a road or a building in a non-contact manner. This method processes images of a single still camera, so that it is not present in a normal condition such as a falling object in an environment including moving objects on a road or a passage, and it is a stationary obstacle that does not exist and hinders traffic It detects objects.

As a technique for detecting an obstacle, there are many techniques such as a vehicle and a mobile robot that detect an obstacle in the traveling direction while moving. However, it is possible to detect an obstacle even when they are not moving. These devices measure the distance using a stereo camera, an ultrasonic sensor, a laser or the like to grasp three-dimensional irregularities, and determine that there is an obstacle when the vehicle is higher than a road surface or a floor surface. Thereby, a moving / stationary object on the road is detected.

Further, there are the following methods for processing a still single camera image. First, a background image that does not contain a moving object without any falling objects is prepared in advance, and the difference between this background image and the current image is calculated. Judgment (background subtraction method, reference: image processing industry application list, second volume). Thus, an object that is not in the background is extracted regardless of whether the object is moving.

[0005] Both methods detect both moving and stationary objects, but it is possible to distinguish between a moving object and a stationary object by repeating the method every unit time. As for the method based on image processing described later, many studies on the detection of moving objects have been made, and there is also research on stably detecting moving objects in response to environmental changes by appropriately updating the background image.

[0006]

However, such a conventional method has the following problems. In the method of measuring the distance that is often used for detecting obstacles of moving objects, it is necessary to determine whether an object exists based on the difference in height from the road surface. Is difficult to detect. Also, in the method using image processing, the background image also changes due to changes in brightness and appearance due to weather and time, and it is difficult to extract only obstacles with simple background subtraction.

In order to cope with this, when a moving object is detected, a method of appropriately updating the background is adopted. In this method, a stationary obstacle which does not exist in the first background image is used as a background image at the time of updating. Since the obstacle is captured, a stationary obstacle cannot be detected. As means for solving these problems, we have proposed a "stationary obstacle detection method" (Japanese Patent Application No. 2000-206289, hereinafter,
"Last proposal").

According to this method, in an environment including a moving object on a road or a passage, an image sampled from a moving image including a moving object of a stationary single camera is divided into small regions, and each small region is prepared in advance. By calculating the normalized correlation with the corresponding area of the background image that does not include the moving object and does not have any obstacles, by repeating the process for images sampled at regular time intervals, such as a falling object The present invention relates to a method and an apparatus for detecting a stationary obstacle that does not exist in a normal state and hinders traffic, and is capable of detecting even an object having a small thickness, which is difficult with the conventional method, which is resistant to brightness fluctuation. There is an advantage that an obstacle can be detected even when the background slowly changes due to weather or the progress of an incident.

However, the previously proposed stationary obstacle detection method has the following problems. Since no consideration is given to the distribution of small areas where obstacles have been detected on the image, the size of the obstacles on the image cannot be detected. In such a case, the detection is reported a plurality of times. The use of normalized correlation makes it difficult to detect brightness changes erroneously.However, in small areas without very clear patterns, shadows of moving objects may be erroneously captured when updating the background. May adversely affect the background update and lead to erroneous detection.

[0010] When the camera vibrates, a slight displacement of the image occurs, so that the correlation value with the background image changes, so that it is often erroneously determined that the moving object is passing, and the background is updated. Does not proceed quickly, a slow change in the way the background looks due to a change in time or weather may be erroneously detected as an obstacle. Therefore, when the camera vibrates due to restrictions on the installation location and fixing method of the camera, the probability of erroneous detection increases.

[0011]

According to a first aspect of the present invention, there is provided a method and apparatus for detecting a stationary obstacle on a road by processing an image of a single still camera.
In a method and an apparatus for detecting a stationary obstacle that does not exist in a normal state such as a falling object and hinders traffic in an environment including a moving object on a road or a passage, an image sampled from a moving image is reduced. For each small area, calculate the normalized correlation value with the corresponding area of the background image that does not include a moving object prepared in advance and does not have any obstacles for each small area, and calculate the average brightness value within the small area at regular intervals. Repeatedly performed on the image to be sampled, the amount of change in the correlation value and the amount of change in the average luminance value are determined by a threshold,
It is characterized in that a stationary obstacle on the road is detected while the background image is appropriately updated based on these determinations / evaluations.

According to a second aspect of the present invention, there is provided a method and apparatus for detecting a stationary obstacle on a road, which processes an image captured by a single still camera, thereby detecting an obstacle including a moving object on a road or a passage. In a method and apparatus for detecting a stationary obstacle that does not exist in a normal state such as a falling object and hinders traffic, an image sampled from a moving image is cut into small regions, and each small region is Repeatedly calculate the normalized correlation value and the average brightness value in the small area with the corresponding area of the background image that does not include the moving object prepared in advance and does not have any obstacles to the image sampled at regular time intervals, The amount of change in the correlation value and the amount of change in the average luminance value are determined using a threshold, and the connection between the obstacle candidate regions on the image is also evaluated. Based on these determinations and evaluations, While updating the appropriate background image Te, and detects a stationary obstacle on the street.

According to a third aspect of the present invention, there is provided a method and apparatus for detecting a stationary obstacle on a road, which processes an image of a single still camera, thereby detecting an obstacle including a moving object on a road or a passage. In a method and apparatus for detecting a stationary obstacle that does not exist in a normal state such as a falling object and hinders traffic, a background image that does not include a moving object prepared in advance and does not include an obstacle is reduced to a small area. For each small area, for each small area, calculate the normalized correlation value for the area corresponding to the corresponding position of the image sampled from the moving image and for each area slightly shifted up, down, left and right from the corresponding position, and calculate the maximum The value is taken as the correlation value at that point in time in the small area and the average value in the small area is calculated at the position of the maximum value. Repeatedly performed for the image, to determine a change amount of the change amount and the average luminance value of the correlation value with the threshold value,
The method is also characterized in that the connection between the obstacle candidate areas on the image is evaluated, and the stationary obstacle on the road is detected while the background image is appropriately updated based on the determination and evaluation.

According to a fourth aspect of the present invention, there is provided a method and apparatus for detecting a stationary obstacle on a road, which processes an image captured by a single still camera to be used in an environment including a moving object on a road or a passage. In a method and apparatus for detecting a stationary obstacle that does not exist in a normal state such as a falling object and hinders traffic, a background image that does not include a moving object prepared in advance and does not include an obstacle is reduced to a small area. Select several small areas in advance that have a clear pattern from segmentation, brightness dispersion in the area, edges, etc., and are sensitive to positional deviation, and for an image sampled from a moving image, it corresponds to the corresponding position of the small area selected first. Area and
First, calculate the normalized correlation value for each area slightly shifted up, down, left, and right from the corresponding position, and take the weighted average for several selected points of the shift amount that gives the maximum value. The amount of positional deviation of the sampled image is obtained, and for all the small regions of the background image, the small region at the position shifted from the corresponding position by the amount of positional deviation obtained here is extracted from the sampled image, and the normalized correlation value and The calculation of the average luminance value in the small area is repeatedly performed on the image sampled at regular intervals, and the amount of change in the correlation value and the amount of change in the average luminance value are determined using a threshold. It also evaluates the connection of the obstacle candidate areas at, and based on these judgments and evaluations, updates the background image as appropriate and detects stationary obstacles on the road. It is characterized in.

According to a fifth aspect of the present invention, there is provided a method and apparatus for detecting a stationary obstacle on a road according to the present invention, wherein the average luminance value is replaced by a denominator of the correlation value. The amount of change in the root mean square of the brightness in the small area, which is a part, is determined by a threshold value. A method and apparatus for detecting a stationary obstacle on a road according to claim 6 of the present invention that solves the above-described problem also evaluates the connection between the obstacle candidate areas on the image in the method and apparatus according to claim 2 or 3. It is characterized by the following. A method and apparatus for detecting a stationary obstacle on a road according to claim 17 of the present invention that solves the above-mentioned problems is a method and apparatus according to claim 2 or 3, wherein a part of the denominator of the correlation value is used instead of the average luminance value. It is characterized in that the amount of change in the root mean square of the luminance in a certain small area is determined by a threshold value.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 Basic Concept In this system, an image sampled from a moving image including a moving object is divided into small regions, and as shown in FIG. The normalization correlation with the corresponding area of the background image that does not include the moving object prepared in advance and does not have any obstacles and the calculation of the average luminance value of all the pixels in the small area are performed on the image sampled at regular time intervals. repeat. The small region has a height L and a width K in FIG. The normalized correlation is
For example, it is obtained by equation (1).

[0017]

(Equation 1)

FIG. 2 shows an example in which the change over time of the correlation value and the average luminance value of a certain small area is graphed. If the amount of change in the correlation value or the luminance average value is large for a small area as in section A in FIG. 2, it is determined that the moving object is passing, and the correlation value is as in sections B and C in FIG. If the amount of change in both the luminance and the average luminance value is continuous for a certain number of times or more, it can be determined that the background is reflected in the small area. If the correlation value of the small area regarded as the background in this way is a value close to 1 as in the section B of FIG. 2, the image is the same as the corresponding small area of the previously prepared background image. Therefore, it is set as a background update candidate area.

When the correlation value is smaller than a certain threshold value as in the section C in FIG. 2, this small area is determined as a stationary obstacle candidate area in which a stationary object not shown in the previously prepared background image exists. After the above processing has been performed for all the small areas in the image, areas other than the small areas where the moving object is determined to be passing are smaller than a certain threshold (see FIG. 3).
In the flowchart of the above, it is expressed as an object region, and is hereinafter referred to as an object region. By performing labeling on ()), the approximate size of the obstacle on the image can be determined.

Therefore, since the same label can be determined for the stationary obstacle candidate area if the label is the same, the detection of the same object needs to be reported only once. The labeling here means that the connection between the obstacle candidate areas on the image is also evaluated. Generally, in a binary image,
When the area of the pixel value 1 is scattered in the background of the pixel value 0, it refers to the operation of numbering this area. When the area is represented as a set of pixels of the pixel value 1, the area 4
When the connection of only pixels is considered (referred to as four connected components),
When considering a total of 8 pixels including 4 pixels adjacent in the oblique direction (there is an 8 connected component (Reference: Revised Image Engineering 174)
page). Next, the background is updated only in a background update candidate area that is not adjacent to the object area so that the shadow of the object is not erroneously captured as the background.

FIGS. 3 and 4 show flowcharts of the method for detecting a stationary obstacle on the road according to this embodiment. First, a current image sampled from a moving image including a moving object is fetched (step S1). Thereafter, a small area for which correlation calculation has not been performed is selected (step S2), and the normalized correlation of the small area with the background image is selected. Is calculated according to equation (1) (step S3), and the average luminance of the pixels in the small area is calculated (step S4).

Next, the temporal change amount of the calculated correlation value or average luminance value is small (step S5), the correlation value is close to 1 (steps S6, S9), and the small change amount is N
If (N is a natural number) times (step S8), the image is the same as the corresponding small area of the background image prepared in advance, so that it is determined to be a background update candidate area (step S10). If the calculated temporal change amount of the correlation value or the average luminance value is small (step S5), but the correlation value is not close to 1 (step S6), it is determined that the area is an object area (step S7).

The temporal change amount of the calculated correlation value or average luminance value is small (step S5), the correlation value is not close to 1 (steps S6, S9), and the small change amount is N (N
(A natural number) times (step S8), the small area is determined to be a stationary obstacle candidate area including a stationary object that is not shown in the previously prepared background image (step S8).
11). If the temporal change amount of the calculated correlation value or average luminance value is not small (step S5), or if the small change amount does not continue N (N is a natural number) times (step S5).
8) A stationary obstacle is not detected in the small area.

Thereafter, if the above-mentioned judgment has not been completed for all the regions (step S12), the process returns to the selection of the small region for which the correlation calculation has not been completed (step S2), and the above processing is performed for all the small regions. (Step S1
2) Perform labeling of the object area (step S1)
3). Then, a stationary obstacle candidate area is selected (step S15), and if the label is not a detected report label (step S1).
4) A detection report is made (step S17), and the detected report label is stored (step S19). If the detected report label is a label (step S16), the stationary obstacle is kept until all the candidate areas are selected. The selection of the candidate area is repeated (step S15).

Further, a background update candidate area is selected (step S19), and if it is not adjacent to the object area (step S2).
0), the background is updated (step S21), and if adjacent to the object area (step S20), the selection of the background update candidate area is repeated until all the candidate areas have been selected (step S22).

[Embodiment 2] Response to camera vibration In this embodiment, instead of performing the process of calculating the normalized correlation value for the corresponding small area of the current image and the background image in the embodiment 1, FIG. As shown, a normalized correlation value is calculated at a position slightly shifted up, down, left, and right corresponding to the position of the corresponding small area of the current image, and the maximum value is used as a correlation value used in subsequent determinations. The average luminance is calculated at the position of the maximum value. In FIG. 4, assuming that the vibration direction is only the left-right direction, only the shift in the left-right direction is considered, and the normalized correlation value is calculated for the small region shifted to the right or left to obtain the maximum value.

[Third Embodiment] Response to Camera Vibration In the third embodiment, instead of performing the process of calculating the normalized correlation value for the corresponding small area of the current image and the background image in the first embodiment, the following is performed. Change to That is, for the background image, the luminance variance and edges of the pixels in the small area are calculated in advance, and a plurality of small areas having clear patterns and sensitive to vibration are selected.
Similarly, the normalized correlation value is calculated at the position of the corresponding small area and the position slightly shifted up, down, left and right, and the amount of shift that gives the maximum value is obtained for each small area.

The amount of this shift is weighted according to the normalized maximum correlation value obtained earlier, and an average is taken to obtain the position shift of the current image with respect to the background image. By taking the weighted average, the influence of the area where the object is passing can be reduced as shown in FIG. The normalized correlation value and the average luminance value with respect to the background image are calculated for all the small areas by shifting the calculated image position shift.

[Embodiment 4] Calculation method of luminance change amount In the method of the first embodiment, the average change amount of the luminance values of the pixels in the small area is used. Since the root-mean-square of the luminance value of the pixel (the denominator of the equation (1)) is obtained, a stationary object is detected in the same flow as in the first embodiment using the amount of change in this value as the amount of change in luminance.

[0030]

As described above, according to the present invention, the following effects can be obtained as described in detail with reference to the embodiments. 1) A stationary road obstacle in a camera image can be detected. 2) By using the amount of change in the average luminance value in addition to the amount of change in the correlation value with the background image, it is possible to prevent errors in background updating that tend to occur in small areas without clear patterns in the previously proposed method. Therefore, the probability of erroneous detection can be reduced. 3) For a stationary obstacle candidate area extracted from a time-series change in the amount of change in the average luminance value in addition to the amount of change in the correlation value with the background image, the two-dimensional connection on the image is evaluated for the stationary obstacle. The approximate size of an object can be detected, and detection of a stationary obstacle having a size over a plurality of regions can be prevented from being reported more than once. 4) The probability of erroneous detection when the camera vibrates can be reduced as compared with the second embodiment. 5) According to the third embodiment, the probability of erroneous detection can be reduced when the camera vibrates with a smaller amount of calculation than in the second embodiment. 6) According to the fourth embodiment, since the luminance value can be calculated simultaneously with the calculation of the correlation with the background image, the amount of calculation can be reduced as compared with the first embodiment.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing a current image and a background image divided into small areas.

FIG. 2 is a graph showing a time transition of a normalized correlation value and an average luminance.

FIG. 3 is a flowchart for detecting a stationary obstacle.

FIG. 4 is a flowchart for detecting a stationary obstacle.

FIG. 5 is an explanatory diagram showing a corresponding small area position of a current image and a position slightly shifted up, down, left, and right.

FIG. 6 is an explanatory diagram showing a method of selecting a plurality of small regions having a clear pattern and being sensitive to vibration, and obtaining a position shift that gives the maximum value of the normalized correlation in these regions.

[Explanation of symbols]

 K Small area height L Small area height

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA07 AA51 BB05 CC40 DD06 DD14 FF01 FF04 FF44 FF61 JJ03 JJ09 JJ26 MM07 QQ13 QQ24 QQ25 QQ36 QQ39 QQ41 RR08 SS09 SS13 UU05 5B057 AA12 CB02 CA02 CA02 CA02 CA02 DB05 DB09 DC32 5C054 FC01 FC05 FC12 FC13 FC16 GB01 GB12 HA30 5H180 AA01 CC04

Claims (7)

[Claims] An image of a stationary single camera is processed so that a stationary object which does not exist in a normal state such as a falling object in an environment including a moving object on a road or a passage, and impedes traffic. In the method and apparatus for detecting an obstacle, an image sampled from a moving image is divided into small regions, and each small region is compared with a corresponding region of a background image that does not include a moving object prepared in advance and does not have an obstacle. The normalized correlation value and the calculation of the average luminance value in the small area are repeatedly performed on the image sampled at regular intervals, and the amount of change in the correlation value and the amount of change in the average luminance value are determined by a threshold. A method and apparatus for detecting a stationary obstacle on a road while appropriately updating a background image based on determination / evaluation. 2. Processing an image of a stationary single camera so that the stationary or non-existent stationary object which does not exist in a normal state such as a falling object in a road or an environment including a moving object and impedes traffic in an environment including a moving object. In the method and apparatus for detecting an obstacle, an image sampled from a moving image is divided into small regions, and each small region is compared with a corresponding region of a background image that does not include a moving object prepared in advance and does not have an obstacle. The calculation of the normalized correlation value and the average luminance value in the small area is repeatedly performed on the image sampled at regular time intervals, and the amount of change in the correlation value and the amount of change in the average luminance value are determined by a threshold. It also evaluates the connection of the obstacle candidate areas on the above and detects stationary obstacles on the road while appropriately updating the background image based on these judgments and evaluations. Method and apparatus. 3. Processing an image of a stationary single camera so that the stationary or non-existent condition such as a falling object does not exist in a normal condition such as a fallen object in an environment including a moving object on a road or a passage, so that traffic is prevented. In the method and apparatus for detecting an obstacle, a background image that does not include a moving object prepared in advance and does not have an obstacle is cut into small regions, and each small region corresponds to a corresponding position of an image sampled from a moving image. A normalized correlation value is calculated for each of the region and each region slightly shifted up, down, left, and right from the corresponding position, and the maximum value is taken as the correlation value at that time of the small region, and the small region is calculated at the position of the maximum value. The calculation of the average luminance value is repeatedly performed on the images sampled at regular intervals, and the amount of change in the correlation value and the amount of change in the average luminance value are determined as threshold values. In addition, the connection of the obstacle candidate areas on the image is also evaluated, and the stationary obstacle on the road is detected while appropriately updating the background image based on these determinations / evaluations. Method and apparatus. 4. Processing an image of a stationary single camera so that the stationary or non-existent stationary object, which does not exist in a normal state such as a falling object in a road or a passage including an obstacle, impedes traffic. In the method and apparatus for detecting an obstacle, a background image that does not include a moving object prepared in advance and does not include an obstacle is divided into small areas, and a small pattern that has a clear pattern based on luminance dispersion and edges in the area and is sensitive to displacement is provided. Select several areas in advance, and for the image sampled from the moving image, first, for the area corresponding to the corresponding position of the previously selected small area, and for the area slightly shifted up, down, left and right from the corresponding position, normalized correlation values First, the background image and the sub-image are calculated by taking the weighted average of several points where the amount of deviation giving the maximum value is selected. The amount of positional deviation of the sampled image is obtained, and for all the small regions of the background image, a small region at a position shifted from the corresponding position by the amount of positional deviation obtained here is extracted from the sampled image, and the normalized correlation value and The calculation of the average luminance value in the small area is repeatedly performed on the image sampled at regular intervals, and the amount of change in the correlation value and the amount of change in the average luminance value are determined using a threshold. And a method for detecting a stationary obstacle on a road while appropriately updating a background image based on these determinations / evaluations. 5. The method according to claim 1, wherein, instead of the average luminance value, a change in the root mean square of the luminance in the small area, which is a part of the denominator of the correlation value, is determined by a threshold value. And equipment. 6. The method and apparatus according to claim 2, wherein the connection between the obstacle candidate areas on the image is also evaluated. 7. The method according to claim 2, wherein, instead of the average luminance value, a change in the root mean square of the luminance in the small area, which is a part of the denominator of the correlation value, is determined by a threshold value. Method and apparatus.