JP2000057323A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor which can extract only an object area without being affected by the pixel value of a background area. SOLUTION: This image processor 30 includes an image memory 305 for temporarily storing image data generated from the output of a camera 20 and also storing the image data processed by an image processing board 307 and this board 307 for performing fault detection processing, based on the image data stored in the memory 305 and the output of a laser radar 25. Then the board 307 produces differential image data from the input image data and the background image data and binarizes the differential image data. The respective areas included in the binarized differential image data are discriminated into an obstacle area and the other areas according to distances observed by a laser radar 25. Further binarization threshold is decided by the relation between the processing capability of the laser radar 25 and the area to be detected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus capable of extracting an object area without being affected by pixel values of a background area.

[0002]

2. Description of the Related Art In order to extract only an object region from input image data captured by a CCD (charge coupled device) camera or the like, there is a method of binarizing input image data with an appropriate threshold value. There is also widely known a method in which difference image data is created from input image data and background image data in which an object region is not shown, and the difference image data is binarized to extract only the object region. In any of these methods, the binarization threshold plays an important role.

In the image processing apparatus disclosed in Japanese Patent Publication No. Hei 6-79332, a histogram is created for each of input image data and background image data, and the two histograms are compared. Find the threshold. Using this binarization threshold, the input image data is binarized to extract the object region.

In the moving object image extraction method disclosed in Japanese Patent Application Laid-Open No. 7-320035, difference image data of two background image data at different times is obtained, and a predetermined value α is added to the maximum value of the difference image data. The determined value is determined as a binarization threshold. Using this binarization threshold, the object area is extracted by binarizing the difference image data between the input image data and the background image data.

[0005]

In the former image processing apparatus, it is assumed that the pixel value of the object area is sufficiently smaller (or larger) than the pixel value of the background area. However, in a process that assumes a real environment such as a process for extracting a falling object on a road, the distribution of the pixel value of the background region and the pixel value of the object region are not always uniform, and the pixel value of the object region is more than the pixel value of the background region. It is rare that the value is always sufficiently small (or large). Therefore, there is a problem that the application range of this image processing apparatus is narrow. Further, the image processing apparatus sets a pixel value that can separate the background area and the other area as the binarization threshold. For this reason, the background region is less often detected as an object region, but there is a problem that an object region having brightness close to the background region cannot be extracted.

[0006] The binarization threshold value obtained by the latter moving object image extraction method is determined so as not to be affected by dark current or noise of the CCD camera. On the other hand, in an object region having a brightness close to the background region, the pixel value of the difference image data indicates a small value. Therefore, there is a problem that an object region having a brightness close to the background region cannot be extracted.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to provide an image processing apparatus capable of extracting an object area without being affected by the pixel value of a background area. To provide.

Another object of the present invention is to provide an image processing apparatus capable of extracting only an object area without being affected by pixel values of a background area.

[0009]

According to the first aspect of the present invention, there is provided an image processing apparatus comprising: means for creating a first difference image based on an input image and a background image; Means for binarizing with a first threshold value to create a first binary difference image, object region extracting means for extracting an object region from the first binary difference image, And a first threshold updating means for updating the threshold value of the input image for an area other than the object area obtained from the input image one frame before. Means for creating a second difference image based on the background image and a second difference image at a second threshold value.
Means for generating a second binary difference image by converting to a value, means for counting the number of regions in the second binary difference image, and a second process in accordance with the number of regions. Second threshold value updating means for updating the threshold value, and first threshold value setting means for setting the first threshold value to a number determined according to the second threshold value. including.

[0010] According to the first aspect of the invention, the second 2
The second threshold is updated according to the number of regions in the value difference image, and the first threshold is determined. For this reason, the number of regions (noise regions) other than the detected object region can be made substantially constant. The first threshold value thus determined is determined so that an object region having a pixel value similar to that of the background region can be extracted while the detected noise region is kept to a certain number or less. Therefore, it is possible to provide an image processing apparatus capable of extracting an object region without being affected by the pixel value of the background region.

According to a second aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the object region extracting means comprises a first
Object candidate area extracting means for extracting an object candidate area from the value difference image, distance measuring means for measuring a distance to an object corresponding to the object candidate area, and an object candidate area according to an output of the distance measuring means. And a discriminating means for discriminating between the object region and the region other than the object region.

According to the second aspect of the invention, in addition to the functions and effects of the first aspect, the object candidate area is narrowed down based on the measured distance. Therefore, it is possible to provide an image processing apparatus capable of extracting only the object region without being affected by the pixel value of the background region.

According to a third aspect of the present invention, in addition to the configuration of the second aspect, the object candidate area extracting means extracts the object area obtained by the discriminating means from the input image one frame before. Means for extracting an object candidate area from the first binary difference image for an area excluding the area other than.

According to a third aspect of the present invention, in addition to the functions and effects of the second aspect of the present invention, an area other than the object area (noise area) is discriminated from the input image one frame before the input image. Then, an object candidate area is extracted from the set area. For this reason,
The number of noise regions in the object candidate region can be reduced. Thereby, erroneous detection of the object region can be reduced, and the object region can be detected efficiently.

According to a fourth aspect of the present invention, in addition to the configuration according to any one of the first to third aspects, the second threshold value updating means includes an area in the second binary difference image. If the number is less than or equal to a predetermined upper limit number, the first threshold setting means includes means for decrementing the second threshold value by the first predetermined number. Means for setting a value obtained by subtracting the second predetermined number from the second threshold value as the first threshold value if the number of regions is larger than the predetermined upper limit number.

[0016]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a road monitoring apparatus according to an embodiment of the present invention. In the following description, the same parts are denoted by the same reference numerals. Since their names and functions are the same, repetition of the description will be appropriately omitted.

Referring to FIG. 1, a road monitoring device 70 includes a pole 60 provided near a road, a camera 20 provided above the pole 60 and a laser radar 25 for distance measurement, It includes an image processing device 30 connected to the laser radar 25 by a cable and provided below the pole 60. The image processing device 30 is connected to the line 5
0 and is connected to the control center 40 to detect whether an obstacle is falling on the road surface.
Report to The control center 40 is a central control center that manages / controls traffic conditions on roads.

The image data between the camera 20 and the image processing device 30 is transmitted by NTSC (National Television System
It is transmitted and received in a form that conforms to standards such as the mittee method. As a result, when transmitting and receiving image data, the conventional ordinary interface can be used as it is. For example, analysis of a road image using a personal computer or the like, recording of a monitor or image on a video device by a TV (Television), and the like can be easily performed.

Referring to FIG. 2, image processing apparatus 30 includes an analog / digital converter 303 for converting an analog signal output from camera 20 into a digital signal and an image data output from analog / digital converter 303 once. An image memory 3 for storing and storing image data as a result of processing by an image processing board 307 described later.
05, and an image processing board 307 for performing an obstacle detection process based on the image data stored in the image memory 305 and the output of the laser radar 25.

Referring to FIG. 3, laser radar 25 includes a light emitting and receiving unit 202 for emitting and receiving laser light, and a light emitting and receiving unit 202.
The first light for reflecting the light emitted therefrom downward in the drawing.
And a second reflecting mirror 206 for reflecting the light reflected by the first reflecting mirror 204 rightward in the drawing.
And a control unit (not shown) for controlling the rotation of the first and second reflecting mirrors 204 and 206 to guide the light emitted from the light emitting and receiving unit 202 in a desired direction. The first reflecting mirror 204 is a plane mirror, and
It rotates around the axis 208 in the same direction as the light emitted from the light emitting element 02 (the direction of the arrow in the figure). The second reflecting mirror 206 is a plane mirror and rotates around an axis perpendicular to the plane of the paper (the direction of the arrow in the figure). By the rotation of the first and second reflecting mirrors 204 and 206, the horizontal direction and the vertical direction of the light emitted from the laser radar 25 are determined, respectively. Laser radar 2
5 calculates the distance from the time from light emission to light reception.

Referring to FIG. 4, each part of image processing apparatus 30 operates as follows. The image processing device 30 captures, as pre-processing, a background on which no obstacle is reflected, as shown in FIG.
The background image data as shown in FIG.
(S1). FIG. 5 taken from the camera 20
The input image data as shown in FIG.
5, and pre-processing such as smoothing is performed on the input image data (S2).

The difference processing is performed between the preprocessed input image data and the background image data, and the difference image data as shown in FIG. 5C is stored in the image memory 305 (S3). This difference processing is processing for obtaining an absolute value of a difference between pixel values between corresponding pixels of two images, and is represented by the following equation (1). At this time, when the pixel value of the corresponding pixel of the noise area mask image data described later is 1, the pixel value after the difference processing is set to 0. This makes it possible to exclude a region determined as a noise region in the input image data one frame before.

[0023]

(Equation 1)

The difference image data is binarized using a predetermined binarization threshold, and the image data as shown in FIG. 5D is obtained. Labeling of the binarized difference image data is performed, and a candidate area for an obstacle is detected (S
4). The labeling of the difference image data is performed by a method of combining pixels in the vicinity of eight concatenations. For example, referring to FIG. 6, since area 82 and area 84 are connected in the vicinity of eight joints, the same label is assigned.

For each obstacle candidate obtained in the process of S4, the distance to the object or background corresponding to each pixel constituting the candidate area is measured using the laser radar 25 (S5). It is assumed that the calibration of the camera 20 and the laser radar 25 has been performed in advance.

The shape of the area is recognized from the measurement result,
It is determined whether it is an obstacle (S6). For example, if the shape of the area has unevenness, it is determined to be an obstacle area, and if not, it is determined to be a background area other than an obstacle such as a road surface. Alternatively, the distance image of the background may be stored in the image memory 305 in advance, and it may be determined whether or not the obstacle is an obstacle by comparing with the distance image. If it is determined that the area is an obstacle area (YES in S6), the image processing board 307 notifies the control center 4 of the detection of the obstacle and the position of the obstacle via the line 50.
Report to 0. In addition, for the binarization threshold value optimization process (S9) to be described later, the pixel value of a pixel constituting the obstacle region is set to 1, and the pixel values of the other pixels are set to 0.
Obstacle area mask image data as shown in (E) is created (S7).

If it is determined that the area is not an obstacle (NO in S6), the pixel value of the noise area mask image data (FIG. 5F) corresponding to the area is set to 1 (S8). ). The processing from S6 to S8 is performed for all obstacle candidates detected in S4. It is assumed that a pixel value other than 1 such as a pixel value 0 is set for pixels other than noise in the noise area mask image data.

Using the input image data, the background image data and the obstacle area mask image data, a binarization threshold value used in the obstacle candidate detection process (S4) for the input image data of the next frame is calculated (S4). S9). This processing will be described later.

Thereafter, background image update processing is performed (S1).
0), the process is returned to S2. The updating of the background image is performed by a method called an exponential smoothing method. The pixel values of the coordinates (x, y) of the input image data, the background image data, and the obstacle area mask image data at time t are represented by I (t,
x, y), B (t, x, y) and OM (t, x, y), the pixel value of the coordinates (x, y) of the updated background image data is expressed by the following equation (2). It is.

[0030]

(Equation 2)

Note that the constant α is a value of a value from 0 to 1 experimentally obtained. As the value of the constant α increases, the updated background image data is more affected by the currently input image data. Therefore, it is possible to follow a sharp change in the input image data. on the other hand,
As the value of the constant α decreases, the updated background image data is less affected by the current input image data. For this reason, the degree of change of the background image data can be reduced. In a pixel where OM (t, x, y) = 1,
In order to update the background image except for the obstacle area, the pixel value of the background image one frame before is used as it is.

The binarization threshold value optimization processing (S9 in FIG. 4) will be described with reference to FIG. The difference image data between the input image data and the background image data of the t-th frame is created according to the following equation (3) (S21). The reason why the difference value of the pixel masked by the obstacle region mask image data is set to 0 when the difference image data is created is that the pixel is not desired to be detected as a region in the processing described later.

[0033]

(Equation 3)

The image processing device 30 sets a predetermined upper threshold value as the threshold value (S2).
2). The difference image data is binarized with the upper threshold, and image data as shown in FIG. 8 is obtained (S23). By the binarization, the pixel value of a pixel having a difference value equal to or larger than the threshold value is set to 1, and the pixel values of other pixels are set to 0. The binarized difference image data is subjected to erosion processing twice (S24), and thereafter is subjected to expansion processing twice (S25). As a result, noise components having a small area are removed.

Referring to FIG. 9, the contraction processing and expansion processing will be described. The contraction processing is a process of 8 of a pixel having a pixel value of 0.
This is a process of replacing the pixel value of a pixel near the concatenation with 0. For example, when the image data shown in FIG. 9A is subjected to one contraction process, the image data shown in FIG. 9B is obtained. The expansion process is a process of replacing a pixel value of a pixel in the vicinity of eight consecutive pixels of a pixel with a pixel value of 1 with 1. For example, FIG.
By performing one expansion process on the image data shown in (B), the image data shown in FIG. 9A is obtained.

The image data shown in FIG.
Consider a case where one expansion process is performed after performing one contraction process. For the sake of simplicity, the number of times of the contraction process and the number of the expansion processes are each one, but the same effect can be obtained with two times. The image data shown in FIG.
An area 86 corresponding to a small area noise component and an area 88 corresponding to an obstacle are included. When the image data shown in FIG. 10A is subjected to one contraction process, the area 86 is removed, and the image data shown in FIG. 10B is obtained. Further, when the image data shown in FIG. 10B is subjected to one expansion process, finally, the image data shown in FIG. 10C including only the area 88 corresponding to the obstacle is obtained.

After the processing of S25, the dilation processing is performed twice on the binarized difference image data (S26), and the erosion processing is performed twice (S27). As a result, it is possible to integrate a plurality of divided regions in the vicinity.

For example, consider a case where the image data shown in FIG. 11A is subjected to one expansion processing and then to one contraction processing. The image data shown in FIG. 11A includes two regions 90 and 92 located near each other. Referring to FIG. 11B, after one expansion process, 2
An area 94 in which the two areas 90 and 92 are integrated is obtained. Referring to FIG. 11 (C), when one more contraction process is performed, two regions 90 and 92 are finally integrated, and image data after one region 94 is obtained.

After the processing in S27, the number of areas having a pixel value of 1 in the binarized difference image data is counted (S2).
8). Similar to the process of S8 in FIG. 4 described above, a region connected near eight concatenations is defined as one region. As the number of these areas increases, the measurement processing by the laser radar 25 (S in FIG. 4)
5)).

It is determined whether or not the number of detection areas is larger than a predetermined detection upper limit number (S29). The value of the upper detection limit is determined in relation to the processing time of the laser radar 25. That is, the detection upper limit number is determined so that the laser radar 25 can measure the distances to all the detection regions in one measurement process (the process of S5 in FIG. 4).

If the number of detection areas is equal to or smaller than the detection upper limit number (NO in S29), the laser radar 25 can be used as a rangefinder for more areas. For this reason, the image processing device 30 decrements the threshold by one (S31),
The processing of 23 and below is repeated.

If the number of detection areas is larger than the upper limit of detection (YES in S29), the threshold value is incremented by one (S30), and the value is set as a binarization threshold value in the next processing of S4. . Using this binarized threshold, S in FIG.
The number of obstacle candidates detected in the process 4 is equal to or less than the maximum number that can be measured by the laser radar 25.

In the above-described image processing apparatus 30, the binarization threshold is set on the assumption that a noise region accompanying the difference processing is detected. For this reason, the binarization threshold is set lower. When the binarization of the differential image data is performed using the binarization threshold value, it is possible to detect an obstacle region in which the pixel value is similar to the background. However, a noise region is also detected at the same time. This noise region is removed by performing distance measurement using the laser radar 25. Therefore, the image processing apparatus 30 can extract only the obstacle region without being affected by the pixel value of the background region.

The road monitoring device 7 according to the present embodiment
In the case of 0, the laser radar 25 is used for distance measurement. However, the present invention is not limited to this, and any device capable of measuring distance, such as a millimeter wave radar, may be used.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[Brief description of the drawings]

FIG. 1 is an external view of a road monitoring device according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a road monitoring device.

FIG. 3 is a cross-sectional view illustrating a configuration of a laser radar.

FIG. 4 is a flowchart of a process performed by the road monitoring device.

FIG. 5 is a diagram illustrating an example of various image data.

FIG. 6 is a diagram illustrating a region connected near eight joints.

FIG. 7 is a flowchart of a binarization threshold value optimization process performed by the road monitoring device.

FIG. 8 is a diagram illustrating an example of binary difference image data created by a binarization threshold value optimization process.

FIG. 9 is a diagram illustrating expansion processing and contraction processing.

FIG. 10 is a diagram illustrating a process of removing a noise component by a contraction process and an expansion process.

FIG. 11 is a diagram illustrating a process of integrating regions divided by an expansion process and a contraction process.

[Explanation of symbols]

 Reference Signs List 20 camera 25 laser radar 30 image processing device 40 control center 50 line 60 pole 70 road monitoring device 303 analog / digital converter 305 image memory 307 image processing board

Claims (4)

[Claims] 1. A method according to claim 1, further comprising the steps of:
Means for creating a first differential image; means for creating a first binary difference image by binarizing the first differential image with a first threshold value; An object area extracting unit for extracting an object area from the value difference image; and a first threshold updating unit for updating the first threshold, the first threshold updating unit Means for creating a second difference image based on the input image and the background image for an area other than the object area obtained from the input image one frame before; Means for binarizing with a threshold value of 2 to create a second binary difference image; means for counting the number of regions in the second binary difference image; A second threshold value updating means for updating the second threshold value according to the number; When, and a first threshold setting means for setting the number determined according to the second threshold to the first threshold value, the image processing apparatus. 2. The object region extracting unit, comprising: an object candidate region extracting unit for extracting an object candidate region from the first binary difference image; and measuring a distance to an object corresponding to the object candidate region. The method according to claim 1, further comprising: a distance measuring unit for discriminating the object candidate region into the object region and a region other than the object region according to an output of the distance measuring unit. Image processing device. 3. An object candidate area extracting unit, wherein the first binary difference is set for an area excluding an area other than the object area obtained by the discriminating unit from the input image one frame before. The image processing apparatus according to claim 2, further comprising: a unit configured to extract the object candidate region from an image. 4. The second threshold value updating means, if the number of the areas in the second binary difference image is equal to or less than a predetermined upper limit number, sets the second threshold value to a first upper limit number. Means for decrementing by a predetermined number, wherein the first threshold value setting means sets the second threshold value if the number of the regions in the second binary difference image is larger than the predetermined upper limit number. The image processing apparatus according to claim 1, further comprising: a unit configured to set a value obtained by subtracting a second predetermined number from a threshold value as the first threshold value.