JPH07113973B2 - Image processing device, moving object detection device, and image processing method - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a background image updating method in image processing, and an image processing apparatus such as a moving object detection apparatus or a monitoring apparatus.

[Prior Art] With the recent development of automobile traffic and the like, traffic congestion on expressways has become a problem.

For the purpose of eliminating such a situation as much as possible, a traffic flow measurement technique is being developed in which the flow of a vehicle is measured using a television camera and information is given to the user.

In such conventional mobile body inspection technology such as traffic flow measurement technology, the background image is updated so as to follow the change in brightness, and the vehicle or the like is extracted from the difference image between the background image and the input image. , Measuring traffic flow. For this kind of technology, see the IEICE Technical Committee Report IE76-84, 19
February 1977, PP.67-78).

The technique of the document is characterized in that the reference brightness is obtained by the following formula and the background is updated based on the reference brightness.

| I (t−1) −I (t) | α, then (t) = (t−1) · (1−ε) + I (t) · ε (1) | I (t−1) −I If (t) |> α, then (t) = (t−1) (2) where: Reference luminance value based on exponential average α: Change width of estimated reference luminance value ε: Coefficient of exponential average (1) Formula If the difference between the reference brightness at time t−1 and the brightness value I (t) at time t is within a predetermined value α, it is considered that the brightness of the road portion changes due to changes in weather conditions, etc. It is shown that the reference density value is calculated by the equation (2), and when the difference between the two exceeds α, I (t) is considered to be the change in brightness accompanying the passage of the vehicle, and therefore the reference brightness value is not changed. Indicates that you want to leave.

That is, it is intended to eliminate the influence of the change in the brightness due to the passage of the vehicle in the background update.

[Problems to be Solved by the Invention] In the above-described conventional technology, as can be seen from the above equation (2), an old background image is stored in the vehicle area as it is during a traffic jam where a vehicle exists for a certain period of time or the like. Therefore, there is a problem that the background image is not updated properly.

That is, the conventional technology does not consider updating the background image that follows changes in the brightness of the external environment during traffic jams, and when traffic jams occur when the brightness changes occasionally, such as during dusk, the background image However, there is a problem that the vehicle detection may fail because the update is not performed.

It is an object of the present invention to provide a background updating method and an image processing device that can obtain a background image that follows the external environment even in a special state such as a traffic jam that is seen in a moving object detection device.

[Means for Solving the Problem] In order to achieve the above object, the present invention is a background image updating method for obtaining a new background image based on a background image and an input image, wherein the background image and the background image A background image updating method characterized by obtaining a change between an input image area and a background image area in an area having a difference smaller than a predetermined value by comparing the input image and using the obtained change to obtain a new background image. I will provide a.

To achieve the above object, the present invention provides a means for comparing a background image with an input image, and as a result of the comparison, obtains a change between the input image area and the background image area in an area having a difference smaller than a predetermined value. There is provided an image processing apparatus comprising: a means and a means for obtaining a new background image by using the obtained change and updating the background image.

Further, the present invention is to generate a new background image based on the background image and the input image, to update the background image, a means to compare the background image and the input image, the result of the comparison, from the predetermined value A means for obtaining a change between the input image area and the background image area in an area having a small difference; a means for applying the obtained change to a background image of an area having a difference larger than a predetermined value as a result of the comparison; and the new background. Among the input images used by the means for generating an image and updating the background image, a means for replacing the input image in an area having a larger difference than the predetermined value with the image subjected to the obtained change. An image processing device is provided.

Furthermore, according to the present invention, a means for generating a new background image based on the background image g and the input image f, a means for comparing the background image and the input image, and a result of the comparison, a difference from a predetermined value is determined. in small region 1, the input image area f ₁ and the background image and means for determining a rate of change r of the region g _1, and said change rate r, the result of the comparison, the background image g ₂ large region 2 of the difference than a predetermined value
And an input image f _{1 in an} area having a smaller difference than a predetermined value and a predetermined number A (0 ≦ A ≦ 1), A × (f ₁ + r × g ₂ ) + (1-A) × g Accordingly, there is provided an image processing device, characterized in that it has means for obtaining a new background image.

By the way, the above formula A × (f ₁ + Γ × g ₂ ) + (1-A) × g is given by A × (f ₁ + (1-A) × g ₁ in the region ₁ and It shows that a new background image is obtained by A × Γ × g ₂ + (1-A) × g ₂ .

As the change, it is preferable to use a change in brightness obtained from the difference between the maximum and minimum densities or the average density of each of the input image and the background image or the difference between two peak densities in the image.

The present invention also provides the moving object detecting device, which is the image processing device, further comprising means for detecting a moving object based on the background image and the input image.

[Operation] According to the background image updating method of the present invention, a change between the input image area and the background image area in an area having a smaller difference than a predetermined value is obtained by comparing the background image and the input image, and the change is obtained. To obtain a new background image.

Further, according to the image processing apparatus of the present invention, the background image and the input image are compared with each other, and as a result of the comparison, a change between the input image area and the background image area in an area having a difference smaller than a predetermined value is obtained. Then, a new background image is obtained by using the change.

Further, according to the image processing apparatus of the present invention, the background image and the input image are compared, and as a result of the comparison, the change rate between the input image area and the background image area in the area where the difference is smaller than the predetermined value is obtained. . Then, as a result of the comparison, the background image in the area having a larger difference than the predetermined value is subjected to the change, and is replaced as the input image in the area having the larger difference than the predetermined value in the input image. Then, a new background image is generated based on the input image and the background image.

Furthermore, according to the image processing apparatus of the present invention, the background image g and the input image f are compared, and as a result of the comparison, the input image area and the background image area in the area 1 where the difference is smaller than a predetermined value are compared. The rate of change r is determined. Then, the change rate r, the background image g _{2 of the} area 2 in which the comparison result has a larger difference than a predetermined value, and the input image f of the area having a smaller difference than the predetermined value.
A new background image is obtained by the calculation of A × (f ₁ + Γ × g ₂ ) + (1−A) × g using ₁ and a predetermined number A (0 ≦ A ≦ 1).

Further, according to the moving body detection device of the present invention, the moving body is detected from the background image and the input image described above.

As described above, the present invention is based on the experimental result that the rate of change in the background brightness and the like is substantially constant, based on the rate of change in the brightness between the input image and the corresponding background area of the background image, An optimum background image is created by correcting the change in the brightness of the moving object area of the input image.

That is, according to the present invention, the difference between the areas of interest between the input image and the background image is calculated, and a certain value is used to determine the area with a large difference (the area of a moving body such as a vehicle) and the area with a small difference (the background of a road surface, etc.). Area) and correct it by estimating the background brightness etc. of the area corresponding to the moving body of the input image from the ratio of the change in brightness etc. of the area corresponding to the background of the input image and the background image. An input image is obtained, and a new background image is obtained using the corrected input image and the background image.

Therefore, even if the brightness of the input image is not affected by abrupt changes in brightness (including noise), it is possible to follow the changes in brightness, etc. Can be created, so that the vehicle and the like can be satisfactorily extracted.

[Embodiment] An embodiment of the present invention will be described below by taking a moving body detection device as an example.

First, an outline of the moving body detection device according to the present embodiment will be described.

FIG. 7 shows the configuration of the moving body detection apparatus according to this embodiment.

The moving object detection device according to the present embodiment includes a camera 101 that captures an image, an image processing unit 100 that processes the captured image, a CPU that controls the entire device and processes the processing result of the image processing unit 100.
112, a monitor 111 for displaying images and various information.

The image processing unit 100 includes an A / D converter 102, an image memory 103, an inter-image operation circuit 104, a binarization circuit 105, and a binary image inversion circuit 10.
6, concentration frequency distribution calculation circuit 107, histogram memory 10
8. A linear combination circuit 109 and a D / A converter 110 are provided.

The image memory 103 has, for example, k gray-scale image memories G _{1 to} G _{k of} 256 × 256 pixels, and ₁ binary image memory B _{1 to} B _l for storing binary images as necessary. Prepare

The operation will be described below.

Based on a command from the CPU 112, the image processing unit 100 captures an image signal captured by the camera 101, and an A / D converter.
The image data is converted into density data of 128 gradations by the 102 and stored in the image memory 103.

Further, the image processing unit 100 uses the data in the image memory 103 based on a command from the CPU 112 to perform inter-image calculation, 2
The inter-image calculation circuit 104 and the binarization circuit 1 perform processes such as binarization, binary image inversion, density frequency distribution calculation, and linear combination calculation, respectively.
05 binary image inversion circuit 106, density frequency distribution calculation circuit 107,
It is processed by the linear combination calculation circuit 109, etc., and if necessary, the processing result is converted to a video signal by the D / A converter 110 and monitored.
Display at 111.

The histogram memory 108 is a memory for storing the frequency distribution and the like.

Next, an outline of a moving body (for example, vehicle) extraction using a background image and a flow measuring process of the moving body will be described.

FIG. 8 shows the flow of this processing.

First, an image is input from the camera 101 (step 120), a moving body extraction process (step 121) described later is performed, and then a measurement process (step 122) is performed.

After that, the predetermined background image update interval reference time t1 is compared with the elapsed time t from the time when the background was updated last time, and if the elapsed time becomes large, the background image update processing (step 124) is performed, The time t is initialized (step 125).

If the elapsed time has not reached the background image update interval reference time t1, the time t is incremented (step 126).

As described above, the background image is updated, the moving body is extracted, and the flow is measured in real time.

Here, a method of extracting the moving body described above (hereinafter, the case where the moving body is a vehicle) will be described.

FIG. 9 shows a conceptual diagram of this moving body extraction processing.

In this process, first, the image processing unit obtains a difference image 3 between the input image 1 and the background image 2, binarizes the difference image 3 with a predetermined threshold value, and creates a binary image 4. After that, in the present embodiment, only one lane worth of vehicles are extracted from this image 30,
For this image, the CPU 112 measures the number of vehicles, individual vehicle speeds, etc. for each lane31.

Next, details of the background image update processing portion, which is the central portion of the present invention, will be described.

FIG. 1 shows a functional diagram of the first background image update processing part.

First, when the input image f (n) 1 from the camera is stored in the image memory after A / D conversion, the inter-image calculation circuit causes the background image g (n-1) stored in the image memory in advance.
3. Find the difference image from 2. Next, the binarization circuit binarizes this difference image with a predetermined threshold value ε, and the binary image 4
To create. That is, an area where the difference value is larger than ε is represented by “1” (hatched portion in FIG. 1), and a small area is represented by “0” (white portion).

Next, the binary image inversion circuit inverts the value of each pixel of the binary image 4 (“0” → “1”, “1” → “0”) to create an inverted binary image 5 ( The shaded value is "1" and the white value is "0".

Here, f ₁ (n) represents the road surface area image 6 of the input image f (n) 1, and f ₂ (n) represents the vehicle area image 27. | f (n) -g (n-1) If | ε, then f (n) = f ₁ (n) (3) | f (n) -g (n-1) |> ε then f (n) = f ₂ (n) (4) , F (n) = f ₁ (n) + f ₂ (n) (5) Then, first, the road surface area image f ₁ (n) 6 is input to the input image f
(N) 1 and the inverted binary image 5
Multiply by the multiplication circuit 15 to obtain.

The vehicle area image f ₂ (n) 27 is the input image f (n) 1
And the binary image 4 are multiplied by the multiplication circuit 26, but the vehicle area image f ₂ (n) 27 is not always necessary in the present invention.

On the other hand, f ₁ of the input image 1 of the background image g (n-1) 2
Area g ₁ (n−1) 7 corresponding to (n), and f
_The region g ₂ (n-1) 8 corresponding to ₂ (n) is set as the background image g.
(N-1) 2 and the inverted binary image 5 are stored in the inter-image calculation circuit 104
The background image g (n-1) 2 and the binary image 4 are supplied to the multiplication circuit 16 of
Are respectively input to the multiplication circuit 17 to obtain the values.

At this time, the following equation holds.

g (n-1) = g 1 (n-1) + g 2 (n-1) Also ... (6), by multiplying the binary or reverse binary images, f
_{1 (n) 6, f 2} (n) 27, g 1 (n-1) 7, and g
_{In 2} (n-1) 8, the shaded area in FIG. 1 is an effective area having a certain density value, but the other white area is an invalid area having a density value of "0".

Next, since the change in the brightness of the road surface due to the illumination change changes linearly with the brightness of the target object (road surface, vehicle, etc.) (constant rate of change), the background image g (n-1) 2 is changed. The change rate γ of the brightness of the input image f (n) 1 is calculated.

First, the density product frequency distribution calculation circuit 107 uses the previously calculated image f
The density frequency distribution of ₁ (n) is obtained 9 (see FIG. 2A), and the maximum density max {f ₁ (n)} is calculated from the density frequency distribution.
Obtain the minimum density min {f ₁ (n)} 10.

On the other hand, the background image g ₁ (n-1) corresponding to f ₁ (n)
The density frequency distribution of 7 is calculated 11, and the maximum density max is calculated from the distribution 11.
Obtain {g ₁ (n-1)} and minimum density min {g ₁ (n-1)} 12. Next, using these density values, the change rate γ of the brightness of the input image 1 with respect to the background image 2 is obtained by the equation (7) 13.

The relationship between the density frequency distribution and the change rate γ is shown in FIG. 2 (c).

Note that f ₁ (n) and g ₁ (n-1) are white lines on the road surface,
It is preferable to include a wide density distribution from the dark portion to the bright portion in order to improve the accuracy of the change rate γ. In addition, of the two peaks in FIGS. 2A and 2B, the left peak represents a dark portion of the road surface and the right peak represents a whitish portion such as a white line. The above equation (7) means the inclination γ in FIG. 2 (c).

Here, using the rate of change γ and g ₂ (n-1), the input image in the f ₂ (n) region is replaced with the following formula of f ₂ ′.

f ₂ ′ (n) = γ × g ₂ (n−1) (8) Further, the values of the constants A and B (there is a relation of the equation (9)),
It is known that a new background image g (n) can be created by the linear combination operation of the current background image f (n) and the previous background image g (n-1) represented by the equation (10).

A + B = 1 g (n) = A * f (n) + B * g (n-1) (10) Then, the expression (10) is calculated by the expressions (5), (6), and (8) as follows. Represented.

g (n) = A × {f ₁ (n) + γ × g ₂ (n-1)} + B ×
g (n-1) = A × f 1 (n) + A × γ × g 2 (n-1) + B × g (n-
1) ... (11) The execution of the operation of the equation (11) is performed by the linear combination operation circuit 109,
As shown in FIG. 1 in particular, (because decimal values, it is desirable to scale up in advance about 2 ¹⁰ times) constant A Type Type image f ₁ (n) to multiplier circuit 18 , divided by 2 ¹⁰ by 10-bit shift down-shift circuit 19 and the output, obtaining a first term of formula (11).

Further, multiplied constants A and the rate of change γ of the (previously 2 ¹⁰ degree scale-up) in multiplier circuit 20 multiplies the output result and the background image g ₂ (n-1) 8 in multiplier circuit 21, the by scaling down 2 ¹⁰ as above by the shift circuit 22 outputs, determining the second item of the equation (11).

Further, the constant B (pre-2 ¹⁰ times previously scaled up) a background image g (n-1), type 2 to multiplier circuit 23, scaled down by 2 ¹⁰ in the same manner as described above by the shift circuit 24 to output Then, the third term of the equation (11) is obtained.

Then, a new background image g (n) 28 is obtained by adding these three obtained results by the addition circuit 25.

The new background image g (n) 28 is stored in the image memory, and the background image g (n-
1) Processed as 2.

As described above, according to the present embodiment, it is possible to eliminate the influence of the vehicle portion of the input image f (n) on the background update. Therefore, even if there are many vehicles in the input image due to traffic congestion or the like, the brightness can be reduced. The background image can be reliably updated by following changes in

Further, even if noise in which the brightness changes abruptly occurs in the input image, that part is corrected based on the brightness of the background image, as can be seen from equations (4) and (8). Therefore, a new background image can be created without any problem, and vehicle extraction can be performed accurately.

In addition, the present embodiment has a high possibility of being implemented by hardware and is capable of high-speed background updating.

Further, it may be simplified by approximating the above processing. In this case, instead of executing the equation (11), g (n) = f ₁ (n ) + Γ × g ₂ (n-1) (11 ') is obtained and the background image g is obtained by executing the equation (11').
(N) Find 28.

In this case, although it can be realized by setting the coefficients A = 1 and B = 0 in the equation (11), it goes further and the coefficients A and B and the multiplication circuit 1 of the background update processing part in FIG.
The 8, 20, 23 and shift circuits 19, 24 can be deleted.

FIG. 3 shows a functional diagram of the second background update processing portion.

Further, as a method of obtaining the change rate γ, the input image f
It is also possible to use the ratio of the average density values of ₁ (n) and the background image g ₁ (n-1), and in this case, instead of the maximum / minimum density detections 10 and 12, equations (12), (13 ) Is executed instead of the change rate calculation 13 by the formula (14).
The change rate calculation 33 for executing

FIG. 4 shows a functional diagram of the third background update processing portion.

Further, as another method for obtaining the change rate γ, the input image f ₁
(N) and the ratio of the difference in peak density between the road surface and the white line portion of the background image g ₁ (n-1) can be used. In this case, the maximum / minimum detection of the background update portion in FIG. 1 can be used.
Instead of 10 and 12, as shown in Fig. 5, peak concentration detection
Instead of the change rate calculation 13, 34 and 35 may be provided with a change rate calculation 36 that executes the equation (15).

FIG. 5 shows a functional diagram of the fourth background updating portion.

FIG. 6 shows a conceptual diagram of the processing of the peak density detection 34, 35.

First, the maximum and minimum densities maxf ₁ and minf ₁ are obtained from the density frequency distribution of the input image f ₁ (n) shown in FIG. 6 (a), and the frequencies l _i on the bright side and the dark side are checked from the average value. Then
Concentration level peak that takes the peak value of each area
(F _1h ) and peak (f _1d ) are calculated.

Similarly, peak (g _1h ) and peak (g _1d ) are obtained from the density frequency distribution of the background image g ₁ (n-1) shown in FIG. 6 (b).

Then, the inclination change rate γ in FIG. 6 (c) is obtained.

In this way, by avoiding the influence of noise near the maximum density or the minimum density of the density frequency and obtaining the rate of change in brightness by the average density value and peak density value that represent the overall brightness of the road surface, stable The corrected image f ₂ (n) thus obtained can be obtained.

Therefore, it is possible to more accurately extract the vehicle.

In this embodiment, a new background image is obtained by the method based on the equations (10) and (11 ′).
The correction of the moving object partial area of the input image in the background update processing is the main point, and other methods can be similarly realized and are effective.

Further, not only the detection of the vehicle but also the detection of other moving objects such as a person can be similarly realized.

As described above, according to the present embodiment, it is possible to create a background image while considering a change in brightness even during a traffic jam or the like, so that a traveling vehicle, a stopped vehicle, or the like can be stably extracted.

In the above embodiments, the present invention has been described in detail by taking a moving body detection device as an example. However, the present invention is not limited to detection of a moving body, and may be, for example, an image processing device such as an abnormality monitoring device in a factory. The same can be realized in.

Further, in the anomalous embodiment, an example of using brightness as a change between the background image and the input image has been described, but for example, a local temperature anomaly monitoring device in the furnace, or a test paper such as a thermotape or litmus paper outdoors In the monitoring device, color may be used instead of or in combination with luminance.

[Effects of the Invention] As described above, according to the present invention, a background updating method capable of obtaining a background image following the external environment even in a special state such as a traffic jam seen in a moving object detection apparatus, and
An image processing device can be provided.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing the functional configuration of the first background image update processing portion, FIG. 2 is an explanatory view showing the concept of change rate calculation processing using maximum and minimum densities, and FIG. Fourth, a functional block diagram showing a functional configuration of a background image update processing portion,
FIG. 6 is a functional block diagram showing the functional configuration of the third background image update processing portion, FIG. 5 is a functional block diagram showing the functional configuration of the fourth background image update processing portion, and FIG. 6 uses the peak density level. FIG. 7 is an explanatory view showing the concept of change rate calculation, FIG. 7 is a block diagram showing the configuration of a moving body detection device, FIG. 8 is a flow chart showing the flow of processing of moving body flow measurement, and FIG. 9 is moving body extraction. It is explanatory drawing which shows the flow of a process. 1 ... Input image, 2 ... Background image, 3 ... Difference image, 4 ... Binary image, 5 ... Inverted binary image, 13 ... Change rate calculation circuit, 14 ... Subtraction circuit, 15 ... Multiplication circuit, 25 ... Addition circuit, 101 ... camera, 1
02 ... A / D conversion circuit, 103 ... Image memory, 104 ... Image arithmetic circuit, 105 ... Binary circuit, 106 ... Binary image inverting circuit, 107 ...
Concentration frequency distribution calculation circuit, 108 ... Histogram memory, 109
… Linear combination calculation circuit, 111… Monitor, 112… CPU.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Norio Tanaka 5-2-1 Omika-cho, Hitachi City, Ibaraki Prefecture, Ltd. Inside the Omika Plant, Hitachi, Ltd. (56) Reference JP-A 1-211083 (JP, A) JP-A-63-198193 (JP, A) JP-A-62-114064 (JP, A)

Claims (7)

[Claims] 1. An image processing apparatus having means for generating a new background image based on the background image g and the input image f and updating the background image, the means for comparing the background image g and the input image f. And means for obtaining the rate of change r in brightness between the input image f ₁ and the background image g ₁ in a region where the difference is smaller than a predetermined value as a result of the comparison,
As a result of the comparison, a means for multiplying the background image g ₂ corresponding to a region having a difference larger than a predetermined value by the obtained change rate r,
Of the input image f used by the means for generating the new background image, means for replacing the input image f ₂ corresponding to a region having a larger change than the predetermined value with an image r × g ₂ multiplied by the change rate. An image processing apparatus comprising: 2. A means for comparing the background image g and the input image f, and as a result of the comparison, a rate of change r between the input image f ₁ and the background image g ₁ in an area having a change smaller than a predetermined value is obtained. Means, the rate of change r, and as a result of comparison, a background image g _{2 in an} area having a larger difference than a predetermined value, an input image f _{1 in an} area having a smaller difference than a predetermined value, and a predetermined number A (0 ≦ A ≦ 1) and using A
An image processing apparatus comprising: a unit that obtains a new background image by a calculation of × (f ₁ + r × g ₂ ) + (1-A) × g and updates the background image. 3. The image processing apparatus according to claim 1, wherein the change rate r is the maximum of each of the input image f ₁ and the background image g ₁ in an area having a difference smaller than a predetermined value. An image processing apparatus characterized by using a rate of change in brightness obtained from a difference in minimum density. 4. The image processing apparatus according to claim 1, wherein the change rate r is an average density of each of the input image f ₁ and the background image g ₁ in an area having a difference smaller than a predetermined value. An image processing apparatus, characterized in that a change rate of brightness obtained from the above is used. 5. The image processing apparatus according to claim 1, wherein the change rate r is within the respective images of the input image f ₁ and the background image g ₁ in an area having a difference smaller than a predetermined value. Using the rate of change in brightness obtained from the two peak densities of the input image f ₁ in the region where the difference is smaller than the predetermined value
Among the two peak densities, the density Peak f ₁ h having the highest occurrence frequency on the bright side of the intermediate value between the maximum and minimum densities of the input image f _{1 and} the occurrence frequency on the dark side of the intermediate value being the highest. High density Peak f ₁ d, and the background image g ₁ in an area where the difference is smaller than the predetermined value.
The two peak densities in the middle are density Peakg ₁ h that has the highest occurrence frequency on the bright side of the intermediate value between the maximum and minimum values of the background image g1, and the highest occurrence frequency on the dark side of the intermediate value. The image processing apparatus is characterized in that the density is Peakg ₁ d, and the change rate r of the brightness is obtained according to r = (Peakf ₁ h-Peakf ₁ d) / (Peakg ₁ h-Peakg ₁ d). 6. An image processing apparatus according to claim 1, 2, 3, 4 or 5, further comprising means for detecting a moving body based on the background image and the input image. . 7. An image processing method for correcting the input image f, which is used when a new background image is generated based on the background image g and the input image f and the background image is updated. The input image f is compared, and as a result of the comparison, the rate of change in brightness r between the input image f ₁ and the background image g ₁ in the area where the difference is smaller than the predetermined value is obtained. The background image g ₂ corresponding to the region having a larger difference is multiplied by the obtained change rate r to generate a new background image. Among the input images f used by the means, the change is larger than the predetermined value. An image processing method, characterized in that an input image f ₂ corresponding to a region is replaced with an image r × g ₂ multiplied by the change rate.