JPH0467276A - Method and device for detecting moving object by image processing - Google Patents

Abstract

PURPOSE:To selectively extract only the moving object from in image information in which a background and the moving object are mixed by averaging a storage image, and subtracting the averaged image from the image supplied successively. CONSTITUTION:A standard screen S generated by a standard screen generating circuit 2 is stored in a standard screen memory 3. A subtracting circuit 4 subtracts the standard screen from image data of that time point at every time gamma. The standard screen S is a screen of a background in a visual field of a camera, therefore, a result of operation of the subtracting circuit 4 becomes a screen obtained by removing a background from the screen of that time point. That is, the subtracting circuit 4 outputs image data of a moving body at that time point. The image data of this moving body is stored in a moving body memory 5.

Description

DETAILED DESCRIPTION OF THE INVENTION "Field of Industrial Application" The present invention relates to a method and apparatus for detecting a moving object using image processing, which is suitable for use in detecting a moving object outdoors with a complex background.

"Prior Art" Conventionally, moving objects have been detected by performing various calculations on images captured from a video camera or the like. For example, there are a method of finding a shift that minimizes the difference between successive images (between frames), and a method of finding it from a ratio between Fourier-transformed images.

``Problem to be solved by the invention J1 By the way, the above-mentioned moving object detection method requires a system that can perform high-speed processing in order to perform advanced calculations. However, when a device based on a conventional detection method is used outdoors where the background has complex movements, the conventional moving object detection method and device cannot detect the target moving object. Since there are many other movements (such as the swaying of trees), it is difficult to distinguish these movements from the movements of the moving object.

This invention was made in view of the above-mentioned problems.
The object of the present invention is to provide a method and device for detecting a moving object using image processing that can selectively extract only moving objects other than objects that are constantly present from among image information containing a mixture of background and moving objects. .

"Means for Solving the Problem" In order to solve the above-mentioned problem, in the invention according to claim I, the images of a predetermined period in the past are accumulated and stored in a storage means with respect to sequentially supplied images. The invention according to claim 2, characterized in that the images stored in the storage means are averaged, and the averaged 73 images are subtracted from the sequentially supplied images.
Standard image generation means for capturing sequentially supplied images at predetermined time intervals, storing cumulative images in a past predetermined period in the storage means, and averaging and outputting the cumulative images output by the storage means. The invention according to claim 3, further comprising: subtracting means for subtracting and outputting the averaged image outputted by the standard image generating means from the sequentially supplied images. In the moving object detection method using image processing described above, the sequentially supplied images are temporarily stored in a current image storage means, and the averaged image is subtracted from the image stored in the current image storage means. It is characterized by

"Action" According to the invention described in claims 1 and 2, the images of a predetermined period in the past are accumulated and stored in a storage means, and the images stored in the storage means are A standard image means performs averaging, and a subtraction means subtracts the averaged image from the sequentially supplied images.

According to the invention set forth in claim 3, in the invention set forth in claim I, the sequentially supplied images are stored in the current image storage means, and the standard image is subtracted from the image stored in the current image storage means. It is characterized by

"Embodiments" Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention. Further, FIG. 2 is a conceptual diagram for explaining the method of detecting a moving object according to the present invention.

First, in FIG. 1, 2 is a standard screen generation circuit,
Video data captured by the camera is sequentially supplied at intervals of time Δ shown in FIG. This standard screen generation circuit 2 operates at time t.
For, at time Δ in the past time τ minutes (Δτ≧Δt)
A standard screen S is generated by averaging the n screens. This standard screen S is a screen of only the background (stationary object) obtained by removing video information of a moving object from the screen captured by the camera, and serves as a reference screen for detecting a moving object. Here, generation of the standard screen S and removal of video information of a moving object will be explained.

First, a screen matrix consisting of i dots x j dots at an arbitrary time t is expressed as DIJ (d IJ, t ). Also, the matrix of the desired standard screen S is S I
Let J (S IJ, t ). On the other hand, the past n screens for the above-mentioned time τ minutes are Dx, +(t-τ), D,
J (t-τ-Δτ), Z)11(t-τ-2・Δτ)
.

・・・・・・・・・・・・DiJ(mu-τ-(n-1)
Δτ). Next, when calculating the average matrix i:)tJ (d.J) of these n data matrices, for each dot, It has the characteristic that it moves between Δτ and masks one object.In other words, if there is no change between each screen (hereinafter referred to as a frame), the value will be averaged as it is, but on the other hand, if there is a moving object Because the position of the moving object changes in each frame, it is averaged and masked.Furthermore, any instantaneous changes (changes in brightness, noise, etc.) are also averaged, so Therefore, the final average matrix f51J (d, J) will be masked at least at time (
1τ-(n-1)Δτ) to time (t-τ), only the background image is removed, with the image of the object moving within the field of view of the camera removed. That is, the present invention is characterized in that the screen containing only this background image is set as the standard screen S. Therefore, the matrix S IJ (S I
J, i) is S+J(s+,,1.t) --! −
ΣD IJ (at t-r-(n-1)Δ)...
(2) becomes. Note that the time τ may be set to "0", and in that case, the standard screen S will change from time t to time (t(n-1)Δ
It is generated based on the past up to τ).

The standard screen S obtained in this way has the following characteristics.

(1) The S/N ratio of the standard screen S is improved by F times.

Consequently, irregular noise within the screen is significantly reduced.

(2) The moving objects (or abnormalities) included in each of the n screens past the time (τ+nΔτ) are 1/n on the standard screen S, and are therefore almost eliminated.

(3) Sudden changes in screen brightness (flashes caused by lightning, shading by clouds, etc.) are averaged out and removed.

Since the standard screen S has the above characteristics, it can be used as a reference screen for detecting a moving object at any time t.

The standard screen S generated by the standard screen generation circuit 2 in this manner is stored in the standard screen memory 3. 4 is a subtraction circuit, which subtracts the standard screen S from the image data at that time every time Δτ.

Since the standard screen S is the background screen in the field of view of the camera as described above, the calculation result of the subtraction circuit 4 is a screen obtained by removing the background from the screen at that time. That is, the subtraction circuit 4 outputs the image data of the moving object at that point in time. This image data of the moving object is stored in the moving object memory 5.

Next, the above-mentioned standard screen generation circuit 2 and its memory 3
The configuration will be explained with reference to the block diagram shown in FIG.

FIG. 3 is a block diagram showing the configuration of the standard screen generation circuit 2 according to this embodiment. In this figure, parts corresponding to the respective parts in the block diagram shown in FIG. 1 are given the same reference numerals and explanations will be omitted. In the figure, 10 is A/
It is a D converter, which converts the video signal output by the camera (the path shown) into a digital signal according to the basic clock signal CL (every hour), and converts the video signal outputted by the camera (the path shown) into a digital signal, and sends the video data to the current frame memory l and adders 14-1 to 14, which will be described later. -n (n: number of frames). Next, 11 is a frame memory address counter, which generates the above-mentioned address signal AD and frame signal F'R according to the above-mentioned basic clock signal CL. 1-15
-Output to rl. The frame memory selection counter 12 includes a selection signal C8, ~C8, and which frame memory 15-8~ for selecting which frame memory 15□~15-, , the video data is outputted to according to the frame signal FR. 15-, outputs a control signal CD for selecting whether to output the video data stored in . Each of the select signals C81 to C8o is connected to the gate circuit 1.
3-1 to 13-1. Further, the control signal CD is supplied to the selector circuit 16. Gate circuit 11. -1
3-n performs an AND operation on the video data output from frame memories 15- to 15-n and select signals C81 to C8n, and outputs the result to adders 14- and 14-o. That is, the gate circuits 13-1 to 13-, .
Only when the select signals CS, .about.C5,l become high level, the video data stored in the frame memories 15-1 to 15-, . . . are output to the adders 14-1 to 14-n. The adders 14, . -15-n. Frame memory 15~
15-, . . . sequentially store the supplied video data according to the address signal. That is, the frame memory 15-
In each of 1 to 15-n, video data of a plurality of frames are stored in a duplicated (added) manner, and the video data of each frame memory is shifted by a time Δτ. (Details will be explained later). Next, the selector circuit 16 selectively outputs the video data stored in any one of the frame memories 15-1 to 15-n to the multiplier 17 in accordance with the control signal CD. The multiplier 17 multiplies the selected f video data by a constant K (=I/n) and outputs the result to the subtracter 18 . The subtracter 18 is supplied with the video data for each time stored in the current frame memory l, and calculates the time Δ from the selected video data.
Subtract the video data for each τ.

The video data R resulting from the subtraction is stored in the mobile memory 5. Since the f-co video data stored in the mobile object memory 5 is data only for the mobile object, its speed and distance are measured by various external devices (as shown).

Next, regarding the operation of this embodiment described above, FIG. 4(a)
This will be explained with reference to the timing charts shown in (h). Note that here, in order to simplify the explanation, we will use the standard screen S.
The frame memory for creating is assumed to be 3 frames. First, a standard screen S is created by turning on the power or instructing an operation. First, at time t0, A
/D converter IO converts the video captured by the camera into video data A and sends it to the current frame memory l and adders 14I~
Output to 14-3. The video data A is stored in the current frame memory I. Further, the frame memory address counter 11 outputs the aforementioned address signal AD and frame signal PR in accordance with the basic clock signal CL. Next, the frame memory selection counter 12 sets the select signal C8 to a low level in accordance with the frame signal FR (see FIG. 4(b)). Therefore, gate circuit 13. , is closed, and only the video data A output from the adder 14-1 is stored in the frame memory 15 (see FIG. 4(e)). Although the video data A is also stored in the frame memories 15-2 and 15-3 (see FIG. 4 (f) and (g)), when creating the standard screen S for the first time, the previous data There is no need to consider whether or not to add to.

Therefore, in this case, the select signal C8 and CS
s is set to a high level as shown in FIGS. 4(C) and (d).

Next, time t. From time to time t, video data from the video camera is stored in the current frame memory l every time.

Next, time t. At time t1 after Δτ from A/D
The converter IO converts the video captured by the camera into video data B and outputs the video data B to the current frame memory 1 and the adder 14. The video data B is stored in the current frame memory l. In addition, the frame memory address counter 11
outputs the aforementioned address signal AD and frame signal PR in accordance with the basic clock signal CL. Next, the frame memory selection counter 12 receives the frame signal FR.
Accordingly, the select signal C81 is set to high level, and the select signal C8 is set to low level (see FIGS. 4(b) and 4(c)). Therefore, the gate circuit 13 becomes an oven, and in the adder 14-, the frame memory 15
- Video data A stored in 1 and video data B above
are added (assumed to be video data (A+B)). This video data (A and B) is stored in the frame memory 15-1 (see FIG. 4(e)). Also gate circuit 1.3
-, are closed, so the video data B is stored as is in the frame memory 15-3 via the adder 14-2 (see FIG. 4(f)). Note that from time t1 to time t.

The period up to the above-mentioned time t. Since it is the same as up to -tl, the explanation will be omitted below.

Next, at time t, which is further Δτ after time t,
The A/D converter 10 converts one image captured by the camera into video data C and sends it to the current frame memory 1 and adder 14.
Output to. The video data C is stored in the current frame memory l. Next, frame memory selection counter 1
2 sets the select signal C8 to a high level and sets the select signal CS3 to a low level in accordance with the frame signal PR (see FIGS. 4(c) and 4(d)). Select signal C
81 maintains a high level state, the gate circuit 13-3 becomes an oven, and in the adder 14,
Video data (A+B) and video data C are added (
Video data (A+B+C)). Further, the gate circuit 13-2 functions as an oven, and in the adder 14-t, the video data B stored in the frame memory 15 and the video data C are added (video data (B+C
).

This video data (B+C) is stored in the frame memory 15-2.
(see FIG. 4(f)). Also, since the gate circuit 13-3 is closed, the frame memory 15-3 is closed.
3, the video data C is stored as is through the adder 14-8 (see FIG. 4(g)).

In this way, the video data AB and C output by the camera are
First, it is stored in the frame memory 15-1. and,
At time t3 when Δτ has elapsed, video data D is stored in the current frame memory 1 (see FIG. 4(e)). Further, the frame memory selection counter I2 selects the output data of the frame memory 15-1 in response to the control signal CD.

As a result, the video data (A+B+C) stored in the frame memory 15- is supplied to the multiplier 17 via the selector circuit 16. Then, in the multiplier 17,
It is multiplied by a multiplier K (-1/n) and supplied to the subtracter 18 as an averaged standard screen S. The subtracter 18 subtracts the video data (A+B+C) from the video data D stored in the current frame memory l (see FIG. 4).
h)). In this case, if the video data D includes a moving object, the video data R output by the subtractor 18 will be video data of only the moving object. This video data R is stored in the mobile memory 5.

On the other hand, at the same time t3, the frame memory selection counter 12 sets the select signal C81 to low level and sets the select signal CS3 to high level according to the frame signal PR (see FIGS. 4(b) and 4(d)). Also,
The select signal C8 maintains a high level state (see FIG. 4(c)).

Therefore, the gate circuit 13-8 is closed, and only the video data D is stored in the frame memory 15-1 as shown in FIG. 4(e). In addition, the gate circuit 13-
Since frame memory t5-.7 is open, frame memory t5-. The video data (B+
C) and video data D are added (B+C
+D) is newly stored (see FIG. 4(f)). Furthermore, since the gate circuit 13-3 is open, video data (c+D) is stored in the frame memory 15-3 (see FIG. 4(g)).

Next, at time t4, the frame memory selection counter 12 controls the frame memory 15-2 by the control signal CD.
Select output data. As a result, frame memory 1
5. The video data (B+C+D) stored in is supplied to the multiplier 17 via the selector circuit 16. Then, in the multiplier 17, the video data (B+C+D) is multiplied by a multiplier K(-1/n) and supplied to the subtracter 18 as an averaged standard screen 5 (-(B+C+D)'). The subtracter 18 subtracts the video data (B+C+D) from the video data E stored in the current frame memory 1.
)' (see Figure 4(h)).

In this case, if the video data E includes a moving object, the video data R output by the subtractor 18 will be video data of only the moving object. This video data R is stored in the mobile memory 5.

On the other hand, at the same time t4, the frame memory 15-1
．． 15-2 and 15-3 each contain video data (D
+E), video data E, and video data (C+D+E) are stored (see FIGS. 4(e) and (g)).

Then, at time t5, the frame memory 15-3
The video data (C+D+E) stored in is multiplied by a multiplier K(-1/n) to form an averaged standard screen s (=(c+D×E)'). And the fourth
As shown in Figure (h), from the video data F stored in the current frame memory l to the video data (C+D+E)
' is subtracted and stored as video data R in the mobile memory 5. And frame memory t5-1+15-
At t and 15, video data (D + E +
F), video data (E+P), and video data F are stored (see FIGS. 4(e) and (f)).

Hereinafter, at time t and after, the frame memory 15-
The video data stored in t, 15-t and 15-3 (
The standard screen S) is sequentially supplied to the subtracter I8 to obtain new video data R, which is stored in the mobile memory 5. −
Also, video data G at each time, )1. ···teeth,
Frame memories 15.15- and 15-3ζa are sequentially shifted by 3 frames and
be remembered.

Then, at each time after time t3, the video data R stored in the mobile body memory 5 is output to the outside, and the moving speed, shape, etc. of the mobile body are measured based on the video data R.

As described above, in this embodiment, a standard screen S is created by averaging a plurality of past frames, and when a moving object enters the field of view of the camera, based on the standard screen S,
An image of only the moving object is taken out and stored in the moving object memory 5.

Note that in the embodiment described above, the frame address counter 11. Frame memory selection counter I2, gate circuit 1
3-I to If11, the selector circuit 16, the multiplier 17, the subtracter 18, etc. are configured by hardware, but
These may be realized by software.

Further, the above-mentioned moving object is not limited to the type or state of the material forming it, and may be any object that has brightness and darkness contrast with the background.

Moreover, the above-mentioned camera is not limited to being sensitive only to visible light, but may also be a means sensitive to invisible light (ultraviolet rays, infrared rays, etc.), X-rays, laser beams, etc., for example.

"Effects of the Invention J As explained above, in this invention, the images of a predetermined period in the past are accumulated and stored in a storage means, and the images stored in the storage means are is averaged by a standard image means, and the averaged image is subtracted by a subtraction means from the sequentially supplied images, so that only moving objects can be selectively detected from image information containing a mixture of background and moving objects. Benefits that can be extracted are obtained.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.
FIG. 2 is a flowchart for explaining the operation of the embodiment, and FIG. 3 is a block diagram showing the detailed configuration of the embodiment.
FIGS. 4(a) to 4(h) are timing charts for explaining the operation of this embodiment. 1...Current frame memory (current image storage means),
2... Standard screen generation circuit, 3... Standard screen memory (storage means), 4...
Subtractor (subtraction means), 5...mobile memory. Applicant FMT Fujikura Electric Wire Co., Ltd. Figure 1 Figure 2

Claims (3)

[Claims] (1) For images supplied sequentially, the images of a predetermined period in the past are accumulated and stored in a storage means, the images stored in the storage means are averaged, and the average is calculated from the images supplied sequentially. A moving object detection method using image processing characterized by subtracting a converted image. (2) It comprises a storage means, captures sequentially supplied images at predetermined time intervals, stores cumulative images in the past predetermined period in the storage means, and stores the cumulative images output by the storage means. Image processing characterized by comprising: a standard image generating means for averaging and outputting; and a subtracting means for subtracting and outputting the averaged image output by the standard image generating means from the sequentially supplied images. moving object detection device. (3) In the moving object detection method using image processing according to claim 1, the sequentially supplied images are temporarily stored in a current image storage means, and the averaged images are extracted from the images stored in the current image storage means. A moving object detection method using image processing characterized by subtracting an image.