JP2004151980A - Moving object detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method that can detect a moving object over a wide range of moving speed. <P>SOLUTION: The method acquires three images A to C in ongoing time series order. An acquisition interval T1 between the images B and C and an acquisition interval T2 between the images A and B are set differently. The image A and the image B are differentiated and binary-coded to produce an image D, and the image A and the image C are differentiated and binary-coded to produce an image E. If a moving object moves at relatively high speed or moves at relatively low speed, the image D of the shorter acquisition interval or the image E of the longer acquisition interval includes movement components (a set of "1" pixels) respectively. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method for detecting a moving object in a predetermined area by capturing an image of a predetermined area with an imaging device and processing image data of the captured area.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there has been known a method of detecting whether or not a moving object exists in a predetermined area by obtaining a difference image between two images having different shooting times at a predetermined time.
[0003]
In addition, three images (first, second, and third images) having different photographing times are acquired at equal intervals, and a first difference image from the first image and the second image is obtained as a first image and a second image. There is a method that obtains a second difference image from a third image and performs a logical product of the first and second difference images to remove background noise and detect a moving target (for example, And Patent Document 1.).
[0004]
[Patent Document 1]
Japanese Patent Publication No. 6-64613 [0005]
[Problems to be solved by the invention]
When the moving object moves at a high speed, the moving object may pass through the angle of view of the photographing device between the photographing time and the next photographing time. In this case, since no moving component exists on the difference image, the moving object cannot be detected. As described above, if the speed of the moving object is higher than the acquisition interval of the image used for the processing, the moving object may not be detected in some cases.
[0006]
On the other hand, when the moving object moves at a low speed, the moving component cannot be obtained by the difference processing if the shift amount of the image of the moving object in the two images to be differentiated is small enough to be lower than the resolution of the image sensor. .
[0007]
When three images are used as in the above-described conventional example, the probability that a moving object can be detected based on a difference image is higher than when two images are used. When moving at a very high speed or moving at a very low speed, a moving object may not be detected from the difference image.
[0008]
Therefore, an object of the present invention is to provide a method capable of detecting a moving object over a wide range in terms of moving speed.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, a moving object detection method according to the present invention includes:
In a method of imaging a predetermined region with an imaging device and processing a captured image data to detect a moving object in the region,
Obtaining an image group having first, second and third images, which are first, second and third images in chronological order,
Selecting a set of two images from the group of images to obtain a first difference image; and selecting a set of images different from the selected set of two images from the group of images to obtain a second difference image. Obtaining an image;
A time interval T1 from the acquisition of the first image to the acquisition of the second image is different from a time interval T2 from the acquisition of the second image to the acquisition of the third image.
[0010]
The first difference image may be a difference image between the first image and the third image, and the second difference image may be a difference image between the second image and the third image. In this case, it is preferable that the time interval T1 be longer than the time interval T2.
[0011]
The method may further include the step of binarizing each of the first and second difference images.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment according to the present invention will be described below with reference to the accompanying drawings. In the following description, an example in which the moving object detection method according to the present invention is applied to a surveillance camera system will be described. However, the present invention is not limited to this, and can be applied to various fields.
[0013]
FIG. 1 is a configuration diagram illustrating an embodiment of a surveillance camera system. This system 2 is for acquiring and recording a still image of a moving object by adjusting the direction and magnification of a camera when a moving object is detected in the monitoring area.
[0014]
The surveillance camera system 2 generally includes a surveillance camera device 4 (hereinafter, simply referred to as a camera) for patrol monitoring of a plurality of surveillance areas, and a computer 6 having a program for controlling the camera 4 and performing predetermined image processing. And
[0015]
The camera 4 includes an imaging unit 8 having a CCD sensor (not shown) as an imaging element and a lens mechanism (not shown) for forming an image on the CCD sensor. The lens mechanism is a zoom lens mechanism with a variable photographing magnification, and is provided with a zoom actuator 10 for changing the photographing magnification of the lens mechanism. The camera 4 also includes a pan / tilt actuator 12 for rotating and driving the imaging unit 8 by a predetermined angle in each of the tilt direction and the pan direction.
[0016]
A camera control unit 15, an image processing unit 16, an image storage unit 18, a timer 20, and a communication control unit 22 are connected to a CPU (central processing unit) 14 of the computer 6. The camera control unit 15 controls the zoom and pan / tilt actuators 10 and 12 to control the visual field of the camera 4. The image processing unit 16 is for performing various image processing described later on the time-series image data captured by the camera 4 and stored in the image storage unit 18. The image storage unit 18 stores a time-series image input in time series from the camera 4, an image processed by the image processing unit 16, and a still image of a moving object. The timer 20 has a built-in clock and provides time information to the CPU 14. The communication control unit 22 is for performing communication with the outside. For example, the communication control unit 22 stores a still image stored in the image storage unit 18 through a network (for example, the It is for transmitting to an external terminal installed in a local monitoring center or the like.
[0017]
2 and 3 are flowcharts showing an example of a process for performing monitoring by the monitoring camera system of FIG. First, when the system is started in step S1, the actuators 10 and 12 of the camera 4 are controlled so that the monitoring area to be first patrol enters the field of view of the camera 4. Note that the area to which the camera was facing when the system was activated may be the area to be monitored first. The camera 4 monitors a monitoring area to be first patrol for a preset time (step S2).
[0018]
In step S3, the image processing is repeated until the traveling time at the current traveling position ends.
[0019]
Specifically, with reference to the subroutine of FIG. 3 and FIG. 4, in step S31, three images A, B, and C are selected in the order of the latest photographing time. These images are images relating to the same monitoring area. The computer 6 acquires an image from the CCD sensor of the camera 4 at a fixed interval T, that is, in synchronization with a vertical drive (VD) signal. The acquisition intervals of the images A, B, and C for detecting a moving object are set at intervals. It is an integral multiple of T. The acquisition interval T1 between the images B and C and the acquisition interval T2 between the images A and B are set to be different. As an example, assuming that a VD signal is generated 30 times per second, that is, an image is acquired at 30 fps (frames per second), T1 is set to 27 cycles of the VD signal and T2 is set to 4 cycles (images A and The acquisition interval of C is 31 cycles).
[0020]
In step S32, the images A and B are differentiated, and subsequently, binarization processing is performed to remove noise and the like, thereby obtaining an image D. If the moving object is not reflected in both images, the pixels of the difference image D are all “0”. Similarly, in step S33, the images A and C are differentiated, and a binarization process is performed to obtain an image E. As a result, when the moving object moves at a relatively high speed, the moving component (a set of “1” pixels) is included in the image D having a short acquisition interval when the moving object moves at a relatively low speed, and in the image E having a long acquisition interval when the moving object moves at a relatively low speed. Will be included.
[0021]
In step S34, an image F as a logical sum of the images D and E is obtained. As a result, it is possible to detect a moving object over a wide range in terms of the moving speed based on the image F. After obtaining the logical sum image F, the flow of the image processing ends.
[0022]
The acquisition interval T1 between the second newest image B and the oldest image C is preferably set to be larger than the acquisition interval T2 between the newest image A and the image B. As described above, the images A and B are used to detect a relatively fast moving object, and the images A and C are used to detect a relatively slow moving object. When T1 is smaller than T2, the moving object is detected. When the speed is very low, the moving component may not be obtained from the difference image between the images A and C. It is preferable that the time interval T2 between the images A and B is set small enough to detect a very high-speed moving object (for example, as described above for four periods of the VD signal), but T1 is set smaller than T2. Then, a slow moving object cannot be detected.
[0023]
Returning to FIG. 2, when a moving object is detected based on the logical sum image F in step S4, the process proceeds to step S5, in which the actuators 10 and 12 of the camera 4 are controlled so that the moving object comes to the center of the camera image. To For example, as shown in FIG. 5A, when the moving object 24 comes to a position closer to the edge of the screen 26 than to a fixed area 28 near the center of the camera screen 26, the pan / tilt actuator 12 is controlled to move. The camera posture is adjusted so that the object 24 is at the center of the screen 26 'as shown in FIG. Further, the zoom actuator 10 is controlled to adjust the photographing magnification. In step S6, a still image of the moving object is recorded in the image storage unit 18.
[0024]
If no moving object is detected in step S4, that is, if all the pixels of the logical sum image F are “0”, the flow returns to step S2.
[0025]
When the patrol time at the current patrol position ends in step S2, the process returns to step S1, controls the field of view of the camera 4 to monitor the next monitoring area to be patrol, and repeats the operations in steps S2 to S6.
[0026]
According to the present embodiment, a moving object can be detected over a wide range from a low speed to a high speed. Regarding the distance from the camera, when the moving object moves relatively far, the moving object moves slowly on the imaging surface even if the moving speed is high, and the moving component is obtained by using images having the same acquisition interval as in the conventional example. Requires a large acquisition interval. However, with this setting, a moving object that moves relatively near the camera at a relatively high speed cannot be captured. On the other hand, by making the acquisition intervals different as in the present embodiment, it is possible to catch an object moving quickly in a distant place and an object moving fast in a near place. That is, according to the present embodiment, a moving object can be detected over a wide range in terms of a distance range.
[0027]
(Experiment)
The present inventors acquired two images at different photographing times under the following conditions, and experimented to determine whether a moving object can be detected based on a difference image.
[0028]
(1) Camera specification pixels: 320 pixels wide
Imaging pixel pitch: 0.0075mm
Focal length: 2.18mm
VD signal: 30 fps
(2) Moving object size: 500 mm in width perpendicular to the camera optical axis
Moving direction: linear motion in the direction perpendicular to the camera optical axis (lateral direction of the screen) Moving speed: 100 to 5000 mm / sec
Distance from camera to object: 250-10000mm
(In this range, the entire object having a width of 500 mm is projected onto three or more pixels on the imaging surface.)
[0029]
Results when two images are captured while changing the speed of the moving object and the distance from the camera (distance from the imaging surface) within the above-mentioned range for the image acquisition intervals of 4 periods and 31 periods of the VD signal. Are shown in FIGS. 6A and 6B, respectively. In the figure, a white portion indicates a case where discovery is possible, and a black portion indicates a case where it is impossible. A moving object can be found when the difference image has three or more moving components and an object having a width of 500 mm is picked up by three or more pixels in any of the two images.
[0030]
As shown in the figure, if the image acquisition interval is one, there is a combination of the distance from the camera and the moving speed that cannot be found. However, when the two results with different image acquisition intervals are combined, it can be seen that the moving object can be found in all combinations of the distance of the moving object from the camera and the moving speed within the above range. .
[0031]
By the way, in this embodiment, three images are acquired at different time intervals. However, in this method, when a noise component of the CCD sensor or another moving object reflected in the image other than the target object is detected. There is. In particular, when the monitoring area is set to a place where the background component easily changes, such as outdoors, there is a high possibility that a plurality of moving objects will be detected at the same time. In the above-described method, since all moving components appear in the image subjected to the logical sum processing when a plurality of moving objects are detected, the position of the target moving object on the image cannot be accurately identified from the logical sum image. As a result, steps S5 and S6 (FIG. 2) for storing the still image of the moving object cannot be performed.
[0032]
Labeling is known as a method for identifying a plurality of components in an image. However, labeling, which requires a large amount of calculation, is not suitable for a moving object detection method that needs to repeat image capture and processing at a high speed so as to be able to cope with a subject that moves to some extent.
[0033]
Hereinafter, a simple method for extracting a set having the largest area from a set of a plurality of difference components (“1” pixels) will be described with reference to the flowcharts of FIGS. .
[0034]
First, in step S71, an image Q as shown in FIG. 9 is obtained by performing a process of difference, binarization, and logical sum using three images as described with reference to FIG. In the figure, there are five sets of “1” pixels, which are referred to as P1, P2, P3, P4, and P5, respectively. In step S72, the number of “1” pixels in the image Q is projected and counted on the vertical axis and the horizontal axis of the image Q (that is, every time a “1” pixel is detected, the number of pixels in the vertical direction and the horizontal direction is counted). The count value of the corresponding position is incremented by one.) FIG. 10 shows a marginal distribution indicating the values thus aggregated. If the number of projected pixels is equal to or smaller than the threshold (th_h on the horizontal axis and th_v on the vertical axis) in step S73, it is regarded as noise and is not included in the subsequent processing (see FIG. 11). For example, a small area such as the set P5 or an area surrounded by a dotted line in FIG. 11 such as a part of the sets P1 and P4 is excluded from the processing target. Further, in step S74, a section in which the length of the peripheral distribution that continues in the axial direction is equal to or less than the threshold (le_h on the horizontal axis and le_v on the vertical axis) is also regarded as noise and not included in the subsequent processing. For example, in FIG. 11, among the sections L1, L2, L3, and L4, the section having the length L2 smaller than the threshold le_h is excluded from the processing target.
[0035]
In step S75, it is determined whether there is an area (that is, a rectangular area having a predetermined width in the vertical direction and the horizontal direction) extracted from the image by the processing using the two types of thresholds. If there is a cut-out area, the flow proceeds to step S76. Referring to FIG. 12, a region R1 having a horizontal width L1 and a vertical width L4 and a region R2 having a horizontal width L1 and a vertical width L3 are cut out. If there is no region cut out in step S75, it is determined that there is no moving object, and the flow ends.
[0036]
In step S76, it is determined whether there are a plurality of cut-out areas. If there is more than one, the flow goes to step S77. In the example of the figure, there are two regions, R1 and R2. The region R1 including the sets P1 and P2 is cut out as a rectangular region continuous by a predetermined length in the vertical direction and the horizontal direction. This is because there was a continuous component in the horizontal direction of the set P3 when axial projection was performed. is there. In such a case, it is necessary to perform a process of dividing the sets P1 and P2 as described later.
[0037]
At step S77, the count of "1" pixels included in each area (two regions in the example of FIG R1, R2), determine the area R _m having the maximum. R1 is determined as _{R m} in the example of FIG.
[0038]
In step S78, counted by projecting the number of "1" pixel in the region R _m on the vertical and horizontal axes of the region R _m, determine the marginal distribution. Since the noise component has been removed in the previous processing, the processing using the thresholds such as th_h, th_v, le_h, and le_v described above is not performed on the obtained marginal distribution.
[0039]
If there is only one region cut out in step S76, the flow proceeds to step S79. An area cut out in step S79 as _{R m,} the process proceeds to step S78.
[0040]
In step S81, it is determined whether or not the marginal distribution is divided into sections. If divided, the process proceeds to step S82, the from region R _m, cut out the sub-region as a rectangular region having a machine direction and a transverse direction in the section. In the illustrated example, from the region R1, the lateral width L5, the subregion _{r 1} of the vertical width L4, the width L6, a subregion _{r 2} of the longitudinal width L4 is cut out (see FIG. 13).
[0041]
In step S83, the count of "1" pixels included each of the plurality of sub-areas, determining the sub-region r _m with the maximum. _{R 1} is calculated in two sub regions _r 1, _{r 2} as _{r m} in the example of FIG.
[0042]
In step S84, the projection of the sub-region r _{m (r 1} in the example of FIG containing the set _P1) to the original image Q (i.e., converts the sub-region r _m the coordinates of the original image Q), final Then, an extracted image H as shown in FIG. 14 is obtained, and the flow ends. The extracted image determines the position of the sub-region r _m on the image is used to acquire a still image of a moving object of interest records perform posture control and the imaging magnification adjustment of the camera.
[0043]
In step S81, if the marginal distribution is not divided (i.e., if the set of "1" pixel is only one in the region R _m), the flow proceeds to step S85. In step S85, the acquired extracted image by projecting a region R _m to the original image Q, the flow ends.
[0044]
The above description relates to one embodiment of the present invention, and the present invention is not limited to this, and can be variously modified. For example, in the embodiment, the moving object is detected based on the difference between the newest image and the oldest image in time series, and the difference between the newest image and the second newest image. The present invention is not limited to which two of them are used as a set to determine two sets of difference images. However, when obtaining a difference image from the oldest image and the second newest image, even if the difference image includes a moving component, it is at least the most necessary to start controlling the camera to obtain a still image of the moving object. Since it is after acquiring a new image, there is a possibility that the moving object may be out of the angle of view of the camera when controlling the camera. Therefore, it is preferable to use the newest image for any set as in the above embodiment. .
[0045]
【The invention's effect】
According to the present invention, in a method of detecting a moving object in a predetermined area (for example, a monitoring area) by processing the captured image data by capturing an image of the predetermined area (for example, a monitoring area), the object speed and the distance from the imaging surface are determined. Thus, a moving object can be detected over a wide range.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a surveillance camera system using one embodiment of a moving object detection method according to the present invention.
FIG. 2 is a flowchart showing an entire monitoring process using the monitoring camera system of FIG. 1;
FIG. 3 is a flowchart showing a subroutine of image processing in step S3 of FIG. 2;
FIG. 4 is a view showing three images, a difference image, and a logical sum image used in the image processing of FIG. 3;
FIG. 5 is a view for explaining camera view control in step S5 of FIG. 2;
FIG. 6 shows whether or not a moving object can be detected when two images are captured while changing the speed of the moving object and the distance from the camera with respect to an image acquisition interval of 4 periods and 31 periods of a VD signal. FIG.
FIG. 7 is a flowchart showing a first part of a process for extracting a specific moving component set.
FIG. 8 is a flowchart showing a second part of the extraction process of a specific moving component set.
FIG. 9 is a view showing a logical sum image obtained in step S71 of FIG. 7;
FIG. 10 is a diagram showing a marginal distribution obtained in step S72 of FIG. 7;
FIG. 11 is a diagram for explaining noise removal in steps S73 and S74 of FIG. 7;
FIG. 12 is a diagram showing a cut-out area in step S75 of FIG. 7;
FIG. 13 is a view showing a marginal distribution obtained in step S78 in FIG. 7 and a sub-region having the maximum number of “1” pixels obtained in step S83 in FIG. 8;
FIG. 14 is a view showing an extracted image obtained in step S84 in FIG. 8;
[Explanation of symbols]
2: surveillance camera system, 4: camera device (imaging device), 6: computer, 8: imaging unit, 10: zoom actuator, 12: pan / tilt actuator.

Claims (4)

In a method of imaging a predetermined region with an imaging device and processing a captured image data to detect a moving object in the region,
Obtaining an image group having first, second and third images, which are first, second and third images in chronological order,
Selecting a set of two images from the group of images to obtain a first difference image; and selecting a set of images different from the selected set of two images from the group of images to obtain a second difference image. Obtaining an image;
A detection method characterized in that a time interval T1 from obtaining a first image to obtaining a second image is different from a time interval T2 from obtaining a second image to obtaining a third image. . 2. The method according to claim 1, wherein the first difference image is a difference image between the first image and the third image, and the second difference image is a difference image between the second image and the third image. Detection method. 3. The method according to claim 2, wherein the time interval T1 is larger than the time interval T2. The detection method according to claim 1, further comprising a step of performing a binarization process on each of the first and second difference images.

Images