JP2005339276A - Method, program and device for detection of moving body - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a program, method and device for detection of moving body strong to flicker or change in environmental light and capable of sharply detecting a moving body of a color close to the background. <P>SOLUTION: The method comprises executing a first processing for deriving, for each pixel α of a plurality of frame images taken at a plurality of frame times, a two-point contrast C between each pixel α and another pixel β having a predetermined positional relation with this pixel α; a second processing for deriving, with respect to each pixel α, a difference D between a two-point contrast C derived for one frame among the plurality of frame images and a two-point contrast C derived for the other frame image at a frame time preceding the one frame image; and a third processing for determining, in case that the difference D is a predetermined value or more for any one pixel α, that this pixel α is changed, and recording information for this pixel α to a recording device. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a program, a method, and an apparatus for detecting a moving object in an image captured by an imaging apparatus such as a CCD camera. In particular, the present invention is robust to flicker and changes in ambient light, or approximates the background. The present invention relates to a moving object detection program, method, and apparatus capable of sensitively detecting a moving object of color.

  There are a background difference method and an inter-frame difference method as known methods for detecting a moving object such as a person or a car from an image taken by an imaging means such as a CCD camera.

  The background subtraction method detects a difference between a target image and a background image prepared in advance as a moving object. However, in an environment where changes in sunshine, movement of a background object, etc. occur, every time the background changes Since it is necessary to replace the image, there is a problem that a large amount of background images must be prepared when the background changes in various ways, and the method of replacing the background image at an appropriate timing is unresolved. It remains as an issue.

  The inter-frame difference method updates the image of the immediately previous frame (or an image before a predetermined time) as a background image one after another, and detects a moving object based on the difference between the background image and the target image (current image). Although the above problem in the background subtraction method is solved, there is a problem that a slow image change cannot be detected. For example, if a person in an image that is a moving object is stationary for a certain time, Even the video portion is regarded as the background, and there is a problem that an erroneous detection result that the person disappears or appears even though the person is reflected in each frame image is output.

Furthermore, both of the above methods have a problem that when the colors of the background and the moving object are approximate, the moving object cannot be detected, and the flickering of the fluorescent lamp or the flicker due to the imaging quality is likely to occur.
Junya Morita, Yoshio Iwai, Masahiko Taniuchi, "Estimation of background images and removal of shadows in the room", Transactions of Information Processing Society of Japan, Vol. 44, SIG8 (2003) Isao Baba, Ken Ohashi, Manji Oi, Hideaki Matsuo, Toshiro Ejima, “Person Tracking Based on Interframe Difference Method”, Image Sensing Symposium (2000)

  An object of the present invention is to provide a moving object detection program, method, and apparatus that solve problems in the conventional background subtraction method and interframe subtraction method and are robust against changes in sunlight and flicker.

  Another object of the present invention is to provide a moving object detection program, method, and apparatus capable of more sensitively detecting a moving object having a color similar to the background.

  Another object of the present invention is to provide a moving object detection program, method, and apparatus that can adjust how fast an object that moves is detected as a moving object.

The present invention solves the above problems,
A moving object detection program for detecting a moving object in a frame image that is imaged at each frame time by an imaging device, the processing device regarding each pixel α of a plurality of frame images imaged at a plurality of frame times A first process for deriving a contrast C between two points of each pixel α and another pixel β having a predetermined positional relationship with each pixel α, and one frame image among the plurality of frame images A difference D between the derived point-to-point contrast C and the point-to-point contrast C derived for another frame image captured in the frame time preceding the one frame image is derived for each pixel α. In the second process, when the difference D is greater than or equal to a predetermined value for any pixel α, it is assumed that there is a change in the pixel α. A moving object detection program for executing a third process of recording information relating to the element α in a recording device, and a movement for detecting a moving object in a frame image captured every frame time by the imaging device An object detection method, for each pixel α of a plurality of frame images captured in a plurality of frame times, between the two points of the pixel α and another pixel β having a predetermined positional relationship with the pixel α A first step of deriving a contrast C, the two-point contrast C derived for one frame image among the plurality of frame images, and other images captured in a frame time preceding the one frame image A second step of deriving the difference D of the contrast C between the two points derived for the frame image for each pixel α and any one of the pixels α. A moving object comprising: a third step of recording information relating to the pixel α in a recording device on the assumption that there is a change in the pixel α when the difference D is equal to or greater than a predetermined value. A detection method or a moving object detection device that detects a moving object in a frame image captured every frame time by an imaging device, and for each pixel α of a plurality of frame images captured in a plurality of frame times, A point-to-point contrast deriving means for deriving a point-to-point contrast C between each pixel α and another pixel β having a predetermined positional relationship with each pixel α, and one frame image among the plurality of frame images The point-to-point contrast C derived for the previous frame image and the other frame image captured in the frame time preceding the one frame image. Difference derivation means for deriving the difference D of the contrast C between the two points for each pixel α, and when the difference D is greater than or equal to a predetermined value for any pixel α, there is a change in the pixel α As a moving object detection apparatus, the apparatus includes a change pixel information recording unit that records information on the pixel α in the recording apparatus.

  That is, the present invention uses a contrast between two points that can reduce the influence of changes in ambient light, and changes to the image when the difference D between the frame images before and after the contrast between the two points is a predetermined value or more. Therefore, it is possible to accurately detect a moving object even when flicker or sunshine changes occur.

As such a point-to-point contrast, when the pixel value of each pixel α is g (α) and the pixel value of another pixel β having a predetermined positional relationship with the pixel α is g (β), The point-to-point contrast C (α, β) represented by (1) can be used.
C (α, β) = (g (α) −g (β)) / (| g (α) | + | g (β) |) (1)

  Further, since the contrast between the two points is derived from the two pixels α and β having a predetermined positional relationship, the moving object is detected by reflecting information on the texture (pattern). Therefore, even if the color of the moving object approximates the background, it is possible to detect the moving object sensitively if there is a difference in pattern or texture between the two.

  The change in the pixel α is detected when the average of the difference D in the image is X and the variance X is used as a predetermined value X + σ, and the absolute value of the difference D is larger than X + σ. Is possible.

  In addition, the information about the pixel α in which a change is detected, which is recorded in the recording apparatus in the present invention, can be, for example, information for specifying the pixel α, and thereby the shape of the moving object from each frame image. And the size can be extracted.

  For the pixel α in which the change is detected, the pixel value of the pixel α in the one frame image is recorded in the recording device. For the pixel in which no change is detected, for example, the pixel value corresponding to white (R = 255, G = 255, B = 255), and the like, if a pixel value corresponding to a specific color is recorded in the recording device, an image obtained by extracting a moving image from each frame image can be generated. It becomes possible.

  Further, the pixel value used for deriving the contrast between two points of the present invention can be any value such as the RGB value of each pixel, but it is particularly preferable to use the brightness and the color difference. .

The lightness and color difference are known in various definitions depending on the purpose. In the present invention, the lightness y and the color differences r and b defined by the following formulas (2) to (4) are used. It is particularly preferable to do this.
y = 0.299R + 0.587G + 0.144B (2)
r = 0.500R + 0.419G−0.081B (3)
b = −0.160R−0.332G + 0.500B (4)

  In the present invention, three types of two-point contrast are derived for the brightness y and the two color differences r and b, or from each pixel α and four different pixels β1 to β4 adjacent to the pixel α. Deriving four types of contrast between two points, or summing each pixel α and four different pixels β1 to β4 adjacent to the pixel α for each of the brightness y and the two color differences r and b. It is possible to derive twelve types of contrast between two points and to record information on a pixel in which a predetermined number of differences D for each two-point contrast is equal to or greater than a predetermined value on a recording apparatus. With this, it becomes possible to realize more precise moving object detection.

  Further, in the present invention, with respect to the point-to-point contrast C derived for one frame image, a plurality of 2 derived for a plurality of other frame images captured in a plurality of frame times preceding the one frame image. Each difference D of the point-to-point contrast C is derived, and for any pixel α, when a predetermined number of the differences D derived for the plurality of other frame images is equal to or greater than a predetermined value, It is also possible to determine that there is a change in the pixel α, and by adjusting the number of other frame images to be used and a predetermined number according to the frame time, at what speed It is possible to adjust whether a moving object is detected as a moving object.

  DESCRIPTION OF EMBODIMENTS Hereinafter, an embodiment of a program for executing a moving object detection method of the present invention and a moving object detection device will be described with reference to the accompanying drawings.

  FIG. 1 is an explanatory diagram illustrating an exemplary configuration of a moving object detection device of the present invention or an exemplary configuration of a device suitable for executing the method and program of the present invention.

  As shown in the figure, the moving object detection device 10 includes a processing device (CPU) 12, a ROM 13, a RAM 14, a hard disk 15 and other recording devices connected to each other via a bus line 11, information such as a CCD camera 16, an LCD and a CRT. A display device 17 and an operation input device 18 such as a keyboard and a mouse are provided.

  Here, the processing device 11 executes the moving object detection program of the present invention recorded on the hard disk 15 to thereby perform a contrast calculation means 12a between two points, a difference derivation means 12b, a changed pixel information recording means 12c, a moving object image. Functional blocks such as the generation unit 12d are realized.

  The CCD camera 16 captures an image with a predetermined resolution (for example, 100 dpi) every predetermined frame time (for example, 10 seconds), and the image is recorded in the image recording unit of the recording apparatus for each frame. It is recorded in 15a.

  The moving object detection apparatus 10 operates in the following manner.

  The point-to-point contrast deriving means 12a calculates the point-to-point contrast value C in the following procedure for all the pixels of each newly captured frame image.

First, three pixel values g of lightness y, color differences r, and b are calculated from the RGB values of all the pixels according to the following formulas (5) to (7).
y = 0.299R + 0.587G + 0.144B (5)
r = 0.500R + 0.419G−0.081B (6)
b = −0.160R−0.332G + 0.500B (7)

Further, the contrast between two points for each pixel value g of each pixel is derived according to the following equation (8) for the four patterns shown in FIG.
Cn = (g (α) −g (βn)) / (| g (α) | + | g (βn) |)
However, n = 1 to 4 (8)

  Therefore, for each pixel, 12 types (= 3 pixel values × 4 patterns) of the two-point contrast C are derived, and the derived two-point contrast is calculated for each pixel of each frame. Are recorded in the point-to-point contrast recording unit 15b of the recording apparatus.

Further, the difference deriving means 12b is configured such that the two-point contrast C (F0) for the latest frame image F0 and the two-point contrast C (F1), C ( From F2), 12 types of differences D (F1) and D (F2) are derived for each pixel according to the following equation (9).
D (Fm) = | C (F0) −C (Fm) |
However, m = 1, 2 (9)

  At the same time, for each of the differences D (F1) and D (F2), the average X (for all the pixels in the image) and the variance σ in the image plane are derived, and the derived difference D is derived. , X and variance σ are recorded in the difference recording unit 15c of the recording apparatus.

  The change pixel information recording unit 12c compares the differences D (F1) and D (F2) with the sum of the mean X and the variance σ derived for the differences D (F1) and D (F2), It is determined whether the differences D (F1) and D (F2) exceed the sum of the average X and the variance σ.

For each pixel, the change pixel information recording unit 12c applies the determination result to the following conditions A and B, and when both the conditions A and B are satisfied, the change has occurred in the pixel ( Assuming that the moving object was photographed on the pixel), the information (coordinate value) for identifying the pixel and the RGB value of the pixel in the frame image F0 are recorded in the change pixel information recording unit 15d. When either one of B is not established, the information (coordinate value) for specifying the pixel is assumed that no change has occurred in the pixel (the moving object has not been photographed in the pixel). , And RGB values (R = 255, G = 255, B = 255) corresponding to white are recorded in the change pixel information recording unit 15d.
Condition A: There are 6 or more differences D (F1) exceeding the sum of the average X and the variance σ.
Condition B: There are 6 or more differences D (F2) exceeding the sum of the average X and the variance σ.

  The moving object image generating means 12d displays the moving object as an original color image based on the information specifying each pixel in the change pixel information recording unit 15d and its RGB value, and the background part other than the moving object is displayed. A moving object detection image in a white state is generated and displayed on the information display device 15.

  3-6 is explanatory drawing which compares and shows the moving object detection image produced | generated by the said moving object detection apparatus, and the moving object detection image produced | generated by the conventional background difference method, (A) is an image used as a frame image F1 or F2 (an image used as a background image in the background difference method), and (b) is an image used as a frame image F0 (an object in the background difference method). (Image used as an image), (c) shows a moving object detection image generated by the background subtraction method, and (d) shows a moving object detection image generated by the moving object detection device of the present invention.

  As is apparent from FIGS. 3 to 6, in the background subtraction method, there are many portions that fail to detect the portion of the clothes that approximates the background ground color, and there is also a lot of noise that seems to be caused by flicker. On the other hand, in the moving object detection image obtained by the present invention, the portion of the clothes that is close to the ground color is detected sharply, and the image has less noise.

  Although one embodiment of the present invention has been described above, the present invention can be variously modified within the scope described in the claims.

  For example, in the above-described embodiment, brightness and two independent color differences are used as pixel values. However, only one or two of them are used, or instead of these pixel values, or In addition to these pixel values, other pixel values such as RGB values can also be used.

  In the above-described embodiment, the pixel values of all independent adjacent pixels (β1 to β4) are used for deriving the contrast between two points, but only the pixel values of some adjacent pixels are used, or It is also possible to use pixel values of pixels other than adjacent pixels.

  Further, in the above-described embodiment, it is determined whether or not the difference D is larger than the sum of the average X of the contrast between the two points having the predetermined value and the variance σ, and the difference D larger than the predetermined value is a predetermined number. Although the case where the presence or absence of a change in a pixel is determined based on whether or not there are six or more pixels has been described, the selection of a predetermined value or a predetermined number is arbitrary, and the accuracy of moving object detection can be adjusted by adjusting these values. Can be adjusted.

  Further, in the above-described embodiment, the case where the two frame images F1 and F2 preceding the one frame image F0 are used as the comparison target images has been described. However, the comparison target frame images may be single. It is also possible to use three or more preceding frame images as comparison target frame images.

  In addition, when the number of frame images used as comparison targets and the number of comparison target frame images are more than a predetermined number are satisfied according to the frame time of the imaging device. It is possible to adjust at what speed the moving object is detected as a moving object depending on whether or not it is determined that the pixel has changed.

  In the above-described embodiment, for the pixel that has changed, the information about the pixel and the pixel value of the pixel are recorded in the changed pixel information recording unit 15d. However, the information about the pixel In this case, it is possible to generate an image showing the shape of the moving object.

  In the above-described embodiment, the case has been described in which information about pixels that have not changed is also recorded in the changed pixel information recording unit 15d. However, it is also possible not to record information regarding this.

  INDUSTRIAL APPLICABILITY The present invention can be used for observing the behavior of visitors at an article store, an exhibition hall of an article including an art gallery or a museum, a service providing base such as a game center, and what is attracting attention from time to time. It can be used for grasping the needs of visitors by conducting dynamic survey behaviors.

  Similarly, the present invention can be used for observing the behavior of a person at school, office, home, etc., and by observing the behavior of a specific individual, the needs of the individual are grasped and a service that meets the needs. Can be applied to a comfortable living environment system.

  In addition, the present invention can be used in a monitoring system for the purpose of security or crime prevention. For example, when a suspicious person, which is a moving object, is detected, an alarm or notification is given, and the imaging resolution or imaging It is possible to realize a monitoring system having functions such as increasing the speed or automatically performing zoom shooting of a suspicious person based on pixel information in which movement is detected.

  In addition, the present invention can be used for an observation system and an observation system for an arbitrary moving object and an object whose form changes, such as a traffic and traffic monitoring system for detecting a car and an observation system for observing the ecology and growth of a living organism. Is possible.

Explanatory drawing which shows the structure of the moving object detection apparatus which concerns on one embodiment of this invention. Explanatory drawing which shows the positional relationship of two pixels used for derivation | leading-out of the contrast between two points in one Embodiment of this invention. Explanatory drawing which compares and shows the moving object detection image produced | generated by the moving object detection apparatus of this invention, and the moving object detection image produced | generated by the background difference method. Explanatory drawing which compares and shows the moving object detection image produced | generated by the moving object detection apparatus of this invention, and the moving object detection image produced | generated by the background difference method. Explanatory drawing which compares and shows the moving object detection image produced | generated by the moving object detection apparatus of this invention, and the moving object detection image produced | generated by the background difference method. Explanatory drawing which compares and shows the moving object detection image produced | generated by the moving object detection apparatus of this invention, and the moving object detection image produced | generated by the background difference method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving object detection apparatus 11 Bus line 12 Processing apparatus 12a Two-point contrast deriving means 12b Difference deriving means 12c Changed pixel information recording means 12d Moving object image generating means 13 ROM
14 RAM
15 Hard disk 15a Image storage unit 15b Contrast recording unit between two points 15c Difference recording unit 15d Changed pixel information recording unit 16 CCD camera 17 Information display device 18 Operation input device

Claims (8)

A moving object detection program for detecting a moving object in a frame image captured every frame time by an imaging device, the processing device,
For each pixel α of a plurality of frame images imaged in a plurality of frame times, a first point C for deriving a contrast C between two points of the pixel α and another pixel β having a predetermined positional relationship with the pixel α And processing
Among the plurality of frame images, the point-to-point contrast C derived for one frame image and the point-to-point derived for another frame image captured in a frame time preceding the one frame image. A second process for deriving a difference D of contrast C for each pixel α;
When the difference D is greater than or equal to a predetermined value for any pixel α, it is assumed that there has been a change in the pixel α, and a third process for recording information relating to the pixel α in the recording apparatus is executed. A moving object detection program characterized by that. The point-to-point contrast C is such that the pixel value of each pixel α is g (α), and the pixel value of another pixel β having a predetermined positional relationship with the pixel α is g (β).
C (α, β) = (g (α) −g (β)) / (| g (α) | + | g (β) |) (1)
The moving object detection program according to claim 1, wherein the moving object detection program is derived as follows. In the first process, for each pixel α, four types of the two-point contrasts C1 to C4 are derived between the pixel α and four different pixels β1 to β4 adjacent to the pixel α.
In the second process, the two-point contrasts C1 to C4 derived for the one frame image and the corresponding two-point contrasts C1 to C4 derived for the other frame image. The differences D1-D4 are derived for each of the pixels α,
When a predetermined number of the differences D1 to D4 is greater than or equal to a predetermined value for any pixel α, information regarding the pixel α is recorded in the recording device as a change in the pixel α. The moving object detection program according to claim 1, wherein the third process is executed. In the first process, the contrast y between two points Cy, Cr, and Cb is derived with respect to lightness y that is three types of pixel values and two color differences r and b,
In the second processing, the contrast between the two points Cy, Cr, Cb derived for the one frame image and the corresponding point-to-point contrast Cy, Cr, Cb derived for the other frame image. The differences Dy, Dr, Db of Cb are derived for each of the pixels α,
When a predetermined number of the differences Dy, Dr, and Db is greater than or equal to a predetermined value for any pixel α, information regarding the pixel α is assumed to have changed in the pixel α. The moving object detection program according to any one of claims 1 to 3, wherein a third process to be recorded is executed. In the second processing, among the plurality of frame images, a plurality of other images captured at a plurality of frame times preceding the one frame image with a contrast C between two points derived for the one frame image. A difference D for each two-point contrast C derived for the frame image is derived,
For any pixel α, when a predetermined number of the differences D derived from the plurality of other frame images is greater than or equal to a predetermined value, it is assumed that there is a change in the pixel α. The moving object detection program according to any one of claims 1 to 4, wherein a third process of recording information about α in the recording device is executed.   For the pixel α that has been changed in the third process, the pixel value of the pixel α in the one frame image is recorded in the recording device, and the pixel that has not been changed. 6. The moving object detection program according to claim 1, wherein a pixel value corresponding to a specific color is recorded in the recording device for α. A moving object detection method for detecting a moving object in a frame image captured every frame time by an imaging device,
For each pixel α of a plurality of frame images imaged in a plurality of frame times, a first point C for deriving a contrast C between two points of the pixel α and another pixel β having a predetermined positional relationship with the pixel α And the steps
Among the plurality of frame images, the point-to-point contrast C derived for one frame image and the point-to-point derived for another frame image captured in a frame time preceding the one frame image. A second step of deriving a difference D of contrast C for each pixel α;
A third step of recording information relating to the pixel α in the recording device on the assumption that there is a change in the pixel α when the difference D is greater than or equal to a predetermined value for any pixel α. The moving object detection method characterized by this. A moving object detection device that detects a moving object in a frame image captured every frame time by an imaging device,
Two points for deriving a contrast C between two points of each pixel α and another pixel β having a predetermined positional relationship with each pixel α with respect to each pixel α of a plurality of frame images captured in a plurality of frame times. Inter-contrast deriving means,
Among the plurality of frame images, the point-to-point contrast C derived for one frame image and the point-to-point derived for another frame image captured in a frame time preceding the one frame image. Difference deriving means for deriving the difference D of the contrast C for each pixel α;
When the difference D is greater than or equal to a predetermined value for any of the pixels α, it is assumed that there has been a change in the pixel α, and there is a change pixel information recording unit that records information about the pixel α in the recording device. The moving object detection apparatus characterized by the above-mentioned.

Images