JP4268953B2 - Motion detection apparatus and program thereof - Google Patents

Description

  The present invention relates to a motion detection device and a program for detecting a motion region in a video.

  As the term ubiquitous is commonly used, an environment where computers can be used in various situations of daily life is being prepared. In order to build an environment in which such computers are ubiquitous and support users' real-world work, in particular, technology for detecting and recognizing real-time situations and conditions such as movements in the real world is required. It has been.

  In view of these problems, a technique for detecting motion in a video using image information has been provided. As these conventional techniques, a method based on image difference is known. A particular problem here is that the luminance difference and chromaticity signal difference between the moving object and the background change due to factors such as the shooting environment.

  For example, even with the same moving object and background, if the intensity of ambient light (illumination) increases, the brightness difference between the moving object and the background increases, and the intensity of ambient light (illumination) decreases. The brightness difference between the moving object and the background is reduced. This causes erroneous detection of a moving object. As such an example, an example of trying to solve a problem in the case where a shadow or the like is included in the detection target region is known (see, for example, Patent Document 1). In the apparatus described in this document, by taking into account images with low appearance frequency such as shadows that move with moving objects and specular reflection images due to the floor surface, specular reflection images due to shadows and floor surfaces in the detection target area, belongings, etc. It is said that it can cope with accidental deformation caused by

However, in the above conventional technology, only changes such as shadows and specular reflections due to the movement of an object are taken into account, and the input is made depending on the illumination conditions of the environment in which the image is captured and the photoelectric conversion characteristics of the camera used for the image capture. When the luminance or chrominance signal difference changes, the motion area in the video cannot be detected correctly. In recent years, the performance of a relatively inexpensive and easy-to-use camera such as a WEB camera has not been so high, and such a defect makes it very difficult to detect a motion region in a real environment.
JP 2004-240997 A "Moving Object Extraction Device"

  The present invention has been made in view of the above-described problems of the prior art, and is a motion detection device for detecting a motion region in a video, particularly in shooting conditions, shooting environment, camera photoelectric conversion characteristics, and time direction. An object of the present invention is to provide a motion detection device and a program robust to noise.

  In order to achieve this object, a motion detection device according to the present invention inputs video including captured motion, stores video data in a memory, and a plurality of video data stored in the video input device. Motion detection means for generating a motion pattern image from a group of frame images, motion information calculation means for calculating motion information including a motion region or a motion direction from the motion pattern image generated by the motion detection means, and the motion information Motion information output means for outputting a motion information calculation result in the calculation means in correspondence with each frame image input by the video input means.

In a preferred aspect of the above motion detection means, means for reading out, from the memory, frame images corresponding to the number of detection target frames set in advance from the latest one among a plurality of frame image groups stored in the video input means. And means for sequentially obtaining pixel values at the same pixel position of all frame images, means for logarithmically converting each of the obtained pixel values, and time from transition of the converted values for all pixel positions of the acquired frame image. Means for calculating the differential value, means for setting the order number of the frame image from which the pixel value is obtained when the calculated time differential value satisfies a threshold value, and evaluation of all pixel positions Means for generating a motion pattern image having a pixel value as a value. According to this aspect, by using the time differential value of the logarithmic conversion of the pixel value as a determination target, it is possible to detect motion robustly in the imaging conditions, the imaging environment, and the photoelectric conversion characteristics of the camera. Furthermore, noise tolerance in the time direction can be improved by targeting a plurality of frame images from the latest one.

In another preferred aspect of the motion detection means, frame images corresponding to the number of detection target frames set in advance from the latest one among a plurality of frame image groups stored in the video input means are stored from the memory. A means for reading out, a means for sequentially obtaining pixel values at the same pixel position of all frame images for all pixel positions of the obtained frame image, a pixel value of the pixel of the obtained frame image, and a corresponding frame image immediately thereafter Among the pixel values of a pixel, a means for obtaining a ratio of a larger pixel value to a smaller pixel value, a means for dividing the obtained ratio value by a time interval between frames, and the divided value satisfies a threshold value In addition, a means for setting the order number of the frame image from which the pixel value has been acquired as an evaluation value for the pixel position and a motion pattern image having the evaluation value for all pixel positions as the pixel value are generated. And a means. According to this aspect, the same effect can be obtained even if the evaluation value is obtained based on the result of dividing the pixel value ratio by the time interval between frames without using the logarithmic conversion of the pixel value.

  In another preferred mode of the motion information calculation means, for all pixels to which the pixel value of the input motion pattern image is assigned, the pixel value is set to the pixel to which the pixel value is assigned among the adjacent 8 pixels. Means for calculating the decreasing direction, and means for calculating the motion direction of each motion region by taking the average of the directions for all the calculated pixels. According to this aspect, the movement direction with respect to time can be calculated.

  As described above, according to the present invention, according to the above-described configuration, a motion region in a video can be detected, and it is related to shooting conditions, shooting environment, noise caused by camera photoelectric conversion characteristics, and time direction. It is possible to provide a device and a program that are robust against noise.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a functional block diagram of a motion detection device according to the present embodiment. As shown in FIG. 1, the apparatus includes a video input unit 1, a motion detection unit 2, a motion information calculation unit 3, and a motion information output unit 4.

  In the video input unit 1, a video including captured motion is input and each frame image is stored in the memory 5. The input image uses a camera using a CCD as an image sensor. The motion information output unit 4 adds and outputs motion information to each frame image stored in the video input unit 1 based on the motion information calculation result in the motion information calculation unit 3. The motion information is not particularly limited as long as it is information including a motion region and a motion direction in each frame image in the target video, and the motion pattern generated by the motion detection unit 2 of the present apparatus. Information such as images and information obtained by normal image processing can be included.

  FIG. 2 shows a block configuration diagram of the motion detector 2. The frame image reading unit 11 reads frame images for the number of detection target frames set in advance from the latest one from the memory. The pixel value acquisition unit 12 acquires the pixel values at the same pixel position of all frame images in order from the latest one for all pixel positions of the acquired frame image. The logarithmic conversion unit 13 performs logarithmic conversion on each acquired pixel value. The time derivative calculation unit 14 calculates a time derivative value from the transition of the converted value. When the calculated time differential value satisfies the threshold value, the evaluation value setting unit 15 sets the order number of the frame image from which the pixel value is acquired as the evaluation value of the pixel position. The pattern image generation unit 16 generates a motion pattern image having the evaluation values at all pixel positions as pixel values. In the motion detection unit 2 configured as described above, a motion pattern image is generated from a plurality of frame image groups stored in the video input unit.

  FIG. 3 is a flowchart showing the operation of the motion detector 2 shown in the block configuration of FIG. Hereinafter, it demonstrates in detail according to this flow.

  [Step 1] A preset number N of detection target frames is acquired.

  [Step 2] A time interval (frame rate) t between frames of the input video is acquired.

  [Step 3] Frame images corresponding to the number of frames are read from the memory as I (1), I (2),.

[Step 4] The pixel values p (x, y, n) at the same pixel position (x, y) of all frame images I (n) are acquired in order from the latest. P = {p (x, y, 1), p (x, y, 2), ..., p (x, y, N)}
[Step 5] Each acquired pixel value is logarithmically converted. logP = {log (p (x, y, 1)), log (p (x, y, 2)), ..., log (p (x, y, N))}
[Step 6] A differential value is calculated from the transition of the converted value.

d (logP) / dt = {{log (p (x, y, 1))-log (p (x, y, 2))}
/T,...,{log(p(x,y,N-1))-log(p(x,y,N))}/t}
[Step 7] When the calculated differential value satisfies the threshold value, that is, when the absolute value of the calculated differential value is equal to or greater than the predetermined threshold value, the number of the order of the frame image from which the pixel value is acquired is set to the pixel position. The evaluation value of

  [Step 8] The processing of [Step 4] to [Step 7] is repeated for all pixel positions of the acquired frame image.

  [Step 9] A motion pattern image is generated with the evaluation values at all pixel positions as pixel values.

  At this time, in [Step 7], the latest items are inspected in order, an evaluation value is set when the threshold value is satisfied, and the process proceeds to [Step 8]. That is, even when the threshold value is satisfied for a plurality of frame images, the smallest one of the order numbers of the frame images is set as the evaluation value. If the threshold is not satisfied, the evaluation value is set to 0. In addition, the threshold value to be used may be determined in advance, or may be dynamically calculated using a statistically calculated value such as an average value, a frequency, a deviation value, and the like. Since the method is not particularly different from the conventional method, detailed description thereof is omitted here.

  A supplementary explanation of the motion detection unit 2 that performs the above processing will be given. In general, the value of each pixel in an image can be expressed as a product of the intensity of ambient light (illumination), the reflectance of a subject, and a coefficient relating to photoelectric conversion characteristics (efficiency) of the camera. Here, when the time interval between the frames of the target video is short, the coefficient relating to the intensity of the ambient light and the photoelectric conversion characteristic of the camera is substantially constant between a certain frame image and the immediately following frame image. Can be considered. Therefore, the environment included in each pixel value is obtained by subtracting the value obtained by logarithmically converting the pixel value of the corresponding pixel of the immediately subsequent frame image from the value obtained by logarithmically converting the pixel value of the pixel of the acquired frame image. The coefficient relating to the light intensity and the photoelectric conversion characteristic of the camera is offset, and only the change in the reflectance of the subject due to the movement of the object can be detected based on the magnitude of the absolute value of the subtraction result.

  Further, when a target video is input via a network, a frame image may be lost due to a delay in the transmission system or the like. Therefore, in this embodiment, every time the target video is input in Step 1, the time interval t between frames of the input video is acquired in Step 2. Then, by subtracting the result of logarithmically converting the pixel value by the time interval t between the frames, the change in the reflectance of the subject per unit time is obtained as the time derivative of the value obtained by logarithmically converting the pixel value. (Steps 5 and 6). Accordingly, in step 7, a slight change in the reflectance of the subject that is originally equal to or less than the threshold is prevented from being erroneously detected as a large change due to a missing frame image.

  Thereby, in the present embodiment, it is possible to provide a device that is robust against noise caused by shooting conditions, shooting environment, camera photoelectric conversion characteristics, and noise in the time direction.

  In FIG. 2 or FIG. 3, the evaluation value is obtained based on time differentiation of a value obtained by logarithmically converting the pixel value. However, without using logarithmic transformation, the ratio of the larger pixel value to the smaller pixel value of the pixel value of the pixel of the acquired frame image and the pixel value of the corresponding pixel of the immediately following frame image is obtained. It is obvious that the same result can be obtained even if the evaluation value is obtained based on the result of dividing the obtained ratio value by the time interval t between frames. In this case, when the result of dividing by the time interval t is equal to or greater than a predetermined threshold, the order number of the frame image from which the pixel value is acquired may be used as the evaluation value of the pixel position. The block configuration at this time is shown in FIG. 4, and the difference from FIG. 2 is that instead of the logarithmic conversion unit 13, the pixel value of the pixel of the acquired frame image and the pixel value of the corresponding pixel of the frame image immediately after that Instead of the pixel value ratio calculation unit 13A for obtaining the ratio of the larger pixel value to the smaller pixel value and the time differentiation calculation unit 14, the time differentiation calculation for dividing the obtained ratio value by the time interval between frames. Part 14A.

  Next, FIG. 5 shows a block diagram of the motion information calculation unit 3. The region extraction unit 21 receives a motion pattern image, and extracts a region where pixels assigned pixel values are continuously present from the input motion pattern image. The motion region determination unit 22 determines a motion region when the size of the region satisfies a threshold value. The pixel value decrease direction calculation unit 23 calculates a direction in which the pixel value decreases with respect to the pixel to which the pixel value is assigned among all the adjacent 8 pixels to which the pixel value of the input motion pattern image is assigned. To do. The motion direction setting unit 24 for all image regions calculates the motion direction of each motion region by taking the average of the directions for all the calculated pixels. The motion direction setting unit 25 for each motion region calculates the motion direction of each motion region by taking the average of the directions for all the calculated pixels. The motion information calculation unit 3 configured as described above calculates motion information including a motion region and a motion direction from the motion pattern image generated by the motion detection unit.

  FIG. 6 is a flowchart showing the operation of the motion information calculation unit 3 shown in the block configuration in FIG. Hereinafter, it demonstrates in detail according to this flow.

  [Step 11] A motion pattern image is input.

  [Step 12] From the input motion pattern image, a rectangular area in which pixels to which pixel values other than 0 are assigned continuously exists is extracted. For example, as shown in FIG. 7, when a sphere moves in three frame images 601 to 603, a region surrounded by a wavy line shown in the frame image 604 becomes a rectangular region.

  [Step 13] When the size of the rectangular area satisfies a threshold value, that is, when the size of the rectangular area is equal to or larger than a predetermined first threshold value (and, if necessary, equal to or smaller than a predetermined second threshold value), Determined as a motion area.

  [Step 14] For all pixels to which a pixel value other than 0 is assigned in the input motion pattern image, a direction in which the pixel value decreases with respect to a pixel to which a pixel value other than 0 is assigned among the adjacent 8 pixels. calculate.

  [Step 15] The average of the directions for all the calculated pixels is taken as the movement direction of the entire image area.

  [Step 16] The average of the directions for all the pixels in each detected motion region is taken as the motion direction of each motion region.

  Here, the process of extracting the rectangular area is the same as the area division process used in normal image processing, and these techniques are not particularly different from the conventional techniques, and thus detailed description thereof is omitted here. To do. Moreover, not only a rectangular area but a circular area or an elliptical area may be used. In addition, the threshold value to be used may be determined in advance, or may be dynamically calculated using a statistically calculated value such as an average value, a frequency, a deviation value, and the like. Since the method is not particularly different from the conventional method, detailed description thereof is omitted here.

  A supplementary description of the motion information calculation unit 3 that performs the above processing will be given. If an attempt is made to determine the movement of an object for each pixel, there arises a problem that the determination result is greatly affected by noise. Therefore, in the present embodiment, focusing on the fact that the moving object to be detected should have a finite size, in step 12, a pixel value other than 0 is assigned from the motion pattern image. A rectangular area is extracted as an area having a size equal to or larger than the first threshold in which pixels continuously exist, and the movement of the object is determined based on the extracted rectangular area.

  Thereby, in this embodiment, an apparatus robust against noise in the time direction can be provided.

  Further, the second threshold value that defines the upper limit of the size of the rectangular area determined as the motion area in step 13 is a parameter for eliminating the case where the size of the area is too large with respect to the image size. This is set based on the size of the moving object to be detected on the image.

  In the following, a specific procedure for detecting a motion region in a video in the motion detection device described so far will be described.

  Here, a specific processing procedure when the motion detection unit 2 generates a motion pattern image will be described. Here, for the sake of explanation, it is assumed that the image group is a three-frame image group of 5 × 5 pixel size as indicated by 401 to 403 in FIG. For each pixel, pixel values at the same pixel position in all frame images are acquired in order from the latest one, logarithmically converted, and a differential value is calculated. Thus, the pixels indicated by 404 to 405 satisfy the threshold value. On the other hand, since the pixel indicated by 405 first satisfies the threshold value, an evaluation value of 1 is assigned to this pixel, as indicated by 407. Subsequently, the evaluation value 2 is assigned to the pixel indicated by 404 that does not satisfy the threshold value at 405, as indicated by 406. An evaluation value of 0 is assigned to the other pixels, and an image as shown in 408 is finally generated as a motion pattern image.

  Next, a specific processing procedure when the motion information calculation unit 3 calculates the motion region and the motion direction from the motion pattern image will be described. Here, for the purpose of explanation, a motion pattern image indicated by 408 generated by the above procedure as indicated by reference numeral 501 in FIG. On the other hand, when a rectangular area where pixels to which pixel values are assigned continuously exists is extracted, an area as indicated by 502 is extracted. Here, assuming that the size of the rectangular area satisfies the threshold value, it is determined as a motion area. Further, with respect to all the pixels to which a pixel value other than 0 of the image 501 is assigned, a direction in which the pixel value decreases with respect to a pixel to which a pixel value other than 0 is assigned among adjacent 8 pixels is calculated as 503. Four directions are calculated as shown in FIG. On the other hand, the average of the directions for all the calculated pixels and all the detected pixels in each motion region is taken to determine the motion direction as indicated by 504. Here, since there is one motion region, the motion direction of the entire image region is the same as the motion direction of the detected motion region.

  Although the present invention has been specifically described above based on the embodiments, the present invention is not limited to the above-described embodiments and can be widely applied.

  The apparatus of the present invention can also be realized by a computer and a program, and can be recorded on a recording medium or provided through a network.

The figure for demonstrating the functional block diagram of the system which concerns on this invention. The block block diagram of the motion detection part which concerns on this invention. The processing flowchart of the motion detection part which concerns on this invention. The other block block diagram of the motion detection part which concerns on this invention. The block block diagram of the motion information calculation part which concerns on this invention. The processing flow figure of the motion information calculation part which concerns on this invention. The figure which shows the example of the rectangular area which concerns on this invention. The figure for demonstrating an example of the process which concerns on this invention. The figure for demonstrating an example of the process which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image | video input part 2 Motion detection part 3 Motion information calculation part 4 Motion information output part 5 Frame image memory 11 Frame image reading part 12 Pixel value acquisition part 13 Logarithmic conversion part 13A Pixel value ratio calculation part 14, 14A Time differential value calculation part DESCRIPTION OF SYMBOLS 15 Evaluation value setting part 16 Pattern image generation part 21 Area extraction part 22 Motion area determination part 23 Pixel value reduction direction calculation part 24 Motion direction setting part 25 of all image areas 25 Motion direction setting part of each motion area

Claims (3)

In a motion detection device for detecting a motion region in an image,
Video input means for inputting video including captured motion and storing each frame image in a memory;
Among the plurality of frame image groups stored in the video input means , means for reading out frame images for the number of detection target frames set in advance from the latest one, and all pixel positions of the acquired frame images Means for sequentially obtaining pixel values at the same pixel position in all frame images, means for logarithmically converting each obtained pixel value, means for calculating a time differential value from the transition of the converted values, When the time differential value satisfies a threshold value, a means for setting the order number of the frame image from which the pixel value is acquired as an evaluation value for the pixel position and a motion pattern image using the evaluation value for all pixel positions as the pixel value are generated. Motion detection means having means for
Means for calculating a direction in which a pixel value decreases with respect to a pixel to which a pixel value is assigned among adjacent eight pixels for all pixels to which a pixel value of a motion pattern image generated by the motion detection means is assigned. And a motion information calculating means having means for taking the average of the directions for all the calculated pixels and calculating the motion direction of the entire image area ,
Motion information output means for outputting the motion information calculation result in the motion information calculation means in correspondence with each frame image input by the video input means;
A motion detection apparatus comprising: In a motion detection device for detecting a motion region in an image,
Video input means for inputting video including captured motion and storing each frame image in a memory ;
Among the plurality of frame image groups stored in the video input means , means for reading out frame images for the number of detection target frames set in advance from the latest one, and all pixel positions of the acquired frame images Means for sequentially obtaining pixel values at the same pixel position in all frame images, and the pixel value of the obtained frame image and the pixel value of the corresponding pixel in the immediately following frame image with respect to the smaller pixel value Means for determining the ratio of the larger pixel value; means for dividing the value of the determined ratio by the time interval between frames; and if the divided value satisfies a threshold value, the order of the frame image from which the pixel value is acquired Motion detection means comprising means for setting the number as an evaluation value for the pixel position, and means for generating a motion pattern image using the evaluation values for all pixel positions as pixel values ;
Means for calculating a direction in which a pixel value decreases with respect to a pixel to which a pixel value is assigned among adjacent eight pixels for all pixels to which a pixel value of a motion pattern image generated by the motion detection means is assigned. And a motion information calculating means having means for calculating the motion direction of each motion region by taking an average of the directions for all the calculated pixels ,
Motion information output means for outputting the motion information calculation result in the motion information calculation means in correspondence with each frame image input by the video input means;
A motion detection apparatus comprising: A program characterized in that each means in the motion detection device according to claim 1 or 2 is configured to be executable by a computer.