JP2010015469A - Still area detection method, and apparatus, program and recording medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To robustly detect a still area even under conditions with various disturbance factors. <P>SOLUTION: An initial background creation unit 2 in a still area detection apparatus 10 creates a background image by calculating a weighted mean image by using images acquired from an image input means and creates a threshold image by modeling steady fluctuation by normal distribution. A foreground portion extraction unit 3 calculates a foreground by performing differential calculation between the acquired image and the created background image based on a threshold image. Then, the foreground portion extraction unit 3 calculates a foreground by differential calculation for comparing the acquired image with the created background image by calculating an index value based on the created background image, and then calculates a foreground area by calculating a logical AND of the two foregrounds. A sequential background updating processing unit 5 newly creates background models for an area which is not determined as a foreground by the foreground extraction unit 3 and an area determined as a still area by a still portion evaluation unit 4 by using the acquired images and the created background model. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a method for detecting a still area in an image, an apparatus and program therefor, and a recording medium therefor.

  In video monitoring, there is a function to detect a stationary area in a video, such as a left object or a removed object. In order to detect a still area from an image, a method is generally used in which a background image is created, a foreground is calculated by comparing the background image with the current image, and the foreground still time is evaluated.

  The problem here is various disturbances included in the image. Disturbances that cause problems when detecting a still region include, for example, changes in the lighting environment such as sunshine fluctuations, stationary noise such as temporary blockage of moving objects by passers-by, passing through of leaves, camera AGC: Auto Gain Control, AWB: Auto There are White Balance and noise of the image sensor.

  In order to perform detection robustly, it is necessary to efficiently reduce the disturbance that is applied in a complex manner. There are various methods for removing disturbances, such as noise removal processing such as high-frequency removal filter and low-frequency removal filter in the input image, adaptive threshold method in background difference method, etc. .

  There is an adaptive background update method as a method for dealing with steady fluctuation and temporary shielding. For example, there is Non-Patent Document 1 as a method for modeling the fluctuation of the background, but it is difficult to deal with the effects of sunshine fluctuations and object shadows that are not steady with these methods alone.

Some methods that are robust to sunshine and object shadows use differentiation and correlation. For example, there is Non-Patent Document 2 or the like as long as it uses correlation, but the method using correlation or differentiation is weak against high-frequency noise, and it is difficult to eliminate the influence of noise such as shaking of leaves and element noise.
Zoran Zivkovic, Improved Adaptive Gaussian Mixture Model for Background Subtraction, ICPR2004 Seiya Ito, Shunichi Kaneko, Fast Robust Image Matching by Selective Normalization Correlation, IEEJ (Special Issue on Machine Vision Application Technology in Real Environments), 2004 Nara Institute of Science and Technology, OpenCV, Programming book production team, OpenCV, Programming book, 2007 JP2004-241945

  As described above, in the conventional technology for detecting a static region, there are various methods for dealing with only a part of the disturbance, but for use in actual applications, a plurality of disturbance factors can be dealt with simultaneously. If not, false detection and detection omission occur, causing a problem in stability.

  The present invention has been made in view of such circumstances, and the processing related to detection of a stationary region is divided into a plurality of steps, and a method that is resistant to disturbances is used in each of the steps. The present invention provides a still area detection method capable of detection, an apparatus thereof, a program, and a recording medium.

  Accordingly, a still area detection method according to claim 1 for solving the above-mentioned problem is a still area detection method for detecting a still area from an image, wherein the initial background creation means uses an image acquired from an image input device. An initial background creation procedure for creating a background model composed of a background image and a threshold image in which each pixel value is a stillness determination threshold, and a foreground location extraction unit calculates a background difference between the acquired image and the created background model A foreground location extraction procedure for extracting a foreground area by comparing with the above, and a stillness determination threshold in which a still time obtained by the stationary location evaluation means counting the number of times the extracted foreground region continuously appears as a foreground The stationary part evaluation procedure for determining the above-described part as a stationary region, and the sequential background update means extract the foreground part using the acquired image and the created background model. A sequential background update procedure for creating a new background model for a region not determined as a foreground in order and a region determined as a static region in the static part evaluation procedure, and the initial background generation procedure includes the image input A procedure for creating the background image by calculating a weighted average image using a plurality of images acquired from the apparatus, and a procedure for creating the threshold image by modeling steady fluctuations with a normal distribution. The foreground location extraction procedure includes: a background difference procedure for calculating a foreground by calculating a difference between an image acquired from the image input device and the created background image based on the threshold image; and Index value calculation difference for calculating the foreground by calculating the index value based on the image acquired from the input device and the created background image and comparing the images of the two A procedure, a foreground determining step of calculating the foreground area by calculating the logical product of the foreground and the calculated foreground calculated by the background subtraction procedure by the index value calculating difference procedure.

  The still region detection method according to claim 2 is the still region detection method according to claim 1, wherein, in the index value calculation difference procedure, logarithmic differentiation of an image acquired from the image input device and a background image created by the initial background creation procedure. The image ratio is calculated as the index value.

  The still region detection method according to claim 3 is the still region detection method according to claim 1, wherein in the index value calculation difference procedure, the target pixel of the image acquired from the image input device and the background image created by the initial background creation procedure The normalized cross-correlation and the normalized square difference are calculated for the surrounding area, and the normalized cross-correlation value and the normalized square difference value are calculated according to the weight based on the deviation of the pixel value in the surrounding area of the attention area. A weighted sum is calculated as the index value.

  The still region detection method according to claim 4 is the still region detection method according to any one of claims 1 to 3, wherein the still portion evaluation procedure is performed in a predetermined region of interest in an image acquired from the image input device. The method further includes a procedure of detecting entry of an object into the attention area based on a determination result of the stationary area.

  The still region detection device according to claim 5 is a still region detection device that detects a still region from an image, and uses the image acquired from the image input device to set a threshold value in which a background image and each pixel value are still determination threshold values. Initial background creation means for creating a background model composed of images, and foreground location extraction means for extracting a foreground area by comparing the image acquired from the image input device with the created background model by background difference calculation; , A stationary part evaluation means for judging, as the stationary area, a part where the stationary time obtained by counting the number of times the extracted foreground area continuously appears as a foreground is equal to or more than a predetermined stationary judgment threshold, and the acquisition Using the generated image and the created background model, a region that was not determined to be a foreground by the foreground location extraction procedure and a region that was determined to be a static region by the still location evaluation procedure A sequential background update unit that creates a background model, and the initial background creation unit creates the background image by calculating a weighted average image using a plurality of images acquired from the image input device. And means for creating the threshold image by modeling steady fluctuations with a normal distribution, and the foreground location extracting means includes the image obtained from the image input device and the created image By calculating a foreground by performing a difference calculation with a background image based on the threshold image, and calculating an index value based on the image acquired from the image input device and the created background image An index value calculation difference unit that calculates a foreground by a difference calculation that compares both images, a foreground calculated by the background difference unit, and a foreground calculated by the index value calculation difference unit And a foreground determining means for calculating a foreground region by calculating the logical product.

  The still area detection apparatus according to claim 6 is the still area detection apparatus according to claim 5, wherein the index value calculation difference means is a logarithmic differentiation between the image acquired from the image input apparatus and the background image created by the initial background creation means. The image ratio is calculated as the index value.

  The still region detection device according to claim 7 is the still region detection device according to claim 5, wherein the index value calculation difference unit is an attention pixel of an image acquired from the image input device and a background image generated by the initial background generation unit. The normalized cross-correlation and the normalized square difference are calculated for the surrounding area, and the normalized cross-correlation value and the normalized square difference value are calculated according to the weight based on the deviation of the pixel value in the surrounding area of the attention area. A weighted sum is calculated as the index value.

  The still region detection device according to claim 8 is the still region detection device according to any one of claims 5 to 7, wherein the still portion evaluation means is located within a predetermined region of interest in the image acquired from the image input device. Based on the determination result of the still area, the intrusion of the object into the attention area is detected.

  A program according to a ninth aspect is a program for causing a computer to function as each means constituting the still area detecting device according to any one of the fifth to eighth aspects.

  A recording medium according to a tenth aspect is a computer-readable recording medium on which the program according to the ninth aspect is recorded.

  According to the above invention, a stationary region can be detected robustly even under conditions with disturbance factors.

  FIG. 1 is a block diagram showing a schematic configuration of a still area detection apparatus according to an embodiment of the invention.

  The static region detection apparatus 10 includes an input image acquisition processing unit 1, an initial background creation unit 2, a foreground location extraction unit 3, a static location evaluation unit 4, and a sequential background update unit 5.

  The static area detection apparatus 10 includes hardware resources such as a CPU (Central Processor Unit), a memory (RAM), a hard disk drive device, and a communication device, and the hardware resources and installed software and the like. The processing according to each of the functional blocks 1 to 5 is executed by the cooperation. The image obtained by this process can be displayed on a display (display device) (not shown).

  The input image acquisition processing unit 1 executes the input image acquisition step of S101. Specifically, an image is acquired from an image input device such as a camera, DVR, or VTR. The input image acquisition unit 2 can be realized by a known communication device connected to the image input device.

  The initial background creation processing unit 2 executes the initial background creation step of S102. In S102, a background model is created using one or a plurality of images acquired in the input image acquisition step of S101. In S102, by using a method of creating a background model on the assumption that image fluctuations have a normal distribution, modeling the effects of steady fluctuations such as the leaves of trees in the background area swaying in the wind. Can do. This method can be realized by the background difference method using the normal distribution described in Non-Patent Document 3.

  The background model is composed of a background image and a threshold image. In the threshold processing of an image, the same threshold value is normally used for all pixels. When a different threshold value is used for each pixel, this is expressed by an image that records a threshold value corresponding to each pixel. This image is referred to as a threshold image. For example, the threshold value when threshold processing is performed for the coordinates (x, y) of the image A is obtained by referring to the coordinates (x, y) of the threshold image.

  The background image is calculated by taking a weighted average of a plurality of images, and the threshold image is calculated by modeling steady fluctuations with a normal distribution. This makes it possible to create a background image that is not easily affected by temporary fluctuations.

  Each pixel value of each threshold image is obtained by calculating a standard deviation of pixel values for N frames. Or you may use the average value of the own series of the background difference value in each pixel like following Formula. That is, assuming that the pixel value of the existing threshold image T is T (x, y) and the pixel value of the difference image D is D (x, y), the pixel value T ′ (x, y) of the new threshold image T ′ is It can be expressed by the following formula.

T ′ (x, y) = (1−α) × T (x, y) + α × D (x, y)
The background image is obtained by a weighted average of the images. That is, if the pixel value of the existing background image M is M (x, y) and the pixel value of the current image I is I (x, y), the pixel value M ′ (x, y) of the new background image M ′. Can be expressed by the following equation.

M ′ (x, y) = (1−α) × M (x, y) + α × I (x, y)
The initial value of M may be an average of one or a plurality of images acquired first.

  When the threshold image is T, the background difference calculation determines that there is a difference when the absolute difference D between M and I is larger than a value obtained by multiplying the threshold by a constant k.

D (x, y) = | M (x, y) -I (x, y) |
D (x, y) ≧ k × T (x, y): Foreground D (x, y) <k × T (x, y): Background The foreground location extraction unit 3 executes the foreground location extraction step of S103. In S103, the image acquired in the input image acquisition step of S101 is compared with the background that has already been created, and a portion having a difference is extracted as the foreground.

  More specifically, the foreground location extraction unit 3 calculates the foreground by performing a difference calculation between the image acquired in S101 and the background image created in S102 based on the threshold image (background difference procedure). Next, by calculating an index value based on the image acquired in S101 and the background image generated in S102, a foreground is calculated by a difference calculation that compares the two images (index value calculation difference procedure). A foreground region is calculated by calculating a logical product of the foreground calculated by the background difference procedure and the foreground calculated by the index value calculation difference procedure (foreground determination procedure). Examples of the index value include a ratio of a logarithmic differential image of the image acquired in S101 and the background image generated in S102. As other index values, normalized cross-correlation and normalized square difference are calculated for the area around the target pixel of the image acquired in S101 and the background image created in S102, and the pixels in the area around the target area are calculated. The weighted sum of the normalized cross-correlation value calculated according to the weight based on the standard deviation of the value and the normalized squared difference value is exemplified.

  For the static part, use the logical product of the foreground part calculated by the background difference method using the normal distribution described in Non-Patent Document 3 and the foreground part calculated by the logarithmic differential method, or the background model. The foreground location calculated by the background difference method and the foreground location calculated by the correlation-based difference method are calculated. About each difference result, you may add emphasis processes, such as an expansion process and the process which takes the sum of a surrounding pixel.

  The difference method based on logarithmic differentiation and the correlation-based difference method applied in the foreground location extraction unit 3 are comparative methods that are not easily affected by changes in brightness.

  The difference method by logarithmic differentiation is realized by combining the process of taking the logarithm of the image and the process of taking the derivative as shown in Patent Document 1. The difference method by logarithmic differentiation first takes the logarithm of the pixel value of the image to obtain a logarithmic image. Next, a logarithmic differential image is obtained by spatial differentiation of the image. The ratio between the logarithmic differential image obtained for the background image and the logarithmic differential image obtained for the current frame image is taken, and a place where a change of a certain value or more is determined as the foreground place.

  Specifically, the difference method by logarithmic differentiation is realized using the following principle.

  Assuming that the pixel value of an image at a certain time is I, and the pixel value of an image to which brightness change due to environmental brightness, camera AGC, AWB, or the like is added is aI, the following equation can be expressed.

  Here, a is a coefficient indicating a luminance change.

  Here, if a spatial differential is taken for the logarithm image, it can be described as the following mathematical formula.

  Here, if the luminance change works uniformly linearly in the spatial direction, the first term of Equation 2 becomes zero by spatial differentiation, and Equation 3 is obtained. Actually, the luminance change is often non-linear and does not strictly become zero. However, the luminance change is converted to a value that is not easily affected by the luminance change.

  Equations 2 and 3 show spatial differentiation in the x direction, but the same applies to the y direction.

  The calculation of spatial differentiation may be substituted by using a sobel filter in image processing.

  The comparison process is a ratio of an image obtained by applying the processes of Expressions 1 to 3 to the image obtained in S101 and an image obtained by applying the processes of Expressions 1 to 3 to the background obtained in S102 or S105. It is realized by taking.

  The ratio is calculated by dividing the two images, but a process of taking a scale conversion and a process of taking the reciprocal of a part less than 1 are added. Furthermore, an emphasis process such as an expansion process or a process of taking the sum of surrounding pixels may be added.

  Further, the presence or absence of a difference may be converted into a binary image by a predetermined threshold process.

  The above is the principle of the difference method by logarithmic differentiation.

  The correlation-based difference method uses two kinds of correlation values, that is, a normalized cross-correlation between two images I and I ′ and a normalized square difference for pixel values in a region of n × n pixels around the pixel of interest in the image. Calculated as a weighted sum using. As the weight, a value corresponding to the standard deviation of the area of n × n pixels is used. However, n is an arbitrary integer determined in advance.

  Normalized cross-correlation A is obtained by the following equation.

  The normalized square difference B is obtained by the following equation.

  The weighted sum D of the normalized cross-correlation A and the normalized square difference B is calculated by the following equation.

D = αA + (1-α) B
α is a constant in the range of 0 to 1 decided in advance according to the average value of the standard deviation of the standard deviation and I of I _n.

  For example, α = 0.8 when the standard deviation is 20 or more, α = 0.6 when the standard deviation is less than 20, and the like.

  Regarding the method of determining the n × n pixel region, the n × n pixels around the pixels may be used for all the pixels, or the target pixel may be selected and only the selected target pixel may be calculated.

  Further, binarization may be performed according to D by a predetermined threshold process. For example, threshold processing using a predetermined threshold th as a foreground when D ≦ th and as a background when D> th is given as an example.

  The stationary part evaluation part 4 performs the stationary part evaluation step of S104. In S104, the foreground location calculated in the foreground location extraction step in S103 is evaluated to determine whether or not to detect as a still region. Specifically, the time calculated as the foreground portion is continuously counted while looking at the foreground portion in time series, and a portion where the count is equal to or greater than a predetermined stillness determination threshold is determined as a still region. Here, noise removal and area clustering may be performed using shrinkage processing, expansion processing, labeling processing, and the like.

  The sequential background update unit 5 executes the sequential background update step of S105. In S105, a new background model is created using the image obtained in the input image obtaining step in S101 and the background already created. The update location is the background region calculated as the background in the foreground location extraction step in S103 and the region determined as the still region calculated in the static location evaluation step in S104. The background model is updated by calculating the background region and the static region using a background difference method using a normal distribution described in Non-Patent Document 3.

  Specifically, only the background area and the still area are calculated based on the following calculation method of the background image M ′ and the threshold image T ′ in S102 described above.

M ′ (x, y) = (1−α) × M (x, y) + α × I (x, y)
T ′ (x, y) = (1−α) × T (x, y) + α × D (x, y)
As described above, it is possible to detect an area that is stationary for a certain period of time.

  This can also be used as intrusion detection for detecting an intrusion of an object into a specific region of interest in the image. When used as intrusion detection in an area, the determination is made by cutting out the attention area when acquiring the input image or by using only the determination result in the attention area for the stationary part evaluation result. If the stillness determination threshold value is set to a small value, it is possible to immediately detect an object that has entered the area.

  FIG. 2 is a block diagram illustrating a more specific system configuration of the still region detection apparatus 10.

  The foreground location extraction unit 3 of the still region detection apparatus 10 includes functional blocks 31 to 37 and 41 to 47 as shown in FIG. The processing related to the function blocks 31 to 37 and 41 to 47 can be realized by the cooperation of the hardware resources described above and the installed software. The background holding unit 31 and the background candidate holding unit 45 may be configured by a device having a storage function such as a computer memory or a hard disk.

  The procedure (S201 to S220) of still region detection by the still region detection apparatus 10 will be described with reference to FIG.

  S201: An image captured or accumulated from the image input device 11 is output. Examples of the image input device 11 include an imaging device such as a camera, a video storage device such as a DVR or VTR, or a video distribution device.

  S202: The input image acquisition processing unit 1 acquires the video output from the image input device 11 in the process of S201, and the initial background creation processing unit 2, the current image-background difference processing unit 32, the logarithmic differential current image-background difference. Delivered to the processing unit 41.

  S203: The initial background creation processing unit 2 uses the plurality of images acquired from the input image acquisition processing unit 1 in the process of S202, and the background difference method using the normal distribution described in Non-Patent Document 3 above. To create a background model.

  S204: The background holding unit 31 holds the background model created in S203, or the background model updated in S216 (background update processing unit 46) or S218 (sequential background update processing unit 5) in the storage area.

  S205: The current image-background difference processing unit 32 uses the image acquired in the process of S202 and the background model acquired from the background holding unit 31 to use the background difference using the normal distribution described in Non-Patent Document 3 described above. The difference calculation is performed by the method of the method, and the difference image is output.

  S206: The foreground / background mask creation processing unit 33 calculates the foreground area mask image and the background area mask from the difference image calculated in S205 based on the background difference method using the normal distribution described in Non-Patent Document 3 described above. Create an image.

  S207: The current image-background candidate difference processing unit 34 is acquired from the image acquired in the process of S202 and the background candidate holding unit 45 based on the background difference method using the normal distribution described in Non-Patent Document 3 described above. Difference calculation with the background candidate image is performed, and the difference image is output.

  S208: The still candidate mask creation processing unit 35 creates a still candidate area mask image from the difference image calculated in S207 based on the background difference method using the normal distribution described in Non-Patent Document 3 described above.

  S209: The still time evaluation processing unit 36 uses the foreground area mask image calculated in S206 and the still candidate area mask image calculated in S208 to use the background subtraction method using the normal distribution described in Non-Patent Document 3 described above. Based on the above method, the stationary time of the stationary part is evaluated and a stationary time count image is created.

  S210: The still part mask creation processing unit 36 creates a still area mask image from the still time count image created in S209 based on the background difference method using the normal distribution described in Non-Patent Document 3 described above.

  S211: Logarithmic differential current image-background difference processing unit 41 uses the image acquired in S202 (input image acquisition processing unit 1) and the background model acquired from the background holding unit 31, and the principle of the difference method based on the logarithmic differentiation. The difference calculation is performed based on the above and a difference image is output.

  S212: The logarithmic differential foreground mask creation processing unit 42 performs threshold processing based on a predetermined threshold for the difference image calculated in S211 and calculates a foreground area mask image by logarithmic differentiation.

  S213: The logarithmic differential still time evaluation processing unit 43 evaluates the still time by counting the mask areas in time series for the foreground area mask image obtained by the logarithmic differentiation obtained in S212, and creates a still time count image by logarithmic differentiation. .

  S214: The logarithmic differential stationary part mask creation processing unit 44 performs threshold processing based on a predetermined threshold for the still time count image by logarithmic differentiation created in S213, and creates a static region mask image by logarithmic differentiation.

  S215: The background candidate holding unit 45 holds the background candidate model created in the process of S217 by the background candidate update processing unit 47 in the storage area.

  S216: The background update processing unit 46 uses the normal distribution described in Non-Patent Document 3 described above for the background model using the image acquired in the process of S202 and the background region mask image calculated in the process of S206. Update based on the background subtraction method.

  S217: The background candidate update processing unit 47 uses the image acquired in the process of S202 and the background candidate area mask image calculated in the process of S208 to convert the background candidate model into the normal distribution described in Non-Patent Document 3. Update based on the background subtraction method used.

  S218: The sequential background update processing unit 5 is based on the method of the background difference method using the normal distribution described in Non-Patent Document 3 described above for the region determined as the still region in the process of S219 by the stationary part evaluation processing unit 4. The background model and the background candidate model are updated. The updated background model is output to the background holding unit 31 (S204). On the other hand, the updated background candidate model is output to the background candidate holding unit 45 (S215).

  S219: The still part evaluation processing unit 4 determines the still area by taking the logical product of the still area mask image created in the process of S210 and the still area mask image by the logarithmic differentiation created in the process of S214. In this process, noise removal or region clustering may be combined using shrinkage processing, expansion processing, labeling processing, or the like.

  S220: Information such as the position, size, area, and detection time of the still area obtained in the process of S219 is output to the outside. Specifically, the information is presented by a presentation device such as a display attached to the still region detection device 10. Further, the information is recorded in a database stored in a recording device (recording medium such as a hard disk) provided inside or outside the static area detecting device 10, or is networked as information such as a warning to a remote place. It is transmitted to the outside via.

  The processing of S211 to S214 may be replaced with the processing of S301 to S304 respectively executed by the function blocks 51 to 57 to which the correlation-based method illustrated in FIG. 3 is applied. The processing according to the function blocks 51 to 54 shown in FIG. 3 can be realized by the cooperation of the hardware resources described above and the installed software.

  S301: Correlation-based current image-background difference processing unit 51 outputs a difference image by difference calculation based on the principle of the correlation-based difference method using the image acquired in the process of S202 and the background model acquired in S204.

  S302: The correlation-based foreground mask creation processing unit 52 performs threshold processing based on a predetermined threshold for the difference image calculated in the process of S301, and calculates a correlation-based foreground region mask image.

  S303: The correlation-based still-time evaluation processing unit 53 evaluates the still-time by counting the mask regions in time series for the correlation-based foreground region mask image created in the process of S302, and the correlation-based still-time count image Create

  S304: The correlation-based still part mask creation processing unit 54 performs threshold processing based on a predetermined threshold for the correlation-based still time count image created in the process of S303 to create a correlation-based still region mask image. .

  4 to 7 show the appearance of a stationary region (a region where the foreground is stationary for a certain time, not the background) in the background difference method using the normal distribution described in Non-Patent Document 3 and the method according to the present invention. The result of processing for 24 hours of video without a window is determined by determining a portion that has been stationary for 30 seconds as a stationary region and assimilating the portion that has been stationary for 40 seconds into the background.

  FIG. 4 exemplifies a part of a frame of a video for one day from 0 o'clock to 24 o'clock taken with QVGA at about 10 fps.

  5 and 6 are characteristic diagrams showing the results of processing the image of FIG. 4 using the background difference method. The vertical axis is the total number of pixels in the area determined as a still area in the entire image, and the vertical axis is the time axis.

  FIG. 5 is a characteristic diagram showing the result of processing the image of FIG. 4 using only the background difference method using the normal distribution described in Non-Patent Document 3.

  FIG. 6 is a characteristic diagram showing a result of processing the image of FIG. 4 by applying a difference method by logarithmic differentiation in the still region detection apparatus 10 according to the invention.

  FIG. 7 is a characteristic diagram showing a result of processing the image of FIG. 4 by applying the correlation-based difference method in the still region detection apparatus 10 according to the invention. Here, the correlation-based difference is calculated by dividing the image into 32 × 24 regions and using the 10 × 10 region of interest.

  Since the used video is a video of only the state of sunshine fluctuations and does not show any object blocking or leaving / taking away, ideally it is desirable to detect all frames as the background. A pixel detected as a still region is a pixel that is erroneously detected as a stationary region appears instead of a background as a result of sunshine fluctuation. Comparison of superiority and inferiority of each method is shown in FIGS. 5 to 7 showing the number of erroneously detected pixels. Can do.

  As can be seen from the characteristic diagram of FIG. 5, when only the background difference method using the normal distribution described in Non-Patent Document 3 is used, fluctuations that are not temporary changes due to sunshine fluctuations or the like are mistakenly regarded as still areas. Detected.

  As is apparent from the comparison between the characteristic diagrams of FIGS. 6 and 7 and the characteristic diagram of FIG. 5, it can be seen that the stationary region detection apparatus 10 according to the present invention can greatly suppress erroneous detection due to sunshine fluctuations.

  FIG. 8 to FIG. 11 show the results of processing according to the present invention by determining a place where an image with QVGA and 20 fps is stationary for 30 seconds as a still area and assimilating the place where the image is stationary for 40 seconds with the background. FIG. 8 shows the video used. Here, the video is a video where the still area is left 7 seconds after the start of shooting, and the take-out occurs 2 minutes later, and the person goes back and forth in front of the static area, and the background blinds are shaken by the wind. The shooting time is at sunset. An inexpensive network camera is used as the camera, and AGC and AWB are applied.

  FIG. 9 is a characteristic diagram and an image showing the result of processing by the background real-time update method described in Non-Patent Document 3.

  FIG. 10 is a characteristic diagram and an image showing a processing result when the difference method based on logarithmic differentiation is used in the still region detection apparatus 10 according to the invention.

  FIG. 11 is a characteristic diagram and an image showing a processing result when the correlation-based difference method is used in the still region detection apparatus 10 according to the invention.

  9 to 11, the vicinity where the still area is placed is used as a correct area mask, and the total number of pixels of the area determined as the still area detected in the mask is the correct value, and the still area detected outside the mask is used. The total number of pixels in the determined area is plotted as an incorrect answer value.

  As is apparent from the characteristic diagram of FIG. 9, when processing is performed by the background real-time update method according to Non-Patent Document 3, a slightly incorrect answer area (Wong) is generated, which is shown in FIGS. As described above, it can be seen that when the still area detection method according to the present invention is used, only a still area can be detected without erroneous detection.

  In the conventional still area detection method, erroneous detection is likely to occur due to the influence of a plurality of disturbances such as a change in illumination environment, temporary blocking of a moving area, stationary noise, camera AGC, AWB, and image sensor noise. As is clear from the above description, according to the still region detection apparatus 10 according to the embodiment of the present invention, it is possible to robustly detect a still region under conditions with these various disturbance factors.

  In addition, this invention is not limited to the above-mentioned embodiment, A computer is used as the process which concerns on each functional block 1-5, 31-37, 41-47, 51-54 which comprises the still region detection apparatus 10. FIG. It is good also as a mode of the program to function. The program may be provided by being stored in a computer-readable recording medium or provided through a network.

1 is a block diagram showing a schematic configuration of a still area detection device according to an embodiment of the invention. The block diagram which showed the specific system configuration | structure of the foreground location extraction part which concerns on embodiment of invention. The block diagram which showed the system configuration | structure which performs the correlation based difference method applied to the foreground location extraction part which concerns on embodiment of invention. The figure which illustrated the one day image | video from 0 o'clock to 24 o'clock image | photographed by QVGA at about 10 fps in the time series. FIG. 5 is a characteristic diagram showing a result of processing the image of FIG. 4 using only the background difference method using the normal distribution described in Non-Patent Document 3. The characteristic view which shows the result of having processed the image of FIG. 4 by applying the difference method by logarithmic differentiation in the still area detection apparatus which concerns on embodiment of invention. The characteristic view which shows the result of having processed the image of FIG. 4 by applying the correlation-based difference method in the still region detection apparatus according to the embodiment of the invention. An image displaying a video and a correct area mask when a still area is left 7 seconds after the start of shooting and the removal occurs after 2 minutes. The characteristic view and image which showed the result processed by the method of the real-time update of the background of a nonpatent literature 3. The characteristic view and image which showed the processing result at the time of using the difference method by logarithmic differentiation in the still region detection apparatus which concerns on embodiment of invention. The characteristic view and image which showed the processing result at the time of using the correlation based difference method in the static region detection apparatus which concerns on embodiment of invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Input image acquisition process part 2 ... Initial background creation process part (initial background creation means)
3. Foreground location extraction unit (foreground location extraction means)
4 ... Stationary part evaluation part (stationary part evaluation means)
5 ... Sequential background update unit (sequential background update means)
DESCRIPTION OF SYMBOLS 10 ... Static region detection apparatus 41 ... Logarithmic differential current image-background difference processing part 42 ... Logarithmic differential foreground mask creation processing part 43 ... Logarithmic differential still time evaluation processing part 44 ... Logarithmic differential static part mask creation processing part 51 ... Correlation base present Image-background difference processing unit 52 ... correlation-based foreground mask creation processing unit 53 ... correlation-based still time evaluation processing unit 54 ... correlation-based still location mask creation processing unit

Claims (10)

A still region detection method for detecting a still region from an image,
An initial background creation procedure for creating a background model composed of a background image and a threshold image in which each pixel value is a stillness determination threshold using an image acquired by the initial background creation means from the image input device;
A foreground location extraction procedure in which a foreground location extraction means compares the acquired image and the created background model by background difference calculation to extract a foreground region;
A stationary part evaluation procedure in which a stationary part evaluation unit determines a part where a stationary time obtained by counting the number of times the extracted foreground area has continuously appeared as a foreground is equal to or greater than a predetermined stationary judgment threshold as a stationary area. When,
The sequential background update means uses the acquired image and the created background model to determine a region that is not determined to be a foreground by the foreground location extraction procedure and a region that is determined to be a static region by the still location evaluation procedure. A sequential background update procedure for creating a new background model,
The initial background creation procedure includes:
Creating the background image by calculating a weighted average image using a plurality of images acquired from the image input device; and
Creating a threshold image by modeling steady fluctuations with a normal distribution;
The foreground location extraction procedure includes:
A background difference procedure for calculating a foreground by performing a difference calculation between an image acquired from the image input device and the created background image based on the threshold image;
An index value calculation difference procedure for calculating a foreground by calculating a difference between the image acquired from the image input device and the created background image and comparing the images of the both;
A still region detection method comprising: a foreground determination procedure for calculating a foreground region by calculating a logical product of a foreground calculated by the background difference procedure and a foreground calculated by the index value calculation difference procedure. The index value calculation difference procedure calculates a ratio of a logarithmic differential image of an image acquired from the image input device and a background image created by the initial background creation procedure as the index value. Stationary area detection method. In the index value calculation difference procedure, a normalized cross-correlation and a normalized square difference are calculated for a region around the target pixel of the image acquired from the image input device and the background image generated in the initial background generation procedure, The weighted sum of the normalized cross-correlation value and the normalized square difference value is calculated as the index value according to a weight based on a deviation of pixel values in a region around the region. Stationary area detection method. The stationary part evaluation procedure is:
2. The method according to claim 1, further comprising a step of detecting an intrusion of an object into the attention area based on a determination result of a still area within a predetermined attention area in an image acquired from the image input device. 4. The method for detecting a still area according to any one of 3 above. A still area detection device for detecting a still area from an image,
Using the image acquired from the image input device, an initial background creating means for creating a background model composed of a background image and a threshold image in which each pixel value is a stillness determination threshold;
A foreground location extracting means for extracting a foreground region by comparing the image acquired from the image input device and the created background model by a background difference calculation;
A stationary part evaluation means for determining a stationary area where a stationary time obtained by counting the number of times the extracted foreground area continuously appears as a foreground is equal to or greater than a predetermined stationary determination threshold;
Using the acquired image and the created background model, a new background model is created for a region that has not been determined to be a foreground by the foreground location extraction procedure and a region that has been determined to be a static region by the still location evaluation procedure. And sequential background updating means for
The initial background creating means includes
Means for creating the background image by calculating a weighted average image using a plurality of images acquired from the image input device;
Means for creating the threshold image by modeling a steady fluctuation by a normal distribution,
The foreground location extracting means includes
Background difference means for calculating a foreground by performing a difference calculation between the image acquired from the image input device and the created background image based on the threshold image;
Index value calculation difference means for calculating a foreground by calculating a difference between the image acquired from the image input device and the created background image and comparing the images of the two;
A still area detection apparatus comprising: a foreground determination unit that calculates a foreground area by calculating a logical product of a foreground calculated by the background difference unit and a foreground calculated by the index value calculation difference unit. 6. The index value calculation difference means calculates a ratio of a logarithmic differential image of an image acquired from the image input device and a background image created by the initial background creation means as the index value. Stationary area detector. The index value calculation difference unit calculates a normalized cross-correlation and a normalized square difference for a region around the target pixel of the background image generated by the image acquired from the image input device and the initial background generation unit, and the target 6. The weighted sum of the normalized cross-correlation value and the normalized square difference value is calculated as the index value according to a weight based on a pixel value deviation in a region around the region. Stationary area detector. The stationary part evaluation means detects an intrusion of an object into the attention area based on a determination result of a stationary area within a predetermined attention area in an image acquired from the image input device. Item 8. The static region detection device according to any one of Items 5 to 7.   A program that causes a computer to function as each unit that constitutes the still area detection device according to any one of claims 5 to 8.   A computer-readable recording medium on which the program according to claim 9 is recorded.