JP2010128947A - Abnormality estimation apparatus, abnormality estimation method and abnormality estimation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To stabilize normality/abnormality classification according to motion features of moving objects. <P>SOLUTION: Video captured by a camera is input into a video input part 2 of an abnormality estimation apparatus 1. A motion feature extraction part 3 generates a plurality of time series motion features from the input video, selects from the features according to a selection criterion, and outputs a time series combination of selected features to an identification part 4. The identification means 4 classifies the input combination of features as normal/abnormal by an SVM technique, and computes N logical values indicating whether or not the feature elements should be used in final identification. A display part 5 displays time series normal/abnormal flags as identification results. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for identifying a non-stationary degree of a scene from a feature amount based on a movement of an animal body included in an image.

  In recent years, video surveillance systems equipped with a large number of cameras have become widespread, and the efficiency of surveillance work is required. For this reason, functions for extracting information from video such as a moving object detection function and a person tracking function have been incorporated into the video monitoring system in addition to the video storage function. As one of such functions, there is a demand for non-stationary detection technology that automatically detects scenes that are different from usual, and there is an increasing need to incorporate this into an existing video surveillance system in a video module.

  In order to detect changes in the position of the moving object in the video and changes in the direction of movement as a clue to detect an unsteady scene from the video, a feature value is set as a set of the coordinate of the position at each time and the motion vector at that coordinate By doing so, an unsteady scene can be detected from the difference in the feature amount. However, the following problems have been raised in the detection of non-stationary scenes.

  The first problem is that the motion vector extraction process is easily affected by noise, and it is difficult to stably separate the steady state and the unsteady state due to the large variation in the feature amount.

  The second problem is how to input the time change of the motion vector to the discriminator. When a motion vector is stored for each pixel of the image, the information is several times the number of frames of pixels in the order of thousands to hundreds of thousands. In this way, the amount of memory used for calculation and the time required for statistical processing become enormous, and it is not suitable for input to real-time detection processing. It is possible to reduce the amount of memory by holding information only about the area where the moving object exists, but if the number of areas changes with time, the dimension number of the feature quantity will change, It is difficult to handle after statistical processing. Therefore, there is a need for a method for extracting a change in the position of the moving object in the video and a change in the direction of movement as a stable low-dimensional feature value.

  The third problem, the what we should use the feature amount as information to separate the stationary and non-stationary, it is believed that differ by the image observed there and the camera installation site, in advance what feature quantity best It is difficult to determine whether or not there is a method for automatically selecting a feature amount. It is not always necessary to use all of them, and features that do not contribute to steady / unsteady separation may become noise in the identification calculation, and calculation cost may be wasted.

For the first problem, Non-Patent Document 1 proposes a method for obtaining a spatial and temporal motion average in a moving body region. This method has been proposed as a feature quantity based on movement for identifying whether a moving area is a human or not, and is effective when comparing each area. However, it is not a comparison for each region, but a method for setting the feature amount of the entire image is not shown. In order to extract the feature amount of the entire image, it is necessary to devise a technique that can reflect the motion characteristics of the main moving objects included in the image.
Tetsuji Hashita, Kazuhiko Kusumi, Yasushi Yagi “Detection of Pedestrians in Outdoor Scenes Focusing on Spatio-Temporal Features of Movement in Change Regions” IEICE Transactions D-II, vol. J87-D-II, no. 5, pp. 1104-1111, 2004. “Computer Vision” New Technology Communications edited by Takashi Matsuyama, Yoshinori Kuno, Satoshi Imiya 3.3.4 pp. 172 Shin, Watanabe, Sakakibara "Video Analysis Method Using Temporal Template" IEICE Technical Report, PRMU2002 Baba "Statistics of angle data" Statistical Institute of Statistical Mathematics Vol. 28, No. 1, pp. 41--45, 1981 Zeng Wenhua, Ma Jian, “A Novel Incremental SVM Learning Algorithm,” The 8th International Conference on Computer Supported Cooperatives. 658--662, 2003.

  In the conventional technique, the motion vector extraction process is easily affected by noise, and the feature amount using the motion vector has many variations, so that stable steady / unsteady classification is difficult. Even if an existing method for obtaining a spatial and temporal motion average in a moving body region is used, a method for making a feature amount of an entire image, not a comparison for each region, is not shown.

  In addition, when a motion vector is stored for each pixel of an image, the amount of features is in the order of thousands to several hundreds of thousands, and the amount of memory used for calculation and the time required for statistical processing become enormous, which is not suitable for real-time processing. If the amount of memory is reduced by holding only information on the area where the animal exists, the number of dimensions of the feature quantity will change if the number of areas changes over time, making it difficult to use for statistical processing. .

  Furthermore, as information that separates stationary and non-stationary, the extractable movement features include the position of the moving object, the direction of movement, the size, and the time. Among these pieces of information, there is a difference between stationary and non-stationary. Use of a non-contributing one may cause noise in the identification calculation and may lead to a waste of calculation cost. Therefore, there is a need for a means for selecting a combination of which elements to be reflected in the feature quantity among the movement features of the moving object, such as the position, the movement direction, the size, and the continuous time of the movement.

  This invention is made | formed in view of such a situation, and makes it a solution subject to classify | categorizing steady / unsteady stably from the motion characteristic of a moving body.

  Accordingly, in order to solve the above-mentioned problem, the invention according to claim 1 extracts a feature amount based on the movement of the moving object included in the video captured by the imaging device, and performs statistical processing to unsteady scenes. A motion feature extracting unit that extracts a feature amount based on a motion of a moving object in an input video from the photographing device, and a non-input image from the feature amount extracted by the motion feature extracting unit. And an identification means for identifying continuity.

  According to a second aspect of the present invention, the motion feature extraction unit includes a plurality of feature amount extraction modules that generate time-series feature amounts from the input video, and feature amounts generated by the feature amount extraction modules. And feature amount selection means for selecting based on a selection criterion and outputting a time series of combinations of the selected feature amounts to the identification unit.

  According to a third aspect of the present invention, each of the feature quantity extraction modules extracts a moving body region in the input video and obtains a binary image of the moving body region and a background region, and the input video. In each time-series image in the middle, a motion vector array (xyuv) in which a two-dimensional vector indicating the position (xy) of each pixel and the motion direction (uv) of the pixel is paired A motion vector calculation means for obtaining a moving vector, and an integrated vector calculation means for integrating the information on the moving body region and the information on the motion vector arrangement and obtaining the pair of the integrated vectors as a feature quantity. Yes.

  According to a fourth aspect of the present invention, the integrated vector calculating means generates a motion history image obtained by temporally superimposing the time-series binary images by assigning numerical values to the moving body region and setting the background region to zero. It is characterized by comprising history generation means and mask processing means for performing mask processing on the motion vector array using the motion history image.

  According to a fifth aspect of the present invention, each of the feature quantity extraction modules further includes a continuous time counting unit that counts a time at which the integrated vector pair continuously appears, and the integrated vector includes the count time. The feature quantity added to the pair is output to the feature quantity selection unit.

  According to a sixth aspect of the present invention, the discriminating unit classifies the N-dimensional discriminating feature amount, which is a combination of the feature amounts of the feature amount extraction modules, into a steady / non-stationary state using an SVM (support vector machine) method. N indicating whether or not the element is finally used for identification according to the element of the N-dimensional feature amount from the identification unit, the support vector group generated by the SVM identification unit, and the stationary / non-stationary identification result And a feature value evaluating means for feeding back the logical values to the identification device, and displaying a time series of steady and non-steady plugs as identification results.

  The invention according to claim 7 is a method of extracting feature quantities based on the movement of a moving object included in an image photographed by a photographing apparatus, and identifying a non-stationary state of a scene by statistical processing. A feature extraction unit extracts a feature quantity based on the movement of a moving object in an input video from the imaging device, and an identification unit extracts the input from the feature quantity extracted in the motion feature extraction step. And an identification step for identifying non-stationarity of the image.

  According to an eighth aspect of the present invention, in the motion feature extraction step, each of a plurality of feature amount extraction modules generates a time-series feature amount from the input video, and a feature amount selection unit includes the feature amount selection unit, A feature amount selection step of selecting each feature amount generated in the feature amount generation step based on a feature amount selection criterion and outputting a time series of a combination of the selected feature amounts to the identification unit. It is characterized by.

  In the invention according to claim 9, in the feature quantity generation step, a moving body region in the input video is extracted, a change region extraction step for obtaining a binary image of the moving body region and a background region, and in the input video In each time-series image, a motion vector array (xyuv) in which two-dimensional vectors indicating the position (xy) of each pixel and the motion direction (uv) of the pixel are paired is obtained. A motion vector calculation step to be obtained, and an integrated vector calculation step for integrating the information on the moving body region and the information on the motion vector arrangement, and obtaining the integrated vector pair as a feature amount. .

  A tenth aspect of the present invention relates to a nonstationary degree estimation program that causes a computer to function as the nonstationary degree estimation device according to any one of the first to sixth aspects.

  According to the first to tenth aspects of the present invention, steady / unsteady is stably classified from the motion characteristics of the moving object, and high-precision and unsteady detection can be performed at high speed without being affected by noise. .

  FIG. 1 shows an example of the configuration of an unsteady degree estimation apparatus according to an embodiment of the present invention. The estimation apparatus 1 includes video data from a photographing apparatus (for example, a digital camera or a digital video camera) of a video surveillance system via a network. Is entered.

  The pre-stationary degree estimation apparatus 1 is constituted by a computer, and includes a video input unit 2 for inputting the video data, and a motion feature extraction for extracting motion features of moving objects from the input video of the video input unit 2 in time series. Unit 3, an identification unit 4 for obtaining a numerical value indicating the non-stationary degree of the motion feature based on the time-series motion feature extracted by the motion feature extraction unit 3, and a display unit for displaying the numerical value obtained by the identification unit 4 5.

  The processing functions of the units 2 to 5 are realized by the cooperation of computer hardware and software. At this time, the identification unit 4 can output a logical value indicating whether or not the value is unsteady to the display unit 5 by determining the threshold value of the unsteady degree. The display unit 5 output here may be a monitor, for example.

  The nonstationary degree estimation device 1 stores normal computer components, for example, arithmetic means (for example, CPU: Central Processor Unit) that controls the processing of each of the units 2 to 5 and processing data of the units 2 to 5. It has an input means such as a possible memory (RAM), a hard disk drive device capable of storing the processing data of the units 2 to 5, a communication device for network connection, a pointing device and a keyboard. Hereinafter, the processing content of each said parts 2-5 is demonstrated concretely.

(1) Video input unit 2
The video data input to the video input unit 2 may be, for example, an AVI file or a JPEG image sequence, and the video data is input to the video input unit 2 frame by frame. Here, the video input unit 2 is realized by a communication device.

(2) Motion feature extraction unit 3
The motion feature extraction unit 3 extracts the motion feature of the object by using the input video data for each of a plurality of frames. The motion feature extracted here is a numerical value having information on the region of the moving object included in the video, its motion direction, and its change.

  Specifically, the motion feature extraction unit 3 is roughly divided into a feature quantity extraction module group 6 and a feature quantity selection unit 7 as shown in FIG. The feature quantity extraction module group 6 includes k types of feature quantity extraction modules. In FIG. 2, each feature quantity extraction module is shown as a feature quantity 1 extraction module to a feature quantity k extraction module. Each of the feature quantity extraction modules extracts a feature quantity k from the feature quantity 1 using the time series image frames input from the video input section 2 and outputs the time series to the feature quantity selection section 7 respectively. Yes.

  The feature amounts 1 to k mean features that can be extracted from the time series of image frames, that is, features of image changes caused by movement of moving objects in the image frames. Here, it will be described as a motion vector. However, spatio-temporal features (features of the number of pixels x number of frames in which binary images with a moving region of "1" and a background of "0" are expanded in a row and arranged for multiple frames) and between frames A difference feature (a feature quantity in the number-of-pixels dimension, which is an array of pixels of a binary image generated by a difference between adjacent frames) may be used.

  The feature quantity selection unit 7 outputs N-dimensional discriminating feature quantities in combination with any one of the feature quantities 1 to k based on the feature quantity selection criteria in time series. This feature quantity selection criterion is a standard indicating which of each feature quantity is used, that is, a set of numerical values corresponding to the feature quantity 1 to the feature quantity k, and is input from the outside with a GUI (Graphic User Interface) or a command prompt. can do. In this case, the input means may be used for input. The feature quantity selection criterion may be a threshold value defined in advance in a program or the like.

  FIG. 3 shows a configuration example of each of the feature quantity extraction modules. Here, each feature amount extraction module includes a change area extraction unit 8, a motion vector calculation unit 9, an integrated vector unit 10, a continuous time counting unit 13, and an array storage unit 14.

  To describe a specific process, first, the motion vector calculation unit 9 obtains a motion vector array. This motion vector is made to correspond to the position (xy) of each pixel in the image and the two-dimensional vector indicating the motion direction (uv) of the pixel at the detection time of the position (xy). It is set as an array of a set (x.y.u.v). The obtained motion vector (x.y.uv) is output to the integrated vector calculation unit 10.

  Hereinafter, in this embodiment, what is generally called an optical flow and a processed vector such as spatially and temporally averaged are combined as a motion vector. In order to obtain this optical flow, a method based on luminance gradient or a method based on region matching described in Non-Patent Document 2 may be used. Next, the change region extraction unit 8 outputs a binary image obtained by separating the moving object region and the background region to the integrated vector calculation unit 10.

  The integrated vector calculation unit 10 obtains a vector integrated as a feature amount of the entire image from the binary image output from the change region calculation unit 8 and the motion vector output from the motion vector calculation unit 9, Output to the continuous time counting unit 13. Here, the output of the motion vector calculation unit 9 is a time series of a set of numerical values composed of motion vectors (four-dimensional information determined by start point coordinates and end point coordinates) corresponding to each pixel.

  For example, it is output as a time series of a set of (representative vector start point x coordinate, representative vector start point y coordinate, representative vector end point x coordinate, representative vector end point y coordinate). The integrated vector calculation unit 10 integrates change area information (binary image) and motion vector information, and outputs a time series of integrated motion vectors indicating the motion characteristics of the entire image.

  FIG. 4 shows a configuration example of the integrated vector calculation unit 10. Here, the integrated vector calculation unit 10 includes a motion history image generation unit 11 and a mask processing unit 12. The motion history image generation unit 11 generates an image (motion history image) on which the temporal change of the change region is superimposed. This motion history image is used to know the magnitude and direction of the movement of the moving object during a certain period of time, and Non-Patent Document 3 shows the method.

  Specifically, a time-series binary image is input to the motion history image generation unit 11 from the change region extraction unit 8, and numerical values other than zero are applied to pixels in the motion region, and other regions are set to zero. A mask image is generated. This mask image is input to the mask processing unit 12. In addition, a motion vector array (x, y, u, v) is input from the motion vector calculation unit 9 to the mask processing unit 12. The mask processing unit 12 obtains an integrated vector by averaging the vectors within the mask of the mask image. Here, the motion vectors are integrated from the spatial average and the temporal average.

  An example of the integration process of the mask processing unit 12 will be described with reference to FIG. In FIG. 5A, a change area at time t and a vector indicating the direction and magnitude of the movement of the area are indicated by arrows. FIG. 5B similarly shows the state at time t + k. FIG. 5C shows a state in which the change areas at time t and time t + 1 are superimposed. This state corresponds to a motion history image.

  That is, the process of the motion history image generation unit 11 corresponds to the process between FIGS. 5 (a) to 5 (c). Here, only two times of time t and time t + k are superposed, but the interval k and how many times are superposed are variable parameters and can be arbitrarily changed. For the overlapped change region in FIG. 5C, the vertical and horizontal lengths of the inscribed rectangles are obtained for each region, and the rectangle outer circumference or the rectangular area is equal to or larger than a certain threshold value, or a certain number from the larger one Select the area. The threshold value and the variable parameter may be defined in a program or the like.

  Then, the motion history image generation unit 11 outputs a mask image in which the selected area is “1” and the other areas are “0”. In FIG. 5D, vector integration is performed only for the selected region. At this time, the average of the vectors inside the mask is obtained, and this is used as an integrated vector. FIG. 5D shows processing in the integrated vector calculation unit. An arrow P in FIG. 5D corresponds to a representative vector. FIGS. 5E and 5F show specific processed image examples of FIGS. 5C and 5D. Hereinafter, two examples of the integration processing method will be described.

In the first method, first, a vector indicating the direction and magnitude of the movement of each of the pixels in the change area A ^(t) ₁ and the change area A ^(t) _{2 at} time t in FIG. Numerical values corresponding to v ^(t) ₁ and vector v ^(t) ₂ are assigned. Similarly, a vector v indicating the direction and magnitude of the movement of each of the pixels in the change area A ^{(t + k)} ₁ and the change area A ^{(t + k)} _{2 at} time t + k in FIG. ^The numerical values corresponding to ^{(t + k)} ₁ and the vector v ^{(t + k)} ₂ are given.

  Next, in FIG. 5C, when each pixel in the rectangle has the numerical value, a numerical value obtained by integrating the numerical values of both the rectangles is given to a portion where the rectangles overlap. When the numerical value corresponding to the vector is a set of (start point x, start point y) and (end point x, end point y) of the vector, the average is obtained as the average of each of these four numerical values.

  In addition, the numerical value corresponding to the vector may be divided into N regions in the vector direction (360 degrees), and numerical values from 1 to N may be applied. In this case, the average value of the numerical values is given to the portion where the rectangles overlap. The way of dividing the direction is, for example, four divisions (N = 4) of 0 degrees to 45 degrees and 315 degrees to 360 degrees, 45 degrees to 135 degrees, 135 degrees to 225 degrees, and 225 degrees to 315 degrees. The numerical values 1 to 4 may be assigned to each area.

  In this case, discontinuity occurs in the numerical value at the boundary of 315 degrees. Therefore, when a vector in this direction is frequently generated, the error becomes large, so care must be taken. Therefore, if the occurrence frequency in the direction of 45 degrees is low, it is divided into four as described above, and if the occurrence frequency in the direction of 0 degrees is low, 0 degrees to 90 degrees, 90 degrees to 180 degrees, 180 degrees to 270 degrees, A direction with low occurrence frequency is brought to a discontinuous boundary, such as four divisions of 270 to 360 degrees. When it is necessary to correspond more accurately, or when the direction of low occurrence frequency is unknown, the correlation coefficient of angle statistics described in Non-Patent Document 4 can be applied.

  The second method is a method in which a set of (start point x, start point y) and (end point x, end point y) corresponding to the vector of the region having the largest area among the change regions is used as a representative vector.

  Here, a processing example of the continuous time counting unit 13 will be described. Here, a vector (a time series of numerical values corresponding to a time series of images) integrated by the integrated vector calculation unit 10 is input. At this time, the continuous time counting unit 13 counts continuous times when the integrated vector is not zero (counts frames where the temporal average of motion vectors is not zero). The counted continuous time is stored as a feature amount in a data array together with the vector integrated in the array storage unit (eg, memory) 14 and is output to the feature amount selection unit 7.

  The continuous time counted here indicates how long the moving object appears continuously. By adding this to the feature amount, the identification unit 4 keeps moving for a long time. When there is a body, this can be detected as non-stationary.

  FIG. 6 shows a processing flow of each of the above feature quantity extraction modules. Here, it is assumed that the input video frames are I (x, y, ts), I (x, y, ts + 1),, I (x, y, t). Further, “x, y” is the coordinate value of the image, t is the current time, s is the number of past frames necessary to obtain the current feature amount, and the start of processing is t = 0.

S01: First, an optical flow is calculated. Here, it is assumed that an image for which the optical flow is obtained is I _o (x, y, t) = (u (x, y, t), v (x, y, t)).

  S02. S03: In parallel with S01, a moving object region is obtained by background difference or the like (S02), and the region is labeled and the number of pixels in each region is calculated (S03).

Here, the area (number of pixels) for the n regions from the larger number of pixels to which the moving object region is labeled is defined as A ₀ (x, y, t),. . . , A _n (x, y, t).

S04: A spatial average (center coordinate, direction vector) is calculated for the image subjected to the calculation processing in S03. Here, when the pixel at coordinates (x, y) belongs to the k-th moving region at time t and the number of pixels is M, A _k (x, y, t) = M. The output of the spatial average (center coordinate, direction vector) calculation is calculated as follows (X _s (t), Y _s (t), u _s (t), v _s (t)).
X _s (t) = (A _k (x, y, t)! = 0 x average)
Y _s (t) = (average of y where A _k (x, y, t)! = 0)
_{u s (t) = 1 /} (Σ k = 1~n A k (x, y, t)) × Σ k = 1~n u (x, y, t) × A k (x, y, t)
_{v s (t) = 1 /} (Σ k = 1~n A k (x, y, t)) × Σ k = 1~n v (x, y, t) × A k (x, y, t)
Then, “t = t + 1” is set, and S101. Returning to S102, processing of the next frame is continued.

S05: A time average (center coordinate, direction vector) is calculated (S05). Here _{(X s (t-s)} , Y s (t-s), u s (t-s), v s (t-s)) ,. . . (X _s (t), Y _s (t), u _s (t), vs _s (t)) time average (X _st (t), Y _st (t), u _st (t), v _st ( t)) is determined as an average for each component.

S06 to S08: It is confirmed whether or not the motion vector (u _st (t), v _st (t)) obtained as a result of obtaining the time average in S05 is a continuous zero vector (0, 0) (S06).

  Then, the number of frames [C (t) = 0] of consecutive zero vectors (0, 0) is excluded (S07), and the number of frames that are not consecutive zero vectors (0, 0) [C (t) = C (t -1) +1] is counted (S08).

S09: The number of frames C (t) counted in S08 is added to the time average obtained in S05 (X _st (t), Y _st (t), u _st (t), v _st (t), C ( t)) is stored in the data array as a feature quantity.

(3) Identification unit 4
FIG. 7 shows a configuration example of the identification unit 4. Here, the identification unit 4 is input with time-series data of N-dimensional identification feature quantities composed of combinations of feature quantities 1 to k. The SVM discriminator 15 in FIG. 7 outputs to the feature quantity evaluation unit 6 the result of statistically discriminating the discriminating feature quantity from the input time series data using the SVM (support vector machine) method. To do. Here, as an SVM algorithm for time-series input, as described in Non-Patent Document 5, an unsupervised online identification can be used using an incremental one-class SVM.

  The feature quantity evaluation unit 16 uses the support vector group generated in the identification process using the SVM classifier 15, and uses the element for the identification in correspondence with the element of the N-dimensional feature quantity. Judge whether or not. Here, N logical values indicating ON / OFF are fed back to the SVM discriminator 15. A support vector is a subset of each input feature value and corresponds to a feature value located at the stationary / non-stationary discrimination boundary. Therefore, by checking the support vector group and the steady-state / non-stationary distribution, It can be estimated which dimension of the feature value contributes to the separation of stationary and unsteady.

  FIG. 8 shows identification using feature amounts obtained from actual images, and shows distributions of stationary, non-stationary, and support vectors on two-dimensional feature axes of the feature amounts. In FIG. 8 (a), the support vectors are distributed unevenly around a stationary island (considered as a stationary class), and non-stationary is distributed outside the stationary distribution. However, in FIG. 8B, the support vector and the stationary state overlap, and the unsteady state also has a large overlap with the stationary state.

  Therefore, the feature quantity evaluation unit 16 can be used as an indicator of whether or not the feature dimension contributes to the steady / unsteady discrimination by evaluating the degree of bias of the support vector. Here, for each dimension of the feature quantity, the ratio of the dispersion of the support vector to the stationary variance or the ratio of the non-stationary variance to the stationary variance is obtained, and the larger the ratio value, the better the separation in the feature dimension. N logical values with the high degree of separation being ON and the low dimension being OFF are fed back to the SVM discriminator 15, and the time series of the steady / non-stationary flags are displayed on the display unit 5. The data is output and displayed on the screen by the display unit 5.

  As described above, according to the non-stationary degree estimation apparatus 1, the motion feature extraction unit 3 stably extracts information based on the motion vector of the moving object in the video as a low-dimensional feature amount, and the statistical discriminator 4. Therefore, unsteady detection with high accuracy can be performed at high speed. In particular, the motion feature extraction unit 3 adds the continuity of motion to the feature amount, and averages the motion vector spatially and temporally, reflecting the magnitude of the motion of the moving object region and the size of the region itself. As a result, the feature amount is stabilized, and the amount of calculation due to the number of dimensions of the feature amount is reduced. In this respect, high-speed and low-cost unsteady detection is realized. Further, the classifier 4 uses the feature amount evaluation unit 16 to evaluate the feature amount from the support vector and the stationary / non-stationary partial distribution state, and selects an element of the feature amount that effectively contributes to the identification. The influence of noise that does not contribute to stationary / non-stationary is suppressed, and the calculation cost of the identification calculation is also reduced.

In addition, this invention is not limited to the said embodiment, A device structure etc. can be suitably changed within the range described in each claim. For example, as shown in FIG. 9, the continuous time counting unit 13 of each feature amount extraction module can be eliminated. In this case, S06 to S09 in FIG. 6 are omitted, and the vector (time series of numerical values corresponding to the time series of the image) integrated by each integrated vector calculation unit 10, that is, the time average (X _st Although (t), Y _st (t), u _st (t), and v _st (t)) are used as feature quantities, any normal moving object can be suitably detected as an unsteady degree.

  The present invention can also be constructed as a program that causes a computer to function as the non-stationary degree estimation device 1. This program may cause a computer to execute all the processes of the respective units 1 to 5 or may execute a part of the processes such as the motion feature extraction unit 3 and the identification unit 4.

  This program can be provided to a computer by downloading from a website or the like. The program is stored in a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, Blu-ray Disk (registered trademark). It may be provided to a computer. Since the program code itself read from the recording medium realizes the processing of the above embodiment, the recording medium also constitutes the present invention.

The structural example figure of the non-stationary degree estimation apparatus which concerns on embodiment of this invention. The structural example figure of the movement feature extraction part. The structural example figure of the same each feature-value extraction module. The structural example figure of the integrated vector calculation part. (A) is a schematic diagram showing the change area at time t, and the direction and magnitude of the movement of the change area, (b) is a schematic diagram showing the state shifted to time t + 1, and (c) is (a)-( (c) Schematic diagram in which change areas are superimposed, (d) is a schematic diagram showing vector integration of selected areas, and (e) and (f) are diagrams showing specific processing examples. The processing flowchart of each feature amount extraction module. The structural example figure of the identification part. (A) and (b) are comparative examples of bias of support vectors. The block diagram which shows the other examples of each feature-value extraction module.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Unsteady degree estimation apparatus 2 ... Video | video input part 3 ... Motion feature extraction part (motion feature week output means)
4 ... Identification part (identification means)
DESCRIPTION OF SYMBOLS 5 ... Display part 6 ... Feature-value extraction module group 7 ... Feature-value selection part (feature-value selection means)
8 ... Change area selection section (change area extraction means)
9: Motion vector calculation unit (motion vector calculation means)
10: Integrated vector calculation unit (integrated vector calculation means)
11 ... Motion history image generation unit (motion image generation means)
12 ... Mask processing section (mask processing means)
13 ... Continuous time counting section (continuous counting means)
14 ... Array storage unit 15 ... SVM discriminator (SVM discriminating means)
16... Feature amount evaluation unit (feature amount evaluation means)

Claims (10)

A device that extracts feature quantities based on movements of moving objects included in a video imaged by an imaging device and identifies unsteady scenes by statistical processing,
Movement feature extraction means for extracting feature amounts based on the movement of the moving object in the input video from the imaging device;
Identification means for identifying non-stationaryness of the input video from the feature amount extracted by the motion feature extraction means;
An unsteady degree estimation device comprising: The motion feature extraction unit includes a plurality of feature quantity extraction modules that generate time-series feature quantities from the input video;
Feature quantity selection means for selecting a feature quantity generated by each feature quantity extraction module based on a feature quantity selection criterion, and outputting a time series of combinations of the selected feature quantities to the identification unit;
The nonstationary degree estimation apparatus according to claim 1, further comprising: Each feature amount extraction module extracts a moving body region in the input video, and a change region extracting means for obtaining a binary image of the moving body region and a background region;
In each time-series image in the input video, a motion vector array (x.y.u) in which two-dimensional vectors indicating the position (x.y) of each pixel and the movement direction (u.v) of the pixel are paired. V) motion vector calculation means for obtaining
Integrated vector calculation means for integrating the information of the moving body region and the information of the motion vector arrangement, and obtaining the integrated vector pair as a feature amount;
The nonstationary degree estimation apparatus according to claim 2, comprising: The integrated vector calculation means includes a motion history generation means for generating a motion history image obtained by temporally superimposing the binary images in time series, applying a numerical value to the moving body region, and setting the background region to zero.
Mask processing means for performing a mask process on the motion vector array using the motion history image;
The nonstationary degree estimation apparatus according to claim 3, further comprising: Each of the feature quantity extraction modules further includes a continuous time counting unit that counts a time during which the integrated vector pair appears continuously,
The unsteady degree estimation apparatus according to any one of claims 2 to 4, wherein a feature amount obtained by adding the count time to the integrated vector pair is output to the feature amount selection unit. The identification means includes an SVM identification means for classifying an N-dimensional identification feature quantity combining the feature quantities of each feature quantity extraction module into a steady / non-stationary state using an SVM (support vector machine) method;
N logical values indicating whether or not the element is finally used for identification according to the element of the N-dimensional feature quantity from the support vector group generated by the SVM identification means and the stationary / non-stationary identification result And feature quantity evaluation means for feeding back to the identification device,
The unsteady degree estimation apparatus according to any one of claims 2 to 5, wherein a time series of steady and unsteady plugs is displayed as an identification result. A method of extracting feature quantities based on the movement of an animal body included in an image captured by an imaging device and identifying unsteadiness of a scene by statistical processing,
A motion feature extraction step, wherein the motion feature amount extraction means extracts a feature amount based on the movement of the moving object in the input video from the imaging device;
An identifying step for identifying non-stationarity of the input video from the feature amount extracted in the motion feature amount extraction step;
A non-stationary degree estimation method characterized by comprising: The motion feature extraction step includes:
Each of a plurality of feature amount extraction modules generates a feature amount in time series from the input video, and
A feature quantity selecting unit selects each feature quantity generated in the feature quantity generation step based on a feature quantity selection criterion, and outputs a time series of combinations of the selected feature quantities to the identification unit A selection step;
The nonstationary degree estimation method according to claim 7, wherein: The feature generation step includes
Extracting a moving body region in the input video, and a change region extracting step for obtaining a binary image of the moving body region and a background region;
In each time-series image in the input video, a motion vector array (x.y.u) in which two-dimensional vectors indicating the position (x.y) of each pixel and the movement direction (u.v) of the pixel are paired. V) a motion vector calculation step for obtaining
An integrated vector calculating step of integrating the information of the moving body region and the information of the motion vector arrangement, and obtaining the integrated vector pair as a feature amount;
The nonstationary degree estimation method according to claim 8, wherein:   A non-stationary degree estimation program that causes a computer to function as the non-stationary degree estimation device according to claim 1.

Images