JP2008287478A - Abnormality detection apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an abnormality detection apparatus capable of detecting with high accuracy any abnormal condition different from the normal condition from certain multidimensional feature data. <P>SOLUTION: The abnormality detection apparatus includes a feature data extraction means for extracting multidimensional feature data from input data, an angle calculation means for calculating the angle between a cone showing a normal space previously calculated in a feature data space and the feature data extracted by the feature data extraction means, and an abnormality determining means for determining the angle as an abnormality if the angle is greater than a predetermined value. The cone can be a polygonal pyramid, an approximate polygonal pyramid, a circular cone, etc. Determinations can be made with higher accuracy than before, the amount of calculations required to determine abnormalities is reduced and is constant regardless of the number of targets, and real-time processing is possible. Also, any feature vector, e.g., CHLAC data, is usable. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an anomaly detection apparatus and an anomaly detection method, and in particular, an anomaly detection apparatus capable of detecting an abnormal state different from normal from any multidimensional feature data obtained from any object to be detected with high accuracy, and The present invention relates to an abnormality detection method.

  Conventionally, as an abnormal state monitoring system, an anomaly monitoring system using a camera, such as a video monitoring in the security field and a monitoring system for elderly care, has been widely used. However, these were systems that were monitored by humans, and it took time and money. Therefore, there has been a demand for a system for automatically detecting abnormal conditions using a computer.

As a study of recognition of abnormal motion of an arbitrary object such as a person or a car, which is a kind of abnormal state, the following patent document 1 published by some of the present inventors describes a three-dimensional higher-order local autocorrelation feature ( Hereinafter, a technique for recognizing abnormal operation using CHLAC data) is disclosed. The abnormal motion detection apparatus of Patent Document 1 generates inter-frame difference data from moving image data input from a video camera, and obtains feature data from three-dimensional data composed of a plurality of inter-frame difference data by three-dimensional higher-order local autocorrelation. Extracting and calculating the distance between the latest feature data and the partial space based on the principal component vector obtained from the past feature data by the principal component analysis method. And when this distance is larger than predetermined value, it determines with it being abnormal. By learning the normal operation of the object as a partial space and detecting the abnormal operation as a deviation from the partial space, for example, even when there are a plurality of persons in the screen, it is possible to detect one abnormal operation.
JP 2006-079272 A

  In the above-described conventional abnormality detection method, whether or not there is an abnormality is determined based on the distance between a partial space indicating a normal region based on the principal component vector obtained by the principal component analysis method and the latest feature data. However, there is a problem that there may be characteristic data that should be judged as abnormal in the vicinity of the subspace indicating the normal region based on the principal component vector obtained by the principal component analysis method or in the subspace. .

  The object of the present invention is to solve such a problem, and not only the feature data relating to the operation but also the abnormal state different from the normal from any multidimensional feature data (feature vector) obtained from any target to be detected is high. The object is to provide an abnormality detection device and an abnormality detection method that can be detected with high accuracy.

  The abnormality detection apparatus according to the present invention includes feature data extraction means for extracting multidimensional feature data from input data, a cone indicating a normal space obtained in advance in the feature data space, and the feature data extraction means. The main features include an angle calculating means for calculating an angle with the extracted feature data, and an abnormality determining means for determining an abnormality when the angle is larger than a predetermined value.

In the above-described abnormality detection apparatus, the cone-shaped body indicating the normal space is also characterized in that it is a polygonal pyramid whose sides are defined by a plurality of basis vectors.
In the above-described abnormality detection apparatus, the polygonal pyramid is a distribution of normal samples in a normal subspace in which each side is defined by a basis vector obtained by a non-negative matrix factorization method. Another feature is that it is a polygonal cone that approximates the range.

  In the above-described abnormality detection device, the polygonal pyramid is normal in a normal subspace in which each side is defined by a plurality of basis vectors generated based on a principal component vector obtained from a normal sample by principal component analysis. Another feature is that it is a polygonal cone that approximates the distribution range of the sample.

  In the above-described abnormality detection device, the cone-shaped body indicating the normal space is also characterized in that it is a cone defined by direction vector information indicating the center of the cone and angle information indicating the spread of the cone. Further, the above-described abnormality detection device is further characterized in that it further includes scale normalizing means for normalizing the scale of the feature data.

  Further, the abnormality detecting apparatus described above further includes a learning unit that calculates a cone that indicates a normal space from a plurality of feature data extracted by the feature data extracting unit based on input data for learning. There is also a feature.

  The abnormality detection apparatus further includes a cluster dividing unit that divides the feature data into a plurality of clusters, and the learning unit calculates a cone that indicates a normal space for each divided cluster, and the abnormality determination The means is characterized in that the determination is performed for each cluster, and the feature data is determined to be abnormal only when the feature data is determined to be abnormal in all clusters.

  Further, in the above-described abnormality detection device, the feature data extraction unit generates inter-frame difference data from moving image data including a plurality of input image frame data, and three-dimensional data including the plurality of inter-frame difference data. Another feature is that the motion feature data is generated by three-dimensional higher-order local autocorrelation.

  The abnormality detection method of the present invention includes a step of extracting feature data from input data, a cone indicating a normal space obtained in advance in the feature data space, and feature data extracted by the feature data extraction means. The main feature includes a step of calculating an angle and a step of determining an abnormality when the angle is larger than a predetermined value.

The present invention has the following effects.
(1) It is possible to determine an abnormality even for an event that has been determined to be normal even though it is abnormal in the past, and the accuracy of the abnormality determination is improved.
(2) The amount of calculation for feature extraction and abnormality determination is small, the amount of calculation is constant regardless of the number of objects, and real-time processing is possible.

(3) It can cope with various monitoring targets, and any arbitrary vector such as CHLAC data can be adopted as a feature vector.
(4) In the case where CHLAC is used to perform abnormality determination by angle, abnormality can be detected robustly on the target scale.
(5) The present invention is applicable not only to abnormality detection but also to general recognition tasks (face recognition, human recognition, etc.).

  First, as an example of the present invention, the detection of abnormal operation using CHLAC data is disclosed as an example, but the present invention is not limited to CHLAC data, and any multidimensional feature obtained from any target to be detected. From the data (feature vector), not only abnormal operation but also an abnormal state different from normal, that is, an event having a small occurrence probability can be detected with high accuracy.

  Next, although “abnormal” is defined, the abnormality itself cannot be defined so that all abnormal events cannot be listed. Therefore, in this specification, an abnormality is defined as “not normal”. If normal is a region where the distribution is concentrated when considering the statistical distribution of features, for example, it is possible to learn from the statistical distribution. Then, an operation that greatly deviates from the distribution is regarded as abnormal.

  For example, in a security camera installed on a passage or road, general motion such as walking is learned and recognized as normal motion, but motion like suspicious behavior is periodic motion like walking motion. Instead, it is an event that has a very low probability of occurring, and is therefore recognized as an abnormality.

  In the present invention, it is found that normal feature vectors are distributed in a normal space region in which one or a plurality of (hyper-solid) polygonal pyramid shapes or conical regions are combined in the feature vector space. It is judged whether the vector exists inside one of the above-mentioned one or plural polygonal cones or cone-shaped regions, and is normal if it exists inside any one of the polygonal cones or cones. Otherwise, it is determined as abnormal. Hereinafter, detection of abnormal operation using CHLAC data will be described as an example of an embodiment of the present invention.

  FIG. 1 is an explanatory diagram illustrating a determination method of Example 1 (polygonal pyramid) of the present invention. FIG. 1 shows the CHLAC feature data space in three dimensions (actually, for example, 251 dimensions) for ease of explanation, and the normal operation space is a polygonal pyramid containing normal operation CHLAC feature data. Exists inside.

  FIG. 1B shows an arrangement example in a cross section when cut by a plane intersecting a sample feature vector x to be determined and a base vector representing each side of the polygonal pyramid, and FIG. 1C shows an origin and a sample. A cross section is shown in the case of cutting along a plane including a perpendicular drawn from the feature vector x to the surface of the nearest polygonal pyramid.

  When the surface of the polygonal pyramid closest to the feature vector x is a plane between the sides (base vectors) of the polygonal pyramid (range A in FIG. 1B), the angle θ is the plane and the feature vector. The angle formed by x. When the surface of the polygonal pyramid closest to the feature vector x is a side (basis vector) (range B in FIG. 1B), the angle θ is the difference between the side (base vector) and the feature vector x. The angle to make.

  Accordingly, a line connecting the positions of the feature vectors having the same angle θ is a position like a line L shown in FIG. In the feature data of the abnormal operation, the angle θ with the surface of the polygonal pyramid indicating the normal operation space is large. Therefore, whether the angle θ is equal to or greater than a predetermined threshold is determined.

  FIG. 5 is a block diagram showing the configuration of the abnormality detection apparatus according to the present invention. The video camera 10 outputs moving image frame data of a target person or device in real time. The video camera 10 may be a monochrome camera or a color camera. The computer 11 may be a known personal computer (PC) provided with a video capture circuit for capturing a moving image, for example. The present invention is realized by creating and installing a program to be described later on an arbitrary known computer 11 such as a personal computer.

  The monitor device 12 is a well-known output device of the computer 11 and is used, for example, to display to the operator that an abnormal operation has been detected. The keyboard 13 and the mouse 14 are well-known input devices used for input by the operator. In the embodiment, for example, moving image data input from the video camera 10 may be processed in real time, or may be temporarily stored after being stored in a moving image file and processed.

  FIG. 6 is a flowchart showing the processing contents at the time of detection of the abnormality detection processing of the present invention. In S10, it waits until the input of the frame data from the video camera 10 is completed, and the frame data is input (read into the memory). The image data at this time is, for example, 256 gray scale data.

  In S11, a feature vector is generated from the input frame data by a method described later. The feature vector may be, for example, 251 dimensional CHLAC feature data described later. In S12, as shown in Equation 1 below, the feature vector is normalized by a vector norm. That is, the feature vector x is divided by the vector norm (vector length, | x |).

  In S13, an unprocessed cluster is selected. The number of clusters, the basis vector xb of each cluster, and the threshold value are obtained in advance by a learning process described later.

  In S14, the angle between the surface of the polygonal pyramid and the input feature vector is obtained by the following method. If the above-described basis vectors are obtained, the processing speed is increased. However, in the following method, all normal feature vectors obtained at the time of learning may be used as they are as the basis vectors. Assuming that the set of basis vectors is Xb and the normalized input vector to be determined is y, the convex polygonal pyramid (S) and the angle θ to the polygonal pyramid are expressed by the following Equation 2.

By converting cos θ into sin θ, Equation 2 can be transformed as shown in Equation 3. If α that minimizes the last equation in Equation 2 is obtained, X _b α becomes a projection vector onto the surface of the polygonal cone, θ is obtained. However, when the minimum angle θ formed by the input vector and the vector in the polygonal cone is larger than π / 2, that is, when ∀λ> π / 2, the solution of the last equation is α = 0. Therefore, when the obtained solution is α = 0, the minimum angle is θ = π / 2 (upper limit).

Since the last expression of Equation 3 is a well-known convex quadratic programming problem, θ can be obtained by solving the convex quadratic programming problem. In order to solve the convex quadratic programming problem by a computer, for example, the quadprog function of Optimization Toolbox of the scientific calculation software MATLAB (registered trademark) system sold by Cybernet System Co., Ltd. can be used. The algorithm for the convex quadratic programming problem is described in Non-Patent Documents 1 and 2 below, for example.
Coleman, TF and Y. Li, OA Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on some of the Variables, O SIAM Journal on Optimization, Vol. 6, Number 4, pp. 1040-1058, 1996. Gill, PE and W. Murray, and MH Wright, Practical Optimization, Academic Press, London, UK, 1981.

  The angle θ can also be obtained by the following method. Since the equation in the second line from the bottom of Equation 3 is a well-known NNLS (Non-negative Least Squares) problem, θ can be obtained by solving the NNLS problem.

  In order to solve the NNLS problem with a computer, for example, the nnnls function of Optimization Toolbox of the scientific calculation software MATLAB (registered trademark) system sold by Cybernet System Co., Ltd. can be used.

The algorithm of the NNLS problem is described in Non-Patent Documents 3 and 4 below, for example.
Bro, R., de Jong, S., 1997. A fast non-negativity constrained linear least squares algorithm. Journal of Chemometrics 11, 393-401. Lawson, C., Hanson, R., 1995. Solving Least Squares Problems. SIAM, Philadelphia, PA.

  In S15, it is determined whether or not the calculated angle θ is less than the abnormality determination threshold value. If the determination result is negative, the process proceeds to S16, but if the determination result is affirmative, the process proceeds to S17. The threshold is determined during learning. In S16, it is determined whether or not the processing has been completed for all clusters. If the determination result is negative, the process proceeds to S13, but if the determination is affirmative, the process proceeds to S18.

  In S17, “normal” is output, and in S18, “abnormal” is output. In S19, it is determined whether or not the end of the process is instructed. If the determination result is negative, the process proceeds to S10 and the process is repeated. It is determined whether or not the sample data input by the above detection process is abnormal.

  FIG. 7 is a flowchart showing the processing contents during learning of the abnormality detection processing according to the first embodiment of the present invention. This process is executed prior to the detection process of FIG. In S30, sample frame data for motion learning is input. In S31, a feature vector is generated as in S11. In S32, normalization by a vector norm is performed as in S12. In S33, the feature vectors are clustered by the Mean Shift method.

  FIG. 14 is a flowchart showing the contents of the clustering process in S33. The inventors invented the Mean Shift method using the von Mises-Fisher distribution. This will be described below. In S120, one unprocessed sample point is selected as the initial value of the reference point ξ. In S121, the reference point ξ is updated based on the update formula shown in Formula 4. κ is a parameter representing the degree of concentration in the von Mises-Fisher distribution, and is set by experiment.

  In S122, it is determined whether or not ξ has changed, that is, whether or not the absolute value of the difference between ξ after the update and ξ before the update is equal to or less than a predetermined value. If the determination result is negative, the process returns to S121. If yes, the process proceeds to S123. In S123, it is determined whether or not processing has been completed for all sample points. If the determination result is negative, the process proceeds to S120, but if the determination is affirmative, the process proceeds to S124. Through the above processing, the reference point ξ corresponding to each sample point is obtained. In S124, the points at close positions are integrated. That is, a set of samples having reference points whose difference between reference points ξ is equal to or less than a predetermined value is regarded as one cluster.

  Returning to FIG. 7, in S34, one unprocessed cluster is selected. In S35, the base vector of the polygonal pyramid is obtained by the following method, for example. A polygonal pyramid showing a normal feature vector distribution (outermost boundary surface) can be expressed by a feature vector representing each vertex = a base vector, and the shape of the polygonal pyramid is unchanged even if the feature vector inside the polygonal pyramid is removed. is there. Therefore, by using a well-known Leave One Out Method, feature vectors existing inside the polygonal pyramid (not serving as a vertex of the polygonal pyramid) are removed to obtain a base vector.

FIG. 2 is an explanatory diagram showing the leave one-out method. Be removed non-independent samples x _t from a set of samples, as shown in FIG. 2, the shape of the polygonal pyramid is unchanged. Therefore, the non-independent sample _xt is removed by the method described below, and only the independent sample _xb is left.

FIG. 16 is a flowchart showing the contents of the base vector extraction processing by the leave one-out method of S35. Suppose that there is a sample set X = x ₁ ,..., X _N consisting of N feature vectors. In S150, the variable i is set to 1, and in S151, the minimum angle between the polygonal cone formed by the set X˜ excluding x _i by solving the convex quadratic programming problem or the NNLS problem and the vector x _i ψ _i is calculated. In S152, 1 is added to the variable i. In S153, it is determined whether or not the variable i is greater than N. If the determination result is negative, the process proceeds to S151. If the determination is positive, the process proceeds to S154. Transition.

In S154, the variable i is set to 1. In S155, it is determined whether or not the angle ψ _i is larger than a predetermined threshold value s (positive small value). If the determination result is negative, the process proceeds to S157. However, if the determination is affirmative, the process proceeds to S156. In S156, it registers the vector x _i as base vector x _b. Here, if ψ _i is too large (more than the threshold value t), it is not registered in the basis vector as an outlier. Eventually, only those with s <ψ _i <t are registered.
In S157, 1 is added to the variable i. In S158, it is determined whether or not the variable i is greater than N. If the determination result is negative, the process proceeds to S155. finish. By applying the leave-one-out method, the number of basis vectors becomes very small compared to the number of original feature vectors.

  Returning to FIG. 7, in S <b> 36, it is determined whether or not the processing has been completed for all clusters. If the determination result is negative, the process proceeds to S <b> 34, but if the determination is positive, the process ends. Through the learning process as described above, one or a plurality of polygonal pyramid basis vectors indicating the normal range are obtained.

  Next, details of the processing will be described. FIG. 12 is a flowchart showing the contents of the feature vector generation process of S11. In S100, difference data is generated and binarized. In S100, "motion" information is detected for the moving image data, and difference data is generated for the purpose of removing stationary objects such as the background. As a method of taking the difference, an inter-frame difference method that extracts a change in luminance of a pixel at the same position between adjacent frames is adopted, but an edge difference that extracts a luminance change portion in a frame or both are adopted. May be.

Further, binarization is performed by automatic threshold selection to remove color information and noise unrelated to “movement”. As a binarization method, for example, a fixed threshold, a discriminative least square automatic threshold method disclosed in Non-Patent Document 5 below, a threshold 0 and a noise processing method (all differences except for 0 in a grayscale image are set to have motion = 1). A method of removing noise by a known noise removing method can be employed. By the above preprocessing, the input moving image data becomes a string of frame data having logical values of “moved (1)” and “not moved (0)” as pixel values.
Noriyuki Otsu, “Automatic threshold selection method based on discriminant and least-squares criteria” IEICE Transactions D, J63-D-4, P349-356, 1980.

  In S101, correlation pattern count processing for pixel data for one frame is performed to generate frame CHLAC data. Although details will be described later, 251-dimensional feature vector data is generated. For extraction of motion features from time-series binary difference data, cubic higher-order local autocorrelation (CHLAC) features are used. The Nth-order autocorrelation function is as shown in Equation 5 below.

Here, f is a time-series pixel value (difference value), and N displacements a _i (i = 1,..., N) viewed from the reference point (target pixel) r and the reference point are represented in the difference frame. A three-dimensional vector having time as a component in two-dimensional coordinates. Further, the integration range in the time direction is a parameter indicating how much correlation in the time direction is taken. However, the frame CHLAC data in S101 is data for each frame, and CHLAC feature data is obtained by integrating (adding) this for a predetermined period in the time direction.

  There are an infinite number of higher-order autocorrelation functions depending on the direction of displacement and the order, but the higher-order local autocorrelation function is limited to a local region. In the three-dimensional high-order local autocorrelation feature, the displacement direction is limited to a 3 × 3 × 3 pixel local area centered on the reference point r, that is, in the vicinity of 26 of the reference point r. In the calculation of the feature amount, the integral value of Expression 1 becomes one feature amount for one set of displacement directions. Therefore, feature amounts are generated as many as the number of combinations of displacement directions (= mask patterns).

  The number of feature amounts, that is, the dimension of the feature vector corresponds to the type of mask pattern. In the case of a binary image, since the pixel value 1 is multiplied by 1 regardless of how many times it is, a term of square or higher is deleted as an overlapping term of a square with a different multiplier only. In addition, patterns that overlap in the integration operation (parallel movement: scan) of Formula 5 are deleted while leaving one representative pattern. Since the expression on the right side of Expression 1 always includes a reference point (f (r): the center of the local area), the representative pattern includes the center point, and the entire pattern fits within the local area of 3 × 3 × 3 pixels. select.

  As a result, the number of types of mask patterns including the center point is 352 with a total number of selected pixels of one: one, two: 26, three: 26 × 25/2 = 325. However, there are 251 types of mask patterns except for overlapping patterns in the integration operation (parallel movement: scan) of Equation 1. That is, the three-dimensional higher-order local autocorrelation feature vector for one three-dimensional data is 251 dimensions.

  When the pixel value is a multi-value gray image, for example, when the pixel value is a, the correlation value is a (0th order) ≠ a × a (primary) ≠ a × a × a (secondary). Thus, even if the selected pixels are the same, those having different multipliers cannot be redundantly deleted. Therefore, in the case of multi-value, the number is 2 when the number of selected pixels is 1 and 26 when the number of selected pixels is 2, and the number of mask patterns is 279 in total.

  The nature of the cubic higher-order local autocorrelation feature is additive to the data because the displacement direction is limited to the local region, and position invariance in the data because it is integrated throughout the entire screen.

  In S102, the frame CHLAC data is stored in correspondence with the frame. In S103, the latest frame CHLAC data obtained in S101 is added to the current CHLAC data, and frame CHLAC data corresponding to a frame older than a predetermined period is subtracted from the current CHLAC data to generate new CHLAC data. ,save.

  FIG. 19 is an explanatory diagram showing the contents of real-time processing of a moving image according to the present invention. The moving image data is a sequence of frames. Therefore, a time window having a certain width is set in the time direction, and a frame set in the window is set as one three-dimensional data. Each time a new frame is input, the time window is moved, and the old frame is deleted to obtain finite three-dimensional data. The length of this time window is desirably set equal to or longer than one cycle of the operation to be recognized.

  Actually, only one frame is stored in order to obtain the difference in the image frame data, and the frame CHLAC data corresponding to the frame is stored for the time window. That is, when a new frame is input at time t in FIG. 19, the frame CHLAC data corresponding to the immediately preceding time window (t−1, t−n−1) has already been calculated. However, in order to calculate the frame CHLAC data, the last three difference frames are required, but since the (t-1) frame is the end, the frame CHLAC data is calculated up to the one corresponding to the (t-2) frame. Yes.

  Therefore, using the newly input t frame, frame CHLAC data corresponding to the (t−1) frame is generated and added to the CHLAC data. Also, the frame CHLAC data corresponding to the oldest (t-n-1) frame is subtracted from the CHLAC data. Through such processing, the CHLAC feature data corresponding to the time window is updated.

  FIG. 13 is a flowchart showing the contents of the three-dimensional higher-order local autocorrelation feature extraction process in S101. In S110, 251 correlation pattern counters are cleared. In S111, one unprocessed pixel is selected (the target pixel is sequentially scanned in the frame). In S112, one unprocessed mask pattern is selected.

  FIG. 17 is an explanatory diagram showing autocorrelation processing coordinates in a three-dimensional pixel space. In FIG. 17, the xy planes of three difference frames temporally adjacent to each of the t−1 frame, the t frame, and the t + 1 frame are shown side by side. In the present invention, correlation is performed for pixels inside a cube of 3 × 3 × 3 (= 27) pixels centered on the reference pixel of interest.

  The mask pattern is information indicating a combination of pixels to be correlated, and the pixel data selected by the mask pattern is used for calculating the correlation value, but the pixels not selected by the mask pattern are ignored. As described above, the target pixel (center pixel) is always selected in the mask pattern. Further, when considering correlation values from the 0th order to the 2nd order in a binary image, the number of patterns after eliminating the overlap in a 3 × 3 × 3 pixel cube is 251.

  FIG. 18 is an explanatory diagram showing an example of an autocorrelation mask pattern. FIG. 18A shows the simplest 0th-order mask pattern of only the target pixel. (2) is a primary mask pattern example in which two hatched pixels are selected, (3) and (4) are tertiary mask pattern examples in which three hatched pixels are selected, There are many other patterns.

Returning to FIG. 13, in S113, the correlation value is calculated using Equation 5 described above. F (r) f (r + a ₁ )... F (r + a _N ) in Equation 5 is multiplied by the pixel value of the difference binarized three-dimensional data of the coordinates corresponding to the mask pattern (= correlation value, 0 or 1) It corresponds to that. Further, the integration operation of Equation 5 corresponds to moving (scanning) the pixel of interest within the frame and adding the correlation value with a counter corresponding to the mask pattern (counting 1).

  In S114, it is determined whether or not the correlation value is 1. If the determination result is affirmative, the process proceeds to S115. If the determination result is negative, the process proceeds to S116. In S115, the correlation pattern counter corresponding to the mask pattern is incremented by one. In S116, it is determined whether or not the processing has been completed for all patterns. If the determination result is affirmative, the process proceeds to S117. If the determination result is negative, the process proceeds to S112.

  In S117, it is determined whether or not the processing has been completed for all the pixels. If the determination result is affirmative, the process proceeds to S118, but if the determination result is negative, the process proceeds to S111. In S118, a set of pattern counter values is output as 251 dimensional frame CHLAC data.

  FIG. 3 is an explanatory diagram illustrating a determination method according to the second embodiment (approximate polygonal pyramid 1) of the present invention. In FIG. 3, in order to simplify the explanation, the CHLAC feature data space is three-dimensional (actually, for example, 251 dimensions). In the first embodiment, a polygonal pyramid including the distribution range of normal feature vectors is obtained. In the second embodiment, a polygonal pyramid that approximates the distribution range of normal feature vectors at the time of learning is obtained. A pyramid is used to determine whether or not there is an abnormality as in the first embodiment. Examples of how to obtain approximate polygonal pyramids will be described below.

<How to find an approximate polygonal pyramid 1>
FIG. 8 is a flowchart showing the processing contents at the time of learning of the abnormality detection processing of the present invention using the method 1 for obtaining the approximate polygonal pyramid. The only difference from the flowchart of the first embodiment shown in FIG. 7 is that S35 and S45 are different, and the processing of the other steps is the same, so the processing of S45 will be described. In the method 1 for obtaining an approximate polygonal pyramid, a polygonal pyramid having a small number of basis vectors and a normal feature vector distribution range is obtained. The number of corners is arbitrary. However, in the approximate polygonal pyramid 1, it is assumed that all the elements of the feature vector are non-negative. For example, CHLAC feature data or the like satisfies this non-negative condition, and data that does not satisfy this condition may be dealt with by preprocessing such as rotating the distribution so that it becomes a non-negative region. Note that methods other than the approximate polygonal pyramid 1 in the present invention do not make such a premise and can be applied to arbitrary feature vectors.

Specifically, for example, the following equation 6 is solved by a known non-negative matrix factorization method as disclosed in the following non-patent document 6 to obtain an approximate polygonal pyramid basis. Find a vector. X is a feature vector group constituting a polygonal pyramid, W is a basis vector of an approximate polygonal pyramid, and H is a coupling coefficient of the basis vectors.
Lee, D., & Seung, H. (1999). Learning the parts of objects by non-negative matrix factorization.Nature, 401, 788-791.

  Specifically, the processing shown in the following formula 7 is performed. (1) Initial values of W and H are set at random. (2) W and H are updated according to the equation as shown in Equation 7. (3) If the determination formula shown in (3) has converged (the change in value was less than or equal to a predetermined value before and after the update), the process is terminated, and W becomes the basis vector of the approximate polygonal pyramid. If not, W and H are normalized by the update formula shown in (3), and the process proceeds to (2).

  At the time of learning, an approximate polygonal pyramid is obtained by the above process, and at the time of detection, an abnormality is determined by the process shown in FIG.

  FIGS. 3B and 3C are explanatory diagrams illustrating a determination method according to the third embodiment (approximate polygonal pyramid 2) of the present invention. In FIG. 3, in order to simplify the description, the CHLAC feature data space is three-dimensional (actually, for example, 251 dimensions). In the third embodiment, an approximate polygonal pyramid is obtained in the same manner as in the second embodiment described above, but the method of obtaining it is different, and the obtained approximate polygonal pyramid approximates the distribution range of normal feature vectors. Examples of how to obtain an approximate pyramid will be described below.

<How to find an approximate polygonal pyramid 2>
FIG. 3B is an explanatory diagram illustrating a determination method according to the third embodiment. In Example 3 (how to obtain an approximate polygonal pyramid 2), a polygonal pyramid basis vector is calculated by a known principal component analysis. The first eigenvector obtained as a result of the principal component analysis indicates the center direction of the polygonal pyramid, and the second and subsequent eigenvectors indicate the spreading direction of the polygonal pyramid perpendicular to the first eigenvector. Therefore, an approximate polygonal pyramid basis vector is calculated from the distribution of feature vectors on the axis of the eigenvector after the second eigenvector.

  FIG. 9 is a flowchart showing the processing contents at the time of learning of the abnormality detection processing of the present invention using the method 2 for obtaining the approximate polygonal pyramid. The only difference from the flowchart of the first embodiment shown in FIG. 7 is that S35 is replaced by S55 and S56, and therefore the processing of S55 and S56 will be described.

  In S55, a principal component vector is output by principal component analysis. Since the principal component analysis method itself is well known, an outline will be described. First, in order to construct a partial space including a polygonal pyramid, a principal component vector is obtained from a CHLAC feature data group by principal component analysis. The M-dimensional CHLAC feature vector x is expressed as follows.

  Note that M = 251. A matrix U (eigenvector) in which the principal component vectors are arranged in a column is expressed as Equation 9 below.

  Note that M = 251. The matrix U in which the principal component vectors are arranged in columns is obtained as follows. The covariance matrix R is shown in Equation 10 below.

  The matrix U is obtained from the eigenvalue problem of the following equation 11 using this covariance matrix R.

  The diagonal matrix Λ of eigenvalues is expressed by the following equation 12.

The cumulative contribution rate β _K up to the K-th eigenvalue is expressed as Equation 13 below.

Here, the base vector of the polygonal pyramid is calculated using eigenvectors u ₁ ,..., U _K up to a dimension where the cumulative contribution rate β _K becomes a predetermined value (for example, β _K = 0.99). Note that the optimum value of the cumulative contribution rate β _K is determined by experiments or the like because it is considered to depend on the monitoring target and detection accuracy.

  In S56, an approximate polygonal pyramid basis vector is calculated from the obtained principal component vector. Two methods for calculating an approximate polygonal pyramid basis vector from the principal component vector will be described below.

<How to find an approximate polygonal pyramid 2A>
FIG. 3C is an explanatory diagram illustrating a first calculation method of the method 2 for obtaining an approximate polygonal pyramid. In the first calculation method, a point having a distance k times the standard deviation σ from the center of the distribution of the feature vector projected onto the i-th eigenvector (i> 1) is set as a base vector. That is, the (2i−3) th basis vector = e ₁ + kσe _i and the (2i−2) th basis vector = e ₁ −kσe _i . Note that e _i is the i-th eigenvector, and k is an arbitrary real number whose optimum value is determined by experiment, for example, but may be k = 3, for example.

  According to this method, the number of basis vectors that is twice the number of dimensions by principal component analysis is obtained. In addition, by using a point having a distance k times as large as the standard deviation σ as a base vector, the influence of noise, outlier sample, or the like is reduced.

<How to find an approximate polygonal pyramid 2B>
The principal component analysis is performed in the same manner as the above-described method 2A for obtaining the approximate polygonal pyramid, but the normal range is manually set from the distribution of the feature vectors projected onto the i-th eigenvector (i> 1), for example. That is the first (2i-3) th basis vector _{= e 1 + L 1 e i} , the (2i-2) th basis vector = e ₁ -L ₂ e _i. Note that e _i is the i-th eigenvector. L ₁ and L ₂ are manually set parameters, and may be, for example, the maximum value and the minimum value of the sample projected on each axis. In this method as well, basis vectors having twice the number of dimensions obtained by principal component analysis can be obtained.

  At the time of learning, an approximate polygonal pyramid is obtained by the above process, and at the time of detection, an abnormality is determined by the process shown in FIG.

  FIG. 4 is an explanatory diagram illustrating a determination method according to the fourth embodiment (cone) of the present invention. In FIG. 4, in order to simplify the description, the CHLAC feature data space is three-dimensional (actually, for example, 251 dimensions). In the fourth embodiment, a cone (direction vector γ and spread angle φ) including a normal feature vector distribution range is obtained, and an angle θ between the cone and sample feature data x is obtained to determine whether or not there is an abnormality. Is. Examples will be described below.

  FIG. 10 is a flowchart showing the processing contents at the time of detection in the embodiment 4 of the abnormality detection processing of the present invention. Since the difference from the flowchart of the first embodiment shown in FIG. 6 is only that S14 is replaced from S64 to S66, the processing of the difference will be described.

  In S64, the scale of each element of the feature vector is adjusted so that the normalized feature vector y has an isotropic distribution on a sphere centered on the origin. Assuming that U and Λ are eigenvectors and eigenvalue diagonal matrices obtained by principal component analysis in the learning process described later, the vector ~ z after the scaling process is expressed by Equation 14.

  In S65, the vector ~ z is normalized by the vector norm to become z (|| z || = 1) by the same processing as S12. In S66, the angle between the feature vector z and the direction vector γ as shown in Equation 15 is calculated, and the value obtained by subtracting φ is the angle θ to the cone (cone).

  FIG. 11 is a flowchart showing the processing contents during learning in the fourth embodiment of the abnormality detection processing of the present invention. Since the only difference from the flowchart of the first embodiment shown in FIG. 7 is S85 to S88, the processing of the difference will be described. In S85, principal component analysis is performed to obtain an eigenvector U and an eigenvalue diagonal matrix Λ.

  In S86, the scale of each element of the feature vector is adjusted by the same processing as in S64. In S87, normalization by the vector norm is performed by the same process as in S65. In S88, the direction vector γ and the spread angle φ are obtained by a 1-class SVM (1-class Support Vector Machine) method.

  FIG. 15 is a flowchart showing the contents of the direction vector calculation process in S88. Since the 1-class SVM method itself is well known, an outline will be described. In FIG. 4C, the circle on the (hyper) spherical surface defined by the direction vector γ and the spread angle φ can be expressed as an intersection line of the (hyper) sphere and the (hyper) plane. Therefore, if the projection point of the sample (normalized to norm 1) and the origin are separated and the (super) plane b having the maximum distance from the origin is obtained, the normal vector to the hyperplane b and From the distance from the origin of the (hyper) plane, the direction vector γ and the angle range φ are obtained. That is, this is equivalent to the problem of 1-class SVM, and the problem of 1-class SVM shown in the following equation 16 should be solved. Equation 16 is a convex quadratic programming problem to be finally solved in the 1-class SVM problem.

  Note that ν is a parameter designated by the user, and may be, for example, ν = 0.01. As described above, the above problem can also be solved using MATLAB of Cybernet System Co., Ltd., but here, for example, it can also be specifically solved as follows. In S130, the variable t = 1 is set, and δ is initialized randomly within a range satisfying the condition of Expression 17.

  In S131, reserve variables O, SV, and ρ are calculated as shown in Equation 18.

  In S132, iεSV satisfying the condition of Expression 19 is selected.

  In S133, it is determined whether or not i is present. If the determination result is negative, the process proceeds to S134, but if the determination is affirmative, the process proceeds to S136. In S134, i∈1,..., N satisfying the condition of Expression 19 is obtained. In S135, it is determined whether or not i is present. If the determination result is negative, the process ends. If the determination is affirmative, the process proceeds to S136. In S136, j is obtained based on Equation 20.

  In S137, δ is updated based on Equation 21.

  In S138, it is determined whether or not δ satisfies the condition shown in Expression 22. If the determination result is negative, the process proceeds to S139, but if the determination is affirmative, the process proceeds to S140.

In S139, δ _j is projected so as to fall within a predetermined range [0, 1 / νN]. That is, if δ> 1 / νN, δ = 1 / νN, and if δ <0, δ = 0, and the robot moves to the nearest point (end point). Then, δ _i is recalculated. In S140, t = t + 1 and the process proceeds to S131. The normal vectors γ and φ are obtained by the following Expression 23 using δ and ρ obtained by the above processing.

  Returning to FIG. 11, in S89, it is determined whether or not the processing has been completed for all clusters. If the determination result is negative, the process proceeds to S84, but if the determination is positive, the process ends. In the fourth embodiment, the calculation load during detection is light and processing can be performed at high speed.

  As described above, the embodiment for detecting the abnormal operation has been described. However, the present invention may be modified as follows. In the embodiment, an example in which information on a polygonal cone or cone indicating a normal operation area is generated in advance by the learning phase has been disclosed. However, abnormal information is detected while updating information on the polygonal cone or cone. For example, it is possible to follow changes in the environment (changes in the normal state).

  Further, for example, polygonal cone or cone information is updated using a part of data at a predetermined period longer than a frame interval such as 1 minute, 1 hour or 1 day, and the polygonal pyramid is fixed until the next update. Or you may detect abnormal operation | movement using the information of a cone. In this way, the amount of processing in learning is reduced.

It is explanatory drawing which shows the determination method of Example 1 (polygonal pyramid) of this invention. It is explanatory drawing which shows the leave one out method. It is explanatory drawing which shows the determination method of Example 2 (approximate polygonal pyramid 1) of this invention. It is explanatory drawing which shows the determination method of Example 4 (cone) of this invention. It is a block diagram which shows the structure of the abnormality detection apparatus by this invention. It is a flowchart which shows the processing content at the time of the detection of the abnormality detection process of this invention. It is a flowchart which shows the processing content at the time of learning of the abnormality detection process of Example 1 of this invention. It is a flowchart which shows the processing content at the time of learning of the abnormality detection process of this invention using the calculation method 1 of an approximate polygonal pyramid. It is a flowchart which shows the processing content at the time of learning of the abnormality detection process of this invention using the calculation method 2 of an approximate polygonal pyramid. It is a flowchart which shows the processing content at the time of the detection of Example 4 of the abnormality detection process of this invention. It is a flowchart which shows the processing content at the time of learning of Example 4 of the abnormality detection process of this invention. It is a flowchart which shows the content of the feature vector generation process of S11. It is a flowchart which shows the content of the solid high-order local autocorrelation feature extraction process of S101. It is a flowchart which shows the content of the clustering process of S33. It is a flowchart which shows the content of the direction vector calculation process of S88. It is a flowchart which shows the content of the base vector extraction process by the leave one out method of S35. It is explanatory drawing which shows the autocorrelation process coordinate in a three-dimensional pixel space. It is explanatory drawing which shows the example of an autocorrelation mask pattern. It is explanatory drawing which shows the content of the real-time process of the moving image by this invention.

Explanation of symbols

x ... sample vector θ ... angle 10 which is an abnormality determination index ... video camera 11 ... computer 12 ... monitor device 13 ... keyboard 14 ... mouse

Claims (10)

Feature data extraction means for extracting multidimensional feature data from input data;
Angle calculation means for calculating the angle between the cones indicating the normal space obtained in advance in the feature data space and the feature data extracted by the feature data extraction means;
Abnormality determining means for determining an abnormality when the angle is greater than a predetermined value;
An abnormality detection device characterized by comprising:   The anomaly detection apparatus according to claim 1, wherein the cone-shaped body indicating the normal space is a polygonal pyramid whose sides are defined by a plurality of basis vectors.   3. The abnormality detecting device according to claim 2, wherein the polygonal pyramid is a polygonal pyramid whose sides are defined by basis vectors obtained by a non-negative matrix factorization method. .   The polygonal pyramid is a polygonal pyramid that approximates the distribution range of normal samples in a normal subspace, in which each side is defined by a plurality of basis vectors generated based on principal component vectors obtained from normal samples by principal component analysis. The abnormality detection device according to claim 2, wherein:   The anomaly detection apparatus according to claim 1, wherein the cone-shaped body indicating the normal space is a cone defined by direction vector information indicating a center of the cone and angle information indicating a spread of the cone.   6. The abnormality detection apparatus according to claim 5, further comprising scale normalizing means for normalizing the scale of the feature data.   2. The learning apparatus according to claim 1, further comprising learning means for calculating a cone representing a normal space from a plurality of feature data extracted by the feature data extraction means based on input data for learning. Anomaly detection device. Furthermore, a cluster dividing means for dividing the feature data into a plurality of clusters is provided,
The learning means calculates a cone indicating a normal space for each divided cluster,
The abnormality determination unit performs determination for each cluster, and determines the feature data as abnormal only when the feature data is determined as abnormal in all clusters. Anomaly detection device.   The feature data extracting unit generates inter-frame difference data from moving image data composed of a plurality of input image frame data, and operates by three-dimensional high-order local autocorrelation from three-dimensional data composed of the plurality of inter-frame difference data The abnormality detection apparatus according to claim 1, wherein characteristic data is generated. Extracting feature data from the input data;
A step of calculating an angle between a cone indicating a normal space obtained in advance in the feature data space and the feature data extracted by the feature data extraction unit;
An abnormality detection method comprising: determining an abnormality when the angle is larger than a predetermined value.

Images