JP2019046278A - Information processor, control method, computer program, storage medium, and model creation device - Google Patents

Abstract

To provide an information processor capable of further improving the determination accuracy of a model.SOLUTION: A model creation device 100 includes: a learning data acquisition part 101 for acquiring sensor output data as learning data; a feature amount calculation part 102 for calculating a feature amount from the learning data; a data division part 103 for dividing the learning data based on the calculated feature amount; and a model creation part 104 for creating a model using the learning data after division. Additionally the model creation device includes: a verification data acquisition part 105 for acquiring sensor output data as verification data; an evaluation part 106 for evaluating the verification data based on the feature amount of the verification data and the created model; and a data redivision part 107 for redividing the learning data based on the learning data and the verification data according to the result of the evaluation. The model creation part creates a model using the learning data after redivision when the redivision of learning data is performed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technology for creating a model based on sensor data and video data.

  2. Description of the Related Art Conventionally, a model creation method has been studied in which sensor data and video data are acquired from various sensors and whether or not there is an abnormality in these data. One of the model creation methods for determining whether there is an abnormality in the data, creating a model from the data collected at normal time, and comparing and determining the data to be determined and the created model There is.

For example, in the model creation method disclosed in Patent Document 1, learning data is generated by extracting and storing feature vectors via sensor signals. Then, a feature vector of this learning data is divided by clustering to create a model.
In Non-Patent Document 1, moving objects in video data are extracted, and clustering is performed using the position, size, and motion of the extracted moving object region as a feature amount. Then, based on the appearance probability of the cluster in each pixel of the frame of the video data, the image is divided into a plurality of areas and a model is created in each area.
As described above, the classification accuracy of the model can be improved by dividing the distribution of learning data by clustering to create the model.

JP, 2014-032657, A

J. Li, S. Gong, T .; Xiang, "Scene Segmentation for Behavior Correlation", In Proceedings of the European Conference on Computer Vision (ECCV), Marseille, France, 2008.

  However, in these methods, a model is created using only a specific class of training data. Therefore, it is difficult to determine non-specific class data similar to learning data. As described above, when non-specific class data is obtained in model creation using only learning data of a specific class, there is a problem that the data can not be utilized for learning.

  The main object of the present invention is to provide an information processing apparatus capable of further improving the determination accuracy of a model.

  The present invention is an information processing apparatus for creating a model for determining whether or not there is an abnormality in output data of a sensor, and from learning data acquisition means for acquiring output data of the sensor as learning data, and from the learning data Feature quantity calculation means for calculating feature quantity, division means for dividing the learning data based on the calculated feature quantity, model generation means for preparing a model using the learning data after the division, and the sensor Verification data acquisition means for acquiring the output data of the data as verification data, and evaluation means for evaluating the verification data based on the feature amount of the verification data calculated by the feature amount calculation means and the created model. And re-dividing means for re-dividing the learning data based on the learning data and the verification data according to the result of the evaluation, and the model Forming means is characterized by creating the model by using the learning data after the subdivision when the subdivision of the learning data is performed by the subdivision means.

  According to the present invention, it is possible to further improve the determination accuracy of the model.

BRIEF DESCRIPTION OF THE DRAWINGS The figure which shows an example of a structure of the model production apparatus which concerns on 1st Embodiment. The figure which shows an example of distribution of the feature-value at the time of using the output value of two sensors as a feature vector as each element. FIG. 3 is a view showing an example of a distribution after redivision for the distribution of feature amounts shown in FIG. 2; The flowchart which shows an example of the process sequence regarding the process of the learning part of a model production apparatus. 6 is a flowchart illustrating an example of a processing procedure related to processing of a determination unit of the model generation device. A figure showing an example of composition of a model creation device concerning a 2nd embodiment. The figure which shows an example of a learning image. The figure which shows an example of a verification image. The flowchart which shows an example of the process sequence regarding the process of the learning part of a model production apparatus. 6 is a flowchart illustrating an example of a processing procedure related to processing of a determination unit of the model generation device. A figure showing an example of composition of a model creation device concerning a 3rd embodiment. The flowchart which shows an example of the process sequence regarding the process of the learning part of a model production apparatus.

Hereinafter, each embodiment will be described with reference to the accompanying drawings, taking an information processing apparatus to which the present invention is applied as a model creating apparatus as an example.
The model creating apparatus described below evaluates a model created using learning data of a specific class, and recreates the model based on the evaluation result.
In the description of each embodiment, a specific class is described as normal data, and a non-specific class is described as abnormal data. However, the present invention is not limited to the discrimination of two classes of normal and abnormal, and other class discrimination and You may use for multiclass discrimination.

First Embodiment
FIG. 1 is a diagram showing an example of the configuration of a model creating apparatus 100 according to the present embodiment.
The model creation device 100 mainly includes a learning unit 121 and a determination unit 122. Further, it has a model storage unit 11 a and a display device 130. The model creation device 100 has a CPU 200 and a memory 300. The CPU 200 reads a program or the like stored in the memory 300 (for example, a RAM) to control the operation of the model creating apparatus 100.
In addition to the display connected to the information processing apparatus (for example, PC) or the model creation apparatus, the display apparatus 130 may be a display of a PC or tablet at a remote place connected via a network. Good.

The learning unit 121 includes a learning data acquisition unit 101, a feature quantity calculation unit 102, a data division unit 103, a model generation unit 104, a verification data acquisition unit 105, an evaluation unit 106, and a data redivision unit 107.
The determination unit 122 includes a determination data acquisition unit 111, a feature quantity calculation unit 112, and an identification unit 113.

[Functional configuration of learning unit 121]
The learning data acquisition unit 101 included in the learning unit 121 acquires output data (output value) of a sensor attached to the device as learning data. Here, as the sensor, various sensors such as a heat sensor, an angle sensor, and a vibration sensor can be used. The acquired learning data is output to the feature amount calculation unit 102.
The feature amount calculation unit 102 calculates a feature amount from the data received via the learning data acquisition unit 101.
Here, the feature amount is a multi-dimensional feature vector in which output data (output value) of each sensor is an element of the vector. The feature vector may be normalized to have a magnitude of one. Further, as other feature quantities, outputs of the respective sensors may be put together in a certain section of the time series and transformation such as discrete wavelet transform may be applied as a feature vector.
The feature vector will be described below.

FIG. 2 is a diagram showing an example of the distribution of feature amounts when output data (output values) of two sensors are used as feature vectors.
Feature amounts 201 of learning data indicated by “o” in FIG. 2 correspond to one feature vector. Further, the cluster information 202, the cluster information 203, and the feature amount 204 of verification data indicated by “x” illustrated in FIG. 2 will be described later. The extracted feature amount is output to the data division unit 103 or the evaluation unit 106.

Returning to the description of FIG. 1, the data dividing unit 103 divides learning data using the feature amount calculated via the feature amount calculating unit 102. As a data division method, for example, clustering can be performed by applying a mixed normal distribution to the distribution of learning data on the feature space, and the data can be divided.
Further, the number of clusters in the mixed normal distribution may be set to a predetermined discrete value, or the number of candidate discrete values may be selected from the candidates for the data matching using the Akaike information amount standard or the like.
Also, in addition to clustering by mixed normal distribution, it is also possible to divide data into a cluster number C of a predetermined discrete value by the k-neighbor method, or division may be performed using other known methods.
By data division, as shown in FIG. 2, learning data can be divided into learning data belonging to each of cluster information 202 and 203. The feature amount 201 of each learning data and the cluster information 202 and 203 are output to the model generation unit 104.

The model creating unit 104 creates a model by dividing the feature amount 201 of learning data into cluster information 202 and 203. As a method of creating a model, for example, with respect to each learning data divided by the data dividing unit 103, a model is created with the barycentric position and the variance covariance of the mixed normal distribution fitted on the feature space as a normal range. Can.
In addition to the mixed normal distribution, “One-Class Support Vector Machine” is applied to the divided learning data to learn the support vector and the weight of each support vector from the learning data. The learned support vector, the respective coefficients and the threshold may be used as a discriminant model.
Further, the “model” described here is a model that represents the distribution of learning data in the feature space, as long as it can be determined that the distribution is normal and the outside is abnormal. Good. The obtained model is output to the evaluation unit 106.

Similar to the learning data acquisition unit 101, the verification data acquisition unit 105 acquires sensor data from various sensors attached to the device. Then, for the acquired data, the user selects data (abnormal data) when an abnormality occurs as verification data.
As a method of selecting verification data, for example, it is possible to configure selection by displaying a plot of sensor data when an abnormality occurs on the display screen, and designating it through the display screen by the user. The selection method may use other methods.
The selected verification data is output to the feature amount calculation unit 102, and the feature amount calculation unit 102 calculates the feature amount 204 of the verification data indicated by “x” in FIG. The calculated feature quantity 204 is output to the evaluation unit 106.

  The evaluation unit 106 evaluates, based on the feature amount 204 of the verification data and the model created by the model creation unit 104, whether or not the model can determine the feature amount 204 of the verification data as abnormal. As a determination method, when the created model is a mixture normal distribution, the Mahalanobis distance with the verification data is calculated using Equation 1 shown below, using the barycentric position and the variance covariance of each mixture normal distribution.

：検証データの特徴量、μ：正規分布の重心位置、Σ：正規分布の分散共分散行列であるとする。評価部１０６は、各混合正規分布とのマハラノビス距離が最小となるものを選択し、距離が閾値以下であれば正常と判定し、閾値以上であれば異常と判定する。
また、モデルに「Ｏｎｅ−Ｃｌａｓｓ Ｓｕｐｐｏｒｔ Ｖｅｃｔｏｒ Ｍａｃｈｉｎｅ」を用いている場合、学習したサポートベクトルとそれぞれの係数とペナルティ項を用いて以下に示す式２を用いて評価する here,

: Feature amount of verification data, μ: center of gravity of normal distribution, Σ: variance-covariance matrix of normal distribution. The evaluation unit 106 selects one that minimizes the Mahalanobis distance with each mixed normal distribution, determines that it is normal if the distance is equal to or less than a threshold, and determines that it is abnormal if it is equal to or greater than the threshold.
In addition, when “One-Class Support Vector Machine” is used for the model, evaluation is performed using Equation 2 shown below using the learned support vectors, respective coefficients and penalty terms.

：検証データの特徴量、

：写像関数、

：学習したペナルティ項、sign()：符号関数であるとする。この結果f(x)が＋１であれば正常と判定され、−１であれば異常と判定される。 Where w: learned coefficient,

: Characteristic amount of verification data,

: Mapping function,

: Learned penalty term, sign (): It is assumed that it is a sign function. As a result, if f (x) is +1, it is judged as normal, and if it is -1, it is judged as abnormal.

When the evaluation unit 106 determines that the verification data is "abnormal", it can be correctly determined that the verification data is abnormal. Therefore, the created model is output to the model storage unit 11a and stored.
On the other hand, when the verification data can not be determined to be abnormal, the learning data and the verification data are output to the data redivision unit 107.

  The data re-division unit 107 first calculates the degree of similarity between the learning data and the verification data using the learning data and the verification data. Equation 3 shown below can be used to calculate the degree of similarity.

：学習データの特徴量、

：検証データの特徴量、

：スケールファクター、

：距離関数であるとする。この式３では、選択した２つの学習データの特徴量の距離と比べて相対的により近い距離で検証データ２０４の特徴量がある場合、２つの学習データの類似度を０とするようになっている。 here,

: Feature amount of learning data,

: Characteristic amount of verification data,

: Scale factor,

: Suppose that it is a distance function. In this formula 3, when there is a feature amount of the verification data 204 at a distance relatively close to the distance between the feature amounts of the two selected learning data, the similarity between the two learning data is set to 0. There is.

と、正規化類似度行列

を算出し、クラスタ数Ｃ（所定の離散値）を設定する。また、クラスタ数Ｃは赤池情報量基準などを算出することにより決定してもよい。
次に、正規化類似度行列Ｌの固有値・固有ベクトルを算出し、行列

を作成する。次に、Xの行ベクトルを正規化した行列

を作成する。
次に、

のｎ本のベクトルに対してｋ近傍法を適用して、クラスタ番号ｃを割り当てる。
次に、

の第ｉ行のベクトルに割り当てられたクラスタ番号ｃを、特徴ベクトル

のクラスタ番号ｃとする。 Next, using the calculated similarity, the diagonal matrix

And the normalized similarity matrix

Is calculated, and the number of clusters C (predetermined discrete value) is set. Further, the number of clusters C may be determined by calculating the Akaike information amount standard or the like.
Next, the eigenvalues and eigenvectors of the normalized similarity matrix L are calculated.

Create Next, a matrix of normalized row vectors of X

Create
next,

The cluster number c is assigned by applying the k-nearest neighbor method to n vectors of.
next,

The cluster number c assigned to the vector of the i-th row of, the feature vector

Cluster number c.

FIG. 3 is a diagram showing an example of the distribution after redivision for the distribution of the feature shown in FIG.
As shown in FIG. 3, the feature vectors 201 of learning data are clustered so as not to include the feature vector 204 of verification data by data redivision by the data redivision unit 107, and the learning data is divided into cluster information 203, 305 and 306 respectively. It has been subdivided. The learning data 201 re-divided into each of the cluster information 203, 305, and 306 is output to the model creating unit 104, and the model is created again.

When a plurality of verification data are obtained, the similarity is calculated using Equation 3 based on the learning data and the plurality of verification data. Then, it is possible to separate normal data and verification data using known Local Fisher Discriminant Analysis (LFDA).
As shown below, in the local Fisher discriminant analysis (equation 6), a local intra-class scatter matrix (equation 4) and a local inter-class scatter matrix (equation 5) are determined using learning data and verification data.

By calculating the eigenvalues and eigenvectors, it is possible to divide learning data and verification data while performing dimensional reduction. Further, in the present invention, the data re-division method is not limited to the above method, and data may be divided using methods other than these.
The divided data is output to the model generation unit 104, and the divided learning data is used to generate a model.

[Functional Configuration of Determination Unit 122]
The determination data acquisition unit 111 of the determination unit 122 acquires, as determination data, output values of various sensors attached to the device, as in the learning data acquisition unit 101. The acquired determination data is output to the feature amount calculation unit 112.

  The feature amount calculation unit 112 calculates a feature amount from the acquired determination data. The feature amount calculation method can be calculated by the same process as that of the feature amount calculation unit 102, and thus the description thereof is omitted here. The calculated feature amount is output to the identification unit 113.

The identification unit 113 acquires the model created from the model storage unit 11 a, and determines the feature amount of the determination data acquired from the feature amount calculation unit 112.
As the determination method, for example, when the model is a mixture normal distribution, a mixture normal distribution having the shortest distance to the center of gravity of the mixture normal distribution with respect to the feature amount of the determination data is selected, and the Mahalanobis distance with the mixture normal distribution ( Formula 1) is calculated, and if the distance is equal to or less than the threshold value, it is determined as "normal". If the distance is equal to or greater than the threshold value, it is determined as "abnormal".
When the created model is "One-Class Support Vector Machine", the above-described equation 2 is applied to the feature amount of the determination data, and when the result is +1, it is determined as "normal". If the result is -1, it is determined as "abnormal". The determination result determined by the identification unit 113 is output to the display device 130.

[Processing flow of learning unit 121]
FIG. 4 is a flowchart showing an example of the processing procedure of the model creating apparatus 100.
The process shown in FIG. 4 mainly relates to the process of the learning unit 121, and is controlled by the CPU 200.

The CPU 200 acquires learning data via the learning data acquisition unit 101 (S401).
As an acquisition method, for example, data may be acquired from a sensor attached to a device and used as learning data, or sensor data may be acquired as a learning data from a stored HDD or the like. The acquired learning data is sent to the feature amount calculation unit 102.

  The CPU 200 calculates the feature amount of the learning data from the acquired learning data (S402). The feature amount may be calculated based on a feature vector having the output value of each sensor as an element. The calculated feature amount 201 is sent to the data dividing unit 103.

The CPU 200 divides learning data based on the feature amount 201 of learning data via the data dividing unit 103 (S403).
In dividing the learning data, a mixed normal distribution can be applied to the distribution of the feature amount on the feature space, and the learning data can be divided according to which normal distribution. In addition, data can be divided by performing clustering using the k-neighbor method with the number of clusters C (predetermined discrete value).
By this processing, the learning data is divided into data in the cluster information 202 and data in the cluster information 203. Information related to the divided learning data is sent to the model generation unit 104.

The CPU 200 creates a model for each divided data based on the learning data divided by the data division unit 103 and the data redivision unit 108 (S404).
As a method of creating a model, for example, the barycentric position and variance covariance of the mixed normal distribution determined by the data dividing unit 103 may be used as a model. Further, in addition to the mixed normal distribution, a support vector, respective coefficients and threshold values may be calculated using “One-Class Support Vector Machine” for each divided data, and these may be used as a model.
The created model is sent to the evaluation unit 106.

  The CPU 200 acquires verification data via the verification data acquisition unit 105 (S405). The verification data may acquire detection results from various sensors attached to the device, and the user may select data when an abnormality occurs in the device. The acquired verification data is sent to the feature amount calculation unit 102.

  The CPU 200 calculates the feature amount 204 of the verification data via the feature amount calculation unit 102 (S406). Note that the feature amount calculation of verification data can be calculated using the same method as the method of calculating the feature amount of learning data. The feature amount 204 of the calculated verification data is sent to the evaluation unit 106.

The CPU 200 verifies whether the feature quantity 204 of the verification data can be determined to be abnormal through the evaluation unit 106 (S407).
As a verification method, using a mixed normal distribution (cluster information 202 and 203), the normal distribution 202 (cluster information 202) having the closest distance between the determination data and the barycentric position of each normal distribution is selected. The Mahalanobis distance with the feature quantity 204 of the determination data is calculated using the equation 1 described above, using the center of gravity position and the variance covariance of the selected normal distribution 202. The CPU 200 determines “normal” if the calculated distance is within the threshold. If it is equal to or higher than the threshold, it is determined as "abnormal".

The CPU 200 determines whether or not the determination result of the verification data is determined as “abnormal” in the process of step S407 (S408).
If it is determined that the verification data is abnormal (S408: Yes), the process proceeds to step S410. If not (S408: No), that is, if it is not determined that the verification data is "abnormal", the CPU 200 sends the feature amount 201 of learning data and the feature amount 204 of verification data to the data redivision unit 107. And output. Then, the process proceeds to step S409.

The CPU 200 redivides data based on the feature amount 201 of learning data and the feature amount 204 of verification data via the data re-division unit 107 (S409).
As a data repartitioning method, for example, the similarity between data is calculated using the equation 3 described above. At this time, verification data is used as a condition for similarity calculation. Eigenvalues and eigenvectors are calculated based on the determined similarity, a matrix in which the top C of the eigenvalues are arranged in diagonal elements is generated, and the row vectors are clustered by the k-neighbor method. Then, the learning data is divided based on the clustering result assigned to the i-th row.
The learning data 201 and the cluster information 203, 305, 306 (see FIG. 3) which is the clustering result are sent again to the model creating unit 104. The CPU 200 returns to step S404 and creates a model again.
The CPU 200 stores the created model in the model storage unit 11a (S410).

[Processing flow of determination unit 122]
FIG. 5 is a flowchart illustrating an example of the processing procedure of the model creating apparatus 100.
The process shown in FIG. 5 mainly relates to the process of the determination unit 122 and is controlled by the CPU 200.

  The CPU 200 acquires determination data from various sensors attached to the device (S421). The acquired determination data is output to the feature amount calculation unit 112.

  The CPU 200 calculates the feature amount of the determination data via the feature amount calculation unit 112 (S422). The feature amount can be calculated using the same feature amount calculation method as that of the feature amount calculation unit 102. The calculated feature amount of the verification data is output to the identification unit 113.

The CPU 200 acquires the model created from the model storage unit 11a via the identification unit 113, and determines the feature amount of the determination data acquired from the feature amount calculation unit 112 (S423).
As a determination method, for example, when the model is a mixed normal distribution, the Mahalanobis distance between each normal distribution and the determination data is calculated using Equation 1 described above. The one with the smallest Mahalanobis distance (Equation 1) with each normal distribution is selected, and if the distance is equal to or less than the threshold value, it is determined as "normal". If it is equal to or higher than the threshold, it is determined as "abnormal".
Note that the identification processing in the present invention is not limited to the normal distribution, and determination may be performed using another method.

  The CPU 200 displays the determination result by the identification unit 113 on the display screen of the display device 130 (S424).

As described above, the model creating apparatus 100 according to the present embodiment creates a model that can determine the verification data as "abnormal" by dividing the learning data so as not to include the verification data and creating the model. can do. Therefore, it is possible to detect an abnormality of the device, and it is possible to further improve the determination accuracy of the model.
Further, in the present embodiment, the user can confirm that the model creation result is output to the display device. Therefore, the user can create a model while confirming by performing selection and addition of verification data.
Moreover, although verification data are described as abnormal data in each embodiment, either of normal data or abnormal data may be used as verification data. At this time, when the evaluation unit 106 evaluates the normal data as abnormal, the data redivision unit 107 implements data redivision.
Further, in the present embodiment, it is possible to provide a model creation apparatus capable of creating a model while sequentially adding abnormal data that has not been detected.

Second Embodiment
In the present embodiment, a model creation device in a video surveillance system will be described.
In addition, about the structure same as each structure in 1st Embodiment, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

FIG. 6 is a diagram showing an example of the configuration of a model creating apparatus 500 according to the present embodiment.
In the present embodiment, it is assumed that a scene of an outdoor intersection is taken as an example on the assumption that abnormality detection is performed using imaging data (video data). The present invention can be applied to scenes of various places such as facilities and public facilities such as stations.

The model generation device 500 mainly includes a learning unit 521 and a determination unit 522. Further, it has a region / model storage unit 51 a and a display device 130.
The model creating device 500 has a CPU 200 and a memory 300. The CPU 200 reads a program or the like stored in the memory 300 (for example, a RAM) and controls the operation of the model generation device 500.

The learning unit 521 includes a learning video acquisition unit 501, a feature quantity calculation unit 502, a region division unit 503, a model creation unit 104, a verification video acquisition unit 505, and an evaluation unit 106.
The determination unit 522 includes a determination video acquisition unit 511, a feature quantity calculation unit 512, and an identification unit 113.
In the present embodiment, a method of dividing the video into a plurality of regions based on the acquired learning video and creating a normal model in each of the divided regions will be described.

[Functional configuration of learning unit 521]
The learning video acquisition unit 501 of the learning unit 521 acquires video data for learning (hereinafter, referred to as a learning video).
FIG. 7 is a diagram showing an example of a learning image. A learning image 601 shown in FIG. 7 is an image in which each shooting target (hereinafter referred to as a target) 602 to 605 in a shooting scene is moving. The objects 602 to 605 shown in FIG. 7 will be described later.
Note that the learning video 601 may be configured to be acquired from a monitoring camera connected to the model creating apparatus 500. In addition, the video taken and recorded by the surveillance camera may be acquired from an HDD (hard disk drive) or a digital video recorder. The obtained learning image 601 is output to the feature amount calculation unit 502.

Returning to the description of FIG. 6, the feature amount calculation unit 502 detects a target for the acquired learning image 601 and calculates a feature amount. For detection of an object, an object in the image can be detected by template matching using a template prepared in advance.
The user may set an object to be extracted along with the shooting scene. In addition, it is possible to extract the foreground region as a target by removing the background by the background subtraction method etc. using the background image prepared in advance. Also, the detection method is not limited to the above method, and other known techniques may be used.

Thus, for example, the positions, ranges, and the like of the objects 602 to 605 in the shooting scene shown in FIG. 7 are detected as information, and image data of the positions and ranges for the objects 602 to 605 detected from the image are used as feature amounts. Can be calculated.
In addition, the gradient direction histogram and texture feature amount which are known can be used for calculation of a feature amount. Further, it is also possible to extract a multi histogram of optical flow (MHOF) feature value in which a motion vector is extracted from video data, the motion vector is divided into directions, and intensities are added to form a histogram. Also, the feature amount is not limited to the above, and other feature amounts may be used.
The calculated feature amount is output to the region division unit 503 or the evaluation unit 106.

  The area division unit 503 divides the area based on the position information and the feature amount acquired via the feature amount calculation unit 502. As the region division method, for example, the similarity can be calculated using Equation 7 shown below.

：対象の特徴量、

：画像上での対象の位置、

：スケールファクター（クラスタ重心位置とデータ位置の距離）、

：距離閾値、である。この類似度を使って、スペクトラルクラスタリングの手法を用いてクラスタリングを行ない、画像の各ピクセルでのクラスタの出現確率を算出し、その出現確率を基準に複数の領域へと分割する。分割された領域、及び、学習映像から抽出した特徴量はモデル作成部１０４へと送られる。 here,

: Target feature amount,

: The position of the object on the image,

: Scale factor (distance between cluster center position and data position),

: Distance threshold. Using this similarity, clustering is performed using a spectral clustering method, the appearance probability of the cluster at each pixel of the image is calculated, and the image is divided into a plurality of regions based on the appearance probability. The divided area and the feature quantity extracted from the learning video are sent to the model creating unit 104.

The verification video acquisition unit 505 acquires video data for verification (hereinafter, referred to as verification video).
FIG. 8 is a diagram showing an example of a verification video.
The verification image 707 illustrated in FIG. 8 is an example of an image in which the person 708 is falling. In this verification image 707, the position and range in the frame in which the person 708 which is the object is falling are input. The position and range are input, for example, by the user selecting a frame of video via the GUI and specifying the position and range on the image. Thus, for example, a frame in which an abnormal action has occurred, such as a person falling, is used as a verification video, and information on the occurrence position and range of the abnormality is input.
The acquired image 707 is output to the feature amount calculation unit 502, and the feature amount of the person 708 is calculated.

The region redivision unit 507 performs region redivision on the basis of the feature amounts of the objects 602 to 605 in the learning video 601 and the position information and feature amounts of the object 708 in the verification image.
In addition to calculating the degree of similarity using Equation 3 described above, the degree of similarity can be calculated using Equation 8 shown below as the region re-division method.

Here, α is an arbitrary value, and L is a label of data (learning data = + 1, verification data = 0). By doing this, it is possible to calculate so that the degree of similarity between the learning data and the verification data becomes zero. After the similarity is calculated, the eigenvalues are calculated and clustering is performed by the k-neighbor method. Thereby, feature quantities can be clustered.
In the image, the appearance probability of each clustered cluster in each pixel is calculated, and the area is divided based on the appearance probability. The re-divided area is output to the model creating unit 104, and the model is created again.

[Functional Configuration of Determination Unit 522]
The determination video acquisition unit 511 of the determination unit 522 acquires, for example, video data from the monitoring camera as determination data. The acquired determination data is output to the feature amount calculation unit 512.

The feature amount calculation unit 512 detects a person as an object from the determination video and calculates a feature amount.
Note that the person detection and the feature amount calculation can be performed by the same process as the feature amount calculation unit 502, so the description will be omitted. The calculated feature amount is output to the identification unit 113.

[Processing flow of learning unit 521]
FIG. 9 is a flowchart showing an example of the processing procedure of the model creating device 500.
The process shown in FIG. 9 mainly relates to the process of the learning unit 521, and is controlled by the CPU 200.

The CPU 200 acquires a learning video 601 (S801). The acquired learning image 601 is sent to the feature amount calculation unit 502.
The CPU 200 calculates the feature amount of the learning image 601 from the acquired learning image 601 through the feature amount calculating unit 502 (S802). The feature amount may be calculated by, for example, performing background subtraction using a background image prepared in advance and extracting a foreground area in the video. In addition, it is possible to detect the position (region) of the symmetrical (person) by detecting by template matching.
Specifically, feature amounts such as HOG feature amounts and HOF feature amounts are calculated for the objects 602 to 605. The calculated position information of the target and the feature amount are sent to the area dividing unit 503 as learning data.

The CPU 200 divides the area using learning data based on the learning video 601 through the area dividing unit 503 (S 803).
In the area division, the similarity can be calculated using Equation 8 described above based on the calculated position information and feature amount, and the area can be divided using a method using spectral clustering.
Specifically, cluster numbers are assigned to learning data by calculating eigenvalues and eigenvectors of the degree of similarity and performing clustering using the k-nearest neighbor method. Then, the appearance probability of clusters is calculated for each pixel of the image, and divided into regions based on the appearance probability of clusters allocated to each pixel.
Information on the divided area and the position / feature amount of learning data is sent to the model creating unit 104.

  The CPU 200 creates a model for each area based on the information related to the area divided by the area dividing unit 503 or the area re-dividing unit 507 (S804).

  The CPU 200 acquires the verification video 707 via the verification video acquisition unit 505 (S805). As the verification video, for example, a video captured in advance by a monitoring camera may be recorded, and a scene in which the user is performing an abnormal action may be selected from the video. The acquired verification image 707 is sent to the feature amount calculation unit 502.

  The CPU 200 calculates the feature of the verification video 707 via the feature calculation unit 502 (S806). Note that the feature amount of the verification image 707 can be calculated using the same method as the method of calculating the feature amount of the learning image 601. The calculated feature amount of the verification video 707 is sent to the evaluation unit 106 as verification data.

The CPU 200 verifies via the evaluation unit 106 whether or not the verification data can be determined as abnormal (S807). This verification can be performed by the same process as step S407 (see FIG. 4).
The CPU 200 identifies whether or not the determination result of the verification data is determined to be "abnormal" in the process of step S807 (S808).
If it is determined that the verification data is abnormal (S808: Yes), the process proceeds to step S810. If not (S808: No), that is, if it is not determined that the verification data is "abnormal", the CPU 200 outputs the learning data and the verification data to the area redivision unit 507. Then, the process proceeds to step S809.

The CPU 200 re-divides the area based on the learning data and the verification data via the area re-division unit 507 (S809).
For re-division of the region, for example, the degree of similarity is calculated using the feature amount of the acquired learning data and verification data and the position on the image according to Equation 8 described above. Then, based on the calculated degree of similarity of the learning / verification data, the data can be clustered using spectral clustering, and the area can be divided based on the position of the target at which the similar feature quantity occurs.
The area division result is sent to the model generation unit 104. The CPU 200 returns to step S804 and creates a model again.
The CPU 200 stores the divided area and the model of each area in the area / model storage unit 51a (S810).

[Processing flow of determination unit 522]
FIG. 10 is a flow chart showing an example of the processing procedure of the model creating device 500.
The process shown in FIG. 10 mainly relates to the process of the determination unit 522, and is controlled by the CPU 200.

  The CPU 200 acquires the verification video 707 via the determination video acquisition unit 511 (S821). The acquired verification image 707 is sent to the feature amount calculation unit 512 as determination data.

  The CPU 200 calculates the feature amount of the determination data via the feature amount calculation unit 512 (S822). The feature amount can be calculated using a feature amount calculation method similar to that of the feature amount calculation unit 502. The calculated feature amount of the determination data (the feature amount of the determination image) is sent to the identification unit 113.

The CPU 200 reads the model created from the model storage unit 11a via the identification unit 113, and determines the feature amount of the determination data acquired from the feature amount calculation unit 112 (S823).
The CPU 200 displays the determination result by the identification unit 113 on the display screen of the display device 130 (S824).

As in the model creating apparatus 500 according to the present embodiment, the present invention can be configured as a monitoring system of equipment and a model creating apparatus in the field of a monitoring system using video.
Also, according to the present invention, it is possible for the user to re-create a model by sequentially adding verification data.

Third Embodiment
FIG. 11 is a diagram showing an example of the configuration of a model creating apparatus 900 according to the present embodiment.
A difference from the model creating apparatus 500 shown in FIG. 6 is that in the model creating apparatus 900, the learning unit 921 includes a model recreating unit 908. The same reference numerals are given to the same components as the components in the first and second embodiments, and the description thereof will be omitted.

The model re-creation unit 908 of the learning unit 921 creates a model using both the learning image and the verification image divided by the region re-division unit 507.
As a model creation method, based on the feature amount of the person 708 at the time of occurrence of abnormality being input in the verification video and the feature amount of the persons 602 to 605 in the learning video being divided by the region redivision unit 507, This can be done using Support Vector Machine (SVM) or the like.

  At this time, SVM can be learned by setting the label of the learning image to +1 and the label of the verification image to -1. If the number of verification video data is extremely small, the verification video may be weighted and learned, or the learning video may be undersampled to balance the number of data of the learning video and the verification video to create a model. You may Model creation provides support vectors and their respective coefficients and thresholds.

Moreover, it is also possible to verify whether it is misdetected as abnormality, using the learning image | video which is not used for learning by sampling. Further, in the present invention, the method of model re-creation is not limited to the Support Vector Machine, and other methods may be used to create a model. For example, it is also possible to sequentially learn and update the added verification data using an online learning method or the like.
In this model creation, learning of the model may be performed using verification videos for each of the divided learning data.
The model of each area created by this is sent to the evaluation unit 106, and it is evaluated again whether the created model can detect an abnormality.

[Processing flow of learning unit 921]
FIG. 12 is a flowchart illustrating an example of the processing procedure of the model generation device 900.
The process shown in FIG. 9 mainly relates to the process of the learning unit 921 and is controlled by the CPU 200.
Further, the processes of steps S1001 to S1006 shown in FIG. 12 and the processes of steps S1008 and S1010 are the same as the processes of steps S801 to S806 and steps S808 and S810 shown in FIG. .
The processes of steps S1007, S1009, and S1011 will be described below.

The CPU 200 verifies via the evaluation unit 106 whether or not the verification video can be determined as abnormal (S1007).
The CPU 200 re-divides the area based on the learning video and the verification video through the area re-dividing unit 507 (S1009).
The CPU 200 re-creates a model based on the divided learning video and verification video (S1011).
The model can be re-created using Support Vector Machine. By this model re-creation process, support vectors and coefficients and threshold values corresponding to the support vectors are obtained in each area, and these become models. The model of each re-created area is sent to the evaluation unit 106.

The embodiments described above are for more specifically describing the present invention, and the scope of the present invention is not limited to these examples.
Further, although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the embodiments, and includes design and the like within the scope of the present invention. In addition, each embodiment may be implemented by combining the above-described embodiments.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read out a computer program. It can also be realized by the process to be executed. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

  100 ... model creation device, 101 ... learning data acquisition unit, 102, 112 ... feature value calculation unit, 103 ... data division unit, 104 ... model creation unit, 105 ... verification data Acquisition unit 106 Evaluation unit 107 Data redivision unit 11a Model storage unit 111 Determination data acquisition unit 113 Identification unit 121 Learning unit 122: Judgment unit, 130: Display device, 200: CPU, 300: Memory.

Claims (11)

An information processing apparatus that creates a model that determines whether or not there is an abnormality in output data of a sensor.
Learning data acquisition means for acquiring output data of the sensor as learning data;
Feature amount calculation means for calculating a feature amount from the learning data;
A dividing unit that divides the learning data based on the calculated feature amount;
Model creation means for creating a model using the learning data after the division;
Verification data acquisition means for acquiring output data of the sensor as verification data;
Evaluation means for evaluating the verification data based on the feature amount of the verification data calculated by the feature amount calculation means and the created model;
And re-dividing means for re-dividing the learning data based on the learning data and the verification data according to the result of the evaluation.
The model creating means creates the model using the learning data after the redivision, when the redivision of the learning data is performed by the redivision means.
Information processing device. The re-division means calculates the similarity between the learning data and the verification data, and divides the learning data based on the calculated similarity.
An information processing apparatus according to claim 1. The similarity may be calculated based on the distance between feature amounts of the learning data and the verification data.
The information processing apparatus according to claim 2. The similarity may be calculated using labels of the learning data and the verification data.
The information processing apparatus according to claim 2. The verification data is data including information indicating that an abnormality has occurred in a device to which the sensor is attached.
The information processing apparatus according to any one of claims 1 to 4. The output data of the sensor is imaging data,
The learning data acquisition means acquires a learning video as the learning data from the imaging data,
The verification data acquisition unit acquires a verification image as the verification data from the imaging data.
An information processing apparatus according to claim 1. The model creating means creates a model using both the learning data and the verification data.
The information processing apparatus according to any one of claims 1 to 6. A control method of an information processing apparatus for creating a model for determining whether or not there is an abnormality in output data of a sensor,
Acquiring output data of the sensor as learning data;
Calculating a feature amount from the learning data;
Dividing the learning data based on the calculated feature amount;
Creating a model using the learning data after the division;
Acquiring output data of the sensor as verification data;
Calculating a feature amount from the verification data, and evaluating the verification data based on the calculated result and the created model;
And a step of creating the model using the learning data after the redivision, when the learning data is subdivided based on the learning data and the verification data according to the result of the evaluation. And
Control method of information processing apparatus. A computer program for causing a computer to operate as an information processing apparatus that creates a model that determines whether or not there is an abnormality in output data of a sensor,
Learning data acquisition means for acquiring output data of the sensor as learning data;
Feature amount calculation means for calculating a feature amount from the learning data;
A division unit that divides the learning data based on the calculated feature amount;
Model creation means for creating a model using the learning data after the division;
Verification data acquisition means for acquiring output data of the sensor as verification data,
An evaluation unit that evaluates the verification data based on the feature amount of the verification data calculated by the feature amount calculation unit and the created model;
According to the result of the evaluation, it functions as a re-division unit that re-divides the learning data based on the learning data and the verification data,
When the redivision of the learning data is performed by the redivision unit, the learning data after the redivision is used to create the model.
Computer program.   A storage medium storing the computer program according to claim 9. A model creation apparatus for creating a model that determines whether or not there is an abnormality in output data of a sensor,
Learning data acquisition means for acquiring output data of the sensor as learning data;
Feature amount calculation means for calculating a feature amount from the learning data;
A dividing unit that divides the learning data based on the calculated feature amount;
Model creation means for creating a model using the learning data after the division;
Verification data acquisition means for acquiring output data of the sensor as verification data;
Evaluation means for evaluating the verification data based on the feature amount of the verification data calculated by the feature amount calculation means and the created model;
And re-dividing means for re-dividing the learning data based on the learning data and the verification data according to the result of the evaluation.
The model creating means creates the model using the learning data after the redivision, when the redivision of the learning data is performed by the redivision means.
Model creation device.

Images