JP6069095B2 - Image identification device, image identification method, and image identification program - Google Patents

Description

  The present invention relates to an image identification device, an image identification method, and an image identification program for identifying an image.

  It has been conventionally practiced to learn an image in a normal state without any abnormality and to make an abnormal scene that does not belong to any of the learned patterns. At this time, when learning / identification is performed based on the movement pattern for each moving object in the image (see, for example, Non-Patent Document 1), the trajectory of the moving object is extracted, and learning / identification is performed based on the trajectory. There are cases where it is performed (for example, see Non-Patent Document 2). These methods are used when the number of moving objects (people, cars) in the image is small (one or several), or when the moving path of moving objects is limited, such as when a car runs on a road. Is valid.

  Note that in this specification, an image means a still image or an image for one frame constituting a moving image. A video has the same meaning as a moving image, and is a set of a series of images.

Anomaly detection of work motion by scene division focusing on human motion, Sanae Shimizu, Hidekazu Hirayu, Hiroji Asai, Yoshinori Niwa, IEICE Technical Report, 2007-CVIM-160, 2007. Detection of abnormal behavior using a situation-dependent model, Hiromi Okamoto, Shuichi Nishio, Noboru Babaguchi Fujiki Morii Norihiro Hamada, IEICE Technical Report, DE2008-19, PRMU2008-37, 2008

  However, in a situation where there are a relatively large number of moving objects, such as when a person is walking on a sidewalk or a site in a facility, and the movement route thereof is not limited, the method of Non-Patent Document 1 is used. It is difficult to observe each moving object with a resolution that can identify the state of the scene based on the movement of each moving object. In addition, it is difficult to learn a locus pattern by the method of Non-Patent Document 2.

  As described above, in the conventional technology, when an image at a time including an abnormal scene is detected from a video including a relatively large number of moving objects, it is difficult to extract and determine a feature amount based on the motion of the object. There is a problem. This is because it is necessary to use a feature value that reflects the relationship between the movement directions of the moving objects as well as the individual movements of the moving objects.

  The present invention has been made in view of such circumstances, and provides an image identification device, an image identification method, and an image identification program capable of detecting occurrence of an abnormal situation in an image by identifying an input image. The purpose is to provide.

The present invention includes a changing area extraction means for extracting an image change region from an input image sequence, and vector extracting means for extracting a motion vector on the basis of the image change region, in the spatial domain when the predetermined, of the motion vector A flow texture feature amount generating unit that generates a flow texture feature amount indicating a relationship between directions; and an identification unit that identifies an image based on the flow texture feature amount and outputs information of an identification result; To do.

The present invention, the flow texture feature generating means, a matrix in which the value indicating the direction of the motion vector quantized in rows and columns, or by a matrix in which the direction and vector correlation value of the motion vector in rows and columns The flow texture feature amount is expressed.

According to the present invention, the flow texture feature quantity generation means calculates the frequency of the motion vector direction of an animal body other than the animal body that is present within a certain distance with respect to the motion vector direction of the animal body. Expressed in the form of a matrix and the frequency in the direction of the motion vector of the moving object other than the moving object that exists within the fixed distance, each of which has a different center position for each moving object. the combination of things, or, with respect to the motion vector direction of a moving object is present within a certain distance around the animal body, in the form of a vector correlation value between the motion vector direction of a moving object other than the animal body matrix a representation, and a plurality of positions around each per animal body exists within a different said predetermined distance, direction and vector of the motion vector of a moving object other than the moving object Combinations of a representation of a and Le correlation values in a matrix format, or a numerical value was determined uniformity of distribution of the motion vector when considered pre kidou-out vector and texture, the by any one of It is characterized by expressing a flow texture feature amount.

The present invention relates to an image identification method performed by an image identification device for identifying an image from an input image, a change region extraction step for extracting an image change region from an input image series, and a motion vector based on the image change region. A vector extraction step , a flow texture feature amount generating step for generating a flow texture feature amount indicating a relationship between the directions of the motion vectors in a predetermined space-time region, and image identification based on the flow texture feature amount. And an identification step of outputting identification result information.

The present invention, the flow in the texture feature generating step, the matrix in which the value indicating the direction of the motion vector quantized in rows and columns, or by a matrix in which the direction and vector correlation value of the motion vector in rows and columns The flow texture feature amount is expressed.

According to the present invention, in the flow texture feature amount generation step, the frequency of the motion vector direction of an animal body other than the animal body existing within a certain distance with respect to the motion vector direction of the animal body is determined. Expressed in the form of a matrix and the frequency in the direction of the motion vector of the moving object other than the moving object that exists within the fixed distance, each of which has a different center position for each moving object. the combination of things, or, with respect to the motion vector direction of a moving object is present within a certain distance around the animal body, in the form of a vector correlation value between the motion vector direction of a moving object other than the animal body matrix a representation, and respectively the center position for each of the plurality of moving object is present within a different said predetermined distance, the direction of the motion vector of a moving object other than the moving object Combinations of a representation the vector correlation values into a matrix format, or a numerical value was determined uniformity of distribution of the motion vector when considered pre kidou-out vector and texture, the by any one of It is characterized by expressing a flow texture feature amount.

  The present invention is an image identification program for causing a computer to function as the image identification device.

  According to the present invention, the motion vector due to the motion of the moving object included in the image is regarded as a temporal and spatial texture, and is expressed in a low-dimensional feature amount, so that not only the individual motions of the moving object but also a plurality of moving objects. Reflecting the flow to be produced, it is possible to detect the occurrence of an abnormal situation in the image.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows the processing operation of the image identification device shown in FIG. It is explanatory drawing which shows the processing operation of the image identification device shown in FIG.

  Hereinafter, an image identification device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a video input unit for inputting an input video. The video input unit 1 inputs video by inputting the output of an imaging device such as a camera or reading video data from a video file. Reference numeral 2 denotes a flow information detection unit that detects a hierarchical flow, that is, an optical flow or a motion vector for each moving object, from the video input in the video input unit 1. Reference numeral 21 denotes an optical flow detector that detects an optical flow from the input video. Reference numeral 22 denotes a motion vector detection unit that detects a motion vector from the input video.

  Reference numeral 3 denotes a flow texture feature generation unit that performs flow sampling by a predetermined method, calculates a relationship between the directions in a predetermined space-time region, and generates a flow texture feature. Reference numeral 4 denotes a learning / identification unit that performs learning and identification using flow texture features. Reference numeral 41 denotes a dictionary storage unit that stores dictionary data generated and updated according to the learning result in the learning / identification unit 4. Reference numeral 5 denotes a determination output unit that outputs the identification result as a numerical value.

  Next, the operation of the image identification device shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the image identification device shown in FIG. First, the video input unit 1 inputs an input video (step S1). Here, it is assumed that a monitoring video is input as an input video and used for the purpose of detecting an abnormal scene from the monitoring video. As a video input by the video input unit 1, first, a long video for learning is required. The more accurate the video that does not contain abnormal scenes, the more accurate the dictionary can be created. It does not necessarily have to be a series of video files, and there may be many time-series images and short video files. The subject may be anything, but there are some movements of animal groups such as roads where many cars pass, streets where many people go, and station premises. When a flow is generated, it is difficult to cope with the conventional method, and the effect of the present invention is exhibited. Therefore, in the following description, it will be described as an example of a surveillance video with a lot of traffic. In addition, the size of the image and the size of the moving object included are not specified.

  Next, the flow information detection unit 2 detects a hierarchical flow, that is, an optical flow or a motion vector for each moving object, from the video input by the video input unit 1 (step S2). The optical flow detector 21 detects the optical flow in each frame of the video. There are many detection algorithms, but for example, the Lucas-Kanade method, the Horn-Schunkck method, the method by block matching, and the like can be used. The motion vector detection unit performs foreground extraction based on background estimation, and calculates a region of an object unit such as a person or a car and an average motion vector in the region from the temporal change of the foreground region.

  Next, the flow texture feature generation unit performs flow sampling by a predetermined method, calculates a relationship between these directions in a predetermined space-time region, and generates a flow texture feature (step S3). . The flow texture feature generation unit 3 regards the optical flow and the global motion vector obtained by the flow information detection unit 2 as the temporal and spatial texture of the video, generates texture information, and inputs the feature vector of the learning / identification unit Output as. Here, a method using GLCM (Gray Level Co-occurrence Matrix), which is a method used for texture analysis, will be described.

GLCM describes a relationship between a reference pixel p and a pixel q at a certain distance from the pixel p as a matrix, and is defined as shown in equation (1).

  This shows the relationship of pixel values between two pixels p and q that are at a fixed distance from each other. i and j can be pixel values themselves, or i or j can be a difference value between pixel values.

Therefore, if the optical flow at the position of each pixel is obtained from the image, this is used instead of the pixel value, and the same matrix is obtained by classifying i and j in Equation (1) into a quantized optical flow class. Can be obtained. This is the flow texture feature. The flow quantization method can be described by an 8 × 8 matrix when, for example, d ^k (k = 1,..., 8) is defined as eight formulas of 45 degrees, as shown in equation (2). . Here, f (p) indicates the direction of the flow vector at the pixel p, and can take values of d ^k (k = 1,..., 8).

となる。 For example, in FIG. 3, when considering only the point p and the region of a certain distance point (shown by a circle) from there, the flow direction class at p is d ² , and the flow direction at points q1, q2, q3 within a certain distance. Is d ³ , d ² , d ¹ ,

It becomes.

In the matrix on the right side of equation (3), 1 is given to the elements of d ² d ¹ , d ² d ² , and d ² d ³ , and the others are zero.

  In addition to describing the relationship between pixels, it can be extended to describe the relationship between objects. The same description can be made by applying the equation (2) to the average flow of a certain object region and the average flow of another object region within a certain distance. This can be used to describe the texture features of global motion vectors. The object region can be extracted as a region occupying a certain area or more having a gradient of movement in the same direction for a certain short period of time.

As shown in equation (2), by setting both the rows and columns of the matrix G to the quantized flow direction, it is possible to describe how and in which direction the flow is distributed. I can't reflect that. Therefore, as shown in the equation (4), by setting one of the row and the column as the vector correlation value (corr (p, q)), the flow vectors of the reference pixel p and the surrounding pixel q are different. Can be a matrix that reflects. As a method of obtaining the correlation value of the vector, there is a method of obtaining an inner product (v (p) · v (q)) (Equation (5)). The vector correlation value quantization method takes a value from 0 to 1 if it is an inner product, for example, so that this is expressed as 0.1 in increments of c ^j (j = 1,..., 10) as shown in equation (4). When defined, it can be described by a 10 × 10 matrix. Here, f (p) represents a vector correlation value in the pixel p, and can assume a value of c ^j (j = 1,..., 10).

  Furthermore, using the matrix G obtained as described above, a numerical value such as uniformity of motion vector distribution may be used as the feature amount. In this case, the feature amount used for texture analysis using GLCM can be applied as it is. Examples include the equations (6) to (9).

標準偏差：

エントロピー：

２次モーメント：

average:

standard deviation:

Entropy:

Second moment:

  Next, the learning / identification unit 4 learns or identifies the texture information output from the flow texture feature generation unit 3 as a feature vector (step S4). The learning / identification unit 4 includes a dictionary storage unit 41, and stores dictionary data as a learning result in the dictionary storage unit 41 during learning. At the time of identification, the dictionary data is read from the dictionary storage unit 41 for identification. The degree of similarity or abnormality is determined based on the degree of matching with the dictionary. The distinction between the learning mode and the identification mode and whether the dictionary data is updated or not in the learning mode are input by the user from the outside at the time of execution. As a learning / identification method, a general machine learning algorithm can be used.

  For example, there is a method of expressing a feature vector distribution as a probability distribution model. The parameters of this model are estimated at the time of learning, and at the time of identification, the degree of similarity or degree of abnormality is output as a probability from the goodness of fit to the model. Further, there is a method of clustering distributions of feature vectors in the feature space and determining the similarity or abnormality based on whether the class belongs to any class or whether the class does not exist. In addition, there is a method of learning model parameters for separating two classes, which are abnormal or not at the time of learning, using an SVM classifier, and using this for identification.

  Next, the determination output unit 5 outputs the identification result obtained by the learning / identification unit 4 at the time of identification (step S5). Depending on what is used as the above-described machine learning method, the output type is a binary output indicating whether it is abnormal or not, a scalar value indicating the similarity, and the like.

  As explained above, while reflecting the movement of individual moving objects, it also reflects changes in the flow of movements created by multiple moving objects, making motion vectors feature quantities and reducing them suitable for learning and identification processing. When determining the similarity and abnormality level of moving images using dimension feature quantities, the movement of a moving object such as a person or a car is converted into a feature quantity so that a large number of moving objects can move in the image. In addition to learning how individual animals move, even if they exist, learn the path of movement and the flow of movements created by multiple moving objects to determine the degree of similarity and abnormality. did.

  In order to realize this, the optical flow of the image and the motion vector of the moving object are regarded as textures in the space and time, and the feature value is obtained. Thus, the optical flow of the input image is obtained, the average motion vector for each moving object is obtained, Quantize those directions into texture features, learn from the texture features obtained from the learning sample as input, create a dictionary, use the dictionary to identify the texture features obtained from the test sample, and extract the sampled feature points The feature information is converted into feature vectors.

  As a result, the motion vector by the motion of the moving object included in the image is regarded as a temporal and spatial texture and expressed in a low-dimensional feature value, so that not only the individual movements of the moving object but also the flow created by multiple moving objects. Reflecting it, it becomes possible to determine the similarity or abnormality of the video.

  You may make it implement | achieve the image identification device in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  By identifying the input image, it can be applied to applications where it is essential to detect an abnormal situation in the image.

  DESCRIPTION OF SYMBOLS 1 ... Video input part, 2 ... Flow information detection part, 21 ... Optical flow detection part, 22 ... Motion vector detection part, 3 ... Flow texture characteristic generation part, 4 ... Learning Identification unit, 41 ... dictionary storage unit, 5 ... determination output unit

Claims (7)

Change area extraction means for extracting an image change area from the input image series;
Vector extraction means for extracting a motion vector based on the image change region;
A flow texture feature amount generating unit for generating a flow texture feature amount indicating a relationship between directions of the motion vectors in a predetermined space-time region ;
An image identification apparatus comprising: an identification unit configured to identify an image based on the flow texture feature amount and output information of the identification result. The flow texture feature generating means, a matrix in which the value indicating the direction of the motion vector quantized in rows and columns, or the flow texture features by a matrix in which the direction and vector correlation value of the motion vector in rows and columns The image identification apparatus according to claim 1, wherein a quantity is expressed. The flow texture feature quantity generation means includes:
Expressing the frequency of the motion vector direction of an animal body other than the above-mentioned animal body within a certain distance with respect to the motion vector direction of the animal body in a matrix form, and a plurality of animal bodies the combination of those respectively present within the predetermined distance position differs around, representing the frequency of the motion vector direction of a moving object other than the moving object in a matrix format for each,
Alternatively, a vector correlation value with a motion vector direction of a moving object other than the moving object that is present within a certain distance with respect to the moving object direction of the moving object in the form of a matrix, and present within the predetermined distance position differs around each of a plurality of each animal body, the combination of a representation in the form of a direction vector correlation values and a matrix of motion vectors of a moving object other than the animal body,
Or, the numerical values determine the uniformity of distribution of the motion vector when regarded before kidou-out vector and texture,
The image identification apparatus according to claim 1 , wherein the flow texture feature amount is expressed by any one of the following. An image identification method performed by an image identification device for identifying an image from an input image,
A change area extraction step for extracting an image change area from the input image series;
A vector extraction step of extracting a motion vector based on the image change region ;
A flow texture feature value generating step for generating a flow texture feature value indicating a relationship between directions of the motion vectors in a predetermined space-time region ;
An image identification method comprising: an identification step of identifying an image based on the flow texture feature amount and outputting information of the identification result. The flow in the texture feature generating step, the matrix in which the value indicating the direction of the motion vector quantized in rows and columns, or the flow texture features by a matrix in which the direction and vector correlation value of the motion vector in rows and columns The image identification method according to claim 4, wherein a quantity is expressed. In the flow texture feature generation step,
Expressing the frequency of the motion vector direction of an animal body other than the above-mentioned animal body within a certain distance with respect to the motion vector direction of the animal body in a matrix form, and a plurality of animal bodies the combination of those respectively present within the predetermined distance position differs around, representing the frequency of the motion vector direction of a moving object other than the moving object in a matrix format for each,
Alternatively, a vector correlation value with a motion vector direction of a moving object other than the moving object that is present within a certain distance with respect to the moving object direction of the moving object in the form of a matrix, and present within the predetermined distance position differs around each of a plurality of each animal body, the combination of a representation in the form of a direction vector correlation values and a matrix of motion vectors of a moving object other than the animal body,
Or, the numerical values determine the uniformity of distribution of the motion vector when regarded before kidou-out vector and texture,
The image identification method according to claim 4 , wherein the flow texture feature quantity is expressed by any one of the following.   An image identification program for causing a computer to function as the image identification device according to claim 1.

Images