JP2018049566A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To authenticate an object accurately by use of a similarity calculation parameter suitable for a location or field of view of a camera.SOLUTION: An image processing apparatus includes: acquisition means of acquiring an image captured by at least one imaging apparatus; extraction means of extracting feature quantity from an image acquired by the acquisition means; storage means of storing a registration dictionary in which feature quantities of images are registered; first calculation means of calculating a similarity calculation parameter for calculating similarity between the feature quantities registered in the registration dictionary and feature quantity extracted from the image captured by the imaging apparatus, on the basis of the feature quantity extracted from the image captured by the imaging apparatus, for each of the imaging apparatuses; and second calculation means of calculating the similarity between the feature quantity registered in the registration dictionary and feature quantity of the image for authentication acquired by the acquisition means, for the imaging apparatus having captured the image for authentication, on the basis of the similarity calculation parameter calculated by the first calculation means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for authenticating an object in an image.

There is known a person authentication technique for determining who is shown in an image captured by an installed camera. Applications of this person authentication technology include criminal investigations using facial photographs and lost child search. In these application destinations, since the installation location and the angle of view differ for each installed camera, the imaging conditions (for example, the direction of the person and the lighting conditions) for imaging a person also differ for each camera. Therefore, there is a need for a person authentication technique that is robust to imaging conditions.
In general, in the person authentication technology, a person is included in the registered image based on the degree of similarity calculated between the captured person image and a previously registered person image (hereinafter referred to as “registered image”). Who is or who is not. How to calculate the similarity is one of the important factors that determine the accuracy of person authentication, and various methods have been proposed.

For example, in the technique disclosed in Non-Patent Document 1, a parameter for calculating the similarity is learned in advance from a large amount of data, and the similarity is calculated based on the learned parameter. A large amount of data including various imaging conditions is used to learn parameters in advance, thereby realizing person authentication that is robust to the imaging conditions to some extent.
In the technique disclosed in Patent Document 1, an input subspace is calculated from an input pattern, a plurality of dictionary subspaces are calculated from a dictionary pattern corresponding to a predetermined category, and the input subspace and the plurality of dictionary parts are calculated. Each space is linearly transformed using an orthogonalization matrix. Then, the similarity between the linearly transformed input subspace and the plurality of linearly transformed dictionary subspaces is calculated. The orthogonalization matrix for linear transformation is generated as a transformation matrix that makes it easy to distinguish between persons included in a registered image by orthogonalizing a plurality of dictionary subspaces.

Japanese Patent No. 4343125

H. V. Nguyen and L.M. Bai, “Cosse Similarity Metric Learning for Face Verification”, 2010, ACCV (Annual Conference of Computer Vision).

However, with the techniques described in the above-mentioned documents, since the similarity calculation parameter suitable for the installation location and the angle of view of the camera cannot be obtained, the accuracy of person authentication may be reduced. That is, in the technique of Non-Patent Document 1, since the similarity calculation parameter is learned from a large amount of data in advance, calculating the similarity calculation parameter suitable for the installation location and the angle of view of the camera in advance is not possible. Can not. The technique of Patent Document 1 calculates a transformation matrix for orthogonalizing a partial space for each person included in a registered image, and is suitable for the installation location and angle of view of a camera that captures a person to be authenticated. The similarity calculation parameter cannot be calculated.
The present invention has been made to solve the above-described problems, and an object of the present invention is to authenticate an object with higher accuracy using a similarity calculation parameter suitable for a camera installation location and an angle of view.

  In order to solve the above problems, according to an aspect of the image processing device of the present invention, an acquisition unit that acquires an image captured by at least one imaging device, and a feature amount from the image acquired by the acquisition unit Extraction means for extracting image data, storage means for storing a registered dictionary in which image feature values are registered, and for each of the imaging devices, based on the feature values extracted from the image captured by the imaging device, First calculation means for calculating a similarity calculation parameter for calculating a similarity between a feature quantity extracted from an image captured by an imaging device and a feature quantity registered in the registration dictionary; and the acquisition For the imaging device that captures the authentication target image, the similarity between the feature amount of the authentication target image acquired by the means and the feature amount registered in the registration dictionary A second calculation means for calculating, based on the similarity calculation parameter calculated by the calculating means, an image processing apparatus including a is provided.

  According to the present invention, it is possible to authenticate an object with higher accuracy by using a similarity calculation parameter suitable for the installation location and angle of view of the camera.

FIG. 2 is a block diagram illustrating an example of a hardware configuration of the image authentication system according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a control program of the image authentication system according to the first embodiment. FIG. 3 is a diagram schematically illustrating a console screen of the image authentication system according to the first embodiment. 5 is a flowchart illustrating monitoring / authentication processing of the image authentication system according to the first embodiment. The figure which showed typically the local area | region set to an authentication image. 5 is a flowchart showing image authentication processing of the image authentication system according to the first embodiment. The figure which showed notionally an example of the content displayed as an authentication result. 5 is a flowchart showing similarity calculation parameter calculation processing of the image authentication system according to the first embodiment. The figure which showed typically the example which calculates the origin suitable for the installed camera. The figure which showed typically the example when the calculated origin is not in the appropriate position on feature-value space. The figure which showed typically the screen which shows the sample collection condition for a calculation which the image authentication system which concerns on Embodiment 1 displays during a similarity calculation parameter calculation process. 5 is a flowchart showing dictionary registration processing of the image authentication system according to the first embodiment. The figure which showed typically the example which calculates the Mahalanobis matrix suitable for the installed camera. 10 is a flowchart showing dictionary registration processing of the image authentication system according to the second embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. In addition, not all combinations of features described in the present embodiment are essential for the solving means of the present invention.

Embodiment 1
In the image authentication system according to the first embodiment of the present invention, an example will be described in which a similarity calculation parameter is calculated for each installed camera, and a specific person is authenticated from the captured images collected.
<Hardware configuration>
FIG. 1 is a block diagram illustrating an example of a hardware configuration of the image authentication system according to the present embodiment. This image authentication system includes an image authentication apparatus 1, cameras 2 to 4, and a network 5.
The image authentication device 1 is an image authentication device that authenticates a specific person from an image, and a specific configuration will be described later.
The cameras 2 to 4 are so-called network cameras and function as imaging devices. Each of the cameras 2 to 4 includes a visible light camera unit that includes a lens, an imaging element such as a CCD and a CMOS sensor, and captures visible light, and a communication device for connection to a network. Note that other cameras having a communication function may be used as the cameras 2 to 4. Further, the camera unit is not limited to one that captures visible light, but may be an infrared camera unit that captures infrared light.
The image authentication apparatus 1 and each of the cameras 2 to 4 are connected via a network 5 and can perform data communication with each other.
1 shows a configuration using three cameras (cameras 2 to 4), the number of cameras is not limited to this.

The image authentication device 1 includes a CPU (Central Processing Unit) 11, a ROM 12, a RAM 13, a secondary storage device 14, a communication device 15, a video output device 16, an operation input device 17, and a connection bus 18. I have.
The CPU 11 controls the entire image recognition apparatus 1 by executing a control program stored in the ROM 12 or the RAM 13. The ROM 12 is a non-volatile memory and stores a control program and various parameter data. The control program is executed by the CPU 11 and realizes each process described later. The RAM 13 is a volatile memory, and temporarily stores images, control programs, and execution results thereof.

The secondary storage device 14 is a rewritable secondary storage device such as a hard disk or a flash memory, and stores images received from the cameras 2 to 4 via the communication device 15. Further, the secondary storage device 14 stores information such as control programs, various setting contents, and processing results as necessary. Information stored in the secondary storage device 14 is output to the RAM 13 as necessary, and is used by the CPU 11 to execute a control program.
The communication device 15 is a wired communication unit, for example, and performs data communication with the cameras 2 to 4 via the network 5. Note that the communication device 15 may be a wireless communication unit.
The video output device 16 is a monitor device such as a CRT or a TFT liquid crystal, and displays an image acquired from the RAM 13, an execution result of the control program, and the like.
The operation input device 17 is a device for inputting an operation from a user such as a keyboard and a mouse. User input (instruction input from the user) to be described later is acquired via the operation input device 17.
The connection bus 18 connects the devices constituting the image authentication device 1 and performs data communication with each other.

In the present embodiment, the processing described later is described as being realized by software using the CPU 11 of the image authentication device 1, but part or all of the processing of the image authentication device 1 is realized by hardware. It doesn't matter. As hardware in this case, for example, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used.
Further, software describing processing described later may be acquired via a network or various storage media and executed by a processing device (CPU, processor) such as a personal computer.
Further, the cameras 2 to 4 may execute part or all of the authentication processing of the image authentication apparatus 1. In that case, the cameras 2 to 4 may be provided with hardware necessary for processing performed by the image authentication apparatus 1.

<Functional configuration of image authentication device>
FIG. 2 is a block diagram illustrating an example of a functional configuration realized by executing a control program in the image authentication apparatus 1 according to the present embodiment. The image authentication apparatus 1 according to the present embodiment includes an authentication image acquisition unit 21, a local region setting unit 22, a feature extraction unit 23, a feature storage unit 24, a dictionary registration unit 25, a similarity calculation parameter calculation unit 26, and a similarity calculation parameter storage. Unit 27, similarity calculation unit 28, authentication result determination unit 29, and display unit 30.
The authentication image acquisition unit 21 acquires an authentication image (authentication target image) that is an authentication target based on the captured images acquired from the cameras 2 to 4, and outputs the authentication image to the local region setting unit 22. Here, the “authentication image” is an image in which an object to be authenticated is captured, and in the present embodiment, is an image in which a human face is captured.

The local region setting unit 22 extracts coordinate information of feature points such as eyes and nose of a person's face from the authentication image acquired from the authentication image acquisition unit 21, and sets a local region based on the extracted feature points. Then, the local region setting unit 22 outputs the local region together with the authentication image to the feature extraction unit 23.
The feature extraction unit 23 outputs the feature amount extracted from the local region of the authentication image acquired from the local region setting unit 22 to the feature storage unit 24. Further, the feature extraction unit 23 also outputs the extracted feature amount to the dictionary registration unit 25 and the similarity calculation parameter calculation unit 26. In the present embodiment, the feature amount is extracted for each local region of the authentication image.
The feature storage unit 24 stores the feature amount acquired from the feature extraction unit 23 in the secondary storage device 14. You may make it memorize | store in RAM13. The stored feature amount is output to the similarity calculation unit 28 upon request.

The dictionary registration unit 25 stores the feature amount acquired from the feature extraction unit 23 in the secondary storage device 14 as registration dictionary data. Alternatively, the registration dictionary may be stored in the RAM 13. Here, the “registration dictionary” is a set of data in which a feature quantity is associated with a person ID (generally, a person name or the like) corresponding to the feature quantity. A method for acquiring the person ID will be described later. The registration dictionary may include feature amounts of a plurality of persons, or may include a plurality of feature amounts for one person. The registration dictionary is output to the similarity calculation parameter calculation unit 26 and the similarity calculation unit 28 in response to a request.
The similarity calculation parameter calculation unit 26 calculates a similarity calculation parameter based on the feature amount acquired from the feature extraction unit 23 or the feature amount of the registered dictionary acquired from the dictionary registration unit 25, and stores the similarity calculation parameter storage To the unit 27. Details of the calculation processing of the similarity calculation parameter will be described later.

The similarity calculation parameter storage unit 27 stores the similarity calculation parameter acquired from the similarity calculation parameter calculation unit 26 in the secondary storage device 14. The stored similarity calculation parameter is output to the similarity calculation unit 28 upon request.
The similarity calculation unit 28 uses the similarity calculation parameter acquired from the similarity calculation parameter storage unit 27 as the similarity between the feature amount acquired from the feature storage unit 24 and the registration dictionary acquired from the dictionary registration unit 25. And calculate. The calculated similarity is output to the authentication result determination unit 29.
The authentication result determination unit 29 determines, based on the similarity acquired from the similarity calculation unit 28, who is the authentication result, that is, who is the person in the authentication image or who is not in the registration dictionary. To do. The authentication result is output to the display unit 30.
The display unit 30 displays the authentication result acquired from the authentication result determination unit 29 on the video output device 16.

<Overview of overall processing>
The process according to the present embodiment includes three processes: a similarity calculation parameter calculation process, a monitoring / authentication process, and a dictionary registration process. In the similarity calculation parameter calculation process, a similarity calculation parameter corresponding to the installed cameras 2 to 4 is calculated. In the monitoring / authentication process, it is determined whether a specific person is captured while taking and storing captured images using the cameras 2 to 4. In the dictionary registration process, the feature amount of the person to be authenticated is added to the registration dictionary.
Note that at the start of each process, it is assumed that the cameras 2 to 4 are already installed in different locations, and the image authentication apparatus 1 is ready to immediately acquire images from the cameras 2 to 4 and start the process described later.

FIG. 3 is a diagram illustrating an example of a console screen displayed on the video output device 16 by the image authentication system according to the present embodiment. On this console screen 40, buttons 41 to 43 for instructing an operation mode, buttons 44 for instructing various settings, a button 45 for instructing termination, and a pointer 46 are displayed. Thereby, a menu indicating which process is executed is displayed. The user operates the operation input device 17 and selects a process in the menu with the pointer 46. By selecting any one of the buttons 41 to 43 of “monitoring / authentication mode”, “dictionary registration mode”, and “adjustment mode”, the image authentication device 1 executes processing to be described later.
For example, when the user selects the “setting” button 44 with the pointer 46, the image authentication apparatus 1 executes processing for changing various settings of the image authentication system of the present embodiment. When the user selects the “end” button 45 with the pointer 46, the image authentication apparatus 1 ends the image authentication system of the present embodiment.

<Monitoring / authentication processing>
First, the monitoring / authentication process in which the image authentication apparatus 1 according to the present embodiment determines whether a specific person is captured by authenticating according to a user input while capturing image data of a person using the cameras 2 to 4 will be described. To do. This process is executed by the user operating the operation input device 17 and pointing the “monitoring / authentication mode” button 41 on the console screen 40 of FIG.
Note that during this monitoring / authentication process, the cameras 2 to 4 always capture images, and continue to store captured images in the secondary storage device 14. At this time, the image authentication device 1 stores the image in association with a label indicating which camera has captured the captured image.
The monitoring / authentication process proceeds according to a user input acquired via the operation input device 17. There are two types of user input: “end instruction” for ending the monitoring / authentication process and “authentication instruction” for executing image authentication. These two user inputs can be received at any time during the monitoring / authentication process shown in FIG.

FIG. 4 shows an example of the processing flow of the monitoring / authentication process executed by the image authentication apparatus 1. The actual processing will be described in detail with reference to this figure.
First, the authentication image acquisition unit 21 ends this processing flow when there is a “termination instruction” of user input (Yes in S1). If there is no “end instruction” (No in S1), and if there is an “authentication instruction” input by the user (Yes in S2), an authentication image is acquired (S3). As described above, the authentication image is an image showing a person's face. In order to acquire the authentication image, it is only necessary to detect a human face from the captured image stored in the secondary storage device 14 and cut out the detected human face. As a method for detecting a human face from a captured image, a known technique may be used.
For example, the techniques disclosed in the following documents can be used.
P. Viola and M.M. Jones, “Robust Real-Time Face Detection”, In page II: 747, 2001.
Based on the detected coordinates on the face image, the face image is cut out from the captured image. At this time, image normalization may be performed to make the in-plane rotation of the face relative to the image plane constant. For example, processing such as rotating the image so that the straight line connecting both eyes of the face is horizontal to the image may be performed.

In the present embodiment, a human face is detected from a plurality of captured images stored in the secondary storage device 14, and the detected human face, that is, an authentication image is cut out. The user may select which captured image is to be detected and cut out. For example, the user may designate a time range, and a captured image captured within the time range may be set as a processing target.
Subsequently, the local region setting unit 22 sets a local region for all authentication images (S4). Here, first, the position of an organ (organ position) such as the eyes and nose of the face is detected, and a local region is set based on the detected organ position. That is, the local area setting unit 22 sets a plurality of local areas for one authentication target image.

FIG. 5 is a diagram schematically showing a local region set in the authentication image. As shown in this figure, a local region 51 for the right eye, a local region 52 for the left eye, and a local region 53 for the mouth are set for the authentication image, that is, the human face image 50.
In the present embodiment, a rectangular area centered on the detected organ position is set as a local area. However, other methods may be used as the local area setting method. It may not be a rectangular area centered on the organ position. For example, it may be an area based on a position that is a predetermined number of pixels away from the organ position, or may be a circular area instead of a rectangular area. Further, a predetermined area of the face image may be determined in advance without using the detected organ position as a reference. A plurality of local regions may be set for one organ position, and the number of local regions is not limited to three shown in FIG.
As a method for detecting the organ position, a known technique may be used. For example, the technique disclosed in Japanese Patent Application Laid-Open No. 2009-211177 can be used.

Subsequently, the feature extraction unit 23 extracts a feature amount from each local region of the authentication image (S5). A known technique may be used as a method for extracting the feature amount. For example, LBP (Local Binary Pattern) feature values can be used. Alternatively, a HOG (Histogram of Oriented Gradient) feature amount or a SIFT (Scale-Invariant Feature Transform) feature amount may be used. Moreover, you may use the feature-value which mixed these. You may use the feature-value extracted by CNN (Convolutional Neural Network) learned beforehand. Further, the extracted feature quantity may be dimensionally compressed using PCA (Principal Component Analysis) or the like.
Subsequently, the feature storage unit 24 stores the extracted feature amount in the secondary storage device 14 (S6). Alternatively, the feature amount may be stored in the RAM 13.

Subsequently, an image authentication process is performed (S7). The image authentication process will be described in detail below. FIG. 6 is a diagram illustrating an example of a processing flow of image authentication processing.
First, the similarity calculation unit 28 acquires a registration dictionary from the dictionary registration unit 25 (S11). The registration dictionary is obtained by registering in advance the feature amount of the image of the person the user wants to authenticate, and is registered in advance by the user. The dictionary registration process will be described later.
Subsequently, the similarity calculation unit 28 acquires a similarity calculation parameter from the similarity calculation parameter storage unit 27 (S12). It is assumed that the similarity calculation parameter is calculated in advance and stored in the similarity calculation parameter storage unit 27. The calculation process of the similarity calculation parameter will be described later.
Subsequently, the similarity calculation unit 28 acquires a feature quantity to be authenticated from the feature storage unit 24 (S13). As described above, the feature amount acquired at this time is at least one feature amount acquired from the authentication image. One feature amount corresponds to one face of a person. In the present embodiment, since feature amounts are extracted for each local region for one face, one feature amount includes feature amounts for a plurality of local regions.

Subsequently, the similarity calculation unit 28 calculates the local similarity for the combination of the feature quantity to be authenticated and all the feature quantities in the registration dictionary (S14). Here, the “local similarity” is a similarity calculated for each local region. As described above, since the feature quantity to be authenticated and the registered dictionary are the feature quantities extracted for each local area, the similarity is calculated for each local area. A method for calculating the local similarity for one local region will be described below.
As a similarity calculation method, a known technique can be used. For example, the cosine similarity shown in the following formula (1) can be used.

ここで、・は内積を意味し、Ｓは特徴量ベクトルｘ’、ｙ’間の類似度である。さらに、特徴量ベクトルｘ’、ｙ’を以下の式（２）、式（３）で示す。
ｘ’＝ｘ_ｃ−μ_ｃ （２）
ｙ’＝ｙ−μ_ｃ （３）
と表す。

Here, · means an inner product, and S is the similarity between the feature quantity vectors x ′ and y ′. Furthermore, feature quantity vectors x ′ and y ′ are represented by the following formulas (2) and (3).
x ′ = x _c −μ _c (2)
y ′ = y−μ _c (3)
It expresses.

x _c represents a feature vector extracted from the authentication image captured by the camera c. y represents a feature vector registered in the registration dictionary. μ _c is an origin vector for angle calculation. That is, Equation (1) measures the cosine similarity between the feature vectors x _c and y with μ _c as the origin. The mu _c is similarity calculation parameters in the present embodiment. method will be described later calculation of mu _c.
Up to this point, the method for calculating the similarity for one local region has been described. Similarities are similarly calculated for other local regions.
Subsequently, the similarity calculation unit 28 calculates the integrated similarity based on the calculated local similarity for all the combinations of the feature quantities to be authenticated and the registered dictionary (S15). Here, the “integrated similarity” is a similarity obtained by integrating a plurality of local similarities calculated for each local region.
A known technique may be used for the integration method. For example, it is possible to use a method such as selecting an average of local similarity and a maximum value of local similarity. In addition, the weight for each local region may be calculated in advance, and the local similarity may be integrated with the weighted average. In the present embodiment, the local similarity calculation (S14) and the integrated similarity calculation (S15) are collectively referred to as similarity calculation. Further, the calculated integrated similarity may be normalized so that the value range is, for example, 1 to 1000.

Subsequently, the authentication result determination unit 29 determines who is or is not in the registration dictionary for all persons in the authentication target image based on the integrated similarity (S16). . Specifically, if the highest integrated similarity is greater than a predetermined threshold, the authentication result determination unit 29 determines that the person is in the registered dictionary corresponding to the integrated similarity. Conversely, if the highest similarity is equal to or less than a predetermined threshold, the authentication result determination unit 29 determines that no one is in the registration dictionary.
Thus, the image authentication process of FIG. 6 is completed.
Thereafter, the process returns to S7 in FIG. 4 and the subsequent processing is executed. That is, the display unit 30 displays the authentication result acquired from the authentication result determination unit 29 (S8).

FIG. 7 is a diagram schematically illustrating an example of a display screen of the authentication result. As shown in FIG. 7, on this display screen 60, images of persons registered in the registration dictionary (in this example, person A) 61a, 61b are displayed in the left column, and authentication results are displayed in the right column. Yes. In the present embodiment, among the authentication images, images 62a to 62c of persons who are determined to be the principal, similarities 63a to 63c corresponding to the images, the names of the captured cameras, and imaging times are displayed. By displaying in this way, it is possible to confirm when and on which camera the person registered in the registration dictionary was captured.
The above is the monitoring / authentication process.

In the present embodiment, the feature amount of the authentication image is stored in advance, and the image authentication process is executed when the user authentication instruction is given. However, the present invention is not limited to this, and the feature amount is extracted. Image authentication processing may be executed.
In this embodiment, the image authentication process is executed for all the authentication images acquired in S3 of the monitoring / authentication process flow (FIG. 4). However, a part of the authentication images is selected and the image authentication process is executed. You may make it do. For example, authentication images may be thinned out at predetermined intervals, or feature amounts extracted from authentication images may be clustered, and a representative feature amount may be selected from one cluster for image authentication processing. .

<Similarity calculation parameter calculation processing>
Next, processing in which the image authentication apparatus 1 according to the present embodiment calculates similarity calculation parameters corresponding to the installed cameras 2 to 4 will be described. This process is executed by selecting “adjustment mode” from the console screen of FIG. This process is preferably performed before the above-described monitoring / authentication process. As a result, the monitoring / authentication process can be performed with the similarity calculation parameter suitable for each camera installation location and angle of view.
FIG. 8 shows an example of the processing flow of similarity calculation parameter calculation processing. The actual processing will be described in detail with reference to this figure.
It is assumed that the cameras 2 to 4 continue to capture images at a predetermined frame rate from the start to the end of this flow, and continue to store the captured images in the secondary storage device 14.

First, the authentication image acquisition unit 21 selects one camera for which the calculation of the similarity calculation parameter has not been completed (S21).
Subsequently, the authentication image acquisition unit 21 selects a captured image captured by the camera selected in S21 from the images stored in the secondary storage device 14, and acquires a human face image from the captured image. (S22). In order to know which camera has captured the captured image, a label stored in association with the captured image may be referred to. The method for acquiring a person's face image from the captured image is the same as the method described in the above-described monitoring / authentication process, and thus the description thereof is omitted. In the present embodiment, the captured image stored in the secondary storage device 14 is acquired. However, the captured image may be acquired directly from each camera.

Since the local region setting (S23) and the feature amount extraction (S24) are the same as those described above, the description thereof is omitted.
Subsequently, the feature storage unit 24 stores the feature amount (S25). At this time, in the present embodiment, a label indicating the feature amount extracted for calculating the similarity calculation parameter is stored in association with the feature amount.
Subsequently, if the number of feature amounts extracted for calculating the similarity calculation parameter is less than a predetermined number (No in S26), the process returns to S22 and the subsequent processing is repeated. That is, feature amount collection (extraction and storage) is repeated until a sufficient number of feature amounts are collected to calculate the similarity calculation parameter. If the collected feature quantity is greater than the predetermined number (Yes in S26), the process proceeds to S27.
Subsequently, the similarity calculation parameter calculation unit 26 calculates a similarity calculation parameter using the collected feature amount (S27). As described above, the similarity calculation parameter in the present embodiment is the origin of cosine similarity. Hereinafter, a method of calculating the similarity calculation parameter corresponding to one local area will be described.
The origin μ _c corresponding to the camera c is calculated by averaging the feature quantity vectors x _ci collected for each installed camera as shown in the following formula (4).

ここで、ｘ_ｃｉはカメラｃが撮像した認証画像から収集したｉ番目の特徴量ベクトルであることを示している。また、Ｎは収集した特徴量ベクトルのサンプル数である。この原点μ_ｃ、すなわち類似度算出パラメタを用いて、監視・認証処理で類似度を算出する。

Here, x _ci indicates the i-th feature vector collected from the authentication image captured by the camera c. N is the number of collected feature vector samples. Using this origin μ _c , that is, the similarity calculation parameter, the similarity is calculated by the monitoring / authentication process.

  FIG. 9 is a diagram schematically showing an effect obtained by calculating the origin, and represents a feature vector space of a certain local region imaged by a certain camera. Each white dot in the figure represents a feature vector, and a set of feature vectors, that is, a cluster, is represented by a dotted line. FIG. 9 shows a predetermined origin 71, a feature vector cluster 72 of the authentication image collected by the installed camera, a feature vector cluster 73 of the person A, a feature vector cluster 74 of the person B, and collected by the installed camera. The origin 75 calculated based on the authentication image is shown. When calculating similarity based on an angle like cosine similarity, the resolution of distinguishing feature vectors is higher when the origin is inside the cluster created by the feature vector to be authenticated.

  Since the predetermined origin 71 is outside the cluster created by the feature vector to be authenticated, for example, the feature vector of person A (one of the feature vectors in 73) and the feature vector of person B are When the angle is measured with reference to the origin 71 determined in the above, the angle is almost zero. For this reason, when the cosine similarity is calculated, it should be a different person, but a high similarity that is erroneously determined as the principal is calculated. The predetermined origin 71 is generally calculated based on a large number of authentication images prepared in advance.

However, the feature vector distribution of the authentication image collected by the installed camera may be different from the feature vector distribution of the authentication image prepared in advance. In particular, the installed camera has a limited angle of view and location, so that images with specific lighting conditions and face orientations are larger than authentication images prepared in advance. In such a case, an erroneously high similarity is calculated, leading to erroneous authentication. Therefore, by using the origin 75 calculated based on the authentication image collected by the installed camera, it enters the inside of the feature vector cluster 72 of the authentication image collected by the installed camera. As a result, the cosine similarity can be calculated using an origin suitable for authentication for each camera installation location and angle of view, and it is expected that false authentication will be reduced.
However, the position of the origin in the feature amount space calculated based on the authentication image collected by the installed camera does not always come to a position suitable for authentication.

  FIG. 10 shows a feature vector space as in FIG. 9. A predetermined origin 76, a feature vector cluster 77 of an authentication image collected by an installed camera, a feature vector cluster 78 of a person C, installation The origin 79 calculated based on the authentication images collected by the camera is shown. In this case, the predetermined origin 71 is not suitable for calculating the cosine similarity as in FIG. 9, but the origin 79 calculated based on the authentication image collected by the installed camera is the feature vector of the person C. It is inside the cluster 78. In this case, if the cosine similarity is calculated, erroneous authentication may occur in which the person C of the feature vector cluster 78 and another person are erroneously determined as the principal. Thus, if the calculated origin falls inside someone's feature vector cluster, it cannot be said to be an origin suitable for authentication.

  Therefore, in this embodiment, instead of one feature amount for one authentication image, a plurality of feature amounts are extracted by setting a plurality of local regions, and finally the similarity for each of the plurality of feature amounts is obtained. Integrated similarity is calculated. This reduces false authentication when there is a local area where the calculated similarity calculation parameter is not suitable for authentication. In other words, even if a similarity calculation parameter that is not suitable for authentication in a certain local area is calculated, and a high similarity that determines another person as the person is calculated, other similarity calculation parameters that are suitable for authentication can be calculated. Since it is finally integrated with the similarity of the local area, it can be expected to reduce false authentication.

In the present embodiment, a plurality of feature amounts are extracted by setting a plurality of local regions, but a plurality of feature amounts may be extracted by other methods. For example, a plurality of feature amounts may be extracted by extracting different types of feature amounts.
In this embodiment, the average of feature vectors is used as the origin, but the present invention is not limited to this. In addition, the origin may be calculated using another method. For example, a weighted average may be used instead of a simple average of feature vectors. At this time, it is preferable to normalize the weight value so that the origin is inside the cluster created by the feature vector to be authenticated.
The above is the calculation method of the similarity calculation parameter corresponding to one local region. This is calculated for all other local regions.

Returning to FIG. 8, subsequently, the similarity calculation parameter storage unit 27 stores the calculated similarity calculation parameter (S28).
Subsequently, if the similarity calculation parameter calculation process corresponding to all the cameras has been completed (Yes in S29), the similarity calculation unit 28 completes the similarity calculation parameter calculation process. If there is a camera that has not been completed (No in S29), the process returns to S21, and the image authentication apparatus 1 repeats the subsequent processing.
The above is the description of the similarity calculation parameter calculation processing.

  FIG. 11 is a diagram schematically illustrating an example of a screen 80 displayed during execution of the similarity calculation parameter calculation processing. FIG. 11 schematically shows a state in which a predetermined number of authentication image numbers (number of feature quantities) necessary for adjusting the similarity calculation parameter is presented to the user. FIG. 11 shows that the camera 2 and the camera 3 have enough images (81a, 82a, 81b, 82b) to calculate the similarity calculation parameter, and the camera 4 is still insufficient. (81c, 82c). In this way, by indicating to the user that a predetermined number of images have been accumulated, it is possible to visually know how many images should be collected in order to calculate the similarity calculation parameter. .

  In this embodiment, the similarity calculation parameter calculation processing is not completed unless the similarity calculation parameters corresponding to all the cameras are calculated. However, the present invention is not limited to this. For example, the similarity calculation parameter calculation process may be interrupted before completing the calculation of some or all of the similarity calculation parameters. In that case, the similarity calculation parameter used in the monitoring / authentication process may be prepared by another method. For example, a default similarity calculation parameter prepared in advance may be used. Alternatively, although the predetermined number is insufficient, the similarity calculation parameter may be calculated using the feature amount collected for each camera. Alternatively, feature quantities that are insufficient for calculation may be further collected in the background of the monitoring / authentication process, and the similarity calculation parameter may be calculated when a predetermined number is reached. In that case, a default similarity calculation parameter prepared in advance may be used until the calculation of the similarity calculation parameter is finished, and when the calculation is finished, the calculated similarity calculation parameter may be replaced.

<Dictionary registration process>
Next, a dictionary registration process for registering a person to be authenticated will be described. This process is executed by selecting “dictionary registration mode” from the console screen of FIG.
FIG. 12 shows an example of the processing flow of dictionary registration processing. The actual processing will be described in detail with reference to this figure.
First, the authentication image acquisition unit 21 acquires an image for dictionary registration (S31). Note that the dictionary registration image does not necessarily have to be captured by the cameras 2 to 4, and an image captured in advance may be used.
The subsequent processing from local region setting (S32) to feature extraction (S33) is the same as the processing S23 and S24 in FIG.

Subsequently, the dictionary registration unit 25 registers the feature amount in the dictionary (S34). At this time, the person ID of the registered person is registered in association with the feature quantity, that is, stored in the secondary storage device 14. What is necessary is just to acquire person ID linked | related with a feature-value by user input.
The above is the description of the dictionary registration process.
It is determined by the above-described monitoring / authentication process whether or not the person registered here is reflected in the collected captured image. Note that the number of persons to be registered is not particularly limited, and a plurality of feature amounts may be registered per person.
The above is description of Embodiment 1 in this invention.

As described above, in this embodiment, authentication authentication is acquired from the captured image captured by the installed camera, and the similarity calculation parameter corresponding to the installed camera is calculated, thereby realizing person authentication robust to the imaging conditions. Explained how to do.
By installing the camera, imaging conditions such as the orientation of a person to be imaged and illumination conditions are fixed to some extent. Therefore, by using similarity calculation parameters specialized for imaging conditions such as camera installation location and angle of view, people (objects) can be authenticated more accurately than using similarity calculation parameters prepared in advance. can do.

In the first embodiment, the cosine similarity is used as the similarity and the origin at the time of calculating the cosine similarity is calculated as the similarity calculation parameter. However, another similarity may be used. For example, the Mahalanobis matrix M may be calculated as the similarity calculation parameter using the inverse of the Mahalanobis distance as the similarity S as shown in the following formulas (5) and (6).

ここで、ｘ_ｃ、ｙは、それぞれ、カメラｃで撮像された人物の特徴量ベクトル、登録辞書の特徴量ベクトルを表す。

Here, x _c and y represent a feature vector of a person imaged by the camera c and a feature vector of a registered dictionary, respectively.

FIG. 13 is a diagram schematically showing the effect when the Mahalanobis matrix is calculated as the similarity calculation parameter.
FIG. 13 shows the distribution of the feature vector, as in FIGS. In this distribution, an average vector 91 calculated in advance, a metric space 92 based on a Mahalanobis matrix calculated in advance, and an average vector 93 newly calculated based on an authentication image collected by an installed camera are shown. Further, the distribution shows a metric space 94 based on the newly calculated Mahalanobis matrix, a feature vector cluster 95 of person D, and a feature vector cluster 96 of person E.
Here, the metric space 92 means that the Mahalanobis distance from the average vector 91 calculated in advance is equal on the ellipse of the metric space 92. That is, the Mahalanobis distance of the metric space 92 is a distance scale in which the distance in the diagonal direction from the lower left to the upper right in FIG. 13 is shorter than the normal Euclidean distance. The distance space 94 is the same, and the distance from the upper left to the lower right in FIG. 13 is a shorter distance scale than the normal Euclidean distance.

Thus, the distance scale calculated from the authentication image collected by the installed camera is different from the distance scale calculated in advance. In the metric space 92 based on the Mahalanobis matrix calculated in advance in FIG. 13, the distance from the lower left to the upper right in FIG. 13 is a short distance measure. For this reason, the Mahalanobis distance between the feature vector of person D (any one point in the feature vector cluster 95 of person D) and the feature vector of person E is close. That is, there is a possibility that they are determined to be the same person. On the other hand, in the metric space 94 based on the newly calculated Mahalanobis matrix, the distance between the feature quantity vector of the person D and the feature quantity vector of the person E is conversely long, so that the possibility of being determined as a different person increases. As described above, the method of this embodiment can be applied to a method of setting a similarity calculation parameter in advance learning, such as Mahalanobis distance.
In the present embodiment, the authentication image is an image showing a person's face. However, the authentication image is not limited to this, and may be an image showing a human body or any other object.
As described above, in the present embodiment, a person (object) can be authenticated with higher accuracy by using a similarity calculation parameter suitable for the installation location and angle of view of the camera.

Embodiment 2
In the first embodiment, the example in which the similarity calculation parameter is calculated using the feature amount collected from the captured image of the camera has been described. In the present embodiment, an example in which the similarity calculation parameter is calculated during the dictionary registration process will be described. In addition, description is abbreviate | omitted about the location similar to Embodiment 1, and only a different location is demonstrated.
In the second embodiment, as in the first embodiment, an example in which a cosine similarity is used as a similarity used for authentication will be described. Since the monitoring / authentication process and the similarity calculation parameter calculation process are the same as those in the first embodiment, description thereof is omitted.
The dictionary registration process will be described.

FIG. 14 shows an example of the processing flow of dictionary registration processing in the present embodiment. The actual processing will be described in detail with reference to this figure.
Since registration image acquisition (S41) to dictionary registration (S44) are the same as those in the first embodiment (S31 to S34 in FIG. 12), description thereof is omitted.
Subsequent to S44, the similarity calculation parameter calculation unit 26 updates the similarity calculation parameter using the registration dictionary acquired from the dictionary registration unit 25 (S45). The updated similarity calculation parameter σ calculated in the similarity calculation parameter calculation processing is expressed by the following equation (7).
σ = w ₀ σ ₀ + w _r σ _r (7)
Here, σ ₀ is an average vector of feature values collected by the camera at the installation location, σ _r is an average vector of all feature values of the registered dictionary, and w ₀ and w _r are the respective similarity calculations. A weighting factor for the parameter.

Generally, compared to the number of features that can be collected with the installed camera, the number of features in the registered dictionary is small, so if you take the average including both features, the effect of the features collected with the installed camera will be affected. Become dominant. Therefore, in the present embodiment, the average vectors calculated respectively are added with predetermined weights so that the influence of the average vector obtained by the registration dictionary can be adjusted.
Note that σ ₀ may be calculated in advance, or all the feature values collected by the camera at the installation location are stored in the secondary storage device 14, and σ ₀ is calculated when calculating equation (7). It may be calculated.
Subsequently, the similarity calculation parameter storage unit 27 stores the similarity calculation parameter σ calculated by Expression (7) in the secondary storage device 14 (S46).
In the present embodiment, when a new feature amount is registered in the registration dictionary, the similarity calculation parameter is calculated every time. However, it may not be calculated every time. For example, it may be calculated when a predetermined number of feature amounts are newly registered in the registration dictionary.

In the present embodiment, the average vector σ _r of all the feature values of the registered dictionary is calculated by the equation (7), but may be calculated using other methods. For example, an average vector for each person in the registered dictionary may be calculated first, and then the average vector of each person may be further averaged. By doing in this way, when there is a big difference in the number of registration for every person, it can suppress that the influence of a person with many registration numbers does not become large too much.
The above is the description of the second embodiment of the present invention.
As described above, according to the present embodiment, not only the feature amount collected from the captured image of the installed camera but also the feature amount of the registration dictionary is reflected in the calculation of the similarity calculation parameter, thereby capturing the registered image. It is possible to calculate the similarity calculation parameter more suitable for the monk condition.

  DESCRIPTION OF SYMBOLS 1 ... Image authentication apparatus, 2, 3, 4 ... Camera, 5 ... Network, 21 ... Authentication image acquisition part, 22 ... Local area | region setting part, 23 ... Feature extraction part, 24 ... Feature storage part, 25 ... Dictionary registration part, 26 ... Similarity calculation parameter calculation unit, 27 ... Similarity calculation parameter storage unit, 28 ... Similarity calculation unit, 29 ... Authentication result determination unit, 30 ... Display unit

Claims (11)

Acquisition means for acquiring an image captured by at least one imaging device;
Extraction means for extracting feature amounts from the image acquired by the acquisition means;
Storage means for storing a registered dictionary in which image feature values are registered;
For each of the imaging devices, based on the feature amount extracted from the image captured by the imaging device, the feature amount extracted from the image captured by the imaging device and the feature amount registered in the registration dictionary First calculation means for calculating a similarity calculation parameter for calculating a similarity between
The similarity between the feature quantity of the authentication target image acquired by the acquisition means and the feature quantity registered in the registration dictionary is calculated by the first calculation means for the imaging device that captured the authentication target image. Second calculation means for calculating based on the calculated similarity calculation parameter;
An image processing apparatus comprising: The first calculation means calculates a plurality of similarity calculation parameters for one image acquired by the acquisition means,
The second calculating unit calculates a plurality of similarities based on the plurality of similarity calculating parameters calculated by the first calculating unit, and the calculated plurality of similarities are set as one similarity. Integrated into the
The image processing apparatus according to claim 1. A setting unit that sets a plurality of local regions for one image acquired by the acquiring unit;
The first calculation means calculates the similarity calculation parameter for each of the plurality of local regions set by the setting means;
The image processing apparatus according to claim 2. The first calculation means calculates the similarity calculation parameter based on the feature amount registered in the registration dictionary in addition to the feature amount extracted from the acquired image.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The similarity calculation parameter calculated by the first calculation means is an origin of angle calculation in the feature amount space,
The second calculation means calculates a similarity based on an angle using an angle calculation origin calculated by the first calculation means.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The extraction means extracts a plurality of feature amounts from the image,
The origin of the angle calculation calculated by the first calculation means is an average of the plurality of feature amounts extracted by the extraction means.
The image processing apparatus according to claim 5. The similarity calculation parameter calculated by the first calculation means is a Mahalanobis matrix,
The second calculating means calculates the similarity based on the Mahalanobis matrix calculated by the first calculating means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The acquisition means acquires the image to calculate a plurality of the similarity calculation parameters according to user input,
The first calculation unit calculates a plurality of the similarity calculation parameters based on the image acquired by the acquisition unit.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. A display means for presenting to the user that the acquisition means has acquired a predetermined number of images for calculating a plurality of the similarity calculation parameters;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Obtaining an image captured by at least one imaging device;
An extraction step of extracting a feature amount from the image acquired in the acquisition step;
For each of the imaging devices, based on the feature amount extracted from the image captured by the imaging device, the feature amount extracted from the image captured by the imaging device and the feature amount registered in the registration dictionary Calculating a similarity calculation parameter for calculating a similarity between,
The similarity between the authentication target image acquired in the acquisition step and the feature amount registered in the registration dictionary is based on the similarity calculation parameter calculated for the imaging device that captured the authentication target image. A calculating step;
An image processing method comprising:   A program for causing a computer to operate as the image processing apparatus according to any one of claims 1 to 9.

Images