JP6873639B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to an image processing device, an image processing method and a program for authenticating an object in an image.

There is known a person authentication technology for determining who is in the image captured by the installed camera. Applications of this person authentication technology include criminal investigations using facial photographs and searching for lost children. In these application destinations, since the installation location and the angle of view are different for each installed camera, the imaging conditions (for example, the direction of the person and the lighting conditions) for photographing a person are also different for each camera. Therefore, robust person authentication technology is required for imaging conditions.
Generally, in the person authentication technology, a person is among the registered images based on the degree of similarity calculated between the captured image of the person and the image of the person registered in advance (hereinafter referred to as "registered image"). It determines who it is or who it is not. How to calculate this similarity is one of the important factors that determine the accuracy of person authentication, and various methods have been proposed.

For example, in the technique disclosed in Non-Patent Document 1, parameters for calculating similarity are learned in advance from a large amount of data, and the similarity is calculated based on the learned parameters. By learning the parameters in advance using a large amount of data including various imaging conditions, it is possible to realize person authentication that is robust to the imaging conditions to some extent.
Further, in the technique disclosed in Patent Document 1, an input subspace is calculated from an input pattern, a plurality of dictionary subspaces are calculated from a dictionary pattern corresponding to a predetermined category, and the input subspace and a plurality of dictionary portions are calculated. Each of the spaces is linearly transformed using an orthogonalization matrix. Then, the similarity between the linearly transformed input subspace and the linearly transformed plurality of dictionary subspaces is calculated. The orthogonalization matrix for linear transformation is generated as a transformation matrix in which a plurality of dictionary subspaces are orthogonalized to make it easier to distinguish between persons included in the registered image.

Japanese Patent No. 4343125

H. V. Nguyen and L. Bai, "Cosine Science Learning for Face Verification", 2010, ACCV (Annual Computer of Computer Vision).

However, with the techniques described in the above-mentioned documents, it is not possible to obtain similarity calculation parameters suitable for the camera installation location and angle of view, so that the accuracy of person authentication may decrease. That is, in the technique of Non-Patent Document 1, since the similarity calculation parameter is learned in advance from a large amount of data, it is not possible to calculate the similarity calculation parameter suitable for the camera installation location and angle of view in advance. Can not. Further, the technique of Patent Document 1 calculates a transformation matrix that orthogonalizes the subspace of each person included in the registered image, and is suitable for the installation location and angle of view of the camera that images the person to be authenticated. It is not possible to calculate the similarity calculation parameter.
The present invention has been made to solve the above problems, and an object of the present invention is to authenticate an object with higher accuracy by using a similarity calculation parameter suitable for a camera installation location and an angle of view.

In order to solve the above problems , according to an aspect of the present invention, an acquisition means for acquiring an image captured by at least one image pickup device and an extraction means for extracting a feature amount from the image acquired by the acquisition means. When a storage unit for storing a feature quantity of a person in the image to be authenticated, for each of the imaging device, on the basis of the feature quantity extracted from the image captured by the image capturing apparatus, captured by the imaging device Based on the first calculation means for calculating the similarity calculation parameter for identifying whether the image to be authenticated includes the person to be authenticated and the similarity calculation parameter calculated according to the image pickup apparatus. An image processing apparatus is provided that includes a second calculation means for calculating the degree of similarity between the converted extracted feature amount and the stored feature amount.

According to the present invention, it is possible to authenticate an object with higher accuracy by using a similarity calculation parameter suitable for a camera installation location and an angle of view.

The block diagram which shows an example of the hardware configuration of the image authentication system which concerns on Embodiment 1. FIG. The block diagram which showed the functional structure of the control program of the image authentication system which concerns on Embodiment 1. FIG. The figure which showed typically the console screen of the image authentication system which concerns on Embodiment 1. FIG. The flowchart which shows the monitoring / authentication processing of the image authentication system which concerns on Embodiment 1. The figure which shows typically the local area to be set in the authentication image. The flowchart which shows the image authentication processing of the image authentication system which concerns on Embodiment 1. The figure which conceptually showed an example of the content to be displayed as an authentication result. The flowchart which shows the similarity calculation parameter calculation process of the image authentication system which concerns on Embodiment 1. The figure which showed typically the example of calculating the origin suitable for the installed camera. The figure which showed typically the example when the calculated origin was not in an appropriate position in the feature space. The figure which showed typically the screen which shows the sample collection situation for calculation which the image authentication system which concerns on Embodiment 1 displays during the similarity calculation parameter calculation process. The flowchart which shows the dictionary registration process of the image authentication system which concerns on Embodiment 1. The figure which showed typically the example of calculating the Mahalanobis matrix suitable for the installed camera. The flowchart which shows the dictionary registration process of the image authentication system which concerns on Embodiment 2.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example as a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. It is not limited to the embodiment. Moreover, not all combinations of features described in the present embodiment are essential for the means of solving the present invention.

Embodiment 1
In the image authentication system according to the first embodiment of the present invention, an example in which a similarity calculation parameter is calculated for each installed camera and a specific person is authenticated from the captured images captured will be described.
<Hardware configuration>
FIG. 1 is a block diagram showing an example of the hardware configuration of the image authentication system according to the present embodiment. This image authentication system includes an image authentication device 1, cameras 2 to 4, and a network 5.
The image authentication device 1 is an image authentication device that authenticates a specific person from an image, and a specific configuration will be described later.
The cameras 2 to 4 are so-called network cameras and function as an imaging device. Each of the cameras 2 to 4 includes a visible light camera unit that includes a lens and an image pickup element such as a CCD or a CMOS sensor to capture visible light, and a communication device for connecting to a network. As the cameras 2 to 4, other cameras having a communication function may be used. Further, the camera unit is not limited to the one that captures visible light, and an infrared camera unit or the like that captures infrared rays may be used.
The image authentication device 1 and the cameras 2 to 4 are connected to each other via the network 5, and data communication is possible with each other.
Further, although FIG. 1 shows a configuration using three cameras (cameras 2 to 4), the number of cameras is not limited to this.

The image authentication device 1 includes a CPU (Central Processing Unit) 11, a ROM 12, a RAM 13, a secondary storage device 14, a communication device 15, a video output device 16, an operation input device 17, and a connection bus 18. I have.
The CPU 11 controls the entire image recognition device 1 by executing a control program stored in the ROM 12 or the RAM 13. The ROM 12 is a non-volatile memory and stores a control program and various parameter data. The control program is executed by the CPU 11 and realizes each process described later. The RAM 13 is a volatile memory, and temporarily stores an image, a control program, and an execution result thereof.

The secondary storage device 14 is a rewritable secondary storage device such as a hard disk or a flash memory, and stores images received from the cameras 2 to 4 via the communication device 15. Further, the secondary storage device 14 stores information such as a control program, various setting contents, and processing results as needed. The information stored in the secondary storage device 14 is output to the RAM 13 as needed, and is used by the CPU 11 for executing a control program or the like.
The communication device 15 is, for example, a wired communication unit, and performs data communication with the cameras 2 to 4 via the network 5. The communication device 15 may be a wireless communication unit.
The video output device 16 is a monitor device such as a CRT or a TFT liquid crystal, and displays an image acquired from the RAM 13, an execution result of a control program, and the like.
The operation input device 17 is a device for inputting an operation from a user such as a keyboard or a mouse. User input (instruction input from the user), which will be described later, is acquired via the operation input device 17.
The connection bus 18 connects each device constituting the image authentication device 1 and performs data communication with each other.

In the present embodiment, the processing described later will be described as being realized by software using the CPU 11 of the image authentication device 1, but a part or all of the processing of the image authentication device 1 will be realized by hardware. It doesn't matter. As the hardware in this case, for example, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used.
Further, software describing the processing described later may be acquired via a network or various storage media and executed by a processing device (CPU, processor) such as a personal computer.
Further, the cameras 2 to 4 may execute a part or all of the authentication process of the image authentication device 1. In that case, the cameras 2 to 4 may be provided with the hardware necessary for the processing performed by the image authentication device 1.

<Functional configuration of image authentication device>
FIG. 2 is a block diagram showing an example of a functional configuration realized by executing a control program in the image authentication device 1 in the present embodiment. The image authentication device 1 of the present embodiment includes an authentication image acquisition unit 21, a local area setting unit 22, a feature extraction unit 23, a feature storage unit 24, a dictionary registration unit 25, a similarity calculation parameter calculation unit 26, and a similarity calculation parameter storage. A unit 27, a similarity calculation unit 28, an authentication result determination unit 29, and a display unit 30 are provided.
The authentication image acquisition unit 21 acquires an authentication target image (authentication target image) based on the captured images acquired from the cameras 2 to 4, and outputs the authentication image to the local area setting unit 22. Here, the "authentication image" is an image in which an object to be authenticated is captured, and in the present embodiment, is an image in which a person's face is captured.

The local area setting unit 22 extracts coordinate information of feature points such as eyes and nose of a person's face from the authentication image acquired from the authentication image acquisition unit 21, and sets a local area based on the extracted feature points. Then, the local area setting unit 22 outputs the local area together with the authentication image to the feature extraction unit 23.
The feature extraction unit 23 outputs the feature amount extracted from the local area of the authentication image acquired from the local area setting unit 22 to the feature storage unit 24. Further, the feature extraction unit 23 also outputs the extracted feature amount to the dictionary registration unit 25 and the similarity calculation parameter calculation unit 26. In this embodiment, the feature amount is extracted for each local region of the authentication image.
The feature storage unit 24 stores the feature amount acquired from the feature extraction unit 23 in the secondary storage device 14. It may be stored in the RAM 13. The stored feature amount is output to the similarity calculation unit 28 as requested.

The dictionary registration unit 25 stores the feature amount acquired from the feature extraction unit 23 in the secondary storage device 14 as the data of the registration dictionary. Alternatively, the registered dictionary may be stored in the RAM 13. Here, the "registered dictionary" is a set of data in which a feature amount and a person ID (generally, a person name or the like) corresponding to the feature amount are associated with each other. The method of acquiring the person ID will be described later. The registered dictionary may include a plurality of features of a person, or may include a plurality of features for one person. Further, the registered dictionary is output to the similarity calculation parameter calculation unit 26 and the similarity calculation unit 28 according to the request.
The similarity calculation parameter calculation unit 26 calculates the similarity calculation parameter based on the feature amount acquired from the feature extraction unit 23 or the feature amount of the registered dictionary acquired from the dictionary registration unit 25, and stores the similarity calculation parameter. Output to unit 27. The details of the calculation process of the similarity calculation parameter will be described later.

The similarity calculation parameter storage unit 27 stores the similarity calculation parameter acquired from the similarity calculation parameter calculation unit 26 in the secondary storage device 14. The stored similarity calculation parameter is output to the similarity calculation unit 28 in response to a request.
The similarity calculation unit 28 also includes a similarity calculation parameter obtained from the similarity calculation parameter storage unit 27 for the similarity between the feature amount acquired from the feature storage unit 24 and the registered dictionary acquired from the dictionary registration unit 25. And calculate. The calculated similarity is output to the authentication result determination unit 29.
The authentication result determination unit 29 determines, based on the similarity degree obtained from the similarity calculation unit 28, the authentication result, that is, who or not the person in the authentication image is in the registered dictionary. To do. The authentication result is output to the display unit 30.
The display unit 30 displays the authentication result acquired from the authentication result determination unit 29 on the video output device 16.

<Overview of overall processing>
The process in this embodiment includes three processes: a similarity calculation parameter calculation process, a monitoring / authentication process, and a dictionary registration process. In the similarity calculation parameter calculation process, the similarity calculation parameters corresponding to the installed cameras 2 to 4 are calculated. In the monitoring / authentication process, it is determined whether or not a specific person is captured while taking and storing captured images using cameras 2 to 4. In the dictionary registration process, the feature amount of the person to be authenticated is added to the registered dictionary.
At the start of each process, the cameras 2 to 4 are already installed in different places, and the image authentication device 1 can immediately acquire an image from the cameras 2 to 4 and start the process described later.

FIG. 3 is a diagram showing an example of a console screen displayed on the video output device 16 by the image authentication system according to the present embodiment. On the console screen 40, buttons 41 to 43 for instructing the operation mode, buttons 44 for instructing various settings, a button 45 for instructing the end, and a pointer 46 are displayed. As a result, a menu of which process to execute is displayed. The user operates the operation input device 17 and selects a process in the menu with the pointer 46. By selecting any of the buttons 41 to 43 of the "monitoring / authentication mode", "dictionary registration mode", and "adjustment mode", the image authentication device 1 executes the process described later.
For example, when the user selects the "setting" button 44 with the pointer 46, the image authentication device 1 executes a process of changing various settings of the image authentication system of the present embodiment. Further, when the user selects the "end" button 45 with the pointer 46, the image authentication device 1 terminates the image authentication system of the present embodiment.

<Monitoring / Authentication processing>
First, the monitoring / authentication process of determining whether or not a specific person is captured by the image authentication device 1 of the present embodiment while capturing image data of a person using cameras 2 to 4 and authenticating according to user input will be described. To do. This process is executed by the user operating the operation input device 17 and instructing the button 41 of the "monitoring / authentication mode" of the console screen 40 of FIG. 3 with the pointer 46.
During this monitoring / authentication process, it is assumed that the cameras 2 to 4 constantly take images and store the captured images in the secondary storage device 14. At this time, the image authentication device 1 stores the captured image in association with a label indicating which camera captured the captured image.
Further, this monitoring / authentication process proceeds according to the user input acquired via the operation input device 17. There are two user inputs, an "end instruction" for ending the monitoring / authentication process and an "authentication instruction" for executing image authentication. These two user inputs can be received at any time during the monitoring / authentication process shown in FIG.

FIG. 4 shows an example of the processing flow of the monitoring / authentication process executed by the image authentication device 1. The actual processing will be described in detail with reference to this figure.
First, the authentication image acquisition unit 21 ends this processing flow when there is a user-input "end instruction" (Yes in S1). When there is no "end instruction" (No in S1), and when there is a user-input "authentication instruction" (Yes in S2), an authentication image is acquired (S3). As described above, the authentication image is an image in which a person's face is captured. In order to acquire the authentication image, the face of a person may be detected from the captured image stored in the secondary storage device 14, and the face of the detected person may be cut out. As a method for detecting a person's face from a captured image, a known technique may be used.
For example, the techniques disclosed in the following documents can be used.
P. Viola and M. Jones, "Robust Real-Time Face Detection", Inpage II: 747, 2001.
A face image is cut out from the captured image based on the detected coordinates on the face image. At this time, image normalization may be performed in order to make the in-plane rotation of the face with respect to the image surface constant. For example, the image may be rotated so that the straight line connecting both eyes of the face is horizontal to the image.

In the present embodiment, the face of a person is detected from a plurality of captured images stored in the secondary storage device 14, and the face of the detected person, that is, the authentication image is cut out. The user may select which captured image is to be detected and cut out. For example, the user may be allowed to specify a time range, and the captured image captured within the time range may be processed.
Subsequently, the local area setting unit 22 sets the local area for all the authentication images (S4). Here, first, the positions of organs (organ positions) such as the eyes and nose of the face are detected, and a local region is set based on the detected organ positions. That is, the local area setting unit 22 sets a plurality of local areas for one authentication target image.

FIG. 5 is a diagram schematically showing a local region set in the authentication image. As shown in this figure, a local region 51 for the right eye, a local region 52 for the left eye, and a local region 53 for the mouth are set for the authentication image, that is, the face image 50 of a person.
In the present embodiment, the rectangular region centered on the detected organ position is set as the local region, but another method may be used as the method for setting the local region. It does not have to be a rectangular area centered on the organ position. For example, it may be a region based on a position separated by a predetermined number of pixels from the organ position, or may be a circular region instead of a rectangular region. In addition, a predetermined region of the facial image may be defined in advance without using the detected organ position as a reference. Further, a plurality of local regions may be set for one organ position, and the number of local regions is not limited to the three shown in FIG.
A known technique may be used for the method of detecting the organ position. For example, the technique disclosed in Japanese Patent Application Laid-Open No. 2009-21177 can be used.

Subsequently, the feature extraction unit 23 extracts the feature amount from each local region of the authentication image (S5). A known technique may be used for the method of extracting the feature amount. For example, an LBP (Local Binary Pattern) feature amount can be used. Alternatively, a HOG (Histogram of Oriented Gradient) feature quantity or a SIFT (Scale-Invariant Feature Transition) feature quantity may be used. Moreover, you may use the feature quantity which mixed these. The feature amount extracted by CNN (Convolutional Neural Network) learned in advance may be used. Further, the extracted feature amount may be dimensionally compressed using PCA (Principal Component Analysis) or the like.
Subsequently, the feature storage unit 24 stores the extracted feature amount in the secondary storage device 14 (S6). Alternatively, the feature amount may be stored in the RAM 13.

Subsequently, the image authentication process is performed (S7). The image authentication process will be described in detail below. FIG. 6 is a diagram showing an example of a processing flow of the image authentication process.
First, the similarity calculation unit 28 acquires a registered dictionary from the dictionary registration unit 25 (S11). In the registration dictionary, the feature amount of the image of the person to be authenticated by the user is registered in advance, and it is assumed that the feature amount is registered by the user in advance. The dictionary registration process will be described later.
Subsequently, the similarity calculation unit 28 acquires the similarity calculation parameter from the similarity calculation parameter storage unit 27 (S12). It is assumed that this similarity calculation parameter is calculated in advance and stored in the similarity calculation parameter storage unit 27. The calculation process of the similarity calculation parameter will be described later.
Subsequently, the similarity calculation unit 28 acquires the feature amount to be authenticated from the feature storage unit 24 (S13). As described above, the feature amount acquired at this time is at least one or more feature amount acquired from the authentication image. One feature corresponds to one person's face. Further, in the present embodiment, since the feature amount is extracted for each local region for one face, one feature amount includes the feature amount for each of a plurality of local regions.

Subsequently, the similarity calculation unit 28 calculates the local similarity for the combination of the feature amount to be authenticated and all the feature amounts of the registered dictionary (S14). Here, the "local similarity" is the similarity calculated for each local region. As described above, since the feature amount to be authenticated and the registered dictionary are the feature amount extracted for each local area, the similarity is calculated for each local area. The method of calculating the local similarity for one local region will be described below.
A known technique can be used as the similarity calculation method. For example, the cosine similarity shown in the following equation (1) can be used.

ここで、・は内積を意味し、Ｓは特徴量ベクトルｘ’、ｙ’間の類似度である。さらに、特徴量ベクトルｘ’、ｙ’を以下の式（２）、式（３）で示す。
ｘ’＝ｘ_ｃ−μ_ｃ （２）
ｙ’＝ｙ−μ_ｃ （３）
と表す。

Here, · means the inner product, and S is the degree of similarity between the feature vectors x'and y'. Further, the feature vector x'and y'are shown by the following equations (2) and (3).
x'= x _c − μ _c (2)
y'= y−μ _c (3)
It is expressed as.

x _c represents a feature amount vector extracted from the authentication image captured by the camera c. y represents a feature amount vector registered in the registration dictionary. μ _c is the origin vector for angle calculation. That is, the equation (1) measures the cosine similarity between the _{feature vectors x c and y with μ c} as the origin. This μ _c is a similarity calculation parameter in this embodiment. The calculation method of μ _c will be described later.
So far, the method of calculating the similarity for one local region has been described. The similarity is calculated for other local regions in the same manner.
Subsequently, the similarity calculation unit 28 calculates the integrated similarity based on the calculated local similarity for all the features to be authenticated and the combination of the registered dictionaries (S15). Here, the "integrated similarity" is a similarity obtained by integrating a plurality of local similarities calculated for each local region.
A known technique may be used for the method of integration. For example, a method such as selecting the average of local similarity and the maximum value of local similarity can be used. In addition, the weight for each local region may be calculated in advance, and the local similarity may be integrated by the weighted average. In the present embodiment, the local similarity calculation (S14) and the integrated similarity calculation (S15) are collectively referred to as the similarity calculation. Further, the calculated integrated similarity may be normalized so that the value range is, for example, 1 to 1000.

Subsequently, the authentication result determination unit 29 determines who or not the registered dictionary for all the persons in the image to be authenticated based on the integrated similarity (S16). .. Specifically, if the highest integrated similarity is larger than a predetermined threshold value, the authentication result determination unit 29 determines that the person is a person in the registered dictionary corresponding to the integrated similarity. On the contrary, when the highest degree of similarity is equal to or less than a predetermined threshold value, the authentication result determination unit 29 determines that the dictionary is not any of the registered dictionaries.
This completes the image authentication process of FIG.
After that, the process returns to S7 in FIG. 4 and the subsequent processing is executed. That is, the display unit 30 displays the authentication result acquired from the authentication result determination unit 29 (S8).

FIG. 7 is a diagram schematically showing an example of an authentication result display screen. As shown in FIG. 7, on the display screen 60, images of persons (persons A in this example) 61a and 61b registered in the registration dictionary are displayed in the left column, and the authentication result is displayed in the right column. There is. In the present embodiment, the images 62a to 62c of the person determined to be the person, the corresponding similarity 63a to 63c, the imaged camera name, and the imaging time are displayed among the authenticated images. By displaying in this way, it is possible to confirm when and which camera the person registered in the registration dictionary was photographed.
The above is the monitoring / authentication process.

In the present embodiment, the feature amount of the authenticated image is stored in advance, and the image authentication process is executed when the user is instructed to authenticate, but the present invention is not limited to this, and the feature amount is extracted. The image authentication process may be executed.
Further, in the present embodiment, the image authentication process is executed for all the authentication images acquired in S3 of the monitoring / authentication processing flow (FIG. 4), but some authentication images are selected and the image authentication process is executed. You may try to do it. For example, the authentication images may be thinned out at predetermined intervals, or the feature amounts extracted from the authentication images may be clustered, and a representative feature amount may be selected from one cluster to perform the image authentication process. ..

<Similarity calculation parameter calculation process>
Next, a process in which the image authentication device 1 of the present embodiment calculates the similarity calculation parameters corresponding to each of the installed cameras 2 to 4 will be described. This process is executed by selecting the "adjustment mode" from the console screen of FIG. It is preferable that this process is performed before the above-mentioned monitoring / authentication process in terms of time. As a result, monitoring / authentication processing can be performed with similarity calculation parameters suitable for each camera installation location and angle of view.
FIG. 8 shows an example of the processing flow of the similarity calculation parameter calculation process. The actual processing will be described in detail with reference to this figure.
It is assumed that the cameras 2 to 4 continue to capture images at a predetermined frame rate from the start to the end of this flow, and continue to store the captured images in the secondary storage device 14.

First, the authentication image acquisition unit 21 selects one camera for which the calculation of the similarity calculation parameter has not been completed (S21).
Subsequently, the authentication image acquisition unit 21 also selects an image captured by the camera selected in S21 from the images stored in the secondary storage device 14, and acquires a person's face image from the image. (S22). In order to know which camera captured the captured image, the label stored in association with the captured image may be referred to. Further, since the method of acquiring the face image of a person from the captured image is the same as the method described in the above-mentioned monitoring / authentication process, the description thereof will be omitted. In the present embodiment, the captured image stored in the secondary storage device 14 is acquired, but the captured image may be acquired directly from each camera.

Since the local area setting (S23) and the feature amount extraction (S24) are the same as the above-mentioned methods, the description thereof will be omitted.
Subsequently, the feature storage unit 24 stores the feature amount (S25). At this time, in the present embodiment, a label indicating that the feature amount is extracted for calculating the similarity calculation parameter is associated with the feature amount and stored.
Subsequently, if the number of feature quantities extracted for calculating the similarity calculation parameter is less than a predetermined number determined in advance (No in S26), the process returns to S22 and the subsequent processing is repeated. That is, the collection (extraction and storage) of the features is repeated until a sufficient number of features for calculating the similarity calculation parameter are collected. If the collected feature amount is more than the predetermined number (Yes in S26), the process proceeds to S27.
Subsequently, the similarity calculation parameter calculation unit 26 calculates the similarity calculation parameter using the collected features (S27). As described above, the similarity calculation parameter in this embodiment is the origin of the cosine similarity. Hereinafter, a method of calculating the similarity calculation parameter corresponding to one local region will be described.
_{The origin μ c} corresponding to the camera c is calculated by averaging _{the feature vector x ci} collected for each installed camera as shown in the following equation (4).

ここで、ｘ_ｃｉはカメラｃが撮像した認証画像から収集したｉ番目の特徴量ベクトルであることを示している。また、Ｎは収集した特徴量ベクトルのサンプル数である。この原点μ_ｃ、すなわち類似度算出パラメタを用いて、監視・認証処理で類似度を算出する。

Here, x _ci indicates that it is the i-th feature vector collected from the authentication image captured by the camera c. Further, N is the number of samples of the collected feature vector. Using this origin μ _c , that is, the similarity calculation parameter, the similarity is calculated by the monitoring / authentication process.

FIG. 9 is a diagram schematically showing the effect of calculating the origin, and represents a feature vector space of a certain local region imaged by a certain camera. The white circle points in the figure represent the feature vector, and the set of feature vectors, that is, the cluster is represented by the dotted line. FIG. 9 shows a predetermined origin 71, a feature amount vector cluster 72 of the authentication image collected by the installed camera, a feature amount vector cluster 73 of the person A, a feature amount vector cluster 74 of the person B, and the feature amount vector cluster 74 collected by the installed camera. The origin 75 calculated based on the authentication image is shown. When calculating the similarity based on the angle like the cosine similarity, the resolution for distinguishing the feature vector is higher when the origin is inside the cluster created by the feature vector to be authenticated.

Since the predetermined origin 71 is outside the cluster created by the feature vector to be authenticated, for example, the feature vector of person A (one of the feature vectors in 73) and the feature vector of person B are predetermined. When the angle is measured with reference to the origin 71 defined in, the angle is almost 0. Therefore, when the cosine similarity is calculated, a high degree of similarity that is supposed to be a different person but is erroneously determined to be the person is calculated. The predetermined origin 71 is generally calculated based on a large number of authentication images prepared in advance.

However, the feature amount vector distribution of the authenticated image collected by the installed camera may be different from the feature amount vector distribution of the authenticated image prepared in advance. In particular, since the angle of view and the location of the installed camera are limited, the number of images with specific lighting conditions and face orientation is larger than that of the authenticated image prepared in advance. In such a case, an erroneous high degree of similarity is calculated, leading to erroneous authentication. Therefore, by using the origin 75 calculated based on the authentication image collected by the installed camera, the feature amount vector cluster 72 of the authentication image collected by the installed camera is entered. As a result, the cosine similarity can be calculated using the origin suitable for authentication for each camera installation location and angle of view, and it is expected that false authentication will be reduced.
However, the position on the feature space of the origin calculated based on the authentication image collected by the installed camera does not always come to the position suitable for authentication.

FIG. 10 shows a feature amount vector space as in FIG. 9, a predetermined origin 76, a feature amount vector cluster 77 of the authentication image collected by the installed camera, a feature amount vector cluster 78 of the person C, and an installation. The origin 79 calculated based on the authentication image collected by the camera is shown. In this case, it is the same as in FIG. 9 that the predetermined origin 71 is not suitable for calculating the cosine similarity, but the origin 79 calculated based on the authentication image collected by the installed camera is the feature vector of the person C. It is inside the cluster 78. In this case, when the cosine similarity is calculated, erroneous authentication may occur in which the person C of the feature vector cluster 78 and another person are erroneously determined to be the person. In this way, if the calculated origin falls inside someone's feature vector cluster, it cannot be said to be a suitable origin for authentication.

Therefore, in the present embodiment, a plurality of feature quantities are extracted by setting a plurality of local regions instead of one feature quantity for one authentication image, and finally the similarity of each of the plurality of feature quantities is obtained. The integration similarity is calculated by integrating. This reduces erroneous authentication when there is a local area where the calculated similarity calculation parameter is not suitable for authentication. That is, even if a similarity calculation parameter that is not suitable for authentication is calculated in a certain local area and a high similarity that determines another person as the person is calculated, another similarity calculation parameter that is suitable for authentication can be calculated. Since it is finally integrated with the similarity of the local area, reduction of false authentication can be expected.

In the present embodiment, a plurality of feature quantities are extracted by setting a plurality of local regions, but a plurality of feature quantities may be extracted by other methods. For example, a plurality of feature quantities may be extracted by extracting different types of feature quantities.
Further, in the present embodiment, the average of the feature amount vectors is used as the origin, but the present invention is not limited to this. In addition, the origin may be calculated by using another method. For example, a weighted average may be used instead of a simple average of the feature vectors. At this time, it is advisable to normalize the weight value so that the origin is inside the cluster created by the feature vector to be authenticated.
The above is the calculation method of the similarity calculation parameter corresponding to one local region. This is calculated for all other local regions.

Returning to FIG. 8, the similarity calculation parameter storage unit 27 subsequently stores the calculated similarity calculation parameter (S28).
Subsequently, if the similarity calculation parameter calculation process corresponding to all the cameras is completed (Yes in S29), the similarity calculation unit 28 completes the similarity calculation parameter calculation process. If there is a camera that has not been completed yet (No in S29), the process returns to S21, and the image authentication device 1 repeats the following processing.
The above is the explanation of the similarity calculation parameter calculation process.

FIG. 11 is a diagram schematically showing an example of the screen 80 displayed during the execution of the similarity calculation parameter calculation process. FIG. 11 schematically shows a state in which a predetermined number of images (features) of the authentication images required for adjusting the similarity calculation parameter are presented to the user. FIG. 11 shows that the camera 2 and the camera 3 have a sufficient number of images for calculating the similarity calculation parameters (81a, 82a, 81b, 82b), and the camera 4 is still insufficient. (81c, 82c). In this way, by presenting to the user that a predetermined number of images have been accumulated, it is possible to visually understand how many more images should be collected in order to calculate the similarity calculation parameter. ..

In the present embodiment, the similarity calculation parameter calculation process is not completed unless the similarity calculation parameters corresponding to all the cameras are calculated, but the present embodiment is not limited to this. For example, the similarity calculation parameter calculation process may be interrupted before the calculation of some or all of the similarity calculation parameters is completed. In that case, the similarity calculation parameters used in the monitoring / authentication process may be prepared by another method. For example, the default similarity calculation parameter prepared in advance may be used. Alternatively, although the number is not sufficient, the similarity calculation parameter may be calculated from the feature quantities collected for each camera. Alternatively, the feature amount that is insufficient for calculation may be further collected in the background of the monitoring / authentication process, and the similarity calculation parameter may be calculated when a predetermined number is reached. In that case, the default similarity calculation parameter prepared in advance may be used until the calculation of the similarity calculation parameter is completed, and then replaced with the calculated similarity calculation parameter after the calculation is completed.

<Dictionary registration process>
Next, the dictionary registration process for registering the person to be authenticated will be described. This process is executed by selecting the "dictionary registration mode" from the console screen of FIG.
FIG. 12 shows an example of the processing flow of the dictionary registration process. The actual processing will be described in detail with reference to this figure.
First, the authentication image acquisition unit 21 acquires an image for dictionary registration (S31). The image for registering the dictionary does not necessarily have to be captured by the cameras 2 to 4, and the image captured in advance may be used.
Subsequent processes of local area setting (S32) to feature extraction (S33) are the same as the processes S23 and S24 of FIG. 8 described above, and thus description thereof will be omitted.

Subsequently, the dictionary registration unit 25 registers the feature amount in the dictionary (S34). At this time, the person ID of the registered person is registered in association with the feature amount, that is, stored in the secondary storage device 14. The person ID associated with the feature amount may be acquired by user input.
The above is the explanation of the dictionary registration process.
Whether or not the person registered here is reflected in the captured image is determined by the above-mentioned monitoring / authentication process. The number of persons to be registered is not particularly limited, and a plurality of feature quantities may be registered per person.
The above is the description of the first embodiment of the present invention.

As described above, in the present embodiment, the authentication image is acquired from the image captured by the installed camera, and the similarity calculation parameter corresponding to the installed camera is calculated to realize the person authentication robust to the imaging conditions. I explained how to do it.
By installing the camera, the orientation of the person to be imaged and the imaging conditions such as the lighting conditions are fixed to some extent. Therefore, by using the similarity calculation parameters specialized for the imaging conditions such as the camera installation location and the angle of view, the person (object) can be authenticated with higher accuracy than using the similarity calculation parameters prepared in advance. can do.

Modification Example In the above-described first embodiment, the cosine similarity is used as the similarity, and the origin at the time of calculating the cosine similarity is calculated as the similarity calculation parameter, but another similarity may be used. For example, as shown in the following equations (5) and (6), the reciprocal of the Mahalanobis distance may be used as the similarity S, and the Mahalanobis matrix M may be calculated as the similarity calculation parameter.

ここで、ｘ_ｃ、ｙは、それぞれ、カメラｃで撮像された人物の特徴量ベクトル、登録辞書の特徴量ベクトルを表す。

Here, x _c and y represent the feature vector of the person captured by the camera c and the feature vector of the registered dictionary, respectively.

FIG. 13 is a diagram schematically showing the effect when the Mahalanobis matrix is calculated as a similarity calculation parameter.
FIG. 13 shows the distribution of the feature vector as in FIGS. 9 and 10. This distribution shows a pre-calculated average vector 91, a metric space 92 based on a pre-calculated Mahalanobis matrix, and a newly calculated average vector 93 based on an authentication image collected by an installed camera. Further, this distribution shows a distance space 94 based on the newly calculated Mahalanobis matrix, a feature vector cluster 95 of the person D, and a feature vector cluster 96 of the person E.
Here, the metric space 92 means that the Mahalanobis distance from the pre-calculated average vector 91 is equal on the ellipse of the metric space 92. That is, the Mahalanobis distance in the metric space 92 is a distance scale in which the diagonal distance from the lower left to the upper right in FIG. 13 is shorter than the normal Euclidean distance. The same applies to the metric space 94, in which the distance from the upper left to the lower right in FIG. 13 is a shorter distance scale than the normal Euclidean distance.

In this way, the distance scale calculated in advance and the distance scale calculated from the authentication image collected by the installed camera are different. In the distance space 92 based on the Mahalanobis matrix calculated in advance in FIG. 13, the distance from the lower left to the upper right in FIG. 13 is a short distance scale. Therefore, the Mahalanobis distance between the feature amount vector of the person D (any one point in the feature amount vector cluster 95 of the person D) and the feature amount vector of the person E is close. That is, it may be determined that they are the same person. On the other hand, in the distance space 94 based on the newly calculated Mahalanobis matrix, on the contrary, since the distance between the feature amount vector of the person D and the feature amount vector of the person E is long, there is a high possibility that the person is determined as another person. As described above, the method of the present embodiment can also be applied to a method such as the Mahalanobis distance in which the similarity calculation parameter is set by prior learning.
Further, in the present embodiment, the authentication image is an image in which a person's face is captured, but the present invention is not limited to this, and an image in which a human body or any other object is captured may be used.
As described above, in the present embodiment, the person (object) can be authenticated with higher accuracy by using the similarity calculation parameter suitable for the installation location and the angle of view of the camera.

Embodiment 2
In the first embodiment, an example of calculating the similarity calculation parameter using the feature amount collected from the captured image of the camera has been described. In this embodiment, an example of calculating the similarity calculation parameter at the time of dictionary registration processing will be described. The same parts as in the first embodiment will be omitted, and only the parts with differences will be described.
In the second embodiment, as in the first embodiment, an example in which the cosine similarity is used as the similarity used for authentication will be described. Since the monitoring / authentication process and the similarity calculation parameter calculation process are the same as those in the first embodiment, the description thereof will be omitted.
The dictionary registration process will be described.

FIG. 14 shows an example of the processing flow of the dictionary registration process in the present embodiment. The actual processing will be described in detail with reference to this figure.
Since the registration image acquisition (S41) to the dictionary registration (S44) are the same as those in the first embodiment (S31 to S34 in FIG. 12), the description thereof will be omitted.
Following S44, the similarity calculation parameter calculation unit 26 updates the similarity calculation parameter using the registered dictionary acquired from the dictionary registration unit 25 (S45). The updated similarity calculation parameter σ calculated by this similarity calculation parameter calculation process is represented by the following equation (7).
σ = w ₀ σ ₀ + w _r σ _r (7)
Here, sigma ₀ is the mean vector of the collected feature quantity at the site of the camera, the sigma _r is the mean vector of all feature quantities of registered dictionary, w _0, w _r is calculated for each of the similarity It is a weighting coefficient for a parameter.

Generally, the number of features in the registered dictionary is smaller than the number of features that can be collected by the installed camera, so if you take the average including both features, the effect of the features collected by the installed camera will be affected. It becomes dominant. Therefore, in the present embodiment, the average vectors calculated for each are added by a predetermined weight determined in advance so that the influence of the average vector obtained by the registered dictionary can be adjusted.
Incidentally, sigma ₀ is may be previously calculated in advance, a feature amount collected at the site of the camera may be stored all in the secondary storage device 14, a sigma ₀ when calculating the formula (7) It may be calculated.
Subsequently, the similarity calculation parameter storage unit 27 stores the similarity calculation parameter σ calculated by the equation (7) in the secondary storage device 14 (S46).
In the present embodiment, the similarity calculation parameter is calculated every time a new feature amount is registered in the registration dictionary, but it is not necessary to calculate it every time. For example, it may be calculated when a predetermined number of features are newly registered in the registration dictionary.

Further, in the present embodiment, the average vector σ _r of all the features of the registered dictionary is calculated by the equation (7), but it may be calculated by using another method. For example, the average vector for each person in the registered dictionary may be calculated first, and then the average vector for each person may be further averaged. By doing so, when there is a large difference in the number of registrations for each person, it is possible to suppress the influence of the person with a large number of registrations from becoming too large.
The above is the description of the second embodiment of the present invention.
As described above, according to the present embodiment, not only the feature amount collected from the captured image of the installed camera but also the feature amount of the registered dictionary is reflected in the calculation of the similarity calculation parameter to take the registered image. It is possible to calculate the similarity calculation parameter that is more suitable for the monk condition.

1 ... image authentication device, 2, 3, 4 ... camera, 5 ... network, 21 ... authentication image acquisition unit, 22 ... local area setting unit, 23 ... feature extraction unit, 24 ... feature storage unit, 25 ... dictionary registration unit, 26 ... Similarity calculation parameter calculation unit, 27 ... Similarity calculation parameter storage unit, 28 ... Similarity calculation unit, 29 ... Authentication result determination unit, 30 ... Display unit

Claims (11)

An acquisition means for acquiring an image captured by at least one imaging device, and
An extraction means for extracting a feature amount from an image acquired by the acquisition means, and an extraction means.
A storage means for storing the feature amount of the image of the person to be authenticated, and
For each of the imaging device, based on the feature quantity extracted from the image taken by the imaging device, the imaging device for identifying whether a person is included to be the authentication object on the image captured by the The first calculation means for calculating the similarity calculation parameter and
A second calculation means for calculating the similarity between the extracted feature amount converted based on the similarity calculation parameter calculated according to the imaging device and the stored feature amount, and
An image processing device comprising. The first calculation means calculates a plurality of similarity calculation parameters for one image acquired by the acquisition means.
The second calculation means calculates a plurality of similarities based on the plurality of similarity calculation parameters calculated by the first calculation means, and the calculated plurality of similarity is used as one similarity. Integrate into
The image processing apparatus according to claim 1. Further, a setting means for setting a plurality of local areas for one image acquired by the acquisition means is provided.
The first calculation means calculates the similarity calculation parameter for each of the plurality of local regions set by the setting means.
The image processing apparatus according to claim 2. The first calculation means calculates the similarity calculation parameter based on the stored feature amount in addition to the feature amount extracted from the acquired image.
The image processing apparatus according to any one of claims 1 to 3. The similarity calculation parameter calculated by the first calculation means is the origin of angle calculation in the feature space.
It said second calculating means, the degree of similarity is calculated based on the angle of the first calculating means with the origin of the calculated angle calculated,
The image processing apparatus according to any one of claims 1 to 4. The extraction means extracts a plurality of feature quantities from the image and
The origin of the angle calculation calculated by the first calculation means is the average of the plurality of feature quantities extracted by the extraction means.
The image processing apparatus according to claim 5. The similarity calculation parameter calculated by the first calculation means is a Mahalanobis matrix.
The second calculation means calculates the similarity based on the Mahalanobis matrix calculated by the first calculation means.
The image processing apparatus according to any one of claims 1 to 4. The acquisition means acquires the image in order to calculate a plurality of the similarity calculation parameters according to the user input.
The first calculation means calculates a plurality of the similarity calculation parameters based on the image acquired by the acquisition means.
The image processing apparatus according to any one of claims 1 to 7. A display means for presenting to the user that the acquisition means has acquired a predetermined number of the images for calculating the plurality of similarity calculation parameters is further provided.
The image processing apparatus according to any one of claims 1 to 8. A step of acquiring an image captured by at least one imaging device, and
An extraction step of extracting a feature amount from an image captured by the imaging device, and
For each of the imaging device, on the basis of the feature quantity extracted from the image taken by the imaging device, the imaging device for identifying whether a person is included as a authentication target image captured by Steps to calculate similarity calculation parameters and
The degree of similarity between the extracted feature amount converted based on the similarity calculation parameter calculated according to the imaging device and the feature amount of the image of the person to be recognized stored in the storage means. The steps to calculate and the steps to calculate
An image processing method characterized by having. A program for operating a computer as the image processing device according to any one of claims 1 to 9.

Images