JP2018005805A - Information processing system, image processing device, information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the time required for biological determination while suppressing a decrease in the accuracy of biological determination in an information processing system that performs biological determination to determine whether a person captured in an image is a living body.SOLUTION: The information processing system performs biological determination to determine whether a person captured in an image is a living body using information pertaining to the identification space of a living body pre-extracted from the image of the living body, the information processing system comprising: an imaging unit for capturing an image; a feature information extraction unit for extracting the feature information of a person captured in the image; a first storage unit for storing information pertaining to an unnecessary element, among first projection vector elements projected to a discrimination space including the identification space of the living body, that is not used for biological determination; and a biological determination unit for determining whether the person captured in the image is a living body using a second projection vector with the unnecessary element removed from the first projection vector and the feature information extracted by the feature information extraction unit.SELECTED DRAWING: Figure 6

Description

  The present application relates to an information processing system, an image processing device, an information processing device, and a program.

  As an authentication method for authenticating a user, an image authentication technique such as face authentication that does not require an operation such as a password input and can prevent spoofing due to loss or theft of an IC card has become widespread.

  Also, by extracting areas such as eyes and mouth from the face image taken by the camera and judging whether the area changes in the time direction, it is a photograph whether the photographed face is an actual face. There is known a living body determination technique for determining whether or not (see, for example, Patent Document 1).

  For example, in an information processing apparatus that performs face authentication, impersonation due to a face photograph or the like can be reduced by using biometric determination as disclosed in Patent Document 1.

  However, for example, in the conventional biometric determination technique as disclosed in Patent Document 1, when determining the movements of various people, it is possible to reduce the time required for determination while suppressing a decrease in determination accuracy. Was accompanied by difficulties.

  Embodiments of the present invention have been made in view of the above problems, and in an information processing system that performs a living body determination for determining whether a person photographed in an image is a living body, the accuracy of the living body determination The purpose is to reduce the time required for the living body determination while suppressing the decrease of the above.

  In order to solve the above problem, an information processing system according to an embodiment of the present invention uses information on a living body identification space extracted in advance from a living body image to determine whether a person photographed in the image is a living body. An information processing system for performing biometric determination, wherein a photographing unit for photographing the image, a feature information extracting unit for extracting feature information of a person photographed in the image, and the feature information extracting unit extract the feature information A first storage unit that stores information on unnecessary elements that are not used for biometric determination among elements of a first projection vector that project characteristic information onto a discrimination space including the identification space of the biological body; A living body that determines whether or not the person photographed in the image is a living body using the second projection vector obtained by removing the unnecessary elements from the projecting vector and the feature information extracted by the feature information extracting unit. Judgment part and Having.

  According to an embodiment of the present invention, in an information processing system that performs a living body determination for determining whether or not a person photographed in an image is a living body, it is necessary for the living body determination while suppressing a decrease in the accuracy of the living body determination. Time can be shortened.

It is a figure showing an example of composition of an information processing system concerning one embodiment. 2 is a diagram illustrating an example of a hardware configuration of an image forming apparatus according to an embodiment. FIG. It is a figure which shows the hardware structural example of the information processing apparatus which concerns on one Embodiment. 1 is a diagram illustrating an example of a functional configuration of an image forming apparatus according to a first embodiment. 4 is another example of a functional configuration of the image forming apparatus according to the first embodiment. It is a figure which shows the example of a function structure of the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows the example of the face image which concerns on 1st Embodiment. It is a figure for demonstrating the HALC feature-value which concerns on 1st Embodiment. It is a figure for demonstrating the CHLAC feature-value which concerns on 1st Embodiment. It is a figure for demonstrating an example of the discriminant analysis which concerns on 1st Embodiment. It is a figure for demonstrating another example of the discriminant analysis which concerns on 1st Embodiment. It is a figure which shows the example of the projection vector which concerns on 1st Embodiment. It is a figure which shows the example of the unnecessary element of the projection vector which concerns on 1st Embodiment. 6 is a flowchart illustrating an example of unnecessary element extraction processing according to the first embodiment; 6 is a flowchart illustrating an example of biometric determination processing of the image forming apparatus according to the first embodiment. It is a figure for demonstrating the subspace method which concerns on 2nd Embodiment. It is a figure which shows the image of the partial space of biometric information which concerns on 2nd Embodiment, and non-biometric information. It is a figure which shows the image of the partial space obtained by the principal component analysis which concerns on 2nd Embodiment. It is a figure which shows the image of the partial space on the complement space which concerns on 2nd Embodiment. It is a figure which shows the example of the projection vector which concerns on 2nd Embodiment. It is a figure which shows the example of the unnecessary element of the projection vector which concerns on 2nd Embodiment. It is a flowchart which shows the example of the extraction process of the unnecessary element which concerns on 2nd Embodiment. It is a flowchart which shows the example (1) of the biometric determination process which concerns on other embodiment. It is a figure which shows the example of the setting screen of the biometric determination mode which concerns on other embodiment. It is a flowchart which shows the example (2) of the biometric determination process which concerns on other embodiment.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment. For example, the information processing system 100 photographs an image forming apparatus 101 such as an MFP (Multifunction Peripheral / Product) having a plurality of functions such as printing, copying, scanning, and facsimile transmission / reception, and a user 103 of the image forming apparatus 101. Camera 102.

  An image forming apparatus (an example of an image processing apparatus) 101 uses a camera 102 to capture 103 images of a user who uses the image forming apparatus 101 and extracts a face image of the user 103. Further, the image forming apparatus 101 performs face authentication of the user 103 using, for example, the extracted face image.

  In addition, the image forming apparatus 101 is configured so that, for example, a person (for example, a user 103) photographed in an image by the camera 102 is a living body in order to prevent authentication break-through through impersonation due to an act of holding a photograph of another person over the camera 102 or the like. Biological determination is performed to determine whether or not.

  For example, the image forming apparatus 101 includes biological information, which is information on a living body identification space extracted in advance from a biological image, and information on a non-biological identification space extracted in advance from an image of a non-living body (such as a photograph). Non-biological information is stored in the storage unit in advance. Further, the image forming apparatus 101 extracts the feature information of the user 103 from the image taken by the camera 102, and determines the distance, similarity, etc. between the extracted feature information and the biological information and non-biological information stored in the storage unit. Based on this, it is determined whether or not the person photographed in the image is a living body.

  Further, the image forming apparatus 101 according to the present embodiment stores in advance information on an unnecessary space (hereinafter referred to as an unnecessary element) that is not used for the biological determination among the identification spaces used for the biological determination. In addition, feature information extraction, similarity calculation, and the like are executed.

  The unnecessary element information stored in the image forming apparatus 101 can be stored in the storage unit of the image forming apparatus 101 as, for example, unnecessary element information created in advance by another information processing apparatus. Alternatively, the image forming apparatus 101 may have a function of generating unnecessary element information.

  The image forming apparatus 101 according to the present embodiment removes unnecessary elements from the first projection vector that projects the feature information extracted from the captured image onto a discrimination space including a living body identification space and a non-living body identification space. The biological information of the feature information is determined using the excluded second projection vector.

  Preferably, the image forming apparatus 101 extracts feature information excluding unnecessary elements when extracting feature information extracted from a captured image.

  As a result, the image forming apparatus 101 can reduce processing related to unnecessary elements among the feature information extraction processing and the biological determination processing, and thus can speed up the biological determination processing. Further, by storing in advance elements that do not affect (or have little influence on) the determination result of the biological determination as unnecessary elements that are not used for the biological determination, it is possible to suppress a decrease in the accuracy of the biological determination.

  As described above, according to the present embodiment, in the information processing system 100 that performs biological determination for determining whether or not the person photographed in the image is a living body, the biological determination is performed while suppressing a decrease in the accuracy of the biological determination. Can be shortened.

  The configuration of the information processing system 100 illustrated in FIG. 1 is an example. For example, the image forming apparatus 101 is another image processing apparatus that performs face authentication, such as a PC (Personal Computer), a tablet terminal, a smartphone, a video conference device, an attendance / exit management device, an ATM (Automatic teller machine), a navigation device, and the like. An image processing apparatus may be used.

  Further, the information processing system 100 may include another information processing apparatus that creates the above-described unnecessary space information.

<Hardware configuration>
(Hardware configuration of image forming apparatus)
FIG. 2 is a diagram illustrating an example of a hardware configuration of the image forming apparatus 101 according to the embodiment. The image forming apparatus 101 includes a main body 210 that realizes various image forming functions such as a copy function, a scanner function, a fax function, and a printer function, and an operation unit 220 that receives a user operation. Note that accepting a user operation is a concept including accepting information (including a signal indicating a coordinate value of a screen) input according to the user operation. The main body 210 and the operation unit 220 are connected via a dedicated communication path 240 so that they can communicate with each other. The communication path 240 may be, for example, a USB (Universal Serial Bus) standard, but may be of any standard regardless of wired or wireless.

  The main body 210 can perform an operation corresponding to the operation received by the operation unit 220. The main body 210 can also communicate with an external device such as a client PC (personal computer), and can also perform an operation according to an instruction received from the external device.

  First, the hardware configuration of the main body 210 will be described. As shown in FIG. 2, the main body 210 includes a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, a RAM (Random Access Memory) 213, a storage unit 214, a communication I / F (Interface) 215, and a connection I. / F216, engine unit 217, human body detection sensor 218, system bus 219, and the like.

  The CPU 211 controls the overall operation of the main body 210 by executing a program stored in the ROM 212 or the storage unit 214 using the RAM 213 as a work area (work area). For example, the CPU 211 uses the engine unit 217 to realize various functions such as the copy function, the scanner function, the fax function, and the printer function described above.

  The ROM 212 is, for example, a non-volatile memory that stores a basic input / output system (BIOS) executed when the main body 210 is started, various settings, and the like. The RAM 213 is a volatile memory used as a work area for the CPU 211. The storage unit 214 is a non-volatile storage device that stores, for example, an OS (Operating System), application programs, various data, and the like, and includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. The

  The communication I / F 215 is a network interface for connecting the main body 210 to the network 250 and performing communication with an external device connected to the network 250. The connection I / F 216 is an interface for communicating with the operation unit 220 via the communication path 240.

  The engine unit 217 is hardware that performs general-purpose information processing and processing other than communication for realizing functions such as a copy function, a scanner function, a fax function, and a printer function. The engine unit 217 includes, for example, a scanner (image reading unit) that scans and reads an image of a document, a plotter (image forming unit) that performs printing on a sheet material such as paper, and a fax unit that performs fax communication. . Furthermore, the engine unit 217 may include specific options such as a finisher that sorts printed sheet materials and an ADF (automatic document feeder) that automatically feeds documents.

  The human body detection sensor 218 is a sensor that detects a human body within a detection range around the image forming apparatus 101. For example, a pyroelectric sensor or the like is used.

  A system bus 219 is connected to each of the above components, and transmits an address signal, a data signal, various control signals, and the like.

  Next, the hardware configuration of the operation unit 220 will be described. As shown in FIG. 2, the operation unit 220 includes a CPU 221, ROM 222, RAM 223, flash memory 224, communication I / F 225, operation panel 226, connection I / F 227, internal camera 228, external connection I / F 229, and system bus 230. Etc.

  The CPU 221 controls the operation of the entire operation unit 220 by executing a program stored in the ROM 222 or the flash memory 224 using the RAM 223 as a work area (working area).

  The ROM 222 is a non-volatile memory that stores, for example, a BIOS executed when the operation unit 220 is activated, various settings, and the like. The RAM 223 is a volatile memory used as a work area for the CPU 221. The flash memory 224 is, for example, a non-volatile storage device that stores an OS, application programs, various data, and the like.

  The communication I / F 225 is a network interface for connecting the operation unit 220 to the network 250 and performing communication with an external device connected to the network 250.

  The operation panel 226 accepts various inputs in accordance with user operations and displays various types of information (for example, information in accordance with accepted operations, information indicating the operation status of the image forming apparatus 101, setting states, and the like). The operation panel 226 is configured by, for example, a liquid crystal display (LCD) having a touch panel function, but is not limited thereto. The operation panel 226 may be configured by, for example, an organic EL (Electro Luminescence) display device equipped with a touch panel function. Further, the operation panel 226 can be provided with an operation unit such as a hardware key or a display unit such as a lamp in addition to or instead of this.

  The connection I / F 227 is an interface for communicating with the main body 210 via the communication path 240.

  The internal camera 228 is a camera that captures an image around the image forming apparatus 101, and is an example of the camera 102 in FIG.

  The external connection I / F 229 is an interface such as a USB for connecting an external device. The external device connected to the external connection I / F 229 includes, for example, the external camera 231. The external camera 231 is a camera that captures an image around the image forming apparatus 101, and is another example of the camera 102 in FIG. The image forming apparatus 101 includes one or both of an internal camera 228 and an external camera 231.

  The system bus 230 is connected to each of the above components, and transmits an address signal, a data signal, various control signals, and the like.

(Hardware configuration of information processing device)
FIG. 3 is a diagram illustrating a hardware configuration example of the information processing apparatus according to the embodiment. The image processing apparatus according to the present embodiment may be an information processing apparatus as shown in FIG. 3, for example. Further, another information processing apparatus for creating unnecessary space information described above with reference to FIG. 1 has a hardware configuration as shown in FIG.

  The information processing apparatus 300 has a general computer configuration. For example, the CPU 301, the RAM 302, the ROM 303, the storage device 304, the network I / F 305, the input device 306, the display device 307, the external I / F 308, and the bus 309 etc.

  The CPU 301 is an arithmetic device that implements each function of the information processing device 300 by reading a program or data stored in the ROM 303, the storage device 304, or the like onto the RAM 302 and executing processing. A RAM 302 is a volatile memory used as a work area for the CPU 301. The ROM 303 is a non-volatile memory that can retain programs and data even when the power is turned off.

  The storage device 304 is a large-capacity storage device such as an HDD or an SSD, and stores an OS, application programs, various data, and the like. A network I / F 305 is a communication interface for connecting the information processing apparatus 300 to the network 250 or the like.

  The input device 306 is, for example, a pointing device such as a mouse or an input device such as a keyboard, and is used to input each operation signal to the information processing device 300. The display device 307 is a display device such as a display, and displays a processing result or the like by the information processing device 300.

  The external I / F 308 is an interface with an external device. The external device includes, for example, an external camera 310 that captures an image around the information processing device 300, a recording medium, and the like.

  A bus 309 is connected to each of the above components, and transmits an address signal, a data signal, various control signals, and the like.

[First Embodiment]
<Functional configuration>
FIG. 4 is a diagram illustrating an example of a functional configuration of the image forming apparatus according to the first embodiment. The image forming apparatus 101 realizes the functional configuration shown in FIG. 4 by executing a predetermined program by at least one of the CPU 211 and the CPU 221 in FIG.

  The functional configuration shown in FIG. 4 is an example. For example, at least some of the components included in the operation unit 220 may be included in the main body 210, and at least some of the components included in the main body 210 may be included in the operation unit 220. It may be included.

(Functional structure of the main unit)
The main body 210 of the image forming apparatus 101 includes a human body detection unit 421, a state control unit 422, an image forming unit 423, a communication unit 424, and the like.

  The human body detection unit 421 is means for detecting a human body around the image forming apparatus 101 using, for example, the human body detection sensor 218 shown in FIG. 2, and is executed by, for example, the CPU 211 (or CPU 221) in FIG. Realized by a program. For example, when a human body is detected within the detection range of the human body detection sensor 218, the human body detection unit 421 notifies the state control unit 422 that the human body has been detected.

  The state control unit 422 is means for controlling the power state of the operation unit 220 and the main body 210, and is realized by, for example, a program executed by the CPU 211 (or CPU 221) of FIG. For example, the state control unit 422 saves power when the image forming apparatus 101 is not used for a predetermined time and consumes less power than the normal state in which the image forming process can be performed. Transition to the state. In the power saving state, the image forming apparatus 101 can reduce power consumption, for example, by stopping the functions of the operation unit 220, the engine unit 217 of the main body 210, and the storage unit 214.

  In addition, when the image forming apparatus 101 receives a notification indicating that a human body has been detected from, for example, the human body detection unit 421 while the image forming apparatus 101 is in the power saving state, the state control unit 422 causes the operation unit 220 to return from the power saving state to the normal state. It returns to the normal state that can operate. For example, the state control unit 422 notifies the operation unit 220 of a return command via the communication unit 424, thereby returning the operation unit 220 from the power saving state to the normal state.

  The image forming unit 423 is a unit that executes various image forming functions (for example, printing, copying, scanning, faxing, and the like) provided in the image forming apparatus 101. For example, the engine unit 217 in FIG. 2 and the CPU 211 in FIG. This is realized by a program executed in

  The communication unit 424 is a means for communicating with the operation unit 220, and is realized by, for example, the connection I / F 216 in FIG. 2 and a program executed by the CPU 211 in FIG.

(Function configuration of the operation unit)
The operation unit 220 of the image forming apparatus 101 includes, for example, a photographing unit 401, a face image detection unit 402, a feature information extraction unit 403, a living body determination unit 404, a noise determination unit 405, an unnecessary space extraction unit 406, an image authentication unit 407, a display. It includes an input unit 408, a biological information storage unit 409, a non-biological information storage unit 410, an identification space information storage unit 411, an unnecessary space information storage unit 412, a communication unit 413, and the like.

  The imaging unit (acquisition unit) 401 is a unit that captures an image around the image forming apparatus 101 using the camera 102 such as the internal camera 228 or the external camera 231 shown in FIG. 2, for example, the CPU 221 (FIG. Alternatively, it is realized by a program executed by the CPU 211). The imaging unit 401 acquires, for example, images of a user in the vicinity (for example, the front) of the image forming apparatus 101 at predetermined time intervals.

  The face image detection unit 402 is means for detecting a face area from an image captured by the image capturing unit 401, and is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG. An example of a face image detected by the face image detection unit 402 is shown in FIG.

  In FIG. 7A, the user 103 in front of the image forming apparatus 101 is photographed in an image 710 photographed by the photographing unit 401. As described above, it is assumed that the camera 102 of the image forming apparatus 101 is installed so that the user 103 who uses the image forming apparatus 101 is included in the image 710 photographed by the photographing unit 401.

  The face image detection unit 402 detects, for example, a face image 720 of the user 103 as shown in FIG. 8B from the image 710 captured by the imaging unit 401 using, for example, a known pattern matching technique. Then, the detected face image 720 is extracted.

  The face image detection method by the face image detection unit 402 may be any method, but here, as an example, an outline of face detection by the Haar-like feature will be described.

  In face detection using the Haar-like feature, the Haar-like feature value is a value obtained by subtracting the sum of the pixel values of the white region from the sum of the pixel values of the black region in the rectangle to be calculated in the search region. Is used. Since the rectangular arrangement location described above may be arranged anywhere in the search area, there may be tens of thousands of arrangement locations in the search area. On the other hand, each weak classifier in the search area is weighted by prior learning by boosting, and dozens of weak classifiers with higher importance are selected to create a strong classifier. With this strong discriminator (by a black and white rectangular number pattern indicating the likelihood of a human face), it is possible to determine whether or not the area is a human face area.

  The feature information extraction unit 403 is means for extracting feature information (feature amount) of a person included in an image captured by the image capturing unit 401, and is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG. The

  The feature information extraction unit 403 extracts feature information indicating the operation from the time-series face images extracted by the face image detection unit 402 from images captured by the imaging unit 401 at predetermined time intervals. As an example of the feature information extracted by the feature information extraction unit 403, a cubic higher-order local auto correction (CHLAC) can be used.

  Here, before describing the three-dimensional higher-order autocorrelation feature (hereinafter referred to as CHLAC feature), a higher-order local autocorrelation feature (HLAC: Higher-order Local Auto Correction) is a predecessor of the CHLAC feature. ).

(HLAC feature)
For higher order local autocorrelation feature quantities (hereinafter referred to as HLAC feature quantities), higher order autocorrelation at a point r on the binarized image is calculated.

  Since the target pixel r (reference point) and the neighboring region are expected to have a high correlation, the HLAC feature amount is calculated by calculating the correlation from the local region instead of calculating the correlation of the entire region. Is required. The autocorrelation function of the Nth order (N is a natural number) is expressed by the following (Expression 1).

ここで、ｆは時系列の画像であり、ｒは参照点、ａｉ（ｉ＝１，...，Ｎ）はＮ個の変位である。次元を２次、変位方向を３×３に限定した場合、２値の動画像のＨＡＬＣ特徴量における変位ベクトルの組合せは、例えば、図８に示すように、０次が１個、１次が４個、２次が２０個の計２５次元のベクトルとして表される。

Here, f is a time-series image, r is a reference point, and ai (i = 1,..., N) are N displacements. When the dimension is limited to the second order and the displacement direction is limited to 3 × 3, the combination of the displacement vectors in the HALC feature amount of the binary moving image is, for example, as shown in FIG. 4 and 20 are expressed as 25-dimensional vectors.

  This HLAC feature quantity has two properties, one invariance and additivity.

  The position invariance is a property that the feature amount does not change regardless of the position in the image. In addition, for example, when the feature amount of the motion of the person A is fA and the feature amount of the motion of the person B is fB, the person A and the person B perform the motion in the same image. The amount is fA + fB.

(CHLAC feature)
Subsequently, a CHLAC feature amount used as feature information in the present embodiment will be described.

  The HLAC feature value is used to obtain the correlation of the two-dimensional local region. The CHLAC feature value is used to obtain the correlation of the local region to which time is added. The CHLAC feature amount can be obtained by integrating the Nth-order autocorrelation function shown in the following (Equation 2) with the time of the time-series image.

時間方向の積分範囲は、どの程度の時間方向の相関を取るかによって決定する任意のパラメータである。

The integration range in the time direction is an arbitrary parameter determined by how much correlation in the time direction is taken.

  The number of combinations of displacement vectors of CHLAC feature values by binary moving images is 0 for the 1st order, 13 for the 1st order, and 237 for the 2nd order, and is expressed as a 251-dimensional vector in total. Similar to the HLAC feature quantity, the CHLAC feature quantity has two properties of position invariance and additivity.

  Here, an image of a method of calculating the CHLAC feature amount will be described with reference to FIG.

  In FIG. 9, for example, time series images 901 to 904 are taken at predetermined time intervals by the photographing unit 401 or the like.

  Next, for example, the feature information extraction unit 403 calculates the difference between the time-series images 901 to 904 and binarizes the obtained difference image. In the example of FIG. 9, an image 911 shows a binary difference image between the image 901 and the image 902. An image 912 shows a binary difference image between the image 902 and the image 903. Further, the image 913 shows a binary difference image between the image 903 and the image 904.

  Subsequently, the feature information extraction unit 403 calculates, for the binarized image 912, the correlation between the own HLAC feature amount and the preceding and following images 911 and 913 up to the second order.

  Further, the feature information extraction unit 403 calculates and integrates CHLAC for all pixels in the image 912.

  Thereby, for example, one feature vector (CHLAC feature amount) corresponding to three binary images shown in a frame 920 in FIG. 9 is generated. Note that when a minute motion such as a face is identified as in the present embodiment, it is desirable to normalize the norm for the generated feature vector.

  As described above, the CHLAC feature amount can be used as an example of the feature information extracted by the feature information extraction unit 403.

  As another example, the feature information extraction unit 403 uses a time-series image to calculate where the small area in the image has moved in the image of the next frame by block matching, and moves each area. May be used as feature information. Here, the feature information extraction unit 403 performs the following description on the assumption that the CHLAC feature amount is extracted.

  Preferably, the feature information extraction unit 403 extracts, for example, a feature amount excluding unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from a 251-dimensional CHLAC feature amount vector. Thereby, for example, when 10 unnecessary elements are stored in the unnecessary space information storage unit 412, the feature information extraction unit 403 only needs to perform 241 dimensional calculation out of the 251 dimensional feature information. Processing can be speeded up.

  Returning to FIG. 4, the description of the functional configuration of the operation unit 220 is continued.

  The biometric determination unit 404 includes information on a biometric identification space extracted in advance from a biometric image (hereinafter referred to as biometric information) and information on a non-biological identification space extracted in advance from a non-living body (for example, a photograph). Hereinafter, biometric determination is performed using non-biometric information). The biometric determination unit 404 is realized, for example, by a program executed by the CPU 221 (or CPU 211) in FIG.

  The biometric determination unit 404 compares the feature information extracted by the feature information extraction unit 403 with biometric information stored in advance in the biometric information storage unit 409 and non-biological information stored in advance in the non-biological information storage unit 410. It is determined whether or not the person photographed in the image is a living body (biological determination).

  Note that there are a discriminant analysis, a subspace method, and the like as a living body determination method. In the first embodiment, as an example, the following description will be given assuming that the living body determination unit 404 performs living body determination by discriminant analysis.

(Discriminant analysis)
Here, an outline of discriminant analysis will be described with reference to FIGS.

  For example, as shown in FIG. 10A, it is assumed that there is a discrimination space generated from three pieces of biological information (biological information 1 to 3) and five pieces of non-biological information (non-biological information 1 to 5). The biometric information 1 to 3 is an example of a biometric identification space (biometric information) that is stored in the biometric information storage unit 409 and extracted in advance from a biometric image. The non-biological information 1 to 5 is an example of a non-biological identification space (non-biological information) extracted in advance from a non-biological image stored in the non-biological information storage unit 410.

  As an example, when the feature information extracted by the feature information extraction unit 403 is projected onto the discrimination space, the biometric determination unit 404 extracts the feature information extracted by the feature information extraction unit 403 if the closest identification space is biometric information. Is biometric feature information. On the other hand, when the feature information extracted by the feature information extraction unit 403 is projected onto the discrimination space, the biometric determination unit 404 extracts the feature information extracted by the feature information extraction unit 403 if the closest identification space is non-biometric information. Is non-biological feature information.

  For example, in the example of FIG. 10A, the feature information 1001 projected onto the discrimination space is closest to “living body 1”, and thus the living body determination unit 404 determines that the feature information 1001 is the living body feature information. To do. On the other hand, in the example of FIG. 10B, the feature information 1002 projected onto the discrimination space is closest to “non-biological information 2”, so that the biometric determination unit 404 has the feature information 1002 as non-biological feature information. Judge that there is.

  The distance between the feature information projected on the discrimination space and each identification space (biological information 1 to 3 and non-biological information 1 to 5) is, for example, the distance from the center (center of gravity) of each identification space. To be judged. The distance is, for example, the Euclidean distance or Mahalanobis distance.

  As another example, when the biometric determination unit 404 projects the feature information extracted by the feature information extraction unit 403 onto the discrimination space, the biometric determination unit 404 does not belong to the closest identification space, but rather is a certain range from the biometric identification space. You may look at the distance.

  For example, the feature information projected onto the discrimination space in FIG. 11 is within a predetermined distance range 1101 from the biological information 1, within a predetermined distance range 1102 from the biological information 2, or within a predetermined distance range 1103 from the biological information 3. If it is within the living body, the living body determination unit 404 determines that it is biological feature information.

  In this method, since the information necessary for the biometric determination is the distance between the center of the identification space of the biometric information 1 to 3 and the feature information, it is between the center of the identification space of the non-biological information 1 to 5 and the feature information. The distance need not be calculated. In this method, the similarity is higher as the distance from the center of the identification space of the biological information 1 to 3 is shorter, and the similarity is lower as the distance is longer.

  In the present embodiment, the living body determination unit 404 performs the following description on the assumption that the living body determination is performed by discriminant analysis.

  Returning to FIG. 4, the description of the functional configuration of the operation unit 220 is continued.

  The noise determination unit 405 is a unit that determines whether there is an image on which noise affecting the living body determination is present due to, for example, flickering of a fluorescent lamp among time-series images captured by the imaging unit 401. The noise determination unit 405 is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG.

The noise determination unit 405 may use any method for determining the noise, but as an example, the noise determination unit 405 calculates the sum of the luminance values of each pixel in two images arranged in time series. To do. Further, the noise determination unit 405 can determine that noise is present when the calculated sum value exceeds the threshold value.

  The unnecessary space extraction unit 406 is not used for biometric determination among the elements of the projection vector (first projection vector) that projects the feature information extracted by the feature information extraction unit 403 into the discrimination space as shown in FIG. The element information is extracted and stored in the unnecessary space information storage unit 412.

  Further, the unnecessary space extraction unit 406 obtains a projection vector (second projection vector) obtained by removing unnecessary elements stored in the unnecessary space information storage unit 412 from the elements of the first projection vector, and the identification space information storage unit 411. To remember.

  Here, the discriminant space and the projection vector will be described.

  Discriminant analysis is a method for obtaining a space that minimizes intra-class variance of its own class (identification space) and maximizes inter-class variance with other classes. When there are a plurality of classes, a space that minimizes the variance within each class and maximizes the interclass variance between the classes is obtained.

  The intra-class variance W is expressed by the following (Equation 3).

また、クラス間分散Ｂは、次の（式４）で表される。

The interclass variance B is expressed by the following (Equation 4).

上記の（式３）、（式４）において、データはｄ次元のベクトルｘで表される。また、クラス数はｃ≧２、総データ数はｎ 、データ集合はＸ＝｛ｘ｝、データ全体の平均値は_aveｘであるものとする。また、クラスｉのデータ集合をｘ_i、データ集合ｘ_iのデータ数をｎ_i、データ数ｎ_iのデータの平均を_aveｘ_iとする。Ｔは転置行列を示す。

In the above (Expression 3) and (Expression 4), the data is represented by a d-dimensional vector x. Further, the number of classes is c ≧ 2, the total number of data is n, the data set is X = {x}, and the average value of the entire data is _ave x. Further, the data set of classes i x _i, the number of data of the data set x _i n _i, the average of the data of the data number n _i and _ave x _i. T indicates a transposed matrix.

Here large eigenvalue C-1 or lambda ₁ of W ^-1 B, ..., the eigenvector corresponding to _{λ C-1 A 1, ...} , a A _C-1. Using this, the d × (C−1) matrix A = [A ₁ , A ₂ ,..., A _C-1 ] is determined. That the dimension reduction in C-1 dimensional space A ^{T x} data x using the matrix, as projected onto the discriminant space.

  Here, the biological information that is the identification space of the living body stored in the biological information storage unit 409 and the non-biological information that is the non-biological identification space stored in the non-biological information storage unit 410 are 251-dimensional data x. The following will be described. However, the data x may use data other than 251 dimensions.

FIG. 12 is a diagram illustrating an example of a projection vector according to the first embodiment. The projection vector shown in FIG. 12 is an example of a projection vector when performing discriminant analysis of eight classes (eight kinds of identification spaces). When performing 8-class discriminant analysis, the discriminant space has 7 dimensions, and there are seven eigenvectors A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , and A _{7 that} are projected onto the discriminant space. become.

In FIG. 12, the element number indicates a 251-dimensional dimension. A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , and A ₇ represent the values of the eigenvectors.

The eigenvectors A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , A ₇ that are projected onto the discrimination space include one or more pieces of biological information and one or more pieces of non-biological information. It is.

  The unnecessary space extraction unit 406 extracts unnecessary elements that are unnecessary elements (dimensions) from, for example, a projection vector as shown in FIG. For example, the unnecessary space extraction unit 406 determines in FIG. 12 that the element corresponding to the element number for which the absolute value of all the values corresponding to each element number is less than the threshold is an unnecessary element, and the unnecessary space information storage unit 412 is stored. For example, FIG. 13 shows an example in which the threshold value is “0.010” in FIG.

FIG. 13 is a diagram illustrating an example of unnecessary elements of the projection vector according to the first embodiment. In the example of FIG. 13, the element numbers ₃ , ₄ , _7, and 246 indicate that all the values corresponding to the eigenvectors A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , A ₇ have the threshold value “0”. Since it is less than “.010”, the unnecessary space extraction unit 406 determines that it is an unnecessary element and stores it in the unnecessary space information storage unit 412.

  Preferably, the unnecessary space extraction unit 406 obtains a projection vector (second projection vector) obtained by removing unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) shown in FIG. Store in the identification space information storage unit 411.

  Note that the threshold value “0.010” is an example, and it is assumed that an element having a small influence on the determination result of the discriminant analysis is determined in advance so as to be extracted as an unnecessary element. Alternatively, the threshold value may be set by an administrator, a user, or the like using the operation panel 226 or the like.

  The biometric determination unit 404 according to the present embodiment uses a second projection vector obtained by removing unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) as shown in FIG. Then, the feature information extracted by the feature information extraction unit 403 is discriminated and analyzed. For example, the living body determination unit 404 uses the second projection vector stored in the identification space information storage unit 411 by the unnecessary space extraction unit 406 to determine whether or not the person photographed in the image by the shooting unit 401 is a living body. Determine whether.

  Preferably, when extracting the feature information, the feature information extraction unit 403 excludes unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from the 251-dimensional feature information, Perform extraction.

  Returning to FIG. 4, the description of the functional configuration of the operation unit 220 is continued.

  The image authentication unit 407 is means for executing conventional image authentication such as face authentication, and is realized, for example, by a program executed by the CPU 221 (or CPU 211) in FIG. For example, the image authentication unit 407 permits the use of the image forming apparatus 101 when the image authentication is successful and the biometric determination unit 404 determines that the person photographed in the image is a living body.

  The display input unit 408 is a means for displaying a display screen such as an operation screen or a setting screen on the operation panel 226 of FIG. 2 and accepting an input operation to the operation panel 226. For example, the CPU 221 (or CPU 211 of FIG. ).

  The biometric information storage unit 409 is a means for storing biometric information that is information on a biometric identification space extracted in advance from a biometric image. For example, the biometric information storage unit 409 is executed by the flash memory 224, the RAM 223 in FIG. 2, and the CPU 221 in FIG. This is realized by a program or the like.

  The non-biological information storage unit 410 is a unit that stores non-biological information that is information on a non-biological identification space extracted in advance from a non-biological image. This is realized by a program executed by the CPU 221.

  The identification space information storage unit (second storage unit) 411 includes a feature information extracted by the feature information extraction unit 403, which is obtained by removing unnecessary elements from the first projection vector that projects the biological information and the non-biological information. This is means for storing two projection vectors. The identification space information storage unit 411 is realized by, for example, a program executed by the flash memory 224, the RAM 223 in FIG. 2, the CPU 221 in FIG.

  The unnecessary space information storage unit (first storage unit) 412 does not use the feature information extracted by the feature information extraction unit 403 for biometric determination among the first projection vectors projected onto the biometric information and the non-biometric information. A means for storing information on unnecessary elements. The unnecessary space information storage unit 412 is realized by, for example, a program executed by the flash memory 224, the RAM 223 in FIG. 2, the CPU 221 in FIG.

  With the above configuration, the image forming apparatus 101 can omit calculation of unnecessary elements in, for example, feature information extraction processing by the feature information extraction unit 403 or biometric determination processing in the biometric determination unit 404. Can be

  In addition, since the image forming apparatus 101 extracts, as unnecessary elements, elements that have a small influence on the determination result of the biological determination from among the elements included in the first projection vector, it is possible to suppress a decrease in the accuracy of the biological determination. .

  Therefore, according to the image forming apparatus 101 according to the present embodiment, in the image forming apparatus 101 that performs the living body determination for determining whether or not the person photographed in the image is a living body, a decrease in the accuracy of the living body determination is suppressed. However, the time required for the living body determination can be shortened.

  Note that the functional configuration of the image forming apparatus 101 illustrated in FIG. 5 is an example. For example, the image forming apparatus 101 does not have to include the unnecessary space extraction unit 406.

  FIG. 5 is another example of the functional configuration of the image forming apparatus according to the first embodiment. The image forming apparatus 101 shown in FIG. 5 has a configuration in which the unnecessary space extraction unit 406 is removed from the image forming apparatus 101 shown in FIG.

  In this case, the image forming apparatus 101 stores unnecessary elements acquired by other information processing apparatuses in the unnecessary space information storage unit 412 in advance. In addition, the image forming apparatus 101 stores in advance in the identification space information storage unit 411 the second projection vector obtained by removing other unnecessary elements from the first projection vector acquired by another information processing apparatus.

  As a result, also in the image forming apparatus 101 shown in FIG. 5, in the image forming apparatus 101 that performs biometric determination for determining whether or not the person photographed in the image is a living body, while suppressing a decrease in the accuracy of the biometric determination. The time required for the living body determination can be shortened.

  FIG. 6 is a diagram illustrating an example of a functional configuration of the information processing apparatus according to the first embodiment. An information processing apparatus 600 illustrated in FIG. 6 includes a photographing unit 401, a feature information extraction unit 403, a noise determination unit 405, an unnecessary space extraction unit 406, and a biological information storage unit 409, among the functional configurations of the image forming apparatus 101 illustrated in FIG. A non-biological information storage unit 410, an identification space information storage unit 411, and an unnecessary space information storage unit 412.

  With the above configuration, the information processing apparatus 600 acquires information on unnecessary elements that are not used for biometric determination from the first projection vector from the image captured by the imaging unit 401 and stores the information in the unnecessary space information storage unit 412. To do. In addition, the information processing apparatus 600 stores the second projection vector obtained by removing unnecessary elements from the first projection vector in the identification space information storage unit 411. The information on the unnecessary elements stored in the unnecessary space information storage unit 412 by the information processing apparatus 600 and the information on the second projection vector stored in the identification space information storage unit 411 are, for example, the image forming apparatus 101 shown in FIG. It is memorized and used.

<Process flow>
Subsequently, a flow of processing of the living body determination method according to the first embodiment will be described.

(Unnecessary element extraction processing)
FIG. 14 is a flowchart illustrating an example of unnecessary element extraction processing according to the first embodiment. This process shows an example of a process of extracting unnecessary elements by the unnecessary space extracting unit 406 of the image forming apparatus 101 or the information processing apparatus 600 and a process of generating a second projection vector.

  In step S1401, the feature information extraction unit 403 reads images captured by the imaging unit 401 at predetermined time intervals.

  In step S1402, the feature information extraction unit 403 extracts feature information (eg, CHLAC feature amount) from the image read in step S1401.

  Subsequently, the unnecessary space extraction unit 406 executes an unnecessary space extraction process 1400 shown in steps S1403 to S1409.

  In step S1403, the unnecessary space extraction unit 406 creates the discrimination space described above.

  In the processing of steps S1404 to S1407, the unnecessary space extraction unit 406 determines whether each element is an unnecessary element from element numbers 1 to 251 (Nmax) of the projection vector as shown in FIG. 12, for example. Do.

  For example, in step S1401, the unnecessary space extraction unit 406 initializes the element number N to 1.

  In step S1405, the unnecessary space extraction unit 406 determines whether the absolute value of each value of the element number N is less than a threshold value (for example, “0.010”).

  If the absolute value of each value of the element number N is not less than a threshold value (for example, “0.010”), the unnecessary space extraction unit 406 causes the process to proceed to step S1407. On the other hand, when each value of the element number N is less than a threshold value (for example, “0.010”), the unnecessary space extraction unit 406 shifts the processing to step S1406.

  In step S1406, the unnecessary space extraction unit 406 stores the element number N determined as the absolute value of each value below the threshold (determined as an unnecessary element) in the unnecessary space information storage unit 412 as an unnecessary element. Remember.

  In step S1407, the unnecessary space extraction unit 406 determines whether the element number N is greater than or equal to Nmax, which is the maximum value (for example, 251) of the element number N.

  If the element number N is not greater than or equal to Nmax, the unnecessary space extraction unit 406 causes the process to proceed to step S1408. On the other hand, if the element number N is greater than or equal to Nmax, the unnecessary space extraction unit 406 causes the process to proceed to step S1409.

  In step S1408, the unnecessary space extraction unit 406 adds 1 to the element number N, returns the process to step S1405, and repeats the same process.

  In step S1409, the unnecessary space extraction unit 406, for example, a projection vector (first projection vector) obtained by removing unnecessary elements stored in the unnecessary space information storage unit 412 as shown in FIG. 2nd projection vector) is created. The unnecessary space extraction unit 406 stores the created second projection vector in the identification space information storage unit 411.

  By the above processing, for example, information such as element numbers 3, 4, 7, 246, etc., among the projection vectors shown in FIG. 13 is stored in the unnecessary space information storage unit 412. For example, among the projection vectors shown in FIG. 13, a 247-dimensional projection vector excluding element numbers 3, 4, 7, and 246 is stored in the identification space information storage unit 411.

(Biological determination processing)
FIG. 15 is a flowchart illustrating an example of biometric determination processing of the image forming apparatus according to the first embodiment.

  In step S1501, the imaging unit 401 captures a plurality of time-series images at predetermined time intervals.

  In step S1502, the face image detection unit 402 performs a process of detecting a face image from the image captured by the image capturing unit 401, and determines whether a face image is detected.

  When the face image is not detected, the face image detection unit 402 returns the process to step S1501 and repeats the same process. On the other hand, when a face image is detected, the face image detection unit 402 shifts the processing to step S1503.

  In step S1503, the feature information extraction unit 403 reads information on unnecessary elements (for example, element numbers 3, 4, 7, 246, etc.) stored in the unnecessary space information storage unit 412.

  In step S1504, the feature information extraction unit 403 extracts feature information (eg, CHLAC feature amount) from the face image detected by the face image detection unit 402.

  Preferably, at this time, the feature information extraction unit 403 extracts feature information of element numbers (dimensions) excluding unnecessary elements (for example, element numbers 3, 4, 7, 246, etc.) read from the unnecessary space information storage unit 412. To do. This is because the feature information of unnecessary elements is not used in the biometric determination process by the biometric determination unit 404. Thereby, the load of the feature information extraction processing by the feature information extraction unit 403 is reduced, so that the processing can be speeded up.

  In step 1505, the living body determination unit 404 reads the second projection vector stored in the identification space information storage unit 411.

  In step S <b> 1506, the biometric determination unit 404 performs biometric determination of the face image detected by the face image detection unit 402 using the read second projection vector and the feature information extracted by the feature information extraction unit 403. Accordingly, the photographing unit 401 determines whether the person photographed in the image is a living body or a non-living body (for example, a photograph).

  In step S1507, the living body determination unit 404 branches the process according to the result of the living body determination, for example.

  If it is not determined that the face image is a living body, the living body determination unit 404 returns the process to step S1501 and repeats the same process. On the other hand, if it is determined that the face image is a living body, the process proceeds to the next step (S1508).

  It is arbitrary how to use the determination result of the biological determination in the next step.

  As an example, in step S <b> 1508, the image authentication unit 407 executes face authentication processing of the detected face image, and permits the use of the image forming apparatus 101 when the face authentication is successful. On the other hand, the image authentication unit 407 does not permit the use of the image forming apparatus 101 when face authentication is not successful.

  As described above, according to the present embodiment, the feature information extraction unit 403 and the biometric determination unit 404 perform the process by excluding unnecessary elements stored in the unnecessary space information storage unit 412, so that the biometric determination process is accelerated. Is done. Moreover, since the unnecessary element memorize | stored in the unnecessary space information storage part 412 is an element (dimension) judged not to influence biometric determination, the fall of the precision of biometric determination can be suppressed.

  Therefore, according to the present embodiment, in the information processing system 100 that performs biological determination for determining whether or not the person photographed in the image is a living body, it is necessary for the biological determination while suppressing a decrease in the accuracy of the biological determination. Time can be shortened.

[Second Embodiment]
In the first embodiment, an example in which discriminant analysis is used for biological determination has been described. However, discriminant analysis is an example of a biological determination method, and the biological determination method may be another method.

  In the second embodiment, an example in which the subspace method is used for biological determination will be described.

<Functional configuration>
The functional configuration of the information processing system 100 according to the second embodiment is the same as the functional configuration of the information processing system 100 illustrated in FIGS.

  However, the living body determination unit 404 according to the second embodiment performs living body determination using the subspace method.

  In addition, the unnecessary space extraction unit 406 according to the second embodiment is unnecessary in common with the biological information stored in the biological information storage unit 409 and the non-biological information stored in the non-biological information storage unit 410. Elements are extracted by principal component analysis and stored in the unnecessary space information storage unit 412.

(Biometric judgment by subspace method)
Here, a living body determination method using the subspace method will be described.

  FIG. 16 is a diagram for explaining the subspace method according to the second embodiment. When the subspace method is used for the biometric determination, the angle of the class (identification space) can be used as the similarity (similarity representing the biologicalness).

  For example, in the 251-dimensional discriminant space 1600 shown in FIG. 16, “biological information 1” prepared in advance is an m-dimensional (m is a natural number) partial space P1601, and feature information for performing biometric determination is n-dimensional (n is a natural number). Of the subspace Q1602. At this time, n canonical angles θi, which are angles between the partial spaces, can be defined between the partial space P1601 and the partial space Q1602 (m ≧ n).

  The n canonical angles θi are expressed by the following (formula 5).

このとき、類似度は次の（式６）で定義される。

At this time, the similarity is defined by the following (formula 6).

この類似度は、２つの部分空間が一致するとき１．０となり、２つの部分空間が直交するとき０．０となる。

This similarity is 1.0 when the two subspaces match and 0.0 when the two subspaces are orthogonal.

  As an example, the biometric determination unit 404 can calculate the similarity between the feature information for performing biometric determination and each class (identification space), and can determine the class with the highest similarity as the determination result of the biometric determination.

  For example, it is assumed that there are eight classes (identification spaces) of biological information 1 to 3 and non-biological information 1 to 5. The similarity between the feature information and the biological information 1 is “0.75”, the similarity between the feature information and the biological information 2 is “0.96”, and the similarity between the feature information and the biological information 3 is “0. 23 ”, the similarity between the feature information and the non-biological information 1 is“ 0.52 ”. The similarity between the feature information and the non-biological information 2 is “0.33”, the similarity between the feature information and the non-biological information 3 is “0.45”, and the similarity between the feature information and the non-biological information 4 is Assume that “0.68” and the similarity between the feature information and the non-biological information 5 is “0.11”.

  In this case, the biometric determination unit 404 can classify the biometric information 2 that has a higher degree of similarity to the characteristic information for performing biometric determination than other classes, and determine that the biometric characteristic information is biometric characteristic information.

  As another example, the biometric determination unit 404 can use biometric information as a target and set a class equal to or higher than a threshold value as a determination result of the biometric determination.

  For example, the similarity between the feature information and the biological information 1 is “0.66”, the similarity between the feature information and the biological information 2 is “0.86”, and the similarity between the feature information and the biological information 3 is “ It is assumed that the similarity between the feature information and the non-biological information 1 is “0.52”. The similarity between the feature information and the non-biological information 2 is “0.33”, the similarity between the feature information and the non-biological information 3 is “0.45”, and the similarity between the feature information and the non-biological information 4 is It is assumed that the similarity between the feature information and the non-biological information 5 is “0.11”.

  The target threshold is set to “0.80”. In this case, the biometric determination unit 404 is biometric feature information because there is the biometric information 2 in which the similarity to the feature information for performing biometric determination is greater than or equal to the threshold value among the target biometric information 1 to 3. It can be judged.

  In this method, the living body determination unit 404 may not calculate the similarity with the non-biological information 1 to 5. As another method, the living body determination unit 404 may add to the determination condition that the similarity to the non-biological information 1 to 5 is equal to or less than another threshold value.

(Extraction of unnecessary elements by principal component analysis)
Next, a method for extracting unnecessary elements by principal component analysis will be described.

  In the second embodiment, the unnecessary space extraction unit 406 has a higher contribution ratio when the principal component analysis is performed on all the classes of information that is common to each class (identification space). The element of the principal component vector of (threshold value) is stored in the unnecessary space information storage unit 412 as an unnecessary element.

  FIG. 17 is a diagram illustrating an image of a partial space of biological information and non-biological information according to the second embodiment. In FIG. 17, since it is difficult to draw a three-dimensional or more space at the same origin, the 251-dimensional discrimination space 1700 is represented by a three-dimensional space.

  In the example of FIG. 17, three partial spaces of “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 1” 1703 are shown in the discrimination space 1700.

  The unnecessary space extraction unit 406 performs principal component analysis to extract unnecessary elements that exist in common in each class (“biological information 1”, “biological information 2”, “non-biological information 1”), and contributes. A principal component vector having a higher rate of X% is extracted.

  For example, the unnecessary space extraction unit 406 performs principal component analysis according to the following procedure, and extracts a principal component vector having a higher contribution rate of X%.

  First, the unnecessary space extraction unit 406 obtains the covariance matrix S from the feature values of the biological information and the non-biological information. The covariance matrix S is obtained by the following (Expression 7).

この共分散行列Ｓの各要素(Ｓxx, Ｓxy, Ｓyy)については、下記のように求めることができる。

Each element (Sxx, Sxy, Syy) of the covariance matrix S can be obtained as follows.

  The vector X shown in the following (Equation 8) is introduced.

同様にして、ベクトルＹを導入する。ベクトルＹは、（式８）のＸをＹに置き換えることにより得られる。

Similarly, the vector Y is introduced. The vector Y is obtained by replacing X in (Equation 8) with Y.

  The variance using the vector X and the vector Y can be expressed in the form of a product of a vertical vector and a horizontal vector as shown in (Equation 9).

（式９）において、「'」は、転置を表す。

In (Formula 9), “′” represents transposition.

  Note that the number of elements in the covariance matrix S varies depending on the number of classes of biological information and non-biological information. A covariance matrix S shown in (Expression 7) shows an example in which the number of classes is two. When the number of classes is 3, the covariance matrix is [3 × 3], and when the number of classes is 4, the covariance matrix is [4 × 4].

  Next, the unnecessary space extraction unit 406 obtains the eigenvalue λ and eigenvector μ of the covariance matrix S using the relational expression shown in (Expression 10).

また、不要空間抽出部４０６は、求めた固有値λの中で、値が大きい順にλ1…nを決定し、決定したλ1…nの各々について、（式１０）を用いて、固有ベクトルμ1…nの値を求める。

Further, the unnecessary space extraction unit 406 determines λ1... N in descending order from the determined eigenvalue λ, and uses (Equation 10) for each of the determined λ1. Find the value.

  Next, the unnecessary space extraction unit 406 sets the eigenvector μ1 corresponding to the eigenvalue λ1 having the largest value among the obtained eigenvalues λ as the first principal component vector. Here, the eigenvalue λ1 corresponds to the dispersion in that direction.

  The number of principal component vectors obtained by the unnecessary space extraction unit 406 is determined from the eigenvalue λi by the cumulative contribution rate obtained by the following (Expression 11).

なお、累積寄与率は、寄与率を所定の順序で順次に加えたものである。例えば、第１主成分から、第ｋ主成分（ｋは２以上の整数）までの寄与率の和を、第ｋ主成分までの累積寄与率と呼ぶ。第ｋ主成分までの累積寄与率は、ｋ個の主成分が、全体の情報量の何パーセントの情報を保有しているかを示している。

The cumulative contribution rate is obtained by sequentially adding contribution rates in a predetermined order. For example, the sum of contribution rates from the first principal component to the k-th principal component (k is an integer of 2 or more) is referred to as a cumulative contribution rate up to the k-th principal component. The cumulative contribution rate up to the k-th principal component indicates what percentage of the total information amount the k principal components have.

  In the present embodiment, a principal component vector necessary for biometric determination is extracted by setting a threshold value (X%) indicating what percentage of the total information amount is the principal component.

  For example, from the three subspaces of “biological information 1” 1701, “biological information 2” 1702 and “non-biological information 1” 1703 shown in FIG. 17, the contribution ratio of the principal component analysis is 95% (X% threshold value). When a principal component vector satisfying one example) is extracted, a partial space 1801 obtained by principal component analysis as shown in FIG. 18 is obtained.

  The partial space 1801 obtained by this principal component analysis is a space (vector) common to the three partial spaces of “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 1” 1703. It can be determined that it is not effective for biometric determination.

  Subsequently, the unnecessary space extraction unit 406 projects onto the complementary space of the partial space 1801 and creates a partial space excluding the partial space 1801.

  For example, the projected subspace is Pc, the input subspace is P, the projection matrix is P = [p1, p2, ..., pn], the constraint subspace (complementary space) is C, and the projection matrix is C = [c1 , c2,..., cn], K is the number of bases used in P, and M is the number of bases used for projection in C.

  At this time, the base vector pci of the subspace Pc obtained by projecting P onto C is expressed by the following (Equation 12).

例えば、図１７に示す「生体情報１」１７０１、「生体情報２」１７０２、「非生体情報３」１７０３を、部分空間１８０１の補空間に射影したときのイメージを図１９に示す。

For example, FIG. 19 shows an image when “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 3” 1703 shown in FIG. 17 are projected onto the complementary space of the partial space 1801.

  For example, in the 251-dimensional discriminant space 1700, when the partial space 1801 obtained by the principal component analysis has 30 dimensions, the number of elements of the projection vector is 251. Further, a complementary space 1900 of the partial space 1801 obtained by the principal component analysis is a 221-dimensional space. In the example of FIG. 19, images of a partial space 1901 of “biological information 1” 1701, a partial space 1902 of “biological information 2” 1702, and a partial space 1903 of “non-biological information 1” on the complementary space 1900 are shown. ing.

  FIG. 20 is a diagram illustrating an example of a projection vector according to the second embodiment. The projection vector shown in FIG. 20 is an example of a projection vector for projecting feature information to be subjected to biological determination to a complementary space 1900 (221 dimensions) of a partial space 1801 obtained by principal component analysis.

  The unnecessary space extraction unit 406 extracts unnecessary elements that are unnecessary elements (dimensions) from, for example, a projection vector as shown in FIG. For example, the unnecessary space extraction unit 406 determines in FIG. 20 that the element corresponding to the element number for which the absolute value of all the values corresponding to each element number is less than the threshold is an unnecessary element, and the unnecessary space information storage unit 412 is stored. For example, FIG. 21 shows an example where the threshold value is “0.010” in FIG.

  FIG. 21 is a diagram illustrating an example of unnecessary elements of the projection vector according to the second embodiment. In the example of FIG. 21, the element numbers 3, 4, 7, and 246 have the absolute values of all values corresponding to the eigenvectors μ1 to μ221 being less than the threshold value “0.010”. It is determined as an unnecessary element and stored in the unnecessary space information storage unit 412.

  Preferably, the unnecessary space extraction unit 406 obtains a projection vector (second projection vector) obtained by removing unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) shown in FIG. Store in the identification space information storage unit 411.

  Also in the present embodiment, the living body determination unit 404 uses the second projection vector stored in the identification space information storage unit 411, and extracts the feature information extracted by the feature information extraction unit 403, as in the first embodiment. Perform discriminant analysis.

  Preferably, when extracting the feature information, the feature information extraction unit 403 also excludes unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from the 251D feature information, Perform extraction.

  The principal component vector with the higher contribution rate is information common to each class (identification space). Therefore, in this embodiment, since unnecessary elements common to each class are removed from the second projection vector used for the biometric determination, each class is differentiated, and the determination accuracy of the biometric determination is improved. I can expect.

<Process flow>
FIG. 22 is a flowchart illustrating an example of unnecessary space extraction processing according to the second embodiment.

  In step S2201, the feature information extraction unit 403 reads images captured by the imaging unit 401 at predetermined time intervals.

  In step S2202, the feature information extraction unit 403 extracts feature information (for example, CHLAC feature amount) from the image read in step S2201.

  Subsequently, the unnecessary space extraction unit 406 executes an unnecessary space extraction process 2200 shown in steps S2203 to S2212.

  In step S2203, the unnecessary space extraction unit 406 extracts unnecessary elements that exist in common in each class (for example, “biological information 1”, “biological information 2”, and “non-biological information 1” in FIG. 17). The principal component analysis described above is performed.

  In step S2204, the unnecessary space extraction unit 406 extracts the principal component vector having the cumulative contribution rate of X% described above.

  In step S2205, the unnecessary space extraction unit 406 creates a partial space 1801 (common partial space) obtained by principal component analysis as shown in FIG. 18, for example.

  In step S2206, the unnecessary space extraction unit 406 creates a complementary space 1900 of the partial space 1801 obtained by principal component analysis as shown in FIG. 19, for example.

  In the processing of steps S2207 to S2210, the unnecessary space extraction unit 406 determines whether each element is an unnecessary element from element numbers 1 to 251 (Nmax) of the projection vector, for example, as shown in FIG. Do.

  For example, in step S2207, the unnecessary space extraction unit 406 initializes the element number N to 1.

  In step S2208, the unnecessary space extraction unit 406 determines whether the absolute value of each value of the element number N is less than a threshold value (for example, “0.010”).

  If the absolute value of each value of the element number N is not less than a threshold value (for example, “0.010”), the unnecessary space extraction unit 406 causes the process to proceed to step S2210. On the other hand, when the absolute value of each value of the element number N is less than a threshold value (for example, “0.010”), the unnecessary space extraction unit 406 causes the process to proceed to step S2209.

  In step S2209, the unnecessary space extraction unit 406 stores in the unnecessary space information storage unit 412 the element number N for which the absolute value of each value is determined to be less than the threshold (determined as an unnecessary element) as an unnecessary element. Remember.

  In step S2210, the unnecessary space extraction unit 406 determines whether the element number N is greater than or equal to Nmax, which is the maximum value (for example, 251) of the element number N.

  If the element number N is not equal to or greater than Nmax, the unnecessary space extraction unit 406 causes the process to proceed to step S2211. On the other hand, if the element number N is greater than or equal to Nmax, the unnecessary space extraction unit 406 causes the process to proceed to step S2212.

  In step S2211, the unnecessary space extraction unit 406 adds 1 to the element number N, returns the process to step S2208, and repeats the same process.

  When the process proceeds to step S2212, the unnecessary space extraction unit 406, for example, a projection vector (first projection vector) as shown in FIG. 20 excluding unnecessary elements stored in the unnecessary space information storage unit 412 (see FIG. 20). 2nd projection vector) is created. The unnecessary space extraction unit 406 stores the created second projection vector in the identification space information storage unit 411.

  By the above processing, for example, information such as element numbers 3, 4, 7, 246, etc., among the projection vectors shown in FIG. 21 is stored in the unnecessary space information storage unit 412. Further, for example, among the projection vectors shown in FIG. 21, a 247-dimensional projection vector excluding element numbers 3, 4, 7, 246 is stored in the identification space information storage unit 411.

  Note that the flow of the biological determination process according to the second embodiment is the same as the biological determination process of the first embodiment shown in FIG. However, the biometric determination unit 404 according to the second embodiment performs biometric determination using the above-described subspace method using the second projection vector stored in the identification space information storage unit 411.

  As described above, according to the present embodiment, the feature information extraction unit 403 and the biometric determination unit 404 perform the process by excluding unnecessary elements stored in the unnecessary space information storage unit 412, so that the biometric determination process is accelerated. Is done.

  Further, according to the present embodiment, since unnecessary elements common to each class are removed from the second projection vector used for the biometric determination, each class is differentiated, and the determination accuracy of the biometric determination is improved. The effect can be expected.

[Other Embodiments]
The processing shown in the first embodiment and the second embodiment is an example, and various modifications and applications are possible.

  As an example, the information processing system 100 can improve the accuracy of biometric determination by discarding an image captured by the image capturing unit 401 and using the next image.

  FIG. 23 is a flowchart illustrating an example (1) of the biometric determination process according to another embodiment. Note that the processing in steps S1501 and S1502-1508 in FIG. 23 is the same as the biometric determination processing according to the first embodiment shown in FIG.

  In step S2301, the noise determination unit 405 determines whether the amount of noise included in the image captured by the imaging unit 401 is less than a predetermined value.

  If the amount of noise included in the image captured by the image capturing unit 401 is equal to or greater than a predetermined value, the noise determination unit 405 discards the image captured by the image capturing unit 401 and returns the process to step S1501.

  On the other hand, when the amount of noise included in the image captured by the imaging unit 401 is equal to or greater than a predetermined value, the noise determination unit 405 shifts the process to step S1503 and starts the biometric determination process.

  As a result of the above processing, the information processing system 100 can improve the accuracy of the biometric determination by discarding the image captured by the imaging unit 401 and using the next image when noise is present. .

  As another example, the image forming apparatus 101 performs high-speed biological determination using information on unnecessary elements stored in the unnecessary space information storage unit 412 and the second projection vector stored in the identification space information storage unit 411. Whether or not can be set.

  For example, the image forming apparatus 101 can display a biometric determination mode setting screen 2400 as shown in FIG. 24 on the operation panel 226 and accept settings by an administrator, a user, or the like. In the example of FIG. 24, the biometric determination mode setting screen 2400 includes a “high-speed processing mode” button 2401 for selecting high-speed biometric determination using unnecessary element information and the second projection vector, and normal biometric determination. A “normal processing mode” button 2402 is displayed.

  The administrator or the user selects the high-speed biometric determination using the unnecessary element information and the second projection vector by selecting the “high-speed processing mode” button 2401 from the biometric determination mode setting screen 2400. be able to. In addition, the administrator or the user selects the “normal processing mode” button 2402 from the biometric determination mode setting screen 2400, thereby performing high-speed biometric determination using information on unnecessary elements and the second projection vector. It is possible to select a normal biometric determination that is not performed.

  FIG. 25 is a flowchart illustrating an example (2) of the biometric determination process according to another embodiment. Note that the processes in steps S1501, S1502, and S1504 to S1508 in FIG. 25 are the same as the biometric determination process according to the first embodiment shown in FIG. 15, and here, the differences from the first embodiment are mainly described here. Give an explanation.

  In step S2501, the biometric determination unit 404 determines whether the image forming apparatus 101 is set to a high-speed processing mode in which biometric determination is performed at high speed using unnecessary element information and the second projection vector.

  When the high-speed processing mode is set, the biological determination unit 404 shifts the process to step S1503 and executes the biological determination process of the first embodiment illustrated in FIG.

  On the other hand, when the high-speed processing mode is not set, the biometric determination unit 404 performs normal biometric determination processing using biometric feature information, non-biological feature information, and the like.

  With the above processing, for example, in the image forming apparatus 101 shown in FIG. 5, even when necessary information is not stored in the identification space information storage unit 411 and the unnecessary space information storage unit 412, the image forming apparatus 101 performs biometric determination. Will be able to do.

100 Information processing system 101 Image forming apparatus (an example of an image processing apparatus)
401 Imaging unit 403 Feature information extraction unit 404 Biometric determination unit 406 Unnecessary space extraction unit 411 Identification space information storage unit (second storage unit)
412 Unnecessary space information storage unit (first storage unit)

JP 2006-330936 A

Claims (11)

An information processing system for performing a living body determination for determining whether a person photographed in an image is a living body using information on a living body identification space extracted in advance from a living body image,
A photographing unit for photographing the image;
A feature information extraction unit for extracting feature information of a person photographed in the image;
First information storing unnecessary element information that is not used for the living body determination among elements of a first projection vector that projects the feature information extracted by the feature information extracting unit onto a discrimination space including the identification space of the living body. A storage unit of
Whether the person photographed in the image is a living body using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
An information processing system having The feature information extraction unit includes:
The information processing system according to claim 1, wherein the feature information excluding the unnecessary elements is extracted from the image photographed by the photographing unit. The feature information is a vector of a plurality of dimensions,
3. The information processing system according to claim 1, wherein the information on the unnecessary element includes information for specifying a dimension that is not used for the biometric determination among the plurality of dimension vectors.   4. The information processing system according to claim 1, further comprising a second storage unit that stores the second projection vector obtained by removing the unnecessary element from the first projection vector. 5.   Of the elements of the first projection vector, elements whose absolute values corresponding to a plurality of identification spaces including the identification space of the living body are less than a threshold value are determined as the unnecessary elements, and the first storage The information processing system according to any one of claims 1 to 4, further comprising an unnecessary space extraction unit stored in the unit.   Among the elements of the first projection vector, an element that is present in common in a plurality of identification spaces including the identification space of the living body and the identification space of a non-living body is determined as the unnecessary element by principal component analysis, The information processing system according to any one of claims 1 to 5, further comprising an unnecessary space extraction unit that is stored in one storage unit. An information processing apparatus included in the information processing system according to any one of claims 1 to 6,
An acquisition unit for acquiring the image;
A feature information extraction unit for extracting feature information of a person photographed in the image;
From the feature information extracted by the feature information extraction unit, information on unnecessary elements that are not used for biometric determination is extracted and stored in a first storage unit, and a projection vector is created by removing the unnecessary elements from the feature information. An unnecessary space extraction unit stored in the second storage unit,
An information processing apparatus. An image processing apparatus that performs a living body determination for determining whether a person photographed in an image is a living body using information on a living body identification space extracted in advance from a living body image,
A photographing unit for photographing the image;
A feature information extraction unit for extracting feature information of a person photographed in the image;
First information storing unnecessary element information that is not used for the living body determination among elements of a first projection vector that projects the feature information extracted by the feature information extracting unit onto a discrimination space including the identification space of the living body. A storage unit of
Whether the person photographed in the image is a living body using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
An image processing apparatus. The image processing apparatus according to claim 8,
A main body provided with at least an engine part;
An operation unit for receiving an operation instruction to the main body;
Have
An image processing apparatus comprising: the feature information extraction unit, the first storage unit, and a living body determination unit in the operation unit. A computer for performing a living body determination for determining whether a person photographed in an image is a living body using information on a living body identification space extracted in advance from a living body image,
A photographing unit for photographing the image;
A feature information extraction unit for extracting feature information of a person photographed in the image;
First information storing unnecessary element information that is not used for the living body determination among elements of a first projection vector that projects the feature information extracted by the feature information extracting unit onto a discrimination space including the identification space of the living body. A storage unit of
Whether the person photographed in the image is a living body using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
Program to function as. Obtaining a captured image; and
Extracting feature information of a person photographed in the image;
Extracting unnecessary element information not used for biometric determination from the extracted feature information and storing it in the first storage unit;
Creating a projection vector excluding the unnecessary elements from the feature information and storing it in a second storage unit;
A program that causes a computer to execute.

Images