JP6707906B2 - Information processing apparatus, information processing system, authentication method, and program - Google Patents

Description

The present application relates to an information processing device, an information processing system, an authentication method, and a program.

As an authentication method for authenticating a user, image authentication technology such as face authentication, which does not require an operation such as password input and can prevent spoofing due to loss or theft of an IC card, has been widespread.

For example, an image processing apparatus that authenticates a user by comparing and collating information generated from a face image obtained by capturing an area including the face of the user with authentication information registered in advance is known. (For example, see Patent Document 1).

However, for example, an authentication method using a captured image such as face authentication has lower authentication accuracy than an authentication method using a widely used IC card, and there is a risk of spoofing with a face photograph. However, it has been difficult to improve the accuracy of authentication.

Embodiments of the present invention have been made in view of the above problems, and improve the accuracy of user authentication in an information processing apparatus that authenticates a user with a captured image such as face authentication. With the goal.

In order to solve the above problems, an information processing apparatus according to an embodiment of the present invention is an information processing apparatus that authenticates a user, and a wireless communication unit that acquires predetermined information from a wireless tag and the acquired predetermined information. a wireless authentication means for performing wireless authentication to authenticate the wireless tag based on information, and imaging means for capturing an image, and image authentication means for performing image authentication for authenticating the user who is captured in the image, the image Determination means for performing image determination for determining whether the imaged determination target is a living body satisfying a predetermined condition, and determining that the determination target captured in the image is a living body satisfying the predetermined condition If the wireless authentication and the image authentication are permitted, whether the user of the wireless tag permitted the wireless authentication and the user permitted the image authentication are the same user. determine the case the user and the user who image authentication is permitted in the wireless tag wireless authentication is permitted is the same user, to allow the use of the information processing apparatus of said user Device authentication means.

According to the embodiments of the present invention, it is possible to improve the accuracy of user authentication in an information processing apparatus that authenticates a user with a captured image such as face authentication.

It is a figure which shows the example of a functional structure of the image processing apparatus which concerns on 1st Embodiment. It is a figure which shows the example of the hardware constitutions of the image processing apparatus which concerns on 1st Embodiment. 6 is a flowchart illustrating an example of an actual object determination process based on motion information of a person area according to the first embodiment. It is an image figure until a person area is detected according to the first embodiment. It is a figure explaining the real thing judgment using the motion information of the person area concerning a 1st embodiment. 6 is a flowchart showing an example of an actual object determination process based on motion information of a face area according to the first embodiment. FIG. 6 is an image diagram until motion information is calculated from a face area according to the first embodiment. It is a flow chart which shows an example of real thing judgment processing when a background field concerning a 1st embodiment is less than a threshold. It is an image figure when the background area concerning a 1st embodiment is less than a threshold. 6 is a flowchart illustrating an example of an actual object determination process including outlier removal according to the first embodiment. It is a figure explaining the outlier removal which concerns on 1st Embodiment. It is a flow chart which shows the example of the real thing judgment processing which normalizes the direction of a person concerning a 1st embodiment. It is an image figure which normalizes the direction of a person concerning a 1st embodiment. It is a flow chart which shows an example of real thing judgment processing including update of dictionary data concerning a 1st embodiment. It is a figure explaining the operation information used by the update of the dictionary data which concerns on 1st Embodiment. It is a figure which shows the structural example of the information processing system which concerns on 1st Embodiment. It is a figure which shows an example of the imaging range of the camera which concerns on 1st Embodiment, and the detection range of each sensor. It is a figure which shows the example of the hardware constitutions of the image forming apparatus which concerns on 1st Embodiment. FIG. 3 is a diagram showing an example of a functional configuration of the image forming apparatus according to the first embodiment. It is a figure which shows the example of the user information which concerns on 1st Embodiment. 6 is a flowchart illustrating an example of wireless authentication processing according to the first embodiment. 6 is a flowchart illustrating an example of image authentication processing according to the first embodiment. It is a figure for demonstrating extraction of the face image which concerns on 1st Embodiment. 6 is a flowchart showing an example of image determination processing according to the first embodiment. 6 is a flowchart showing an example of device authentication processing according to the first embodiment. 9 is a flowchart showing an example of image determination processing according to Modification 1. 9 is a flowchart showing an example of image determination processing according to modification 2. 14 is a flowchart showing an example of timeout processing of device authentication processing according to Modification 3; 9 is a flowchart showing an example of alternative authentication processing according to Modification 4. 9 is a flowchart showing an example of image authentication processing according to Modification 5. 13 is a flowchart showing an example of transmission control processing according to modification 6. It is a figure (1) for explaining layering concerning a 2nd embodiment. It is a figure (2) for explaining layering concerning a 2nd embodiment. It is a figure (3) for explaining layering concerning a 2nd embodiment. It is a figure (4) for demonstrating the hierarchization which concerns on 2nd Embodiment. It is a figure (5) for explaining layering concerning a 2nd embodiment. 9 is a flowchart showing an example of image determination processing according to the second embodiment.

Hereinafter, embodiments will be described in detail.

<Image processing device: Functional configuration>
FIG. 1 is a diagram illustrating an example of a functional configuration of the image processing apparatus according to the first embodiment. As shown in FIG. 1, the image processing apparatus 10 includes a camera image receiving unit 11, a person area detecting unit 12, a face area detecting unit 13, a background area ratio determining unit 14, and a background area ratio determining threshold setting unit 15. The operation information determining unit 16, the inter-image correlation calculating unit 17, the flow histogram calculating unit 18, the outlier value removing unit 19, the image orientation normalizing unit 20, and the dictionary updating unit 21 are included. The image processing apparatus 10 also includes a person area motion dictionary 22, a face area motion dictionary 23, and a motion information determination threshold value setting unit 24.

The camera image receiving unit 11 acquires a plurality of images captured by an image capturing device such as a camera. The camera image receiving unit 11 acquires, for example, a background image acquired in advance or an image including a person area. The camera image receiving unit 11 also acquires time-series images at predetermined time intervals.

A person area detection unit (an example of a detection unit) 12 detects a person area from a plurality of images acquired from the camera image reception unit 11. The person area detection unit 12 detects a person area using, for example, a person area detection method based on background difference or image processing. When using the background difference, for example, the person area detection unit 12 determines a portion of the time-series image or the like that matches the background image acquired in advance as a background area, and a portion that does not match the background image as a non-background area. The non-background area is determined as the person area. The method for detecting the person area is not limited to this.

The face area detection unit (an example of a detection unit) 13 detects a face area from the image acquired by the camera image reception unit 11. The face area detection unit 13 can use a face detection method based on background difference or image processing. As the face detection method, the face area detection unit 13 can use, for example, face detection based on Haar-like features, but is not limited to this.

In this face detection, a Haar-like feature quantity that is a value obtained by subtracting the sum of the pixel values of the white area from the sum of the pixel values of the black area in the rectangle to be calculated in the search area is used. .. In addition, in the detection by the Haar-like feature, it is general to discriminate the area difference of the sum of the pixel values of the rectangular small areas by a classifier created by Boosting.

Since the above-mentioned rectangular layout location may be located anywhere in the search area, there are tens of thousands of possible layout locations in the search area. With respect to these, each weak classifier in the search area is weighted by pre-learning by Boosting, and dozens of weak classifiers having higher importance are selected to create a strong classifier. With this strong discriminator (a black-and-white rectangular pattern indicating the human face-likeness), it is possible to determine whether the area is a human face area.

The background area ratio determining unit 14 calculates the ratio of the background area and the non-background area from the image acquired from the camera image receiving unit 11, and determines whether the background area is equal to or more than the threshold value or less than the threshold value, for example.

The background area ratio determination threshold setting unit (second threshold value setting unit) 15 sets a threshold value (second threshold value) used for the background area ratio determination unit 14 to determine the background area ratio. The background area ratio determination threshold setting unit 15 sets, for example, the ratio of the background area not to be processed. The background area ratio determination threshold value setting unit 15 may set the threshold value in consideration of a balance such as, for example, the detection accuracy of a person or the priority of processing speed.

For example, when giving priority to the processing speed, the background area ratio determination threshold setting unit 15 can calculate the motion information from the entire image by setting the threshold value of the ratio of the human area (non-background area) to be low. Becomes In this case, since the motion information is not calculated according to the shape of the detection area, for example, the processing amount and processing time can be reduced.

Further, the background area ratio determination threshold value setting unit 15 sets the threshold value of the ratio of the person area (non-background area) to be high when the detection accuracy of the person is prioritized. In this case, although the processing amount and processing time required when the motion information is calculated, it is possible to improve the detection accuracy of the motion determination.

The motion information determination unit (determination unit) 16 outputs the images obtained from the person region detection unit 12 and the face region detection unit 13 to the inter-image correlation calculation unit 17 and the flow histogram calculation unit 18, and the inter-image correlation calculation unit. The motion information is acquired from 17 or the flow histogram calculation unit 18.

The motion information determination unit 16 stores the motion information acquired from the inter-image correlation calculation unit 17 and the flow histogram calculation unit 18 and the motion information (dictionary data) set in the person region motion dictionary 22 and the face region motion dictionary 23 in advance. Calculate the similarity. For example, when the motion information determination unit 16 determines that the images are similar to each other by a predetermined threshold value or more, the motion information determination unit 16 uses the image of the person obtained from the person region detection unit 12 or the face region detection unit 13 as a real image (a person actually at that place). ) Is determined. In other words, the motion information determination unit 16 determines whether the determination target obtained from the person region detection unit 12 or the face region detection unit 13 is a living body that satisfies a predetermined condition.

As the discriminator used for comparison, for example, linear discriminant analysis and mutual subspace method (for example, Seiji Hotta, "Subspace Method Starting with an Example-Invitation to Pattern Recognition"), Robotics Society of Japan, 69th Robotics Seminar , May 22, 2012) and the like. In addition, the threshold used for the determination can be changed by the motion information determination threshold setting unit 24.

The inter-image correlation calculation unit (an example of a calculation unit) 17 calculates the motion information obtained from, for example, the inter-image correlation value for the image acquired from the motion information determination unit 16. The inter-image correlation calculation unit 17 takes, for example, the difference between the current frame image and the previous frame image, and in the binarized image, for example, calculates the 25-dimensional local autocorrelation feature amount (higher-order autocorrelation feature (HLAC feature)). Ask. The 25th dimension is, for example, the total sum of correlations in the local area when the dimension is quadratic and the displacement direction is limited to 3×3 pixels. The combination of the displacement vectors of the HLAC features of the binary moving image is represented as a total of 25-dimensional vectors, one of which is the 0th order, four of which is the second order and 20 of which is the second order, except for those which are equivalent by translation. ..

In addition, the inter-image correlation calculation unit 17 further calculates time-series correlations to calculate all 251-dimensional vectors, and uses them as motion information. The 251 dimension is the total sum of correlations in the local region when the dimension is quadratic and limited to 3×3×3 pixels. The number of combinations of displacement vectors of the stereoscopic higher-order autocorrelation feature (CHLAC feature) from the binary moving image is 1 for the 0th order, 13 for the 1st order, and 2 for the 2nd order There are 237, which are expressed as a total of 251 dimensional vectors.

The former 25-dimensional information is the correlation in the difference image, and the latter information is the correlation between the previous difference image and the next difference image. The inter-image correlation calculation unit 17 calculates the amount of 251 dimensional information using these two pieces of information (for example, Hideaki Imanishi et al., “Detection of suspicious behavior using CHLAC feature and subspace method”, information processing). See Academic Research Report 2012 etc.).

The flow histogram calculation unit (an example of a calculation unit) 18 calculates motion information for the image acquired from the motion information determination unit 16 by, for example, block matching. The flow histogram calculation unit 18 calculates whether the small area of the previous frame image has moved within the current frame image by block matching. The movement of each area is represented by, for example, a vector.

For example, the flow histogram calculation unit 18 calculates the optical flow, which is vector information indicating the movement of each small area included in the plurality of images acquired from the motion information determination unit 16, as the motion information using block matching.

Note that block matching is an example, and a motion vector calculation method such as a gradient method may be used (for example, Hiroshi Nagahashi, Tokyo Institute of Technology, “Image Analysis Theory (7)”, http://www.isl.titch). .Ac.jp/ to Nagahashilab/member/longb/imageanalysis/LecterNotes/ImageAnalysis07.pdf, P17-18, etc.). The flow histogram calculation unit 18 uses the obtained vector as a histogram for each gradient direction as operation information.

The outlier value removal unit 19 calculates the variance of the magnitude of the optical flow from the optical flow of each pixel in the extraction region, and removes the optical flow whose variance is equal to or greater than a certain value. Here, an optical flow having a variance of a certain value or more is called an outlier (for example, Hiroshima City University Image Media Engineering Special Thesis, http://www.ime.info.hiroshima-cu.ac.jp/~hiura). /Lec/ime/2010/5.pdf, P19-21, etc.).

The optical flow is, for example, a velocity vector of each pixel obtained from a change in the luminance value of the pixel caused by the movement of an object having temporally continuous image data. It is possible to detect a moving body using such a velocity vector.

The image orientation normalization unit 20 detects a plurality of feature points designated in advance as targets in the image area. The image orientation normalization unit 20 maps the feature points to a three-dimensional model prepared in advance, and normalizes the image orientation (the orientation of the imaging target) by facing the front. The image orientation normalizing unit 20 normalizes not only the image input from the camera image receiving unit 11 but also the image when the person area motion dictionary 22 or the face area motion dictionary 23 has data that does not face the front. Is also good.

The dictionary updating unit (updating means) 21 uses the operation information at the time of determining the actual object when the operation information determining unit 16 determines that the image is a real object, and then when it is surely not a photograph, for example, by password input by compound authentication. Update the dictionary data. In addition, the dictionary updating unit 21 updates the dictionary data of the person area movement dictionary 22 when the movement information of the person area is used. The dictionary update unit 21 updates the dictionary data of the face area motion dictionary 23 when the motion information of the face area is used.

Further, the dictionary updating unit 21 may update the dictionary data by performing conversion for identification in advance according to the discriminator used by the motion information determining unit 16. In this way, by updating the dictionary data, it is possible to obtain a dictionary that corresponds to the shooting environment, and it is possible to improve the accuracy of the actual information determination by the motion information determination unit 16.

The person area motion dictionary 22 stores the movement information (dictionary data) of the person area learned in advance. The person area motion dictionary 22 is created, for example, from motion information acquired under the situation of photographing a person image. The person area motion dictionary 22 may be converted in advance for the discriminator in association with the discriminator used in the motion information determination unit 16. For example, when the discriminator is the constrained subspace method, it is projected to the generalized difference subspace as a prior transformation (for example, Kazuhiro Fukui, “Subspace method and discriminator”, 13th Image Sensing Symposium, 2007, 6). See the 7th of the month).

The face area motion dictionary 23 stores the face area motion information (dictionary data) that has been learned in advance. The face area motion dictionary 23 is created based on motion information acquired under the situation where a face image is captured. The face area action dictionary 23 may be converted in advance for the discriminator in association with the discriminator used in the action information determination unit 16.

The motion information determination threshold value setting unit (first threshold value setting unit) 24 is a threshold value used by the motion information determination unit 16 to determine whether the person area or the face area person included in the image acquired by the camera image reception unit 11 is a real person. (First threshold value) is set. The motion information determination threshold value setting unit 24 can set or change the threshold value by a user operation, for example. The motion information determination threshold value setting unit 24 improves the accuracy of the motion information determination unit 16 in the physical determination by setting an appropriate threshold value in accordance with the user's shooting environment such as the distance to the shooting target and the presence or absence of lighting. It becomes possible.

<Image processing device 10: hardware configuration>
FIG. 2 is a diagram illustrating an example of the hardware configuration of the image processing apparatus according to the first embodiment. The image processing apparatus 10 illustrated in FIG. 2 includes, for example, an operation unit 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a CPU (Central Processing Unit) 34, and a camera 35.

The operation unit 31 is realized by, for example, a keyboard, a mouse, a touch panel, or the like. The operation unit 31 receives various operations performed by, for example, a user.

The ROM 32 is a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 32 stores programs and data such as BIOS (Basic Input/Output System), OS (Operation System) settings, and network settings that are executed when the image processing apparatus 10 is started. The RAM 33 is a volatile semiconductor memory (storage device) that temporarily holds programs and data.

The CPU 34 is an arithmetic device that realizes control and functions of the entire image processing apparatus 10 by reading programs and data from a storage device such as the ROM 32 and the RAM 33 onto the RAM 33 and executing processing.

The camera 35 is an example of an image pickup apparatus provided to pick up an image of a person whose face is recognized. The camera 35 is, for example, a digital camera or the like and is capable of acquiring images and videos, but is not limited to this.

The image processing apparatus 10 can realize various processes described later by the above hardware configuration.

<Actual judgment processing based on motion information of person area>
FIG. 3 is a flowchart showing an example of an actual object determination process based on motion information of a person area according to the first embodiment. In the example of FIG. 3, it is determined whether or not the determination target imaged by the camera 35 is a real object (for example, a photograph) using the motion information of the person area. In other words, it is determined based on the motion information of the person area whether or not the determination target imaged by the camera 35 is a living body that satisfies a predetermined condition.

As shown in the example of FIG. 3, when the image processing apparatus 10 acquires an image by the camera image receiving unit 11 in advance in the absence of a person (S10), the image is set as a background image. Further, when the image processing apparatus 10 acquires an image by the camera image receiving unit 11 (S11), the image processing apparatus 10 determines whether or not there is a difference between the background image acquired in advance by the person area detecting unit 12 and the image acquired in the process of S11. Yes (S12).

When the person area detection unit 12 determines that there is a difference from the background image (YES in S12), the person area detection unit 12 detects the difference area as a person area (S13). If the person area detection unit 12 determines that there is no difference from the background image (NO in S12), the process returns to S11.

Next, the image processing apparatus 10 determines whether or not there was a person area in the previous frame image (S14). When the image processing apparatus 10 determines that the previous frame image has a human region (YES in S14), the inter-image correlation calculation unit 17 or the flow histogram calculation unit 18 calculates motion information (S15). If the image processing apparatus 10 determines that there is no human area in the previous frame image (NO in S14), the process returns to S11.

In the process of S15, the inter-image correlation calculation unit 17 or the flow histogram calculation unit 18 calculates the motion information using, for example, the current frame image in which the human region is detected in the process of S13 and the previous frame image.

Alternatively, the flow histogram calculation unit 18 does not limit to the current frame image and the previous frame image, but uses the optical flow, which is vector information indicating the movement of each small area included in the plurality of images in which the person area is detected, as the operation information. It may be calculated or the like.

In this way, the inter-image correlation calculation unit 17 or the flow histogram calculation unit 18 calculates the motion information when a plurality of images of the person area can be acquired in time series.

Next, the image processing device 10 uses the motion information calculated in the process of S15 to determine whether the person in the person area detected from the current frame image by the motion information determination unit 16 is a real person (a person actually at that place). It is determined (S16). In the process of S16, the motion information determination unit 16 calculates the similarity between the motion information calculated in the process of S15 and the motion information (dictionary data) stored in the person region motion dictionary 22 to determine whether it is a real object. ..

When the motion information determination unit 16 determines that the similarity between the motion information calculated in the process of S15 and the dictionary data of the person area motion dictionary 22 is equal to or more than a predetermined threshold value (YES in S16), A person in the person area of the current frame image is determined as a real person (S17). In other words, the motion information determination unit 16 determines that the imaged determination target is a living body that satisfies a predetermined condition. If the motion information determination unit 16 determines that it is less than the predetermined threshold value (NO in S16), it determines that it is a photograph (S18), and ends the process. In other words, the motion information determination unit 16 determines that the imaged determination target is not a living body that satisfies a predetermined condition.

<Image until human area is detected>
FIG. 4 is an image diagram until the person area according to the first embodiment is detected. FIG. 4A shows an example of a background image acquired in advance, and FIG. 4B shows an example of an image obtained by the camera 35. FIG. 4C shows an example of the difference area detected as the person area.

An example of detecting a person area based on, for example, the background difference will be described using the background image shown in FIG. 4A and the image shown in FIG. 4B. The person area detection unit 12 calculates the absolute value by dividing the value of the luminance value of each pixel of the background image shown in FIG. 4A from each pixel of the image shown in FIG. 4B, for example.

The background image shown in FIG. 4A and the image shown in FIG. 4B are gray images of 2 pixels×2 pixels (values are luminance values of 256 gradations), for example. Here, in the background image shown in FIG. 4A, "pixels in the first column are 50 and 150, and pixels in the second column are 200 and 3", and in the image shown in FIG. Is 120, 35, and the pixels in the second column are 10, 150".

At this time, in the difference image shown in FIG. 4C, for example, “pixels in the first column are 70, 115, pixels in the second column are 190, 147”. This difference image can be used as it is, or it is also possible to use only the value of an area having a value (having a difference from the background) of a predetermined value or more in the difference image.

<Actual judgment using motion information>
FIG. 5 is a diagram illustrating an actual determination using motion information of a person area according to the first embodiment. FIG. 5A shows an image diagram of motion information calculated from images of a plurality of person areas. 5B shows an example of the brightness value of the current frame image, and FIG. 5C shows an example of the brightness value of the target block in the previous frame image. Further, FIG. 5D is a diagram illustrating a comparison between the calculated motion information and the dictionary data of the person area motion dictionary 22.

In the example of FIG. 5A, the optical flow obtained by using the block matching by the inner product is represented by an arrow as the motion information calculated from the images of the plurality of person regions. A specific example until the optical flow shown in FIG. 5A is calculated using the current frame image shown in FIG. 5B and the previous frame image shown in FIG. 5C will be described. 5B and 5C show an example in which a block size of 3 pixels×3 pixels is used, for example.

As shown in FIG. 5B, the brightness value is shown in the block to be calculated in the current frame image. Further, in the current frame image, “●” shown in FIG. 5B corresponds to the luminance value “90” of the target block in the previous frame image shown in FIG. 5C.

For example, the flow histogram calculation unit 18 calculates where the block of interest of the previous frame image shown in FIG. 5C has moved within the current frame image. The inner product is calculated with the block to be calculated.

For example, “a” in Expression 1 indicates the matrix shown in FIG. 5B, and “b” in Expression 1 indicates the matrix shown in FIG. 5C.

The flow histogram calculation unit 18 calculates the angle (cos θ) using the inner product of the two matrices described above, and sets the calculated value as the similarity between the two blocks. In the example of FIGS. 5B to 5C, for example, cos θ=0.8503.

As described above, when “cos θ” is used, if the two blocks have a high similarity, the value approaches “1.0”, and as the two blocks have a lower similarity, the values approach “0.0”. Therefore, the position of the block of interest of the previous frame image shown in FIG. 5C is in the vicinity in the current frame image shown in FIG. 5B, and the position with the highest similarity is the block of interest shown in FIG. 5C. Is the moved position.

For example, in the example of FIG. 5B and FIG. 5C, the movement distance is “−3” in the x-axis direction and “+4” in the y-axis direction, so the optical flow is (−3, 4).

Next, the motion information determination unit 16 performs a principal component analysis on the above-described motion information vector and compares it with the dictionary data of the person region motion dictionary 22 using the subspace method shown in FIG. To calculate. In the example of FIG. 5D, one shows the input subspace and the other shows the subspace of the dictionary data.

Here, the smaller the angle between the subspaces, the higher the degree of similarity. Therefore, for example, when the threshold is set to “10°” and the angle between the subspaces is less than 10°, the motion information determination unit 16 determines that the calculated motion information is a human motion. However, when the angle between the subspaces is 10° or more, it is determined that the calculated motion information is not a real object (photograph).

The above-mentioned similarity uses a canonical angle which is an angle between subspaces. For example, it is possible to define n (m≧n) between the m-dimensional subspace P and the n-dimensional subspace Q. Further, the canonical angles (n canonical angles θi) are calculated using, for example, Equations 2 and 3.

The degree of similarity is calculated by using the above canonical angle and Expression 4.

Here, a case will be described in which, for example, the basis vector 1 in the input subspace is represented by Equation 5, the basis vector 2 is represented by Equation 6, the basis vector 1 in the dictionary data subspace is represented by Equation 7, and the basis vector 2 is represented by Equation 8.

In the above-mentioned case, when calculated using the basis vector 2 in the input subspace and the basis vector 1 in the subspace of the dictionary data, “cos ² θ ₁ =0.9412” is obtained. Further, when calculated using the basis vector 1 in the input subspace and the basis vector 2 in the subspace of the dictionary data, “cos ² θ ₁ =0.0917” is obtained. From this, the degree of similarity is calculated as “0.5165”.

When it is determined that the degree of similarity calculated as described above is equal to or greater than a predetermined threshold value, the motion information determination unit 16 determines that the person in the person area of the current frame image is a real person.

In the above example, the human motion is judged from the similarity to the dictionary data. However, if the non-human motion is divided into several classes and learned, it is determined whether the human motion is a human motion according to the classified class. You may decide.

<Real judgment processing based on motion information of face area>
FIG. 6 is a flowchart showing an example of the actual object determination process based on the motion information of the face area according to the first embodiment. In the example of FIG. 6, the face area is extracted by detecting the face of a person, and it is determined whether the target of the determination captured by the camera 35 is a real object (for example, a photograph) by using the motion information of the face area. In other words, it is determined based on the motion information of the face area whether or not the determination target imaged by the camera 35 is a living body that satisfies a predetermined condition.
The process of FIG. 6 is different from the process of FIG. 3 in that a background image is not acquired in advance.

As shown in FIG. 6, when the camera image receiving unit 11 acquires an image (S20), the image processing apparatus 10 determines whether the face area detecting unit 13 has detected a face area (S21). When the image processing apparatus 10 determines that the face area is detected by the face area detection unit 13 (YES in S21), the detected face area is cut out (S22). If the face area detection unit 13 determines that the face area is not detected (NO in S21), the process returns to S20.

Next, the image processing device 10 determines whether or not there was a face area in the previous frame image (S23). When the image processing apparatus 10 determines that the face area was present in the previous frame image (YES in S23), the inter-image correlation calculation unit 17 or the flow histogram calculation unit 18 calculates motion information (S24). If the image processing apparatus 10 determines that there is no face area in the previous frame image (NO in S23), the process returns to S20.

Next, the image processing apparatus 10 determines whether the person in the face area detected from the current frame image by the motion information determination unit 16 is a real person by using the motion information calculated in the process of S24 (S25). In the process of S25, the motion information determination unit 16 calculates the similarity between the motion information calculated in the process of S24 and the dictionary data stored in the face area motion dictionary 23 to determine whether the motion information is a real product.

When the motion information determination unit 16 determines that the similarity between the motion information calculated in S24 and the dictionary data of the face area motion dictionary 23 is equal to or more than a predetermined threshold value (YES in S25), The person in the face area of the current frame image is determined as a real person (S26). In other words, the motion information determination unit 16 determines that the imaged determination target is a living body that satisfies a predetermined condition. If it is determined that the motion information determination unit 16 is less than the predetermined threshold value (NO in S25), the motion information determination unit 16 determines that the person in the face area of the current frame image is a photograph (S27), and ends the process.

In the case of detecting the face area described above, the processing can be performed without being affected by the flicker of the entire image due to, for example, illumination or the like, or the influence of the texture such as the background or the non-background.

<Image until motion information is calculated from face area>
FIG. 7 is an image diagram until the motion information is calculated from the face area according to the first embodiment. FIG. 7A shows an example of an image obtained by the camera, and FIG. 7B shows an example of a detected face area. FIG. 7C is an image diagram of motion information calculated from a plurality of images of the face area.

The face area detection unit 13 detects the face area shown in FIG. 7B from the image shown in FIG. 7A by using, for example, template matching. Similarly to the calculation of the motion information in FIG. 5, the face area detection unit 13 holds a template of a face image that is held in advance (for example, 11 pixels×11 pixels in a grayscale image) and the image shown in FIG. For example, the inner product calculation is performed using the respective brightness values (256 gradations) of 22 pixels×37 pixels).

The face area detection unit 13 calculates the angle (cos θ) by the inner product of two matrices obtained from the above-described face image template and the image shown in FIG. The angle is the similarity. For example, “a” in Expression 1 described above indicates each search area in the image shown in FIG. 7A, and, for example, “b” in Expression 1 is a template of a face image.

As described above, when “cos θ” is used, the closer the similarity is to “1.0”, and the lower the similarity is to “0.0”. Therefore, the face area detecting unit 13 sets in advance. An area equal to or larger than the threshold value is detected as a face image.

The local peak may be calculated in consideration of the case where a plurality of face images are present in the image shown in FIG. It is also possible to use another face detection method such as the face detection based on the Haar-like feature described above.

The motion information calculated from the image of the face area shown in FIG. 7B (the image cut out from the face area detected by the face area detecting unit 13) and the image of the face area in the previous frame image is used. Thus, the optical flow shown in FIG. 7C can be obtained.

<When the background area is less than the threshold>
FIG. 8 is a flowchart showing an example of an actual object determination process when the background area according to the first embodiment is less than the threshold value. In the example of FIG. 8, when the human area is detected by the process shown in FIG. 3, when the ratio of the background area is less than the threshold value, the motion information is calculated not from the human area but from the entire area.

As shown in FIG. 8, when the image processing apparatus 10 acquires an image in advance in a situation where there is no person (S30), the image processing apparatus 10 sets the image as a background image. Further, when the image processing apparatus 10 acquires an image by the camera image reception unit 11 (S31), the background image acquired in advance by the person area detection unit 12 is compared with the image acquired by the process of S31, and a difference is obtained. Determine if there is.

When there is a difference as described above, the image processing apparatus 10 determines whether the background area ratio determination unit 14 determines that the background area is less than the threshold value set by the background area ratio determination threshold value setting unit 15 (S32). When the background area ratio determination unit 14 determines that the background area is less than the threshold value (YES in S32), the image processing apparatus 10 sets the entire image as the person area (S33). If the background area ratio determination unit 14 determines that the background area is not less than the threshold value (NO in S32), the image processing apparatus 10 returns to the processing of S31.

Next, the image processing apparatus 10 determines whether or not there was a person area in the previous frame image (S34). Since the processing of S34 to S38 is the same as the processing of S14 to S18 shown in FIG. 3, detailed description thereof will be omitted here.

<Image until the operation information is calculated when the background area is less than the threshold>
FIG. 9 is an image diagram when the background area according to the first embodiment is less than the threshold value. 9A shows an example of an image obtained by the camera, and FIG. 9B shows an example of a detected person (face) region. FIG. 9C is a diagram for explaining the ratio between the person area and the non-person area (background area). FIG. 9D shows an image diagram of the motion information calculated from the entire plurality of images.

The person area detecting unit 12 and the face area detecting unit 13 detect the person (face) area shown in FIG. 9B from the image shown in FIG. 9A by background difference, time series difference, face detection, and the like. The background area ratio determination unit 14 obtains the total pixels of the person area and the total pixels of the non-person area (background area). In the example of FIG. 9C, for example, when the total pixel of the person area is “500” and the total pixel of the non-person area is “70”, the background area ratio determination unit 14 determines 500/570=0. Therefore, the ratio of the person area is calculated as “87.7%”.

If the threshold value of the human area set by the background area ratio determination threshold setting unit 15 is “75%”, the background area ratio determination unit 14 determines that the ratio of the human area is “87. Since it is “7%”, it is determined that the threshold value is exceeded. In this way, for example, when the ratio of the person area exceeds the threshold value, it is determined that the influence of the non-person area is small, and in the example of FIG. 9C, the motion information is calculated from the entire image. That is, the motion information can be calculated without being aware of the person area or the background area.

It is possible to obtain the optical flow shown in FIG. 9D by using the motion information calculated from the image shown in FIG. 9A and the previous frame image.

<Including outlier removal>
FIG. 10 is a flowchart showing an example of the actual object determination process including the outlier value removal according to the first embodiment. In the example of FIG. 10, the outlier value is removed when the motion information of the optical flow is calculated by the above-described processing. In the example of FIG. 10, it is possible to calculate more accurate motion information by removing the outlier. Note that the processes of S40 to S44 and S48 to S50 shown in FIG. 10 are the same as the processes of S30 to S34 and S36 to S38 shown in FIG. 8, so detailed description thereof will be omitted here.

After the processing of S44 shown in FIG. 10, when the image processing apparatus 10 calculates the optical flow using the current frame image and the previous frame image by the flow histogram calculation unit 18 (S45), the deviation value removal unit 19 causes the deviation. The value is removed (S46). When the deviation value is removed in the process of S46, the image processing apparatus 10 calculates a flow histogram (S47), and proceeds to the process of S48.

<About removal of outliers>
FIG. 11 is a diagram for explaining outlier removal according to the first embodiment. FIG. 11A shows an example of a normal least squares method, and FIG. 11B shows an example in which an outlier value is considered. If the flow histogram calculation unit 18 uses the sum of squares (least squares) of the difference between pixel values when calculating the motion information using the optical flow based on the block matching described above, the accuracy decreases due to the outlier. Therefore, in the image processing apparatus 10, the outlier value removing unit 19 removes the outlier value.

11A and 11B, the horizontal axis represents the x coordinate and the vertical axis represents the y coordinate. The plotted points shown in FIGS. 11A and 11B represent pixel values of the matching target area (current frame image), and the straight lines shown in FIGS. 11A and 11B. Indicates a matching target area (previous frame image).

The smaller the sum of squares of the differences between the pixel values of the matching target area (current frame image) and the matching target area (previous frame image), the higher the degree of similarity, that is, the higher the possibility of moving to that area. In the examples of FIGS. 11A and 11B, when the sum of squares of the difference between the pixel values is small, the points plotted on the straight line are arranged. On the other hand, when some pixels have a unique value (outlier) due to the influence of changes in illumination, etc., the value of the sum of least squares becomes large due to the influence of the value. Correct results will not be obtained.

Therefore, as shown in FIG. 11B, the outlier value removing unit 19 removes the peculiar value (outlier value) from the calculation target of the least square sum. In the example of FIG. 11B, the plotted points of black circles are outliers. When the corresponding outliers are excluded as shown in FIG. 11B, the plotted points are arranged on a straight line, and the similarity of the matching target is high.

By removing the eccentricity value in this manner, it is possible to improve the optical flow calculation accuracy in the calculation of the motion information, and it is possible to improve the detection accuracy in the above-described actual object determination.

<Actual judgment process for normalizing the direction of people>
FIG. 12 is a flow chart showing an example of an actual object determination process for normalizing the orientation of a person according to the first embodiment. In the example of FIG. 12, for example, after detecting a person (face) area, the orientation of the person (face) is normalized to calculate motion information. In the example of FIG. 12, by normalizing the direction of the person, more accurate motion information can be calculated. Note that the processes of S60 to S63 and S65 to S69 shown in FIG. 12 are the same as the processes of S30 to S33 and S34 to S38 shown in FIG. 8, and thus detailed description thereof is omitted here.

After the processing of S63 shown in FIG. 12, the image processing apparatus 10 normalizes the orientation of the person by the image orientation normalization unit 20 (S64). As described above, the image orientation normalization unit 20 detects a feature point designated as a target in advance from within the image subregion, maps it to a three-dimensional model prepared in advance, and turns it to face the front of the person. Normalize orientation. When the orientation of the person is normalized in the process of S64, the image processing apparatus 10 proceeds to the process of S65.

<Image diagram to normalize the direction of people>
FIG. 13 is an image diagram for normalizing the orientation of a person according to the first embodiment. 13A shows an example of an image obtained by a camera, and FIG. 13B shows an example of an image in which the orientation of a person (face) is corrected.

When the image orientation normalization unit 20 obtains the image shown in FIG. 13A, the orientation of the person (face) is detected by detecting the specified feature points in the image area. In the example of FIG. 13A, for example, it is detected that the direction of the person (face) or body is facing right.

The image orientation normalization unit 20 also maps the feature points detected from the image area shown in FIG. 13A to a three-dimensional model, and normalizes them by directing them to the front as shown in FIG. 13B. ..

By normalizing the image orientation in this way, it is possible to improve the calculation accuracy of the motion information, and it is possible to improve the detection accuracy in the above-described actual object determination.

<Including update of dictionary data>
FIG. 14 is a flowchart showing an example of an actual object determination process including the update of dictionary data according to the first embodiment. In the example of FIG. 14, the dictionary data is updated with the operation information used in the actual product determination process when a user's operation such as input of a password is performed after the actual product determination process. Since the processing of S70 to S78 shown in FIG. 14 is the same as the processing of S10 to 18 shown in FIG. 3, detailed description thereof will be omitted.

After the processing of S78 shown in FIG. 14, the image processing apparatus 10 determines whether or not the user specifying operation has been performed by the dictionary updating unit 21 (S79). In the process of S79, the dictionary updating unit 21 determines whether or not an operation for identifying the user, such as a password input by the user, has been performed.

The image processing apparatus 10 updates the dictionary data (S80) when the dictionary updating unit 21 determines that the user specifying operation has been performed (YES in S79). In the process of S80, the dictionary updating unit 21 updates the motion information calculated in the process of S75, for example, by adding it to the dictionary data of the person area motion dictionary 22. If the dictionary updating unit 21 determines that the user specifying operation has not been performed (NO in S79), the image processing apparatus 10 ends the process.

<Operation information used for updating dictionary data>
FIG. 15 is a diagram illustrating operation information used in updating the dictionary data according to the first embodiment. The arrow shown in FIG. 15 indicates operation information in the case of optical flow.

When the user specifying operation is performed after it is determined that the person in the image is the real person, the dictionary updating unit 21 determines that the person in the image is the real person as shown in FIG. The dictionary data is updated by adding the motion information used when the determination is made to the person area motion dictionary 22. Note that the dictionary updating unit 21 stores in the face area motion dictionary 23 when the motion information used when the person in the image is determined to be a real person is, for example, the motion information calculated from the face area. Update the dictionary data by adding.

By successively updating the dictionary data in this manner, it is possible to create dictionary data according to the usage environment of the user, and it is possible to improve the detection accuracy in the above-described actual object determination.

According to the above-described embodiment, the motion information is calculated from the region corresponding to the person, and the motion information is used to determine whether the person in the image is a real person, thereby reducing the local area of the face such as the eyes and mouth. It is possible to make a more appropriate judgment than to make a judgment only by detecting a partial area, and it is possible to improve the detection accuracy of a person.

Although the above-described example describes the embodiment for the purpose of face recognition or the like, the scope of application is not limited to this. For example, it is possible to determine whether a picture is included in the images taken by the camera.

This makes it possible, for example, to determine that a photograph has been captured by the camera, perform image conversion on that area, and capture it. For example, if you want to capture photos one by one, you can correct the distortion by affine transformation etc. by repeating the work of holding the photos over the camera without scanning them one by one, and capture them in a normalized state. It will be possible. Note that the timing of performing the conversion process and capturing the photographic data is the case where it is determined that the photograph has been captured by the camera (the actual person has not been photographed) according to the above-described embodiment.

Next, an example of the authentication method of the image forming apparatus to which the above-described determination method of the real thing is applied will be described.

<System configuration>
FIG. 16 is a diagram showing a configuration example of the information processing system according to the first embodiment. In FIG. 16, the information processing system 100 includes an image forming apparatus 101 capable of communicating with an RFID (Radio Frequency Identifier) tag 104 possessed by a user 105.

The image forming apparatus 101 is an image forming apparatus such as an MFP (Multifunction Peripheral) having a printer, a scanner, a copy function, a facsimile function, or a printer, and is an example of an information processing apparatus that authenticates a user.

The image forming apparatus 101 includes an RF tag reader 103 that receives predetermined information such as a tag ID from an RFID tag 104 within a predetermined range, and a camera 102 that captures an image.

RFID is a technique for performing near field communication using an electromagnetic field, radio waves, or the like between the RFID tag 104, which stores predetermined information such as a tag ID, and the RF tag reader 103, and is an example of near field communication. Is.

For example, when the RFID tag 104 is a passive tag, the RF tag reader 103 emits a predetermined radio wave in the detection range (predetermined range) of the RF tag reader 103 described later. Further, when the RFID tag 104 receives a radio wave radiated from the RF tag reader 103, it operates by using the received radio wave as electric power and transmits predetermined information such as a tag ID stored in advance to the RF tag reader 103.

The passive tag is an RFID tag 104 that operates using radio waves from the RF tag reader 103 as an energy source, and does not need to have a battery built therein. The antenna of the passive tag reflects a part of the radio wave from the RF tag reader 103, and information such as the tag ID is returned on the reflected wave. Since the intensity of this reflection is very small, the passive tag has a shorter communication distance than an active tag that transmits radio waves with the power of its own device, but is inexpensive and operates almost permanently.

The signal transmission method of the RFID tag 104 is classified into an electromagnetic induction method and a radio wave method. The electromagnetic induction method is a method in which the coil of the RFID tag 104 and the antenna coil of the RF tag reader 103 are magnetically coupled to transmit an energy/signal. The electromagnetic induction method can transfer energy more efficiently than the electric wave method, so the development has advanced. For example, Felica (registered trademark) uses this electromagnetic induction method. In the electromagnetic induction method, for example, frequencies of 130 to 135 kHz and 13.56 MHz are used. In addition, the communicable distance of the radio wave induction method is within 1 m, for example.

On the other hand, the radio wave system is a system in which radio waves are exchanged between the antenna of the RFID tag 104 and the antenna of the RF tag reader 103 to transmit energy/signal. The radio wave system radiates radio waves to the space and transmits the radio waves, so that it is possible to communicate with the RFID tag 104 located farther than the electromagnetic induction system. In the radio wave system, for example, frequencies such as 433 MHz, 900 MHz band, and 2.45 GHz are used. Note that the communicable distance of the radio wave method is about 3 to 5 m for a passive tag. The active tag is capable of communication of, for example, about several km as long as output power is allowed.

The camera 102 is, for example, an imaging device installed such that an image captured by the camera 102 includes a user who is using the image forming apparatus 101.

FIG. 17 is a diagram showing an example of the imaging range of the camera and the detection range of each sensor according to the embodiment. FIG. 17 shows an example of an imaging range 251 of the camera 102, a detection range 252 of the RF tag reader 103, and a detection range 253 of a mobile body sensor unit described later.

In FIG. 17, the detection range 252 of the RF tag reader 103 when viewed from the upper surface of the image forming apparatus 101 is shown. In the example of FIG. 17, the image forming apparatus 101 has, for example, a patch antenna or the like inside the image forming apparatus 101, and has a fan shape (for example, in the downward direction in FIG. 17) on the front surface of the image forming apparatus 101. A detection range 252 having an angle of 150° is formed.

The RF tag reader 103 emits a predetermined radio wave to the detection range 252 of the RF tag reader 103 when the user 105 is detected by the moving body sensor unit in the detection range 253 of the moving body sensor unit. Thereby, for example, when the user 105 having the RFID tag 104 approaches a predetermined distance (for example, within 3 m) from the front surface of the image forming apparatus 101, predetermined information (for example, tag ID) stored in the RFID tag 104. Is automatically transmitted to the image forming apparatus 101.

In the present embodiment, the image forming apparatus 101 stores the information of the user registered in advance, and based on the tag ID received from the RFID tag 104 and the information of the user registered in advance, the RFID tag. Authentication of 104 (hereinafter referred to as wireless authentication) is performed.

Further, the camera 102 is activated (operated) when the user is detected by the moving body sensor unit in the detection range 253 of the moving body sensor unit, and starts imaging.

The image forming apparatus 101, for example, based on the face image of the user included in the image captured by the camera 102 and the face feature information of the user registered in advance, face authentication of the user captured in the image. (Hereinafter referred to as image authentication).

Further, the image forming apparatus 101 determines whether the person detected from the image captured by the camera 102 is a real (living body) or a photograph (non-living body) by the above-described real-life determination processing. In other words, the image forming apparatus 101 determines whether the determination target captured by the camera 102 is a living body that satisfies a predetermined condition. It should be noted that the determination as to whether the real object (living body) is a photograph (non-living body) by this determination processing can be performed as long as it can be determined that the person detected from the image is not a spoofed non-living body such as a photograph. .. Impersonation is performed by a person using a photograph or the like.

In the present embodiment, when it is determined that the object is the “real object” by the actual object determination process described above, that is, when it is determined that the person detected from the image is not impersonated by a non-living body such as a photograph, it is detected from the image. It is assumed that the person who is said to be "living body".

In addition, when it is determined as a “photograph” by the actual determination process described above, that is, when the person detected from the image is determined to be a spoofed non-living body such as a photograph, the person detected from the image Is expressed as "non-living body".

In the image forming apparatus 101, the user of the RFID tag 104 authorized by the wireless authentication and the user authorized by the image authentication are the same user, and the person imaged in the image is the living body. When it is determined (that is, when it is determined that the imaged determination target is a living body that satisfies a predetermined condition), the user is permitted to use the image forming apparatus 101.

According to the image forming apparatus 101 according to the present embodiment, the user 105 of the image forming apparatus 101 can improve the accuracy of image authentication such as face authentication only by having the RFID tag 104. Further, according to the image forming apparatus 101 according to the present embodiment, it is possible to reduce spoofing caused by a user's facial photograph or the like.

Note that the system configuration shown in FIG. 16 is merely an example. For example, the image forming apparatus 101 may be various information processing apparatuses such as a PC, a tablet terminal, a smartphone, a game machine, a video conference apparatus, an electronic blackboard, and a projector that authenticate the user. Further, the camera 102, the RF tag reader 103, and the like may be externally attached.

Further, the RFID tag 104 may be an RFID active tag, a semi-active tag, or the like. The RFID tag 104 is an example of a wireless tag. The wireless tag may be a short-range wireless system different from RFID, for example, a Bluetooth (registered trademark) Low Energy (hereinafter referred to as BLE) or a wireless terminal that performs communication by NFC (Near Field Communication) or the like. Further, instead of the RFID tag 104, a mobile terminal equipped with a wireless tag or the like may be used.

Further, the image authentication is various authentication methods such as fingerprint authentication, iris authentication, vein authentication, etc. for authenticating the user by using the image of the user and the characteristic information about the user's living body. Is also good.

<Hardware configuration>
(Hardware configuration of image forming apparatus)
FIG. 18 is a diagram illustrating an example of the hardware configuration of the image forming apparatus 101 according to the first embodiment. The image forming apparatus 101 includes a main body 310 that realizes various image forming functions such as a copy function, a scanner function, a fax function, and a printer function, and an operation unit 320 that receives a user operation. Note that accepting a user operation is a concept including accepting information (including a signal indicating a coordinate value of a screen) input according to a user operation. The main body 310 and the operation unit 320 are communicably connected to each other via a dedicated communication path 330. As the communication path 330, for example, a USB (Universal Serial Bus) standard can be used, but any standard may be used regardless of whether it is wired or wireless.

The main body 310 can perform an operation according to the operation received by the operation unit 320. The main body 310 can also communicate with an external device such as a client PC (personal computer), and can also perform an operation in accordance with an instruction received from the external device.

First, the hardware configuration of the main body 310 will be described. As shown in FIG. 18, the main body 310 includes a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312, a RAM (Random Access Memory) 313, a storage unit 314, a communication I/F (Interface) unit 315, and a connection. It has an I/F unit 316, an engine unit 317, a moving body sensor unit 318, a system bus 319, and the like.

The CPU 311 centrally controls the operation of the main body 310. The CPU 311 controls the overall operation of the main body 310 by executing a program stored in the ROM 312, the storage unit 314, or the like, using the RAM 313 as a work area (work area). For example, the CPU 311 uses the engine unit 317 to realize various functions such as the copy function, the scanner function, the fax function, and the printer function described above.

The ROM 312 is a non-volatile memory that stores, for example, a BIOS (Basic Input/Output System) executed when the main body 310 is started up and various settings. The RAM 313 is a volatile memory used as a work area of the CPU 311 or the like. The storage unit 314 is, for example, a nonvolatile storage device that stores an OS (Operating System), application programs, various data, and the like, and is configured by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. It

The communication I/F unit 315 is a network interface for connecting the main body 310 to the network 340 and performing communication with an external device connected to the network 340. The connection I/F unit 316 is an interface for communicating with the operation unit 320 via the communication path 330.

The engine unit 317 is hardware that performs general-purpose information processing and processing other than communication for realizing functions such as a copy function, a scanner function, a fax function, and a printer function. The engine unit 317 includes, for example, a scanner (image reading unit) that scans and reads an image of a document, a plotter (image forming unit) that prints on a sheet material such as paper, a fax unit that performs fax communication, and the like. .. Further, the engine unit 317 may include a specific option such as a finisher for sorting printed sheet materials and an ADF (automatic document feeder) for automatically feeding a document.

The moving body sensor unit 318 is a sensor that detects a moving body within a detection range around the image forming apparatus 101, and for example, a pyroelectric sensor or the like is used.

The system bus 319 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

Next, the hardware configuration of the operation unit 320 will be described. As illustrated in FIG. 18, the operation unit 320 includes a CPU 321, a ROM 322, a RAM 323, a flash memory unit 324, a communication I/F unit 325, an operation panel unit 326, a connection I/F unit 327, and an external connection I/F unit 328. And a short-range wireless unit 329, a camera unit 332, a system bus 331, and the like.

The CPU 321 centrally controls the operation of the operation unit 320. The CPU 321 controls the overall operation of the operation unit 320 by executing a program stored in the ROM 322, the flash memory unit 324, or the like using the RAM 323 as a work area (work area). For example, the CPU 321 realizes various functions such as displaying information (image) according to the input received from the user.

The ROM 322 is, for example, a non-volatile memory that stores BIOS executed when the operation unit 320 is started up, various settings, and the like. The RAM 323 is a volatile memory used as a work area or the like of the CPU 321. The flash memory unit 324 is, for example, a non-volatile storage device that stores an OS, application programs, various data, and the like.

The communication I/F unit 325 is a network interface for connecting the operation unit 320 to the network 340 and performing communication with an external device connected to the network 340.

The operation panel unit 326 accepts various inputs according to the user's operation and displays various information (for example, information according to the accepted operation, information indicating the operation status of the image forming apparatus 101, setting status, etc.). .. The operation panel unit 326 is composed of, for example, a liquid crystal display (LCD) having a touch panel function, but is not limited to this. The operation panel unit 326 may be composed of, for example, an organic EL (Electro Luminescence) display device having a touch panel function. Further, the operation panel unit 326 may be provided with an operation unit such as a hardware key or a display unit such as a lamp in addition to or instead of this.

The connection I/F unit 327 is an interface for communicating with the main body 310 via the communication path 330. The external connection I/F unit 328 is an interface such as a USB for connecting an external device.

The near field wireless unit 329 is a near field wireless device for communicating with a wireless tag within a predetermined communication range by near field wireless communication. For example, the short-range wireless unit 329 includes the RF tag reader 103 of FIG. 16 and the like, and includes a passive RFID tag that returns a response to a radio wave transmitted from the short-range wireless unit 329, or an active RFID that transmits a radio wave from its own device. Predetermined information is received from a tag or the like. The short-range wireless unit 329 may be a short-range wireless device that performs short-range wireless communication such as BLE and NFC described above.

The camera unit 332 is an imaging device that captures an image within the imaging range.

The system bus 331 is connected to each component described above and transmits an address signal, a data signal, various control signals, and the like.

<Functional configuration>
FIG. 19 is a diagram illustrating an example of the functional configuration of the image forming apparatus according to the first embodiment. The image forming apparatus 101 realizes the functional configuration shown in FIG. 19 by executing a predetermined program by at least one CPU of the CPU 311 and the CPU 321 of FIG.

Note that the functional configuration shown in FIG. 19 is an example. For example, at least some of the components included in the operation unit 320 may be included in the main body 310, and at least some of the components included in the main unit 310 may be included in the operation unit 320. May be included.

(Functional structure of the main unit)
The main body 310 of the image forming apparatus 101 includes a moving body detection unit 401, a state control unit 402, an image forming unit 403, a storage unit 404, a communication unit 405, and the like.

A moving body detection unit (moving body detection unit) 401 is a unit that detects a moving body (for example, a person) around the image forming apparatus 101 using the moving body sensor unit 318, and for example, the CPU 311 (FIG. 18). Alternatively, it is realized by a program operating in the CPU 321). For example, when a moving body is detected in the detection range 253 of the moving body sensor unit 318, the moving body detection unit 401 notifies the state control unit 402 and the like that the moving body is detected.

The state control unit 402 is a unit that controls the power state of the operation unit 320 and the main body 310, and is realized by, for example, a program operated by the CPU 311 (or CPU 321) in FIG. 18. For example, if the state control unit 402 has not used the image forming apparatus 101 for more than a preset time, the state control unit 402 consumes less power than the normal state in which the image forming process can be executed. Shift to the state. In the power saving state, the image forming apparatus 101 can reduce power consumption by, for example, stopping the functions of the operation unit 320, the engine unit 317 of the main body 310, and the storage unit 314.

Further, when the state control unit 402 according to the present embodiment receives a notification indicating that a moving body is detected from the moving body detection unit 401, for example, while the image forming apparatus 101 is in the power saving state, the operation unit 320. Is returned from the power saving state to a normal state in which normal operation is possible. For example, the state control unit 402 returns the operation unit 320 from the power saving state to the normal state by notifying the operation unit 320 of the return instruction via the communication unit 405.

The image forming unit 403 is a unit that executes various image forming functions (for example, printing, copying, scanning, and faxing) of the image forming apparatus 101. For example, the engine unit 317 of FIG. 18 and the CPU 311 of FIG. It is realized by a program or the like that operates in.

The storage unit (storage unit) 404 stores various information including user information A 406, which is information of the user of the image forming apparatus 101 registered in advance. The storage unit 404 is realized by, for example, the storage unit 314 of FIG. 18, the RAM 313, and a program operating by the CPU 311 of FIG.

The communication unit 405 is means for communicating with the operation unit 320, and is realized by, for example, the connection I/F unit 316 in FIG. 18 and a program operating in the CPU 311 in FIG.

(Functional configuration of the operation unit)
The operation unit 320 of the image forming apparatus 101 includes, for example, a wireless communication unit 411, a wireless authentication unit 412, a transmission control unit 413, an image capturing unit 414, an image authentication unit 415, a face area detection unit 416, a characteristic information extraction unit 417, and image determination. It has a unit 418, a device authentication unit 419, a password authentication unit 420, a noise determination unit 421, a hierarchy determination unit 422, a timer management unit 423, a communication unit 424, a user information management unit 425, a storage unit 426, and the like.

The wireless communication unit (wireless communication unit) 411 is, for example, a unit that receives predetermined information from a wireless tag by short-range wireless communication using the short-range wireless unit 329 of FIG. Alternatively, it is realized by a program operating in the CPU 311). For example, the wireless communication unit 411 receives predetermined information such as the tag ID stored in the RFID tag 104 from the RFID tag 104 within the detection range 252 of the RF tag reader 103 in FIG.

The wireless authentication unit (wireless authentication unit) 412 is a unit that authenticates the wireless tag that has transmitted the predetermined information based on the predetermined information received by the wireless communication unit 411 and the user information registered in advance. The wireless authentication unit 412 is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG. The wireless authentication unit 412 identifies the RFID tag 104 based on the tag ID of the RFID tag 104 received by the wireless communication unit 411 and the user information registered in advance (for example, user information a431, user information A406, etc.). Certify.

For example, the wireless authentication unit 412 permits authentication of the RFID tag 104 when the tag ID of the RFID tag 104 received by the wireless communication unit 411 is registered in the user information in advance. In addition, the wireless authentication unit 412 outputs information (for example, a user number described later) for identifying the user of the RFID tag 104 authorized for authentication to the device authentication unit 419, the storage unit 426, and the like.

The transmission control unit 413 is a unit that controls the transmission of radio waves by the wireless communication unit 411, and is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG. The transmission control unit 413 controls whether or not the RF tag reader 103 of FIG. 16 transmits a predetermined radio wave to be transmitted to the detection range 252 of the RF tag reader 103.

For example, when the moving body detection unit 401 of the main body 310 detects a moving body in the detection range 253 of the moving body sensor unit 318, the transmission control unit 413 causes the wireless communication unit 411 to transmit radio waves. Further, the transmission control unit 413 causes the wireless communication unit 411 to stop the transmission of radio waves when the moving body detection unit 401 of the main body 310 does not detect a moving body around the image forming apparatus 101.

The image capturing unit (image capturing unit) 414 is, for example, a unit that captures an image of the periphery of the image forming apparatus 101 using the camera unit 332 in FIG. 18, and, for example, by a program operating in the CPU 321 (or CPU 311) in FIG. Will be realized. For example, the image capturing unit 414 captures an image of the user in front of the image forming apparatus 101 by capturing an image in front of the image forming apparatus 101.

The image authentication unit (image authentication unit) 415 authenticates the user included in the image captured by the image capturing unit 414 based on the image captured by the image capturing unit 414 and the user information registered in advance. The image authentication unit 415 is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG. The image authentication unit 415 uses the image captured by the image capturing unit 414 to extract the feature information of the user's face extracted by the feature information extracting unit 417 and the user information registered in advance (for example, the user information a431 and the user information. A406, etc.), and the face authentication of the user included in the image is performed.

For example, when the feature information corresponding to the face feature information extracted by the feature information extraction unit 417 is included in any of the user information registered in advance, the image authentication unit 415 authenticates the user included in the image. to approve. Further, the image authentication unit 415 outputs information (for example, user information described later) for identifying the user who is permitted to be authenticated, to the device authentication unit 419, the storage unit 426 and the like.

In the present embodiment, the image authentication unit 415 is assumed to perform face authentication of the user using a known face authentication technique (see, for example, Patent Document 1). The image authentication unit 415 also performs various known image authentication methods for authenticating the user based on the characteristic information regarding the user's living body included in the image captured by the image capturing unit 414, such as fingerprint authentication and iris authentication. The user may be authenticated by vein authentication or the like.

The face area detection unit 416 is means for detecting a face area from the image captured by the image capturing unit 414, and is realized by, for example, a program operating by the CPU 321 (or CPU 311) in FIG. 18.

The feature information extraction unit 417 is a unit that extracts face feature information from the face image when the image captured by the imaging unit 414 includes a face image, and operates, for example, by the CPU 321 (or CPU 311) in FIG. 18. It is realized by a program. The face feature information includes, for example, information such as the contour of the face, the shape and relative position of each part such as the eyes, nose, chin, and cheekbones. The face feature information is an example of the biometric feature information.

The image determination unit (image determination unit) 418 determines whether the person included in the image captured by the image capturing unit 414 is a living body (real object) or a non-living body (photograph) by the above-described actual object determination processing. In other words, the image determination unit 418 determines whether the determination target included in the image captured by the imaging unit 414 is a living body that satisfies a predetermined condition. The image determination unit 418 includes, for example, the functional configuration shown in FIG. 1 and is realized by a program that operates in the CPU 321 (or CPU 311) in FIG. For example, when the image determination unit 418 determines “actual” by the actual determination process described in FIGS. 3, 6, 8, 10, 12, and 14 of the first embodiment, the person captured in the image. Is determined to be a living body.

The device authentication unit (device authentication means) 419 determines whether the information of the user authorized by the wireless authentication unit 412, the information of the user authorized by the image authentication unit 415, and the photograph by the image determination unit 418. Authenticate the user based on whether it is a real product or not. The device authentication unit 419 is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG.

For example, in the device authentication unit 419, the user of the wireless tag authorized by the wireless authentication unit 412 and the user authorized by the image authentication unit 415 are the same user, and the person captured in the image. If is judged to be a living body, the user is allowed to log in. A user who is permitted to log in can use various image forming functions of the image forming apparatus 101.

As a preferred example, the device authentication unit 419 includes the identification information (user information) of the user of the wireless tag output from the wireless authentication unit 412 and the identification information of the user (user of the user output from the image authentication unit 415. If the information and) match, it is determined that the users are the same.

It should be noted that “when the two pieces of identification information match” includes not only the case where the two pieces of identification information completely match but also the case where the two pieces of identification information can be determined to be substantially the same user identification information. .. For example, the user identification information obtained from the image authentication unit 415 is an 8-digit employee ID, and the user identification information obtained from the wireless authentication unit 412 is a 10-digit login in which 2 characters are added to the employee ID. In the case of ID, it is determined that the two pieces of identification information match.

In this way, the device authentication unit 419 is the same when the user identification information obtained from the wireless authentication unit 412 and the user identification information obtained from the image authentication unit 415 are identification information regarding the same user. It may be determined to be the user.

The password authentication unit (password authentication unit) 420 is a unit that receives a password input by the user to the operation panel unit 326 of FIG. 18, for example, by a program that operates on the CPU 321 (or CPU 311) of FIG. Will be realized.

The noise determination unit (noise determination unit) 421 includes noise that affects the actual determination, such as a flicker of a fluorescent lamp, in the time-series camera image group used by the image determination unit 418 for the actual determination process. Determine if there is an image. The noise determination unit 421 discards an image including noise that affects the actual determination by the image determination unit 418, and waits for the next image to improve the accuracy of the actual determination. The noise determination unit 421 is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG.

For example, the noise determination unit 421 determines noise based on the difference amount between images. For example, the noise determination unit 421 obtains the difference (absolute value) between the luminance values of the pixels in the two images arranged in time series, and calculates the sum. When the calculated sum total exceeds the preset threshold value, the noise determination unit 421 determines that noise is present because the difference amount of the entire image is too large.

The layer determination unit 422 layers the actual determination processing by the image determination unit 418. Hierarchical means that the acquired characteristic information of the living body is finally judged as an image or a real body (living body) through a plurality of discriminators. The details of the hierarchy determination unit 422 will be described later in the second embodiment.

The timer management unit (timer management unit) 423 manages the valid period of the authentication result by the wireless authentication unit 412, the authentication result by the image authentication unit 415, the determination result by the image determination unit 418, etc. using a timer. The timer management unit 423 is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG.

For example, when a user carrying a wireless tag (RFID tag 104) passes in front of the image forming apparatus 101 and wireless authentication is permitted, if the authentication result remains forever, erroneous authentication or unauthorized authentication will occur. Can be a factor that causes. The timer management unit 423 deletes or initializes the authentication result by the wireless authentication unit 412, the authentication result by the image authentication unit 415, the determination result by the image determination unit 418, and the like after a predetermined time has passed, thereby erroneously authenticating. And reduce fraudulent authentication.

The communication unit 424 is a unit for communicating with the main body 310, and is realized by, for example, the connection I/F unit 327 in FIG. 18 and a program operating in the CPU 321 (or CPU 311) in FIG. The connection I/F unit 327 is supplied with power even when the operation unit 320 is in the power saving state, and the connection I/F unit 327 can receive the return request transmitted from the main body 310. I shall.

The user information management unit 425 is a unit that manages the user information a431 stored in the storage unit 426, and is realized by, for example, a program that operates in the CPU 321 (or CPU 311) in FIG.

The storage unit (storage unit) 426 is, for example, a unit that stores user information a431, biometric information 432 used by the hierarchy determination unit 422, non-biometric information 433, and the like. The storage unit 426 is realized by, for example, the flash memory unit 324 of FIG. 18, the RAM 323, and a program operating by the CPU 321 of FIG.

The storage unit 426 may store the person area motion dictionary 22 and the face area motion dictionary 23 of FIG. 1 included in the image determination unit 418.

(User information)
Here, an example of the user information a431 will be described.

FIG. 20 is a diagram showing an example of user information according to the first embodiment. The user information a431 is an example of user information registered in advance. In the example of FIG. 20, the user information a431 includes “user number”, “name”, “mail address”, “login ID”, “login password”, “wireless tag ID”, “face feature information”. Etc. is included.

The “user number” is, for example, an identification number (or identification information) unique to each user data, such as a serial number assigned when the information of each user is registered in the user information a431. , Is an example of identification information for identifying a user. The “user number” may be, for example, a user ID such as an employee ID.

"Name" is information on the name of the user. “Email address” is information on the user's email address. The “login ID” and the “login password” are examples of authentication information for the user to log in to the image forming apparatus 101.

The “wireless tag ID” is information of the tag ID (identification information) transmitted by the RFID tag 104 possessed by each user, and is represented by, for example, an 8-digit number. The "wireless tag ID" is an example of predetermined information transmitted from the wireless tag, and the predetermined information may include information other than numbers, for example, user identification information.

The “face characteristic information” is obtained by previously acquiring and storing information such as the contour of the face of the user of the image forming apparatus 101 and the shape and relative position of each part such as eyes, nose, chin, cheekbones, and the like. It is characteristic information about the user's face. The data format of the user characteristic information shown in FIG. 20 is an example, and the data format may be any format. The “face characteristic information” is an example of biological characteristic information.

The storage unit 426 of the operation unit 320 can store, for example, about 300 to 1800 user information items a431 as illustrated in FIG. For example, the user information management unit 425 of the operation unit 320 stores at least a part of the user information A406 stored in the storage unit 404 of the main body 310 in the storage unit 426 of the operation unit 320 as user information a431. I'll do it. As a result, the wireless authentication unit 412, the image authentication unit 415, the password authentication unit 420, and the like use the user information a431 stored in the storage unit 426 of the operation unit 320, and the usage stored in the storage unit 404 of the main body 310. Authentication can be performed faster than using the personal information A 406. The storage unit 426 of the operation unit 320 can read data, for example, without waiting for activation of the storage unit 314 (for example, HDD) that constitutes the storage unit 404 of the main body 310.

<Process flow>
Next, the flow of processing of the information processing method by the image forming apparatus (information processing apparatus) 101 will be described.

(Wireless authentication process)
FIG. 21 is a flowchart showing an example of the wireless authentication process according to the first embodiment.

In step S2101, when the wireless communication unit 411 receives the wireless tag ID (an example of predetermined information) from the wireless tag (RFID tag 104), the wireless authentication unit 412 executes the process of step S2102 and subsequent steps.

When the process proceeds to step S2102, the wireless authentication unit 412 refers to the user information (user information a431, user information A406) and acquires the user number corresponding to the wireless tag ID received by the wireless communication unit 411. .. For example, in the user information a431 illustrated in FIG. 20, when the wireless tag ID received by the wireless communication unit 411 is “00522213”, the wireless authentication unit 412 sets “” as the user number corresponding to the wireless tag ID “00522213”. 0001" is acquired.

In step S2103, the wireless authentication unit 412 performs processing depending on whether or not the user number corresponding to the wireless tag ID received by the wireless communication unit 411 can be acquired from the user information (user information a431, user information A406). Branch.

If the corresponding user number cannot be acquired, the wireless authentication unit 412 moves the process to step S2106, determines that wireless authentication is not permitted (wireless authentication NG), and ends the process. On the other hand, when the corresponding user number can be acquired, the wireless authentication unit 412 moves the process to step S2104.

Upon proceeding to step S2104, the wireless authentication unit 412 determines that wireless authentication is permitted (wireless authentication OK).

In step S2105, the wireless authentication unit 412 stores the wireless authentication result (OK) indicating that the wireless authentication is permitted and the user number corresponding to the wireless tag ID acquired from the user information in the device authentication unit 419. It is output to the unit 426 and the like.

Through the above process, the wireless authentication unit 412 permits wireless authentication when the user corresponding to the wireless tag ID received by the wireless communication unit 411 is registered in the user information, and uses the wireless authentication permitted. The user's user number. The user number is an example of information for identifying the user, and may be other information such as an employee ID and an email address.

(Image authentication process)
FIG. 22 is a flowchart showing an example of the image authentication processing according to the first embodiment.

In step S2201, the image capturing unit 414 captures an image using the camera unit 332. An example of an image captured at this time is shown in FIG.

In the captured image 2301 shown in FIG. 23A, a user 2302 in front of the image forming apparatus 101 is captured. In this way, the camera unit 332 is installed so that the user in front of the image forming apparatus 101 is included in the captured image 2301.

In step S2202, the face area detection unit 416 detects an image of the face area (hereinafter referred to as a face image) from the image captured in step S2201. An example of the face image detected at this time is shown in FIG. The face area detection unit 416 extracts parts such as the user's face outline 2304, eyes 2305, and nose 2306 from the captured image 2301 captured by the camera unit 332 using a known pattern matching technique or the like. , The face image 2303 of the user is detected.

In step S2203, the image authentication unit 415 determines whether a face image has been detected. If no face image is detected, the image authentication unit 415 returns the process to step S2201 and repeats the same process. On the other hand, when the face image is detected, the image authentication unit 415 shifts the processing to step S2204.

When the process moves to step S2204, the feature information extraction unit 417 extracts the face feature information from the face image 2303 detected in step S2202. For example, the feature information extraction unit 417 acquires information such as the contour of the face and the shape and relative position of each part such as the eyes, nose, chin, and cheekbones from the face image 2303.

In step S2205, the image authentication unit 415 acquires user information. For example, the image authentication unit 415 acquires the user information a431 stored in the storage unit 426.

In steps S2206 to S2210, the image authentication unit 415 compares the face characteristic information extracted in step S2204 with each of the “face characteristic information” included in the user information acquired in step S2205, thereby determining the face. Authenticate.

For example, in step S2206, “i” (i is an integer of 1 or more) is initialized to “1”.

In step S2207, the image authentication unit 415 determines whether or not the face feature information extracted in step S2204 and the “face feature information” corresponding to the user number “i” of the user information match. To do. The matching of the face feature information includes not only the case where the two face feature information completely match, but also the case where the similarity between the two feature information is equal to or more than a predetermined threshold value.

If the face characteristic information does not match, the image authentication unit 415 moves the process to step S2209. On the other hand, if the face feature information matches, the image authentication unit 415 moves the process to step S2211.

In step S2209, the image authentication unit 415 determines whether “i” has reached the number of users (iMAX) registered in the user information. If “i” has not reached “iMAX”, the image authentication unit 415 adds 1 to “i” in step S2210 and returns the process to step S2207. On the other hand, when “i” reaches “iMAX”, the image authentication unit 415 determines in step S2213 that the image authentication has failed (image authentication NG), and ends the process.

On the other hand, in step S2211, the image authentication unit 415 determines in step S2211 that the image authentication has succeeded (image authentication OK). Further, in step S2212, the image authentication unit 415 sets the image authentication result (OK) indicating that image authentication is permitted, and the user number of the user whose detected face image and face feature information match. The data is output to the device authentication unit 419, the storage unit 426 and the like.

By the above processing, the image authentication unit 415 permits the image authentication when the feature information of the face of the user included in the image captured by the image capturing unit 414 is registered in the user information, and the image authentication is performed. Output the user number of the authorized user.

(Image determination process)
FIG. 24 is a flowchart showing an example of the image determination process according to the first embodiment.

In step S2401, the image capturing unit 414 captures an image using the camera unit 332.

In step S2402, the image determination unit 418 executes the physical object determination process described in the first embodiment (for example, FIG. 3, 6, 8, 10, 12, or 14).

In step S2403, the image determination unit 418 branches the process depending on the determination result obtained in the actual determination process executed in step S2402 whether the person included in the image is a real (living body) or a photograph (non-living body).

When it is determined in step S2402 that the object is a "photograph" (non-living body), the image determination unit 418 shifts the processing to step S2406 and determines that the image determination is NG (the person included in the image is not a living body). Then, the process ends.

On the other hand, when it is determined by the actual object determination process that the object is a “real object” (living body), the image determination unit 418 shifts the process to step S2404.

When the process proceeds to step S2404, the image determination unit 418 determines that the image determination is OK (the person included in the image is a living body), and in step S2405, the image determination result (OK) is stored in the device authentication unit 419, the storage unit 426, or the like. Output.

By the above processing, the image determination unit 418 outputs an image determination result (OK) when the person included in the image captured by the image capturing unit 414 is determined to be a “living body”.

(Image forming apparatus authentication processing)
FIG. 25 is a flowchart showing an example of the device authentication process according to the first embodiment. For example, when the moving body detection unit 401 detects a moving body and the state control unit 402 shifts to the normal state, the image forming apparatus 101 starts the authentication process shown in FIG.

When the authentication process is started, the wireless authentication unit 412 executes the wireless authentication process shown in FIG. 21 (step S2501), and the image authentication unit 415 executes the image authentication process shown in FIG. 22 (step S2502). The image determination unit 418 also executes the image determination processing shown in FIG. 24 (step S2503).

In step S2504, the device authentication unit 419 determines whether the wireless authentication by the wireless authentication unit 412 is “OK”, the image authentication by the image authentication unit 415 is “OK”, and the image determination by the image determination unit 418 is “OK”. To judge.

When the wireless authentication is “OK”, the image authentication is “OK”, and the image determination is “OK”, the device authentication unit 419 shifts the processing to step S2505. On the other hand, if any of the wireless authentication, the image authentication, and the image determination is not “OK”, the device authentication unit 419 returns the process to steps S2501, S2502, and S2503. Further, the device authentication unit 419 waits until the wireless authentication is “OK”, the image authentication is “OK”, and the image determination is “OK”.

When the process proceeds to step S2505, the device authentication unit 419 determines whether the user of the wireless tag for which the wireless authentication is permitted and the user for which the image authentication is permitted are the same user. For example, the device authentication unit 419 determines whether the user number output from the wireless authentication unit 412 and the user number output from the image authentication unit 415 match. In addition, when there are a plurality of user numbers for which the wireless authentication is permitted, or a plurality of user numbers for which the image authentication is permitted, and when any one of them matches, the user of the wireless tag for which the wireless authentication is permitted, It is determined that the users who are authorized for image authentication are the same users.

When it is determined that the user of the wireless tag permitted for wireless authentication and the user permitted for image authentication are not the same user, the device authentication unit 419 returns the process to steps S2501, S2502, and S2503. In addition, the device authentication unit 419 waits until the user of the wireless tag for which the wireless authentication is permitted and the user for which the image authentication is permitted become the same user.

On the other hand, when it is determined that the user of the wireless tag for which the wireless authentication is permitted and the user for which the image authentication is permitted are the same user, in step S2506, the device authentication unit 419 logs in the user. Permit (permit the use of the image forming apparatus).

Through the above processing, the device authentication unit 419 causes the user of the wireless tag permitted to perform the wireless authentication and the user permitted to perform the image authentication to be the same user, and the person imaged in the image is the living body. If it is determined that the user is allowed to use the image forming apparatus 101.

As described above, according to the image forming apparatus 101 according to the present embodiment, in an information processing apparatus that authenticates a user with a captured image such as face authentication, impersonation by a non-living body such as a photograph is prevented, and user authentication is performed. The accuracy can be improved.

It should be noted that the configuration and the flow of processing of the above-described embodiment are examples, and various applications and modifications are possible. Subsequently, various modifications of the second embodiment will be described.

<Modification 1>
FIG. 26 is a flowchart showing an example of the image determination process according to the first modification. Note that the processes of steps S2401 and S2402 to S2406 of FIG. 26 are the same as the image determination process according to the first embodiment shown in FIG. 24, and therefore the difference will be mainly described here.

In step S2401, the image capturing unit 414 captures an image using the camera unit 332, and in step S2601, the face area detection unit 416 of the image forming apparatus 101 detects a face image from the captured image. The process of detecting the face image may be the same as step S2202 of FIG.

In step S2602, the image determination unit 418 of the image forming apparatus 101 branches the process depending on whether or not a face image is detected from the captured image.

When the face image is not detected from the captured image, the image forming apparatus 101 returns the process to step S2401 and repeats the same process. On the other hand, when the face image is detected from the captured image, the image forming apparatus 101 shifts the processing to step S2402 and executes the physical object determination processing.

By the above processing, when the captured image does not include a face, it is possible to avoid performing the actual determination processing and reduce the load on the image forming apparatus 101.

<Modification 2>
FIG. 27 is a diagram illustrating an example of image determination processing according to the second modification. Note that the processes of steps S2402 to S2406 of FIG. 27 are the same as the image determination process according to the first embodiment shown in FIG. 24, and therefore the difference will be mainly described here.

In step S2701, the imaging unit 414 uses the camera unit 332 to capture two or more continuous images in time series.

In step S2702, the noise determination unit 421 of the image forming apparatus 101 performs noise determination of the image captured in step S2701. Note that the noise determination is performed, for example, by determining the absolute value of the difference between the luminance values of the pixels of the two images arranged in time series and determining whether the difference between the sums is equal to or greater than a threshold value, as described above. The presence or absence of noise can be determined.

In step S2703, the noise determination unit 421 of the image forming apparatus 101 branches the process depending on whether or not noise is detected in step S2702.

When noise is detected in the captured image, the image forming apparatus 101 returns the process to step S2701 and repeats the same process. On the other hand, when no noise is detected in the captured image, the image forming apparatus 101 shifts the processing to step S2402 and executes the actual object determination processing.

By the above processing, when the captured image contains noise, the image forming apparatus 101 excludes the image containing noise and performs the actual determination process. As a result, the image forming apparatus 101 can reduce the load on the image forming apparatus 101 and reduce erroneous determination.

<Modification 3>
FIG. 28 is a flowchart showing an example of the timeout process of the authentication process according to the modified example 3. For example, when a user user carrying a wireless tag passes near the image forming apparatus 101, wireless authentication may be permitted and the wireless authentication result (OK) may remain. It is desirable to delete such information after a predetermined time has elapsed. In the third modification, an example of a process for deleting or initializing the result information of the wireless authentication, the image authentication, and the image determination after a predetermined time has passed will be described.

Note that the processes of steps S2501 to S2503 and steps S2504 to S2506 of FIG. 28 are the same as the authentication process of the image forming apparatus according to the first embodiment shown in FIG. 25, and therefore the differences will be mainly described here. ..

In step S2801, the timer management unit 423 of the image forming apparatus 101 starts the wireless authentication timer of the user whose wireless authentication is OK in step S2501. The wireless authentication timer is a timer that starts when the wireless authentication is OK and indicates a valid period (for example, 5 minutes) of the authentication result, and is managed individually for each user whose wireless authentication is OK. To be done.

Similarly, in step S2802, the timer management unit 423 starts the image authentication timer of the user whose image authentication is OK in step S2502. The image authentication timer is a timer that is started when the image authentication is OK and indicates a valid period (for example, 5 minutes) of the authentication result, and is managed individually for each user whose image authentication is OK. To be done.

In step S2803, the timer management unit 423 starts the image determination timer when the image determination is OK in step S2503. The image determination timer is a timer that starts when the image determination is OK and indicates the valid period (for example, 3 minutes) of the image determination result.

In step S2804, the timer management unit 423 determines whether or not the wireless authentication timer of each user whose wireless authentication is OK has timed out. If there is a wireless authentication timer that has timed out, in step S2805, the timer management unit 423 invalidates the wireless authentication result (OK) of the user whose wireless authentication timer has timed out (for example, deletes the corresponding wireless authentication result). To).

In step S2806, the timer management unit 423 determines whether or not the image authentication timer of each user whose image authentication is OK has timed out. If there is an image authentication timer that has timed out, in step S2807, the timer management unit 423 invalidates the image authentication result (OK) of the user whose image authentication timer has timed out (for example, deletes the corresponding image authentication result). To).

In step S2808, the timer management unit 423 determines whether the image determination timer has timed out. When the image determination timer times out, in step S2809, the timer management unit 423 invalidates (for example, deletes or initializes) the image determination result of the image determination unit 418.

By the above processing, the processing results of the wireless authentication processing, the image authentication processing, and the image determination processing become invalid (deleted or initialized) after a predetermined time has elapsed. As a result, the image forming apparatus 101 can reduce unnecessary information in the storage area that stores the authentication result and reduce erroneous authentication and unauthorized authentication.

<Modification 4>
For example, when a user wears a mask or sunglasses, it is generally difficult to pass image authentication such as face authentication. Further, if the user does not have a wireless tag, the user cannot pass the wireless authentication. Modification 4 describes an example in which the image forming apparatus 101 provides an alternative authentication means such as password authentication in such a case.

Note that, in FIG. 29, the processes of steps S2501 to S2506 are the same as the authentication process of the image forming apparatus according to the first embodiment shown in FIG. 25, and therefore the difference will be mainly described here.

In step S2901, the password authentication unit 420 of the image forming apparatus 101 displays the password input screen on the operation panel unit 326 of FIG. 18, for example, and accepts the password input by the user.

In step S2902, the device authentication unit 419 of the image forming apparatus 101 determines whether or not the input of the password has been accepted in step S2901. If the input of the password has not been accepted, the image forming apparatus 101 returns the process to steps S2501 to S2503 and S2901. On the other hand, if the input of the password has been received, the device authentication unit 419 shifts the processing to step S2903.

When the process proceeds to step S2903, the device authentication unit 419 determines whether or not the accepted password matches the password of the user whose wireless authentication is OK.

When the accepted password matches the password of the user whose wireless authentication is OK, the device authentication unit 419 shifts the processing to step S2506, and permits the user to use the image forming apparatus 101.

On the other hand, when the accepted password does not match the password of the user whose wireless authentication is OK, the device authentication unit 419 shifts the processing to step S2904.

When the process proceeds to step S2904, the device authentication unit 419 determines whether the accepted password matches the password of the user whose image authentication has been OK.

When the accepted password matches the password of the user whose image authentication is OK, the device authentication unit 419 moves the process to step S2506, and permits the user to use the image forming apparatus 101.

On the other hand, when the accepted password does not match the password of the user whose image authentication is OK, the device authentication unit 419 returns the processing to steps S2501 to S2503 and S2901.

By the above processing, the user of the image forming apparatus 101 can use the image forming apparatus 101 by alternative authentication means (for example, password authentication) even when wireless authentication or image authentication cannot be used. Become.

<Modification 5>
FIG. 30 is a diagram illustrating an example of image authentication processing according to the fifth modification.

In the image authentication processing shown in FIG. 22, the image authentication unit 415 sequentially compares the face feature information extracted from the image with the face feature information of a plurality of users registered in the user information, An example of performing authentication has been described. In Modification 5, when there is a user whose wireless authentication is already OK, the facial feature information extracted from the image is compared with the facial feature information of the user whose wireless authentication is OK, and the image is displayed. An example of authentication will be described.

Note that the processes of steps S2201 to S2204 and steps S2205 to S2213 of FIG. 30 are the same as the image authentication process according to the first embodiment shown in FIG. 22, and therefore the difference will be mainly described here.

In the fifth modification, the feature information extraction unit 417 extracts the face feature information from the face image in step S2204, and then the image authentication unit 415 determines whether or not there is a user whose wireless authentication is OK in step S3001. To judge.

If there is no user whose wireless authentication is OK, the image authentication unit 415 shifts the processing to step S2205 and executes the same processing as in FIG. On the other hand, when there is a user whose wireless authentication is OK, the image authentication unit 415 moves the process to step S3002.

In step S3002, the image authentication unit 415 acquires user information including facial feature information of the user whose wireless authentication has been OK.

In step S3003, the image authentication unit 415 compares the facial feature information extracted in step S2204 with the facial feature information of the user for whom the wireless authentication obtained in step S3002 is OK.

In step S3004, the image authentication unit 415 determines whether or not the facial feature information extracted in step S2204 and the facial feature information of the user for whom the wireless authentication obtained in step S3002 is OK. To do. The matching of the face feature information includes not only the case where the two face feature information completely match, but also the case where the similarity between the two feature information is equal to or more than a predetermined threshold value.

If the face characteristic information does not match, the image authentication unit 415 moves the process to step S3005. On the other hand, if the face feature information matches, the image authentication unit 415 moves the process to step S2211, and executes the process in the case of image authentication OK.

In step S3005, the image authentication unit 415 determines that the image authentication has failed (image authentication NG), and ends the process.

With the above processing, when there is a user whose wireless authentication is OK, the number of users who compare the facial feature information is reduced, so that the processing load of the image forming apparatus 101 is reduced and the processing speed is increased. Can be turned into.

<Modification 6>
FIG. 31 is a flowchart showing an example of transmission control processing according to the sixth modification.

When there is no user in the vicinity of the image forming apparatus 101, the radio wave transmitted by the short-range wireless unit 329 is considered unnecessary. Therefore, in Modification 6, the image forming apparatus 101 controls the transmission of the radio waves of the short-range wireless unit 329 in accordance with the detection result of the mobile body by the mobile body detection unit 401 in order to suppress the useless transmission of the radio waves. A case example will be described.

In step S3101, when the moving body detection state of the moving body detection unit 401 of the image forming apparatus 101 changes, the transmission control unit 413 executes the processing of step S3102 and subsequent steps.

When the process proceeds to step S3102, the transmission control unit 413 determines, based on the detection result of the moving body detection unit 401, whether or not there is a moving body within the detection range of the moving body detection unit 401. When there is a moving body within the detection range of the moving body detection unit 401, the transmission control unit 413 shifts the processing to step S3103. On the other hand, when there is no moving body within the detection range of the moving body detection unit 401, the transmission control unit 413 shifts the processing to step S3105.

When the process proceeds to step S3103, the transmission control unit 413 determines whether the short-range wireless unit 329 is transmitting radio waves. If the short-range wireless unit 329 is not transmitting radio waves, in step S3104, the transmission control unit 413 causes the short-range wireless unit 329 to start transmitting radio waves. On the other hand, when the short-range wireless unit 329 is transmitting a radio wave, the transmission control unit 413 ends the process.

When the process proceeds to step S3105, the transmission control unit 413 determines whether the short-range wireless unit 329 is transmitting radio waves. When the short-range wireless unit 329 is transmitting radio waves, the transmission control unit 413 causes the short-range wireless unit 329 to stop transmitting radio waves in step S3106. On the other hand, when the short-range wireless unit 329 is not transmitting radio waves, the transmission control unit 413 ends the process.

By the above processing, when there is a moving body (person or the like) around the image forming apparatus 101, the image forming apparatus 101 causes the short-range wireless unit 329 to transmit a radio wave so that the moving body is around the image forming apparatus 101. If not, the short-range wireless unit 329 can be stopped from transmitting radio waves.

[Second Embodiment]
In the second embodiment, an example in which the image determination processing of the image forming apparatus 101 according to the first embodiment is layered will be described. Hierarchical means that the extracted biometric feature information (for example, facial feature information) is finally judged through a plurality of discriminators to determine whether or not it is a biometric.

For example, when it is difficult to determine whether a living body is a living body at a time due to the installation environment of the image forming apparatus 101, a usage scene, or the like, the hierarchy determining unit 422 instead of the image determining unit 418 determines the living body. It should be noted that which of the image determination unit 418 and the hierarchy determination unit 422 is to be used for the determination is preferably set by a manager or a user of the image forming apparatus 101 from a setting screen or the like.

<Functional configuration>
The functional configuration of the image forming apparatus 101 according to the second embodiment is similar to the functional configuration of the image forming apparatus 101 according to the first embodiment shown in FIG.

The layer determination unit (another example of the image determination unit) 422 hierarchically determines the image determination process performed by the image determination unit 418. Hierarchizing the determination process means finally determining whether or not the extracted biometric feature information (for example, face feature information) is passed through a plurality of discriminators.

In the present embodiment, “discriminant analysis” is used as a discriminator, “k-means (k-means)” is used as a grouping method, and the hierarchy is divided into three layers of “large classification”, “medium classification”, and “small classification”. An example of the divided case will be described.

The storage unit 426 uses the biological image captured by the image capturing unit 414 to generate biological information 432 that has been created in advance, and the non-living biological image captured by the image capturing unit 414 that has previously created non-living information 433. Is stored.

(Example of layering)
32 to 36 are diagrams for explaining the layering according to the second embodiment.

When the extracted feature information is subjected to discriminant analysis based on the biometric information 432 and the non-biometric information 433 stored in advance in the storage unit 426, the extracted feature information projected on the space stretched by the upper three discriminant axes. When k is grouped by k-means, non-biological information is classified into 5 and biological information is classified into 3 as shown in FIG. In this example, the biometric information 1 and the non-biometric information 3, and the biometric information 3, the non-biometric information 4, and the non-biometric information 5 have very close distributions and are difficult to identify.

Therefore, the hierarchy determining unit 422 creates a space for identification by collecting discriminative items that are close together and performing a discriminant analysis. Each of the five pieces of information (group) classified into the space shown in FIG. 33 is referred to as a “major classification”. For example, as shown in FIG. 33, the hierarchy determination unit 422 sets the biometric information 1 and the non-biometric information 3 as “group A”, and the biometric information 3, the non-biometric information 4, and the non-biometric information 5 as “group B”. And

In the space of “major classification” shown in FIG. 33, the extracted feature information is one of non-biometric information 1, biometric information 2, non-biometric information 2, “group A”, and “group B”. It is intended to be classified into the "major classification" of. The discriminant analysis space is such that the within-class variance of each class is minimized and the inter-class variance is maximized. In the example of FIG. 33, the distributions that are close to each other and whose inter-class variance cannot be increased are grouped into one group, so that the classification space in “major classification” is a space that is easier to classify as a whole. Also, since the initial eight classification problems became five classification problems, the number of dimensions of the identification space was reduced from seven to four, and the processing time for classification can be reduced (the number of identification space dimensions in discriminant analysis is the classification class). Number-1).

When the extracted characteristic information is classified into “group A” or “group B”, it is necessary to determine what is classified in the group, and therefore it is necessary to perform the following identification. ..

When the discriminant analysis is performed only with the biometric information 3, the non-biometric information 4, and the non-biometric information 5 of the “group B”, it is assumed that the distribution is as shown in FIG. Also in this case, for example, when it is difficult to separate the biometric information 3 and the non-biometric information 4, these are grouped as described above, and two classification problems of the “group BI” and the non-biometric information 5 are generated. can do. Each of the two pieces of information (groups) classified into the space shown in FIG. 35 is referred to as “medium classification”.

Also in this case, when the extracted characteristic information is classified into “group B-I”, it is necessary to identify what is classified therein again. That is, it is necessary to classify the extracted feature information into one of the “small classification” of the non-biometric information 4 and the biometric information 3. The distribution of the results of these two discriminant analyzes is shown in FIG. In the example shown in FIG. 36, the non-biometric information 4 and the biometric information 3 can be easily identified.

In the above example, the feature information classified into the non-biometric information 1, the biometric information 2, and the non-biometric information 2 has been discriminated by “major classification”, and the biometric information 1, the non-biometric information 3, and the non-biometric information 5 For the items classified into and, the discrimination is completed by "medium classification". In addition, the classification into the biometric information 3 and the non-biometric information 4 is completed by the “small classification”.

Similarity is often judged by the angle and distance in space. When grouping multiple pieces of feature information, the number of spatial dimensions when calculating angles and distances is reduced, and the number of comparison targets is also reduced. Therefore, all "major classification", "medium classification", and "small separation" are performed. Even in the case, the processing time is shortened. In addition, each identification space is created in a reasonable unit (a group of items that are difficult to classify compared to others) is created, so the identification rate is also improved. Difficult classification is dealt with by creating a dedicated identification space in a situation that does not include others.

<Process flow>
FIG. 37 is a flowchart showing an example of the image determination process according to the second embodiment.

In step S3701, the image capturing unit 414 captures an image using the camera unit 332.

In step S3702, the characteristic information extraction unit 417 extracts the characteristic information of the person from the captured image.

In step S3703, the hierarchy determination unit 422 determines which group, out of the roughly classified groups, the characteristic information acquired in step S3702 belongs to. For example, in the example of FIG. 33, the hierarchy determination unit 422 belongs to which group of the non-biometric information 1, the biometric information 2, the non-biometric information 2, the group A, and the group B, the characteristic information acquired in step S3702. Discriminate and analyze.

In step S3704, the hierarchy determination unit 422 determines whether the characteristic information acquired in step S3702 is included in the non-living body group (in the example of FIG. 33, the non-living body information 1 or the non-living body information 2). .. If the characteristic information acquired in step S3702 belongs to the non-living body group, the hierarchy determining unit 422 ends the process. On the other hand, when the characteristic information acquired in step S3702 does not belong to the non-living body group, the hierarchy determination unit 422 shifts the processing to S3705.

In step S3705, the hierarchy determination unit 422 determines which group, out of the plurality of intermediately classified groups, the characteristic information acquired in step S3702 belongs to. For example, in the example of FIG. 35, the hierarchy determination unit 422 discriminates and analyzes which group of the non-biological information 5 and the group B-I the characteristic information acquired in step S3702 belongs to.

In step S3706, the hierarchy determination unit 422 determines whether the characteristic information acquired in step S3702 is included in the non-living body group (non-living body information 5 in the example of FIG. 35). If the characteristic information acquired in step S3702 belongs to the non-living body group, the hierarchy determining unit 422 ends the process. On the other hand, when the characteristic information acquired in step S3702 does not belong to the non-living body group, the hierarchy determination unit 422 shifts the processing to S3707.

In step S3707, the hierarchy determination unit 422 determines to which group, out of the plurality of sub-classified groups, the characteristic information acquired in step S3702 belongs. For example, in the example of FIG. 36, the hierarchy determination unit 422 discriminates and analyzes which group of the non-biometric information 4 and the biometric information 3 the characteristic information acquired in step S3702 belongs to.

In step S3708, when the characteristic information acquired in step S3702 belongs to the non-living body group, the hierarchy determination unit 422 ends the process. On the other hand, when the characteristic information acquired in step S3702 belongs to the group of living bodies, the hierarchy determination unit 422 shifts the processing to step S3709.

When the process proceeds to step S3709, the hierarchy determination unit 422 outputs the image determination result (OK) indicating that the person included in the image acquired in step S3701 is a living body to the storage unit 426, the device authentication unit 419, and the like.

By the above process, the image forming apparatus 101 can shorten the processing time of the process of determining whether the person included in the image is a living body and improve the identification rate.

[Supplement to Embodiment]
The functions of each unit illustrated in FIG. 19 are realized by the CPU 311 or the CPU 321 illustrated in FIG. It However, the present invention is not limited to this, and at least a part of the function of each unit of the image forming apparatus 101 may be realized by a dedicated hardware circuit (for example, a semiconductor integrated circuit).

A program (control program) executed by the image forming apparatus 101 is a file in an installable format or an executable format, and is recorded on a computer-readable recording medium such as various types of disks, media, and USB memory. It may be configured to be provided. Alternatively, the program executed by the image forming apparatus 101 may be provided or distributed via a network such as the Internet. Further, the various programs may be configured to be provided in advance by being incorporated in a nonvolatile recording medium such as a ROM.

10 Image Processing Device 11 Camera Image Receiving Unit 12 Human Area Detection Unit (Example of Detecting Means)
13 Face Area Detection Unit (an example of detection means)
14 background area ratio determination unit 15 background area ratio determination threshold setting unit (second threshold setting unit)
16 Motion information determination unit (determination means)
17 inter-image correlation calculation unit (an example of calculation means)
18 Flow histogram calculation unit (an example of calculation means)
Reference numeral 19 is a deviation value removal unit 20 is an image orientation normalization unit 21 is a dictionary update unit (update means)
22 person area motion dictionary 23 face area motion dictionary 24 motion information determination threshold value setting section (first threshold value setting means)
31 Operation unit 32 ROM
33 RAM
34 CPU
35 camera 101 image forming apparatus (information processing apparatus)
104 RFID tag (wireless tag)
401 moving body detection unit (moving body detection means)
411 wireless communication unit (wireless communication means)
412 wireless authentication unit (wireless authentication means)
413 Transmission control unit (transmission control means)
414 Imaging unit (imaging means)
415 Image authentication unit (image authentication means)
418 Image determination unit (image determination means)
419 Device authentication unit (device authentication means)
420 Password authentication unit (password authentication means)
421 Noise determination unit (noise determination means)
422 Layer determination unit (another example of image determination means)
423 Timer Management Unit (Timer Management Means)

JP, 2015-35178, A

Claims (21)

An information processing device for authenticating a user,
A wireless communication means for acquiring predetermined information from a wireless tag,
A wireless authentication unit that performs wireless authentication to authenticate the wireless tag based on the acquired predetermined information;
An image pickup means for picking up an image,
An image authentication unit that performs image authentication to authenticate the user captured in the image,
An image determination unit that determines whether the determination target captured in the image is a living body that satisfies a predetermined condition,
When it is determined that the subject of the determination captured in the image is a living body satisfying the predetermined condition, and the wireless authentication and the image authentication are permitted, the wireless tag of the wireless authentication is permitted. It is determined whether the user is the same user as the user who is permitted the image authentication, and the user of the wireless tag permitted the wireless authentication and the user who is permitted the image authentication. And are the same user, a device authentication unit that permits the use of the information processing device by the user,
Information processing device having a. The image determination means,
Detection means for detecting an area corresponding to a person in each of the plurality of images captured by the imaging means,
Using one image in which the area corresponding to the person is detected by the detection means, and one or more other images in which the area corresponding to the person is detected before the detection of the one image, Calculating means for calculating the motion information of the person;
Determination means for determining whether the person detected from the one image is a living body satisfying the predetermined condition, based on the movement information obtained by the calculation means and the movement information of the person set in advance; ,
The information processing apparatus according to claim 1, further comprising: The determination means is
When the motion information obtained by the calculating means and the preset motion information of the person are similar to each other by a predetermined threshold value or more, the person detected from the one image is a living body satisfying the predetermined condition. The information processing apparatus according to claim 2, wherein the information processing apparatus determines that The calculation means is
When the ratio of the area corresponding to the person detected from the one image to the area not corresponding to the person exceeds a threshold value, the motion information of the person is calculated using the entire one image. The information processing device according to claim 2 or 3, characterized in that: The detection means is
The information processing apparatus according to any one of claims 2 to 4, wherein a face area of the person is detected as an area corresponding to the person. The calculation means is
The information processing apparatus according to claim 2, wherein when an optical flow is calculated as the motion information, a predetermined outlier is removed. The calculation means is
The information processing apparatus according to claim 2, wherein the orientation of the person is normalized. After the determination unit determines that the person is a living body that satisfies the predetermined condition, and if a user specifying action that specifies a user is detected, the person is the living body that satisfies the predetermined condition. The information processing apparatus according to any one of claims 2 to 7, further comprising an updating unit that updates the preset motion information of the person using the determined motion information. First threshold value setting means for setting a predetermined threshold value used by the judging means,
Second threshold value setting means for setting a predetermined threshold value used by the calculating means,
The information processing apparatus according to claim 2, further comprising: The image determination means,
The information processing apparatus according to claim 1, wherein it is determined that the target of the determination is a living body that satisfies the predetermined condition by performing a discriminant analysis by dividing into a plurality of layers. Based on a difference amount of pixel values of a plurality of images captured by the image capturing unit, a noise determining unit for determining an image including noise,
The image determination means,
The information processing apparatus according to any one of claims 1 to 10, wherein an image including the noise is excluded to determine that the determination target is a living body that satisfies the predetermined condition. 12. The timer management means for deleting or initializing the authentication result by the wireless authentication means, the authentication result by the image authentication means, or the determination result by the image determination means after a predetermined period has elapsed. The information processing device according to one item. It has a password authentication method that accepts the password input by the user,
The device authentication means,
When the password authentication means accepts the input of the password corresponding to the user of the wireless tag authorized by the wireless authentication means or the user authorized by the image authentication means, The information processing apparatus according to claim 1, wherein use of the information processing apparatus is permitted. A moving body detecting means for detecting a moving body around the information processing device,
14. The transmission control unit according to claim 1, further comprising a transmission control unit that controls the presence or absence of transmission of a radio wave for obtaining the predetermined information from the wireless tag based on a detection result of the mobile body. The information processing device according to item. Image authentication means
15. When there is a wireless tag permitted to be authenticated by the wireless authentication means, the imaged user is authenticated using the characteristic information of the living body corresponding to the user of the wireless tag. The information processing device according to one item. 16. The image determination unit determines whether the determination target is a photograph, thereby determining whether the determination target is a living body satisfying the predetermined condition. The information processing device described. When it is determined that the user of the wireless tag authorized for the wireless authentication is not the same user as the user authorized for the image authentication, the device authenticating unit is configured to perform the wireless authentication. The information processing apparatus according to claim 1, which waits for a user of the wireless tag and a user who is permitted to perform the image authentication to become the same user. If there are a plurality of users of the wireless tag permitted to perform the wireless authentication, or a plurality of users permitted to perform the image authentication, and if any one of them matches, the device authentication means is a user of the wireless tag. The information processing apparatus according to claim 1, wherein it is determined that the user who is permitted to perform the image authentication is the same user. An information processing device that authenticates the user
A wireless communication means for acquiring predetermined information from a wireless tag,
A wireless authentication unit that performs wireless authentication to authenticate the wireless tag based on the acquired predetermined information;
An image pickup means for picking up an image,
An image authentication unit that performs image authentication to authenticate the user captured in the image,
An image determination unit that determines whether the determination target captured in the image is a living body that satisfies a predetermined condition,
When it is determined that the subject of the determination captured in the image is a living body that satisfies the predetermined condition, and the wireless authentication and the image authentication are permitted, the wireless tag of which the wireless authentication is permitted is It is determined whether or not the user and the user who is permitted the image authentication are the same user, and the user of the wireless tag who is permitted the wireless authentication and the user who is permitted the image authentication. And are the same user, a device authentication unit that permits the use of the information processing device by the user,
Program to function as. An authentication method using an information processing device,
The information processing device is
Acquiring predetermined information from the wireless tag,
Performing a wireless authentication to authenticate the wireless tag based on the acquired predetermined information,
Capturing an image,
A step of performing image authentication for authenticating the user captured in the image;
A step of determining whether the determination target captured in the image is a living body that satisfies a predetermined condition,
When it is determined that the subject of the determination captured in the image is a living body that satisfies the predetermined condition, and the wireless authentication and the image authentication are permitted, the wireless tag of which the wireless authentication is permitted is It is determined whether or not the user and the user who is permitted the image authentication are the same user, and the user of the wireless tag who is permitted the wireless authentication and the user who is permitted the image authentication. And are the same user, a step of allowing the user to use the information processing device,
Authentication method including. An information processing system including a short-range wireless device that receives predetermined information from a wireless tag, an imaging device that captures an image, the short-range wireless device, and an information processing device connected to the imaging device. ,
The information processing device,
Wireless communication means for acquiring the predetermined information received by the short-range wireless device,
A wireless authentication unit that performs wireless authentication to authenticate the wireless tag based on the acquired predetermined information;
An image pickup means for picking up an image using the image pickup device;
An image authentication means for performing image authentication for authenticating the imaged user ;
An image determination unit that performs image determination to determine whether the determination target captured in the image is a living body that satisfies a predetermined condition,
When it is determined that the subject of the determination captured in the image is a living body that satisfies the predetermined condition, and the wireless authentication and the image authentication are permitted, the wireless tag of which the wireless authentication is permitted is It is determined whether the user and the user who is permitted the image authentication are the same user, and the user of the wireless tag who is permitted the wireless authentication and the user who is permitted the image authentication. And are the same user, a device authentication unit that permits the use of the information processing device by the user,
Information processing system having.