EP4690151A1

EP4690151A1 - Information processing apparatus, information processing method, and non-transitory computer-readable medium

Info

Publication number: EP4690151A1
Application number: EP24718915.2A
Authority: EP
Inventors: Hiroki Tokunaga; Koichi Sakumoto
Original assignee: Sony Group Corp
Current assignee: Sony Group Corp
Priority date: 2023-03-30
Filing date: 2024-03-26
Publication date: 2026-02-11
Also published as: WO2024204133A1; CN120937062A; JP2024143295A

Abstract

An information processing apparatus including: circuitry configured to: acquire a first image of an iris of a user; acquire a second image of a palm of the user; and perform an authentication process or a registration process based on the first image and the second image, in which the first image and the second image are acquired in parallel.

Description

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

The present disclosure relates to an information processing apparatus, an information processing method, and a non-transitory computer-readable medium, and more particularly, to an information processing apparatus, an information processing method, and a non-transitory computer-readable medium, all of which can perform user authentication more satisfactorily.

<CROSS REFERENCE TO RELATED APPLICATIONS>
This application claims the benefit of Japanese Priority Patent Application JP 2023-055893 filed on March 30, 2023, the entire contents of which are incorporated herein by reference.

In related art, in a head mounted display (HMD) device worn on a head of a user, user authentication using an iris is considered as a user interface suitable for performing the user authentication more easily than user authentication in which a password is input, and a better user experience can be provided.

For example, PTL 1 discloses a see-through HMD device that identifies a user by a pattern of image data obtained by imaging an iris to provide the user with a customized augmented reality experience.

US Patent No. 9202443

Meanwhile, a general HMD device is only mounted with a low-performance in-camera for tracking a line-of-sight, and it is difficult to obtain a sufficient authentication performance by iris authentication using the in-camera. Furthermore, in a case where the user authentication is performed by combining biometric authentication other than the iris authentication in order to obtain the sufficient authentication performance, it is assumed that a period of time required for the user authentication may become longer. Therefore, it may be required to more satisfactorily perform the user authentication by using the in-camera to be mounted on the general HMD device.

The present disclosure has been made in view of such a situation, and is to enable user authentication to be performed more satisfactorily.

An information processing apparatus according to an embodiment of the present disclosure includes circuitry configured to: acquire a first image of an iris of a user; acquire a second image of a palm of the user; and perform an authentication process or a registration process based on the first image and the second image, in which the first image and the second image are acquired in parallel.

An information processing method according to an embodiment of the present disclosure includes: acquiring a first image of an iris of a user; acquiring a second image of a palm of the user; and performing an authentication process or a registration process based on the first image and the second image, in which the first image and the second image are acquired in parallel.

A non-transitory computer-readable medium having embodied thereon a program according to an embodiment of the present disclosure, which when executed by a computer causes the computer to execute an information processing method, the method including: acquiring a first image of an iris of a user; acquiring a second image of a palm of the user; and performing an authentication process or a registration process based on the acquired first image and the acquired second image, in which the first image and the second image are acquired in parallel.

Fig. 1 is a diagram illustrating a configuration example of an embodiment of an HMD device to which the present technology is applied. Fig. 2 is a block diagram illustrating a functional configuration example of the HMD device. Fig. 3 is a diagram illustrating a display example of a palm guide image and a line-of-sight guide image during execution of a user registration process. Fig. 4 is a diagram illustrating a display example of a palm guide image and a line-of-sight guide image in a case where only a palm collation score is less than or equal to a threshold value during execution of a user authentication process. Fig. 5 is a diagram illustrating a display example of a palm guide image and a line-of-sight guide image in a case where both the palm collation score and an iris collation score are less than or equal to threshold values during the execution of the user authentication process. Fig. 6 is a diagram illustrating a display example of a palm guide image and a line-of-sight guide image in a case where only the iris collation score is less than or equal to the threshold value during the execution of the user authentication process. Figs. 7A to 7B are diagrams illustrating determination processes of the palm collation score and the iris collation score. Fig. 8 is a flowchart illustrating the user registration process. Fig. 9 is a flowchart illustrating the user authentication process. Fig. 10 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied.

Hereinafter, a specific embodiment to which the present technology is applied will be described in detail with reference to the drawings.

<Configuration example of HMD device>
Fig. 1 is a diagram illustrating a configuration example of an embodiment of the HMD device that is the wearable display device to which the present technology is applied.

As illustrated in Fig. 1, an HMD device 11 is configured to cover both eyes of a user by a transmissive display 21 when the HMD device is worn on the head of the user. The user can see, for example, a circumference, and his/her hands through a display screen 12 of the transmissive display 21.

The HMD device 11 adopts a user authentication process in which both palm authentication using a palm of the user and iris authentication using an iris of the user are executed in parallel. Therefore, the HMD device 11 simultaneously displays a palm guide image 13 and a line-of-sight guide image 14 on the display screen 12 of the transmissive display 21 during execution of each of a user registration process and a user authentication process. The palm guide image 13 is an image for guiding a posture of the palm of the user, and the line-of-sight guide image 14 is an image for guiding a direction of a line-of-sight of the user.

During the execution of the each of the user registration process and the user authentication process, the user can change the posture of the palm to match a shape of the palm guide image 13, or move the direction of the line-of-sight to gaze at the line-of-sight guide image 14.

Fig. 2 is a block diagram illustrating a functional configuration example of the HMD device 11.

As illustrated in Fig. 2, the HMD device 11 includes the transmissive display 21, an out-camera 22, an in-camera 23, and an information processing apparatus 24. The information processing apparatus 24 has a display control unit 31, a palm collation score calculation unit 32, an iris collation score calculation unit 33, a determination processing unit 34, and a registration template creation unit 35.

The transmissive display 21 displays the palm guide image 13 and the line-of-sight guide image 14 as illustrated in Fig. 1 on the display screen 12 under control of the display control unit 31 during the execution of the each of the user registration process and the user authentication process.

The out-camera 22 has, for example, a performance to an extent required to recognize a situation of the circumference, and performs imaging from a front of the HMD device 11 toward an outer side to acquire palm images in which the palm held by the user in front of a face of the user is captured. Then, the out-camera 22 supplies the palm images to the determination processing unit 34 during the execution of the user registration process, and supplies the palm images to the palm collation score calculation unit 32 during the execution of the user authentication process.

The in-camera 23 has, for example, a performance to an extent required to track the line-of-sight, and performs the imaging toward an inner side of the transmissive display 21 of the HMD device 11 to acquire iris images in which the iris of the user is captured. Then, the in-camera 23 supplies the iris images to the determination processing unit 34 during the execution of the user registration process, and supplies the iris images to the iris collation score calculation unit 33 during the execution of the user authentication process.

The information processing apparatus 24 executes the user authentication process in which the iris authentication using the iris images and the palm authentication using the palm images are performed in parallel.

The display control unit 31 performs display controls on the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21. For example, as described later with reference to Figs. 3 to 6, the display control unit 31 can perform a display control for changing, for example, a position, an orientation, and a shape of the palm guide image 13, and perform a display control for changing a position of the line-of-sight guide image 14.

During the execution of the user authentication process, the palm collation score calculation unit 32 collates feature amounts of the palm captured in the palm images to be supplied from the out-camera 22 with feature amounts of a palm registered in a palm registration template created in advance in the user registration process, and calculates a palm collation score that is a numerical value indicating the collation result. For example, as the numerical value is larger, the palm collation score indicates that a palm imaged during the execution of the user authentication process is of the same person as a palm imaged during the execution of the user registration process.

The iris collation score calculation unit 33 collates feature amounts of the iris captured in the iris images to be supplied from the in-camera 23 with feature amounts of an iris registered in an iris registration template created in advance in the user registration process, and calculates an iris collation score that is a numerical value indicating the collation result during the execution of the user authentication process. For example, as the numerical value is larger, the iris collation score indicates that an iris imaged during the execution of the user authentication process is of the same person as an iris imaged during the execution of the user registration process.

The determination processing unit 34 performs various types of determination processes required during the execution of the each of the user registration process and the user authentication process.

In a case where the determination processing unit 34 determines that the number of the palm images to be supplied from the out-camera 22 and the number of the iris images to be supplied from the in-camera 23 are greater than or equal to a prescribed number required to create the registration template, respectively, during the execution of the user registration process, the determination processing unit supplies the prescribed number of the palm images and the iris images to the registration template creation unit 35, respectively. During the execution of the user registration process, the determination processing unit 34 determines registration success or registration failure in the user registration process on the basis of determining whether or not both the palm registration template and the iris registration template have been created by the registration template creation unit 35.

During the execution of the user authentication process, the determination processing unit 34 determines authentication success or authentication failure in the user authentication process on the basis of the palm collation score to be supplied from the palm collation score calculation unit 32 and the iris collation score to be supplied from the iris collation score calculation unit 33. For example, as described later with reference to Figs. 7A to 7B, the determination processing unit 34 can maintain the palm collation score and the iris collation score for a certain period according to a window size, and use maximum values in the certain period for collation.

The determination processing unit 34 notifies the display control unit 31 that the palm collation score is less than or equal to a threshold value in a case where the palm collation score is less than or equal to the threshold value, and notifies the display control unit 31 that the iris collation score is less than or equal to a threshold value in a case where the iris collation score is less than or equal to the threshold value.

The registration template creation unit 35 performs a process of extracting the feature amounts of the palm from the prescribed number of the palm images to create the palm registration template in which the feature amounts of the palm are registered, and a process of extracting the feature amounts of the iris from the prescribed number of the iris images to create the iris registration template in which the feature amounts of the iris are registered, in parallel.

By executing both the palm authentication and the iris authentication in parallel, the HMD device 11 thus configured can shorten a processing period of time required for the user authentication process as compared with, for example, a case where the palm authentication and the iris authentication are sequentially performed. Similarly, the HMD device 11 can shorten a processing period of time required for the user registration process by executing processes in parallel also in the user registration process. Furthermore, in the HMD device 11, the determination processing unit 34 can maintain the palm collation score and the iris collation score for the certain period, and avoid a failure in authentication of the user himself/herself by using the maximum values in the certain period for the collation. Consequently, the HMD device 11 can perform the user authentication more satisfactorily.

<Display controls of palm guide image and line-of-sight guide image>
The display controls of the palm guide image 13 and the line-of-sight guide image 14 will be described with reference to Figs. 3 to 6.

Fig. 3 illustrates display examples of the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21 during the execution of the user registration process.

During the execution of the user registration process, the palm guide image 13 and the line-of-sight guide image 14 are simultaneously displayed on the display screen 12, and the display control for changing, for example, the position, the orientation, and the shape of the palm guide image 13 and the display control for changing the position of the line-of-sight guide image 14 are performed. Accordingly, both creation of the palm registration template to be performed by using the prescribed number of the palm images acquired by imaging the palm in various postures and creation of the iris registration template to be performed by using the prescribed number of the iris images acquired by imaging the iris in various directions can be executed in parallel. Consequently, for example, the HMD device 11 can shorten the processing period of time required for the user registration process as compared with a user registration process in which the creation of the palm registration template and the creation of the iris registration template are sequentially performed.

Figs. 4 to 6 illustrates display examples of the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21 during the execution of the user authentication process.

First, determination criteria of the authentication success/the authentication failure during the execution of the user authentication process will be described. For example, in a case where the palm collation score exceeds the threshold value and the iris collation score exceeds the threshold value at a certain elapsed timing t, it is determined that the authentication has succeeded. On the other hand, in a case where the palm collation score does not exceed the threshold value at the certain elapsed timing t, or in a case where the iris collation score does not exceed the threshold value, and in both the cases, determination of the authentication success or the authentication failure is performed at a next timing t + 1. Then, in a case where the elapsed timing t has passed a timeout without the palm collation score exceeding the threshold value or without the iris collation score exceeding the threshold value, it is determined that the authentication has failed.

Fig. 4 illustrates display examples of the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21 in a case where only the palm collation score is less than or equal to the threshold value during the execution of the user authentication process.

During the execution of the user authentication process, in a case where only the palm collation score is less than or equal to the threshold value, the display control for changing, for example, the position, the orientation, and the shape of the palm guide image 13 is performed with the line-of-sight guide image 14 fixed. Accordingly, it is possible to acquire the palm images obtained by imaging the palm in the various postures, and increase the palm collation score.

Fig. 5 illustrates display examples of the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21 in a case where the palm collation score is less than or equal to the threshold value and the iris collation score is less than or equal to the threshold value during the execution of the user authentication process.

During the execution of the user authentication process, in a case where the palm collation score is less than or equal to the threshold value and the iris collation score is less than or equal to the threshold value, the display control for changing, for example, the position, the orientation, and the shape of the palm guide image 13 and the display control for changing the position of the line-of-sight guide image 14 are performed. Accordingly, it is possible to acquire the palm images obtained by imaging the palms in the various postures and acquire the iris images obtained by imaging the irises in the various directions, and it is possible to increase the palm collation score and the iris collation score.

Fig. 6 illustrates display examples of the palm guide image 13 and the line-of-sight guide image 14 to be displayed on the display screen 12 of the transmissive display 21 in a case where only the iris collation score is less than or equal to the threshold value during the execution of the user authentication process.

During the execution of the user authentication process, in a case where only the iris collation score is less than or equal to the threshold value, the display control for changing the position of the line-of-sight guide image 14 is performed with the palm guide image 13 fixed. Accordingly, it is possible to acquire the iris images by imaging the iris in the various directions, and it is possible to increase the iris collation score.

Furthermore, when magnitude relationships of the palm collation score and the iris collation score to the threshold values are different, each of the display controls as illustrated in Figs. 4 to 6 return to the beginning, and the display controls are repeated. Furthermore, even when the user authentication process has timed out, the display controls can be performed from the beginning.

In this way, the display controls of the palm guide image 13 and the line-of-sight guide image 14 are performed in accordance with transition of the magnitude relationships of the palm collation score and the iris collation score to the threshold values to facilitate successful authentication.

Accordingly, the HMD device 11 can avoid failure in the authentication of the user himself/herself, for example, as compared with an authentication process in which the palm authentication and the iris authentication are simply combined. In other words, in a case where the determination criteria of the authentication success/the authentication failure described above are used to avoid erroneous success in authentication of another person, since both the palm collation score and the iris collation score are required to exceed the threshold values at the same time to succeed in the authentication of the user himself/herself, a probability of failure in the authentication of the user himself/herself increases as compared with a case where the authentication is performed by using the palm authentication and the iris authentication alone.

Even in such case, the HMD device 11 can reduce the probability of failure in the authentication of the user himself/herself by performing the display controls of the palm guide image 13 and the line-of-sight guide image 14 to facilitate the successful authentication.

<Determination processes of collation scores>
With reference to Figs. 7A to 7B, determination processes of the palm collation score and the iris collation score will be described.

The determination processing unit 34 can maintain the iris collation score and the palm collation score for a certain period according to a predetermined window size, and use the maximum values in the certain period for collation.

For example, in the case where the determination criteria of the authentication success/the authentication failure described above are used to avoid the erroneous success in the authentication of the another person, since both the palm collation score and the iris collation score are required to exceed the threshold values at the same time to succeed in the authentication of the user himself/herself, the probability of failure in the authentication of the user himself/herself increases as compared with the case where the authentication is performed by using the palm authentication and the iris authentication alone.

In other words, as illustrated in Fig. 7A, when the palm authentication and the iris authentication are simply combined, the window size is set to 1 to determine whether or not a reference value is exceeded at the same time. Therefore, both the palm collation score and the iris collation score do not simultaneously exceed the threshold values, and the timing t passes the timeout, and thus it is erroneously determined as the authentication failure.

In contrast, as illustrated in Fig. 7B, the determination processing unit 34 can maintain the iris collation score and the palm collation score for the certain period according to the predetermined window size, and use the maximum values in the certain period for the collation. In other words, the determination processing unit 34 calculates a collation score score (t) at the timing t according to Expression (1) below.

Accordingly, the determination processing unit 34 can correctly succeed in the authentication of the user himself/herself since both the maximum value of the palm collation score and the maximum value of the iris collation score exceed the threshold values within a period (a period of time up to the timeout) in which the user authentication process is performed.

In this way, in the HMD device 11, the probability of failure in the authentication of the user himself/herself can be reduced as compared with a case where the palm authentication and the iris authentication are simply combined.

<Processing example of user registration process>
The user registration process to be performed in the HMD device 11 will be described with reference to the flowchart illustrated in Fig. 8.

For example, when the user wearing the HMD device 11 instructs the execution of the user registration process, the process is started, and in step S11, the determination processing unit 34 initializes the numbers of frames for determining the numbers of the palm images and the iris images acquired.

After process in step S11, processes are performed in steps S12 and S13 in parallel.

In step S12, the display control unit 31 performs a display control to display the palm guide image 13 on the display screen 12 of the transmissive display 21. In step S13, the display control unit 31 performs a display control to display the line-of-sight guide image 14 on the display screen 12 of the transmissive display 21.

After the processes in steps S12 and S13, processes are performed in steps S14 and S15 in parallel.

In step S14, the out-camera 22 supplies the palm images obtained by imaging the palm of the user to the determination processing unit 34, and the determination processing unit 34 acquires the palm images for one frame. In step S15, the in-camera 23 supplies the iris image obtained by imaging the iris of the user to the determination processing unit 34, and the determination processing unit 34 acquires the iris image for one frame.

In step S16, the determination processing unit 34 determines whether or not the number of frames of the palm images acquired in step S14 and the number of frames of the iris images acquired in step S15 are greater than or equal to the prescribed number.

In a case where the determination processing unit 34 determines that the number of the frames is not greater than or equal to the prescribed number of the frames in step S16, the process proceeds to step S17. In step S17, after incrementing the number of the frames, the determination processing unit 34 returns to steps S12 and S13, and thereinafter, a similar process is repeatedly performed.

In contrast, in a case where the determination processing unit 34 determines that the number of the frames is greater than or equal to the prescribed number of the frames in step S16, the process proceeds to steps S18 and S19, and processes are performed in parallel.

In step S18, the determination processing unit 34 supplies the prescribed number of the palm images to the registration template creation unit 35. The registration template creation unit 35 extracts the feature amounts of the palm from the prescribed number of the palm images to create the palm registration template in which the feature amounts of the palm are registered. Note that, in a case where the feature amounts of the palm required for the palm authentication are difficult to be extracted from the prescribed number of the palm images, the creation of the palm registration template is not performed.

In step S19, the determination processing unit 34 supplies the prescribed number of the iris images to the registration template creation unit 35. The registration template creation unit 35 extracts the feature amounts of the iris from the prescribed number of the iris images to create the iris registration template in which the feature amounts of the iris are registered. Note that, in a case where the feature amounts of the iris required for the iris authentication are difficult to be extracted from the prescribed number of the iris images, the creation of the iris registration template is not performed.

In step S20, the determination processing unit 34 determines whether or not the creations of both the palm registration template and the iris registration template has been completed.

In a case where the determination processing unit 34 determines that the creations of both the palm registration template and the iris registration template has been completed in step S20, the process proceeds to step S21. In step S21, a process in a case of the registration success is performed. In the process, the registration template creation unit 35 registers the palm registration template and the iris registration template in a database, and the display control unit 31 causes the transmissive display 21 to display a message indicating that the user registration process has succeeded.

In contrast, in a case where the determination processing unit 34 determines that the creations of both the palm registration template and the iris registration template has not been completed, that is, in a case where either of the creations of the registration template has not been performed in step S20, the process proceeds to step S22. In step S22, a process in a case of the registration failure is performed. In the process, the display control unit 31 causes the transmissive display 21 to display, for example, a message indicating that the user registration process has failed, or a user interface for confirming re-execution of the user registration process. For example, in a case where the user registration process has failed, the user registration process can be re-executed.

After the processes in step S21 or step S22, the user registration process is terminated.

As described above, the HMD device 11 can shorten the processing period of time required for the user registration process by performing a process of acquiring the palm images and the iris images and a process of creating the palm registration template and the iris registration template in parallel.

Note that, the palm images and the iris images may be set by the number of the frames as described above, and may be set by an imaging timing × a frame rate.

<Processing example of user authentication process>
The user authentication process to be performed in the HMD device 11 will be described with reference to the flowchart illustrated in Fig. 9.

For example, when the user wearing the HMD device 11 instructs the execution of the user authentication process, the process is started, and in step S31, the determination processing unit 34 initializes a timing for determining the timeout.

After a process in step S31, the processes are performed in steps S32 and S33 in parallel.

In step S32, the display control unit 31 performs the display control to display the palm guide image 13 on the display screen 12 of the transmissive display 21. In step S33, the display control unit 31 performs the display control to display the line-of-sight guide image 14 on the display screen 12 of the transmissive display 21.

After the processes in steps S32 and S33, processes are performed in steps S34 and S35 in parallel.

In step S34, the out-camera 22 supplies the palm images obtained by imaging the palm of the user to the palm collation score calculation unit 32, and the palm collation score calculation unit 32 acquires the palm images for one frame. In step S35, the in-camera 23 supplies the iris images obtained by imaging the iris of the user to the iris collation score calculation unit 33, and the iris collation score calculation unit 33 acquires the iris images for one frame.

In step S36, the palm collation score calculation unit 32 collates the feature amounts of the palm captured in the palm images acquired in step S34 with the feature amounts of the palm registered in the palm registration template to calculate the palm collation score, and supplies the palm collation score to the determination processing unit 34. In step S37, the iris collation score calculation unit 33 collates the feature amounts of the iris captured in the iris images acquired in step S35 with the feature amounts of the iris registered in the iris registration template to calculate the iris collation score, and supplies the iris collation score to the determination processing unit 34.

In step S38, the determination processing unit 34 determines whether or not both the maximum value of the palm collation score and the maximum value of the iris collation score exceed the threshold values in the certain period according to the window size.

In a case where the determination processing unit 34 determines that both the maximum values of the palm collation score and the maximum value of the iris collation score exceed the threshold values in the certain period according to the window size in step S38, the process proceeds to step S39. In step S39, a process in a case of the authentication success is performed. In the process, the display control unit 31 causes the transmissive display 21 to display a message indicating that the user authentication process has succeeded, or various processes customized for the user are started as the user authentication process has succeeded.

In contrast, in a case where the determination processing unit 34 determines that at least one of the maximum value of the palm collation score or the maximum value of the iris collation score does not exceed the threshold value in the certain period according to the window size in step S38, the process proceeds to step S40.

In step S40, the determination processing unit 34 determines whether or not a current timing has passed the timeout.

In a case where the determination processing unit 34 determines that the current timing has not passed the timeout in step S40, the process proceeds to step S41. In step S41, after the determination processing unit 34 increments the timing, the process returns to steps S32 and S33, and thereinafter, a similar process is repeatedly performed.

In contrast, in a case where the determination processing unit 34 determines that the current timing has passed the timeout in step S40, the process proceeds to step S42. In step S42, a process in a case of the authentication failure is performed. In the process, the display control unit 31 causes the transmissive display 21 to display, for example, a message indicating that the user authentication process has failed, or a user interface for confirming re-execution of the user authentication process. For example, in a case where the user authentication process has failed, the user authentication process can be re-executed.

After the processes in step S39 or step S42, the user authentication process is terminated.

As described above, the HMD device 11 can shorten the processing period of time required for the user authentication process by performing the process of acquiring the palm images and the iris images and a process of calculating the palm collation score and the iris collation score in parallel.

Note that, colors, sizes, shapes, and the like of the palm guide image 13 and the line-of-sight guide image 14 are not limited to those described in each drawing described above, and any color, size, shape, and the like can be adopted. For example, a shape of an eye may be used as the line-of-sight guide image 14, or the line-of-sight guide image 14 with the shape of the eye may perform a blinking movement to induce blinking.

In the present embodiment, the HMD device 11 for augmented reality (AR) including the transmissive display 21 has been described, but the present technology can be applied to a wearable display device including the out-camera 22 and the in-camera 23. For example, as an HMD device for virtual reality (VR) including a non-transmissive display, a display device, may be used, that displays an image imaged by the out-camera 22 on the non-transmissive display in a video see-through manner. Alternatively, the shape may be a shape of a glasses-type display device including the out-camera 22 and the in-camera 23, that is, a so-called eyeglass. Then, various types of authentication may be combined in such manner that the palm, a palm shape, a hand vein, and the like are authenticated by the out-camera 22, and the iris, an eye line, a retina, and the like are authenticated by the in-camera 23.

In a case where only one of the creations of the palm registration template and the iris registration template has been completed in the user registration process, only the other the creation of the registration template may be performed when the user registration process is co-executed. Furthermore, in order to increase a success rate of the authentication of the user himself/herself, it is preferable that the window size is the period of time up to the timeout.

<Configuration example of computer>
Next, a series of the processes described above (an information processing method) may be performed by hardware or can be performed by software. In a case where the series of the processes is performed by the software, a program constituting the software is installed on a general-purpose computer, and the like.

Fig. 10 is a block diagram illustrating a configuration example of an embodiment of a computer on which a program for executing the series of the processes described above is installed.

The program can be recorded in advance on a hard disk 105 or ROM 103 as a recording medium incorporated in the computer.

Alternatively, the program can also be stored (recorded) in a removable recording medium 111 driven by a drive 109. Such removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include, for example, a flexible disk, a compact disc read only memory (CD-ROM), a magneto optical (MO) disk, a digital versatile disc (DVD), a magnetic disk, a semiconductor memory, and the like.

Note that, in addition to installing the program on the computer from the removable recording medium 111 as described above, the program can be downloaded to the computer via a communication network or a broadcasting network and installed on the hard disk 105 to be incorporated. In other words, for example, the program can be wirelessly transferred from a download site to the computer via an artificial satellite for digital satellite broadcasting, or can be transferred by a wire to the computer via a network such as a local area network (LAN) and the Internet.

The computer has a built-in central processing unit (CPU) 102, and an input/output interface 110 is connected to the CPU 102 via a bus 101.

When a command is inputted by the user, for example, operating an input unit 107 via the input/output interface 110, accordingly, the CPU 102 executes a program stored in the read only memory (ROM) 103. Alternatively, the CPU 102 loads a program stored in the hard disk 105 into a random access memory (RAM) 104 to execute the program.

Accordingly, the CPU 102 performs a process according to the flowchart described above or a process to be performed according to a configuration in the block diagram described above. Then, as necessary, the CPU 102 outputs a processing result from an output unit 106, or transmits the processing result from a communication unit 108, and further, causes the hard disk 105 to record the processing result, and the like, via the input/output interface 110, for example.

Note that, the input unit 107 includes a keyboard, a mouse, a microphone, and the like. Furthermore, the output unit 106 includes a liquid crystal display (LCD), a speaker, and the like.

Here, in the present description, the process to be performed by the computer in accordance with the program is not necessarily performed in time series according to orders described in the flowcharts. In other words, the process to be performed by the computer in accordance with the program includes processes to be executed in parallel or independently (for example, a parallel process or an object-based process).

Furthermore, the program may be processed by one computer (one processor) or processed in a distributed manner by a plurality of computers. Moreover, the program may be transferred to a distant computer to be executed.

Moreover, in the present description, a system means a set of a plurality of components (devices, modules (parts), and the like), and it does not matter whether or not all the components are in the same housing. Consequently, a plurality of the devices housed in separate housings and connected via the network and one device in which a plurality of the modules is housed in one housing are both systems.

Furthermore, for example, a configuration described as one device (or one processing unit) may be divided and configured as the plurality of the devices (or the processing units). On the contrary, configurations described above as the plurality of the devices (or the processing units) may be combined and configured as one device (or one processing unit). Furthermore, a configuration other than the configurations described above may be added to the configuration of each device (or each processing unit). Moreover, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or a certain processing unit) may be included in the configuration of another device (or another processing unit).

Furthermore, for example, the present technology can be configured as cloud computing in which one function is shared and jointly processed by the plurality of the devices via the network.

Furthermore, for example, the program described above can be executed by any device. In this case, the device is only required to have a necessary function (a functional block and the like) and obtain necessary information.

Furthermore, for example, each step described in the flowcharts described above can be executed by one device, or can be executed in a shared manner by the plurality of the devices. Moreover, in a case where a plurality of processes is included in one step, the plurality of the processes included in the one step can be executed by one device or shared and executed by the plurality of the devices. In other words, the plurality of the processes included in one step can also be executed as processes of a plurality of steps. On the contrary, the processes described as the plurality of the steps can also be collectively executed as one step.

Note that, in the program to be executed by the computer, the processes in steps describing the program may be executed in time series in the order described in the present description, or may be executed in parallel, or independently at a necessary timing such as when a call is made. In other words, unless there is a contradiction, the process in the each step may also be executed in an order different from the orders described above. Moreover, the processes in the steps describing the program may be executed in parallel with processes of another program, or may be executed in combination with processes of the other program.

Note that, a plurality of the present technologies that has been described in the present description can each be implemented independently as a single unit unless there is a contradiction. Certainly, any of the plurality of the present technologies can be used in any combination. For example, a part or all of the present technologies described in any of the embodiments can also be implemented in combination with a part or all of the present technologies described in other embodiments. Furthermore, a part or all of the present technologies described above may be implemented in combination with another technology not described above.

<Examples of configuration combinations>
Note that, the present technology can also have a configuration below.
(1)
An information processing apparatus including:
circuitry configured to:
acquire a first image of an iris of a user;
acquire a second image of a palm of the user; and
perform an authentication process or a registration process based on the first image and the second image,
in which the first image and the second image are acquired in parallel.
(2)
The information processing apparatus according to (1), further including:
a first camera configured to capture the first image; and
a second camera configured to capture the second image.
(3)
The information processing apparatus according to (2), in which
the first camera is further configured to perform imaging toward an inner side of the information processing apparatus, and
the second camera is further configured to perform imaging toward an outer side of the information processing apparatus.
(4)
The information processing apparatus according to (2), further including:
a transmissive display, in which
the first camera is further configured to perform imaging toward an inner side of the transmissive display, and
the second camera is further configured to perform imaging toward an outer side of the transmissive display.
(5)
The information processing apparatus according to (2), in which the first image is acquired by the first camera and the second image is acquired by the second camera.
(6)
The information processing apparatus according to any one of (1) to (5), in which the circuitry is further configured to:
initiate display of an iris guide image and a palm guide image,
in which the iris guide image and the palm guide image are simultaneously displayed.
(7)
The information processing apparatus according to (6), in which, in the registration process, the circuitry is further configured to:
change a position of the iris guide image; and
change a position, an orientation, or a shape of the palm guide image,
in which the iris guide image and the palm guide image are simultaneously changed.
(8)
The information processing apparatus according to (6), in which, in the authentication process, the circuitry is further configured to:
change a position of the iris guide image; and
change a position, an orientation, or a shape of the palm guide image,
in which the iris guide image and the palm guide image are simultaneously changed.
(9)
The information processing apparatus according to (8), in which, in the authentication process, the circuitry is further configured to:
calculate an iris collation score based on the first image; and
calculate a palm collation score based on the second image,
in which the iris collation score and the palm collation score are simultaneously calculated.
(10)
The information processing apparatus according to (9), in which, in the authentication process, the circuitry is further configured to:
fix the palm guide image and change the position of the iris guide image, based on the palm collation score being greater than a threshold value and the iris collation score being less than the threshold value.
(11)
The information processing apparatus according to (9), in which, in the authentication process, the circuitry is further configured to:
fix the iris guide image and change the position, the orientation, or the shape of the palm guide image, based on the iris collation score being greater than a threshold value and the palm collation score being less than the threshold value.
(12)
The information processing apparatus according to any one of (1) to (11), in which, in the registration process, the circuitry is further configured to:
extract iris feature amounts of the iris from a predetermined number of a plurality of the first images;
create an iris registration template in which the iris feature amounts are registered;
extract palm feature amounts of the palm from a predetermined number of a plurality of the second images; and
create a palm registration template in which the palm feature amounts are registered.
(13)
The information processing apparatus according to (12), in which the palm feature amounts and the iris feature amounts are extracted in parallel, and
In which the palm registration template and the iris registration template are created in parallel.
(14)
The information processing apparatus according to any one of (1) to (13), in which, in the authentication process, the circuitry is further configured to:
collate iris feature amounts of the iris captured in the first image with feature amounts of the iris registered in an iris registration template;
calculate an iris collation score based on the collated iris feature amounts;
collate palm feature amounts of the palm captured in the second image with feature amounts of the palm registered in a palm registration template;
calculate a palm collation score based on the collated palm feature amounts; and
determine that the authentication process is successful based on both the palm collation score and the iris collation score being greater than a threshold value,
in which the iris collation score and the palm collation score are calculated in parallel.
(15)
The information processing apparatus according to (14), in which, in the authentication process, the circuitry is further configured to:
calculate a plurality of the iris collation scores based on a plurality of the second images captured during a period of time;
calculate a plurality of the palm collation scores based on a plurality of the first images captured during the period of time; and
determine that the authentication process is successful based on both a maximum palm collation score of the plurality of the palm collation scores and a maximum iris collation score of the plurality of the iris collation scores being greater than the threshold value,
in which the plurality of the iris collation scores and the plurality of the palm collation scores are calculated in parallel.
(16)
An information processing method including:
acquiring a first image of an iris of a user;
acquiring a second image of a palm of the user; and
performing an authentication process or a registration process based on the first image and the second image,
in which the first image and the second image are acquired in parallel.
(17)
A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method including:
acquiring a first image of an iris of a user;
acquiring a second image of a palm of the user; and
performing an authentication process or a registration process based on the acquired first image and the acquired second image,
in which the first image and the second image are acquired in parallel.
(18)
The non-transitory computer-readable medium according to (17), in which the first image is acquired by a first camera and the second image is acquired by a second camera.
(19)
The non-transitory computer-readable medium according to (17), in which the executed method further includes:
displaying a palm guide image and an iris guide image,
in which the palm guide image and the iris guide image are simultaneously displayed.
(20)
The non-transitory computer-readable medium according to (19), in which the executed method further includes:
changing a position, an orientation, or a shape of the palm guide image; and
changing a position of the iris guide image,
in which the palm guide image and the iris guide image are simultaneously changed.
(21)
An information processing apparatus including:
an in-camera that images an iris of a user wearing a wearable display device to acquire iris images;
an out-camera that images a palm of the user to acquire palm images; and
an information processing unit that executes a user authentication process in which iris authentication using the iris images and palm authentication using the palm images are performed in parallel.
(22)
The information processing apparatus according to (21) described above, in which
the information processing unit has:
an iris collation score calculation unit that collates feature amounts of the iris captured in the iris images with feature amounts of an iris registered in an iris registration template created in advance, and calculates an iris collation score that is a numerical value indicating a collation result; and
a palm collation score calculation unit that collates feature amounts of the palm captured in the palm images with feature amounts of a palm registered in a palm registration template created in advance, and calculates a palm collation score that is a numerical value indicating a collation result,
and
the iris collation score calculation unit and the palm collation score calculation unit perform processes in parallel.
(23)
The information processing apparatus according to (22) described above, in which
the information processing unit further has a determination processing unit that maintains the iris collation score and the palm collation score for a certain period according to a predetermined window size, and uses maximum values in the certain period for collation.
(24)
The information processing apparatus according to (23) described above, in which
the determination processing unit determines that the user authentication process has succeeded in a case where both a maximum value of the iris collation score and a maximum value of the palm collation score in the certain period exceed threshold values.
(25)
The information processing apparatus according to (24) described above, in which
the information processing unit further has a display control unit that performs display controls of a line-of-sight guide image for guiding a direction of a line-of-sight of the user and a palm guide image for guiding a posture of the palm of the user on a display screen of the wearable display device.
(26)
The information processing apparatus according to (25) described above, in which
the display control unit performs a display control for changing the line-of-sight guide image in a case where the iris collation score is less than or equal to the threshold value.
(27)
The information processing apparatus according to (25) described above, in which
the display control unit performs a display control for changing the palm guide image in a case where the palm collation score is less than or equal to the threshold value.
(28)
The information processing apparatus according to (25) described above, in which
the display control unit performs a display control for changing both the line-of-sight guide image and the palm guide image in a case where the iris collation score is less than or equal to the threshold value and the palm collation score is less than or equal to the threshold value.
(29)
The information processing apparatus according to any one of (22) to (28) described above, in which
acquisition of the iris images by the in-camera and acquisition of the palm images by the out-camera are performed in parallel during execution of a user registration process of creating the iris registration template and the palm registration template.
(30)
The information processing apparatus according to (29) described above, in which
the information processing unit further has a registration template creation unit that performs a process of creating the iris registration template from a prescribed number of the iris images and a process of creating the palm registration template from a prescribed number of the palm images in parallel during the execution of the user registration process.
(31)
An information processing method including
by an information processing apparatus,
executing a user authentication process in which iris authentication using iris images to be supplied from an in-camera that images an iris of a user wearing a wearable display device to acquire the iris images, and palm authentication using palm images to be supplied from an out-camera that images a palm of the user to acquire the palm images are performed in parallel.
(32)
A program causing a computer of an information processing apparatus to
execute an information process including a user authentication process in which iris authentication using iris images to be supplied from an in-camera that images an iris of a user wearing a wearable display device to acquire the iris images, and palm authentication using palm images to be supplied from an out-camera that images a palm of the user to acquire the palm images are performed in parallel.

Note that, the present embodiment is not limited to the embodiments described above, and various modifications can be made without departing from the gist of the present disclosure. Furthermore, effects described in the present description are merely examples and are not limited, and other effects may also be provided.

11 HMD device
12 Display screen
13 Palm guide image
14 Line-of-sight guide image
21 Transmissive display
22 Out-camera
23 In-camera
24 Information processing apparatus
31 Display control unit
32 Palm collation score calculation unit
33 Iris collation score calculation unit
34 Determination processing unit
35 Registration template creation unit

Claims

An information processing apparatus comprising:
circuitry configured to:
acquire a first image of an iris of a user;
acquire a second image of a palm of the user; and
perform an authentication process or a registration process based on the first image and the second image,
wherein the first image and the second image are acquired in parallel.
The information processing apparatus according to claim 1, further comprising:
a first camera configured to capture the first image; and
a second camera configured to capture the second image.
The information processing apparatus according to claim 2, wherein
the first camera is further configured to perform imaging toward an inner side of the information processing apparatus, and
the second camera is further configured to perform imaging toward an outer side of the information processing apparatus.
The information processing apparatus according to claim 2, further comprising:
a transmissive display, wherein
the first camera is further configured to perform imaging toward an inner side of the transmissive display, and
the second camera is further configured to perform imaging toward an outer side of the transmissive display.
The information processing apparatus according to claim 2, wherein the first image is acquired by the first camera and the second image is acquired by the second camera.
The information processing apparatus according to claim 1, wherein the circuitry is further configured to:
initiate display of an iris guide image and a palm guide image,
wherein the iris guide image and the palm guide image are simultaneously displayed.
The information processing apparatus according to claim 6, wherein, in the registration process, the circuitry is further configured to:
change a position of the iris guide image; and
change a position, an orientation, or a shape of the palm guide image,
wherein the iris guide image and the palm guide image are simultaneously changed.
The information processing apparatus according to claim 6, wherein, in the authentication process, the circuitry is further configured to:
change a position of the iris guide image; and
change a position, an orientation, or a shape of the palm guide image,
wherein the iris guide image and the palm guide image are simultaneously changed.
The information processing apparatus according to claim 8, wherein, in the authentication process, the circuitry is further configured to:
calculate an iris collation score based on the first image; and
calculate a palm collation score based on the second image,
wherein the iris collation score and the palm collation score are simultaneously calculated.
The information processing apparatus according to claim 9, wherein, in the authentication process, the circuitry is further configured to:
fix the palm guide image and change the position of the iris guide image, based on the palm collation score being greater than a threshold value and the iris collation score being less than the threshold value.
The information processing apparatus according to claim 9, wherein, in the authentication process, the circuitry is further configured to:
fix the iris guide image and change the position, the orientation, or the shape of the palm guide image, based on the iris collation score being greater than a threshold value and the palm collation score being less than the threshold value.
The information processing apparatus according to claim 1, wherein, in the registration process, the circuitry is further configured to:
extract iris feature amounts of the iris from a predetermined number of a plurality of the first images;
create an iris registration template in which the iris feature amounts are registered;
extract palm feature amounts of the palm from a predetermined number of a plurality of the second images; and
create a palm registration template in which the palm feature amounts are registered.
The information processing apparatus according to claim 12, wherein the palm feature amounts and the iris feature amounts are extracted in parallel, and
wherein the palm registration template and the iris registration template are created in parallel.
The information processing apparatus according to claim 1, wherein, in the authentication process, the circuitry is further configured to:
collate iris feature amounts of the iris captured in the first image with feature amounts of the iris registered in an iris registration template;
calculate an iris collation score based on the collated iris feature amounts;
collate palm feature amounts of the palm captured in the second image with feature amounts of the palm registered in a palm registration template;
calculate a palm collation score based on the collated palm feature amounts; and
determine that the authentication process is successful based on both the palm collation score and the iris collation score being greater than a threshold value,
wherein the iris collation score and the palm collation score are calculated in parallel.
The information processing apparatus according to claim 14, wherein, in the authentication process, the circuitry is further configured to:
calculate a plurality of the iris collation scores based on a plurality of the second images captured during a period of time;
calculate a plurality of the palm collation scores based on a plurality of the first images captured during the period of time; and
determine that the authentication process is successful based on both a maximum palm collation score of the plurality of the palm collation scores and a maximum iris collation score of the plurality of the iris collation scores being greater than the threshold value,
wherein the plurality of the iris collation scores and the plurality of the palm collation scores are calculated in parallel.
An information processing method comprising:
acquiring a first image of an iris of a user;
acquiring a second image of a palm of the user; and
performing an authentication process or a registration process based on the first image and the second image,
wherein the first image and the second image are acquired in parallel.
A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising:
acquiring a first image of an iris of a user;
acquiring a second image of a palm of the user; and
performing an authentication process or a registration process based on the acquired first image and the acquired second image,
wherein the first image and the second image are acquired in parallel.
The non-transitory computer-readable medium according to claim 17, wherein the first image is acquired by a first camera and the second image is acquired by a second camera.
The non-transitory computer-readable medium according to claim 17, wherein the executed method further comprises:
displaying a palm guide image and an iris guide image,
wherein the palm guide image and the iris guide image are simultaneously displayed.
The non-transitory computer-readable medium according to claim 19, wherein the executed method further comprises:
changing a position, an orientation, or a shape of the palm guide image; and
changing a position of the iris guide image,
wherein the palm guide image and the iris guide image are simultaneously changed.