JP2007049304A - Personal digital assistant - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a personal digital assistant capable of shortening the time required for authentication processing. <P>SOLUTION: The authentication processing is completed during a series of authentication processing based on a photographed image with the failure of the extraction of characteristic information in a prescribed number of authentication processing (Yes in step S103) as completion conditions (step S105), thus preventing the authentication processing by the photographed image from being repeated in environment. Authentication does not succeed even if a face image is photographed repeatedly, and hence shortening the time required for the authentication processing. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a portable information terminal device, and more particularly to an improvement of a portable information terminal device that performs an authentication process based on a captured image.

  Some portable information terminal devices such as cellular phones include a camera that captures a subject and generates image data. In this type of portable information terminal device, recently, it is urged to capture a face image by the camera during operation, and feature information such as the shape and size of eyes, nose, mouth and contour extracted from the captured face image, Some of them are designed to perform authentication processing (so-called face authentication) by comparing with user feature information stored in advance as registered feature information in a memory provided in the portable information terminal device. As a result of the authentication process, only when the degree of coincidence between the feature information extracted from the captured image and the feature information of the user exceeds the recognition threshold, it is determined that the person who is operating at that time is a true user, By permitting execution of subsequent processing, security can be improved and privacy can be protected (see, for example, Patent Document 1).

In the technique disclosed in Patent Literature 1, when face authentication fails, the person who performs the operation is requested to input a personal identification number. Therefore, when face authentication is performed when a true user is performing an operation and the face authentication fails because the surroundings are dark when the face image is captured, the user enters the password. Subsequent processing can be executed. On the other hand, when a person other than the user is performing the operation, after the face authentication fails, the correct password cannot be input in response to the password input request, and the subsequent processing may be executed. Can not.
JP 2000-278658 A

  When the authentication fails in the authentication process, the applicant of the present application automatically performs the authentication process by photographing again, and when the true user performs an operation, the authentication fails in the first authentication process. Even in this case, an application has been filed first for a configuration in which authentication processing is performed again without the user performing a shooting operation (for example, Japanese Patent Application No. 2005-212806). In this application specification, if the authentication fails, the authentication process is repeated continuously until a predetermined time (for example, 3 seconds) elapses and a timeout occurs, and the authentication process is performed when the authentication is successful. It is going to end.

  In such a configuration, for example, the authentication process is repeatedly performed even in an environment where the surroundings are dark at the time of face authentication and authentication is not successful no matter how many times a face image is taken. Therefore, even if a configuration is adopted in which a password input request is made after a series of authentication processes is completed, the above-mentioned waiting time is required until the password input request is made, and the authentication process is There is a problem that it takes time to finish.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a portable information terminal device that can shorten the time required for authentication processing.

  According to a first aspect of the present invention, there is provided a portable information terminal device comprising: an imaging means for photographing a subject; a feature information storage means for previously storing feature information extracted from a face image as registered feature information; The feature information is extracted from the captured image, and the feature information is compared with the registered feature information to control the authentication processing means, the image pickup means, and the authentication processing means. In the case of failure, the image is taken again, and an authentication process based on the photographed image is performed, thereby providing a continuous authentication means for performing a series of authentication processes. The continuous authentication means has a predetermined number of times during the series of authentication processes. The authentication process is terminated on the condition that the feature information extraction has failed in the authentication process.

  According to such a configuration, when extraction of feature information fails in a predetermined number of authentication processes during a series of authentication processes, the authentication process using a captured image can be stopped. Therefore, it is possible to prevent the authentication process using the photographed image from being repeatedly performed in an environment where authentication is not successful no matter how many times the face image is photographed, for example, when the surroundings are dark. The time required can be shortened.

  In the portable information terminal device according to the second aspect of the present invention, the termination condition is that feature information extraction has failed in a predetermined number of consecutive authentication processes two or more times.

  According to such a configuration, it is possible to stop the authentication process using the photographed image when the feature information extraction has failed continuously a predetermined number of times. If feature information extraction fails continuously, there is a high possibility that the environment will not succeed even if the face image is taken many times, such as the surroundings are dark. Can be stopped appropriately.

  A portable information terminal device according to a third aspect of the present invention comprises a personal identification number authentication processing means for performing an authentication process by inputting a personal identification number when the termination condition is satisfied and the authentication process is terminated.

  According to such a configuration, authentication can be performed by inputting a personal identification number after the authentication process using the photographed image is stopped. Since the authentication process by entering the security code can be performed even in an environment where the surroundings are dark, the true user can quickly authenticate and execute the subsequent process by entering the security code. Can do.

  According to a fourth aspect of the present invention, there is provided a portable information terminal device comprising fingerprint authentication processing means for performing authentication processing by fingerprint input when the end condition is satisfied and the authentication processing ends.

  According to such a configuration, it is possible to perform authentication by inputting a fingerprint after the authentication process using the photographed image is stopped. Since authentication processing by fingerprint input can be performed even in an environment where the surroundings are dark, a true user can quickly perform authentication and perform subsequent processing by performing fingerprint input. .

  According to the portable information terminal device of the present invention, it is possible to prevent the authentication process using the photographed image from being repeatedly performed in an environment where the authentication is not successful no matter how many times the face image is photographed. The time required can be shortened. If the configuration is such that authentication processing using a captured image is stopped when feature information extraction fails for a predetermined number of times, the authentication processing can be stopped more appropriately.

  FIG. 1 is an external view showing an example of a portable information terminal device according to an embodiment of the present invention. A mobile phone 1 is shown as an example of the portable information terminal device. The mobile phone 1 is a so-called foldable mobile phone, in which a display housing 100 and an operation housing 200 are connected via a hinge part 300, and are folded with one surface of the display housing 100 and the operation housing 200 facing each other. be able to.

  In the display housing 100, a main display unit 101, a receiver for receiver 103, and a sub camera 105 are disposed on the inner housing surface when folded, and a sub display portion and a main camera 104 are disposed on the outer housing surface. . In addition, the operation casing 200 has a large number of operation keys 201 and a microphone for transmitting 203 arranged on the inner casing surface when folded. Such a foldable mobile phone 1 can be carried in a compactly folded state, and if the casing is unfolded, the operation key 201 can be pressed while viewing the display on the main display unit 101. . That is, main information display is performed using the main display unit 101, and main operation input is performed using the operation keys 201.

  When the user is operating the operation key 201 while viewing the display on the main display unit 101, the cellular phone 1 captures the user's face image by the sub camera 105 and extracts feature information extracted from the face image. As a result of performing the authentication process based on the above, it has a function (so-called face authentication function) that permits the execution of the subsequent process only when it is determined that the person who is operating at that time is a true user Yes. By performing authentication processing when executing a highly confidential function, security can be improved and privacy can be protected.

  FIG. 2 is a block diagram showing an example of the internal configuration of the mobile phone 1 shown in FIG. The control unit 400 is a processor that controls main operations of the mobile phone 1, and includes an imaging unit drive control unit 401, an authentication processing unit 402, a continuous authentication unit 403, and an alternative authentication processing unit 404. An operation signal from the operation key 201 is input to the control unit 400, and a memory 410, an image input / output unit 430, an audio input / output unit 440, and a communication unit 450 are connected to be able to input / output.

  The image input / output unit 430 includes a main display unit 101, a sub display unit 102, a main camera 104, and a sub camera 105. The main camera 104 and the sub camera 105 are each unitized as an imaging unit. These imaging units include a photoelectric conversion element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a control. A circuit is provided. Here, a photoelectric conversion element that needs to perform initialization processing such as adjustment of offset and gain after power-on of the imaging unit is used.

  The voice input / output unit 440 is provided with a receiving receiver 103 and a transmitting microphone 203. The communication unit 450 transmits / receives audio data, image data, and the like by transmitting / receiving radio waves to / from a base station (not shown).

  The imaging unit drive control unit 401 controls driving of the main camera 104 and the sub camera 105 by transmitting a signal to the image input / output unit 430. In this embodiment, the imaging unit drive control unit 401 controls the driving of the sub camera 105, so that the sub camera 105 captures a face image of a person who operates the operation key 201 while looking at the main display unit 101. The authentication process is performed based on the feature information extracted from the face image. However, the configuration may be such that authentication processing is performed by photographing a face image with the main camera 104.

  The authentication processing unit 402 compares the feature information extracted from the captured image with the feature information (registered feature information) of the user stored in advance in the memory 410, so that the person who is operating at that time is true It is determined whether or not the user is a user. The continuous authentication unit 403 controls the imaging unit drive control unit 401 and the authentication processing unit 402 to perform imaging again when authentication fails, and performs authentication processing based on the captured image. The alternative authentication processing unit 404 performs an alternative authentication process different from the face authentication.

  A display screen data storage unit 411, an image data storage unit 412, a feature information storage unit 413, and an error history information storage unit 414 are allocated to the memory 410. The display screen data storage unit 411 stores screen data to be displayed on the main display unit 101 and the sub display unit 102. The image data storage unit 412 stores image data captured by the main camera 104 and the sub camera 105, image data received from the outside via the communication unit 450, and the like.

  The feature information storage unit 413 stores feature information of face images used for authentication processing. The feature information storage unit 413 can store feature information extracted from two or more face images as registered feature information up to a predetermined maximum number (for example, seven) for each face image. By performing face authentication based on registered feature information extracted from two or more face images, the probability (authentication acceptance rate) of successful authentication when a true user performs face authentication can be increased. It is possible to prevent authentication failure due to subtle differences such as facial expressions and the angle of light. However, the registered feature information used for face authentication is not limited to a configuration that is extracted from seven face images, but is a configuration that performs face authentication based on registered feature information extracted from one face image, for example. There may be.

  The user's face image is previously captured by the user using the main camera 104 or the sub camera 105, and the image data is stored in the image data storage unit 412 of the memory 410. Since face authentication is performed based on registered feature information extracted from these image data and stored in the feature information storage unit 413 of the memory 410, the image data itself is not necessary for face authentication. If it is stored in 410, the user can read the image data and check the face image, so that convenience is improved.

  In the error history information storage unit 414, when authentication fails, information at the time of authentication is stored as error history information.

  However, the facial image feature information and facial image image data used for the authentication process are stored in a storage medium (not shown) that is detachable from the mobile phone 1 such as a SIM (Subscriber Identity Module) card. Such a configuration may be used.

  FIG. 3 is a block diagram illustrating a configuration example of the imaging unit drive control unit 401 in FIG. The imaging unit drive control unit 401 includes an imaging unit activation unit 407, an imaging control unit 408, and an imaging unit stop unit 409. The imaging unit activation unit 407 transmits control signals to the main camera 104 and the sub camera 105 to activate them. The imaging unit 106 (the main camera 104 or the sub camera 105) is activated based on a user operation. Are provided with a power-on unit 407A for turning on power and an initialization processing unit 407B for executing initialization processing.

  The imaging control unit 408 controls the imaging operation of the main camera 104 or the sub camera 105 based on the input signal from the initialization processing unit 407B or the imaging operation by the user, and thereby the main camera 104 or the sub camera 105 at that time. Still image data of a subject for is generated. The imaging unit stop unit 409 stops driving the imaging units 106 by stopping the power supplied to the main camera 104 or the sub camera 105.

  FIG. 4 is a block diagram illustrating a configuration example of the authentication processing unit 402 in FIG. The authentication processing unit 402 includes a feature extraction unit 402A, a matching degree calculation unit 402B, and a matching degree determination unit 402C. When the authentication process is started, feature information such as shape and size of each part of the face such as eyes, nose, mouth and contour is extracted from the face image taken by the sub camera 105 by the feature extraction unit 402A. .

  When the photographed image is a landscape image that is not a face image, or when the surroundings are dark, feature information cannot be extracted from the photographed image, and authentication fails. On the other hand, when the feature information is successfully extracted from the photographed image, the feature information is compared with the registered feature information stored in the feature information storage unit 413 of the memory 410 by the coincidence degree calculation unit 402B. The degree of matching is calculated.

  The coincidence determination unit 402C compares the coincidence calculated by the coincidence calculation unit 402B with a predetermined threshold (authentication threshold), and outputs the comparison result. At this time, if the degree of coincidence exceeds the authentication threshold, it is determined that the user is a true user and the authentication is successful. If the degree of coincidence is smaller than the authentication threshold, the user is not determined to be a true user. Authentication fails. Here, the degree of coincidence is a ratio (%) occupied by the matched feature information among all the compared feature information, and the authentication threshold is usually set to a predetermined value of 90% or more.

  FIG. 5 is a diagram illustrating a display example of the main display unit 101 when setting related to face authentication is performed. When the user reads the security setting screen by operating the operation key 201 (see FIG. 5A) and selects and operates the “face authentication setting” item from the items displayed on the security setting screen, the mobile phone A screen that prompts the user to input a terminal security code assigned to the telephone 1 is displayed on the main display unit 101 (see FIG. 5B). When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  The terminal security code is composed of, for example, an arbitrary character string of 4 to 8 digits, and is set in advance when the mobile phone 1 is purchased, but can be changed by the user operating the operation key 201. When the character string input by operating the operation key 201 does not match the terminal security code set in advance on the terminal security code input screen of FIG. 5B, a message to that effect is displayed on the main display unit 101. (See FIG. 5C). When the character string input on the terminal security code input screen is 3 digits or less, a message indicating that a 4 to 8 character string should be input is displayed on the main display unit 101 (FIG. 5D). reference).

  After the display as shown in FIG. 5C or 5D is displayed on the main display unit 101, a predetermined time elapses or a time-out occurs, or any key included in the operation key 201 is displayed. When operated, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  When the character string input on the terminal personal identification number input screen in FIG. 5B matches the terminal personal identification number set in advance, the display on the main display unit 101 is turned on / off in the mode for performing face authentication. The screen is switched to a face authentication use setting screen for setting OFF (see FIG. 5E). When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the terminal security code input screen shown in FIG. When the face authentication is turned off by operating the operation key 201 on the face authentication use setting screen, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  When face authentication is turned on by operating the operation key 201 on the face authentication use setting screen of FIG. 5E, the display of the main display unit 101 is switched to a face authentication setting screen for performing settings related to face authentication ( (Refer FIG.5 (f)). In the face authentication setting screen of FIG. 5F, items associated with images (face authentication images) representing the user from which registered feature information is extracted, and authentication threshold values at the time of face authentication (security) The item associated with (level setting) and the item associated with the error history are included.

  When an item associated with the face authentication image is selected on this display screen, a screen (face authentication image registration screen) for overwriting registration and confirmation of the face authentication image is displayed. When an item associated with the security level setting is selected, a screen (security level setting screen) for setting an authentication threshold for face authentication is displayed. When an item associated with the error history is selected, a screen for displaying the error history (error history screen) is displayed.

  After performing these various settings, when the operation key 201 is operated to complete the face authentication setting, the display on the main display unit 101 is changed to a display as shown in FIG. 5 (g) or FIG. 5 (h). Switch. That is, if at least one face authentication image is registered, the main display unit 101 displays that face authentication is performed based on the feature information extracted from the face authentication image (see FIG. 5G). ) When the confirmation key 111 included in this screen is selected, the display of the main display unit 101 returns to the security setting screen shown in FIG. On the other hand, if no face authentication image has been registered, the main display unit 101 displays that the face authentication image should be registered (see FIG. 5H), and the confirmation included in this screen. When the key 112 is selected, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG.

  FIG. 6 shows a display example of the face authentication image registration screen. When any item associated with the face authentication image is selected on the face authentication setting screen of FIG. 5 (f), the main display unit 101 displays the face authentication as shown in FIG. 6 (a). The screen is switched to the face authentication image registration screen corresponding to the image for use. The face authentication image registration screen includes an “overwrite registration” item and a “registration data confirmation” item.

  When the “overwrite registration” item is selected and operated from the items displayed on the face authentication image registration screen in FIG. 6A, the sub camera 105 is activated and the sub camera 105 is ready for photographing. After that, an image of the subject with respect to the sub camera 105 is displayed on the main display unit 101 in real time (see FIG. 6B). At this time, the main display unit 101 functions as a finder of the sub camera 105, and the face of the person who is performing the operation while viewing the main display unit 101 is displayed on the finder screen configured by the main display unit 101. It becomes.

  When the user performs registration operation by operating the enter key included in the operation key 201 after the user performs position alignment of the user's face on the finder screen in FIG. 6B, the feature extraction unit of the control unit 400 402A performs a process of extracting feature information from the image photographed by the sub camera 105 at that time.

  At this time, if the user's face is properly aligned, feature information such as the shape and size of the eyes, nose, mouth and contour can be extracted. However, the alignment is appropriately performed. If the user's face is not displayed on the main display unit 101, the feature information cannot be extracted. When the feature information is successfully extracted, the feature information is stored in the feature information storage unit 413 of the memory 410, and the face image at that time is used as a face authentication image in the image data storage unit 412 of the memory 410. Stored in

  When the feature information is successfully extracted, the fact that the feature information based on the face image is stored (registered) in the memory 410 is displayed on the main display unit 101 (see FIG. 6C). When the user selects the confirmation key 113 included in this display screen, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG. On the other hand, if the feature information extraction fails, the fact and the cause of the feature information extraction failure are displayed on the main display unit 101 (see FIG. 6D), and the confirmation key 114 included in this display screen is displayed. When the user selects, the display on the main display unit 101 returns to the viewfinder screen shown in FIG. 5B, and a face image can be taken again by the sub camera 105.

  When the item “Registered data confirmation” is selected and operated from the items displayed on the face authentication image registration screen of FIG. 6A, the corresponding face authentication for use stored in the image data storage unit 412 of the memory 410 is displayed. An image is read and displayed on the main display unit 101 (see FIG. 6E). The user can confirm the registered face authentication image on the face authentication image confirmation screen shown in FIG. When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the face authentication image registration screen shown in FIG.

  When the user performs an operation of deleting the corresponding face authentication image by operating the operation key 201 on the face authentication image confirmation screen in FIG. 6E, the display on the main display unit 101 is the face authentication. The screen is switched to a deletion confirmation screen for confirming whether or not to delete the image for use (see FIG. 6F). This display screen includes a “Yes” key 115 for instructing that the face authentication image should be deleted, and a “No” key 116 for instructing not to delete the face authentication image. When the “No” key 116 is selected by operating the operation key 201 on the deletion confirmation screen in FIG. 6F, the display on the main display unit 101 is the face authentication image shown in FIG. Return to the confirmation screen.

  When the “Yes” key 115 is selected by operating the operation key 201 on the deletion confirmation screen in FIG. 6F, the corresponding face authentication image is stored in the image data storage unit 412 and the feature information storage unit 413 of the memory 410. In addition, a process for deleting the registered feature information is started and a message indicating that the feature is being deleted is displayed on the main display unit 101 (see FIG. 6G). Then, when the processing for deleting the face authentication image and the registered feature information is completed, a message to that effect is displayed on the main display unit 101 (see FIG. 6 (h)), and a predetermined time elapses or a timeout occurs, or In response to any one of the keys included in the operation key 201 being operated, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG.

  FIG. 7 is a view showing a display example of the security level setting screen. When an item associated with the security level setting is selected on the face authentication setting screen shown in FIG. 5F, the display on the main display unit 101 sets an authentication threshold for face authentication as shown in FIG. Switches to the security level setting screen. As the security level, “level 3” having the highest authentication threshold, “level 2” having an authentication threshold lower than “level 3”, and “level 1” having an authentication threshold lower than “level 2” are set in advance. Any security level can be selected on the security level setting screen.

  FIG. 8 is a diagram showing a display example of the error history screen. When the item associated with the error history is selected on the face authentication setting screen of FIG. 5F, the display of the main display unit 101 is displayed on the error history screen for displaying the error history as shown in FIG. Switch. On the error history screen, time information indicating the date and time when authentication failed is displayed as a list of error history information for a predetermined number of times (for example, nine times). When the authentication history is newly failed while the error history information for the predetermined number of times is stored in the error history information storage unit 414 of the memory 410, the oldest error history information is deleted, and the new error history information is stored in the memory. 410 and updated so that the new error history information is displayed on the error history screen.

  FIG. 9 is a diagram illustrating a display example of the main display unit 101 when performing face authentication. In this example, face authentication is performed when the secret mode included in the management function of the mobile phone 1 is switched on / off. When incognito mode is on, not only normal data but also data registered in advance as secret data is displayed. When incognito mode is off, secret data is not displayed and only normal data is displayed. Will be displayed.

  In the management function screen for setting the management function of the mobile phone 1 as shown in FIG. 9A, when the “secret mode” item is selected and operated from the items displayed on the management function screen, the sub camera 105 is activated. After the sub camera 105 is ready for shooting, an image of the subject with respect to the sub camera 105 is displayed on the main display unit 101 in real time (see FIG. 9B), and automatic authentication is started. At this time, when the initialization process of the sub-camera 105 is completed and the camera is ready for shooting, a face image for authentication is shot without the user performing a shooting operation. Is extracted. Then, the extracted feature information is compared with the user feature information stored in advance in the feature information storage unit 413 of the memory 410, whereby authentication processing is automatically performed.

  In this authentication process, when the degree of coincidence between the feature information of the extracted face image and the user feature information stored in the memory 410 exceeds an authentication threshold set in advance on the security level setting screen, It is determined that the operator is a true user. In this case, the user's face image displayed on the main display unit 101 is highlighted by so-called action focus, thereby informing that the authentication has been successful by a visual effect.

  Thereafter, a message indicating that the authentication has succeeded is displayed on the main display unit 101 (see FIG. 9C), and after a predetermined time has elapsed, the screen is switched to a screen for switching on / off the secret mode (FIG. 9D). reference). On the other hand, when the degree of coincidence of the feature information is smaller than the authentication threshold and the authentication fails, the feature information is automatically extracted again from the image photographed by the sub camera 105 at that time, and the feature information is stored in the memory 410. Is compared with the feature information of the user stored in.

  As described above, in this embodiment, when the authentication fails, the authentication process is repeatedly performed continuously until a predetermined time (for example, 3 seconds) elapses and a timeout occurs, and the authentication process ends when the authentication is successful. To do. That is, a series of authentication processes is performed until the end condition is satisfied, with the end condition being successful authentication or timeout.

  For example, when the timeout time is set to 3 seconds, the authentication process is performed about 7 times at the maximum. That is, the time required for one authentication process is about 0.4 seconds, and if the authentication is successful in the first authentication process, the process proceeds to the next process with almost no waiting time after the initialization process of the sub camera 105 is completed. Can do. While the authentication process is being performed repeatedly, a finder screen is displayed on the main display unit 101, and an image of the subject with respect to the sub camera 105 is displayed in real time.

  When the authentication process is repeatedly performed, each time the authentication process is performed a plurality of times, a display corresponding to the matching degree of the feature information is displayed on the main display unit 101 as a comparison result of the feature information in the authentication process. More specifically, the color of the frame 117 of the finder screen displayed on the main display unit 101 is displayed in a color corresponding to the matching degree of the feature information. As a result, since the comparison result of the degree of coincidence of the feature information at the time of the authentication process is displayed, based on the display, whether or not the user's own face image taken at the time of the authentication process is an easily recognizable image. Can be confirmed.

  However, the present invention is not limited to such a configuration, and a configuration in which the degree of coincidence of the feature information is level-displayed on the main display unit 101 using a numerical value or a graph may be used. Further, the present invention is not limited to the configuration in which the comparison results are displayed for all authentication processes, and may be any display that displays the comparison results for at least one authentication process.

  When a predetermined time elapses after the start of automatic authentication and a timeout occurs, a message indicating that the authentication has failed is displayed on the main display unit 101 (see FIG. 9E). When the user selects the re-authentication key 118 included in this display screen, the display on the main display unit 101 returns to the finder screen shown in FIG. 9B, and automatic authentication is started again. On the other hand, when the user selects the terminal security code input key 119 displayed on the main display unit 101 in FIG. 9E, the display of the main display unit 101 is switched to the terminal security code input screen that prompts the user to enter the terminal security code. (Refer FIG.9 (f)).

  When the character string input on the terminal personal identification number input screen of FIG. 9F matches the terminal personal identification number set in advance, the display on the main display unit 101 switches on / off of the secret mode. The screen is switched to the screen for the purpose (see FIG. 9D).

  On the other hand, when the character string input by the operation of the operation key 201 does not match the terminal security code set in advance on the terminal security code input screen of FIG. Is displayed (see FIG. 9G). When the character string input on the terminal security code input screen is 3 digits or less, a message indicating that a 4 to 8 character string should be input is displayed on the main display unit 101 (FIG. 9 (h)). reference). Then, after the display as shown in FIG. 9G or FIG. 9H is performed, a predetermined time elapses and a time-out occurs, or any key included in the operation key 201 is operated. And the display of the main display part 101 returns to the management function screen shown to Fig.9 (a).

  According to the configuration in which the authentication process is automatically performed again when the authentication process fails in the authentication process as in the present embodiment, the true authentication is performed by the first authentication process when the true user performs an operation. Even if it is not determined that the user is the user, the authentication process is performed again without the user performing a shooting operation. In other words, since the authentication process is automatically and repeatedly performed until it is determined that the user is a true user, the labor required for the authentication process can be reduced and the time required for the authentication process can be shortened.

  In face authentication as described above, if authentication is not successful, a series of authentication processing is performed until timeout, so when performing face authentication in an environment where feature information extraction is difficult, such as when the surroundings are dark, Even if the face image is taken many times, the authentication process is repeated until the time-out occurs even though the authentication is not successful. Therefore, in the present embodiment, exceptionally, the authentication process by the continuous authentication unit 403 is stopped and the alternative authentication unit 404 is set on the condition that the authentication has failed in a predetermined number of consecutive authentication processes in a series of authentication processes. Therefore, authentication processing different from face authentication is performed.

  FIG. 10 is a flowchart illustrating an example of the authentication process. In this example, the feature information extraction based on the captured image and the authentication based on the extracted feature information (coincidence calculation) until the time-out occurs after the authentication process by the face authentication is started (Yes in step S101). And a comparison with the authentication threshold value) are repeated, a series of authentication processes are performed.

  If the feature information based on the photographed image is successfully extracted (Yes in step S102), authentication based on the extracted feature information is performed. If the authentication is successful (Yes in step S104), execution of the subsequent processing is permitted (step S108). In the example of the secret mode, a screen for switching on / off the secret mode (FIG. 9D). )) Is displayed on the main display unit 101.

  If the feature information extraction based on the photographed image has failed (No in step S102), it is determined whether the feature information extraction has failed twice in succession during the series of authentication processes (step S103). . If feature information extraction fails twice in succession (Yes in step S103), face authentication is stopped (step S105). In this case, a terminal security code input screen for inputting the terminal security code is displayed on the main display unit 101 (step S106). As an authentication process different from the face authentication, the authentication process based on the input of the terminal security code is an alternative authentication unit. This is executed by 404.

  The terminal security code input screen displayed on the main display unit 101 is the same as that described with reference to FIG. However, when the authentication process is canceled, the PIN code input screen is automatically displayed on the screen, and the PIN code input is not limited to the configuration in which the authentication process can be performed by the PIN code input. The configuration may be such that the authentication process can be performed.

  When the character string input on the terminal security code input screen matches the terminal security code set in advance (Yes in step S107), execution of subsequent processing is permitted (step S108), and the secret mode is set. In the example, a screen (see FIG. 9D) for switching on / off of the secret mode is displayed on the main display unit 101.

  In this embodiment, when feature information extraction fails in a predetermined number of times (for example, twice) during a series of authentication processes in an environment where feature information extraction is difficult, for example, when the surroundings are dark, etc. Can stop the authentication process using the photographed image. Accordingly, it is possible to prevent the authentication process using the photographed image from being repeatedly performed in an environment where the authentication is not successful no matter how many times the face image is photographed.

  In addition, when the feature information extraction fails continuously for a predetermined number of times (for example, twice), the authentication process using the captured image can be stopped. If feature information extraction fails continuously, there is a high possibility that authentication will not succeed even if the face image is taken many times, such as the surroundings are dark. Can be canceled.

  In the present embodiment, authentication can be performed by inputting a personal identification number after the authentication processing using the photographed image is stopped. Since the authentication process by entering the security code can be performed even in an environment where the surroundings are dark, the true user can quickly authenticate and execute the subsequent process by entering the security code. Can do.

  However, the authentication process performed by the alternative authentication unit 404 is not limited to the authentication process based on the password input, but may be another alternative authentication process different from the authentication process based on the photographed image, for example, the authentication process based on fingerprint input. . In the authentication process based on fingerprint input, for example, light can be emitted toward the user's finger that touches the contact surface, and the fingerprint can be recognized based on the reflected light. That is, in the authentication process based on fingerprint input, the user's finger contacts the contact surface, so that the authentication process can be performed even in an environment where the surroundings are dark. Since the method of authentication processing by fingerprint input is a well-known technique, detailed description thereof will be omitted here.

  The alternative authentication unit 404 is not limited to the configuration in which the authentication process is stopped with the failure to extract the feature information in two consecutive authentication processes as an end condition, and the feature information can be extracted in one authentication process. The configuration may be such that the authentication processing is canceled when it fails, or the authentication processing is stopped when the feature information extraction fails in the predetermined number of authentication processing three or more times. May be.

  In addition, when adopting a configuration in which authentication processing is stopped when feature information extraction fails in two or more predetermined times of authentication processing, feature information is extracted in a predetermined number of consecutive authentication processing. The configuration may be such that the authentication process is stopped when the total number of authentication processes for which feature information extraction has failed in a series of authentication processes reaches a predetermined number of times, not limited to the case of failure.

  It is preferable that the shutter sound is not generated when a face image is captured in the authentication process. With such a configuration, it is possible to prevent misunderstandings such as voyeurism due to the generation of shutter sound when authentication processing is performed in an environment where there are people around the train or the like. In the present embodiment, since the shooting operation is not performed during the authentication process, it is possible to prevent misunderstanding such as voyeurism by performing the shooting operation.

  However, when it is not determined that the user is a true user within a predetermined time, the configuration is not limited to a configuration in which a timeout occurs, and a configuration in which authentication processing is repeatedly performed until authentication is successful may be used.

  Face authentication is not limited to automatic authentication in which authentication is started without a user's photographing operation, and is configured such that photographing is performed according to the user's photographing operation and face authentication is performed based on the photographed image. Also good.

  Face authentication is not limited to switching on / off of the secret mode, but other functions with high security and privacy, such as card setting, incoming call restriction, lock setting, PIN change, caller ID notification / non notification It can be performed during various setting operations relating to setting, setting reset, software update, own station number details display, and other data changes.

  The present invention is not limited to the contents of the above-described embodiment, and various modifications can be made within the scope of the claims.

1 is an external view showing an example of a portable information terminal device according to an embodiment of the present invention, and a mobile phone is shown as an example of a portable information terminal device. FIG. 2 is a block diagram showing an example of an internal configuration of the mobile phone shown in FIG. 1. FIG. 3 is a block diagram illustrating a configuration example of an imaging unit drive control unit in FIG. 2. FIG. 3 is a block diagram illustrating a configuration example of an authentication processing unit in FIG. 2. It is the figure which showed the example of a display of the main display part at the time of setting regarding face authentication. It is the figure which showed the example of a display of the image registration screen for face authentication. It is the figure which showed the example of a display of a security level setting screen. It is the figure which showed the example of a display of an error history screen. It is the figure which showed the example of a display of the main display part at the time of performing face authentication. It is the flowchart which showed an example of the authentication process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Cellular phone 101 Main display part 102 Sub display part 103 Receiver for receiver 104 Main camera 105 Sub camera 201 Operation key 203 Microphone for transmission 400 Control part 401 Imaging unit drive control part 402 Authentication process part 402A Feature extraction part 402B Concordance calculation Unit 402C coincidence degree determination unit 403 continuous authentication unit 404 alternative authentication processing unit 407 imaging unit activation unit 407A power-on unit 407B initialization processing unit 408 imaging control unit 409 imaging unit stop unit 410 memory 411 display screen data storage unit 412 image data storage Unit 413 feature information storage unit 414 error history information storage unit 450 communication unit

Claims (4)

Imaging means for photographing a subject;
Feature information storage means for previously storing feature information extracted from the face image as registered feature information;
Authentication processing means for extracting feature information from an image captured by the imaging means and performing authentication processing by comparing the feature information with the registered feature information;
A continuous authentication unit that controls the imaging unit and the authentication processing unit to perform a series of authentication processing by performing imaging again when authentication fails in the authentication processing and performing authentication processing based on the captured image; With
The portable authentication terminal device, wherein the continuous authentication means terminates the authentication process on the condition that the feature information extraction has failed in the authentication process a predetermined number of times during the series of authentication processes.   2. The portable information terminal device according to claim 1, wherein the termination condition is that feature information extraction has failed in a predetermined number of consecutive authentication processes of two or more times.   3. The portable information terminal device according to claim 1, further comprising: a personal identification number authentication processing unit configured to perform an authentication process by inputting a personal identification number when the termination condition is satisfied and the authentication process is terminated. 4.   3. The portable information terminal device according to claim 1, further comprising fingerprint authentication processing means for performing authentication processing by fingerprint input when the end condition is satisfied and the authentication processing is ended.

Images