JP4646731B2 - Portable information terminal device - Google Patents

Description

  The present invention relates to a portable information terminal device, and more particularly to an improvement of a portable information terminal device that performs an authentication process based on a captured image.

  Some portable information terminal devices such as cellular phones include a camera that captures a subject and generates image data. In this type of portable information terminal device, recently, it is urged to capture a face image by the camera during operation, and feature information such as the shape and size of eyes, nose, mouth and contour extracted from the captured face image, Some of them are designed to perform authentication processing (so-called face authentication) by comparing with user feature information stored in advance as registered feature information in a memory provided in the portable information terminal device. As a result of the authentication process, only when the degree of coincidence between the feature information extracted from the captured image and the feature information of the user exceeds the recognition threshold, it is determined that the person who is operating at that time is a true user, By permitting execution of subsequent processing, security can be improved and privacy can be protected (see, for example, Patent Document 1).

However, when a long period of time has elapsed since the registration feature information was stored in the memory, even if the true user is performing an operation, for example, due to changes in the user's body shape or aging, etc. There is a possibility that the degree of coincidence between the feature information extracted from the image and the registered feature information is lowered, and authentication fails. Therefore, in the field of face authentication technology, the feature information extracted from the photographed image when authentication was successful is replaced with the registered feature information stored in the memory at that time, thereby automatically updating the registered feature information. Techniques have been proposed (for example, Patent Documents 2 and 3).
JP 2000-278658 A JP-A-11-167632 JP 2002-163654 A

  In the portable information terminal device, the location where the image is taken when storing the registered feature information in the memory is different from time to time, so factors different from the user's secular change, such as the change in the angle of light hitting the user's face, etc. Depending on the surrounding environment, the degree of coincidence between the feature information extracted from the photographed image and the registered feature information is reduced, and the authentication success rate (the probability of successful authentication when the user himself authenticates, or a person other than the user authenticates If this is done, the probability of authentication failure may decrease.

  For example, the registered feature information stored in the memory is feature information extracted from the face image when the user is exposed to light from the left side, and the photographed image during the authentication process is exposed to the user from the right side If it is a face image of the time, authentication may fail due to a difference in the angle at which the light hits, and the authentication success rate decreases.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a portable information terminal device capable of improving the authentication success rate.

The portable information terminal device according to the first aspect of the present invention is characterized in that image information for photographing a subject and feature information extracted from two or more face images are stored in advance as registered feature information for each face image. When the feature information is extracted from the image captured by the imaging means by the storage means, and the feature information is compared with the registered feature information, and when authentication fails in the authentication process , said the failure information storage means for storing the comparison feature information and the registered feature information, when the authentication succeeds in the authentication process, the feature information of the comparison from the failed before also the registered feature information during the authentication process therewith Are added to the feature information storage means as new registered feature information, and an authentication threshold setting process for performing an authentication threshold setting process for setting an authentication threshold. An authentication processing means, and the authentication processing means compares the feature information with the registered feature information to obtain a digitized match score, and compares the match score with the authentication threshold value. A degree-of-matching determination unit that performs authentication, and the authentication threshold setting processing unit sets the authentication threshold based on the number of registered feature information corresponding to the same user stored in the feature information storage unit Configured as follows.

According to such a configuration, when authentication is successful in the authentication process, the feature information at the time of the authentication process that failed before that can be added as registered feature information . Therefore, in the authentication process performed thereafter, the authentication process is performed using the added registered feature information, so that the authentication success rate can be improved.

For example, the registered feature information already stored in the feature information storage means is feature information extracted from a face image when the user is exposed to light from the left side, and a captured image at the time of authentication processing is displayed from the right side to the user. Consider a case where a face image is exposed to light. In this case, even if authentication fails due to a difference in the angle of light, if the feature information at that time is added to the feature information storage means as registered feature information, the authentication process performed thereafter Even if the user is exposed to light from either the left side or the right side, the authentication can be surely succeeded. Further, the authentication threshold used for the authentication process can be set to an appropriate value according to the number of registered feature information stored in the feature information storage unit. For example, when the number of registered feature information stored in the feature information storage means is small and the probability that the user himself will succeed in authentication is low, setting the authentication threshold low may make it easier for the user himself to succeed in authentication. it can. On the other hand, when the number of registered special information stored in the feature information storage means is large and there is a high probability that a person other than the user will succeed in authentication, the person other than the user succeeds in authentication by setting the authentication threshold value high. Can be difficult.

In the portable information terminal device according to the second aspect of the present invention, the feature information update unit succeeds in authentication in an authentication process performed before a predetermined time has elapsed since the feature information was stored in the failure information storage unit. In addition, the feature information compared with the registered feature information during the failed authentication process is added to the feature information storage means .

According to such a configuration, when authentication is successful in the authentication process performed after a lapse of a certain time from the failed authentication process, the feature information at the failed authentication process is added as registered feature information. Can be prevented. Therefore, when a person other than the user fails in the authentication process, characteristic information when authentication succeeds during the operation performed by the user himself after a certain period of time, and before the person other than the user fails authentication. Can be prevented from being added as registered feature information.

The third portable information terminal apparatus according to the present invention, when the authentication fails in the authentication process includes a re-authentication processing means for performing re-authentication process, the feature information updating means, the authentication during the re-authentication process When successful, the feature information compared with the registered feature information during the authentication process that failed before that is added to the feature information storage unit as new registered feature information.

  According to such a configuration, when the authentication succeeds in the re-authentication process, the feature information at the time of the authentication process that failed before that can be added to the feature information storage unit as the registered feature information. Therefore, in the authentication process performed thereafter, the authentication process is performed using the feature information at the time of the failed authentication process in addition to the registered feature information already stored in the feature information storage unit. Can be improved.

  For example, the registered feature information already stored in the feature information storage means is feature information extracted from a face image when the user is exposed to light from the left side, and a captured image at the time of authentication processing is displayed from the right side to the user. Consider a case where a face image is exposed to light. In this case, even if the authentication fails due to a difference in the angle of light, the authentication process performed thereafter is performed by adding the feature information at that time to the feature information storage unit as the registered feature information. In this case, even if light is shining on the user from either the left side or the right side, the authentication can be surely succeeded.

In the portable information terminal device according to a fourth aspect of the present invention, the feature information updating unit, when the degree of coincidence in the coincidence determination means fails to authenticate is determined to be smaller than a predetermined threshold value than the authentication threshold value, The feature information compared with the registered feature information during the failed authentication process is added to the feature information storage means.

  According to such a configuration, only when the degree of coincidence between the feature information extracted from the captured image and the registered feature information is determined to be greater than or equal to a predetermined threshold value that is smaller than the authentication threshold value and the authentication fails, the failed authentication is performed. Feature information at the time of processing can be added to the feature information storage means. That is, the photographed image is a face image, and the feature of the user stored in advance as the registered feature information in spite of extracting the feature information of the face image such as the shape and size of the eyes, nose, mouth, and contour. When the degree of coincidence with information is smaller than the authentication threshold and it is determined that the user is not a true user, only feature information having a high degree of coincidence with the user's feature information can be added. Thereby, only the feature information when the true user has failed in authentication can be added to the feature information storage means as registered feature information.

In the portable information terminal device according to the fifth aspect of the present invention, the re-authentication processing unit controls the image capturing unit and the authentication processing unit, and performs image capturing again when authentication fails in the authentication processing, and the captured image By performing the authentication process based on the above, a series of authentication processes are performed until the end condition is satisfied, and the feature information update unit performs authentication performed after two or more authentication failures in the series of authentication processes. When the authentication succeeds in the processing, the feature information having the highest degree of coincidence among the feature information compared with the registered feature information during the failed authentication processing is added to the feature information storage means.

  According to such a configuration, when authentication is successful in authentication processing performed after two or more authentication failures, among the feature information compared with the registered feature information during the failed authentication processing, the user's Only feature information having the highest degree of coincidence with feature information can be added. Thereby, only the most useful feature information among the feature information at the time of the failed authentication process can be added to improve the authentication success rate.

In the sixth portable information terminal device according to the invention, for each registered feature information stored in the feature information storage means, the authentication success stores authentication success rate of the authentication processing performed using those registered feature information The feature information update unit adds the new registered feature information by overwriting the registered feature information with the lowest authentication success rate stored in the feature information storage unit.

  According to such a configuration, the registered feature information with the lowest authentication success rate stored in the feature information storage unit can be overwritten with the feature information to be newly stored, that is, the feature information at the time of failed authentication processing. it can. Therefore, it is possible to prevent the authentication success rate from being lowered by overwriting the registered feature information having a high authentication success rate with new feature information.

  According to the portable information terminal device of the present invention, when the authentication process is successful, the authentication success rate is improved by updating the registered feature information based on the failure information at the time of the authentication process that failed before that. be able to. If the registered feature information is updated only when authentication is successful in the authentication process performed before a fixed time has elapsed since the failure information is stored, when a person other than the user fails in the authentication process, It is possible to prevent the registration feature information from being updated based on failure information when authentication is successful during an operation performed by the user himself after a certain period of time and authentication is failed by a person other than the user before that time.

  According to the portable information terminal device of the present invention, when the authentication is successful in the re-authentication process, the feature information at the time of the authentication process that failed before is added to the feature information storage unit as the registered feature information. The rate can be improved. Only when the degree of coincidence between the feature information extracted from the photographed image and the registered feature information is determined to be greater than or equal to a predetermined threshold value that is smaller than the authentication threshold value and authentication fails, the feature information at the time of the failed authentication process is displayed as feature information. If added to the storage means, only the feature information when the true user has failed in authentication can be added to the feature information storage means as registered feature information.

  If the authentication threshold value is set based on the number of registered feature information stored in the feature information storage unit, the authentication threshold value can be set to an appropriate value. In addition, when authentication is successful after two or more authentication failures, the degree of coincidence with the feature information of the user among the feature information compared with the registered feature information during the failed authentication processing If only the feature information with the highest is added, only the most useful feature information among the feature information at the time of the failed authentication process can be added, and the authentication success rate can be improved.

  Further, according to the portable information terminal device of the present invention, the registered feature information with the lowest authentication success rate stored in the feature information storage means can be overwritten with the feature information at the time of the failed authentication process. It is possible to prevent the authentication success rate from being lowered by overwriting the registered feature information with a high success rate with new feature information.

Embodiment 1 FIG.
FIG. 1 is an external view showing an example of a portable information terminal device according to Embodiment 1 of the present invention. A mobile phone 1 is shown as an example of the portable information terminal device. The mobile phone 1 is a so-called foldable mobile phone, in which a display housing 100 and an operation housing 200 are connected via a hinge part 300, and are folded with one surface of the display housing 100 and the operation housing 200 facing each other. be able to.

  In the display housing 100, a main display unit 101, a receiver for receiver 103, and a sub camera 105 are disposed on the inner housing surface when folded, and a sub display portion and a main camera 104 are disposed on the outer housing surface. . In addition, the operation casing 200 has a large number of operation keys 201 and a microphone for transmitting 203 arranged on the inner casing surface when folded. Such a foldable mobile phone 1 can be carried in a compactly folded state, and if the casing is unfolded, the operation key 201 can be pressed while viewing the display on the main display unit 101. . That is, main information display is performed using the main display unit 101, and main operation input is performed using the operation keys 201.

  When the user is operating the operation key 201 while viewing the display on the main display unit 101, the cellular phone 1 captures the user's face image by the sub camera 105 and extracts feature information extracted from the face image. As a result of performing the authentication process based on the above, it has a function (so-called face authentication function) that permits the execution of the subsequent process only when it is determined that the person who is operating at that time is a true user Yes. By performing authentication processing when executing a highly confidential function, security can be improved and privacy can be protected.

  FIG. 2 is a block diagram showing an example of the internal configuration of the mobile phone 1 shown in FIG. The control unit 400 is a processor that controls main operations of the mobile phone 1, and includes an imaging unit drive control unit 401, an authentication processing unit 402, a continuous authentication unit 403, a feature information update unit 404, and a time measuring unit 405. . An operation signal from the operation key 201 is input to the control unit 400, and a memory 410, an image input / output unit 430, an audio input / output unit 440, and a communication unit 450 are connected to be able to input / output.

  The image input / output unit 430 includes a main display unit 101, a sub display unit 102, a main camera 104, and a sub camera 105. The main camera 104 and the sub camera 105 are each unitized as an imaging unit. These imaging units include a photoelectric conversion element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a control. A circuit is provided. Here, a photoelectric conversion element that needs to perform initialization processing such as adjustment of offset and gain after power-on of the imaging unit is used.

  The voice input / output unit 440 is provided with a receiving receiver 103 and a transmitting microphone 203. The communication unit 450 transmits / receives audio data, image data, and the like by transmitting / receiving radio waves to / from a base station (not shown).

  The imaging unit drive control unit 401 controls driving of the main camera 104 and the sub camera 105 by transmitting a signal to the image input / output unit 430. In this embodiment, the imaging unit drive control unit 401 controls the driving of the sub camera 105, so that the sub camera 105 captures a face image of a person who operates the operation key 201 while looking at the main display unit 101. The authentication process is performed based on the feature information extracted from the face image. However, the configuration may be such that authentication processing is performed by photographing a face image with the main camera 104.

  The authentication processing unit 402 compares the feature information extracted from the captured image with the feature information (registered feature information) of the user stored in advance in the memory 410, so that the person who is operating at that time is true It is determined whether or not the user is a user. The continuous authentication unit 403 controls the imaging unit drive control unit 401 and the authentication processing unit 402 to perform imaging again when authentication fails, and performs authentication processing based on the captured image. The continuous authentication unit 403 constitutes a re-authentication processing unit that performs re-authentication processing when authentication fails, and the authentication processing is continuously performed twice or more by the processing of the continuous authentication unit 403. The feature information update unit 404 updates registered feature information for the memory 410. The time measuring unit 405 measures time.

  A display screen data storage unit 411, an image data storage unit 412, a feature information storage unit 413, an error history information storage unit 414, and an authentication success rate storage unit 415 are allocated to the memory 410. The display screen data storage unit 411 stores screen data to be displayed on the main display unit 101 and the sub display unit 102. The image data storage unit 412 stores image data captured by the main camera 104 and the sub camera 105, image data received from the outside via the communication unit 450, and the like.

  The feature information storage unit 413 stores feature information of face images used for authentication processing. The feature information storage unit 413 can store feature information extracted from two or more face images as registered feature information up to a predetermined maximum number (for example, seven) for each face image. By performing face authentication based on registered feature information extracted from two or more face images, the probability (authentication acceptance rate) of successful authentication when a true user performs face authentication can be increased. It is possible to prevent authentication failure due to subtle differences such as facial expressions and the angle of light. However, the registered feature information used for face authentication is not limited to a configuration that is extracted from seven face images, but is a configuration that performs face authentication based on registered feature information extracted from one face image, for example. There may be.

  The user's face image is previously captured by the user using the main camera 104 or the sub camera 105, and the image data is stored in the image data storage unit 412 of the memory 410. Since face authentication is performed based on registered feature information extracted from these image data and stored in the feature information storage unit 413 of the memory 410, the image data itself is not necessary for face authentication. If it is stored in 410, the user can read the image data and check the face image, so that convenience is improved.

  In the error history information storage unit 414, when authentication fails, information at the time of authentication is stored as error history information. For each registered feature information stored in the feature information storage unit 413, the authentication success rate storage unit 415 stores the authentication success rate of the authentication process performed using the registered feature information. Here, the authentication success rate associated with each registered feature information represents the ratio of the number of successful authentications to the number of authentication processes performed using those registered feature information. For example, when authentication processing is performed twice using certain registered feature information and authentication is successful only once, the authentication success rate is 50%.

  However, the facial image feature information and facial image image data used for the authentication process are stored in a storage medium (not shown) that is detachable from the mobile phone 1 such as a SIM (Subscriber Identity Module) card. Such a configuration may be used.

  FIG. 3 is a block diagram illustrating a configuration example of the imaging unit drive control unit 401 in FIG. The imaging unit drive control unit 401 includes an imaging unit activation unit 407, an imaging control unit 408, and an imaging unit stop unit 409. The imaging unit activation unit 407 transmits control signals to the main camera 104 and the sub camera 105 to activate them. The imaging unit 106 (the main camera 104 or the sub camera 105) is activated based on a user operation. Are provided with a power-on unit 407A for turning on power and an initialization processing unit 407B for executing initialization processing.

  The imaging control unit 408 controls the imaging operation of the main camera 104 or the sub camera 105 based on the input signal from the initialization processing unit 407B or the imaging operation by the user, and thereby the main camera 104 or the sub camera 105 at that time. Still image data of a subject for is generated. The imaging unit stop unit 409 stops driving the imaging units by stopping the power supplied to the main camera 104 or the sub camera 105.

  FIG. 4 is a block diagram illustrating a configuration example of the authentication processing unit 402 in FIG. The authentication processing unit 402 includes a feature extraction unit 402A, a matching degree calculation unit 402B, and a matching degree determination unit 402C. When the authentication process is started, feature information such as shape and size of each part of the face such as eyes, nose, mouth and contour is extracted from the face image taken by the sub camera 105 by the feature extraction unit 402A. .

  If the captured image is a landscape image that is not a face image, the feature information cannot be extracted from the captured image, and authentication fails. On the other hand, when the feature information is successfully extracted from the photographed image, the feature information is compared with the registered feature information stored in the feature information storage unit 413 of the memory 410 by the coincidence degree calculation unit 402B. The degree of matching is calculated.

  The coincidence determination unit 402C compares the coincidence calculated by the coincidence calculation unit 402B with a predetermined threshold (authentication threshold), and outputs the comparison result. At this time, if the degree of coincidence exceeds the authentication threshold, it is determined that the user is a true user and the authentication is successful. If the degree of coincidence is smaller than the authentication threshold, the user is not determined to be a true user. Authentication fails. Here, the degree of coincidence is a ratio (%) occupied by the matched feature information among all the compared feature information, and the authentication threshold is usually set to a predetermined value of 90% or more.

  The comparison result output from the matching degree determination unit 402C is input to the feature information update unit 404. The feature information update unit 404 performs processing (feature information update processing) for updating feature information stored in the feature information storage unit 413 based on the input comparison result.

  FIG. 5 is a flowchart illustrating an example of the feature information update process. In this example, when the authentication is successful (Yes in step S101), the feature information update unit 404 adds feature information (successful feature information) compared with the registered feature information at the time of authentication to the feature information storage unit 413 ( Step S102). On the other hand, if the authentication fails (No in step S101), the feature information compared with the registered feature information at the time of the failed authentication is not added to the feature information storage unit 413. When the authentication is successful, by adding the success feature information at that time, it is possible to prevent the authentication success rate from decreasing due to, for example, a change in the user's body shape or aging.

  When the authentication is successful (Yes in step S101), the feature information update unit 404 assumes that a certain condition is satisfied, and the authentication information is unsuccessful in the authentication process performed before the authentication process. The feature information (failure feature information) is added to the feature information storage unit 413 as new registered feature information (step S110). At this time, the feature information storage unit 413 functions as failure information storage means for storing information (failure information) when authentication fails in the authentication process performed before that.

As a premise that the failure feature information is added to the feature information storage unit 413, it is necessary to satisfy the following three conditions. That is,
(1) Successful authentication in the second and subsequent authentication processes in a series of authentication processes, that is, successful authentication in the authentication process performed after one or more authentication failures (Yes in step S103),
(2) Successful authentication in the authentication process performed before a fixed time has elapsed after authentication failure (Yes in step S104), and (3) The degree of coincidence when authentication fails is an authentication threshold value. Is greater than or equal to a predetermined threshold value (Yes in step S105)
is required.

  In the present embodiment, when the authentication is successful, the feature information at the time of the authentication process that failed before that can be added to the feature information storage unit 413 as registered feature information. Accordingly, in the authentication process performed thereafter, the authentication process is performed using the feature information at the time of the failed authentication process in addition to the registered feature information already stored in the feature information storage unit 413. Can be improved.

  For example, the registered feature information already stored in the feature information storage unit 413 is feature information extracted from the face image when the user is exposed to light from the left side, and the captured image at the time of authentication processing is the right side for the user. Consider a case where the face image is exposed to light. In this case, even if the authentication fails due to a difference in the angle of light, the feature information at that time is added to the feature information storage unit 413 as registered feature information, so that authentication is performed thereafter. In the processing, even if light is shining on the user from either the left side or the right side, the authentication can be surely succeeded.

  In addition, when authentication is successful in the authentication process performed after a lapse of a certain time from the failed authentication process, the feature information at the failed authentication process is added to the feature information storage unit 413 as registered feature information. Can be prevented. Therefore, when a person other than the user fails in the authentication process, characteristic information when authentication succeeds during the operation performed by the user himself after a certain period of time, and before the person other than the user fails authentication. Can be prevented from being added as registered feature information.

  Further, only when the degree of coincidence between the feature information extracted from the photographed image and the registered feature information is determined to be equal to or greater than a predetermined threshold smaller than the authentication threshold and the authentication fails, the feature information at the time of the failed authentication processing is displayed. It can be added to the feature information storage unit 413. That is, the photographed image is a face image, and the feature of the user stored in advance as the registered feature information in spite of extracting the feature information of the face image such as the shape and size of the eyes, nose, mouth, and contour. When the degree of coincidence with information is smaller than the authentication threshold and it is determined that the user is not a true user, only feature information having a high degree of coincidence with the user's feature information can be added. Thereby, only the feature information when the true user has failed in authentication can be added to the feature information storage unit 413 as registered feature information.

  In the case where the above three conditions (steps S103 to S105) are satisfied, there are cases where two or more pieces of failure feature information exist when the authentication succeeds in the third and subsequent authentication processes. In this case, all of the two or more pieces of failure feature information can be added to the feature information storage unit 413. However, in the present embodiment, the failure feature information having the highest degree of coincidence among the failure feature information is featured. The information is added to the information storage unit 413.

  That is, if there are two or more pieces of failure feature information (Yes in step S106), the feature information update unit 404 refers to the degree of coincidence at the time of the failed authentication process, and the failure feature information having the highest degree of coincidence. Is selected (step S107). Accordingly, only the most useful failure feature information among the two or more failure feature information can be added to improve the authentication success rate.

  When one piece of failure feature information selected as described above is added to the feature information storage unit 413, the feature information stored in the feature information storage unit 413 has still reached a predetermined maximum number (for example, seven). If not (No in Step S108), the selected failure feature information is additionally stored in the free area of the feature information storage unit 413 (Step S110).

  On the other hand, when the feature information corresponding to the predetermined maximum number of face images is already stored in the feature information storage unit 413, the authentication success rate stored in the authentication success rate storage unit 415 in association with the feature information is calculated. The registered feature information that is referred to and has the lowest authentication success rate is deleted from the feature information storage unit 413 (step S109), and the failure feature information is added to the free space of the feature information storage unit 413 generated thereby (step S109). S110). Thereby, it is possible to prevent the authentication success rate from being lowered by overwriting the registered feature information having a high authentication success rate with new failure feature information.

  However, the failure feature information is not limited to the configuration in which the failure feature information is added to the feature information storage unit 413 only when all of the above three conditions are satisfied. If one or two of the above three conditions are satisfied, the failure feature information is not limited. The configuration may be such that information is added.

  FIG. 6 is a diagram illustrating a display example of the main display unit 101 when setting related to face authentication is performed. When the user operates the operation key 201 to read the security setting screen (see FIG. 6A) and selects and operates the “face authentication setting” item from the items displayed on the security setting screen, the mobile phone A screen prompting the user to input a terminal security code assigned in advance to the telephone 1 is displayed on the main display unit 101 (see FIG. 6B). When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  The terminal security code is composed of, for example, an arbitrary character string of 4 to 8 digits, and is set in advance when the mobile phone 1 is purchased, but can be changed by the user operating the operation key 201. When the character string input by operating the operation key 201 does not match the terminal security code set in advance on the terminal security code input screen of FIG. 6B, that effect is displayed on the main display unit 101. (See FIG. 6C). When the character string input on the terminal security code input screen is 3 digits or less, a message indicating that a 4 to 8 character string should be input is displayed on the main display unit 101 (FIG. 6D). reference).

  After the display as shown in FIG. 6C or FIG. 6D is displayed on the main display unit 101, a predetermined time elapses, or a time-out occurs, or any key included in the operation key 201 is displayed. When operated, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  When the character string input on the terminal personal identification number input screen in FIG. 6B matches the terminal personal identification number set in advance, the display on the main display unit 101 is turned on / off in the mode for performing face authentication. The screen is switched to a face authentication use setting screen for setting OFF (see FIG. 6E). When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the terminal security code input screen shown in FIG. When the face authentication is turned off by operating the operation key 201 on the face authentication use setting screen, the display on the main display unit 101 returns to the security setting screen shown in FIG.

  When face authentication is turned on by operating the operation key 201 on the face authentication use setting screen of FIG. 6E, the display on the main display unit 101 is switched to the face authentication setting screen for performing settings related to face authentication ( (Refer FIG.6 (f)). In the face authentication setting screen of FIG. 6F, items associated with images (face authentication images) representing the user from which registered feature information is extracted, and authentication threshold values at the time of face authentication (security) The item associated with (level setting) and the item associated with the error history are included.

  When an item associated with the face authentication image is selected on this display screen, a screen (face authentication image registration screen) for overwriting registration and confirmation of the face authentication image is displayed. When an item associated with the security level setting is selected, a screen (security level setting screen) for setting an authentication threshold for face authentication is displayed. When an item associated with the error history is selected, a screen for displaying the error history (error history screen) is displayed.

  After performing these various settings, when the operation key 201 is operated to complete the face authentication setting, the display on the main display unit 101 is changed to a display as shown in FIG. 6 (g) or FIG. 6 (h). Switch. That is, if at least one face authentication image is registered, the main display unit 101 displays that face authentication is performed based on the feature information extracted from the face authentication image (see FIG. 6G). ) When the confirmation key 111 included in this screen is selected, the display of the main display unit 101 returns to the security setting screen shown in FIG. On the other hand, if no face authentication image is registered, the main display unit 101 displays that the face authentication image should be registered (see FIG. 6H), and the confirmation included in this screen is displayed. When the key 112 is selected, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG.

  FIG. 7 is a diagram showing a display example of the face authentication image registration screen. When any item associated with the face authentication image is selected on the face authentication setting screen of FIG. 6F, the main display unit 101 displays the face authentication as shown in FIG. The screen is switched to the face authentication image registration screen corresponding to the image for use. The face authentication image registration screen includes an “overwrite registration” item and a “registration data confirmation” item.

  When the “overwrite registration” item is selected and operated from the items displayed on the face authentication image registration screen of FIG. 7A, the sub camera 105 is activated, and the sub camera 105 is ready for photographing. Thereafter, the image of the subject with respect to the sub camera 105 is displayed on the main display unit 101 in real time (see FIG. 7B). At this time, the main display unit 101 functions as a finder of the sub camera 105, and the face of the person who is performing the operation while viewing the main display unit 101 is displayed on the finder screen configured by the main display unit 101. It becomes.

  When the user performs registration operation by operating the determination key included in the operation key 201 after the user performs position alignment of the user's face on the finder screen of FIG. 7B, the feature extraction unit of the control unit 400 402A performs a process of extracting feature information from the image photographed by the sub camera 105 at that time.

  At this time, if the user's face is properly aligned, feature information such as the shape and size of the eyes, nose, mouth and contour can be extracted. However, the alignment is appropriately performed. If the user's face is not displayed on the main display unit 101, the feature information cannot be extracted. When the feature information is successfully extracted, the feature information is stored in the feature information storage unit 413 of the memory 410, and the face image at that time is used as a face authentication image in the image data storage unit 412 of the memory 410. Stored in

  When the feature information is successfully extracted, the fact that the feature information based on the face image is stored (registered) in the memory 410 is displayed on the main display unit 101 (see FIG. 7C). When the user selects the confirmation key 113 included in this display screen, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG. On the other hand, if the feature information extraction fails, the fact and the cause of the feature information extraction failure are displayed on the main display unit 101 (see FIG. 7D), and the confirmation key 114 included in this display screen is displayed. When the user selects, the display on the main display unit 101 returns to the viewfinder screen shown in FIG. 6B, and a face image can be taken again by the sub camera 105.

  When the “registration data confirmation” item is selected from the items displayed on the face authentication image registration screen of FIG. 7A, the corresponding face authentication data stored in the image data storage unit 412 of the memory 410 is displayed. An image is read and displayed on the main display unit 101 (see FIG. 7E). The user can confirm the registered face authentication image on the face authentication image confirmation screen shown in FIG. When the clear key included in the operation key 201 is operated on this display screen, the display on the main display unit 101 returns to the face authentication image registration screen shown in FIG.

  When the user performs an operation of deleting the corresponding face authentication image by operating the operation key 201 on the face authentication image confirmation screen in FIG. 7E, the display on the main display unit 101 displays the face authentication. The screen is switched to a deletion confirmation screen for confirming whether or not to delete the image for use (see FIG. 7F). This display screen includes a “Yes” key 115 for instructing that the face authentication image should be deleted, and a “No” key 116 for instructing not to delete the face authentication image. When the “No” key 116 is selected by operating the operation key 201 on the deletion confirmation screen in FIG. 7F, the display on the main display unit 101 is the face authentication image shown in FIG. Return to the confirmation screen.

  When the “Yes” key 115 is selected by operating the operation key 201 on the deletion confirmation screen in FIG. 7F, the corresponding face authentication image is stored in the image data storage unit 412 and the feature information storage unit 413 of the memory 410. In addition, a process for deleting the registered feature information is started and a message indicating that the feature is being deleted is displayed on the main display unit 101 (see FIG. 7G). Then, when the processing for deleting the face authentication image and the registered feature information is completed, a message to that effect is displayed on the main display unit 101 (see FIG. 7 (h)) and a predetermined time elapses, or a time-out occurs. In response to any one of the keys included in the operation key 201 being operated, the display on the main display unit 101 returns to the face authentication setting screen shown in FIG.

  FIG. 8 is a diagram showing a display example of the security level setting screen. When an item associated with the security level setting is selected on the face authentication setting screen shown in FIG. 6F, the display on the main display unit 101 sets an authentication threshold for face authentication as shown in FIG. Switches to the security level setting screen. As the security level, “level 3” having the highest authentication threshold, “level 2” having an authentication threshold lower than “level 3”, and “level 1” having an authentication threshold lower than “level 2” are set in advance. Any security level can be selected on the security level setting screen.

  FIG. 9 is a diagram showing a display example of the error history screen. When the item associated with the error history is selected on the face authentication setting screen of FIG. 6F, the display of the main display unit 101 is displayed on the error history screen for displaying the error history as shown in FIG. Switch. On the error history screen, time information indicating the date and time when authentication failed is displayed as a list of error history information for a predetermined number of times (for example, nine times). When the authentication history is newly failed while the error history information for the predetermined number of times is stored in the error history information storage unit 414 of the memory 410, the oldest error history information is deleted, and the new error history information is stored in the memory. 410 and updated so that the new error history information is displayed on the error history screen.

  FIG. 10 is a diagram illustrating a display example of the main display unit 101 when performing face authentication. In this example, face authentication is performed when the secret mode included in the management function of the mobile phone 1 is switched on / off. When incognito mode is on, not only normal data but also data registered in advance as secret data is displayed. When incognito mode is off, secret data is not displayed and only normal data is displayed. Will be displayed.

  In the management function screen for setting the management function of the mobile phone 1 as shown in FIG. 10A, when the “secret mode” item is selected and operated from the items displayed on the management function screen, the sub camera 105 is activated. After the sub camera 105 is ready for shooting, an image of the subject with respect to the sub camera 105 is displayed on the main display unit 101 in real time (see FIG. 10B), and automatic authentication is started. At this time, when the initialization process of the sub-camera 105 is completed and the camera is ready for shooting, a face image for authentication is shot without the user performing a shooting operation. Is extracted. Then, the extracted feature information is compared with the user feature information stored in advance in the feature information storage unit 413 of the memory 410, whereby authentication processing is automatically performed.

  In this authentication process, when the degree of coincidence between the feature information of the extracted face image and the user feature information stored in the memory 410 exceeds an authentication threshold set in advance on the security level setting screen, It is determined that the operator is a true user. In this case, the user's face image displayed on the main display unit 101 is highlighted by so-called action focus, thereby informing that the authentication has been successful by a visual effect.

  Thereafter, a message indicating that the authentication has succeeded is displayed on the main display unit 101 (see FIG. 10C), and after a predetermined time has elapsed, the screen is switched to a screen for switching on / off the secret mode (FIG. 10D). reference). On the other hand, when the degree of coincidence of the feature information is smaller than the authentication threshold and the authentication fails, the feature information is automatically extracted again from the image photographed by the sub camera 105 at that time, and the feature information is stored in the memory 410. Is compared with the feature information of the user stored in.

  As described above, in this embodiment, when the authentication fails, the authentication process is repeatedly performed continuously until a predetermined time (for example, 3 seconds) elapses and a timeout occurs, and the authentication process ends when the authentication is successful. To do. That is, a series of authentication processes is performed until the end condition is satisfied, with the end condition being successful authentication or timeout.

  For example, when the timeout time is set to 3 seconds, the authentication process is performed about 7 times at the maximum. That is, the time required for one authentication process is about 0.4 seconds, and if the authentication is successful in the first authentication process, the process proceeds to the next process with almost no waiting time after the initialization process of the sub camera 105 is completed. Can do. While the authentication process is being performed repeatedly, a finder screen is displayed on the main display unit 101, and an image of the subject with respect to the sub camera 105 is displayed in real time.

  When the authentication process is repeatedly performed, each time the authentication process is performed a plurality of times, a display corresponding to the matching degree of the feature information is displayed on the main display unit 101 as a comparison result of the feature information in the authentication process. More specifically, the color of the frame 117 of the finder screen displayed on the main display unit 101 is displayed in a color corresponding to the matching degree of the feature information. As a result, since the comparison result of the degree of coincidence of the feature information at the time of the authentication process is displayed, based on the display, whether or not the user's own face image taken at the time of the authentication process is an easily recognizable image Can be confirmed.

  However, the present invention is not limited to such a configuration, and a configuration in which the degree of coincidence of the feature information is level-displayed on the main display unit 101 using a numerical value or a graph may be used. Further, the present invention is not limited to the configuration in which the comparison results are displayed for all authentication processes, and may be any display that displays the comparison results for at least one authentication process.

  When a predetermined time elapses from the start of automatic authentication and a timeout occurs, a message indicating that the authentication has failed is displayed on the main display unit 101 (see FIG. 10E). When the user selects the re-authentication key 118 included in this display screen, the display on the main display unit 101 returns to the finder screen shown in FIG. 10B, and automatic authentication is started again. On the other hand, when the user selects the terminal security code input key 119 displayed on the main display unit 101 in FIG. 10E, the display on the main display unit 101 is switched to the terminal security code input screen that prompts the user to enter the terminal security code. (Refer FIG.10 (f)).

  When the character string input on the terminal personal identification number input screen of FIG. 10F matches the terminal personal identification number set in advance, the display on the main display unit 101 switches on / off of the secret mode. The screen is switched to the screen for the purpose (see FIG. 10D).

  On the other hand, when the character string input by the operation of the operation key 201 does not match the terminal security code set in advance on the terminal security code input screen of FIG. (See FIG. 10G). When the character string input on the terminal security code input screen is 3 digits or less, a message indicating that a 4 to 8 character string should be input is displayed on the main display unit 101 (FIG. 10 (h)). reference). Then, after the display as shown in FIG. 10G or FIG. 10H is performed, a predetermined time elapses and a time-out occurs, or any key included in the operation key 201 is operated. And the display of the main display part 101 returns to the management function screen shown to Fig.10 (a).

  According to the configuration in which the authentication process is automatically performed again when the authentication process fails in the authentication process as in the present embodiment, the true authentication is performed by the first authentication process when the true user performs an operation. Even if it is not determined that the user is the user, the authentication process is performed again without the user performing a shooting operation. In other words, since the authentication process is automatically and repeatedly performed until it is determined that the user is a true user, the labor required for the authentication process can be reduced and the time required for the authentication process can be shortened.

  In the present embodiment, automatic authentication is started after selecting the secret mode in FIG. 10A, and re-authentication processing is performed when authentication fails in the first authentication processing. If the terminal security code input key 119 is selected when automatic authentication times out, re-authentication processing is performed by inputting the terminal security code.

  When authentication is successful in the second and subsequent authentication processes in automatic authentication, or when authentication is successful by inputting a terminal personal identification number, the failure feature information in the series of authentication processes is registered as feature information storage unit 413. Stored in At this time, if the authentication is successful in the second authentication process in automatic authentication, there is only one failure feature information at the time of the authentication that has just failed, so this failure feature information is used as the feature information storage unit as registered feature information. 413 is stored.

  On the other hand, when there are two or more pieces of failure feature information, such as when authentication is successful in the third and subsequent authentication processing in automatic authentication, or when authentication is successful by inputting a terminal security code, As described above, of the failure feature information, the feature information having the highest matching degree at the time of authentication is stored in the feature information storage unit 413.

Embodiment 2. FIG.
In the first embodiment, the configuration in which the user sets the security level on the security level setting screen displayed on the main display unit 101 has been described. However, in the second embodiment, the configuration is stored in the feature information storage unit 413. Based on the number of registered feature information, a process (authentication threshold setting process) for automatically setting an authentication threshold used at the time of authentication is performed.

  FIG. 11 is a flowchart showing an example of the authentication threshold setting process according to the second embodiment of the present invention. In this example, when the registered feature information is added to the feature information storage unit 413 and the feature information is increased, the authentication threshold is increased. That is, the first threshold value for determining whether or not the registered feature information stored in the registered feature information storage unit 413 is relatively small, and whether or not the registered feature information is relatively larger than the first threshold value. A second threshold value for determination is set in advance, and a security level is set according to a comparison result with these threshold values (step S201).

  When the number of registered feature information stored in the registered feature information storage unit 413 is smaller than the first threshold value, the security level is set to “level 1” having the lowest authentication threshold value (step S202). If the number of registered feature information is greater than or equal to the first threshold and smaller than the second threshold, the security level is set to “level 2”, which is higher than the “level 1” (step S203). If the number of registered feature information is equal to or greater than the second threshold, the security level is set to “level 3”, which is higher than the “level 2” (step S204).

  However, the authentication threshold value is not limited to two of the first threshold value and the second threshold value, and may be configured such that a two-stage security level can be selected by setting only one, or setting three or more. By doing so, it may be configured such that four or more security levels can be selected.

  In the present embodiment, the authentication threshold used for the authentication process can be set to an appropriate value according to the number of registered feature information stored in the feature information storage unit 413. For example, when the number of registered feature information stored in the feature information storage unit 413 is small and the probability of successful authentication of the user (person acceptance rate) is low, the user himself / herself can be authenticated by setting the authentication threshold value low. It can make it easier to succeed. On the other hand, when the number of registered special information stored in the feature information storage unit 413 is large and the probability that a person other than the user succeeds in authentication (acceptance rate of others) is high, by setting the authentication threshold value high, It is possible to make it difficult for a person to succeed in authentication.

  The configuration for setting the authentication threshold based on the number of registered feature information stored in the feature information storage unit 413 is such that the registered feature information is automatically stored in the memory 410 as in the present embodiment. For example, the present invention can be applied to a case where registered feature information is stored by a user input operation.

  It is preferable that the shutter sound is not generated when a face image is captured in the authentication process. With such a configuration, it is possible to prevent misunderstandings such as voyeurism due to the generation of shutter sound when authentication processing is performed in an environment where there are people around the train or the like. In the present embodiment, since the shooting operation is not performed during the authentication process, it is possible to prevent misunderstanding such as voyeurism by performing the shooting operation.

  However, when it is not determined that the user is a true user within a predetermined time, the configuration is not limited to a configuration in which a timeout occurs, and a configuration in which authentication processing is repeatedly performed until authentication is successful may be used. Further, the configuration is not limited to the configuration in which the authentication process is continuously performed, and a configuration in which the authentication process is performed only once after the sub camera 105 is activated and becomes ready for photographing.

  Face authentication is not limited to automatic authentication in which authentication is started without a user's photographing operation, and is configured such that photographing is performed according to the user's photographing operation and face authentication is performed based on the photographed image. Also good.

  When authentication succeeds in the authentication processing performed after the authentication failure, the failure feature information is not limited to the configuration stored in the memory 410 as failure information. For example, a photographed image (failure) when authentication fails (Image) may be stored in the memory 410. In this case, the configuration may be such that the feature information is extracted from the failed image stored in the memory 410 and the feature information is used for the subsequent authentication processing.

  Face authentication is not limited to switching secret mode on / off, but other functions with high security and privacy, such as card setting, incoming call restriction, lock setting, PIN change, caller ID notification / not-notification It can be performed during various setting operations relating to setting, setting reset, software update, own station number details display, and other data changes.

  Further, the authentication process is not limited to face authentication performed based on the face image, and may be performed based on the fingerprint image of the user as long as the image represents the user. In this case, the authentication process can be performed by extracting and comparing the feature information of the fingerprint image.

  The present invention is not limited to the contents of the above-described embodiment, and various modifications can be made within the scope of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS It is the external view which showed an example of the portable information terminal device by Embodiment 1 of this invention, and the mobile telephone is shown as an example of a portable information terminal device. FIG. 2 is a block diagram showing an example of an internal configuration of the mobile phone shown in FIG. 1. FIG. 3 is a block diagram illustrating a configuration example of an imaging unit drive control unit in FIG. 2. FIG. 3 is a block diagram illustrating a configuration example of an authentication processing unit in FIG. 2. It is the flowchart which showed an example of the feature information update process. It is the figure which showed the example of a display of the main display part at the time of setting regarding face authentication. It is the figure which showed the example of a display of the image registration screen for face authentication. It is the figure which showed the example of a display of a security level setting screen. It is the figure which showed the example of a display of an error history screen. It is the figure which showed the example of a display of the main display part at the time of performing face authentication. It is the flowchart which showed an example of the authentication threshold value setting process by Embodiment 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Cellular phone 101 Main display part 102 Sub display part 103 Receiver for receiver 104 Main camera 105 Sub camera 201 Operation key 203 Microphone for transmission 400 Control part 401 Imaging unit drive control part 402 Authentication process part 402A Feature extraction part 402B Concordance calculation Unit 402C coincidence determination unit 403 continuous authentication unit 404 feature information update unit 405 timing unit 407 imaging unit activation unit 407A power-on unit 407B initialization processing unit 408 imaging control unit 409 imaging unit stop unit 410 memory 411 display screen data storage unit 412 Image data storage unit 413 Feature information storage unit 414 Error history information storage unit 415 Authentication success rate storage unit 450 Communication unit

Claims (6)

Imaging means for photographing the subject;
Feature information storage means for previously storing feature information extracted from two or more face images as registered feature information for each face image ;
Authentication processing means for extracting feature information from an image captured by the imaging means and performing authentication processing by comparing the feature information with the registered feature information;
If the authentication fails in the authentication process, and the failure information storage means for storing feature information is compared with the registered feature information,
Feature information update that adds feature information compared with the registered feature information at the time of authentication processing that failed before that to the feature information storage unit as new registered feature information when authentication is successful in the authentication processing Means,
An authentication threshold setting processing means for performing an authentication threshold setting process for setting an authentication threshold;
A degree of coincidence determination in which the authentication processing means obtains a degree of coincidence expressed by comparing feature information with the registered feature information and performs authentication by comparing the degree of coincidence with the authentication threshold. Means,
The portable information terminal device, wherein the authentication threshold setting processing means sets the authentication threshold based on the number of registered feature information corresponding to the same user stored in the feature information storage means . The feature information updating unit, when the authentication succeeds in the authentication process that is performed before the characteristic information in the failure information storage means for predetermined time has elapsed since the stored, and the registered feature information at the time of failed authentication 2. The portable information terminal device according to claim 1, wherein the compared feature information is added to the feature information storage means . If the authentication fails in the authentication process includes a re-authentication processing means for performing re-authentication process,
When the feature information update unit succeeds in authentication during the re-authentication process, the feature information compared with the registered feature information during the authentication process that failed before that is used as the new registered feature information. The portable information terminal device according to claim 1, wherein the portable information terminal device is added to a storage unit. The feature information updating unit, when the degree of coincidence in the coincidence determination means fails to authenticate is determined to be smaller than a predetermined threshold value than the authentication threshold value is compared with the failed registered feature information during the authentication process The portable information terminal device according to claim 1 , wherein the feature information is added to the feature information storage means. The re-authentication processing unit controls the image capturing unit and the authentication processing unit to perform photographing again when authentication fails in the authentication processing, and by performing authentication processing based on the captured image, the end condition is satisfied. It is a series of authentication processes until it is established,
The feature information update means, when authentication succeeds in authentication processing performed after two or more authentication failures in the series of authentication processing, the feature information compared with the registered feature information at the time of the failed authentication processing 4. The portable information terminal device according to claim 3 , wherein the feature information having the highest degree of coincidence is added to the feature information storage means. For each registered feature information stored in the feature information storage means, the authentication success rate storage means for storing the authentication success rate of the authentication processing performed using the registered feature information,
The characteristic information updating means, by overwriting the lowest registered feature information of the authentication success rate that has been stored in the feature information storage unit, according to claim 3, characterized in that adding the new registration feature information portable information terminal device according to.

Images