JP2012065107A - Portable terminal apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable terminal apparatus that easily and conveniently performs authentication.SOLUTION: A portable terminal apparatus 1 includes a determination section 102 for determining that the portable terminal apparatus is held, an imaging section 15 for capturing an image, a storage section 200 for storing an image, and an authentication section 103 for performing authentication by matching the image captured by the imaging section 15 with the image stored beforehand in the storage section. When the determination section 102 determines that the portable terminal apparatus 1 is held, the imaging section 15 starts imaging and the authentication section 103 authenticates the image captured by the imaging section 15 on the basis of the image stored beforehand in the storage section 200.

Description

  The present invention relates to a mobile terminal device such as a mobile phone or a PDA (Personal Digital Assistant).

  2. Description of the Related Art Conventionally, a mobile terminal device is provided with a security function so that a mobile phone is not illegally used when the mobile phone reaches another person's hand due to loss or misplacement. As such a security function, a lock function that does not accept input from the user is provided. In the state where the lock function is set, for example, when an input operation such as a button press by the user is detected, face authentication and password authentication are performed. If the face authentication or password authentication is successful, the lock state is released. On the other hand, if the face authentication and password authentication fails, the lock state is maintained (for example, Patent Document 1).

JP 2009-296112 A

  However, according to the above configuration, in order to release the lock, the user has to perform a special input operation for starting authentication, which is troublesome.

  In addition, in a mobile phone equipped with an electrostatic touch panel in which a user touches the display surface to perform an input operation, input is not possible even if the user touches the display surface with a gloved hand. In such a case, it is inconvenient because the display surface must be operated after the gloves are removed.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a mobile terminal device that can perform authentication simply and conveniently.

  The portable terminal device according to the present invention is a portable terminal device, in which a determination unit that determines that the portable terminal device is held, an imaging unit that captures an image, a storage unit that stores an image, and an imaging by the imaging unit. An authentication unit that performs authentication by collating the image that has been stored with the image stored in advance in the storage unit. Here, when it is determined by the determination unit that the portable terminal device is held, the photographing unit starts photographing, and the authentication unit is based on an image stored in advance in the storage unit. Authenticate images taken by.

  In the mobile terminal device according to this aspect, the determination unit can determine whether or not the mobile terminal device is held based on an acceleration acting on the mobile terminal device.

  In the mobile terminal device according to this aspect, the determination unit can determine whether or not the mobile terminal device is held based on acceleration acting on the mobile terminal device and illuminance in the mobile terminal device.

  The determination unit may determine whether or not the mobile terminal device is held based on acceleration acting on the mobile terminal device and whether or not an object to be detected is close to the mobile terminal device.

  Further, the determination unit determines whether the portable terminal device is held based on acceleration acting on the portable terminal device, the illuminance in the portable terminal device, and whether or not an object to be detected is in proximity to the portable terminal device. It can be judged.

  The mobile terminal device according to this aspect may further include an input unit that receives an input to the mobile terminal device, and a function control unit that sets and cancels a key guard function that invalidates the input to the input unit. Here, the function control unit cancels the key guard function when authentication by the authentication unit is successful.

  In this case, the determination unit determines that the mobile terminal device is placed, and the function control unit sets the key guard function when the determination unit determines that the mobile terminal device is placed. Can do.

  The mobile terminal device according to this aspect may further include an input unit that receives an input to the mobile terminal device, and a function control unit that sets and cancels a key guard function that invalidates the input to the input unit. Here, the determination unit determines that the mobile terminal device is placed, and the function control unit cancels the key guard function when the authentication by the authentication unit is successful, and the determination unit performs the mobile terminal. If it is determined that the device is placed, the key guard function is set.

  In this case, the determination unit can determine whether or not the mobile terminal device is placed based on the acceleration acting on the mobile terminal device.

  Moreover, the said determination part can determine whether the said portable terminal device is placed based on the acceleration which acts on the said portable terminal device, and the illumination intensity in the said portable terminal device.

  Furthermore, the determination unit can determine whether or not the mobile terminal device is placed based on acceleration acting on the mobile terminal device and whether or not an object to be detected approaches the mobile terminal device.

  The mobile terminal device according to this aspect may further include a display unit and a display control unit that controls the display unit. Here, when the authentication by the authentication unit is successful, the display control unit turns on the display unit.

  The mobile terminal device according to this aspect may further include a display unit and a display control unit that controls the display unit. Here, the display control unit causes the display unit to display an image related to face authentication when the determination unit determines that the portable terminal device is held.

It is a figure which shows the external appearance structure of the mobile telephone which concerns on embodiment. It is a block diagram which shows the whole structure of the mobile telephone which concerns on embodiment. It is a flowchart which shows the process sequence regarding the key guard function which concerns on embodiment. It is a flowchart which shows the process sequence which cancels | releases the key guard function which concerns on embodiment. It is a block diagram which shows the whole structure of the mobile telephone which concerns on embodiment. It is a block diagram which shows the whole structure of the mobile telephone which concerns on embodiment. It is a flowchart which shows the process sequence which cancels | releases the key guard function which concerns on embodiment. It is a flowchart which shows the process sequence which cancels | releases the key guard function which concerns on embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an external configuration of the mobile phone 1.

  The mobile phone 1 has a cabinet 10 including a front surface and a back surface. A display 11 is arranged on the front surface of the cabinet 10.

  The display 11 is a display unit that displays an image, and includes a liquid crystal panel 11a and a panel backlight 11b that illuminates the liquid crystal panel 11a. The display surface 11c of the liquid crystal panel 11a appears outside, and the touch sensor 12 (see FIG. 2) is arranged on the display surface 11c side of the display 11. In addition, the display 11 may be comprised by other display elements, such as organic EL.

  The touch sensor 12 has a transparent sheet shape, and the display surface 11 c of the display 11 can be seen through the touch sensor 12. The touch sensor 12 includes a first transparent electrode and a second transparent electrode arranged in a matrix. The touch sensor 12 detects a position on the display surface 11c touched by the user by detecting a change in capacitance between the transparent electrodes, and outputs a position signal corresponding to the position to a CPU described later. . The touch sensor 12 is not limited to the capacitive touch sensor 12 and may be an ultrasonic touch sensor 12 or a pressure sensitive touch sensor 12.

  A microphone (hereinafter abbreviated as “microphone”) 13 and a speaker 14 are arranged on the front surface of the cabinet 10. The user can make a call by capturing the sound from the speaker 14 with his / her ear and emitting the sound to the microphone 13.

  A lens window 15a of the camera module 15 (see FIG. 2) is disposed on the front surface of the cabinet 10. An image of the subject is taken into the camera module 15 from the lens window 15a.

  FIG. 2 is a block diagram showing the overall configuration of the mobile phone 1.

  The mobile phone 1 according to the present embodiment includes a CPU 100, a memory 200, a video encoder 301, an audio encoder 302, a communication module 303, a backlight drive circuit 304, a video decoder 305, an audio decoder 306, acceleration in addition to the above-described components. A sensor 307 and a proximity sensor 308 are provided.

  The camera module 15 has an image sensor such as a CCD, and includes a photographing unit for photographing an image. The camera module 15 digitizes the image signal output from the image sensor, performs various corrections such as gamma correction on the image signal, and outputs the image signal to the video encoder 301. The video encoder 301 performs an encoding process on the imaging signal from the camera module 15 and outputs it to the CPU 100.

  The microphone 13 converts the collected sound into a sound signal and outputs the sound signal to the sound encoder 302. The audio encoder 302 converts an analog audio signal from the microphone 13 into a digital audio signal, encodes the digital audio signal, and outputs the digital audio signal to the CPU 100.

The communication module 303 converts an audio signal, an image signal, a text signal, and the like from the CPU 100 into a radio signal and transmits it to the base station via the antenna 303a. The antenna 30
The radio signal received via 3a is converted into an audio signal, an image signal, a text signal, etc., and output to the CPU 100.

  The backlight drive circuit 304 supplies a voltage signal corresponding to the control signal from the CPU 100 to the panel backlight 11b. The panel backlight 11b is turned on by a voltage signal from the backlight driving circuit 304 and illuminates the liquid crystal panel 11a.

  The video decoder 305 converts the video signal from the CPU 100 into an analog or digital video signal that can be displayed on the liquid crystal panel 11a, and outputs it to the liquid crystal panel 11a. The liquid crystal panel 11a displays an image corresponding to the video signal on the display surface 11c.

  The audio decoder 306 performs a decoding process on the audio signal from the CPU 100, converts the audio signal into an analog audio signal, and outputs the analog audio signal to the speaker 14. Also, the audio decoder 306 performs decoding processing on the sound signals of various notification sounds such as ringtones and alarm sounds from the CPU 100, and audio signals, converts them into analog sound signals, and outputs them to the speaker 14. The speaker 14 reproduces sound, notification sound, and the like based on the sound signal and sound signal from the sound decoder 306.

  The acceleration sensor 307 is a three-axis acceleration sensor, and is arranged to detect acceleration generated in the three directions of the X-axis direction, the Y-axis direction, and the Z-axis direction in FIG. The acceleration sensor 307 outputs an acceleration signal corresponding to the detected acceleration to the CPU 100.

  Proximity sensor 308 is a proximity detection unit for detecting that an object to be detected has approached the back surface of mobile phone 1 and that an object to be detected exists in the vicinity of the back surface of mobile phone 1. When the detected object is in the vicinity or in the vicinity, the capacitance increases, and when the capacitance exceeds the threshold, the proximity sensor 308 outputs a proximity signal to the CPU 100. On the other hand, the proximity sensor 308 does not output a proximity signal to the CPU 100 when the detected part is no longer present in the vicinity. The proximity sensor 308 is not limited to the capacitive proximity sensor 308, and may be, for example, an ultrasonic or inductive proximity sensor 308. In addition, the proximity sensor 308 is preferably arranged at a portion that extends from the center of the mobile phone 1 to the lower part so that the palm and fingers can easily approach when the user holds the mobile phone 1.

  The memory 200 is a storage unit including a ROM and a RAM. The memory 200 stores a control program for giving a control function to the CPU 100. Such a control program includes a face authentication engine for performing face authentication by the authentication unit 103 described later. The face authentication engine includes a processing program for extracting a face area from a captured image and a processing program for determining matching between the extracted face area image and a pre-registered face image (registered face image). It is.

  The memory 200 stores image information captured by the camera module 15 and image information and text information captured from the outside via the communication module 303 in a predetermined file format. Further, the memory 200 stores, as a position definition table, the positions of icons, buttons, and the like displayed on the display surface 11c in association with processing requests regarding files and programs corresponding to the icons. .

  The CPU 100 refers to the position definition table in the memory 200, and based on the position signal from the touch sensor 12, according to the control program, the camera module 15, microphone 13, communication module 303, panel backlight 11b, liquid crystal panel 11a, speaker 14 To work. Thereby, various applications such as a call function and an e-mail function are executed.

  The CPU 100 includes a registration unit 101, a determination unit 102, an authentication unit 103, a function control unit 104, and a display control unit 105. These units are implemented as software functions executed by the CPU 100.

  The registration unit 101 acquires image information from the memory 200, the camera module 15, and the like, and registers this image information in the memory 200 as a registered face image for face authentication. The image stored in the memory 200 includes an image taken by the camera module 15 and an image received through the communication module 303. When the registration unit 101 acquires a registered face image from the camera module 15, the registration unit 101 outputs a control signal to the camera module 15 and causes the camera module 15 to capture an image.

  The determination unit 102 determines the state of the mobile phone 1 based on signals output from the acceleration sensor 307 and the proximity sensor 308. For example, the determination unit 102 determines that the mobile phone 1 is held by detecting the movement of the mobile phone 1 based on the acceleration and detecting an object close to the back surface of the mobile phone 1. In addition, the determination unit 102 detects the movement of the mobile phone 1 based on the acceleration, and determines that the mobile phone 1 is in a state where the mobile phone 1 is placed when it detects that there is no proximity object to the back surface of the mobile phone 1.

  The authentication unit 103 performs authentication by comparing the captured image with a registered face image. The authentication unit 103 is a photographing control unit that causes the camera module 15 to photograph an image, a face analysis unit that analyzes the photographed image and the registered face image, and face matching that collates the photographed image and the registered face image based on the analysis result. Part.

  When the cellular phone 1 is held, the imaging control unit outputs a control signal to the camera module 15, causes the camera module 15 to photograph a landscape in front of the cellular phone 1, and stores the captured image information in the memory Save to 200. The captured image information may be a still image of one frame or a moving image in which a plurality of frames are collected.

  The face analysis unit extracts a face image area (face area) included in the image based on the photographed image information while reading the photographed image information from the memory 200 every frame or every several frames. For example, the face analysis unit analyzes the color of the photographed image, detects the skin color, and further detects feature parts such as eyes, nose, and mouth, thereby extracting a face region. The face analysis unit acquires information for specifying a face area, for example, position information of a face outline (hereinafter referred to as “face area information”). The face analysis unit extracts the position, size, color, and the like of each part such as eyes, nose, and mouth as parameters (feature parameters) that characterize the face image based on the face area information. In addition, the face analysis unit reads the registered face image from the memory 200, analyzes the registered face image similarly to the analysis of the captured image, and extracts the feature parameters of the registered face image. Then, the extracted feature parameters are stored in the memory 200 together with the registered face image.

  The face collation unit collates the feature parameters of the captured image obtained by the face analysis unit with the feature parameters of the registered face image, and calculates the matching ratio thereof. The face matching unit determines whether or not the face included in the captured image matches the registered face image based on the matching rate.

  The function control unit 104 sets and cancels the key guard function according to the determination result of the authentication unit 103. When the key guard function is set, the input from the user through the touch sensor 12 or the like becomes invalid. On the other hand, when the key guard function is canceled, an input from the user is accepted and a process corresponding to the input is executed. The invalidation of the input includes a case where the touch sensor 12 does not detect an input from the user and a case where the CPU does not execute processing based on the input from the touch sensor 12 even if the touch sensor 12 detects an input. Including.

  The display control unit 105 controls the display 11 according to the determination results of the determination unit 102 and the authentication unit 103, the input from the user, and the like. For example, the display control unit 105 outputs a control signal for supplying a voltage to the panel backlight 11b to the backlight driving circuit 304 and outputs a video signal to the video decoder 305, thereby turning on the panel backlight 11b. An image is displayed on the display surface 11c of the liquid crystal panel 11a. Further, by outputting a control signal that does not supply voltage to the panel backlight 11b to the backlight drive circuit 304, the panel backlight 11b is turned off and the image is turned off from the display surface 11c of the liquid crystal panel 11a.

<Processing procedure of the first embodiment>
FIG. 3A is a flowchart showing a processing procedure for storing a registered face image in the memory 200.

  The display control unit 105 reads predetermined operation menu information from the memory 200 and displays an operation menu image on the display surface 11c of the liquid crystal panel 11a as shown in FIG. In this operation menu image, six icon images are displayed.

  When the user touches the display position of the registered icon with a finger from the icons of the operation menu, the touch sensor 12 detects the contact position, and the CPU 100 receives a signal of the contact position. The CPU 100 refers to the position definition table in the memory 200, identifies and starts a process for registering a face image as a process corresponding to the contact position (S101).

  The registration unit 101 outputs a control signal to the camera module 15 to take an image. As a result, the camera module 15 captures the face of the user at the position facing the front of the mobile phone 1, so the registration unit 101 acquires image information of the user's face image as registration information in the memory 200 ( S102).

  The registration unit 101 registers the image of the user's face as a registered user's registered face image by storing the image information output from the camera module 15 in the memory 200 (S103). At this time, the face analysis unit analyzes the registered face image to obtain a feature parameter of the registered user's face image, and the registration unit 101 stores the feature parameter in the memory 200 in association with the registered face image.

  FIG. 3B is a flowchart showing a processing procedure related to setting / releasing of the key guard function. FIG. 3C is a flowchart showing a processing procedure for setting the key guard function. FIG. 4 is a flowchart showing a processing procedure for canceling the key guard function.

  As shown in FIG. 3A, when the registered face image of the registered user of the mobile phone 1 is registered, next, the control mode of the key guard function shown in FIG. 3B is activated. Here, it is first determined whether or not the key guard function is set (S201). If the key guard function is not set (S201: NO), a setting determination process for setting the key guard function is executed (S202). On the other hand, if the key guard function is set (S201: YES), a release determination process for releasing the key guard function is executed (S203).

In the setting determination process of FIG. 3C, the determination unit 102 determines whether or not the mobile phone 1 is placed. That is, when the user is using the mobile phone 1 such as when the user touches the display surface 11c and operates the mobile phone 1 or when the user is viewing an image displayed on the display surface 11c. Then, the user holds the cellular phone 1 in his hand and holds it in front of him and hardly moves it. However, when the user stops using the mobile phone 1, the user keeps the mobile phone 1 away from the front, releases the hand from the back of the mobile phone 1, and places the mobile phone 1 on a desk or in a bag. For this reason, when the mobile phone 1 is moved from a stationary state and there is no proximity object from the back surface of the mobile phone 1, the determination unit 102 can determine that the mobile phone 1 is going to be put out of use.

  Therefore, the determination unit 102 receives an acceleration signal from the acceleration sensor 307. When the mobile phone 1 is greatly separated from the front, at least one of the accelerations in the three-axis directions changes greatly at the beginning of movement. If determining unit 102 determines that any change in acceleration exceeds a predetermined first threshold, it detects that mobile phone 1 has been moved. The first threshold value is set to determine whether or not the movement of the mobile phone 1 is a user's intentional operation. That is, even if an acceleration equal to or lower than the first threshold is detected by a slight motion, it is determined that this is not due to the user's intentional motion for placing the mobile phone 1.

  Further, when the determination unit 102 stops receiving the detection signal from the proximity sensor 308, the determination unit 102 determines that there is no proximity object from the back surface of the mobile phone 1. As described above, when the hand is released from the back surface of the mobile phone 1 and the mobile phone 1 is moved from the front, the determination unit 102 moves the mobile phone 1 away from the user's hand to a desk or the like. It is determined that it is placed (S301: YES).

  When the mobile phone 1 is placed in a bag or on a desk, the mobile phone 1 may malfunction due to an object hitting the mobile phone 1 in the bag, or the display surface of the mobile phone 1 placed on the desk. 11c may be seen or operated by others. In order to prevent these, it is desirable to immediately set the key guard function and turn off the display surface 11c when the mobile phone 1 is placed. Therefore, the function control unit 104 sets a key guard function (S302) and turns off the panel backlight 11b (S303).

  In the release determination process of FIG. 4, since the mobile phone 1 is not in use at first, the key guard function is set and the illumination of the display surface 11c is turned off. Here, when the user tries to use the mobile phone 1, the mobile phone 1 is held by the user. Therefore, first, the determination unit 102 determines whether or not the mobile phone 1 is held (S401).

  That is, in response to the detection signal from the acceleration sensor 307, it is determined whether or not any change in acceleration in the three axis directions exceeds a predetermined second threshold value. When the mobile phone 1 is lifted, the determination unit 102 determines that any acceleration in the three-axis directions has exceeded the second threshold value. Further, the determination unit 102 determines the presence / absence of a detection signal from the proximity sensor 308. If the back side of mobile phone 1 is gripped by hand, determination unit 102 determines that a detection signal is received from proximity sensor 308. Thus, when it is determined that the mobile phone 1 is lifted and held in the hand from the back side, the determination unit 102 detects that the mobile phone 1 is held (S401: YES).

  When the user uses the mobile phone 1, usually, when the user holds the mobile phone 1, the front side of the mobile phone 1 is immediately brought in front of his / her face. Therefore, when the mobile phone 1 is held and used, the authentication unit 103 captures the front of the front of the mobile phone 1 with the camera module 15 (S402). The authentication unit 103 extracts a face image from the captured image. When the face image can be extracted, the authentication unit 103 obtains a feature parameter from the face image, compares the feature parameter of the face image with the feature parameter of the registered face image, calculates the matching rate of these feature parameters, and matches The rate is compared with a predetermined third threshold.

If the registered user uses the mobile phone 1 and faces the front of the mobile phone 1, the registered user's face is photographed, so the relevance rate is higher than the third threshold, and the authentication unit 103 The image is authenticated as a registered user's face. If the face authentication is successful (S403: YES), the display control unit 105 turns on the panel backlight 11b and displays an image on the display surface 11c (S404). As a result, the user can recognize that the face authentication has succeeded. Further, the function control unit 104 cancels the key guard function (S405). Thereby, since the input by the user is accepted, the user can operate the mobile phone 1.

  On the other hand, when another person other than the registered user wants to use the mobile phone 1, even if the face image is extracted from the image captured by the authentication unit 103, the face image is not the registered user's face image. The precision of the parameter is lower than the third threshold value, and authentication using a face image is not possible (S403: NO).

  Even when the registered user intends to use the mobile phone 1, the registered user's face is not at a position facing the front of the mobile phone 1, or the registered user's facial expression and orientation are If they are different, authentication using a face image cannot be performed (S403: NO). Therefore, since the authentication unit 103 performs photographing and face authentication a plurality of times, the authentication unit 103 determines whether the number of face authentications has reached a predetermined number. If the number of face authentications has not reached the predetermined number (S406: NO), the authentication unit 103 repeats photographing and face authentication (S402, S403).

  If the authentication unit 103 is not successfully authenticated even after performing authentication with a face image a predetermined number of times (S403: NO, S406: YES), the function control unit 104 maintains the key guard function and the process ends. (S407). As a result, the input to the mobile phone 1 becomes invalid, so that an erroneous operation or unauthorized use is prevented. In this case, since the panel backlight 11b is turned off, it is possible to prevent others from seeing the display content.

  As described above, according to the present embodiment, in order for the user to use the mobile phone 1, the user only needs to hold the mobile phone 1, and face authentication is started, so that no special operation is required and the user is made aware of it. Face recognition can be performed easily without any problems. Moreover, even when the cellular phone 1 equipped with the electrostatic touch sensor 12 is used with a gloved hand, face authentication can be performed.

  In addition, according to the present embodiment, when the mobile phone 1 is held and used, the user is placed in front of the mobile phone 1, so that the camera module 15 having the lens window 15 a provided in the front is used to Can shoot. Thus, by providing the lens window 15a on the front surface, the camera module 15 can photograph the user with a natural operation of using the mobile phone 1 by the user.

  Further, according to the present embodiment, it is possible to shoot and authenticate the user without giving the user a sense of resistance by starting face authentication in a state where the display surface 11c is not displayed.

  Furthermore, according to the present embodiment, even when the user is reading the e-mail or the like, even if the mobile phone 1 is placed on a desk or the like, when the mobile phone 1 is placed, the image on the display surface 11c is erased, so that There is no worry that you can see the display contents. Moreover, since the key guard function is set by placing the mobile phone 1, there is no need to manually set the key guard function, and it is possible to prevent forgetting to set the key guard function.

  Further, according to the present embodiment, power saving can be achieved by turning off and turning on the panel backlight 11b with the setting and release of the key guard function.

Second Embodiment
In the mobile phone 1 according to the first embodiment, it is detected that the mobile phone 1 is held based on outputs from the acceleration sensor 307 and the proximity sensor 308. In contrast, the cellular phone 1 according to the second embodiment differs from the cellular phone 1 according to the first embodiment in that the cellular phone 1 is detected based on outputs from the acceleration sensor 307 and the illuminance sensor 309. . In addition, the same number is attached | subjected to the thing similar to the structure of 1st Embodiment among the structures of 2nd Embodiment, and the description is abbreviate | omitted.

  FIG. 5 is a block diagram showing the overall configuration of the mobile phone 1.

  The illuminance sensor 309 is disposed on the back surface of the cabinet 10. The illuminance sensor 30924 is an illuminance detection unit that detects the amount of light irradiated on the back surface. It is desirable that the illuminance sensor 309 is arranged at a portion extending from the center to the lower part of the mobile phone 1 so that the user can easily cover the mobile phone 1 with a palm or a finger.

  The determination unit 102 determines the state of the mobile phone 1 based on outputs from the acceleration sensor 307 and the illuminance sensor 309.

  Note that the processing procedure in the second embodiment is substantially the same as the processing procedure in FIGS. 3 and 4 in the first embodiment, but the determinations in S301 and S401 are different.

  In the present embodiment, in S301 in the setting determination process of FIG. 3C, the determination unit 102 determines that any change in acceleration in the three-axis directions detected by the acceleration sensor 307 is greater than the first threshold value. Thus, it is detected that the mobile phone 1 has been moved. Further, the determination unit 102 determines that the back surface of the mobile phone 1 has become bright by determining that the illuminance detected by the illuminance sensor 309 is greater than a predetermined fourth threshold value. As described above, when the hand is released from the back of the mobile phone 1 and the back is brightened, and the mobile phone 1 is moved and moved from the front, the determination unit 102 determines that the mobile phone 1 is placed. to decide.

  In the case of the present embodiment, in S401 in the release determination process of FIG. 4, the determination unit 102 determines that any change in acceleration in the three-axis directions detected by the acceleration sensor 307 is greater than the second threshold value. Then, it is detected that the mobile phone 1 has been moved. The determination unit 102 determines that the back surface of the mobile phone 1 has become dark by determining that the illuminance detected by the illuminance sensor 309 is smaller than a predetermined fifth threshold value. As described above, when the mobile phone 1 is moved and lifted, and is held in the hand from the back side and the back side becomes dark, the determination unit 102 detects that the mobile phone 1 is held. .

  As described above, according to the present embodiment, it is possible to detect that the mobile phone 1 is held based on the output signals of the acceleration sensor 307 and the illuminance sensor 309.

<Third Embodiment>
In the mobile phone 1 of the third embodiment, the fact that the mobile phone 1 is held based on outputs from the acceleration sensor 307, the proximity sensor 308, and the illuminance sensor 309 is that the mobile phone of the first and second embodiments. Different from 1. In addition, the same number is attached | subjected to the thing similar to the structure of 1st and 2nd embodiment among the structures of 3rd Embodiment, and the description is abbreviate | omitted.

  FIG. 6 is a block diagram showing the overall configuration of the mobile phone 1.

Determination unit 102 determines the state of mobile phone 1 based on outputs from acceleration sensor 307, proximity sensor 308, and illuminance sensor 309.

  Note that the processing procedure in the third embodiment is substantially the same as the processing procedure in FIGS. 3 and 4 in the first embodiment, but the determinations in S301 and S401 are different.

  In the case of the present embodiment, in S301 in the setting determination process of FIG. 3C, the determination unit 102 determines that the acceleration change in any of the three axial directions detected by the acceleration sensor 307 is greater than the first threshold value. Then, it is detected that the mobile phone 1 has been moved. Further, the determination unit 102 detects that there is no proximity object on the back surface of the mobile phone 1 by not receiving the detection signal from the proximity sensor 308. Furthermore, the determination unit 102 detects that the back surface of the mobile phone 1 has become bright by determining that the illuminance detected by the illuminance sensor 309 is greater than the fourth threshold value. As described above, the determination unit 102 determines that the mobile phone 1 is placed when the hand is released from the back of the mobile phone 1, the back is bright, and the mobile phone 1 is moved from the front. To detect.

  In the case of the present embodiment, in S401 in the release determination process of FIG. 4, the determination unit 102 determines that any change in acceleration in the three-axis directions detected by the acceleration sensor 307 is greater than the second threshold value. Then, it is detected that the mobile phone 1 has been moved. In addition, the determination unit 102 receives a detection signal from the proximity sensor 308 and detects the presence of an adjacent object on the back surface of the mobile phone 1. Furthermore, the determination unit 102 determines that the back surface of the mobile phone 1 has become dark by determining that the illuminance detected by the illuminance sensor 309 is smaller than the fifth threshold value. As described above, when the mobile phone 1 is lifted, the hand is close to the back surface, and the back surface is dark, the determination unit 102 detects that the mobile phone 1 is held.

  As described above, according to the embodiment, it is possible to more accurately detect that the mobile phone 1 is held based on the output signals of the acceleration sensor 307, the proximity sensor 308, and the illuminance sensor 309.

<Fourth embodiment>
In the mobile phone 1 according to the first embodiment, the panel backlight 11b is turned on when authentication using a face image is possible. In contrast, the cellular phone 1 according to the fourth embodiment is different from the cellular phone 1 according to the first embodiment in that the panel backlight 11b is turned on before authentication using a face image. In addition, the same number is attached | subjected to the thing similar to the structure of 1st Embodiment among the structures of 4th Embodiment, and the description is abbreviate | omitted.

  FIG. 7 is a flowchart showing a processing procedure for canceling the key guard function. Since S501 is the same as S401, S503 and S504 are the same as S402 and S403, and S505 to S507 are the same as S405 to S407, respectively, the description of the processes in S501 and S503 to S507 is omitted.

  When it is detected that the mobile phone 1 is held, the photographing control unit photographs the front of the front of the mobile phone 1 with the camera module 15 in order to perform face authentication. The display control unit 104 displays the captured image on the display surface 11c and turns on the back panel light (S502). Then, the authentication unit 103 captures an image by the camera module 15 and collates it with the registered face image, thereby performing authentication using the face image.

  As described above, according to the above-described embodiment, the face of the user in front of the front face of the mobile phone 1 is photographed before the start of face authentication, and the photographed face image is displayed on the display surface 11c. Thereby, when the user wears sunglasses, a mask, etc., he / she can notice that he / she has to remove the sunglasses, etc. by looking at the image of the face displayed on the display surface 11c.

  Further, according to the above-described embodiment, it is understood that the user is photographed for face authentication when the photographed face image is displayed on the display surface 11c. For this reason, the user can match his / her face to the position and orientation where photographing can be performed, and authentication based on the face image can be performed more quickly.

<Fifth Embodiment>
In the mobile phone 1 according to the first embodiment, the release of the key guard function is determined only by authentication using a face image. On the other hand, in the cellular phone 1 of the fifth embodiment, when the authentication by the face image cannot be performed, the cancellation of the key guard function is determined by further performing another authentication. Different from 1. In addition, the same number is attached | subjected to the thing similar to the structure of 1st Embodiment among the structures of 5th Embodiment, and the description is abbreviate | omitted.

  As an authentication method other than the face authentication, a method such as a method in which the user moves the mobile phone 1 in a predetermined pattern is used instead of a password input method that requires the user to operate the touch sensor 12. For example, the mobile phone 1 is moved and tilted to a predetermined angle, or the mobile phone 1 is moved and moved with a predetermined motion to check whether the user of the mobile phone 1 is a registered user.

  The registration unit 101 includes a face registration unit 101 that registers a registered face image of a registered user, and a pattern registration unit 101 that registers the angle and movement of the mobile phone 1 as a registration pattern. For example, when a user draws a figure such as a star or a rectangle in the air with the mobile phone 1, the pattern registration unit 101 analyzes an acceleration change pattern from when the user starts drawing on the mobile phone 1 to when it finishes drawing. And stored in the memory 200 as a registered acceleration pattern.

  Although not shown, the authentication unit 103 performs authentication based on a face authentication unit for performing authentication using a face image, an angle of the mobile phone 1 held by the user, a direction in which the mobile phone 1 is moved by the user, and the like. A pattern authentication unit 103 is included. The face authentication unit includes a photographing control unit, a face analysis unit, and a face collation unit. The pattern authentication unit includes a pattern analysis unit and a pattern matching unit.

  The pattern analysis unit receives an acceleration signal from the acceleration sensor 307 and obtains a change in acceleration. The pattern collation unit collates the acceleration change obtained by the pattern analysis unit with the acceleration pattern registered in advance by the pattern registration unit 101, and calculates the matching rate of both. The pattern matching unit determines whether or not the authentication is successful by comparing the matching rate with a preset sixth threshold.

  Similar to the procedure of registering a registered user's face image as a registered face image in the processing procedure shown in FIG. 3A, the acceleration change pattern of the mobile phone 1 when moved by the user is stored in the memory 200 as a registered acceleration pattern. Registered in

  The display control unit 105 displays an operation menu on the display surface 11c. Here, an icon representing a process for registering a face image and an icon representing a process for registering an acceleration change pattern are provided in the operation menu. When the user touches the display position of the icon representing the process of registering the acceleration change pattern with a finger, the pattern registration unit 101 starts the process of registering the acceleration change pattern (S101). When the user selects an icon for registering a face image, the face registration unit 101 captures the face image and stores it in the memory 200 as a registered face image as described above.

The user first selects a pattern registration icon, then moves the mobile phone 1 to draw a figure, and when the user finishes drawing, selects the pattern registration icon again. As a result, the pattern registration unit 101 receives an acceleration signal from the acceleration sensor 307 during the period from when the pattern registration icon is first selected until the pattern registration icon is selected again. And the pattern registration part 101 calculates the change of acceleration, and acquires this change pattern of acceleration as registration information to the memory 200 (S102).

  The pattern registration unit 101 stores the calculated change in acceleration in the memory 200 as a registered acceleration pattern (S103).

  FIG. 8 is a flowchart showing a processing procedure for canceling the key guard function. Since S601 to S606 are the same as S401 to S406, description of these processes will be omitted.

  If the face image cannot be authenticated a predetermined number of times, the display control unit 105 displays a predetermined comment such as “Please perform an authentication operation” on the display surface 11c. This prompts the user to authenticate with a pattern of moving the mobile phone 1. When the user moves the mobile phone 1, the pattern analysis unit obtains a change in acceleration from the acceleration by the acceleration sensor 307, and the pattern matching unit calculates a matching rate between the change in acceleration and the registered acceleration pattern.

  If the movement of the mobile phone 1 is the same as the registered pattern, the pattern matching unit determines that the matching rate is greater than the sixth threshold. Accordingly, since it is determined that the mobile phone 1 has been authenticated by the moving pattern (S608: YES), the display control unit 105 displays on the display surface 11c (S604), and the function control unit 104 cancels the key guard function. (S605).

  On the other hand, if the movement of the mobile phone 1 is different from the registered pattern, the pattern matching unit determines that the matching rate is smaller than the sixth threshold. Accordingly, since it is determined that authentication cannot be performed due to the movement pattern of the mobile phone 1 (S608: NO), the function control unit 104 maintains the key guard function (S609), and the display control unit 105 does not display on the display surface 11c. . Then, since it cannot be confirmed that the user is a registered user by face authentication and pattern authentication, the CPU 100 transmits an image photographed by the photographing control unit of the face authenticating unit 103 to a predetermined contact by e-mail (S610). By setting this contact information as the address of the registered user's personal computer, the registered user can confirm the user who illegally uses the mobile phone 1 by looking at the face reflected in the image taken by the personal computer or the like. .

<Other embodiments>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications other than those described above can be made to the embodiments of the present invention. .

  For example, in the first embodiment, when it is detected that the mobile phone 1 is placed, the key guard function is set. In addition to this, or instead of this, face authentication is performed at predetermined time intervals after the key guard is released, and the key guard function can be set when the face image cannot be authenticated. Thereby, since it can be confirmed every predetermined time whether the user of the mobile phone 1 is a registered user, an unauthorized user or the like is further easily discovered.

  In the above embodiment, when the mobile phone 1 is detected to be placed, the key guard function is set. However, by providing an operation key on the side surface of the cabinet 10 and pressing the operation key, the key guard function is set. Can also be set.

Furthermore, in the fourth embodiment, the image captured before starting the face authentication is displayed on the display surface 11c. However, before starting the face authentication, an image related to face authentication other than the captured image is displayed. You can also For example, as an example of an image related to face authentication, an image of a message indicating that photographing for face authentication or face authentication is being performed, and an image such as a predetermined icon representing these are displayed on the display surface 11c. This display allows the user to know that the mobile phone 1 is performing face authentication. As another example of an image related to face authentication, a frame image and an image of a predetermined comment “please put a face in the frame” are displayed. Thereby, the user can avoid the problem that the size of the face of the photographed image is too small to acquire the feature pattern of the photographed image by matching the size of the face reflected on the display surface 11c with the frame.

  In the fifth embodiment, authentication is performed using a pattern in which the user moves the mobile phone 1, but the present invention is not limited to this authentication method. For example, an authentication method that requires an input operation by the user, such as password authentication, may be used.

  Furthermore, in the first embodiment, based on the outputs from the acceleration sensor 307 and the proximity sensor 308, in the second embodiment, based on the outputs from the acceleration sensor 307 and the illuminance sensor 309, in the third embodiment, the acceleration sensor 307. Based on the outputs from the proximity sensor 308 and the illuminance sensor 309, it was determined that the mobile phone 1 was held. On the other hand, it can be determined that the mobile phone 1 is held based on the output from only the acceleration sensor 307. That is, it can be determined whether or not the mobile terminal device 1 is held based on at least the detection signal from the acceleration sensor 307. In this case, the determination accuracy of whether or not the portable terminal device 1 is held is improved by taking into account the outputs of the other sensors 308 and 309 in addition to the output of the acceleration sensor 307.

  In the above embodiment, the acceleration sensor 307 measures the acceleration in the three-axis direction, but the acceleration sensor 307 further measures the gravitational acceleration in the Z-axis direction perpendicular to the display surface 11c and displays the gravity acceleration in the Z-axis direction. The orientation of the surface 11c can also be detected. Thereby, after the portable terminal device is held, if the display surface 11c faces upward, it can be seen that the portable terminal device is in a state of being used. Further, when the mobile terminal device is in the bag and the display surface 11c faces downward, it can be seen that the mobile terminal device is not in a state of being used.

  Furthermore, in the said embodiment, when the key guard function was set, the input from the user through the touch sensor 12 was invalidated. The range to be invalidated may be the entire input or a part thereof. That is, it is possible to set so that input in the entire area or a partial area of the display surface 11c detected by the touch sensor 12 is invalid. In addition, when an operation key is provided on the side surface of the mobile phone 1 or the like, it is possible to set so that input from all or a part of the operation key is invalid.

  Furthermore, in the above embodiment, the mobile phone 1 is used, but a mobile terminal device such as a PDA or PHS can also be used.

  Moreover, in the said embodiment, although the lens window 15a of the camera module 15 was provided in the front surface of the cabinet 10, the lens window 15a of the camera module 15 can also be provided in the back surface of the cabinet 10. FIG. In this case, the registered user knows that when the mobile phone 1 is held, the image is taken by the camera module 15 and face authentication is performed. For this reason, when the registered user holds the mobile phone 1, the camera module 15 causes the face to be photographed with the back side of the mobile phone 1 facing the face. Thereby, since the registered user's face image is acquired, this face image is collated with the registered face image, authentication by the face image can be performed, and the key guard is released. As described above, in order to release the key guard, not only face authentication but also an operation in which the back surface of the mobile phone 1 is directed toward the face is required, so that the user can be identified more accurately.

Furthermore, in the above embodiment, the mobile phone 1 can be provided with a light emitting unit. The light emitting unit is configured by an LED or the like, and emits light based on a control signal from the CPU 100. The CPU 100 outputs a control signal to the light emitting unit when shooting for face authentication is performed or when face authentication is performed. As a result, while shooting for face authentication and face authentication are being performed, light is emitted from the light emitting unit, and shooting and face authentication are notified. For this reason, the user can know that photographing and face authentication are being performed by the light from the light emitting unit. In addition, it can replace with the light emission part which alert | reports this face authentication, or can also provide a vibration part with a light emission part. The vibration unit includes a motor and the like, and vibrates based on a control signal from the CPU 100. When face authentication shooting and face authentication are performed, the vibration unit vibrates in accordance with a control signal from the CPU 100 to notify the face authentication. Further, the CPU 100 may output a control signal to the speaker 14 by performing the face authentication, emit a sound from the speaker 14, and notify the face authentication.

  In the above embodiment, the user can also set in advance whether or not to perform authentication when an incoming call is made in a state where the key guard function is set. In addition, in a state where the key guard function is set, the user can set in advance whether or not to notify an incoming call when an incoming call is received. When notifying an incoming call, the user may set an incoming call notification method in advance. This makes it possible to control incoming call notification during key guard setting and authentication at the time of incoming call based on the user's intention, improving convenience.

  In addition, the embodiment of the present invention can be variously modified as appropriate within the scope of the technical idea shown in the claims. For example, some or all of the above embodiments can be combined.

DESCRIPTION OF SYMBOLS 15 Camera module 102 Judgment part 103 Authentication part 104 Function control part 105 Display control part 200 Memory 307 Acceleration sensor 308 Proximity sensor 309 Illuminance sensor

Claims (13)

In a mobile terminal device,
A determination unit for determining that the mobile terminal device is held;
A shooting section for shooting images;
A storage unit for storing images;
An authentication unit that performs authentication by comparing an image captured by the imaging unit with an image stored in advance in the storage unit,
When the determination unit determines that the portable terminal device is held, the photographing unit starts photographing, and the authentication unit is photographed by the photographing unit based on an image stored in advance in the storage unit. Authenticate images
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1,
The determination unit determines whether the mobile terminal device is held based on acceleration acting on the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1,
The determination unit determines whether or not the mobile terminal device is held based on acceleration acting on the mobile terminal device and illuminance in the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1,
The determination unit determines whether or not the mobile terminal device is held based on acceleration acting on the mobile terminal device and whether or not an object to be detected is in proximity to the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1,
The determination unit determines whether or not the mobile terminal device is held based on acceleration acting on the mobile terminal device, the illuminance in the mobile terminal device, and whether or not an object to be detected is in proximity to the mobile terminal device. to decide,
The portable terminal device characterized by the above-mentioned. In the portable terminal device according to any one of claims 1 to 5,
An input unit for receiving an input to the mobile terminal device;
A function control unit that sets and releases a key guard function that disables input to the input unit, and
The function control unit cancels the key guard function when authentication by the authentication unit is successful.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 6,
The determination unit determines that the mobile terminal device is placed,
The function control unit sets the key guard function when the determination unit determines that the mobile terminal device is placed.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1,
An input unit for receiving an input to the mobile terminal device;
A function control unit that sets and releases a key guard function that disables input to the input unit, and
The determination unit determines that the mobile terminal device is placed,
The function control unit cancels the key guard function when authentication by the authentication unit is successful, and sets the key guard function when it is determined that the mobile terminal device is placed by the determination unit.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 7 or 8,
The determination unit determines whether or not the mobile terminal device is placed based on acceleration acting on the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 7 or 8,
The determination unit determines whether or not the mobile terminal device is placed based on acceleration acting on the mobile terminal device and illuminance in the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 7 or 8,
The determination unit determines whether or not the mobile terminal device is placed based on acceleration acting on the mobile terminal device and whether or not an object to be detected is in proximity to the mobile terminal device.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to any one of claims 1 to 11,
A display unit;
A display control unit for controlling the display unit,
The display control unit turns on the display unit when authentication by the authentication unit is successful. The mobile terminal device according to any one of claims 1 to 11,
A display unit;
A display control unit for controlling the display unit,
When the determination unit determines that the mobile terminal device is held by the determination unit, the display control unit displays an image related to face authentication on the display unit.
The portable terminal device characterized by the above-mentioned.

Images