JP2009159542A - Mobile terminal apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance security when a face image of an unauthorized user taken when lock cancellation fails. <P>SOLUTION: A control unit 17 can perform security code determination processing and image authentication processing and, in performing the security code determination processing, determines whether communication is possible by a communication part 11. In the case of failure as a result of the security code determination processing and communication being possible, the control unit transmits a photography image to a prescribed address to continue the lock and, in the case of failure as a result of the security code determination processing and communication being incapable, stores the photography image in a storage part 16, continues the lock with an input of the security code by an operation part 12 as not acceptable, and in the case of success in the security code determination processing, cancels lock. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a portable terminal device that authenticates an operator for security enhancement.

For example, personal information such as a telephone number, an e-mail address, a name, an address, or a telephone number / e-mail address of an acquaintance registered in the address book is stored in a mobile terminal device such as a mobile phone. There are many.
Therefore, accepting an operation by an unspecified operator is not preferable from the viewpoint of security.

Recently, portable terminals capable of performing face authentication for the purpose of improving security have been known.
For example, an operator's face is photographed by a camera in response to an operation, and feature information such as the position, shape and size of eyes, nose and mouth, or the shape and size of a contour in the photographed face image is stored in advance in a memory. The face authentication process is performed by comparing with the stored feature information of the regular user.
Then, when the degree of coincidence between the feature information and the feature information of the authorized user exceeds a predetermined value, the operator is determined to be an authorized user, and processing corresponding to the operation is performed (for example, see Patent Document 1). .

In addition, if the password (security code) input fails for a predetermined number of times, the camera is activated to take a picture, and the photographed image is sent to a pre-registered address, so that the user can be used in case of unauthorized use. A portable terminal device that can be identified quickly is also known (see, for example, Patent Document 2).
JP 2007-36365 A JP 2003-101640 A

  As disclosed in Patent Document 2, when unlocking fails, the operator is determined to be an unauthorized user, and the face image is transmitted to a specific address so that the authorized user can It is desired to make it easy to identify an unauthorized user who is operating the apparatus.

  However, depending on the usage environment, the captured image may not be transmitted to the registered transmission destination. Therefore, as long as the captured image cannot be transmitted, the security code is input several times mechanically. However, it can be tried and eventually it will be unlocked and used illegally.

  An object of the present invention is to provide a portable terminal device with enhanced security in a case where a face image of an unauthorized user shot when unlocking fails cannot be transmitted.

  In order to solve the above problems, a mobile terminal device of the present invention includes a communication unit that performs wireless communication, an imaging unit that captures an operator's face, an authentication unit that performs image authentication, an operation unit, and the operation unit. When a security code is input by the operation unit in a locked state, a storage unit that stores a security code that releases a lock that restricts at least a part of processing according to an operation in the A security code determination process for determining whether or not the security code matches the security code stored in the storage unit, and a control unit that performs imaging by the imaging unit based on the input of the security code and stores the image in the storage unit; The control unit determines whether the communication unit is communicable when performing the security code determination process, and If it is rejected in the communication determination process and is in a communicable state, the captured image is transmitted to a predetermined address and the lock is continued, and if it is rejected in the security code determination process and communication is not possible, the captured image is While being stored in the storage unit, the security code input by the operation unit cannot be accepted and the lock is continued. When the security code determination process is successful, the lock is released.

  In the mobile terminal device of the present invention, the control unit determines whether or not communication by the communication unit is possible when the power of the device is turned on, and a captured image is stored in the storage unit. It is determined whether or not the image is stored, and if the captured image is stored based on the security code input and stored in the storage unit, the captured image is transmitted to a predetermined address and the operation is performed. The security code may not be accepted by the unit, and the lock may be continued.

  Also, in the mobile terminal device of the present invention, when the control unit is in a state where communication is impossible, the control unit continues to determine whether or not the communication unit has returned to a communicable state and returns to a communicable state. Determining whether or not a photographed image is stored in the storage unit, and if the photographed image photographed based on the security code input is stored in the storage unit, the photographed image is transmitted to a predetermined address. In addition, the lock may be continued by disabling the input of the security code by the operation unit.

  Further, in the mobile terminal device of the present invention, when the control unit stores a captured image captured based on the code input in the storage unit and determines that communication is not possible in the communication unit, the operation unit It may be configured to display or voice notification so as to move to a communicable area after continuing to disallow security code input by the unit.

  In the mobile terminal device of the present invention, the control unit may be configured to delete the captured image that is performed based on the security code input stored in the storage unit when the code determination process is passed. Good.

  According to the mobile terminal device of the present invention, it is possible to enhance security when the face image of an unauthorized user taken when unlocking fails cannot be transmitted.

  FIG. 1 is a block diagram showing an internal configuration of the mobile terminal according to the embodiment of the present invention. Here, a mobile phone 100 is illustrated as the mobile terminal device.

  As shown in FIG. 1, the mobile phone 100 includes a communication unit 11, an operation unit 12, a display unit 13, an imaging unit 14, an authentication unit 15, a storage unit 16, and a control unit 17. Each of the blocks 11, 12, 13, 14, 15, 16, 17 is connected via a bidirectional bus 18 composed of a plurality of lines for data, address, and control. Yes.

  The communication unit 11 captures a wireless communication system, performs wireless communication with a base station (not shown) connected to the communication network, and transmits and receives various data. The various data includes voice data at the time of voice call, mail data at the time of mail transmission / reception, Web page data at the time of browsing the Web (World Wide Web), and the like.

  The operation unit 12 includes keys to which various functions are assigned, such as a power key, a call key, a numeric key, a character key, a direction key, a decision key, and a call key, for example. Is operated, the signal corresponding to the operation content is generated and output to the control unit 17 as an operator instruction.

The display unit 13 is configured using, for example, a liquid crystal display (LCD) or an organic EL display (OLED) configured by arranging a large number of pixels (a combination of light emitting elements of a plurality of colors) vertically and horizontally. An image corresponding to the video signal supplied from the unit 17 is displayed.
The display unit 13 includes, for example, a telephone number of a call destination at the time of a wireless call by the communication unit 11, a telephone number of a call source at the time of an incoming call, contents of received mail and transmitted mail, date, time, remaining battery power, Call success / failure, standby screen, etc. are displayed.

The imaging unit 14 is an imaging unit configured by a photoelectric conversion element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) image sensor and a control circuit thereof.
As will be described later, the imaging unit 14 includes an in-side camera 41 for photographing the operator himself (operator side) and an out-side camera (not shown) for photographing other subjects.
Note that the in-side camera 41 emits light toward the operator's face in a state in which the housing is operated (that is, in an opened state in which the operator holds the housing and directs his / her line of sight toward the display unit 13). It is preferably a camera that is attached to the casing and used for a videophone or the like with the axis set.

The authentication unit 15 uses the face image of the operator captured by the in-side camera 41 and the face image of the authorized user stored in advance in the storage unit 16 for each feature (for example, the position of the eye, nose, mouth, face contour). , Size, shape, etc.), and if the matching rate of features between the operator's face image and the regular user's face image is equal to or greater than a predetermined threshold, the authentication is passed. It is a processing device that executes face authentication processing that determines that authentication fails when it is less than a threshold value.
Note that in the present embodiment, photographing of a face image by the in-side camera 41 and face authentication processing of the authentication unit 15 are collectively referred to as authentication processing.

The storage unit 16 stores various data used for various processes of the mobile phone 100.
The storage unit 16 includes, for example, a computer program executed by the control unit 17, an address book for managing a communication partner's telephone number, e-mail address, face image, and the like, a sound file for playing a ringtone or an alarm sound, a standby Screen image file, various setting data, temporary data used in the process of the program, face image of a regular user used for face authentication processing of the authentication unit 15, face image of a non-regular user described later, Further, a security code for unlocking is stored.
The storage unit 16 includes, for example, a nonvolatile storage device (nonvolatile semiconductor memory, hard disk device, optical disk device, etc.), a randomly accessible storage device (eg, SRAM, DRAM), or the like.

The control unit 17 comprehensively controls the overall operation of the mobile phone 100.
In other words, the control unit 17 performs various processes of the mobile phone 100 (voice calls performed via a circuit switching network, creation and transmission / reception of e-mails, browsing of Internet websites, etc.) according to the operation of the operation unit 12. Control the operation of each block described above (transmission and reception of signals in the communication unit 11, display of an image on the display unit 13, imaging processing in the imaging unit 14, face authentication processing in the authentication unit 15) and the like. To do.
The control unit 17 includes a computer (microprocessor) that executes processing based on a program (operating system, application program, etc.) stored in the storage unit 16, and performs the above-described processing according to a procedure instructed in this program. Execute. That is, instruction codes are sequentially read from programs such as an operating system and application programs stored in the storage unit 16 to execute processing.

In addition, when a security code is input by the operation unit 12 in the locked state, the control unit 17 determines whether or not the input security code matches the security code stored in the storage unit 16. Based on the input of the security code, an image is captured by the image capturing unit 14 and stored in the storage unit 16, and at the same time, the authentication unit 15 performs image authentication based on the captured image.
The control unit 17 also determines whether or not the communication unit 11 is in a communicable state when performing the above-described code determination process. When the code determination process fails and communication is not possible, the photographed image is stored in the storage unit 16 and the security code input from the operation unit 12 cannot be accepted and the lock is continued. If the code determination process is acceptable, control for releasing the lock is also performed. Details of the security code determination process and the image recognition process of the control unit 17 will be described later.

  FIG. 2 is a diagram illustrating an example of the appearance of the mobile phone 100. As shown in FIG. 2, the portable terminal device according to the present embodiment assumes a foldable mobile phone 100. As shown in FIGS. 2A and 2B, the mobile phone 100 includes an upper housing 101, a lower housing 102, and a hinge portion 103.

2A is a diagram illustrating a state (open state) in which the mobile phone 100 is opened, and FIG. 2B is a diagram illustrating a folded state (closed state) of the mobile phone 100. .
As shown in FIG. 2A, the upper housing 101 is provided with the display unit 13 and the in-side camera 41 on one side that is not exposed to the outside when the mobile phone 100 shown in FIG. 2B is closed. ing. Further, as shown in FIG. 2A, an operation unit 12 is disposed on the lower housing 102 on one side that is not exposed to the outside when the cellular phone 100 shown in FIG. 2B is closed. Note that an out-side camera (not shown) is disposed on the back surface of the lower housing 102 (one surface that is not exposed to the outside when the cellular phone 100 is closed).

  That is, in the open state of the mobile phone 100 shown in FIG. 2A, when the operator tries to perform an operation input through the operation unit 12 while looking at the display unit 13, the operator's face image is captured by the in-side camera 41. Can be taken.

The hinge portion 103 opens and closes the upper housing 101 and the lower housing 102, and allows the transition between the open state of the mobile phone 100 shown in FIG. 2 (a) and the closed state shown in FIG. 2 (b). It is a hinge mechanism which has. Note that the open / closed state of the mobile phone 100 is monitored by the control unit 17, and the control unit 17 can detect the closed state of the mobile phone 100.
Specifically, for example, a closed state is detected by the control unit 17 monitoring whether a detection switch (not shown) of the lower casing 102 is pressed by a projection (not shown) arranged on the upper casing 101 ( That is, if the detection switch is pressed, it is determined to be in the closed state, otherwise, it is determined to be in the open state).
In addition, you may perform opening / closing detection not only with a switch but with various sensors.
Further, the in-side camera 41 is not limited to the upper casing 101 but may be disposed in the lower casing 102 or the hinge portion 103.

FIG. 3 is a flowchart showing the operation of the mobile terminal device according to the embodiment of the present invention, and shows the flow of processing for security code determination processing and image authentication processing by the control unit 17.
Hereinafter, with reference to the flowchart of FIG. 3, the details of the processing flow of the security code determination process and the image authentication process of the portable terminal device (control unit 17) according to the embodiment of the present invention shown in FIGS. Explained.

First, in a state where the mobile phone 100 is locked, the control unit 17 displays a security code input screen on the display unit 13 in order to release the lock. When the operator who sees this operates the operation unit 12, the control unit 17 captures this and determines whether or not the input of the security code is started (step S301).
When the input of the security code is started (step S301 “Yes”), the control unit 17 displays a message on the display unit 13 urging to take a picture at a position where the user's face image can be captured (step S302).
Subsequently, the control unit 17 activates face recognition processing by the authentication unit 15 in the background. That is, the imaging unit 14 is activated and image acquisition (preview) is started. When the authentication unit 15 can recognize the operator's face (“Yes” at step S303), the imaging unit 14 captures the operator's face image and stores it in a predetermined area of the storage unit 16 (step S303). S305).

Here, the recognition of the face image performed by the authentication unit 15 is performed by storing the feature information such as the position and shape / size of the eyes, nose, and mouth or the shape and size of the contour in the captured face image. 16 is realized by comparing with the feature information of the regular user stored in advance in 16.
Here, if the degree of coincidence between the feature information and the feature information of the authorized user exceeds a predetermined value, the operator is determined to be an authorized user, and if it is equal to or less than the specified value, the operator is determined to be an unauthorized user. Then, the captured image of the unauthorized user is stored in a predetermined area of the storage unit 16.

  If the face image cannot be recognized (“No” at step S303), the control unit 17 repeatedly executes the processes of steps S302 and S303, but detects a time over of the time allowed for the face image recognition process. If YES in step S304, the process returns to the security code input start determination process in step S301.

Subsequently, the control unit 17 determines whether or not transmission by the communication unit 11 is possible (step S306). Here, the state where transmission is possible means that power is supplied to the mobile phone 100 and the mobile phone 100 is in a range from the base station.
Here, for example, when the power is turned off or the operator is out of service area and is in a transmission disabled state (step S306 “No”), the control unit 17 determines that the transmission of the face image is impossible. (For example, flag area) is saved (set to ON) (step S307), and a message prompting the user to move to a place where transmission is possible is displayed on the display unit 13, and the lock state in which key operations are not accepted is continued. (Step S308).
Note that a message prompting the user to move to a place where transmission is possible may be output as a voice instead of being displayed.

On the other hand, when it is in a transmittable state (step S306 “Yes”), the control unit 17 acquires the input result of the security code (step S309), and the acquired security code is stored in the security code stored in the storage unit 16. It is determined whether or not they match (step S310).
When the security codes match (step S310 “Yes”), the control unit 17 releases the lock state (step S311). On the other hand, if they do not match (step S310 “No”), the fact that the transmission of the face image is possible is stored in the flag area of the storage unit 16 (step S312), and the face image captured via the communication unit 11 is stored in a predetermined area. The message is transmitted to the destination (step S313), and the process returns to the security code input determination process in step S301.

In addition, according to the flowchart shown in FIG. 3, it is shown that the processing is executed in the order of the face recognition processing and the security code determination processing.
Although not shown in the flowchart of FIG. 3, when the control unit 17 passes the security code determination process (step S <b> 310), the imaging performed based on the security code input stored in the storage unit 16. Erase the image.
For this reason, it is possible to adopt a configuration in which a captured image is stored only when it is stored, so that there is an advantage that security can be improved by simple control with simple configuration.

As described above, when performing the security code determination process, the control unit 17 determines whether the communication unit 11 is in a communicable state. If the image is transmitted to a predetermined address and locked, and the security code determination process fails and communication is not possible, the photographed image is stored in the storage unit 16 and the security code input from the operation unit 12 cannot be accepted. Control is performed so that the lock is released when the security code determination process passes.
For this reason, in a state where the operator's face image cannot be transmitted, the unlocking is not performed, and the face image is transmitted even if, for example, a battery is taken out between shooting and transmission. Since the lock is not released until the release, the security is enhanced when the face image of the unauthorized user taken when the unlocking fails cannot be transmitted.

FIG. 4 is a flowchart showing the operation of the mobile terminal device according to the embodiment of the present invention, and shows the flow of processing at the next startup when the power is turned off in a transmission waiting state.
Hereinafter, with reference to the flowchart of FIG. 4, a process flow at the next startup when the power is turned off in the transmission waiting state of the mobile terminal device shown in FIGS. 1 and 2 will be described in detail.

When the power is turned on and restarted, the control unit 17 first refers to the flag area of the storage unit 16 and checks the state of the flag indicating the image transmission disabled state.
When this flag is set to ON (step S401 “Yes”), the control unit 17 further determines whether or not the communication unit 11 is in a communicable state (step ST402).
If the above flag is not set to ON (step ST401 “No”), the control unit 17 sets the mobile phone 100 to a locked state in which the key operation by the operation unit 12 is not accepted (step S404).

When it is determined that the communication unit 11 is in a communicable state because the communication unit 11 has changed from the out-of-service state to the in-service state (step S402 “Yes”), the control unit 17 is stored in a predetermined area of the storage unit 16. The face image of the operating operator is transmitted to a predetermined destination (step S403), and a locked state in which the key operation by the operation unit 12 is not accepted is set (step S404).
On the other hand, when the communication is disabled (“No” at step S402), the control unit 17 displays a message prompting the user to move to a place where transmission is possible on the display unit 13, and does not accept a key operation by the operation unit 12. The lock state is continued (step S405), and the process returns to the transmittable state determination process in step S402.

That is, when the power of the apparatus is turned on, the control unit 17 determines whether or not communication by the communication unit 11 is possible and determines whether or not a captured image is stored in the storage unit 16. If the captured image captured based on the security code input is stored in the storage unit 16, the captured image is transmitted to a predetermined address and the code input by the operation unit 12 cannot be accepted. Continue locking.
Thus, according to the mobile terminal device according to the embodiment of the present invention, the control unit 17 determines whether the face image used for authentication is stored in the storage unit 16 when the power is turned on. Since it is specified that the process has failed, even when the power is turned off without waiting for the final determination, the facial image of the unauthorized user can be transmitted when the power is turned on next time.

FIG. 5 is a flowchart showing the operation of the mobile terminal apparatus according to the embodiment of the present invention, and shows the flow of processing during face image transmission.
Hereinafter, the flow of processing when transmitting the face image of the mobile terminal device shown in FIGS. 1 and 2 will be described in detail with reference to the flowchart of FIG.

The transmission of the operator's face image is started through the security code determination process and the image authentication process by the control unit 17 shown in FIG. 3. At this time, the control unit 17 is stored in a predetermined area of the storage unit 16. Control is performed so as to transmit the face image of the unauthorized user being sent to a predetermined destination via the communication unit 11 (step S501).
If the image transmission fails (step S502 “No”), the control unit 17 determines whether or not the communication unit 11 is in a transmittable state (step S503). For example, the mobile phone 100 is out of service. If it is determined that the area has returned to the communication range due to a transition from the out-of-range state to the in-range state or the like (step S503 “Yes”), the captured image transmission processing shown in step S501 is performed. Returning, the face image of the unauthorized user stored in a predetermined area of the storage unit 16 is transmitted to a predetermined destination, and the captured image is deleted.

  On the other hand, when the communication unit 11 is in a transmission disabled state (step S503 “No”), the control unit 17 displays a message prompting the user to move to a place where transmission is possible on the display unit 13, and the key by the operation unit 12 is displayed. The locked state where the operation is not accepted is maintained (step S504), and the process returns to the communicable state determination process of step S503.

That is, the control unit 17 continues to determine whether or not the device has returned to the service area when the device is out of service area. When the device returns to the service area, the control unit 17 determines whether or not the captured image is stored in the storage unit 16 and stores it. If the captured image captured based on the security code input is stored in the unit 16, the captured image is transmitted to a predetermined address, and the security code input from the operation unit 12 cannot be accepted and the lock is continued. To do.
In addition, when the control unit 17 stores a captured image captured based on the security code input in the storage unit 16 and determines that the communication unit 11 cannot communicate, the control unit 17 accepts the input of the security code by the operation unit 12. After continuing disabling, display or voice notification is made so as to move to a communicable area. For this reason, it is possible to provide a portable terminal device that is friendly to beginners, in addition to being able to transmit an image of a fraudulent user's face taken when returning to the area. In addition, since the image is transmitted when the power is turned on or the area is restored based only on whether or not there is a captured image based on the input of the security code, the software processing can be made very simple.

As described above, according to the portable terminal device according to the embodiment of the present invention, since the lock cannot be continuously released until the image of the unauthorized user is transmitted, the security code input is mechanically performed. Unlocking by repeating trials can be made difficult.
In addition, unless the result of the security code input determination is passed, the operator who has attempted unauthorized use is not allowed to use the telephone call or communication function, so damage due to billing can be prevented. Of course, it is possible to prevent the leakage of personal information caused by browsing the phone book and sent / received mail. In addition, when a legitimate user discovers his / her mobile terminal device by making the transmitted facial image of an unauthorized user into a moving image with sound or adding position information such as GPS (Global Positioning System) It is also possible to help.

  In addition, according to the above-described mobile terminal device according to the embodiment of the present invention, only the mobile phone 100 is illustrated as the mobile terminal device. However, for example, a PDA (Personal Digital Assistants), an electronic notebook, a game having the same configuration is used. The present invention can be similarly applied to a case where it is mounted on a machine or a notebook PC.

Further, the functions of the authentication unit 15 and the control unit 17 shown in FIG. 1 may be realized entirely by software, or at least a part thereof may be realized by hardware.
For example, when the control unit 17 performs security code determination processing, the control unit 17 determines whether the communication unit 11 is in a communicable state. If the security code determination process fails and communication is not possible, the captured image is stored in the storage unit 16 and the security code input from the operation unit 12 cannot be accepted. If the security code determination process continues, the series of data processing for releasing the lock may be realized on the computer by one or a plurality of programs, and at least a part thereof is realized by hardware May be.

It is a block diagram which shows an example of the internal structure of the portable terminal device which concerns on embodiment of this invention. It is a figure which shows an example of the external appearance structure of the portable terminal device which concerns on embodiment of this invention. It is a flowchart which shows operation | movement (security code determination and face authentication determination) of the portable terminal device which concerns on embodiment of this invention. It is a flowchart which shows operation | movement (processing at the time of next starting when a power supply is turned OFF in the transmission waiting state) which concerns on embodiment of this invention. It is a flowchart which shows operation | movement (flow of transmission operation | movement) of the portable terminal device which concerns on embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Communication part, 12 ... Operation part, 13 ... Display part, 14 ... Imaging part, 15 ... Authentication part, 16 ... Memory | storage part, 17 ... Control part, 18 ... Two-way bus, 41 ... Inner side camera, 100 ... Mobile phone.

Claims (5)

A communication unit for performing wireless communication;
An imaging unit for photographing the face of the operator;
An authentication unit that performs image authentication;
An operation unit;
A storage unit that stores a security code for releasing a lock that restricts at least a part of the process according to the operation in the operation unit;
When a security code is input by the operation unit in a locked state, security code determination processing is performed to determine whether the input security code matches a security code stored in the storage unit, and the security code A control unit that performs imaging by the imaging unit based on the input and stores the image in the storage unit,
The controller is
When performing the security code determination process, determine whether or not the communication unit can communicate,
If the security code determination process fails and communication is possible, the captured image is transmitted to a predetermined address and locked.
If the security code determination process fails and communication is not possible, the captured image is stored in the storage unit and the security code input by the operation unit cannot be accepted and the lock is continued.
If the security code determination process passes, the lock is released.
The portable terminal device characterized by the above-mentioned. The controller is
When the power of its own device is turned on, it is determined whether or not communication by the communication unit is possible, it is determined whether or not a captured image is stored in the storage unit, communication is possible, and If the captured image captured based on the security code input is stored in the storage unit, the captured image is transmitted to a predetermined address, and the security code input from the operation unit cannot be accepted, and the lock is continued. The mobile terminal device according to claim 1. The controller is
When the device itself is in a state where communication is not possible, it continues to determine whether or not it has returned to a state where communication is possible, and when it returns to a state where communication is possible, it is determined whether or not a captured image is stored in the storage unit. If the photographed image photographed based on the security code input is stored in the storage unit, the photographed image is transmitted to a predetermined address and the security code input by the operation unit cannot be accepted. The mobile terminal device according to claim 1, wherein locking is continued. The controller is
When a photographed image photographed based on the code input is stored in the storage unit and it is determined that communication is not possible in the communication unit, after continuing the acceptance of the security code input by the operation unit, The mobile terminal device according to any one of claims 1 to 3, wherein the mobile terminal device is displayed or notified by voice so as to move to a communicable area. The controller is
5. The captured image performed based on the security code input stored in the storage unit is deleted when the security code determination process is passed. 5. Mobile terminal device.

Images