JP5523001B2 - Electronic device, unlocking method, program, and recording medium - Google Patents

Description

  The present invention relates to an electronic device, an unlocking method, a program, and a recording medium that prevent unauthorized use by others.

  Conventionally, many mobile phones on the market have a camera function. For example, Patent Document 1 describes a technology of a mobile phone having a face recognition function. Moreover, recent cellular phones have a lock release function that uses the face recognition function to release yesterday's usage restrictions such as a mail function and a call function.

JP 2009-129419 A

  However, the face authentication function sometimes has a problem that the lock is released by the face of a user other than a legitimate user (a user who has registered a face image in the storage unit of the mobile phone).

  An object of the present invention is to provide an electronic device, an unlocking method, a program, and a recording medium that can determine unlocking more accurately than in the prior art when unlocking is performed based on face recognition.

  An electronic apparatus according to the present invention is imaged by an imaging unit, a display unit that can display an image captured by the imaging unit, an operation unit, a storage unit that stores a predetermined image, and the imaging unit. When it is determined whether the captured image displayed on the display unit matches the predetermined image stored in the storage unit, and it is determined that the captured image matches the predetermined image, the function restriction is performed. A lock release unit to be released; and a selection unit that selects a predetermined display area of the display unit based on an operation received by the operation unit, wherein the lock release unit includes the predetermined image of the captured image. When the captured image displayed in the display area matches the predetermined image, the function restriction is released.

  More preferably, in the electronic device, the selection unit selects a plurality of the predetermined display areas based on an operation on the operation unit, and the lock release unit includes a plurality of the captured images on the display unit. If it is determined that the images sequentially displayed and sequentially displayed in the plurality of predetermined display areas coincide with the predetermined image, the function restriction is released.

  More preferably, the electronic device determines whether or not the images displayed in the plurality of predetermined display areas match the predetermined images according to the order selected by the selection unit. .

  The unlocking method of the present invention includes a step of selecting a predetermined display area of the display unit based on an operation by the operation unit, a predetermined image, the predetermined display area, a storing step, and a function. A step of limiting, a step of capturing a captured image by the imaging unit, a step of displaying a captured image captured by the imaging unit on a display unit, and the predetermined display area captured by the imaging unit Determining whether or not the captured image displayed on the screen matches the predetermined image stored in the storage unit, and when determining that the captured image matches the predetermined image, releasing the function restriction And comprising.

  The program of the present invention is a program for selecting a predetermined display area of the display unit on the computer based on an operation by the operation unit, a predetermined image, the predetermined display area, and a procedure for storing and storing, A procedure for performing function restriction, a procedure for capturing a captured image by an imaging unit, a procedure for displaying a captured image captured by the imaging unit on a display unit, and the predetermined display captured by the imaging unit It is determined whether the captured image displayed in the area matches the predetermined image stored in the storage unit, and when it is determined that the captured image matches the predetermined image, the function restriction is released. And execute the procedure.

  The recording medium of the present invention includes a procedure for selecting a predetermined display area of the display unit based on an operation by the operation unit, a predetermined image, the predetermined display area, and a storing and storing procedure on a computer. , A procedure for performing function restriction, a procedure for capturing a captured image by an imaging unit, a procedure for displaying a captured image captured by the imaging unit on a display unit, and a predetermined image captured by the imaging unit It is determined whether the captured image displayed in the display area matches the predetermined image stored in the storage unit, and if it is determined that the captured image matches the predetermined image, the function restriction is released. And a program for executing the program.

  According to the present invention, unauthorized use can be prevented.

FIG. 1 is a block diagram for explaining each configuration of the mobile terminal. FIG. 2 is a diagram illustrating an example of the appearance of the mobile terminal. FIG. 3 is a diagram illustrating a state of the upper housing 101 when a path for unlocking is input. FIG. 4 is a flowchart showing how the face recognition unit 5 and the control unit 7 perform processing. FIG. 5 is a view showing an embodiment showing the state of the upper housing 101 in the unlock mode. FIG. 6 is a view showing an embodiment showing a state of the upper housing 101 in the unlock mode. FIG. 7 is a view showing an embodiment showing the state of the upper housing 101 in the unlock mode. FIG. 8 is a view showing an embodiment showing a state of the upper housing 101 in the unlock mode.

  The first embodiment of the present invention will be described below.

<First Embodiment>
A portable terminal 100 described below is an example of a portable terminal device of the present invention.
FIG. 1 is a block diagram for explaining each configuration of the mobile terminal 100.
As illustrated in FIG. 1, the mobile terminal 100 includes a communication unit 1, an operation unit 2, a display unit 3, a camera 4, a storage unit 6, and a control unit 7.

  The communication unit 1 captures a wireless communication system, performs wireless communication with a base station (not shown) connected to the communication network, and transmits and receives various data. The various data includes voice data at the time of voice call, mail data at the time of mail transmission / reception, Web page data at the time of Web browsing, and the like.

  The operation unit 2 includes, for example, keys to which various functions are assigned such as a power key, a call key, a numeric key, a character key, a direction key, a decision key, and a call key. Is operated, the signal corresponding to the operation content is generated and output to the control unit 7 as an operator instruction.

  The display unit 3 is configured using, for example, a liquid crystal display (LCD) or an organic EL display (OLED) configured by arranging a large number of pixels (a combination of light emitting elements of a plurality of colors) vertically and horizontally. An image corresponding to the video signal supplied from 7 is displayed. The display unit 3 includes, for example, a telephone number of a call destination at the time of wireless call by the communication unit 1, a caller telephone number at the time of incoming call, contents of received mail and transmitted mail, date, time, remaining battery power, Call success / failure, standby screen, etc. are displayed.

The camera 4 is an image pickup means including a photoelectric conversion device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a control circuit thereof, and the like. The camera 4 includes an in-side camera 41 (imaging unit) for photographing the operator himself (operator side) and an out-side camera 42 for photographing other subjects.
Note that the in-side camera 41 emits light toward the operator's face in a state in which the case is operated (that is, in an open state in which the operator holds the case and directs his / her line of sight toward the display unit 3). It is preferably a camera that is attached to the casing and used for a videophone or the like with the axis set.

  The control unit 7 (selection unit, lock release unit) includes an operator's face image (captured image) captured by the in-side camera 41, and a regular user's face image (predetermined image) stored in advance in the storage unit 6. Are extracted by extracting each feature (for example, the position, size, shape, etc. of the eyes, nose, mouth, and face outline), and the matching rate of the features between the operator's face image and the regular user's face image is determined. A processing device that performs a face recognition process that determines that a face matches when it is equal to or greater than a predetermined threshold value, and determines that a face does not match when it is less than a predetermined threshold value .

  The storage unit 6 stores various data used for various processes of the mobile terminal 100. For example, a computer program executed by the control unit 7, an address book for managing a communication partner's telephone number, e-mail address, face image, etc., an audio file for playing a ring tone or an alarm sound, an image file for a standby screen , Various setting data, temporary data used in the process of the program, and the face image of the authorized user used for the face recognition process of the control unit 7 The storage unit 6 is configured by, for example, a non-volatile storage device (non-volatile semiconductor memory, hard disk device, optical disk device, etc.), a randomly accessible storage device (eg, SRAM, DRAM), or the like.

  Further, the control unit 7 comprehensively controls the overall operation of the mobile terminal 100. That is, various processes of the mobile terminal 100 (voice calls performed via a circuit switching network, creation and transmission / reception of e-mails, browsing of Internet websites, etc.) are executed in an appropriate procedure according to the operation of the operation unit 2. As described above, the operation of each block described above is controlled. The control unit 7 includes a computer (microprocessor) that executes processing based on a program (operating system, application program, etc.) stored in the storage unit 6, and performs the above-described processing according to a procedure instructed in this program. Run. That is, instruction codes are sequentially read from programs such as an operating system and application programs stored in the storage unit 6 to execute processing.

In addition, when an operation for executing various functions of the mobile terminal 100 is performed via the operation unit 2, the control unit 7 determines whether to perform face recognition processing for the operation. Execute the process. When it is determined that the face recognition process should be performed in the determination process, the control unit 7 causes the in-side camera 41 to capture the face image of the operator and executes the face recognition process. Otherwise, the face recognition process is performed. The function requested by the operation is not executed.
Details of the determination process of the control unit 7 will be described later.

FIG. 2 is a diagram illustrating an example of the appearance of the mobile terminal 100.
As shown in FIG. 2, the portable terminal 100 of this embodiment assumes a foldable mobile phone. As illustrated in FIG. 2, the mobile terminal 100 includes an upper housing 101, a lower housing 102, and a hinge unit 103.

FIG. 2A is a diagram illustrating a state in which the mobile terminal 100 is opened (open state), and FIG. 2B is a diagram illustrating a state in which the mobile terminal 100 is folded (closed state). .
As shown in FIG. 2A, the display unit 3 and the in-side camera 41 are arranged on the upper casing 101 on one side that is not exposed to the outside when the portable terminal 100 shown in FIG. 2B is closed. ing. In addition, as shown in FIG. 2A, the lower housing 102 is provided with the operation unit 2 on one surface that is not exposed to the outside in the closed state of the mobile terminal 100 shown in FIG. In addition, an out-side camera 42 (not shown) is disposed on the back surface of the lower housing 102 (one surface that is not exposed to the outside when the mobile terminal 100 is closed).
That is, in the open state of the mobile terminal 100 shown in FIG. 2A, when the operator tries to perform an operation input via the operation unit 2 while looking at the display unit 3, the operator's face image is captured by the in-side camera 41. Can be taken.

The hinge portion 103 opens and closes the upper housing 101 and the lower housing 102, and allows the transition between the open state of the mobile terminal 100 shown in FIG. 2A and the closed state shown in FIG. 2B. It is a hinge mechanism which has. Note that the open / closed state of the mobile terminal 100 is monitored by the control unit 7, and the control unit 7 can detect the closed state of the mobile terminal 100. Specifically, for example, a closed state is detected by the control unit 7 monitoring whether or not a detection switch (not shown) of the lower casing 102 is pressed by a projection (not shown) disposed on the upper casing 101 ( That is, if the detection switch is pressed, it is determined to be in the closed state, otherwise, it is determined to be in the open state). In addition, you may perform opening / closing detection not only with a switch but with various sensors.
Further, the in-side camera 41 is not limited to the upper casing 101 but may be disposed in the lower casing 102 or the hinge portion 103.

  In addition, the control unit 7 can accept selection of a predetermined area (predetermined display area) used in an unlock mode described later from the operation unit 2 when not in the unlock mode. Hereinafter, selection of the predetermined area will be described with reference to FIG. Note that how the predetermined area selected at this time is used in the unlock mode will be described in detail later.

  FIG. 3 is a diagram illustrating a state of the display unit 3 in a state where a predetermined area is received. As shown in FIG. 3A, the display unit 3 displays predetermined regions R1, R2, R3, and R4 that are surrounded by broken lines, for example, which are formed from four regions. At the same time, the display unit 3 performs a display requesting selection of an area, for example, “Please select the first area”.

  And the control part 7 receives selection of one of the said 4 area | regions, for example from the cross key of the operation part 2. FIG. In the present embodiment, for example, a predetermined area R1 shown at the upper left of the drawing is selected as the first area.

  After receiving the selection of the first area from the operation section 2, the display section 3 displays, for example, “Please select the second area”, and requests the user to select the predetermined area again (FIG. 3B). )). The display unit 3 displays “Please select a second area” and simultaneously displays a number “1” in the predetermined area R1 indicating that the predetermined area R1 is selected as the first area. May be. When the display unit 3 performs the above-described display, when the operation unit 2 receives selection of one predetermined region from the predetermined regions R2, R3, and R4 (in this embodiment, the predetermined region R2 is the second region). The display unit 3 performs the display shown in FIG. 3C.

  After the selection of the second area is completed, the display unit 3 displays “Please select the third area” this time. At the same time, the display unit 3 may display “2” at the center of the predetermined region R2 selected as the second region. Then, similarly to the previous time, when the selection of the third area is accepted from the operation unit 2 (in this embodiment, it is assumed that the predetermined area R4 is selected as the third area), the display unit 3 is shown in FIG. Display.

  As described above, when the operation unit 2 accepts the selection of the third region, the display unit 3 displays, for example, “Please select the fourth region” and “3” in the predetermined region R4. And the operation part 2 receives selection of a 4th area | region. In this embodiment, it is assumed that the predetermined area R3 is selected as the fourth area.

  Thus, when not in the unlock mode, the control unit 7 accepts selection of the first region, the second region, the third region, and the fourth region. For example, in the present embodiment, the control unit 7 is configured such that “first region... Predetermined region R1” “second region... Predetermined region R2” “third region. A combination of “area... Predetermined area R3” is stored in the storage unit 6.

<First Embodiment>
Next, the unlocking mode will be described with reference to FIGS.
The unlock mode is a mode in which functions such as a call function and a mail function can be used based on the face image captured by the in-side camera 41.
FIG. 4 is a flowchart illustrating a procedure in which the control unit 7 performs processing in the unlock mode. FIG. 5 is a diagram illustrating the state of the display unit 3 when the control unit 7 performs the processes in S5, S6, S7, S8, S9, S10, S11, S12, and S13 of FIG. is there.

  First, the control unit 7 detects by polling whether the mobile terminal 100 has accepted the activation of the unlock mode (S1). For example, the mobile terminal 100 recognizes that the activation of the unlock mode has been accepted when the state of the housing is switched from the closed state to the open state, and transmits the activation acceptance to the control unit 7 (S1). / Y). Further, the control unit 7 may shift to the unlock mode when the operation unit 2 accepts the operation.

  When the unlock mode is activated, the control unit 7 causes the in-side camera 41 to start imaging (S2). Moreover, the control part 7 displays the image which the in side camera 41 started in S2 images on the display part 3 in real time (S3). Moreover, the control part 7 starts the timer not shown (S4). The timer notifies the control unit 7 that the predetermined time p has elapsed when the predetermined time has elapsed since being activated in S4. The predetermined time is set in advance within a range of, for example, several tens of seconds to several hundreds of seconds.

  When the control unit 7 sets the timer in S4, the control unit 7 refers to the storage unit 6 so that the first area is the predetermined area R1, the second area is the predetermined area R2, and the third area is the predetermined area. It is the region R4 and recognizes that the fourth region is the predetermined region R3. And the control part 7 tries polling detection of a user's face in predetermined area | region R1 which is a 1st area | region (S6).

  If the user's face cannot be detected in the predetermined region R1 for a predetermined time (S6 / N), the control unit 7 determines that the lock cannot be released and prevents the user from releasing the lock, so The process ends (S5 / Y, S14).

  If the regular user's face (a user's face image stored in advance in the storage unit 6) can be recognized in the predetermined region R1 before the time-out (FIG. 5A), the control unit 7 The recognition of the user's face is started in the predetermined region R2 (S8). Then, similarly to the face recognition in the first area, when a time-out occurs during the face recognition in the predetermined area R2 (S7 / Y), the control unit 7 ends the lock release mode (S14). If the face recognition in the predetermined area R2 is successful (S8 / Y, FIG. 5B), the controller 7 then starts face recognition in the predetermined area R4, which is the third area (S10).

  If a time-out occurs during face recognition in the predetermined region R4 (S9 / Y), the control unit 7 ends the lock release mode (S14). If the face recognition in the predetermined area R4 is successful within the predetermined time (FIG. 5C), the control unit 7 starts the face recognition in the predetermined area R3 that is the fourth area (S10 / Y ).

  Then, similarly to the processing in the first region to the third region, if a time-out occurs during face recognition in the predetermined region R3 (S11 / Y), the control unit 7 ends the lock release mode (S14). If the face recognition in the predetermined region R3 is successful within a predetermined time (FIG. 5D), the control unit 7 releases the function restriction lock (S13).

  As described above, according to the present embodiment, when unlocking the user, in addition to the face recognition, the order of transition of the place where the face is recognized (in this embodiment, the predetermined area R1 → the predetermined area R2 → the predetermined area R4 → the predetermined area). The order of region R3) can also be used for unlocking.

  Therefore, even if another person who has a face similar to a regular user and is determined to be a regular user only with face recognition technology tries to unlock with his / her face (a face other than the regular user not stored in the storage unit), Unless the other person knows the predetermined place where face recognition is performed, the other person cannot release the lock. Therefore, the mobile terminal 100 can make security robust.

  In the above-described embodiment, only the transition of the face position is combined with the face recognition technology, but the embodiment is not limited thereto.

<Second Embodiment>
For example, when the in-side camera 41 is moved away from or close to the user's face by the user, the change in the size of the user's face that is recognized by the in-side camera 41 and displayed on the display unit 3 is also unlocked. Can be used.

  For example, when the user registers the transition of the face position by the method described with reference to FIG. 3, the control unit 7 includes the face size information in the first area to the fourth area together with the face position information. Store as unlock path.

  By doing so, in the unlock mode, the user can use not only the position of his / her face reflected on the display unit 3 but also the size of the face as a path, so that the security is further increased (FIG. 6).

<Third Embodiment>
Further, the storage unit 6 may store a plurality of regular user face images, and the control unit 7 may use the plurality of face images stored in the storage unit 6 as an unlock path. .

  For example, the user stores in the storage unit 6 four types of face images “face with closed mouth”, “face with open mouth”, “angry face”, and “winking face” in advance. Let me.

  Then, when the user registers the transition of the face position by the method described with reference to FIG. 3, the control unit 7 inputs which of the above-described four types of face images is associated with the first area. , Via the operation unit 2.

  Similarly, in the second area to the fourth area, the control unit 7 accepts selection of a face image associated with each area from the user.

  For example, in this embodiment, “face with closed mouth” in the first area, “face with open mouth” in the second area, “angry face” in the third area, and “wink” in the fourth area Assume that the face image of “the face” is associated.

  In this case, in the unlock mode, the control unit 7 changes the user's face in a predetermined order and recognizes the same face as the face image associated in advance in the first to fourth areas. Only unlock. In this way, the mobile terminal 100 can increase security (FIG. 7).

The present invention is not limited to the embodiment described above.
That is, when implementing the present invention, various modifications, combinations, sub-combinations, and alternatives may be made to the components of the above-described embodiments within the technical scope of the present invention or an equivalent scope thereof.

  Moreover, although embodiment mentioned above demonstrated the portable terminal 100 which is a foldable mobile telephone as an example of a portable terminal device, this invention is not limited to this. For example, other types of portable terminal devices such as a straight type and a slide type may be used instead of the folding type. In addition to a mobile phone, other mobile terminal devices such as a PDA (Personal Digital Assistant) or a digital camera may be used.

  The predetermined area does not have to be plural, and may be one, for example. In this case, the operation unit 2 accepts selection of the location of the predetermined region (predetermined region R5) in the display region of the display unit 3 from, for example, a cross key (FIG. 8). Then, the control unit 7 performs face recognition determination when a face image is recognized in the predetermined area in the unlock mode.

  Moreover, what is used for recognition is not limited to a face image. For example, the image may be a fingerprint image, an iris image, or an identifier or image that is not related to a living body, such as a barcode image or a specific mark image.

  In the unlock mode, the broken line indicating the boundary line between the predetermined areas R1 to R4 may not be displayed on the display unit 3. By not displaying the broken line, the security can be further improved. Of course, the number of predetermined areas is not limited to four. For example, the number of the predetermined areas may be an aspect that can be selected by the user.

  In addition, the present invention may be used when there are two threshold values for the matching rate of face recognition (when there are a high threshold value and a low threshold value). For example, when the unlocking mode is first executed, the control unit 7 calculates a matching rate between the face image captured by the camera 4 and the face image stored in the storage unit 6. Then, the control unit 7 determines whether or not the coincidence rate calculated at this time exceeds the above-described high threshold value.

  If the matching rate exceeds a high threshold, the control unit 7 releases the lock. If the coincidence rate does not exceed the high threshold, the process according to the flow of FIG. 4 is started from the flow of S4 shown in FIG. Then, when determining the coincidence of face recognition in S6, S8, S10, and S12, the control unit 7 uses a low threshold as a threshold for comparison with the coincidence rate calculated in each step.

  In this way, even if a regular user tries to unlock by high-accuracy (that is, a high matching rate is obtained) face recognition, the regular user's face image such as a light hit or a change in hairstyle can be detected. Even if the lock release fails due to the change, the lock release using the present embodiment can be performed next, and the convenience for the user is increased. In addition, when using this embodiment, the user does not repeat the failure of unlocking due to the change of the face image of his / her face in order to provide the user with unlocking by face recognition that does not require a high matching rate. The possibility increases.

DESCRIPTION OF SYMBOLS 100 ... Portable terminal, 101 ... Upper housing | casing, 102 ... Lower housing | casing, 103 ... Hinge part, 1 ... Communication part, 2 ... Operation part, 3 ... Display part, 4 ... Camera, 41 ... In side camera, 42 ... Out Side camera, 6 ... storage unit, 7 ... control unit

Claims (6)

An imaging unit;
A display unit capable of displaying an image captured by the imaging unit;
An operation unit;
A storage unit for storing a predetermined image;
It is determined whether or not a captured image captured by the imaging unit and displayed on the display unit matches a predetermined image stored in the storage unit, and when the captured image matches the predetermined image If it is determined, a lock release unit that releases the function restriction,
A selection unit that selects a predetermined display area of the display unit based on an operation received by the operation unit;
With
The unlocking unit is
An electronic device that releases a function restriction when a captured image displayed in the selected predetermined display area matches the predetermined image among the captured images. The electronic device according to claim 1,
The selection unit selects a plurality of the predetermined display areas based on an operation on the operation unit,
When the unlocking unit determines that the plurality of captured images are sequentially displayed on the display unit and the images sequentially displayed on the selected plurality of predetermined display areas match the predetermined image, the function is limited. Electronic device to be released. The electronic device according to claim 2,
An electronic device that determines whether or not an image displayed in the selected plurality of predetermined display areas matches the predetermined image in accordance with an order selected by the selection unit. Selecting a predetermined display area of the display unit based on an operation by the operation unit;
A step of storing a predetermined image, and the predetermined display area,
A step of function restriction;
Capturing a captured image in the imaging unit;
Displaying a captured image captured by the imaging unit on a display unit;
It is determined whether a captured image captured by the imaging unit and displayed in the selected predetermined display area matches a predetermined image stored in the storage unit, and the captured image and the predetermined image If it is determined that the image matches, the step of releasing the function restriction;
A method for unlocking an electronic device comprising: On the computer,
A procedure for selecting a predetermined display area of the display unit based on an operation by the operation unit;
A procedure for storing a predetermined image and the predetermined display area;
The procedure for function restriction,
A procedure for capturing a captured image in the imaging unit;
A procedure for displaying a captured image captured by the imaging unit on the display unit;
It is determined whether a captured image captured by the imaging unit and displayed in the selected predetermined display area matches a predetermined image stored in the storage unit, and the captured image and the predetermined image If you determine that the image matches,
A program for running On the computer,
A procedure for selecting a predetermined display area of the display unit based on an operation by the operation unit;
A procedure for storing a predetermined image and the predetermined display area;
The procedure for function restriction,
A procedure for capturing a captured image in the imaging unit;
A procedure for displaying a captured image captured by the imaging unit on the display unit;
It is determined whether a captured image captured by the imaging unit and displayed in the selected predetermined display area matches a predetermined image stored in the storage unit, and the captured image and the predetermined image If you determine that the image matches,
A recording medium on which a program for executing the program is recorded.