JP4284309B2 - Emergency operation execution method and electronic device - Google Patents

Description

The present invention relates to an electronic device including an information communication terminal such as a mobile communication terminal or a personal computer, as well as those concerning the emergency operation execution how in the electronic device.

2. Description of the Related Art Conventionally, as this type of electronic device, a mobile phone is known in which functions such as outgoing calls and use of a communication partner list can be restricted by inputting a predetermined personal identification number. In this cellular phone, for example, when a user inputs a personal identification number on a predetermined setting screen, it is confirmed whether or not the user is an authorized user. Then, when a correct password is input and it is confirmed that the user is a legitimate user, an operation prohibit mode is set to prohibit the outgoing call and the use of the communication partner list. In this operation prohibition mode, even if an authorized user or another person tries to make a call or the like, the operation is prohibited. The operation prohibit mode can be canceled by inputting the password again on a predetermined setting screen.
There is also known a mobile phone that can remotely prohibit outgoing calls and the like (see, for example, Patent Document 1). In this cellular phone, a response message is transmitted by a call that is continued a predetermined number of times via a wireless line, and a control signal in a predetermined format is received via the wireless line while the response message is being transmitted. Based on this control signal, the transmission by dial key operation is prohibited. Thereby, when the mobile phone is lost, stolen, or misplaced, unauthorized use by a malicious third party can be prevented.
Japanese Patent Laid-Open No. 11-298600

However, in the conventional mobile phone, in order to limit the functions such as the telephone call, the personal phone number is input by operating the mobile phone or a predetermined call or control is performed on the mobile phone from another phone. There is a problem that it is inconvenient because it requires a complicated operation of transmitting a signal.
Such a problem can also occur in electronic devices other than the mobile phone as long as the function is restricted by accepting the input of the personal identification number or receiving the predetermined call or the like.

The present invention has been made in view of the above problems, and its purpose is to enable a user who operates an electronic device without performing complicated operations such as inputting a personal identification number when starting the operation of the electronic device. identify the user with no conscious, electronic devices that can perform limited and emergency operation of the function according to the setting conditions of abuse associated mode, as well, to provide emergency operation execution how in the electronic device It is.

An emergency operation execution method according to the present invention is an emergency operation execution method for causing an electronic device to execute an emergency operation, and based on a start request, a face recognition control mode for performing automatic control by face recognition of a user of the electronic device , A step of detecting the start of operation of the electronic device in a state where the face recognition control mode is activated, and photographing the face of the user of the electronic device when the start of operation of the electronic device is detected Then, a step of performing face recognition processing by comparing the image data of the face with pre-registered face recognition reference data, and the user who has started the operation based on the result of the face recognition processing, If it is determined that the user is an unauthorized user other than those who are registered in advance as the owner or user of the device, or if it is determined that face recognition is not possible, an unauthorized use support mode that performs emergency actions during unauthorized use is provided. Is set And when determining that the unauthorized use support mode is not set, when determining that the function of the electronic device is restricted, and determining that the unauthorized use support mode is set, A step of executing an emergency operation at the time of unauthorized use.
An electronic device according to the present invention includes an imaging unit provided in a part capable of photographing a face of a user of the electronic device, a face recognition reference data storage unit that stores pre-registered face recognition reference data, and the electronic device Based on the operation detection means for detecting the operation of the device and the activation request, the face recognition control mode for performing automatic control based on the user's face recognition is activated, and the operation of the electronic device is performed in the state in which the face recognition control mode is activated. When the start is detected, the face of the user of the electronic device is photographed, the face image data is compared with the face recognition reference data, and face recognition processing is performed. Based on the result of the face recognition processing When the user who started the operation is determined to be an unauthorized user other than the person who has been registered in advance as the owner or the user of the electronic device, or when it is determined that face recognition is impossible, Unauthorized use to perform emergency actions It is determined whether a corresponding mode is set, and when it is determined that the unauthorized use support mode is not set, it is determined that the function of the electronic device is restricted and the unauthorized use support mode is set. And a control means for performing control so as to execute the emergency operation at the time of unauthorized use .

Note that the “electronic device” includes any electronic device that can use at least one of a plurality of functions when operated by a user, such as a mobile communication terminal and a personal digital assistant (PDA). Equipment is included. Further, as the “mobile communication terminal”, mobile phones such as PDC (Personal Digital Cellular) system, GSM (Global System for Mobile Communication) system, TIA (Telecommunications Industry Association) system, IMT (International Mobile Telecommunications) -2000, etc. Mobile phone standardized by TD-SCDMA (MC: Multi Carrier) which is one of TD-SCDMA (Time Division Synchronous Code Division Multiple Access) methods, PHS (Personal Handyphone System), and the like.
The processing and control in the electronic device can also be realized by executing a predetermined program on a computer provided in the electronic device. Delivery of the program used in the computer may be performed using a recording medium such as an FD or a CD-ROM in which the program is recorded as digital information, or may be performed using a communication network such as a computer network.

According to the present onset Ming, at the time of the operation start of the electronic devices, without performing a complicated operation such as input of a personal identification number, to identify the user in a state in which the user is not aware of to operate the electronic devices, unauthorized use There is an effect that function restriction and emergency operation can be executed in accordance with the setting state of the response mode .

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating an example of a hardware configuration of a mobile phone 10 as a mobile communication terminal that is an electronic apparatus according to the present embodiment. The mobile phone 10 includes a main control unit 110, a wireless communication unit 111, a baseband processing unit 112, a sound input / output unit 113, a display unit 114, and an operation unit 115.

  The main control unit 110 is also called a main processing unit (MPU), and includes a CPU 110a, an internal memory 110b, and the like. The internal memory is composed of RAM, ROM and the like. The main control unit 110 controls each unit such as the wireless communication unit 111 and the baseband processing unit 112 by executing a program such as a predetermined basic OS, middleware, and device driver, and is a native platform on the software configuration. Environment and application execution environment.

  The wireless communication unit 111 includes, for example, a synthesizer, a frequency converter, a high frequency amplifier, and the like, and executes high frequency signal processing for wireless communication with the base station 41 of the mobile phone communication network 40.

  The baseband processing unit 112 performs digital processing for performing telephone and data transmission / reception communication with an external mobile communication terminal or server. The baseband processing unit 112 and the wireless communication unit 111 are connected via a D / A converter or an A / D converter.

  The sound input / output unit 113 includes a microphone, a speaker, an earphone jack, a sound signal processing unit, and the like. The analog audio signal output from the microphone is converted into a digital signal by the sound signal processing unit and sent to the main control unit 110, the baseband processing unit 112, and the like. The speaker receives an analog signal converted from a digital signal by the sound signal processing unit, and outputs a voice during a call, or outputs a ringtone for a mail, a ringing tone for a telephone, music, or the like. The speaker may be configured by separately providing a receiver speaker (receiver) for listening to voice during a call and an external output speaker for outputting ringtones, music, etc. You may comprise with one speaker so that a speaker and an external output speaker may be combined.

  The display unit 114 includes an inward main display (front display), an outward sub display (rear display), and a drawing processing unit that performs drawing processing on each sub display. The main display and the sub display are configured by, for example, an LCD (liquid crystal display), an organic EL (Organic Electroluminescence), an LED (Light-Emitting Diode) array, and the like, and various images are displayed based on commands from the main control unit 110. The display unit 114 is also used as image display means in the present invention.

  The operation unit 115 includes a plurality of keys such as a data input key (ten key, * key, # key), a call start key, a call end key, a scroll key, and a multi-function key. The operation unit 115 is used when the user gives an instruction to start or end a call, select a menu, switch screens, or input information.

  In addition, the mobile phone 10 includes an electronic wallet chip unit 116 formed of a dedicated IC chip or an IC card as an integrated circuit device for electronic money processing used in an electronic wallet service. The electronic wallet chip unit 116 is connected to a reader / writer device (hereinafter referred to as “R / W device”) 30 that is a payment terminal device for electronic wallet services installed in various facilities and stores. An antenna and a wireless communication circuit for performing contactless communication using a communication protocol are included.

  The mobile phone 10 also includes a GPS (Global Positioning System) unit 117, a camera unit 118, a sensor unit 119, a power supply unit 120, a memory card unit 121, an operation start detection unit 122, a clock unit (not shown), and the like.

  The GPS unit 117 includes a GPS receiving module, a GPS antenna, and the like, receives radio waves from a plurality of GPS satellites arranged around the earth, and based on the reception result, the latitude, longitude, and Calculate altitude data.

  As shown in FIG. 2, the camera unit 118 is configured by using a camera module 118a and a pair of cameras (an outward rear camera 108 and an inward front camera 109), and is used when photographing a person or a landscape. It is done. The inward front camera 109 of the set of cameras is also called a self-taking camera, and in this embodiment is used for face recognition and a video phone, which will be described later. The outward-facing rear camera 108 is also referred to as a stranger camera, and is mainly used for photographing a person other than the scenery or the user. Each camera 108 and 109 is configured using an optical lens and an imaging device. As the imaging device, a CCD (Charge Coupled Device) camera or a CMOS camera can be used. The camera module 118a performs optical control of the cameras 108 and 109, and interfaces with the main control unit 110.

  The sensor unit 119 includes a triaxial acceleration sensor, a triaxial geomagnetic sensor, and the like. Based on the output of the sensor unit 119, data indicating the position, orientation, posture, movement, etc. of the mobile phone 10 can be calculated.

  The power supply unit 120 includes a rechargeable battery, a power supply circuit that supplies power of a predetermined voltage from the battery to each unit, a charging circuit that charges the battery, and the like.

  The memory card unit 121 includes a memory card mounting unit (card slot) in which a memory card as a removable external storage medium is mounted, and a memory card controller for data to the memory card.

The operation start detection unit 122 includes contact sensors such as a touch sensor and a rotation sensor provided on a movable part of the housing, a contact switch such as a push switch, a contact switch, and the like. By detecting the change in the casing, the operation of the mobile phone 10, particularly the use start operation of the mobile phone 10 is detected. As the situation of the casing of the mobile phone 10 used for such detection (change in casing), there are the following.
For example, in the case of a foldable type (clamshell type) mobile phone, the closed state in the non-operation state in which the upper part of the casing and the lower part of the casing are in contact with each other and closed, and the lower part of the casing is swung. The state change of the housing between the open state at the time of the operation opened away from the can be used. When the mobile phone is flip-opened and switched from the closed state to the open state, the change in the state is detected by a sensor incorporated in the hinge or the like, or the contact state of the switch is changed, It is possible to automatically recognize that the operation of the telephone 10 has started. In particular, in the case of a swivel type mobile phone in which a part of the upper part of the case can be rotated so that the display and camera side of the upper part of the case are reversed 180 degrees, the case can be specified so that the camera facing the user can be identified. Configure to detect top rotation.
In addition, in the case of a slide-type mobile phone, the closed state when the casing lid is closed and overlapped with the casing body is closed, and the casing lid is slid to expose the operating section of the casing body. A change in the state of the housing between the open state during the open operation can be used. When switching from the closed state to the open state, it is possible to automatically recognize that the operation of the mobile phone 10 has started.
In the case of a straight type mobile phone, the non-operating state when the mobile phone case is not held by the user and the holding state when the mobile phone case is held by the user The change in the state of the housing between can be used. When the non-holding state is switched to the holding state, it can be automatically recognized that the operation of the mobile phone 10 is started.

  The clock part is composed of a clock circuit, etc., and counts the exact date and time, and if it has a time difference table, it counts the world time, time for telephone / mail communication, image shooting time stamp, clock display, etc. Generate information.

3A, 3B, and 3C are a rear view, a front view, and a side view, respectively, showing the appearance of the mobile phone 10. FIG. This mobile phone 10 is a clamshell (folding) type mobile phone having a lid (upper housing) 10A and a main body (lower housing) 10B that can swing with respect to the hinge 10C.
10 A of cover parts have the antenna 100, and can take the open state which can operate the key of the operation part 115 provided in the main-body part 10B, and the closed state which covered the key of the operation part 115. . An external speaker 101, a sub display 102, and a rear camera 108 are provided on the surface of the lid portion 10A in the open state on the opposite side to the user side. In addition, a receiving speaker 103, a main display 104, and a front camera 109 are provided on the surface of the lid 10A facing the user. On the other hand, an operation unit 115 including a plurality of keys, a transmission microphone 105, and the like are provided on the surface of the main body 10B facing the user. The user can operate the keys of the operation unit 115 while viewing the screen of the main display 104. Further, on the side surface of the main body 10B, there are provided a memory card mounting portion (card slot) 106 in which the shutter 106a is opened and closed and the memory card 200 can be attached and detached, and a headphone jack 107 to which headphones are connected in the case of hands-free or the like. It has been.

  FIG. 4 is a functional block diagram showing extracted main functions corresponding to the features of the present invention in the mobile phone 10. The cellular phone 10 according to the present embodiment includes an imaging unit 130, a face recognition reference data storage unit 131, a control information storage unit 132, an operation detection unit 133, and a control unit 134.

  The imaging unit 130 is provided at a site where the face of the user who operates the mobile phone 10 can be photographed, and is configured using the camera unit 118 described above. In the example of FIG. 3, the front camera 109 facing inward is used. When the user's face is photographed by the camera unit 118, the photograph is taken with a predetermined image size and the number of frames (for example, 10 frames / second).

  The face recognition reference data storage means 131 is a means for storing face recognition reference data used for user face recognition. An external storage medium such as the internal memory 110b or the memory card 200 in the main control unit 110 described above is used. Constructed using. The face recognition reference data is obtained and stored in advance for a person to be face-recognized, such as the owner of the mobile phone 10 or an authorized registrant, according to a face recognition method employed by the control means 134 described later. . For example, a personal identification parameter file, which will be described later, consisting of data on the position and size of facial parts such as mouths and eyes existing on the face is stored as the face recognition reference data. The face part data can be obtained from, for example, a gray pattern in the face image area.

The control information storage means 132 is means for storing function control information used for controlling a plurality of functions of the mobile phone 10, and uses an external storage medium such as the internal memory 110b or the memory card 200 in the main control unit 110 described above. Composed. For example, (1) owner, (2) registered user, and (3) unauthorized users other than the owner and registered user should set the following functions as targets of the function control. Can do. These function control information is preset and stored by the owner.
(1) Owner All functions can be used (normal operation)
(2) Registered users other than the owner <Standby>
Voice call (possible / impossible)
Mail transmission (possible / impossible)
Data communication (control)
Button (key) operation (valid / invalid)
Reference outgoing / incoming call history (possible / impossible)
Mail folder reference (Yes / No: Folder itself is not displayed)
Caller ID notification (notification / non-notification)
Address book data reference, editing (possible / impossible)
Schedule data reference, editing (possible / impossible)
Images, music, memory cards, etc.
Browse and edit data folder (Yes / No)
WEB bookmark reference, editing (possible / impossible)
Infrared communication (possible / impossible)
Set wallpaper when mobile phone is open Change settings (Yes / No)
Electronic wallet function (possible / impossible)
<Incoming call>
Answering incoming calls (Yes / No)
Button (key) operation (valid / invalid)
Destination of incoming call (display / hide (3) unauthorized user other than owner / registered user alarm (alarm sound, LED flashing, vibrator activation, etc.)
Display off Off Phone dial lock All functions lock (all lock)
Execution of operations when the unauthorized use support mode (Emergency operation mode) is set

  Here, the owner can set the function control for (2) registered user and (3) unauthorized user in advance. When there are a plurality of registered users, different function control can be set for each registered user. In addition, when face recognition cannot be performed by hiding the camera or the like, (3) function control for an unauthorized user can be applied.

  In addition, for the above mail folder and data folder, a dedicated folder is created for each registered user so that when the registered user operates, the mail and data can be referred only to the folder dedicated to the registered user. Good.

  The operation detection unit 133 is a unit that detects an operation (or operation start) of the mobile phone 10 and is configured using the operation start detection unit 122 described above.

  The control unit 134 is configured using the main control unit 110 described above, and performs the following function control. That is, when the operation of the mobile phone (electronic device) is detected, the control unit 134 captures the face of the user of the mobile phone, and based on the comparison result between the face image data and the face recognition reference data. The user is identified, and functions usable by the user are controlled based on the identification result and the function control information. The main control unit 110 as such control means functions as a face recognition controller or a face recognition engine by executing a predetermined program.

  FIG. 5 is a block diagram illustrating an example of a software interface in the mobile phone 10. The face recognition function is realized by middleware mainly composed of the face recognition controller 140 and the face recognition engine 141. Basic operations of the mobile phone 10 such as telephone and data communication via the mobile phone communication network are executed on an execution environment of a phone platform called native. This native is composed of a basic OS, various middleware, various driver software, and the like. As middleware, software such as a telephone processing software, a mail client, a mail browser, an HTTP client, a WWW browser, and an application manager is activated. The GPS control software in the GPS unit 117, the sensor control software in the sensor unit 119, and the like are executed via this native driver software.

The face recognition controller 140 is controlled by a native application 142 executed natively via a face recognition control API (Application Program Interface). For example, the following face recognition start / end, camera image set, face recognition correct / incorrect notification, and the like are controlled via the face recognition control API.
-Start of face recognition: Starts face recognition and registration of a user to be recognized, and executes initialization processing. At the same time, the personal identification parameters are set in the face recognition engine, and then data used for face recognition is sequentially set. The data used for the face recognition is not limited to a specific type, but is set according to the type of the face recognition engine used for the face recognition process. Examples of data used for face recognition include face feature information data and user data. Here, the “face feature information” is feature amount information used when detecting a face. By detecting facial feature information from the user's facial image and setting it in the facial recognition engine, facial feature point extraction can be stabilized. “User data” is data such as a user's monochrome image (thumbnail image), facial feature point coordinates, personal identification ID, facial expression movement information, and the like.
End of face recognition: The face recognition process is finished.
Camera image set: An image acquired from a camera is converted into a format that can be processed by an engine (for example, a grayscale 8-bit color image) and passed to a face recognition engine.
-Face recognition correct / incorrect notification: Returns a correct / incorrect determination result using a callback function set in advance.

The face recognition controller 140 controls various devices via the device API, and performs the following data / information / file storage and acquisition processing and camera control.
User data storage / acquisition: User data is stored in the specified directory of the internal memory 110b or the memory card 200, or the stored user data is read out and acquired.
Saving / acquiring facial feature information used for face recognition: saving facial feature information in the internal memory 110b or a prescribed directory of the memory card 200, or reading and obtaining the saved facial feature information. This face feature information is output by the face recognition engine 141.
・ Acquisition of personal identification parameter file: Read and obtain a specified file stored in the internal memory 110b or the memory card 200, and recognize the contents of the personal identification parameter as face recognition reference data written in the specified file. Delivered to the engine 141. Here, the “parameter file” is a file in which setting parameters such as a threshold value and a timeout value for face recognition are written.
-Camera control: Change the image size, the number of frames, the image mode, etc. used for starting and ending the camera and photographing the face.

  The face recognition engine 141 is controlled by the face recognition controller 140 via the face image processing API, and image processing for face recognition is performed based on the face image data photographed by the camera and the personal identification parameter. Execute.

  FIG. 6 is a flowchart showing an example of automatic function control by user's face recognition in the mobile phone 10 having the above configuration. In the standby state of the mobile phone 10, it is possible to accept an activation request for a “face recognition control mode” that performs automatic function control based on face recognition (S <b> 101 and S <b> 102). The activation request for the “face recognition control mode” is input, for example, when the owner of the mobile phone 10 operates the operation unit 115. Further, the activation request for the “face recognition control mode” may be notified to the mobile phone 10 by remote telephone or mail.

  Next, upon receiving a request for activation of the “face recognition control mode”, the mobile phone 10 starts the face recognition control mode (S103), and shifts from the normal standby state to the “standby + face recognition standby state”. (S104). When the operation of the mobile phone 10 is started by any user in this standby state, the start of the operation is detected. More specifically, in the folding type (clamshell type) mobile phone 10 shown in FIG. 3, the state change at the time of the opening operation that swings the housing upper part 10A and opens away from the housing lower part 10B is the start of the operation. It is detected by the detection unit 122. By this state change, it can be automatically recognized that the operation of the mobile phone 10 has started.

  Next, when an open operation of the mobile phone 10 is detected, face recognition processing is executed (S105, S106). This face recognition process starts when a camera activation command is sent from the face recognition controller 140 of the main control unit 110 to the camera unit 118. Upon receiving the camera activation command, the camera unit 118 faces the user who has started the operation of the mobile phone 10 based on predetermined shooting conditions (conditions such as image size, number of shooting frames, and shooting mode). The camera 109 starts photographing the user's face. When the face recognition controller 140 of the main control unit 110 receives the face image data captured by the camera unit 118, the face recognition controller 140 passes it to the face recognition engine 141. The face recognition engine 141 performs recognition processing based on user face recognition reference data registered in advance. Various methods can be adopted as the face recognition processing method. For example, a method may be used in which face recognition is performed using face image data of one frame selected from face image data (input image frames) of a plurality of frames. In this case, as the face image data of one frame, for example, face image data (input image frame) of an optimal frame is selected in the process of tracking the photographed user's face. Then, feature point extraction of the face is performed using the face image data (input image frame) of the selected frame.

  Next, the user who has started the operation of the mobile phone 10 is identified based on the result of the face recognition process, and the function control is executed based on the function control information set and stored in advance (S107 to S107). S112). Here, when it is determined that the user who starts the operation is the owner of the mobile phone 10, control is performed so that there is no function restriction and the normal operation can use all functions (S108). When it is determined that the user who starts the operation is a registered user registered in advance, only a predetermined function can be used based on the function control information for the registered user set in advance. (S109).

On the other hand, when the user who starts the operation is determined to be an unauthorized user other than the above-mentioned owner and registered user, or when it is determined that face recognition is not possible, a predetermined emergency operation is further executed. It is determined whether or not “mode” is set (S110). Here, if the “unauthorized use support mode” is not set, the display is turned off so that most functions for normal users cannot be used based on the function control information for preset unauthorized users. Or an alarm output set for an unauthorized user is executed (S111). Also, if the "unauthorized corresponding mode" that is set to execute a predetermined Emergency operation by the owner (S112). As the emergency operation, for example, an email attached with image data of a photographed user's face is sent to a predetermined email address.

Next, after the function control based on the user identification result by the face recognition is executed, it is determined whether or not to end the face recognition control mode (S113). If the face recognition control mode is not terminated, the “standby + face recognition standby state” described above is entered .
In addition, after the function control based on the user identification result by the first face recognition is executed, when the mobile phone 10 in the “standby + face recognition standby state” is continuously operated, a certain level of You may make it perform the function control based on the said face recognition process and the user's identification result by the face recognition for every time interval.

  The face recognition control mode ends when a series of operations by the user ends and the user closes the mobile phone 10 or turns off the power.

  FIG. 7 is an explanatory diagram showing an example of screen transition when setting the function control content for each user. First, when the operation unit 115 of the mobile phone 10 is operated and the face authentication mode setting is selected on the main menu screen, the face authentication mode setting screen 150 is displayed. When “1. registered user” is selected on the face authentication mode setting screen 150, a registered user setting screen 151 is displayed in which a plurality of registered users registered in advance as users to be identified by face authentication are listed. . For each of a plurality of registered users listed on the registered user setting screen 151, an access authority setting screen 152 for function control setting can be displayed and whether or not various functions can be used can be set. In the example of FIG. 7, it is possible to set whether or not to perform voice transmission, mail transmission, button (key) operation, transmission history / incoming history reference, mail folder reference, and the like.

  On the other hand, when “2. Unauthorized person” is selected on the face authentication mode setting screen 150, an unauthorized person mode setting screen 153 for setting a mode for unauthorized users other than the owner of the mobile phone 10 and the registered user is displayed. On the unauthorized person mode setting screen 153, either the “access restriction mode” or the unauthorized use handling mode for performing the emergency operation (hereinafter referred to as “Emergency mode”) can be selected as a setting target. On the access authority setting screen 154 set for the contents of the “access restriction mode”, it is possible to set a function restriction for an unauthorized user and a specific output operation in the case of an unauthorized user. For example, in the example of FIG. 7, ON / OFF can be set for alarm sound output, LED light emission / flashing, vibrator operation, liquid crystal display stop, dial lock, and all lock. Further, on the Emergency mode setting screen 155 for setting the contents of the “Emergency mode”, it is possible to set a specific operation that is urgently executed when an unauthorized user operates. In the example of FIG. 7, it is possible to make settings for ON / OFF of photographing a face photo, ON / OFF of mail transmission, mail transmission destination, and ON / OFF of liquid crystal display.

8 and 11 are sequence flowcharts showing an example of processing in the user registration mode for registering a user to be identified by face authentication. The sequence following FIG. 8 is displayed in FIG.
First, when the user to be registered operates the operation unit 115 and requests activation of the user registration mode in the standby state of the mobile phone 10, the main control unit 110 activates the user registration mode and performs face recognition. The parameter data necessary for Next, in the main control unit 110, the face recognition engine is initialized, and a face recognition application is called and started.

  Next, when a camera shooting start request is output from the main control unit 110 to the camera unit 118, the front camera 109 is activated and the face shooting guide image shown in FIG. 9A is displayed on the display of the display unit 114. The captured image display to be executed is executed. Then, the camera unit 118 shoots the face of the registered user under a predetermined shooting condition and passes it to the main control unit 110. The main control unit 110 stores the face image data received from the camera unit 118 in a predetermined storage area of the internal memory 110 b or the memory card 200.

  Next, the main control unit 110 executes face recognition processing by the recognition engine, notifies the recognition result shown in FIG. 10A or 10B, and displays the result of FIG. 9B on the display of the display unit 114. Confirmation display for displaying the face image of the recognition result shown in FIG.

  Next, when the user to be registered operates the operation unit 115 and requests registration processing, the main control unit 110 executes camera stop processing and the face recognition reference data obtained by the face authentication processing. Is stored in a predetermined storage area of the internal memory 110b or the memory card 200. Thereafter, the main control unit 110 stops the face recognition engine, ends the face recognition application, and returns to the normal standby state.

  Note that the face recognition control mode shown in FIG. 6 can be remotely activated using a telephone or mail. Whether or not this remote activation is permitted can be set on the face recognition activation setting screen shown in FIG.

  FIG. 13 and FIG. 14 are sequence flowcharts showing an example of processing when the face recognition control mode is remotely activated by telephone or mail. The sequence following FIG. 13 is displayed in FIG. By performing such remote activation by telephone or e-mail, when the mobile phone 10 is dropped or stolen, the face recognition control mode can be quickly activated to prevent unauthorized use of the mobile phone 10 in advance.

  The sequence flow shown in FIG. 13 shows remote activation processing by voice (telephone). First, in FIG. 13, when an incoming call is received in the standby state of the mobile phone 10, it is determined whether or not a voice (phone) call is received. Here, in the case of an incoming voice call, the main control unit 110 acquires an incoming call setting parameter and executes an incoming call operation. Then, remote authentication is requested from the caller telephone, information on a predetermined authentication result is received, and it is determined whether or not the information is correct. When the main control unit 110 receives the correct authentication result information, the main control unit 110 activates the face recognition control mode, acquires parameter data necessary for face recognition, and transmits an activation notification notifying the activation to the caller's telephone. To do. Then, the main control unit 110 shifts to the “standby + face recognition standby state” after executing the end-call processing.

  On the other hand, in the case of an incoming mail, as shown in FIG. 14, the main control unit 110 confirms whether the incoming mail is received. In the case of incoming mail, the incoming call setting parameter is acquired and data such as the received mail message is stored in the mailbox. When the remote mail setting is on and the received mail is a predetermined remote mail, a matching process is executed for the keyword in the mail. When the result of this collation is OK, the face recognition control mode is started, parameter data necessary for face recognition is acquired, a reply mail notifying that the face has been started is sent to the communication terminal of the sender, “Standby + face recognition standby state” is entered.

  FIG. 15 is a sequence flow diagram showing an example of processing when the Emergency mode is set. The display parameters are set so that the display unit 114 in the emergency mode is not displayed.

  In the “standby + face recognition standby state” in which the Emergency mode is set, when the operation start of the mobile phone 10 by the user is detected, the main control unit 110 initializes the recognition engine and performs face recognition. Call and start the application.

  Next, when a camera shooting start request is output from the main control unit 110 to the camera unit 118, the front camera 109 is activated. The camera unit 118 shoots the face of the registered user under a predetermined shooting condition and passes it to the main control unit 110.

  Next, the main control unit 110 executes face recognition processing until the face is successfully recognized by the recognition engine. If the face recognition process is successful, the main control unit 110 stores the image data of the unauthorized user's face acquired from the camera unit 118 in the internal memory 110b or a predetermined storage area of the memory card 200. In addition, the main control unit 110 controls to send an e-mail attached with image data of an unauthorized user's face to a predetermined communication destination (for example, to an e-mail address of an owner or a regular registered user).

  Next, the main control unit 110 executes a camera stop process and saves user authentication information, which is face recognition reference data obtained by the face authentication process, in a predetermined storage area of the internal memory 110b or the memory card 200. To do. Thereafter, the main control unit 110 stops the face recognition engine and ends the face recognition application.

As described above, according to the present embodiment, the face recognition reference data used for the user's face recognition is stored in the face recognition reference data storage unit 131 in advance. When the operation of the mobile phone 10 is detected, the user's face is photographed by the imaging means 130, and the image data of the photographed face and the face stored in the face recognition reference data storage means 131 are recorded. The recognition reference data is compared. Based on the comparison result, the operating user is automatically identified, so that it is not necessary to perform complicated operations such as input of a personal identification number for identification of the user. In addition, the user can be identified without the user being conscious of photographing the face with the imaging means 130 of the mobile phone 10. Based on the identification result and the function control information stored in advance, the usable functions can be controlled so that only the functions preset for the user can be used. Therefore, it is possible to identify a user and control the functions that can be used without the user operating the mobile phone 10 being conscious of it, without performing complicated operations such as inputting a personal identification number.
In addition, according to the present embodiment, when the use start operation by the user is detected, the user uses the mobile phone 10 by photographing the face, identifying the user, and controlling the function. When the function is started, predetermined function control for the user can be started promptly.
Further, according to the present embodiment, while the mobile phone 10 is being used, the face is photographed, the user is identified, and the function control is continuously performed at predetermined time intervals. Even when the user who is actually operating the mobile phone 10 is changed to another user during use of the telephone 10, the predetermined function control for the other user can be started immediately.
Further, according to the present embodiment, even when there are a plurality of users who operate and use the mobile phone 10, the functions can be controlled for each of the plurality of users.
Further, according to the present embodiment, functions can be controlled with different contents for each of a plurality of users.
In addition, according to the present embodiment, by not displaying the image of the photographed user's face, it is difficult for the user to understand that the face has been photographed. Identification can be performed more reliably.
Further, according to the present embodiment, at least one function of referring to, editing, moving, and copying data stored in the internal memory 110b of the main control unit 110 or the memory card 200 can be used only by a specific user. Can be controlled.
Further, according to the present embodiment, at least one of the communication partner information and communication history information stored in the internal memory 110b or the memory card 200 is referred to, edited, moved, and copied. The function can be controlled to be used only by a specific user.
In addition, according to the present embodiment, it is possible to control so that at least one function of outgoing call, incoming call, and incoming call notification display of a voice call can be used only by a specific user.
In addition, according to the present embodiment, it is possible to control so that at least one function of mail transmission / reception and incoming notification display can be used only by a specific user.
In addition, according to the present embodiment, the face authentication control mode is automatically set when a call or mail for setting the operation mode is received from a predetermined caller, so that the face authentication control mode can be set remotely. become.

In the above-described embodiment, the inward front camera 109 is used for face recognition. However, when the user operates the rear camera 108 facing the user's face, The facing rear camera 108 may be used for face recognition.
Further, in the above-described embodiment, it has been described that when the user's use start operation is detected, the photographing of the face, the identification of the user, and the function control are performed, but the operation is performed at an arbitrary timing. When the user's instruction operation is detected, the photographing of the face, the identification of the user, and the function control may be performed. For example, when the user who is actually operating the mobile phone 10 is changed to another user while the mobile phone 10 is being used, the user can use it by performing an instruction operation at an arbitrary timing. The user may be allowed to start predetermined function control.
Moreover, although the case of the mobile phone has been described in the above embodiment, the present invention can be similarly applied to other electronic devices other than the mobile phone by including a camera capable of photographing the face of the user who operates.

The block diagram which shows an example of the hardware constitutions of the mobile telephone which concerns on this embodiment. The block diagram of a camera part. (A), (b) and (c) are the rear view, front view, and side view which respectively show the external appearance of a mobile telephone. The functional block diagram which extracted and showed the main functions corresponding to the characteristic part of this invention in a mobile telephone. The block diagram which shows an example of the software interface in a mobile telephone. The flowchart which shows an example of the automatic function control by the user's face recognition in a mobile telephone. Explanatory drawing which shows an example of a screen transition when setting the function control content with respect to each user. The sequence flow figure which shows an example of the process (first half part) in the user registration mode which registers the user of the identification object by face authentication. (A) is explanatory drawing of the image for face photography guides. (B) is explanatory drawing of the face image of a recognition result. (A) And (b) is explanatory drawing of the display screen of the notification of a recognition result, respectively. The sequence flow figure showing an example of processing (second half part) in user registration mode which registers the user of the candidate for recognition by face authentication. Explanatory drawing of the screen of face recognition starting setting. The sequence flow figure which shows an example of the process (first half part) in the case of performing remote starting of the face recognition control mode by telephone or mail. The sequence flow figure showing an example of processing (second half part) in the case of performing remote starting of the face recognition control mode by telephone or mail. The sequence flow figure showing an example of processing when Emergency mode is set up.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Cellular phone 110 Main control part 110b Internal memory 118 Camera part 122 Operation start detection part 130 Imaging means 131 Face recognition reference | standard data storage means 132 Control information storage means 133 Operation detection means 134 Control means

Claims (21)

An emergency operation execution method for causing an electronic device to execute an emergency operation,
Activating a face recognition control mode for performing automatic control based on face recognition of the user of the electronic device based on the activation request;
Detecting the start of operation of the electronic device with the face recognition control mode activated;
A step of photographing a face of a user of the electronic device when the start of operation of the electronic device is detected, and performing face recognition processing by comparing image data of the face with pre-registered face recognition reference data When,
Based on the result of the face recognition process, when it is determined that the user who started the operation is an unauthorized user other than the person who has been registered in advance as the owner or the user of the electronic device, or face recognition is impossible A step of determining whether or not an unauthorized use support mode for executing an emergency operation at the time of unauthorized use is set,
A step of restricting the function of the electronic device when it is determined that the unauthorized use support mode is not set;
A step of executing an emergency operation at the time of unauthorized use when it is determined that the unauthorized use support mode is set;
An emergency operation executing method characterized by comprising: The emergency operation execution method according to claim 1,
The electronic device is configured to be openable and closable so as to be able to take a closed state when not in use and an open state when in use,
An emergency operation execution method, comprising: detecting an operation start of the electronic device by detecting a state change from the closed state when the electronic device is not used to an open state when the electronic device is used. In the emergency operation execution method according to claim 1 or 2,
The emergency operation execution method, wherein the activation request for the face recognition control mode is input by an operation of the electronic device. In the emergency operation execution method according to claim 1 or 2,
An emergency operation execution method, wherein the activation request for the face recognition control mode is received by the electronic device via a communication network. In the emergency operation execution method of Claim 4,
An emergency operation execution method characterized in that a call from a predetermined caller is received as an activation request for the face recognition control mode. In the emergency operation execution method of Claim 4,
An emergency operation execution method characterized by receiving an email containing a predetermined keyword as the activation request for the face recognition control mode. In the emergency operation execution method according to any one of claims 1 to 6,
The emergency operation execution method is characterized in that the emergency operation is an operation of storing the photographed user's face image data. In the emergency operation execution method according to any one of claims 1 to 6,
The emergency operation is an emergency operation execution method characterized in that the photographed user's face image data is attached to an e-mail and transmitted to a predetermined e-mail address. Imaging means provided at a site capable of photographing the face of the user of the electronic device;
Face recognition reference data storage means for storing face recognition reference data registered in advance;
Operation detecting means for detecting an operation of the electronic device;
Based on the activation request, a face recognition control mode for performing automatic control based on user's face recognition is activated, and when the operation start of the electronic apparatus is detected in a state where the face recognition control mode is activated, The user's face is photographed, the face image data and the face recognition reference data are compared, and face recognition processing is performed. Based on the result of the face recognition processing, the user who has started the operation If it is determined that the user is an unauthorized user other than those who are registered in advance as the owner or user of the device, or if it is determined that face recognition is not possible, an unauthorized use support mode that performs emergency actions during unauthorized use is provided. When it is determined whether or not the unauthorized use support mode is set, when it is determined that the unauthorized use support mode is not set, the function of the electronic device is limited and when it is determined that the unauthorized use support mode is set , Emergency at the time of unauthorized use And control means for controlling to perform the work,
An electronic device characterized by comprising: The electronic device according to claim 9.
It is configured to be openable and closable so that it can be closed when not in use and open when in use.
The operation detection means detects an operation start of the electronic device by detecting a state change of the electronic device from a closed state when not in use to an open state when used. The electronic device according to claim 9 or 10,
The activation request for the face recognition control mode is input by operating the electronic device. The electronic device according to claim 9 or 10,
A communication means for communicating via a communication network;
The electronic device, wherein the activation request for the face recognition control mode is received by the communication means via a communication network. The electronic device according to claim 12, wherein
An electronic device characterized in that a call from a predetermined caller is received as an activation request for the face recognition control mode. The electronic device according to claim 12, wherein
An electronic apparatus that receives a mail including a predetermined keyword as a request for starting the face recognition control mode. In the electronic device in any one of Claims 9 thru | or 14,
The electronic device according to claim 1, wherein the control unit executes an operation of storing image data of a user's face photographed by the imaging unit as the emergency operation. In the electronic device in any one of Claims 9 thru | or 14,
The electronic device is characterized in that, as the emergency operation, the control unit executes an operation of attaching image data of a user's face photographed by the imaging unit to a mail and transmitting it to a predetermined mail address. The electronic device according to any one of claims 12 to 14,
Image display means,
When receiving the activation request for the face recognition control mode, the control means performs control so that the image of the user's face used for the face recognition processing is not displayed on the image display means after being captured by the imaging means. An electronic device characterized by that. The electronic device according to any one of claims 9 to 17,
Data storage means for storing data stored by the user;
The electronic device is characterized in that the function to be restricted is at least one of a function of referring to, editing, moving and copying data stored in the data storage means and displaying a storage state of the data. The electronic device of claim 18,
The electronic device characterized in that the data restricted in reference, editing, movement, copying and display is at least one of communication partner information and communication history information. The electronic device according to any one of claims 9 to 17,
The electronic device according to claim 1, wherein the function to be restricted is at least one function of outgoing call, incoming call, and incoming call notification display. The electronic device according to any one of claims 9 to 17,
The electronic device is characterized in that the function to be restricted is at least one function of outgoing mail, incoming mail, and incoming call notification display .