KR20100000274A - Mobile terminal and operation control method thereof - Google Patents

Abstract

PURPOSE: A portable terminal and a conduct control method for locking and unlocking of a portable terminal are provided to use a facial image as an authentication image by setting a locking function of the portable terminal by using a face recognition technology. CONSTITUTION: A face image of a user recording by a camera is stored as the authentication image. The face of the user records by using the camera in an operation locking state(S400, S405). If the face image of the user recording is determined as the authentication image, the operation locked state is deregulated(S450). The screen in which the operation locked state is deregulated is marked.

Description

Mobile terminal and operation control method

The present invention relates to a mobile terminal and a method of controlling the motion thereof, and more particularly, a mobile terminal using a face image of a user as an authentication image by setting a lock function of the mobile terminal using facial recognition technology, and a method of controlling the motion thereof. It is about.

A portable terminal is a portable device that is portable and has one or more functions of performing voice and video calls, inputting and outputting information, and storing data.

In the portable terminal implemented in the form of such a multimedia device, new attempts have been applied in various ways in terms of hardware and software to implement complex functions. With the use of mobile terminals, not only text messaging and voice calls but also paid services such as video calls, web services, and content downloads have increased, and the necessity of protecting various paid communication events from being used by others has increased. .

In addition, since the mobile terminal has a close relationship with daily life, there is an increasing need for protecting personal privacy, that is, restricting access to information included in the mobile terminal of another person.

In general, the mobile terminal has restricted the use of another person's mobile terminal by using an authentication key capable of identifying a user such as a password or an ID. However, in locking and unlocking a mobile terminal or a specific menu, there is a problem in that a password or ID must be input every time, and it is difficult to prevent others from using the mobile terminal due to leakage of an authentication key.

Therefore, in the locking and unlocking of the mobile terminal, it is possible to generate an effect of the authentication key input by using the authentication image without repeatedly entering a password or ID, and only a user of the mobile terminal can use the authentication key. Required.

Accordingly, an object of the present invention is to provide a mobile terminal and an operation control method for storing a face image of a user as an authentication image and analyzing and comparing the current user's face image with the authentication image to lock and unlock the mobile terminal. have.

In accordance with another aspect of the present invention, there is provided a mobile terminal operation control method including a camera, which stores a face image of a user photographed by the camera as an authentication image for the mobile terminal. In the locked state of the portable terminal, photographing and displaying a face of the current user using the camera, and if the face image of the current user is determined as the authentication image, releases the locked state and the locked state. And displaying the released screen.

In addition, the present invention for achieving the above object, in a mobile terminal operation control method comprising a camera, a method for controlling the operation of a mobile terminal, locking one or more facial expression image of the user taken by the camera of the mobile terminal; Storing as an authentication image of a state and setting a specific menu for each of the stored facial expression images; in a locked state of the portable terminal, photographing and displaying a facial expression of a current user using the camera; If it is determined that the facial expression image of the user is one of the authentication images, releasing the lock state and displaying an execution screen of a specific menu corresponding to the facial expression image of the current user.

In addition, the present invention for achieving the above object, in a mobile terminal operation control method comprising a camera, a method for controlling the operation of a mobile terminal, the image of the face of the user photographed by the camera as an authentication image for the mobile terminal In the step of storing, in the execution screen of the specific menu, taking and displaying the face of the current user using the camera, if the face image of the current user is determined as the authentication image, lock the execution screen of the specific menu Changing to a state.

In addition, the present invention for achieving the above object, a portable terminal, a camera for photographing any one of the user's face and the current user's face, a display unit for displaying the current user's face taken by the camera, the camera Storing the face image of the user photographed by the mobile terminal as an authentication image for the mobile terminal, and when the face image of the current user photographed by the camera is determined as the authentication image, the lock state is released and the lock state is released. And a control unit which displays the displayed screen on the display unit.

In addition, the present invention for achieving the above object, a portable terminal, a camera for photographing any one of the user's facial expressions and the current facial expression of the user, a display unit for displaying the facial expression of the current user photographed by the camera Storing at least one facial expression image of the user photographed by the camera as an authentication image of the locked state of the mobile terminal, setting a specific menu for each of the stored facial expression images, and presently photographed by the camera. If it is determined that the facial expression image of the user is one of the authentication image, the control unit for releasing the lock state and displaying the execution screen of the specific menu corresponding to the facial expression image of the current user on the display unit.

In addition, the present invention for achieving the above object, a portable terminal, a camera for photographing any one of the user's face and the current user's face, a display unit for displaying the current user's face taken by the camera, the camera The image of the user's face photographed by the mobile terminal is stored as an authentication image for the mobile terminal, and when the face of the current user photographed by the camera is determined as the authentication image on the execution screen of the specific menu, And a controller for changing the execution screen to a locked state.

According to the present invention, an authentication image can be input without repetitive user input to generate the same effect as the authentication key input, and only a genuine user of the mobile terminal can use the mobile terminal using the authentication image.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The portable terminal described in the present specification includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like.

1 is a block diagram of a portable terminal according to an embodiment of the present invention. Referring to FIG. 1, a portable terminal according to an exemplary embodiment of the present invention will be described in terms of components according to functions.

Referring to FIG. 1, the mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, and an output unit 150. , A memory 160, an interface unit 170, a controller 180, and a power supply unit 190. Such components may be configured by combining two or more components into one component, or by dividing one or more components into two or more components as necessary when implemented in an actual application.

The wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 113, a wireless internet module 115, a short range communication module 117, and a GPS module 119.

The broadcast receiving module 111 receives at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like. The broadcast management server may mean a server for generating and transmitting at least one of a broadcast signal and broadcast related information, or a server receiving at least one of the pre-generated broadcast signal and broadcast related information and transmitting the same to a terminal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal. The broadcast related information may also be provided through a mobile communication network, and in this case, may be received by the mobile communication module 113. Broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcast signals using various broadcast systems, and in particular, digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), and media forward link only (MediaFLO). ), Digital broadcast signals may be received using digital broadcasting systems such as DVB-H (Digital Video Broadcast-Handheld) and ISDB-T (Integrated Services Digital Broadcast-Terrestrial). In addition, the broadcast receiving module 111 may be configured to be suitable for all broadcast systems providing broadcast signals as well as such digital broadcast systems. The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 113 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to voice call signal, video call signal, or text / multimedia message transmission and reception.

The wireless internet module 115 refers to a module for wireless internet access. The wireless internet module 115 may be embedded or external to the portable terminal 100. The short range communication module 117 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used. The GPS (Global Position System) module 119 receives position information from a plurality of GPS satellites.

The A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 123. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration aspect of the terminal.

The microphone 123 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, or the like, and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 113 and output in the call mode. The microphone 123 may use various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates key input data input by the user for controlling the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, a finger mouse, and the like. In particular, when the touch pad has a mutual layer structure with the display unit 151 described later, this may be referred to as a touch screen.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a position of the mobile terminal 100, presence or absence of a user contact, and controls the operation of the mobile terminal 100. Generate a sensing signal. For example, when the portable terminal 100 is in the form of a slide phone, whether the slide phone is opened or closed may be sensed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, a sound output module 153, an alarm unit 155, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal 100 is in a call mode, the mobile terminal 100 displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 may display a captured or received image, respectively, or simultaneously, and display a UI and a GUI.

Meanwhile, as described above, when the display unit 151 and the touch pad have a mutual layer structure and constitute a touch screen, the display unit 151 may be used as an input device in addition to the output device. If the display unit 151 is configured as a touch screen, the display unit 151 may include a touch screen panel, a touch screen panel controller, and the like. In this case, the touch screen panel is a transparent panel attached to the outside and may be connected to a bus inside the portable terminal. The touch screen panel keeps an eye on the contact result and sends corresponding signals to the touch screen panel controller when there is a touch input. The touch screen panel controller processes the signals and then transmits corresponding data to the controller 180 so that the controller 180 can determine whether there is a touch input and which area of the touch screen has been touched.

In addition, the display unit 151 may include a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display. It may include at least one of the display (3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, the external display unit (not shown) and the internal display unit (not shown) may be simultaneously provided in the portable terminal 100.

The sound output module 153 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 153 outputs a sound signal related to a function performed in the portable terminal 100, for example, a call signal reception sound and a message reception sound. The sound output module 153 may include a speaker, a buzzer, and the like.

The alarm unit 155 outputs a signal for notifying occurrence of an event of the portable terminal 100. Examples of events occurring in the mobile terminal 100 include call signal reception, message reception, key signal input, and the like. The alarm unit 155 outputs a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. When a call signal is received or a message is received, the alarm unit 155 may output a vibration to inform this. Alternatively, when a key signal is input, the alarm unit 155 may output a vibration in response to the key signal input. This vibration output allows the user to recognize the occurrence of the event. Of course, the signal for event occurrence notification may be output through the display unit 151 or the sound output module 153.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input or output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM The storage medium may include at least one type of storage medium. In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. Examples of external devices connected to the mobile terminal 100 include a wired / wireless headset, an external charger, a wired / wireless data port, a memory card, a memory card, a SIM / UIM card, and the like, and an audio I / O (Input). / Output) terminal, video I / O (Input / Output) terminal, earphone, and the like. The interface unit 170 may receive data from such an external device or receive power and transmit the data to each component in the mobile terminal 100, and may transmit data in the mobile terminal 100 to an external device. .

The controller 180 typically controls the operation of each unit to control the overall operation of the mobile terminal 100. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia playback module 181 for multimedia playback. The multimedia playback module 181 may be configured in hardware in the controller 180 or may be configured in software separately from the controller 180.

In addition, the controller 180 may include a face recognition module 182 that analyzes data about an image photographed by the camera 121 and recognizes a face's eyes, nose, and the like. The facial recognition module 182 detects a face region for a predetermined image, extracts a feature of a face, extracts brightness, curvature, and the like, and recognizes a face's eyes, nose mouth, and the like. Changes in the shape of the mouth can be recognized. The face recognition module 182 may transfer face recognition information about the image to the controller 180. The face recognition module 182 may be configured in hardware in the controller 180 or may be configured in software separately from the controller 180.

In addition, the power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The portable terminal 100 having such a configuration may be configured to be operable in a communication system capable of transmitting data through a frame or packet, including a wired / wireless communication system and a satellite based communication system. have.

The portable terminal related to the present invention has been described above in terms of components according to functions. Hereinafter, referring to FIGS. 2 and 3, the portable terminal related to the present invention will be further described in terms of components according to appearance. Hereinafter, for convenience of description, a bar type portable terminal having a front touch screen among various types of portable terminals such as a folder type, a bar type, a swing type, a slider type, and the like will be described as an example. However, the present invention is not limited to the bar type portable terminal, but can be applied to all types of portable terminals including the aforementioned type.

2 is a perspective view of a portable terminal according to an embodiment of the present invention viewed from the front. Referring to FIG. 2, the case forming the exterior of the portable terminal 100 is formed by the front case 100A-1 and the rear case 100A-2. Various electronic components are embedded in the space formed by the front case 100A-1 and the rear case 100A-2. At least one intermediate case may be additionally disposed between the front case 100A-1 and the rear case 100A-2. Such cases may be formed by injecting synthetic resin, or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the first sound output module 153a, the first camera 121a, and the first user input unit 130a may be disposed in the main body, specifically, the first front case 100A-1. In addition, a second user input unit 130b, a third user input unit 130, and a microphone 123 may be disposed on the side surface of the rear case 100A-2.

The display unit 151 includes a liquid crystal display (LCD), organic light emitting diodes (OLED), and the like, which visually express information. The display unit 151 may be configured such that the touch pad is superimposed in a layer structure so that the display unit 151 operates as a touch screen to input information by a user's touch.

The first sound output module 153a may be implemented in the form of a receiver or a speaker. The first camera 121a may be implemented to be suitable for capturing an image or a video of a user or the like. The microphone 123 may be implemented in a form suitable for receiving a user's voice, other sounds, and the like.

The first to third user input units 130a, 130b, and 130c may be collectively referred to as a user manipulating portion 130, and may be adopted in any manner as long as the user operates with a tactile feeling.

For example, the user input unit 130 may be implemented as a dome switch or a touch pad that may receive a command or information by a user's push or touch operation, or may be operated as a wheel or jog method or a joystick that rotates a key. It may also be implemented in a manner and the like. In the functional aspect, the first user input unit 130a is for inputting a command such as a start, end, call, etc., and the second user input unit 130b is for inputting a selection of an operation mode. In addition, the third user input unit 130c may operate as a hot-key for activating a special function in the portable terminal 100.

3 is a rear perspective view of the mobile terminal shown in FIG. 2. Referring to FIG. 3, a fourth user input unit 130d, a fifth user input unit 130e, and an interface unit 170 may be disposed on the side of the rear case 100A-2, and the rear case 100A-2 may be disposed. The second camera 121b may be additionally mounted on the rear side of the).

The second camera 121b may have a photographing direction substantially opposite to that of the first camera 121a, and may have different pixels from the first camera 121a. For example, in the case of a video call or the like, the first camera 121a has a low pixel so that the user's face is photographed and transmitted to the other party, and the second camera 121b photographs a general subject and does not transmit it immediately. In many cases, it is desirable to have a high pixel.

The mirror 125 and the flash 126 may be further disposed adjacent to the second camera 121b. The flash 126 illuminates the subject when the subject is photographed by the second camera 121b. The mirror 125 allows the user to see his / her own face or the like when the user wants to photograph (self-photograph) the second camera 121b.

A second sound output module (not shown) may be further disposed on the rear case 100A-2. The second sound output module may implement a stereo function together with the first sound output module 153a and may be used for a call in the speakerphone mode.

In addition, an antenna (not shown) for receiving a broadcast signal may be disposed on one side of the second rear case 100A-2 in addition to an antenna for a call. The antenna may be installed to be pulled out of the rear case 100A-2.

The interface unit 170 serves as a path for allowing the portable terminal 100 to exchange data with an external device. For example, the interface unit 170 may be at least one of a wired or wireless connection terminal for connecting to the earphone, a port for short-range communication, or a power supply terminal for supplying power to the mobile terminal 100. have. The interface unit 170 may be a card socket that accommodates an external card such as a subscriber identification module (SIM) or a user identity module (UIM) and a memory card for storing information.

The rear case 100A-2 is provided with a power supply unit 190 for supplying power to the portable terminal. The power supply unit 190 is, for example, a rechargeable battery, and may be detachably coupled to the rear case 100A-2 for charging.

In the above description, the second camera 121b and the like are disposed on the rear case 100A-2, but the present invention is not limited thereto. In addition, even if the second camera 12b is not separately provided, the first camera 121a may be rotatably formed to be able to shoot up to the photographing direction of the second camera 121b.

4 is a flowchart illustrating a method for controlling the operation of a portable terminal according to the first embodiment of the present invention.

As shown in FIG. 4, first, a menu for setting a facial recognition lock function for a mobile terminal is selected (S400). The facial recognition lock function means that an authentication key for identifying a user such as a password or an ID is replaced with a face image of the user. That is, by analyzing the face image of the current user to determine whether the current user is a genuine user, it can be used in place of the password or ID in the menu requesting the authentication key.

Next, the controller 180 determines whether the camera 121 is selected for face photographing (S405). As a result of determination in S405, when the camera 121 is not selected, the controller 180 calls a face image that has been previously photographed and stored in operation S410. A plurality of face images may be called and one face image may be selected by the user. As a result of the determination in S405, when the camera 121 is selected, the controller 180 photographs the face of the user using the camera 121 (S415).

Next, the controller 180 stores the face image called in step S410 or the face image photographed in step S415 as an authentication image for the mobile terminal 100 (S420). As the face image is stored as the authentication image, the face recognition lock function for the portable terminal 100 is set. Here, the facial recognition lock function for the mobile terminal 100 may be a lock setting of the mobile terminal 100 itself, such as a text message, a web server connection, a login for accessing a specific web server, a voice or video call, etc. In 100, all of the menus requesting an authentication key are included. For example, a user's face image may be used as an authentication image instead of an ID and password even when accessing a specific website or a messenger connection requiring an authentication key such as ID and password.

Next, the menu for setting the facial recognition lock function is terminated (S425).

After the facial recognition lock function is set for the portable terminal 100, when a menu included in the portable terminal 100 is selected (S430), the controller 180 determines whether the menu is locked (S435). If it is determined that the lock state is not S435, an authentication image for releasing the lock state is not required.

On the other hand, when the S435 determines that the menu is locked, the controller 180 activates the camera 121 (S440). Since the authentication image set in step S420 is required to release the lock state, it is for taking a face image of the current user using the camera 121. When the portable terminal 100 is in the locked state, the camera 121 may be automatically activated or activated only when selected by the user. The user may unlock the lock by entering a password. In order to prevent others from capturing an image of the user's face with the mobile terminal 100 and releasing the locked state, the first camera 121a will generally be activated.

Next, the controller 180 photographs the face of the current user using the activated camera 121 and displays the photographed face image on the display unit 151 (S445).

Next, the controller 180 determines whether the face image photographed in step S445 is the authentication image (S450). That is, when the face recognition module 182 analyzes the face image photographed in step S445, the controller 180 may have the same face image stored by the face recognition module 182 and the face image stored as the authentication image in step S420. Determine if it is a range. Even if the photographed face image is not the same image as the authentication image, it may be recognized as an authentication image if the shape and ratio of the face, eyes, nose, mouth, etc. analyzed by the facial recognition module 182 are in the same range. As a result of the determination in S450, when the photographed face image is not the authentication image, the controller 180 retakes the face of the current user using the selected camera 121 (S445).

On the other hand, if it is determined in S450 that the photographed face image is an authentication image, the controller 180 releases the lock state of the menu and displays an execution screen of the menu in which the lock state is released (S455). The execution screen of the menu may be a standby screen of the mobile terminal 100, and may be a web page screen, a text writing screen, a received text screen, or a screen logged into a specific web. For example, when the received text confirmation is in a locked state, the locked state may be released using the face image as the authentication image without having to enter a password. That is, the face image replaces the password or ID in the menu or server connection where the password or ID must be input in the portable terminal 100, and thus all can be applied. The controller 180 may display a message on the display unit 151 indicating that the locked state has been released.

5 is a flowchart illustrating a method of controlling a mobile terminal according to a second embodiment of the present invention.

As shown in FIG. 5, a menu for setting a face recognition lock function for the portable terminal 100 is first selected (S500). That is, by analyzing the face image of the current user to determine whether the current user is a genuine user, when the mobile terminal 100 is locked, the face image may be used instead of the password.

Next, the controller 180 determines whether the camera 121 is selected for face photographing (S505). As a result of the determination in S505, when the camera 121 is not selected, the controller 180 calls a facial expression image which is photographed and stored in advance (S510). As a result of the determination in S505, when the camera 121 is selected, the controller 180 photographs the facial expression of the user using the camera 121 (S515). Facial expressions may appear in various ways and may include smiley faces, expressionless faces, and closed eyes.

Next, the controller 180 stores the facial expression image called in step S510 or the facial expression image photographed in step S515 as an authentication image for a specific menu (S520). Here, the specific menu may be a lock setting of the mobile terminal 100 itself, and may be a text message, a web server connection, a login for accessing a specific web server, a voice or video call, or the like.

The controller 180 determines whether a menu for storing another facial expression image as an authentication image is selected (S525). If the S525 determines that a menu for storing another facial expression image as an authentication image is selected, steps S505 to S520 are repeated, and when a menu for storing another facial expression image as an authentication image is not selected, facial recognition The menu for setting the lock function is terminated (S530). That is, the plurality of facial expression images may be stored as an authentication image capable of releasing the locked state of the portable terminal 100, and after the locked state of the portable terminal 100 is released, a screen executed according to each facial expression image may be changed. Can be.

For example, when the smiley face and the closed face are stored as the authentication image, the locked state of the mobile terminal 100 is released even if one of the smiley face and the closed face is input. However, when the specific menu for the smiling face is the lock setting of the mobile terminal 100 itself, when the lock state is released by the smiling face, the controller 180 displays the standby screen on the display unit 180 and closes the eyes. When the specific menu for the face is a text writing menu, when the locked state is released by the closed face, the controller 180 displays the text writing screen on the display 180. Therefore, not only the lock state of the portable terminal 100 itself can be released by the facial expression image, but also the lock can be immediately entered into a desired specific menu.

Next, after the face recognition lock function is set, when a user input is generated to the portable terminal 100, the controller 180 determines whether the portable terminal 100 is locked (S535). If it is determined that the lock state is not S535, an authentication image for releasing the lock state is not required.

On the other hand, when the S535 determines that the mobile terminal 100 is in the locked state, the controller 180 activates the camera 121 (S540). Since the authentication image set in step S520 is required to release the lock state, this is to capture a face image of the current user using the camera 121. When the portable terminal 100 is in the locked state, the camera 121 may be automatically activated or activated only when selected by the user. The user may release the lock state of the mobile terminal 100 by entering a password. In order to prevent others from capturing an image of the user's face with the mobile terminal 100 and releasing the locked state, the first camera 121a will generally be activated.

Next, the controller 180 photographs the facial expression of the current user using the activated camera 121 and displays the photographed facial expression image on the display unit 151 (S545).

Next, the controller 180 determines whether the facial expression image photographed in step S545 is one of the authentication images (S550). That is, the facial recognition module 182 analyzes the facial expression image photographed in step S545 to confirm the facial expression, and the controller 180 authenticates the facial expression image confirmed by the facial recognition module 182 and in step S520. It is determined whether the facial expression image stored as the image is the same range. Even if the photographed facial expression image is not the same image as the authentication image, if the shape and ratio of the face, eyes, nose, mouth, etc. analyzed by the facial recognition module 182 are within the same range, it may be recognized as the authentication image. The facial recognition module 182 may determine facial expressions by analyzing muscles and brightness around the mouth, eyes, and the like, and confirming a smiling mouth shape or a blinking eye shape. As a result of the determination in S550, when the photographed facial expression image is not the authentication image, the controller 180 re-photographs the facial expression of the current user using the selected camera 121 (S545).

On the other hand, if it is determined in S550 that the photographed facial expression image is an authentication image, the control unit 180 releases the lock state of the portable terminal 100 and displays an execution screen of a specific menu according to the photographed facial expression image. (S555). The execution screen of the specific menu may be a standby screen of the mobile terminal 100, and may be a web page screen, a text writing screen, a received text screen, or a screen logged into a specific web. For example, the user can release the lock state of the mobile terminal 100 by using the smiley facial expression image as an authentication image without having to enter a password, and simultaneously release the lock state of the mobile terminal 100 according to the smiley facial expression. The received text screen may be immediately displayed on the display unit 151.

6 is a flowchart illustrating a method of controlling a mobile terminal according to a third embodiment of the present invention.

As shown in FIG. 6, a menu for setting a face recognition lock function for a mobile terminal is first selected (S600). That is, by analyzing the face image of the current user, and determining whether the current user is a genuine user, it is possible to change the specific menu or the upper menu including the specific menu to the locked state.

Next, the controller 180 determines whether the camera 121 is selected for face photographing (S605). If the camera 121 is not selected as a result of the determination in S605, the controller 180 calls a face image that has been previously photographed and stored in operation S610. As a result of the determination in S605, when the camera 121 is selected, the controller 180 photographs the face of the user using the camera 121 (S615).

Next, the controller 180 stores the face image called in step S610 or the face image photographed in step S615 as an authentication image for the mobile terminal 100 (S620). Next, the menu for setting the facial recognition lock function is terminated (S625). For example, if you want to lock an important document or image, you can lock it with a face image without having to set security separately.

After the facial recognition lock function is set for the portable terminal, if a specific menu is selected, the controller 180 displays an execution screen of the specific menu on the display unit 151 (S630). Here, the specific menu may be a lock setting of the mobile terminal 100 itself. The mobile terminal 100 may include a text message, a web server connection, a login for accessing a specific web server, a voice or video call, each document, an image, a video, and the like. Includes all the menus that can be locked using the authentication image. In addition, the execution screen of the specific menu may be a standby screen of the mobile terminal 100, and may be a sent text screen, a received text screen, a document, a video, or an image.

Next, the controller 180 determines whether the camera 121 is selected (S635). When the camera 121 is not selected as the result of the determination in S635, since an authentication image for changing the specific menu to the locked state is not required, the controller 180 displays the execution screen of the specific menu on the display unit 151. (S630).

On the other hand, when the camera 121 is selected as a result of the determination S635, the controller 180 determines whether the specific menu currently being executed is the camera photographing menu (S640). As a result of the determination in S640, since the camera 121 is already operating in the camera shooting menu, the controller 180 displays the execution screen of the specific menu, that is, the execution screen of the camera shooting menu, on the display unit 151. (S630). In addition, in order to distinguish between the camera shooting menu selection and the camera selection for the face recognition, the camera 121 is different from each other according to a long-key or short-key input to the user input unit 130. Can be set to work.

On the other hand, if it is determined in S640 that the camera photographing menu is not, the camera 121 photographs the current user's face and displays the photographed face image on the display unit 151 (S645).

Next, the controller 180 determines whether the face image photographed in step S645 is the authentication image (S650). That is, when the face recognition module 182 analyzes the face image photographed at step S645, the controller 180 is identical to the face image analyzed by the face recognition module 182 and the face image stored as the authentication image at step S620. Determine if it is a range. Even if the photographed face image is not the same image as the authentication image, it may be recognized as an authentication image if the shape and ratio of the face, eyes, nose, mouth, etc. analyzed by the facial recognition module 182 are in the same range. As a result of the determination in S650, when the photographed face image is not the authentication image, the controller 180 retakes the face of the current user by using the selected camera 121 (S645).

On the other hand, if it is determined in S650 that the photographed face image is an authentication image, the controller 180 changes the execution screen of the specific menu to the locked state (S655). Depending on the setting, the upper menu including the specific menu as well as the execution screen of the specific menu may be changed to a locked state. When is input, the lock may be set only for the picture displayed on the display unit 151, or the lock may be set for the entire picture storage menu including the picture. One specific menu may be changed to a locked state by using one authentication image, and each menu included in the portable terminal may be changed to a locked state by using one authentication image.

7 through 9 are diagrams for describing a method of controlling a mobile terminal according to a first embodiment of the present invention as shown on a screen displayed on a display unit.

As shown in FIG. 7A, a screen 70 including a face recognition lock menu 71 and a number lock menu 72 is displayed on the display unit 151. When the facial recognition lock menu 71 is selected, as shown in FIG. 7B, the first camera 121a is activated to capture the user's face, and displays the captured user's face image. 73) is displayed. The photographed face image of the user is stored as an authentication image for a specific menu. FIG. 8 illustrates a case in which the captured face image of the user is an authentication image of a locked state of the portable terminal, and FIG. 9 illustrates a case in which the face image of the user is an authentication image of a locked state of an inbox.

When the mobile terminal itself is locked, when a user inputs a touch signal or selects an arbitrary key to use the mobile terminal, the mobile terminal itself is locked as shown in FIG. The screen indicating the status is displayed on the display unit 151. When the user selects the third user input unit 130c, as shown in FIG. 8B, the first camera 121a is activated, photographs the face of the current user, and captures the captured face image of the current user. Screen 74 is displayed. Here, the third user input unit 130c is a hot key for activating the camera and may be set to one of a long key and a short key. As a result of the analysis of the facial recognition module 182, if it is determined that the captured image of the current user's face is the same range as the authentication image for the locked state of the mobile terminal itself stored in advance, as shown in (c) of FIG. The locked state of the terminal itself is released, and the screen 75 on which the standby screen is displayed is displayed on the display unit 151.

When the inbox is locked, when the current user inputs a touch signal to check the received message or selects an arbitrary key, as shown in (a) of FIG. 9, the inbox may indicate that the inbox is locked. The screen is displayed on the display unit 151. When the user selects the third user input unit 130c, as shown in FIG. 9B, the first camera 121a is activated, photographs the face of the current user, and captures the captured face image of the current user. Screen 76 is displayed. As a result of the analysis of the facial recognition module 182, if it is determined that the captured image of the current user's face is the same range as the authentication image for the locked state of the pre-stored inbox, as shown in (c) of FIG. The lock state of the box is released, and a screen 77 on which the received message, voice message, and draft box included in the inbox is displayed on the display unit 151.

10 to 12 are diagrams for describing a mobile terminal operation control method according to a second embodiment of the present invention together with a screen displayed on a display unit.

As shown in FIG. 10A, a screen 80 including a face recognition lock function menu 81 and a number lock function menu 82 is displayed on the display unit 151. When the face recognition lock function menu 81 is selected, as shown in FIG. 10B, the first camera 121a is activated to photograph the smiling face expression of the user, and the smiling face expression image of the user. A screen 83 is displayed. The photographed smiling facial expression image of the user is stored as an authentication image for the mobile terminal locked state. However, if the smiley facial expression image is used as the authentication image, after unlocking the portable terminal, the standby screen is displayed. If another facial expression image is to be set as an authentication image for the mobile terminal lock state, as shown in FIG. 10C, the first camera 121a is activated to capture the facial expression of the user's eyes. The screen 84 on which the captured user's closed facial expression image is displayed is displayed. The photographed user's closed facial expression image is stored as an authentication image of the locked state of the portable terminal. When the facial expression image is used as the authentication image, the eye is set to display a text writing screen for writing a text message after unlocking the portable terminal. That is, as the authentication image for the mobile terminal lock state, both the smiley face and the eyes are set.

When the mobile terminal itself is locked, when a user inputs a touch signal or selects an arbitrary key to use the mobile terminal, the mobile terminal itself is locked as shown in FIG. The screen indicating the status is displayed on the display unit 151. When the user selects the third user input unit 130c, as shown in FIG. 11B, the first camera 121a is activated, photographs the facial expression of the current user, and photographs the current user's face. The screen 85 on which the facial expression image is displayed is displayed. As a result of the analysis of the facial recognition module 182, if it is determined that the photographed facial expression image of the current user is the same range as the smiley facial expression image among the authentication images for the pre-stored locked state of the mobile terminal, it is shown in FIG. As described above, the locked state of the portable terminal is released, and the screen 86 displaying the screen according to the smiley facial expression image, that is, the standby screen is displayed on the display unit 151.

On the other hand, when the user selects the third user input unit 130c in FIG. 11A, as shown in FIG. 12A, the screen 87 on which the facial expression image of the current user photographed is displayed is displayed. do. As a result of the analysis of the facial recognition module 182, if it is determined that the captured facial expression image of the current user is the same range as the closed facial expression image among the authentication images for the pre-stored mobile terminal lock state, it is shown in FIG. As described above, the locked state of the portable terminal is released, and the screen 88 displaying a screen according to the closed facial expression image, that is, a text writing screen for writing a text message is displayed on the display unit 151. Therefore, not only the lock state of the portable terminal can be released by the facial expression image, but also the user can directly enter a desired specific menu at the same time as the lock release.

In addition, the portable terminal and its operation control method according to the present invention are not limited to the configuration and method of the embodiments described as described above, the embodiments are all of the embodiments so that various modifications can be made Or some may be selectively combined.

Meanwhile, the present invention can be embodied as processor readable codes on a processor readable recording medium provided in a mobile terminal such as a mobile station modem (MSM). The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and also include a carrier wave such as transmission through the Internet. The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, while the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the above-described specific embodiments, and the present invention belongs without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

1 is a block diagram of a portable terminal according to an embodiment of the present invention;

2 is a front perspective view of a portable terminal according to an embodiment of the present invention;

3 is a rear perspective view of a portable terminal according to an embodiment of the present invention;

4 is a flowchart illustrating a method for controlling operation of a portable terminal according to a first embodiment of the present invention;

5 is a flowchart illustrating a method for controlling operation of a portable terminal according to a second embodiment of the present invention;

6 is a flowchart illustrating a method of controlling a mobile terminal according to a third embodiment of the present invention;

7 to 9 are views for explaining a mobile terminal operation control method according to a first embodiment of the present invention together with a screen displayed on a display unit;

10 to 12 illustrate a method of controlling a mobile terminal according to a second embodiment of the present invention together with a screen displayed on a display unit.

Claims (21)

In the mobile terminal operation control method comprising a camera, Storing a face image of the user photographed by the camera as an authentication image; Capturing and displaying a current user's face using the camera in a locked state; And If it is determined that the photographed face image of the current user is the authentication image, releasing the motion lock state and displaying a screen on which the motion lock state is released. The method of claim 1, The screen in which the locked state is released includes a standby screen, a web page screen, and a message transmission and reception screen. The method of claim 1, And displaying a message informing of the release of the locked state when the operation locked state is released. In the mobile terminal operation control method comprising a camera, Storing at least one facial expression image of the user photographed by the camera as an authentication image and setting a specific menu corresponding to each of the stored facial expression images; Photographing and displaying a facial expression of a current user using the camera in an operation lock state; And If it is determined that the photographed facial expression image of the current user is one of the authentication images, releasing the operation lock state and displaying an execution screen of a specific menu corresponding to the photographed facial expression image of the current user; Mobile terminal operation control method. The method of claim 4, wherein The execution screen of the specific menu may include a standby screen, a web page screen, and a message transmission screen. The method of claim 4, wherein The at least one facial expression image includes a smiling face image, a closed face image, and an expressionless face image. In the mobile terminal operation control method comprising a camera, Storing a face image of the user photographed by the camera as an authentication image; Photographing and displaying a face of a current user using the camera on an execution screen of a specific menu; And  And changing the execution screen of the specific menu to a locked state if it is determined that the photographed face image of the current user is the authentication image. The method of claim 7, wherein The execution screen of the specific menu is a mobile terminal operation control method, characterized in that any one of a document, an image, and a video. The method of claim 7, wherein And when the photographed face image of the current user is determined as the authentication image, changing the upper menu including the execution screen of the specific menu to a locked state. camera; A display unit which displays an image captured by the camera; And A control unit for storing a face image of the user photographed by the camera as an authentication image, and releasing the operation lock state when the face image of the current user photographed by the camera is determined as the authentication image in an operation lock state; A mobile terminal comprising; The method of claim 10, The controller may display the screen on which the lock state is released, on the display unit. The method of claim 11, The screen of which the operation lock state is released includes a standby screen, a web page screen, and a message transmission and reception screen. The method of claim 10, The controller may include a face recognition module configured to analyze the face image of the current user and determine whether the face image matches the authentication image. camera; A display unit which displays an image captured by the camera; And Store one or more facial expression images of the user photographed by the camera as an authentication image, set a specific menu corresponding to each of the stored facial expression images, and, in the operation lock state, the facial expression of the current user photographed by the camera And when the image is determined to be one of the authentication images, release the operation lock state and control the controller. The method of claim 14, The controller may display an execution screen of a specific menu corresponding to the facial expression image of the current user on the display unit. The method of claim 15, The execution screen of the specific menu may include a standby screen, a web page screen, and a message transmission screen. The method of claim 14, The controller may include: a face recognition module configured to analyze the facial expression image of the current user and determine whether the image matches the one of the stored facial expression images. camera; A display unit displaying an execution screen of a specific menu; And When the face image of the user photographed by the camera is stored as an authentication image and the face image of the current user photographed by the camera is determined as the authentication image while the execution screen of the specific menu is displayed, the specific image is specified. The control unit for changing the execution screen of the menu to the locked state. The method of claim 18, The controller may include a face recognition module configured to analyze the face image of the current user and determine whether the face image matches the authentication image. The method of claim 18, The execution screen of the specific menu is a mobile terminal, characterized in that any one of a document, an image, and a video. The method of claim 18, And when the face of the current user is determined as the authentication image, the controller changes the upper menu including the execution screen of the specific menu to a locked state.

Images