KR101960307B1 - Mobile terminal and method for controlling the same - Google Patents

Abstract

A mobile terminal according to the present invention includes: a camera module for capturing a user image for a video call; A wireless communication unit for transmitting and receiving video and audio data for the video call; A display unit for displaying the photographed user image and the received counterpart image; A memory for storing information on the user and the other party; And a controller for searching whether the face of the third party other than the user or the other party is recognized in the user image or the counterpart video and controlling the user to enter the privacy protection mode when the face of the third party is recognized.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile terminal that can protect a privacy of a user in a video call.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

When a video call is currently made to a mobile terminal, the video call screen may be exposed to a third party other than the user and the other party. When a video call screen is exposed to a third party, the information of the user or the other party may be unintentionally transmitted to the third party, which may invade the privacy of the user or the other party.

The present invention proposes a method for protecting a privacy of a user or an opposite party when a video call is made, and a mobile terminal for entering a privacy mode when a user or a third party other than the other party is recognized, And a control method thereof.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to an aspect of the present invention, there is provided a mobile terminal including a camera module for capturing a user image for a video call; A wireless communication unit for transmitting and receiving video and audio data for the video call; A display unit for displaying the photographed user image and the received counterpart image; A memory for storing information on the user and the other party; And a controller for searching whether the face of the third party other than the user or the other party is recognized in the user image or the counterpart video and controlling the user to enter the privacy protection mode when the face of the third party is recognized.

According to another aspect of the present invention, there is provided a method of controlling a video call of a mobile terminal, the method comprising: displaying a captured user image and a received opposite party image during a video call; Searching whether a face of a third party other than a user or a counterpart is recognized in the user image or the counterpart video; And entering a privacy mode when a face of a third person is recognized in the user image or the counterpart video.

The mobile terminal and its control method according to the present invention have the effect of protecting the privacy of the user or the other party by allowing the user or the third party other than the other party to enter the privacy mode when making a video call.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a front perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
4 is a flowchart illustrating a process of entering a privacy mode when a face of a third person is recognized in a captured user image or a received counterpart image in a mobile terminal according to an exemplary embodiment of the present invention.
FIGS. 5A through 9C illustrate an embodiment of entering a privacy mode when a face of a third party is recognized in a user image or a counterpart image in a mobile terminal according to an exemplary embodiment of the present invention.
10A to 10G are views showing a privacy mode according to an embodiment of the present invention.
11A to 11D are views showing a display unit on which a substitute image is displayed.
12A to 12H are views showing an embodiment of a method of editing a substitute image of a user image.
13 is a diagram illustrating a process of entering a privacy mode according to a specific operation of a user in a mobile terminal according to an exemplary embodiment of the present invention.
14A and 14B are diagrams illustrating a video call conversion according to a change in illumination in a mobile terminal according to an embodiment.
FIG. 15 is a view illustrating a display unit when performing a multi-party video call.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), navigation and the like. However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only (Mobile Broadcasting Business Management System), ISDB-T (Integrated Services Digital Broadcasting (ISDB-T)), Digital Multimedia Broadcasting (MBMS) Digital Broadcast-Terrestrial) or the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may be coupled to a base station, an external terminal, or a server on a mobile communication network, such as, but not limited to, Gobal System for Mobile communications (GSM), Code Division Multiple Access (CDMA), Wideband CDMA Transmits and receives wireless signals with one. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies include WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), GSM, CDMA, WCDMA, LTE Evolution (but not limited to) may be used.

The wireless Internet module 113, which performs wireless Internet access through the mobile communication network, is connected to the mobile communication module 110 through the mobile communication network, for example, from the viewpoint that the wireless Internet access by Wibro, HSDPA, GSM, CDMA, WCDMA, LTE, (112). ≪ / RTI >

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a button 136 located on the front, rear, or side of the mobile terminal 100, and a touch sensor (static / static) 137. Although not shown, ), A dome switch, a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141. (The proximity sensor will be described later).

The output unit 150 is for generating output related to the visual, auditory or tactile sense and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154 .

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

In the case where the display unit 151 and the touch sensor 137 have a mutual layer structure or are integrally formed (hereinafter referred to as a 'touch screen'), the display unit 151 may be used as an input device . The touch sensor may be stacked on the display unit 151 to form a layer structure when the touch sensor has a form of a touch film, a touch sheet, a touch pad, or the like. ≪ / RTI >

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in the inner area of the mobile terminal 100 or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch" The act of actually contacting the pointer on the touch screen may be referred to as "contact touch. &Quot; The location where the pointer is proximately touched on the touch screen may refer to a position where the pointer corresponds vertically to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. In this case, the display unit 151 and the audio output module 152 may be a type of the alarm unit 153. The display unit 151 and the audio output module 152 may be connected to the display unit 151 or the audio output module 152, .

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory unit 160 may store a program for processing and controlling the control unit 180 and temporarily store the input / output data (e.g., telephone directory, message, audio, For example. The memory unit 160 may store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia).

In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted It can be a passage to be transmitted to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 may include, for example, a battery, a connection port, a power supply control unit, and a charge monitoring unit.

The battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like. The connection port may be configured as an example of an interface 170 through which an external charger for supplying power for charging the battery is electrically connected.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays , Microprocessors, microprocessors, microprocessors, and other electronic units for carrying out other functions. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

FIG. 2 is a perspective view of a mobile terminal or a portable terminal according to the present invention, as viewed from the front.

The disclosed mobile terminal 100 includes a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body of the mobile terminal 100 includes a casing 101, 102, In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102.

A part such as a USIM card and a memory card, which is required to be detached, such as an auxiliary storage medium 165, may include an opening in the case so that the part can be inserted into the case from the outside. Slots may be formed on the side of the case so that the auxiliary storage medium 165 may be inserted from the side of the mobile terminal 100 and card slots 166 and 167 may be formed on the surface of the rear case 102 It is possible.

The cases 101, 102, and 103 may be formed by injection molding synthetic resin or may be formed of a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the sound output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, and the interface 170 may be disposed in the terminal cases 101 and 102.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 152 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include a plurality of operation units 131, 132, and 133. The operation units 131, 132, and 133 may be collectively referred to as a manipulating portion.

The contents inputted by the first or second operation unit 131 or 132 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, and the like, and the second operation unit 132 inputs a command such as adjustment of the sound output from the sound output unit 152 And the third operation unit 133 can receive commands such as activation / deactivation of the touch recognition mode of the display unit 151 and the like.

In addition to the display unit 151, the operation units 131, 132, and 133 may be provided with touch sensors, It is also possible to receive a command of the user by touch alone.

3 is a rear perspective view of the mobile terminal shown in FIG.

Referring to FIG. 3, a camera 121 'may be further mounted on the rear surface of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2), and may be a camera having the same or different pixels as the camera 121.

For example, the camera 121 may have a low pixel so that the face of the user can be photographed and transmitted to the other party in case of a video call or the like, and the camera 121 ' It is preferable to have a large number of pixels. The cameras 121 and 121 'may be installed in the terminal body such that they can be rotated or pop-up.

A flash 123 and a mirror 124 may be additionally disposed adjacent to the camera 121 '. The flash 123 illuminates the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to illuminate the user's own face or the like when the user intends to shoot himself / herself (self-photographing) using the camera 121 '.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output 152 'on the rear side of the body may implement the stereo function together with the sound output 152 (see FIG. 2) of the front side of the body, and may be used for the implementation of the speakerphone mode during a call.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

For convenience of explanation, it is assumed that the mobile terminal 100 mentioned below may include at least one of the components shown in FIG.

Hereinafter, with reference to FIGS. 4 to 11D, the process of entering the privacy mode based on the photographed user image or the face pattern recognized in the received counterpart image will be described in detail.

4 is a flowchart illustrating a process of entering a privacy mode when a face of a third person is recognized in a captured user image or a received counterpart image in a mobile terminal according to an exemplary embodiment of the present invention.

 The mobile terminal 100 related to an embodiment of the present invention can perform a video call. The control unit 180 displays the user image 201 captured by the camera module 121 and the counterpart image 202 received by the wireless communication unit 110 on the display unit 201. [ (Step S410).

In this case, the control unit 180 can control the screen arrangement in consideration of the size of the display unit 151. [ For example, the display unit 151 may be divided into two, and one may display the counterpart video 202 and the other one may display the user image 201. Also, the counterpart video 202 can be displayed on the display unit 151 largely and the user video 201 can be displayed on the PIP (Picture In Picture) system. However, the present invention is not limited thereto. The mobile terminal 100 displays only the received counterpart video 202 without displaying the user's own video 201 due to the selection of the user or the size restriction of the display unit 151, Can be performed.

The control unit 180 may recognize a pattern of a face recognized in the captured user image 201 or the received counterpart image 202 in operation S420.

According to one embodiment, the controller 180 determines whether the face of the third party other than the user or the other party is recognized on the detected face (S430), and the face of the third party is displayed in the user image 201 or the counterpart image 202 If it is recognized, the user can enter the privacy mode (S440). In the privacy protection mode, in order to protect the privacy of the user or the other party, a video conversation is performed by modifying some or all of the user image 201 or the counterpart video 202 during a video call, And terminating the video call and performing a voice call or the like.

If the face of the third party is not recognized in the user image 201 or the counterpart image 202, a general video call can be performed at step S450. Herein, the term "general video call" refers to a general video call in which a user and a user watch a user video 201 captured by the camera module 121 and a counterpart video 202 received by the wireless communication unit 110, To perform the call.

Hereinafter, a method of entering a privacy mode when a face of a third party is recognized in the user image 201 or the counterpart image 202 in the mobile terminal 100 will be described in more detail.

FIGS. 5A through 9C illustrate an embodiment of entering a privacy mode when a face of a third party is recognized in a user image or a counterpart image in a mobile terminal according to an exemplary embodiment of the present invention.

5A shows a screen in which a user I of the mobile terminal 100 performs a video call with the other party U. FIG. The remote image 202 received by the wireless communication unit 110 is displayed on the main screen of the display unit 151 and the image captured by the camera module 121 is displayed on the lower right side of the remote image 202. [ The user's own image 201 can be displayed.

The control unit 180 may generate a control signal to enter the privacy mode when the face image recognized in the user image 201 or the counterpart image 202 satisfies a predetermined condition. The mobile terminal 100 may include a memory 160 in which information about the user I and the other party U is stored. The control unit 180 searches the memory 160 to recognize the user I and the other party U from the user image 201 and the counterpart video 202, The three characters (X) can be identified.

5B shows a screen in which the face of the third person X other than the user I is recognized in the user image 201 of the mobile terminal 100. [

The comparison between the information stored in the memory 160 and the faces I and X recognized in the user image 201 is made by comparing the face image and the recognized face in the user image 201, The two faces can be regarded as the same.

The control unit 180 then searches whether the face recognized in the user image 201 is a face of a third person X other than the user I and if the face of the third person X is recognized, The terminal 100 may generate a control signal to enter the privacy mode. When the face of the third person X other than the user I is recognized in the user image 201, the video call of the user I is exposed to the third person X, U) can be known. Accordingly, when two or more faces are recognized in the user image 201, the control unit 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy mode in order to protect the privacy of the user I.

FIG. 5B shows a case where one face of a third party is recognized, but the present invention can also be applied to a case where two or more faces of a third party are recognized.

The face of the third person X in the user image 201 or the other party image 202 may be present together with the user I or the other party U but may be a face of the third person X in the user image 201 or the other party image 202, Only the face of the third person X other than the user I or the other party U may be recognized. If the third party X is recognized in the video call irrespective of whether the face of the user I or the other party U is present in the user image 201 or the other party image 202, In order to protect the privacy of the other party (U), it is necessary to enter the privacy mode. The controller 180 searches the faces in the user image 201 and the counterpart image 202 to determine whether any of the detected faces matches the information of the user I and the other user U stored in the memory 160 A control signal may be generated to enter the privacy mode.

5C shows a screen in which the face of a third party X other than the other party U is recognized in the partner video 202 of the mobile terminal 100. [ As shown in the figure, when the third party's face is recognized in the displayed counterpart image 201, the controller 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy mode. At this time, even if two or more third persons' faces are recognized, a control signal can be generated to enter the privacy protection mode. If the face of the third party X other than the other party U is recognized in the other party video 202, the video call of the other party U is exposed to the third party X, There is a concern that information about the user I performing may be known. Therefore, when two or more faces are recognized in the counterpart video 202, the controller 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy mode in order to protect the privacy of the user I.

In the above description, the user enters the privacy mode when the face of the third person X other than the user I or the user U is recognized in the user image 201 or the partner image 202.

Hereinafter, the entry into the privacy mode according to the state of the face of the third person recognized in the user image 201 or the counterpart image 202 will be described in detail with reference to FIGS. 6A to 6D.

6A shows a screen in which a user I of the mobile terminal 100 is performing a video call with the other party U. FIG. The face of the third person Y other than the user I is recognized on the user image 201 displayed on the main screen of the display unit 151. [ However, if the recognized face of the third party is not the front face, a general video call can be performed without entering the privacy protection mode.

At this time, the controller 180 can determine whether the recognized face is a front face by examining conditions such as eyes, nose, mouth, etc. all drawn on the face recognized in the user image 201. For example, when two eyes are recognized on the face screen and the nose is recognized near the center of the face, it can be determined that the face is the front face. In addition, in the case of a face wearing sunglasses or the like, it can be determined that the face is a front face when both of the lenses of the sunglasses are recognized.

If the face of the third person Y is the side face, the controller 180 regards the face of the user I as not staring at the display unit 151 and can decide whether to enter the privacy mode. 6A, a third party's face Y is recognized in the user image 201. However, since the face of the third party Y is a side face, it is regarded as not observing the display unit 151, It is possible not to generate the control signal to enter the mode.

6B shows a screen in which the front face of the third person X is recognized in the user image 201 of the mobile terminal 100. [ When the front face of the third person X is recognized in the displayed user image 201, the controller 180 may generate a control signal to enter the privacy protection mode. It is assumed that a third person X other than the user I is gazing at the display unit 151 of the mobile terminal 100 of the user I in the case where the face of the third person X is recognized have. In order to prevent the video call from being infringed by the third person X, the control unit 180 may generate a control signal for entering the privacy protection mode.

6C shows a screen in which the face of the third person Y other than the other party U is recognized in the counterpart video 202 of the mobile terminal 100. [ However, if the recognized face of the third person Y is not the front face, a general video call can be performed without entering the privacy protection mode.

6D shows a screen in which the front face of the third person X is recognized in the counterpart video 202 of the mobile terminal 100. [ When the front face of the third person X is recognized in the displayed counterpart image 202, the controller 180 may generate a control signal to enter the privacy protection mode.

In the above description, whether or not the face of the third party recognized in the user image 201 or the counterpart image 202 is a front face or not enters the privacy mode.

Hereinafter, whether to enter the privacy mode according to whether the face of the third person recognized in the user image 201 or the counterpart image 202 is larger than a predetermined size will be described in detail with reference to FIGS. 7A through 7C.

Referring to FIG. 7A, faces of third parties (Y ₁ , Y ₂ ) other than the other party U are recognized in the other party image 202. However, if the face size of the third party (Y _1, Y ₂₎ compared to the face size of the recognized person (U) very small, it can be a general video call without entering a privacy mode it performs. Or a face of a predetermined size or more among the faces of the third parties (Y ₁ , Y ₂ ) exists, a general video call can be performed without entering the privacy mode.

The controller 180 is small, the size of the detected face face (Y _1, Y ₂₎ in a state far from the other side (U), considered and privacy to be not stare facial parts of the display of the other (U) Mode can be determined.

7A, when the face size of the third party (Y ₁ , Y ₂ ) is smaller than the face size of the recognized party U, the third party (Y ₁ , Y ₂ ) It is considered that the mobile terminal is not taken care of and the control unit 180 may not generate a control signal for entering the privacy protection mode. Or the faces of the third parties Y ₁ and Y ₂ do not exist, the control unit 180 may not generate the control signal for entering the privacy mode.

Referring to FIG. 7B, faces of third parties (X, Y ₁ , Y ₂ ) other than the other party U are recognized in the partner video 202. At this time, if the size of the face of at least one third person X is larger than a predetermined ratio, the control unit 180 may generate a control signal to enter the privacy mode. The control unit 180 may generate a control signal to enter the privacy mode if there is more than one face X of a predetermined size or more among the faces of the third person X, Y ₁ , and Y ₂ . When a face of a third person X having a relatively large face size is recognized, it can be seen that a third person X other than the other party U is gazing at the display unit of the mobile terminal of the other party U. In order to prevent the video call from being infringed by the third person X, the control unit 180 may generate a control signal for entering the privacy protection mode.

7C shows that the face of the third person (X, Y) other than the user I is recognized in the user image 201. [ At this time, if only the faces of the user I and the third person Y are recognized in the user image 201 because the face size of the third person Y is smaller than the face size of the user I, It may not enter the privacy mode. However, when a face of a third person X having a size larger than a predetermined ratio with respect to the face size of the user I is recognized, a third person X other than the user I moves the user I The user may be looking at the display unit 151 of the terminal 100. Therefore, when the face of the third person X having a size larger than a predetermined ratio with respect to the face size of the user I is recognized in the user image 201, the control unit 180 controls the control unit 180 to enter the privacy protection mode Lt; / RTI > The control unit 180 may generate a control signal to enter the privacy protection mode when one or more faces X of a predetermined size or more among the faces of the third persons Y ₁ and Y ₂ in the user image 201 are recognized have.

Although not shown, the mobile terminal 100 according to the present invention recognizes the face of a third party other than the user I or the other party U in the user image 201 or the partner image 202, When the face size of the third person X in front of the face of the user I or the face of the other person U is larger than a predetermined ratio, May generate a control signal to enter the privacy mode.

In the above description, whether or not the user enters the privacy mode according to the state or size of the face of the third person recognized by the user image 201 or the counterpart image 202 has been described.

Hereinafter, whether a pattern of a face of the third person in the user image 201 or the counterpart image 202 is recognized and the user enters the privacy mode will be described in detail with reference to FIGS. 8A through 9C.

8A shows a screen in which the face of a third person X other than the user I is recognized in the user image 201 of the mobile terminal 100. [ 8B, when the face of the third person X other than the user I is recognized in the displayed user image 201, the controller 180 determines whether the recognized face I, X is stored in the memory 160 Or the face of the person stored in the storage unit.

At this time, the memory 160 may store information on the user I and the other party U together with a phone book or an album, and the controller 180 may store the recognized face I and X and the memory 160, Can be compared with each other. The control unit 180 can first distinguish the user I and the third party X from the user image 201 based on the user I information stored in the memory 160 according to the embodiment. After the user I and the third person X are distinguished from each other, it is possible to search whether the face of the person stored in the memory 160 and the third person X coincides with each other.

The face image stored in the memory 160 can be stored together with the face image in addition to the contact information for each individual so that the control unit 180 can store the face of the third person X recognized in the user image 201, It is possible to compare whether the photographs match or not. It is also possible to compare whether or not the photo album stored in the memory 160 matches the face of the third person X. [ In addition, information such as the SNS may be stored in the memory 160, and the controller 180 may compare the face information of the third person X recognized in the user image 201 with the person information stored in the SNS or the like.

The comparison between the information stored in the memory 160 and the faces I and X recognized in the user image 201 can be made by comparing the two faces in the case where the number of matching points between the two faces is equal to or greater than a predetermined number, Can be considered.

If the face of the third person X is a person stored in the memory 160, the control unit 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy protection mode. When the third person X views the user image 201, the third person X can know who is the video communication partner U of the user I. At this time, for example, even if the user I does not want to know who is making a video call with whom, the third person X who may be the user I knows is who the other party U of the video call is In addition, it is possible to know what the contents of the conversation are, which may invade the privacy of the user (I). Therefore, if the third person X is a person stored in the memory 160, the control unit 180 can allow the mobile terminal 100 to enter the privacy protection mode.

Or if the face of the third person X is not stored in the memory 160 as a result of the search, the controller 180 may generate a control signal to enter the privacy mode. It is possible to give the user I an unpleasant feeling that the privacy of the user I is exposed to the unknown user when the third person X who does not know by the user I is watching the video call of the user I. [ Therefore, if the third party X is not a person stored in the memory 160, the control unit 180 may allow the mobile terminal 100 to enter the privacy mode.

When the face of the third party X other than the other party U is recognized in the displayed counterpart video 202 as shown in FIG. 8C, the control unit 180 displays the face of the third party X in the memory 160 It is possible to search whether or not it matches the face of the stored person. Whether or not the face of the person stored in the memory 160 matches the face U, X recognized in the opposite side image 202 can be determined, for example, when two or more coinciding points between the two faces are equal to or greater than a predetermined number, Can be regarded as the same. The control unit 180 can first discriminate the other party U and the third party X from the counterpart video 202 based on the counterpart U information stored in the memory 160 according to the embodiment. After distinguishing the other party U and the third party X, it is possible to search whether or not the face of the person stored in the memory 160 matches the face of the third person X. [

If the face of the third person X is a person stored in the memory 160, the control unit 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy protection mode. The third party X can know the video call between the other party U and the user I when the third party X sees the other party video 202. [ At this time, even if the user I does not want to know that he / she is making a video call with the other party U, the third party X, which may be a person known to the user I, There is a possibility that the privacy of the user I may be infringed because the fact of the video call between the user I and the user U and the contents of the conversation can be known by viewing the call screen. Therefore, if the third person X is a person stored in the memory 160, the control unit 180 can allow the mobile terminal 100 to enter the privacy protection mode.

Or if the face of the third person X is not stored in the memory 160 as a result of the search, the controller 180 may generate a control signal to enter the privacy mode. The user I is exposed to the unknown person when the third person X who does not know by the user I is watching the video call of the other party U and therefore the user I is given an unpleasant feeling that the private life is exposed have. Therefore, if the third party X is not a person stored in the memory 160, the control unit 180 may allow the mobile terminal 100 to enter the privacy mode.

9A shows a screen in which a face of a third person Y other than the user I is recognized in the user image 201 of the mobile terminal 100. [ At this time, the user I may store the permitted person information in the user image 201 in the memory 160. [

That is, when the user I allows the third party Y to be recognized in the user image 201 of FIG. 9A to participate in the video call, the third party Y In the memory 160 as the permitted person information in a manner of selecting the face of the user. The allowed person information stored in the memory 160 may be added during the progress of the video call or may be preset information. At this time, the information about the user I and the other party U may be stored in advance in the memory 160 and may be stored in the memory 160 in such a manner that the user I selects a face.

The face of another third person X is recognized in addition to the user I, the other party U and the third person Y who are allowed to participate in the video call in the user video 201 as shown in FIG. 9B The control unit 180 can search whether or not the face of another third party X coincides with the face of the person stored in the memory 160 in which the allowable information is stored. The control unit 180 can first distinguish the user I and the third party X from the user image 201 based on the user I information stored in the memory 160 according to the embodiment. After the user I and the third person X are distinguished from each other, it is possible to search whether the face of the person stored in the memory 160 and the third person X coincides with each other. If the new third party X is not stored in the memory 160, the control unit 180 may generate a control signal to enter the privacy mode.

9C shows a screen in which the face of the third person Y other than the user I is recognized in the user image 201 of the mobile terminal 100. [ At this time, the third person Y in the user image 201 may be stored in the memory 160 as a person allowed to participate in the video call by the user I. If the face of the third party X other than the other party U is recognized in the other party image 202, the control unit 180 determines that the recognized face U, X corresponds to the permission information stored in the memory 160 You can search for people included. If the face of the third person X is not stored in the memory 160 as a result of the search, the controller 180 may generate a control signal to enter the privacy mode.

Although not shown, a third party recognized in the other party image 202 may be included in the permission information by the user I and stored in the memory 160. There may also be more than one third party to allow video call participation.

The control unit 180 searches the faces recognized in the user image 201 or the counterpart image 202 to see if there is a face of a third person X other than the user I or the other party U Only when at least two faces are recognized in the user image 201 or the counterpart video 202. If only one face is recognized in the user image 201 or the other party image 202, only the user I or the other party U is considered to participate in the video call, and only when two or more faces are recognized, It is possible to search whether the face of the third person X other than the face I or the face of the other person U exists, and to improve the retrieval efficiency.

In the above description, if the face of a third party other than the user I and the user U is recognized by analyzing the pattern of the face recognized in the user image 201 or the counterpart image 202, Respectively.

However, the mobile terminal 100 according to an embodiment does not analyze the pattern of the face recognized in the user image 201 or the counterpart image 202, and does not analyze the information stored in the memory 160 and the user image 201, The number of faces in the image 202 may be counted to determine whether to enter the privacy mode. Hereinafter, this will be described in detail.

The control unit 180 counts the number of faces satisfying a predetermined condition among the faces recognized in the user image 201 or the counterpart image 202 and enters the privacy protection mode when the number of faces is equal to or greater than the predetermined number A control signal can be generated.

The control unit 180 can control to enter the privacy mode when at least two faces are recognized in the user image 201 or the counterpart image 202. [ When two or more faces are recognized in the user image 201 or the counterpart image 202, the controller 180 determines which one of the user I or the third party X other than the user U It is assumed that the face is the face and the control signal is generated so that the mobile terminal 100 enters the privacy mode. If the face of the third person X other than the user I or the other party U is recognized in the user image 201 or the partner image 202, It is possible to know which party U is performing a video call with. When two or more faces are recognized in the user image 201 or the counterpart image 202, the controller 180 controls the mobile terminal 100 to enter the privacy mode in order to protect the privacy of the user I Signal.

In addition, the controller 180 may control the user to enter the privacy mode when two or more front faces are recognized in the user image 201 or the counterpart image 202. A third party X other than the user I or the opponent U is allowed to look at the display unit of the mobile terminal 100 of the user I or the mobile terminal of the other party U The control unit 180 may generate a control signal for entering the privacy mode in order to prevent the video call from being infringed by the third person X. If the face recognized is the front face Can be determined as described above.

Two or more faces U, I, and X are recognized in the other party image 202 or the user image 201 and the sizes of the largest faces U and I among the two or more faces It is possible to control to enter the privacy protection mode when the size of any one of the remaining faces X is larger than a predetermined ratio or when there are two or more faces U, I, X of a predetermined size or more. The user I or the third party X other than the user U can not recognize the user I or the user I in the case where two or more faces having a relatively large face size are recognized in the partner image 202 or the user image 201. [ It can be said that the user is looking at the display unit of the other party U. In order to prevent the video call from being infringed by the third person X, the control unit 180 may generate a control signal for entering the privacy protection mode.

At this time, the controller 180 recognizes that at least two faces are recognized in the user image 201 or the counterpart image 202, and that there are two or more faces in front of the recognized face, It may be controlled to enter the privacy protection mode when the size of the face of any of the remaining front faces is larger than the predetermined ratio.

According to the embodiment, the memory 160 may store information such as a phone book or an album together with information on the user I and the other party U. The control unit 180 can search whether the recognized face matches the face of the person stored in the memory 160 when at least two faces are recognized in the user image 201 or the counterpart image 202. In the embodiment, The control unit 180 can distinguish the user I and the third person X from the user image 201 based on the user I information stored in the memory 160. [ After the user I and the third person X are distinguished from each other, it is possible to search whether the face of the person stored in the memory 160 and the third person X coincides with each other. If the face of the third person X is a person stored in the memory 160, the control unit 180 may generate a control signal to allow the mobile terminal 100 to enter the privacy protection mode. Or if the face of the third person X is not stored in the memory 160 as a result of the search, the controller 180 may generate a control signal to enter the privacy mode.

According to an exemplary embodiment, the memory 160 may store the user information that the user I allows to participate in the video call in the user image 201 or the counterpart video 202. The control unit 180 may control the user other than the user I, the opponent U and the third person Y who are allowed to participate in the video call to the user video 201 or the video counterpart 202, The face of the third person X can be searched for whether or not the face of the third person X coincides with the face of the person stored in the memory 160 in which the allowable information is stored. If the detected third party X is not a person stored in the memory 160, the control unit 180 may generate a control signal to enter the privacy protection mode.

In the above description, whether to enter the privacy mode based on the number of faces recognized in the user image 201 or the counterpart image 202 has been described.

Hereinafter, the privacy mode of the control unit 180 in the mobile terminal 100 will be described in detail with reference to FIGS. 10A to 10G.

10A to 10G are views showing a privacy mode according to an embodiment of the present invention.

The mobile terminal 100 according to the embodiment is controlled to enter the privacy mode by the control unit 180 when the face of the third person X is recognized in the user image 201 or the counterpart image 202 .

Referring to FIG. 10A, when a control signal is generated to enter the privacy protection mode by the control unit 180, a lock screen may be displayed on the display unit 151. FIG. When the face of the third person X is recognized in the user image 201 captured by the camera module 121 and the lock screen is displayed on the display unit 151, The mobile terminal 100 may block information about what kind of work it is performing. Also, it is possible to prevent the third person X, which is not wanted by the user I, from watching the video call screen of the user I in a situation where the user I is not aware.

10B, when the face of the third person X is recognized in the user image 201 or the counterpart image 202, the control unit 180 controls the display unit 151 of the mobile terminal 100 to display any image May not be displayed. For example, the power of the display unit can be turned off. In this case, no image is displayed on the display unit 151 and the sound output module 152 can be turned off. At this time, since no image is displayed on the display unit 151, the third person X can not know what kind of work the current user is currently doing with the mobile terminal 100, and thus the privacy of the user I is infringed Can be prevented.

At this time, no image may be displayed on all of the display unit 151 and only a part of the counterpart image 202 or the user image 201 displayed on the display unit 151 may not be displayed. For example, in a case where a third party X other than the user I is recognized in the user image 201, the third party X may not know who the other party U of the video call is, 202 may not be displayed.

10C, when the face of the third person X is recognized in the other party image 202, the control unit 180 can stop the transmission of the user image 201 captured by the camera module 121 have. It is possible to prevent the user I from showing the information about the user I to the third person X in the unwanted counterpart video 202 since the user image 201 is not transmitted to the counterpart U. [

Although not shown, the control unit 180 may stop the transmission of the user image 201 even when the face of the third person X is recognized in the user image 201. [ In this case, it may not inform the other party U of who is in the vicinity of the user I.

10D, when the face of the third person X is recognized in the other party image 202, the control unit 180 replaces the user image 201 photographed by the camera module 121 with a substitute image Can be transmitted. It is possible to prevent the third party X in the counterpart video 202, which is not wanted by the user I, from showing information about the user I because the user video 201 is not transmitted to the counterpart but the substitute video is transmitted. have.

Although not shown, the control unit 180 may replace the user image 201 with a substitute image, even when the face of the third person X is recognized in the user image 201. [ In this case, it may not inform the other party U of who is in the vicinity of the user I.

The substitute image may be a preset image, or may be a video image of the user image 201, and may select one of images stored in the memory 160 and transmit the selected image. In order to select one of the stored images, the controller 180 may display a separate pop-up window. A specific description of the substitute image will be described later.

According to the embodiment, even when the control unit 180 searches whether the third person X corresponds to a person stored in the memory 160 as illustrated in FIGS. 7B, 7C, 8B and 8C, It is possible to transmit a substitute image to the user U in place of the user image 201 photographed by the camera module 121. [ The privacy of the user I may be infringed by the third person X who is the search target even while the third party X is searching for the person who is allowed to participate in the video call by the control unit 180, It is possible to transmit the substitute image not the user image 201 even during the search of the control unit 180. [ At this time, instead of transmitting the substitute video, the video transmission may be stopped to the other party U. After the search is completed, the controller 180 may determine whether to enter the privacy mode according to the search result.

10E is a diagram showing a display unit of the other party U of the video call. The control unit 180 may transmit a notification message to the counterpart U if the face of the third person X is recognized in the counterpart video 202. [ The content may be, for example, "Warning: video call intrusion by a third party ", as shown in the figure, but is not limited thereto. The other party U can recognize the third party X who is gazing at his / her display unit by transmitting a notification message to the other party, thereby preventing privacy invasion. Even if the face of the third person X is recognized in the user image 201, a notification message is transmitted to the other party U to notify the third party X that the current video call is being viewed U). At this time, the control unit may stop the transmission of the user image 201 transmitted to the counterpart U with the transmission of the notification message, and may transmit the replacement image instead of the user image 201. [

10F, when the face of the third person X is recognized in the user image 201 captured by the camera module 121, the control unit 180 can stop the reception of the counterpart video 202 . It is possible to prevent the user I from showing the information on the other party U of the video call to the third party X in the unwanted user video 201 because the user I does not receive the counterpart video 202 .

10G, when the face of the third person X is recognized in the user image 201 captured by the camera module 121, the control unit 180 controls the display unit 151 to display the counterpart image 202 ) Can be displayed instead of the alternative image.

If the face of the third person X is recognized in the user image 201 or the counterpart video 202 according to the embodiment, the control unit 180 automatically terminates the video call, (U). Depending on the embodiment, the state may be changed to Voice over Internet Protocol (VoIP) or Short Message Service (SMS).

Although not shown, according to an embodiment, the display unit 151 may display an image being photographed by the camera module 121, and may transmit a substitute image to the other party U. At this time, the sound output module 152 may be turned off so that the third party recognizes that the user is shooting with the camera module 121, thereby preventing privacy from being infringed.

The privacy mode may be performed by performing only one of the operations described with reference to FIGS. 10A to 10G, and the combined operation may be performed together, but the present invention is not limited thereto.

The privacy protection mode of the control unit 180 in the mobile terminal 100 has been described above. Hereinafter, the alternative image of the privacy protection mode will be described in detail with reference to FIGS. 11A to 12H.

11A to 11D are views showing a display unit on which a substitute image is displayed.

In the privacy mode, the substitute image of the user image 201 or the counterpart image 202 may be a preset image, or may be an image of the user image 201 or the counterpart image 202, And may be any one of stored images. In order to select one of the stored images, the controller 180 may display a separate pop-up window.

The alternative image may be an image of the user's image 201 or the image of the other party 202. For example, as shown in Figs. 11A to 11C, a mosaic process of the user image 201 or the counterpart image 202, a process of smoothing the user image 201 or the counterpart image 202, But the present invention is not limited thereto, for example, mosaic processing an area other than the user I or the opponent U in the image 201 or the counterpart image 202, blurring or erasing the screen. A method of mosaicing an area other than the user I or the counterpart U in the user image 201 or the counterpart image 202 shown in Fig. 11C, blurring or deleting the screen will be described later. Alternatively, the replacement image may be an image or an image stored in the memory 160, as shown in FIG. 11D.

12A to 12H are views showing an embodiment of a method of editing a substitute image of a user image.

Referring to FIG. 12A, a counterpart video 202 and a user video 201 are divided into two parts and displayed on a display unit 151. FIG. At this time, the face of the third party (X ₁ , X ₂ ) other than the user I is recognized in the user image 201. The user I may input a control signal to edit an area other than the area to be displayed to the other party U in the user image 201. [ In this case, when the display unit 151 includes the touch sensor 137, for example, a signal may be input by touching a part of the user image 201 of the display unit 151 as a pointer.

As shown in FIG. 12B, when the user I inputs a control signal for editing, a pop-up window 1201 for confirming whether to edit the user image 201 can be displayed, for example. In the pop-up window 1201, an application for selecting whether to edit is displayed. When the user selects " Cancel " in the pop-up window 1201, the editing is canceled and the unedited user image 201 is transmitted to the other party U. [

When the user I selects " edit " in the pop-up window 1201 of Fig. 12B, the editing tool window 1202 can be displayed as shown in Fig. 12C. For example, the editing tool window 1202 may have a circular line, a human line, a free line, and the like as an editing tool.

For example, when a circular line is selected in the editing tool window 1202, a region to be selected by the user is displayed as a circular line as shown in FIG. 12D. At this time, the user I can adjust the selection area by moving, enlarging or reducing the selected circular line as shown in FIG. 12E. For example, when the display unit 151 includes the touch sensor 137, movement of the circular line may be possible by moving the pointer to a desired position while touching the center of the circular line. In addition, the enlargement of the circular line may be performed by selecting the outline of the circular line and dragging it out of the circular shape, and the reduction of the circular line may be possible by selecting the outer line of the circular line and drawing it into the circular shape.

At this time, the user I may enlarge or reduce the user image 201 by a separate operation to more finely control the selection area. According to the embodiment, after the user image 201 is enlarged or reduced in advance, an editing tool may be selected to proceed with editing of the user image.

When the user touches the selection area twice successively or touches the selection area for a long time to complete the area selection, for example, a selection popup window 1203 as shown in FIG. 12F may be displayed. For example, the user I can press the " ok " button when the selection is completed and the " no "

Referring to FIG. 12G, when the region selection of the user is completed, the counterpart image 202, the selection region 205, and the edited image 203 may be displayed on the display unit 151 according to the embodiment. The edited image 203 may be mosaiced, blurred, or deleted except for the area 205 selected by the user. A selection popup window for editing an image may be displayed according to an embodiment.

For example, the user can select to finish editing and send the edited image 203 to the other party U by touching the edited image 203 in Fig. 12G for a long time.

When the editing of the substitute image is completed, the partner image 202 and the edited substitute image 203 are displayed on the display unit 151 as shown in Fig. 12H, and the substitute image 203 is transmitted .

12A to 12H show a process of editing an area other than the selected area, it will be apparent to those skilled in the art that the selected area can be edited according to the embodiment.

13 is a diagram illustrating a process of entering a privacy mode according to a specific operation of a user in a mobile terminal according to an exemplary embodiment of the present invention.

The mobile terminal 100 according to an exemplary embodiment may allow the user I to enter the privacy mode when the user I takes a specific action during a video call. For example, if the user I can not press the end button of the video call during the video call, the user I can perform a predetermined operation to enter the privacy mode. As shown in FIG. 13, the preset specific operation may be a case in which the user I only faces a side face by turning his or her face. In addition, the preset specific operation may include, but is not limited to, a case where the user I blinks consecutively several times, a user shakes the mobile terminal 100, and the like. At this time, the controller 180 searches the memory 160 in which the information about the user I is stored, and determines which of the faces in the user image 201 is the face of the user I.

According to the embodiment, when the face of the third person X other than the user I or the opponent U is recognized in the user image 201 or the counterpart image 202, when entering the privacy mode, It is possible to enter the privacy mode only when there is a predetermined operation of the user. That is, even if the face of the third person X is recognized in the user image 201 or the counterpart image 202, if it is determined that the user does not need to enter the privacy protection mode, the user does not enter the privacy protection mode You can proceed with the video call as it is. On the other hand, if it is determined that the user needs to enter the privacy protection mode, the user can enter the privacy protection mode by performing the predetermined operation. For example, as shown in FIG. 5B, when the face of the third person X is recognized in the user image 201, the user I does not directly enter the privacy protection mode, The user can enter the privacy mode only when the user is shaken. By doing so, the privacy of the user I and the user U can be protected while preventing the user I from unintentionally entering the privacy mode.

Even when the information stored in the memory 160 and the pattern of the face in the user image 201 or the counterpart image 202 are compared to enter the privacy mode according to the embodiment, It is of course possible to enter the privacy mode only. For example, when the face of the third person X other than the user I and the other party U is recognized during the video call, the user I does not directly enter the privacy protection mode, Only when it is blinking can you enter privacy mode. By doing so, the privacy of the user I and the user U can be protected while preventing the user I from unintentionally entering the privacy mode.

14A and 14B are diagrams illustrating a video call conversion according to a change in illumination in a mobile terminal according to an embodiment.

When a change of the proximity sensor 141 or a change in the illuminance of the display unit 151 is detected during the video call, the mobile terminal 100 according to an embodiment of the present invention transmits a voice call, a Voice over Internet Protocol (VoIP) Short Message Service).

When starting a video call as shown in FIG. 14A, the initial value of the illuminance of the display unit 151 can be set. 14B, when a change in illumination is detected in the user image 201 or the counterpart image 202 in the course of proceeding with a video call, the controller 180 controls the state of the video call by voice call, VoIP, SMS, Can be changed. At this time, the display unit 151 includes the touch sensor 137, so that the user I can touch the display unit 151 to change the talk state. In this case, the controller 180 may transmit a message informing the other party U that the call has been switched to a voice call or the like.

FIG. 15 is a view illustrating a display unit when performing a multi-party video call.

15, when a user I attempts a multi-party video call by adding a new party while performing a video call with the other party, or when a user I receives a multi-party video call, the user I ) To enter the privacy mode.

As shown in FIG. 15, when a video call is performed with a plurality of other parties U ₁ to U ₅ , the controller 180 searches the memory 160 storing the person information, the address book, can do. If the other party is not stored in the memory 160, the control unit 180 may generate a control signal to enter the privacy mode. FIG. 15 shows that the transmission of the user image to the other party U ₁ is stopped as the privacy mode. The privacy mode is the same as that described above, and a detailed description thereof will be omitted.

According to the embodiment, when the face of the third person X is recognized in the other party image of the other party during the multi-party video call, the control unit 180 can generate a control signal to enter the privacy mode. It goes without saying that, even when the face of the third person X is recognized in the user image, the user may enter the defensive operation mode.

The mobile terminal 100 according to another exemplary embodiment of the present invention may include an audio output module 152 for outputting a voice signal received during a video call and a microphone 122 for receiving a voice of a user for video call have. At this time, when a voice of a third party X other than the user I is recognized in the voice input to the microphone 122, or when a voice of a third party X other than the other party U is received When the voice is recognized, the control unit 180 may generate a control signal to enter the privacy protection mode of the mobile terminal 100. [ The voice of the user I and the voice of the other party U may be stored in the memory 160. [ The control unit 180 compares the input voice signal with the voice stored in the memory 160 and outputs the voice signal to the mobile terminal 100 when the voice input is not the voice of the user I or the other party U. It is possible to enter the privacy mode.

According to the embodiment, the mobile terminal 100 recognizes the voice of the user I or the voice of the third party X other than the voice of the other party U, (U), the user may enter the privacy mode. The name of the user I or the name of the other party U may be stored in the memory 160.

When the face of the third person X other than the user I or the other party U recognized in the user image 201 or the counterpart image 202 is removed during the privacy protection mode according to the embodiment , The control unit 180 of the mobile terminal 100 can release the privacy protection mode and generate a control signal to proceed with a general video call.

Further, according to an embodiment of the present invention, the above-described method can be implemented as computer-readable code on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet) . In addition, the computer may include a control unit 180 of the terminal.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: mobile terminal 110: wireless communication unit
120: A / V input / output unit 130: user input unit
140: sensing unit 150: output unit
160: memory 170: interface section
180: control unit 190: power supply unit

Claims (13)

A camera module for capturing a user image for a video call;
A wireless communication unit for transmitting and receiving video and audio data for the video call;
A display unit for displaying the photographed user image and the received counterpart image;
A memory for storing information on the user and the other party; And
And a controller for searching for a face of a third party other than the user or the other party in the user image or the counterpart video and controlling the user to enter the privacy mode when the face of the third party is recognized,
Wherein,
And to enter the privacy protection mode when the size of the face of the third party is larger than a predetermined ratio with respect to the face size of the user or the other party. delete The method according to claim 1,
Wherein the control unit controls to enter the privacy protection mode when the face of the third person is front face. delete The method according to claim 1,
A telephone directory or an album is stored in the memory,
Wherein the control unit searches the third person's face for a person stored in the memory and controls the third person to enter a privacy mode if the face of the third person is not stored in the memory. The method according to claim 1,
The memory is configured to store person information that is allowed to participate in a video call set by the user,
Wherein the control unit searches the third person's face for a person stored in the memory and controls the third person to enter a privacy mode if the face of the third person is not stored in the memory. The method according to claim 1,
In the privacy mode, when the face of a third party is recognized in the partner image, or when a face of a third party is recognized in the user image, transmission of the user image is stopped, a substitute image is transmitted, , Or stops the reception of the other party's video. delete delete The method according to claim 1,
Wherein the privacy protection mode displays a lock screen on the display unit or turns off the display unit when a face of a third person is recognized in the user image. delete delete delete

Images