KR20110028811A - Mobile terminal and information providing method thereof - Google Patents

Abstract

PURPOSE: A mobile terminal and an information providing method thereof are provided to supply various service related to a corresponding person through face recognition. CONSTITUTION: A control unit(180) extracts a facial region form an image displays on a display picture, and displays at least one kind of an information item which indicates the information which can be related to a person of the extracted face region. A touch screen is loaded in correspondence to the display screen, and the extracted face region is selected from the image displayed on the display screen through the touch operation on the touch screen.

Description

Mobile terminal and information providing method in mobile terminal

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of providing various services related to a corresponding image through image matching and a method of providing information in the mobile terminal.

A terminal such as a personal computer, a notebook computer, or a mobile phone has various functions, for example, a multimedia device having a complex function such as photographing or photographing of a moving picture, reproduction of music or a moving picture file, (Multimedia player).

The terminal can move It may be divided into a mobile terminal and a stationary terminal, depending on whether it is present. The mobile terminal may be further classified into a handheld terminal and a vehicle mount terminal according to whether a user can directly carry it.

In order to support and increase the function of the terminal, various ways of improving the structural part and / or software part of the terminal have been considered.

The present invention provides an improved type of mobile terminal in terms of user convenience and function provision.

In addition, the present invention provides a mobile terminal that can provide a variety of services associated with the person through the face recognition.

The present invention also provides a mobile terminal capable of searching and editing multimedia content in a more intuitive manner.

In addition, the present invention provides a mobile terminal capable of increasing user convenience in terms of content / service use by associating respective multimedia contents / services, a phone book, a text / image file, etc., by a category related to the person.

Other objects of the present invention will be readily understood through the following description.

One aspect of the present invention for realizing the above object is a mobile terminal, the display; And a controller configured to extract a face region from the image displayed on the display screen and to display at least one information item including information that may be related to the person of the extracted face region on the display screen.

In an embodiment, the controller may display an image of a candidate person having similarity with the extracted face area through face recognition on the display screen, and when any one of the images of the candidate person is selected, An information item including information that may be associated with a person may be displayed on the display screen.

In an embodiment, the control unit may generate a person folder that uses the extracted face area image as a representative image of the person, and includes an information item that may be associated with the person in the extracted face area of the person folder. Can be included as a lower layer.

In an embodiment, the controller requests the external server to provide the related information about the person of the extracted face region through the communication unit of the mobile terminal, and when the information related to the person is received from the external server, The received information may be itemized and displayed on the display screen.

Another aspect of the present invention for realizing the above object is a mobile terminal, the display; And a controller configured to extract a region of interest from an image displayed on the display screen and display at least one information item including information that may be related to the extracted image of the region of interest on the display screen.

As another aspect of the present invention for realizing the above object, a method for providing information in a mobile terminal according to the present invention comprises the steps of: extracting a face region from an image displayed on a display screen; And displaying at least one information item including information that may be associated with a person of the extracted face region on the display screen.

The method may further include, before the extracting of the face region, selecting the face region from the image displayed on the display screen by a touch manipulation on the touch screen provided in the mobile terminal. have.

In an embodiment, the displaying of at least one information item including information that may be related to the person of the extracted face region on the display screen may have similarity with the extracted face region through face recognition. Displaying an image of a candidate person; Selecting one of the images of the candidate person; And displaying an information item including information that may be associated with a person of the selected image.

In an embodiment, the method may further include: generating a person folder having the extracted face area image as a representative image of the person; And including an information item related to the person of the extracted face region as a lower layer of the person folder.

In an embodiment, the method may further include: requesting an external server to provide related information about a person of the extracted face region; Receiving information related to the person from the external server; And itemizing the received information and displaying the received information on the display screen.

As another aspect of the present invention for realizing the above object, a method of providing information in a mobile terminal according to the present invention comprises the steps of: extracting a region of interest from the image displayed on the display screen; And displaying at least one information item including information that may be associated with the extracted image of the ROI on the display screen.

According to the mobile terminal according to the present invention, the following effects are obtained.

The present invention can provide an improved type of mobile terminal in terms of user convenience and function provision.

In addition, the present invention may provide a mobile terminal capable of providing various services related to the person through face recognition.

In addition, the present invention can provide a mobile terminal capable of searching and editing multimedia content in a more intuitive manner.

In addition, the present invention can provide a mobile terminal that can increase the user convenience in terms of content / service use by associating each multimedia content / service, phone book, text / image files, etc. by category associated with the person. have.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. It is to be understood, however, that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. Like reference numerals designate like elements throughout the specification. In addition, when it is determined that the detailed description of the known function or configuration related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the numbers (eg, first, second, etc.) used in the description process of the present specification are merely identification symbols for distinguishing one component from another component.

The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other.

In the present specification, a mobile phone as a mobile terminal will be described. However, the mobile terminal to which the present invention can be applied includes a smart phone, a laptop computer, a digital broadcasting terminal, an MP3 player, and a PDA (Personal Digital Assistants). Of course, any terminal having portability or mobility, such as PMP (Portable Multimedia Player), navigation, etc., may of course be applicable.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention. 2 is a diagram illustrating an appearance of a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 1, the mobile terminal 100 includes a wireless communication unit 110, an A / V (Audio / Video) unit 120, an input unit 130, a sensing unit 140, an output unit 150, The memory unit 160, the interface unit 170, the controller 180, and the power supply unit 190 may be included. The components shown in FIG. 1 are not essential and may implement a mobile terminal having more or fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short range communication module 114, a location information module 115, and the like. .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcast signals using various broadcasting systems, and in particular, digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), and media forward link (MediaFLO). Digital broadcast signals can be received using digital broadcasting systems such as only), digital video broadcast-handheld (DVB-H), integrated services digital broadcast-terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used.

The location information module 115 is a module for checking or obtaining the location of the mobile terminal. A representative example of the location information module is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates information about a distance of one point (object) away from three or more satellites, and information on a time at which the distance information is measured, and then calculates the calculated distance information. By applying the trigonometric method to, three-dimensional positional information according to latitude, longitude, and altitude of one point at a time can be calculated. Furthermore, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module 115 continuously calculates the current position in real time and uses the same to calculate speed information.

The A / V unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame can be displayed on the display module 151. [

The image frame processed by the camera 121 may be stored in the memory unit 160 or transmitted to the outside through the wireless communication unit 110. The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output in the call mode. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The input unit 130 is a user interface for receiving various commands from a user, and the implementation thereof is not particularly limited. For example, the input unit 130 may be implemented as a key pad, a dome switch, a touch pad (constant voltage / capacitance), a jog wheel, a jog switch, and the like.

The input unit 130 may include a plurality of operation units (see 131 and 132 of FIG. 2) for receiving a command for controlling an operation of the mobile terminal 100. The manipulation units may be employed in any manner as long as the user is in a tactile manner to operate while having a tactile feeling. The contents input by the manipulation units 131 and 132 may be variously set. For example, the first manipulation unit 131 receives a command such as start, end, scroll, and the like, and the second manipulation unit 132 adjusts the volume of sound output from the receiver or the touch recognition mode of the display module 151. It may receive a command such as switching to.

In addition, in recent years, a large number of touch screen systems have been adopted due to an increase in space utilization, an enlargement of display screens, and design considerations. It can also be located at).

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like.

The output unit 150 is used to generate an output related to visual, auditory, or tactile, and may include a display module 151, a sound output module 152, an alarm unit 153, a haptic module 154, and the like. have.

The display module 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, the mobile terminal displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays photographed and / or received images, a UI, and a GUI.

The display module 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. 3D display).

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is a transparent LCD. The rear structure of the display module 151 may also be of a light transmission type. With this structure, the user can see the object located behind the terminal body through the area occupied by the display module 151 of the terminal body.

According to an implementation form of the mobile terminal 100, two or more display modules 151 may exist. For example, a plurality of displays may be spaced apart or integrally disposed on one surface of the mobile terminal 100, or may be disposed on different surfaces, respectively.

In addition, when the display module 151 and a sensor for detecting a touch operation (hereinafter, referred to as a “touch sensor”) form a mutual layer structure (hereinafter, abbreviated as “touch screen”), the display module 151 may be In addition to the output device can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display module 151 or capacitance generated at a specific portion of the display module 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure and directionality of the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to a touch controller (not shown). The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can determine which area of the display screen is touched.

In this case, a proximity sensor (not shown) may be disposed in the inner region of the mobile terminal covered by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using physical contact by using an electromagnetic force or infrared rays. Proximity sensors have a longer life and higher utilization than touch sensors.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, and a magnetic proximity sensor.

When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by a change in an electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, an act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position at which the proximity touch is performed by the pointer on the touch screen means a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 152 may output a sound signal related to a function (for example, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or audio signal may also be output through the display module 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to the vibration, the haptic module 154 may be used for the effects of stimulation by the arrangement of pins vertically moving with respect to the contact skin surface, the effect of the injection force of the air through the injection or inlet or the stimulation through the suction force, and the stimulation that rubs the skin surface. Various tactile effects may be generated, such as effects by stimulation through contact of electrodes, effects by stimulation using electrostatic force, and effects of reproducing a sense of warmth and heat using an endothermic or heat generating element.

The haptic module 154 may not only deliver a haptic effect through direct contact, but also implement a haptic effect through a muscle sense of a user's finger or arm. Two or more haptic modules 154 may be provided according to configuration aspects of the mobile terminal 100.

The memory unit 160 may store a program for the operation of the controller 180 and may store input / output data (eg, a phone book, a message, a still image, a video, etc.). The memory unit 160 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

The memory unit 160 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) Magnetic Memory, Magnetic It may include a storage medium of at least one type of disk, optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path with all external devices connected to the mobile terminal 100. The interface unit 170 may receive data from an external device or receive power to transfer the data to each component inside the mobile terminal 100 or to transmit the data inside the mobile terminal 100 to an external device. For example, wired / wireless headset ports, external charger ports, wired / wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input / output (I / O) ports, The video I / O port, the earphone port, and the like may be included in the interface unit 170 (see reference numeral 171 of FIG. 2).

The identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and a universal user authentication module. (Universal Subscriber Identity Module, USIM) and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit 170 serves as a passage through which power from the cradle is supplied to the mobile terminal 100, or various command signals input from the cradle by a user. May be a passage that is delivered to the mobile terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal 100. For example, perform related control and processing for voice calls, data communications, video calls, and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on the touch screen as text and an image, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing the functions. It may be implemented by the controller 180.

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application written in a suitable programming language. In addition, the software code may be stored in the memory unit 160 and executed by the control unit 180.

3 is a front view of a terminal for explaining an operation state of a mobile terminal according to an embodiment of the present invention.

Various types of visual information may be displayed on the display module 151. These pieces of information can be displayed in the form of letters, numbers, symbols, graphics, or icons. At least one of the letters, numbers, symbols, graphics, or icons may be displayed in a predetermined arrangement for inputting such information, thereby being implemented as a keypad. Such a keypad may be called a so-called " soft key ". 3 illustrates receiving a touch applied to a softkey through the front of the terminal body.

The display module 151 may operate in an entire area or may be divided into a plurality of areas. In the latter case, the plurality of areas can be configured to operate in association with each other. For example, an output window 151a and an input window 151b are displayed on the upper and lower portions of the display module 151, respectively. In the input window 151b, a soft key 151c displaying a number for inputting a telephone number or the like is output. When the softkey 151c is touched, a number or the like corresponding to the touched softkey is displayed on the output window 151a.

In FIG. 3, the output window 151a and the input window 151b are separated from each other in different areas of the display screen, but this is merely an example. Without distinction between the output window 151a and the input window 151b, the entire area of the display screen displayed by the display module 151 serves as the output window for information display and the input window for information input. Obviously, you can play a role at the same time. Assuming the following, the same reference numerals as in the output window 151a of FIG. 3 will be given to the display screen displayed by the display module 151.

Hereinafter, a mobile terminal and an information providing method in the mobile terminal according to an embodiment of the present invention will be described in detail.

Hereinafter, the present invention will be described with reference to embodiments related to a method for providing information in a mobile terminal. These embodiments will also be embodied by the flowcharts shown through FIGS. 4 and 19, respectively. Therefore, hereinafter, the present invention will be described based on the above two flowcharts, and for the purpose of understanding the present invention, reference is made to FIGS. 5 to 18 and FIGS. 20 and 21 together.

In addition, in the following, the front of the display screen (refer to reference numeral 151a of FIG. 4 or less) displayed by the display module 151 of the mobile terminal 100 is provided for the convenience of the understanding and explanation of the contents of the invention. The present invention will be described on the assumption that it is implemented as a touch screen. As described above, the touch screen is a device capable of performing both an information input function and an information display function. The information input function may be performed by a touch method. Accordingly, hereinafter, it is assumed that information input from a user, such as input of a specific command, selection of an information item such as a function / menu, etc., is made by a touch operation on the touch screen (i.e., a contact touch). do.

However, the above assumptions are only for ease of understanding of the contents of the present invention and for convenience / concentration of the description, and are not intended to limit the scope of the present invention. For example, the information input function may be operated in addition to a touch operation on a touch screen, operation of a keypad using up-down keys, direction keys, selection keys, etc., handwriting input operation on a touch pad, command input through voice recognition, and proximity. Proximity touch using a sensor or the like, it may be made by the recognition of the movement of the terminal using a motion sensor or the like.

Hereinafter, FIGS. 4 to 21 will be described in sequence.

4 is a flowchart illustrating a method of providing information in a mobile terminal according to an embodiment of the present invention. In describing the flowchart of FIG. 4, reference is made to FIGS. 5 to 18 together.

According to the flowchart of FIG. 4, first, an image including a human face is displayed on a display screen of a terminal [S410].

Hereinafter, for convenience of description, an "image including a human face" will be briefly referred to as a "portrait image". Therefore, in the present specification, a "person image" is sufficient if at least one face is included in the image, regardless of whether the person is a main photographing target.

In the operation process of the terminal, for example, as shown in FIG. 5A, when the person image 10 is displayed on the display screen 151a, there may be various cases.

When the person image is displayed on the display screen, for example, the user may be displayed by loading a person image pre-stored in the terminal, or the person image obtained in the actual imaging process using the camera provided in the terminal is displayed. It may be. Of course, the person image obtained in the imaging process may be an image of a preview state. Alternatively, the person image may be an image received during a video call, an image included in a video being played, or a captured image.

In the state where the person image is displayed on the display screen as described above, the face area (see reference numeral 11 of FIG. 5B) is selected [S420]. Here, the selection of the face area may be based on a touch manipulation by the user on the touch screen as shown in (b) of FIG. 5. For example, referring to FIG. 5B, the user draws a face portion (see reference numeral 11) in the person image 10 displayed on the display screen 151a so as to have a rectangular trajectory, thereby making a face region. Is selecting.

When the face region is selected through the previous step (S420), the controller 180 extracts the face region from the person image displayed on the display screen [S430].

At this time, the "face area extracted" through the present step (S430) (for example, reference numeral 11a of FIG. 5C) refers to the "selected face area" (in other words, through the previous step S420). , A face region corresponding to the trajectory of the user's touch manipulation (see, for example, reference numeral 11 in FIG. 5B) may not necessarily match. The following cases may exist in the extraction method of the facial region.

First, a method of extracting the "selected face area" as it is from the person image through the previous step (S420) may be used. In this case, the controller 180 only needs to extract a portion corresponding to the user's touch trajectory from the person image. Therefore, in this case, the "extracted face area" may have the same specifications (border shape and area, etc.) as the "selected face area".

In the above case, the "selection of the face area" (that is, the touch operation associated with reference numeral 11 of FIG. 5B) in the previous step S420 is a control signal (hereinafter, " A function as a face extraction signal ", as well as a control signal for selecting / designating an image area to be extracted by the face extraction signal (hereinafter, referred to as an" extraction area designation signal ") is performed.

Secondly, a method of extracting an image of a face region having a predetermined specification (eg, a predetermined border shape, size, area, resolution, etc.) regardless of the "selected face region" may be used. For example, if there is a touch operation as shown in (b) of FIG. 5, regardless of the original border shape, size, etc. of the closed curve area according to the touch trajectory, a predetermined specification (for example, the aspect ratio That is to extract an image of the face area having a rectangular border of 1: 1.2 and having a predetermined resolution as in the thumbnail image.

Here, the predetermined standard may be a fixed standard, but may be a plurality of or variable standards that vary the accumulation, ratio, resolution, etc. to be applied to each target image in consideration of the scale, resolution, etc. of the target image. . In addition, although the "face area" extracted by the controller 180 may mean only an image part corresponding to the "face" of the person, an appropriate image including the "face" of the person (for example, , Facial images, such as when you take proof photos or passport photos.

In the above case, the "selection of the face area" (that is, the touch operation associated with reference numeral 11 of FIG. 5B) in the previous step S420 will have only the function as the aforementioned face extraction signal.

Therefore, unlike the first case above, since the face area to be extracted is not directly designated by the user, in this case, the face to be extracted by the controller 180 according to the predetermined standard from the person image The process of detecting the area may be necessary. Detection of such a face region may include detecting a region corresponding to a face color or skin color in a color image, facial features (for example, eyes, nose, mouth, eyebrows, or two eyes). By detecting a feature line of the inverted triangle connecting the nose and the like, a method of detecting a face region may be used. This corresponds to a known technique in the field of face detection / recognition based on image processing, and a detailed description thereof will be omitted.

In addition, when the second method (that is, the controller 180 automatically detects and extracts the “face area” according to the face extraction signal) is further expanded, the step S420 is illustrated in FIG. 4. It may be omitted from the flowchart of.

That is, in the case of step S420 described above, the image portion corresponding to the face area is illustrated as the user directly selects the person image, but this is not necessarily the case. When pressing a specific key button, input of a specific voice, generation of a specific movement / motion of the terminal itself, or input of a specific touch pattern / gesture on the touch screen are predefined as the face extracting signal, the same procedure as in step S420 is performed. "Selection of face area" process may be unnecessary. For example, when there is an input of a touch pattern / gesture on a touch screen such as drawing a "circle" three times in a row with a person image displayed as shown in FIG. The controller 180 recognizes this as the face extraction signal and automatically detects and extracts a face region from the person image.

As described above, according to an embodiment, the step S420 may be omitted. However, in the case of a group picture in which several persons exist in one image, the step S420 may be smoother. For example, in the case of a group photo, one of the plurality of people is selected from the user (in this case, the selection is not necessarily "selection of the face area", and may be a selection of the area where the person to be selected is displayed / located. It may be a clearer way to go through the receiving step.

When the face region of the person is extracted through the step S430, the controller 180 displays an image of the candidate person having similarity with the extracted face area on the display screen [S440]. To this end, the controller 180 may perform a face contrast process between the extracted face region and the person images previously stored in the terminal.

In the face matching process, a known face recognition technique may be used. For example, the controller 180 normalizes only both face parts to the same standard image, and then distances between the two or more feature points, such as center coordinates and lengths of feature points (eyebrows, eyes, nose, mouth, etc.). The degree of matching can be calculated by comparing the outlines of the face such as the length and width of the face with the outline line of the face. It is obvious that various face recognition / contrast techniques can be applied.

The person image to be compared in the face contrast process may be all the person images stored in the terminal, but may be limited to a specific control group that is selected and present in the terminal for use in the face contrast process. It may be.

For example, the specific control may be an image group including one or two representative person images (or representative face images) suitable for face contrast for each person among the person images previously stored in the terminal. In this case, there may be a plurality of image small groups in which similar faces are gathered together in the image group.

Alternatively, the specific control group is a collection of images determined to be similar faces, based on some comparison factors (eg, the coordinates of the feature points, the positional relationship, etc.) used for the face contrast process. It may be a group. For example, the ratio of the width to the face length is divided into a first group of 0.4 to 0.55, a second group of 0.55 to 0.65, a third group of 0.65 to 0.75, a fourth group of 0.75 to 0.9, and a fifth group of 0.9 or more. In other words, the group includes a representative image of a person belonging to the group and classifies it in advance.

According to the above method (that is, a method using a specific control group), the amount of calculation and the calculation time of the controller 180 can be greatly reduced, so that the above-described "face matching process" can be made more quickly and easily. For example, if the face length / width ratio of the face area extracted through the previous step S430 is 0.69, the face area / width ratio belongs to the third group, so that only faces with the person images (or face images) belonging to the third group are present. The collation may be performed to display the result (that is, the candidate image of step S440). Alternatively, the person images belonging to the third group, which are derived based only on the result of the comparison of the face length / width ratio, may be selected / displayed as the candidate image in the step S440 without a specific face contrast process. In the latter case, the reliability of face recognition will be reduced, but can be very good in terms of speed of result display. Alternatively, a method of considering both the reliability of the face recognition and the speed of the result display may include a method of displaying a person image commonly belonging to an image group according to each comparison factor as the candidate image by mixing two or more comparison factors. Could be.

Through the above-described methods, the image of the candidate person selected as having similarity to the extracted face region may be displayed on the display screen as shown in FIG.

Referring to FIG. 5C, an image (hereinafter, simply referred to as a “face image”) 11a of the face region of the person extracted through step S430 is displayed at the center of the display screen 151a. In addition, it can be seen that images 21 to 26 of six candidate persons are displayed around the face image.

However, FIG. 5C is only one example of various screen configurations, and it is obvious that the screen configuration method is not particularly limited. For example, unlike in FIG. 5C, the face image of the person may not be displayed on the display screen. That is, only an image of the candidate may be displayed on the display screen. In addition, the face image of the person may be displayed at a position other than the center of the display screen or on the entire screen. In addition, instead of the face image of the person, the original person image (see reference numeral 10 of FIG. 5A) or a reduced image thereof may be displayed on the display screen together with the images of the candidate person.

In the case of FIG. 5C, although the images of the candidates are arranged in a substantially radial shape around the face image of the person, various modifications are also possible in the method of listing up the images of the candidates. Of course it is possible. For example, the images of candidate candidates may be displayed in a list or displayed in an N ХM array. In addition to the two-dimensional arrangement of the image of the candidate person, it is a matter of course that the three-dimensional arrangement is also possible.

In addition, there is no particular limitation on the sun (size, shape, etc.) in which the image of the candidate person is displayed. In addition, the image of the candidate is not necessarily a person image or a face image of the candidate person, and any image that the user may naturally see or recognize the candidate person by looking at a specific image may be used. For example, this may correspond to a replacement image (a heart image, an image including the text "I love you", etc.) previously registered by the user in association with his / her lover.

In the present specification, the description will be mainly focused on the case where the "image" of the candidate person is displayed, but any information may be displayed if the candidate person can be indicated. For example, the name, nickname, etc. of the candidate person may be displayed in text instead of the image of the candidate person.

In FIG. 5C, six images of the candidate person are displayed, but the number of the candidate person image displayed on the display screen may be more or less than this. Alternatively, if the accuracy of image matching (ie, face matching) in the face matching process described above is high, in some cases, only one image that is determined to be the most similar (ie, most likely to be the same person) may be displayed. have. In this case, step S450 to be described later will perform a function as a kind of confirmation step in which the user again confirms whether the image matching is correct.

Furthermore, if the image matching is very accurate (i.e., if the image matching results in an image that is almost certain to be the same person), it may be possible to skip steps S440 and S450 and go directly to step S460 to be described later. have. That is, in the flowchart of FIG. 4, steps S440 and S450 are inserted in the middle in consideration of an error in face recognition, and may be omitted if face recognition is correct.

As described above, the screen configuration method of FIG. 5C is merely an example and may be variously modified. However, in the present specification, for convenience of explanation / concentration, a screen configuration for displaying an image of a candidate person (a screen configuration for displaying an information item to be described later) is generally the same as in FIG. 5C. Assuming the same or similar screen configuration, the following drawings (see the drawings of FIG. 6 and the following) will be described and described with reference to the drawings.

In this case, the following methods may be used as an array display method of the image of the candidate person. For example, the candidate person's image may be displayed on the display screen in order of accuracy of image matching with the extracted face image. An example is displaying one column from top to bottom in the order of high accuracy on the display screen. Alternatively, when the image of the candidate person is radially displayed on the display screen as shown in (c) of FIG. 5, another example is displaying in the order of clockwise order with the highest accuracy. Or display a three-dimensional arrangement of the candidate's image, wherein the image with the highest accuracy is initially located on the top layer, and according to touch manipulation (eg, flicking or scrolling). Another example may be configured to change the image displayed on the top layer.

In this case, in each image of the candidate person displayed on the display screen, information indicating the accuracy of image matching with the extracted face image may be displayed together (see FIG. 7).

In this case, when there are a plurality of images having similarities to the extracted face images for each candidate person, only one representative image for each candidate person may be displayed on the display screen. Accordingly, for example, a total of six images 21 to 26 displayed on the display screen of FIG. 5C or FIG. 7 may be images of different persons. Of course, it is obvious that a total of six images may be displayed in the order of high accuracy of image matching. In this case, two or more of the six images may be images of the same person.

In the above, with reference to FIG. 5C or FIG. 7, an example of a screen configuration in which an image of a candidate person is displayed in six surrounding areas around the face image 11a has been described.

However, in some cases, there may be a case (hereinafter, the first case) in which there is no image of the same person as the person among the images of the candidate person initially displayed on the display screen. Alternatively, there may be a case in which an image of the same person as the person is not stored in the terminal (hereinafter, the second case).

In consideration of such a case, the display screen displayed in step S440 described above may be configured as shown in FIG. 8 or 9. In FIG. 8, the "Others" item 41 is an example of the first case, and in FIG. 8 or 9, the "New image" item 42 is an example of the second case.

In relation to the first case, for example, in FIG. 8, when the user touch selects the “Others” item 41, the candidate images 21 to 24 initially displayed on the display screen 151a and Other candidate images may be newly displayed on the display screen. Alternatively, as shown in (a) of FIG. 9, any one point (reference numeral 51 in the example of FIG. 9A) of the display screen 151a except for the region where the image of the candidate person is displayed is used as a starting point. Thus, other candidate images may be newly displayed on the display screen through a touch operation such as drawing a semicircle in the direction in which the other point (reference numeral 52 in the example of FIG. 9A) is located. Referring to FIG. 9B, new touch images 21, 27 to 29, which are not displayed in FIG. 9A, are displayed on the display screen 151a by the above touch operation. It is possible to check.

Among the images of the candidate person displayed on the display screen as described above, the user may select any one image determined to be the same as the person of the person image (see reference numeral 10 of FIG. 5A) [S450 and See FIG. 6 (a)].

In this case, the controller 180 displays an information item related to the person of the selected image on the display screen (see S460 and FIG. 6B).

Referring to FIG. 6B, a total of six information items 31 to 36 are radially displayed on the display screen 151a around the position where the face image 11a is displayed. However, this is merely an example, and various modifications may be made to the location of the screen area where the information items are to be displayed, the arrangement of the information items in the screen area, the display format of the information items, and the sorting method. .

In addition, in FIG. 6B, as the information item, the "Phone book" item 31, the "Schedule" item 32, the "Photo album" item 33, the "Message" item 34, " Favorite "item 35 and" My homepage "item 36 are illustrated.

However, when an image of one candidate person is selected by the user in step S450, the information item that may be associated with the selected person, which may be displayed on the display screen through step S460, is determined in the terminal settings. It may be set according to the preset, or may be variously modified or changed according to the user's taste / preference (for example, frequency of use, recent use order, etc.). Of course, it may be added or reconfigured according to the user's selection.

The term information item, as used herein, refers to text, images (eg, icons) that can be represented by aggregating or abstracting the contents, functions, operations, etc. of specific information. ), A thumbnail image, a folder, etc.), or a combination thereof, and the like.

For example, the information item related to the selected person may include a content item related to the selected person, an application item that can be executed in link with the selected person, and an IP accessible to the selected person. It may include at least one of (Internet Protocol) based communication service items.

Here, the content item may be, for example, a text asset (eg, a memo, a diary, a report, a proposal, a homework, a novel, a poem, etc.) directly written by the person, a still image (photo) associated with the person, It may include various multimedia contents such as a video, music that the person prefers or listens to mainly.

Here, the application item is a variety of application programs that can be operated in conjunction with the person, for example, a phone book application (see 31 of FIG. 6 (b)), the schedule management application (FIG. 6 (b) 32), document creation application, image editing application, image archive / management application (see 33 in FIG. 6 (b)), message creation / archive / management application (see 34 in FIG. 6 (b)), phone call An application may correspond to this.

In addition, the IP-based communication service item, for example, personal blog, mini homepage, the company's website of the person, the linked site, the social network service (SNS) (for example, the messenger to which the person is subscribed) , Twitter, myspace, facebook, etc.), and websites frequently accessed by the person. In this case, however, the IP-based communication service item is not necessarily limited to a web site or the like directly related to the person.

For example, a user of the mobile terminal 100 may want to upload a friend's photo or a favorite celebrity's photo to his homepage (see 36 of FIG. 6B), a blog, or the like. . Therefore, the user's own homepage and the like may also be interpreted as an IP-based communication service item that may be indirectly associated with a friend or entertainer.

This may be similarly applied to the above-described content item and application item. For example, a content item that may be related to the person may be content related to the person, but a person associated with the person (eg, a family, a friend, a colleague, a person who can be searched through related search terms, etc.). ) May be content about

Although the flowchart of FIG. 4 illustrates a case in which the user selects any one of the images of the candidate person displayed on the display screen, the description related to the step S450 and the step S460 is described in the second case (that is, the person). Similarly, the same image may be applied to the case where the image of the same person as is not stored in the terminal.

That is, when an image of the same person as the person does not exist in the terminal, the user touch-selects the "New image" item 42 as shown in FIG. 10 (a) or "New" as shown in FIG. The face image 11a may be touched and dragged 55 onto the area where the “image” item 42 is displayed. Even in this case, the controller 180 displays an information item (see 43, 61 to 65 of FIG. 10B) that may be related to the face image 11a as shown in FIG. 10B. Can be displayed on the screen. In this case, the information item displayed on the display screen may be partially (or completely) different from the foregoing case (when any one of candidate images is selected). This is because it may be considered that the information about the person is first registered or stored in the terminal.

Also in the case of FIG. 10B, in consideration of the fact that the person is initially registered, an information item related to image registration / storage (see “Image store” item 61) and an information item related to image editing ( "Photo editor" item (62)), an information item related to the search of related content (see "Contents search" item (63)), and an information item related to a new link link (see "New link" item 43). The case of newly displaying on the display screen 151a is illustrated.

For example, in FIG. 10 (b), when the "Image store" item 61 is touched and selected, the face image 11a (or the original person image 10 may be stored depending on the terminal setting) is stored. Can be. In this case, the location where the face image is stored / classified may be, for example, within a "Photo album" item (for example, reference numeral 33 in FIG. 6B) of the person. Alternatively, in the case of initial registration, a new image storage folder may be created and stored.

In this case, the newly created image archive folder name may be set to be input by the user. If the name, nickname, etc. of the person exists, the image archive folder name may be automatically set to the name, nickname, etc. by the controller 180. This may be equally applied when setting a file name of the image to be stored. For example, if the person's name is "Kimsara", the file name of the image to be stored may be automatically set to "Kimsarah01.jpg".

In addition, in FIG. 10B, when the "Contents search" item 63 is touch-selected by the user, for example, the controller 180 may store the above-mentioned content among the contents previously stored in the terminal 100. Content that may be associated with a person to be registered first may be collected and displayed on a display screen.

Also, as another example, the controller 180 requests an external server to provide related information on the person to be registered first through the wireless communication unit 110, and receives information related to the person from the external server. The received information may be itemized and displayed on a display screen. The external server may be, for example, a broadcast server, a server of a mobile communication company, a content providing server, or the like. In addition to the present case, the same may be applied to other embodiments of the present invention.

As described above, according to an embodiment of receiving information about the person from an external server, without performing a separate external search process, the user may have various information related to the person (for example, an image of the person, Content such as a video, a related article about the person, web site information related to the person, etc.) can be checked in real time.

In addition, the user touches and selects the "New link" item 43 in FIG. 10 (b) to display an application, content, and menu item to be associated with the person to be initially registered in a newly entered screen. You may also choose.

As described above, when any one of the information items displayed on the screen is selected in the state where the display screen of FIG. 6B or 10B is displayed [S470], the controller 180 According to the content (attribute, type, function, etc.) of the selected information item, the corresponding function can be executed immediately or the screen related thereto (for example, a sub menu, etc. is displayed) can be entered [S480]. ]. This will be described with reference to FIGS. 12 to 18 as follows.

For example, referring to FIG. 12, when the user touch selects the “Phone book” item 31 among the information items 31 to 36 displayed on the display screen 151a of the terminal 100 (FIG. 13). (A) of), as shown in (b) of FIG. 12 may enter the phone book edit screen. In this case, the user performs a terminal operation (for example, a touch operation, etc.) in the phone book edit screen to display the face image 11a of the person (or the person's original person image as shown in FIG. 5A), This may be inserted below).

As a method of inserting the face image 11a into the phonebook edit screen, for example, the following two methods can be used. Of course, there may be other methods. In the first method, as described above, after touch-selecting the corresponding information item (that is, the "Phone book" item 31), touch operation or the like is performed again in the phonebook editing screen entered by the touch selection. There may be a method (so-called two-step touch method) for inserting an image into the corresponding screen by performing the same. In the second method, on the display screen itself as in FIG. 12A, a touch such as dragging the face image 11a displayed on the screen to the area where the "Phone book" item 31 is displayed. There may be a method of inserting an image directly into the phonebook editing screen (ie, a so-called one-step touch method, that is, the touch method shown in FIG. 11) through the manipulation. The above-described image insertion method may be equally applied to the case of FIGS. 13 to 15.

As another example, referring to FIG. 13, when the user touch-selects a “Schedule” item 32 among the information items 31 to 36 displayed on the display screen 151a of the terminal 100 (FIG. 13). (A)), the schedule management screen as shown in (b) of FIG. 13 may be entered.

Even in this case, the face image 11a may be inserted into the schedule (see reference numeral 61 in FIG. 13B) using the image insertion method described above. For example, when there is a touch operation according to the above-described one-step touch method, the schedule editing screen (or window) with the face image 11a inserted is displayed, and the user is displayed on the schedule editing screen (or window). By inputting the content related to the schedule, it is possible to perform the schedule registration as shown in (b) of FIG.

In a manner similar to that described above, the user may upload the face image 11a of the person (or may be the original person image 10 of the person as mentioned above) to his homepage or the like. This can be easily confirmed through FIGS. 14 and 15.

In the above, when any one of the information items displayed on the display screen is selected with reference to FIGS. 12 to 15, a method for entering a screen associated with the selected information item or inserting an image into an application associated with the information item is described. The description is based on. However, it is a matter of course that various applications are possible in addition to those described above.

For example, when touch-selecting a "Message" item 34 or inserting an image into the "Message" item 34, the message composition window with the face image 11a or the original person image 10 inserted into the image is inserted. There may also be embodiments in which the message is activated, thereby composing a message and soon sending the message to a specific recipient.

Alternatively, although not specifically illustrated through the drawings, an embodiment may exist such as making a call to the person of the face image by interworking the call function with the face image 11a. It is obvious that there can be numerous applications.

12 to 15, the case in which the extracted face image 11a itself is linked to the related information item or the image is inserted into the related information item has been mainly described. However, the present invention is not limited thereto. That is, in the case of FIGS. 12 to 15, only the embodiments focused on the case of using the person image first displayed on the display screen or the face image itself. However, in other respects, there may be numerous embodiments in which the present invention is used.

For example, while the user is viewing the pre-stored photo images, the user may intend to call or send a message to the person of the specific photo image. Or it may be intended to send a message or an e-mail in which the specific photo image (or the face image of the specific person) is inserted to another person associated with the specific photo image. It may also be intended to upload that particular photographic image to its homepage. Or identify a variety of information related to a person in that particular photographic image (e.g., an appointment with the person in the photographic image, detailed information about the person, other photos or content related to that person or that person) Can be intended.

An example of use from another perspective as described above is illustrated through FIGS. 16-18.

Referring to FIG. 16, when the user touch-selects the “Photo album” item 33 among the information items 31 to 36 displayed on the display screen 151a of the terminal 100 (FIG. 16A). The person images (see 71 to 76 of FIG. 16B) of the same person as the person of the extracted face image 11a may be displayed on the display screen 151a. Here, the person images 71 to 76 may be images that are pre-stored / classified with respect to the person in the terminal 100.

In the display screen as shown in FIG. 16B, the user may select some person images (see 72 to 75 of FIG. 17A) through a predetermined touch operation. In FIG. 17A, some of the person images 72 to 75 are selected through a drag operation of using the reference numeral 57 as the starting point and the ending 58 position as the end point. However, various touch selection methods may exist as well. For example, it is also selectable by a touch operation such as "double-clicking" each of the person images to be selected. Alternatively, the touch screen may be selected by a multi-touch operation in which a touch on another person image is performed using another finger while maintaining a touch on any one person image by one finger.

In this case, the images 72 to 75 selected in the above manner may be visually distinguished from the unselected images 71 and 76 (see FIG. 17B).

In addition, when the above selection operation is completed (for example, a finger is removed from the touch screen after dragging), a list of application items that can be executed using the selected images is displayed through a pop-up window. It may be displayed on the screen (see FIG. 18). Referring to FIG. 18, as application items that are popped up, "MMS transmission", "E-mail transmission", "Photo printing", "Video making", "Image book making", "Image copying", "Image moving", etc. It is illustrated. Of course, the above is just an example, and there may be various application items. In addition, the type, arrangement, configuration, etc. of the application item may vary according to terminal settings or user preferences / preferences.

(B) of FIG. 18 assumes that a "image book production" item is selected among the application items of FIG. 18 (a), and thus an image book produced using the selected person images. To illustrate. In this case, the user may easily upload the content to his or her homepage or a specific website (for example, youtube) using an image book (or a video, etc.) produced in this manner.

As described above, according to the present invention, the user may touch the selected part of the face in the person image of the person who is listening, as described above with reference to FIG. 5 (b), which may be associated with the person. There is an advantage that the information items can be easily accessed. This can greatly reduce the user's effort to access information items associated with a particular person. According to the related art, in order to access an item of information related to a specific person, there is an inconvenience of having to perform a key operation or a touch operation in various steps. In addition, according to the present invention, information items that may be associated with each specific person can be checked at once and easily through one screen, thereby maximizing user convenience.

So far, the method of providing information in a mobile terminal according to an embodiment of the present invention has been described with reference to the flowchart of FIG. 4. Hereinafter, a method of providing information in a mobile terminal according to another embodiment of the present invention will be described with reference to the flowchart of FIG. 19. However, descriptions of contents that may be applied or overlap with the description above will be omitted.

19 is a flowchart illustrating a method of providing information in a mobile terminal according to another embodiment of the present invention.

Referring to the flowchart of FIG. 19, compared with the flowchart of FIG. 4, steps S610, S620, and S630 are the same as those of S410, S420, and S430 of FIG. 4, respectively. That is, when a face image is selected from the person image [S510] while the person image is displayed on the display screen [S520], the controller 180 extracts the face region from the person image. Same as in flowchart. Therefore, a detailed description of the overlapping steps will be omitted, and the description in FIG. 4 will be used as it is.

In addition, although not specifically illustrated in the flowchart of FIG. 19, the same step as that of step S440 or / and step S450 of FIG. 4 may be further included between step S630 and step S640 (see FIG. 20A). Of course, in the flowchart of FIG. 19, a step such as step S440 or step S450 of FIG. 4 does not correspond to an essential step.

In the flowchart of FIG. 19, after the face image 11a of the person is extracted, a person folder (see reference numeral 80 of FIG. 20B) of the extracted face image is generated. [S640].

In generating the person folder, the controller 180 includes at least one information item including information that may be related to the person of the face image as a lower layer of the person folder, thereby providing a kind of hierarchical structure (or data). Classification structure) can be formed [S650]. In this case, when the user touch-selects the person folder whose face image is the representative image (see FIG. 21A), the controller 180 may display information items included in the person folder on the display screen. (See S670 and FIG. 21B).

In the case of FIG. 21B, the "Kimsara-Photo Album" item (or folder, the same as below) 81, the "Kimsara-text message" item 82, "Kimsara-schedule" on the display screen 151a The case where the item 83 is displayed is illustrated. In addition, a homepage, content, an email, a memo, and the like that may be related to the person may be further displayed by different items or folders. At this time, the confirmation of the related item not displayed on one screen may be confirmed through an operation of a direction key (see reference numerals 91 and 92 of FIG. 21) on the touch screen.

In the display screen 151a as shown in FIG. 21B, when any one item of information is selected, the same process as step S480 of FIG. 4 may be performed. This has been described in detail above with reference to FIG. 4, and a detailed description thereof will be omitted.

So far, the mobile terminal and the method of providing information in the mobile terminal according to the present invention have been described mainly based on two embodiments (the embodiment related to the flowcharts of FIGS. 4 and 19). In the description of the present invention, in the above two embodiments, it is assumed that the image displayed on the display screen is a "portrait image", and the description will focus on the case where the "face area" is extracted especially from the "portrait image". It was. However, the present invention is not necessarily limited to the above case.

In the present invention, the portion from which the image is extracted may not necessarily be a face region, and the original image before the image extraction may not necessarily be a person image. For example, the user may want to extract a particular portion of interest from any image displayed on the display screen of the terminal. For example, there may be various objects of interest or regions of interest, such as a face portion of a dog, a favorite cartoon character, a favorite flower, an image portion of a specific object, etc. included in an image.

Therefore, in this case, when the user selects a portion of interest (ie, a region of interest) in the image in the same manner as in FIG. 5 (b) described above, the controller 180 is the same as described above. Alternatively, the region of interest may be extracted from the image displayed on the display screen. Thereafter, the controller 180 may display at least one information item including information that may be related to the extracted image of the ROI on the display screen. Even in this case, the controller 180 may request the external server to provide information that may be related to the extracted image of the ROI.

Although the above has been described with reference to embodiments of the present invention, those skilled in the art may variously modify the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. And can be changed easily.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

2 is a view showing the appearance of a mobile terminal according to an embodiment of the present invention;

3 is a front view of a terminal for explaining an operation state of a mobile terminal according to an embodiment of the present invention.

4 is a flowchart illustrating a method of providing information in a mobile terminal according to an embodiment of the present invention.

5 and 6 are related drawings for explaining the flow chart of FIG.

7 and 8 illustrate one display example and another display example of a candidate person image.

9 is a view for explaining an example of a touch operation for changing a candidate person image.

10 and 11 are diagrams for explaining an example of the touch operation for the "new storage" of the person image.

12 is a diagram illustrating a case where a phonebook application is selected as an application that may be associated with a person image.

FIG. 13 illustrates a case where a schedule management application is selected as an application that may be associated with a person image; FIG.

14 and 15 illustrate a case where a home page connection application is selected as an application that may be associated with a person image.

16 is a diagram illustrating a case where a photo album application is selected as an application that may be associated with a person image.

17 and 18 are diagrams illustrating a case of producing an image book as an example of use of images included in the photo book illustrated in FIG. 16.

19 is a flowchart illustrating a method of providing information in a mobile terminal according to another embodiment of the present invention.

20 and 21 are related diagrams for explaining the flow chart of FIG. 19;

<Description of the symbols for the main parts of the drawings>

100: mobile terminal

110: wireless communication unit

130: input unit

151: display module

180: control unit

181: multimedia module

Claims (20)

display; And A control unit which extracts a face region from the image displayed on the display screen and displays at least one information item representing information that may be associated with a person of the extracted face region on the display screen Mobile terminal comprising a. The method of claim 1, The touch screen is mounted corresponding to the display screen, And the face area to be extracted is selected from the image displayed on the display screen by a touch manipulation on the touch screen. The method of claim 1, The control unit, Through face recognition, an image of a candidate person having similarity with the extracted face area is displayed on the display screen. And when an image of the candidate person is selected, an information item representing information related to the person of the selected image is displayed on the display screen. The method of claim 3, And the controller is arranged to display images of the candidate person in order of accuracy of image matching with the face area. The method of claim 3, The image of the candidate person, the mobile terminal, characterized in that one representative image is displayed for each candidate person having similarity with the face area. The method of claim 3, The controller is configured to display information indicating the accuracy of image matching with the face region in the image of the candidate person. The method of claim 1, The information item, A content item related to the person in the extracted face area, an application item that can be executed by linking with the person in the extracted face area, and an IP (Internet Protocol) based connection that is accessible in connection with the person in the extracted face area. And at least one of the communication service items of the mobile terminal. The method of claim 7, wherein The application item, A mobile terminal comprising at least one of a phonebook application, a schedule management application, a document writing application, an image editing application, a message writing application, and a phone call application. The method of claim 1, The controller may display the at least one information item together with an image of the extracted face region or an entire image including the face region. The method of claim 1, The touch screen is mounted corresponding to the display screen, And the entire image including the extracted face region or the entire image including the face region may be inserted into the information item by a touch manipulation on the touch screen. The method of claim 1, The control unit, And generating a person folder using the extracted face area as a representative image of the person, and including information items related to the person in the extracted face area as a lower layer of the person folder. Mobile terminal. The method of claim 1, Further includes a communication unit, The control unit, Requesting an external server to provide related information about the person in the extracted face region through the communication unit, and when information related to the person is received from the external server, the received information is itemized and displayed on the display screen. Mobile terminal, characterized in that. display; And A control unit which extracts a region of interest from an image displayed on the display screen and displays at least one information item representing information that may be related to the extracted image of the region of interest on the display screen Mobile terminal comprising a. Extracting a face region from an image displayed on a display screen; And Displaying at least one information item representing information related to a person of the extracted face region on the display screen; Information providing method in a mobile terminal comprising a. The method of claim 14, Before extracting the face area, And selecting a face area from the image displayed on the display screen by a touch operation on the touch screen provided in the mobile terminal. The method of claim 14, The displaying of the at least one information item including information that may be related to the person of the extracted face region on the display screen may include: Displaying an image of a candidate person having similarity with the extracted face region through face recognition; Selecting one of the images of the candidate person; And Displaying an information item representing information that may be associated with a person of the selected image Information providing method in a mobile terminal comprising a. The method of claim 14, In the mobile terminal, the image of the extracted face region or the entire image including the face region may be inserted into the information item by a touch manipulation on a touch screen provided in the mobile terminal. How to Provide Information. The method of claim 14, Generating a person folder having the extracted face region image as a representative image of the person; And Including an information item related to the person of the extracted face region as a lower layer of the person folder Information providing method in a mobile terminal further comprising. The method of claim 14, Requesting an external server to provide related information about the person of the extracted face region; Receiving information related to the person from the external server; And And itemizing the received information and displaying the received information on the display screen. Extracting a region of interest from an image displayed on a display screen; And Displaying at least one information item representing information that may be associated with the extracted image of the ROI on the display screen; Information providing method in a mobile terminal comprising a.

Images