KR20110119127A - Mobile terminal and operation method thereof - Google Patents

Abstract

PURPOSE: A portable terminal and operation method thereof are provided to supply a service by utilizing information about an image or an individual in a preview image. CONSTITUTION: A touch screen displays a preview image of a camera(121). A controller(180) displays an individual image corresponding to the individual. The individual image is an image for cropping the preview image. The individual related service displays individual information on one area of the display unit.

Description

Mobile terminal and operation method thereof

The present invention relates to a portable terminal and a method of operating the same. More particularly, the present invention relates to a portable terminal and a method of operating the same, which provide a service by interworking information and spatial information in which the information is generated.

A portable terminal is a portable device that is portable and has one or more functions for making voice and video calls, inputting and outputting information, and storing data. As the functions of the portable terminals are diversified, the portable terminal can provide complex functions such as photographing a picture or a video, playing a music file or a video file, receiving a game, broadcasting, or wireless internet.

Recently, mobile terminals have been widely used to capture video or still images through a camera. In addition, a portable terminal has been developed to provide a real world image and additional information, in particular, the information related to the real world image seen through the camera. Such technology is called augmented reality technology.

Augmented reality refers to a reality in which additional information is inserted into a real environment, where real world images and information are mixed. Such augmented reality allows a user to more easily obtain information related to the real world.

In such a situation, various methods for providing more useful information to the user through augmented reality have been devised.

Accordingly, an object of the present invention, as a technology for implementing augmented reality, provides a mobile terminal and a method of operating the same by providing a service using information on the image of the person on the preview image or information about the person.

According to an aspect of the present invention, there is provided a method of operating a portable terminal, the method comprising: displaying a preview image of a camera on a display unit; When a person in the preview image is selected, enlarging and displaying a person image corresponding to the person; And providing a service related to the person by using person information corresponding to the person.

On the other hand, another operation method of a mobile terminal according to the present invention for achieving the above object, the step of displaying a preview image of the camera on the display unit; If a person image display command is input, displaying a person image pre-stored in a person database on an area of the display unit; And generating and storing a still image in which the preview image and the person image are combined when a photographing command is input.

On the other hand, another portable terminal according to the present invention for achieving the above object, the camera; A touch screen displaying a preview image of the camera; And a controller configured to enlarge and display a person image corresponding to the person when the person in the preview image is selected.

According to at least one of the embodiments of the present invention, it is possible to provide the user with various information or services related to the person in the preview image displayed when the image is taken.

1 is a block diagram of a portable terminal according to an embodiment of the present invention;
2 is a perspective view of a portable terminal according to an embodiment of the present invention;
3 is a rear perspective view of the mobile terminal shown in FIG. 2;
4 is a flowchart illustrating a method of providing information about a person in a preview image according to an embodiment of the present invention;
5 is a flowchart illustrating a method of adding a person image to a preview image according to an embodiment of the present invention;
6 is a flowchart illustrating a method of uploading a preview image according to an embodiment of the present invention;
7 is a reference diagram illustrating a process of converting from a shooting mode to a portrait mode of a mobile terminal according to an embodiment of the present invention;
8 is a reference diagram illustrating a process of displaying person information in a person mode according to an embodiment of the present invention;
9 is a reference diagram illustrating a process of storing a person image in a person mode according to an embodiment of the present invention;
10 is a reference diagram illustrating a process of storing a person image in a person mode according to another embodiment of the present invention;
11 is a reference diagram illustrating a process of adding a person image in a person mode according to an embodiment of the present invention;
12 is a reference view illustrating a process of uploading a preview image in a person mode according to an embodiment of the present invention;
FIG. 13 is a reference view illustrating a process of attempting a call with a specific person in a person mode according to an embodiment of the present invention; and
14 is a reference diagram illustrating a process of creating a message to be transmitted to a specific person in a person mode according to an embodiment of the present invention.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The portable terminal described in the present specification includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like. In addition, suffixes "module" and " part "for the components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Therefore, the "module" and "unit" may be used interchangeably.

1 is a block diagram of a portable terminal according to an embodiment of the present invention. Referring to FIG. 1, a mobile terminal according to an embodiment of the present invention will be described in terms of components according to functions.

1, the portable terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A memory 160, an interface unit 170, a control unit 180, and a power supply unit 190. Such components may be configured by combining two or more components into one component, or by dividing one or more components into two or more components as necessary when implemented in an actual application.

The wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 113, a wireless Internet module 115, a short distance communication module 117, and a GPS module 119.

The broadcast receiving module 111 receives at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like. The broadcast management server may mean a server for generating and transmitting at least one of a broadcast signal and broadcast related information, or a server receiving at least one of pre-generated broadcast signals and broadcast related information and transmitting the same to a terminal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal. The broadcast related information may also be provided through a mobile communication network, and in this case, may be received by the mobile communication module 113. Broadcast related information may exist in various forms.

The broadcast receiving module 111 may receive a broadcast signal using various broadcast systems, and may receive a digital broadcast signal using a digital broadcast system. In addition, the broadcast receiving module 111 may be configured to be suitable for all broadcast systems providing broadcast signals as well as such digital broadcast systems. Broadcast signals or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 113 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to voice call signal, video call signal, or text / multimedia message transmission and reception.

The wireless Internet module 115 refers to a module for wireless Internet access, and the wireless Internet module 115 can be embedded in the mobile terminal 100 or externally. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short-range communication module 117 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

A GPS (Global Position System) module 119 receives position information from a plurality of GPS satellites.

The A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 123. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 123 receives an external sound signal by a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 113 and output when the voice data is in the call mode. The microphone 123 may use various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates key input data input by the user for controlling the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (constant voltage / capacitance), etc. that may receive a command or information by a user's push or touch operation. In addition, the user input unit 130 may be configured as a jog wheel or a jog method or a joystick that rotates a key, a finger mouse, or the like. Particularly, when the touch pad has a mutual layer structure with the display unit 151 described later, it can be called a touch screen.

The sensing unit 140 senses the current state of the portable terminal 100 such as the open / close state of the portable terminal 100, the position of the portable terminal 100, Thereby generating a sensing signal. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like.

The sensing unit 140 may include a proximity sensor 141, a pressure sensor 143, a motion sensor 145, and the like. The proximity sensor 141 may detect the presence or absence of an object approaching the portable terminal 100 or an object present in the vicinity of the portable terminal 100 without mechanical contact. The proximity sensor 141 may detect a proximity object by using a change in an alternating magnetic field or a change in a static magnetic field, or by using a change rate of capacitance. Two or more proximity sensors 141 may be provided according to the configuration aspect.

The pressure sensor 143 may detect whether pressure is applied to the portable terminal 100, the magnitude of the pressure, and the like. The pressure sensor 143 may be installed at a portion where the pressure of the portable terminal 100 is required depending on the use environment. If the pressure sensor 143 is installed in the display unit 151, a touch input through the display unit 151 and a greater pressure than the touch input are applied according to a signal output from the pressure sensor 143. The pressure touch input can be identified. Also, the magnitude of the pressure applied to the display unit 151 at the time of the pressure touch input can be determined according to the signal output from the pressure sensor 143. [

The motion sensor 145 detects the position and movement of the portable terminal 100 using an acceleration sensor, a gyro sensor, or the like. An acceleration sensor that can be used for the motion sensor 145 is a device that converts an acceleration change in one direction into an electric signal and is widely used along with the development of MEMS (micro-electromechanical systems) technology. The acceleration sensor measures the acceleration of a small value built in the airbag system of an automobile and recognizes the minute motion of the human hand and measures the acceleration of a large value used as an input means such as a game There are various types. Acceleration sensors are usually constructed by mounting two or three axes in one package. Depending on the usage environment, only one axis of Z axis is required. Therefore, when the acceleration sensor in the X-axis direction or the Y-axis direction is used instead of the Z-axis direction for some reason, the acceleration sensor may be mounted on the main substrate by using a separate piece substrate.

The gyro sensor is a sensor for measuring the angular velocity, and it can sense the direction of rotation with respect to the reference direction.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, an audio output module 153, an alarm unit 155, and a haptic module 157.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the portable terminal 100 is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the portable terminal 100 is in the video communication mode or the photographing mode, the photographed or received images can be displayed individually or simultaneously, and the UI and the GUI are displayed.

Meanwhile, as described above, when the display unit 151 and the touch pad form a mutual layer structure and constitute a touch screen, the display unit 151 is an input device capable of inputting information by a user's touch in addition to the output device. May also be used.

If the display unit 151 is configured as a touch screen, it may include a touch screen panel, a touch screen panel controller, and the like. In this case, the touch screen panel is a transparent panel attached to the outside and may be connected to the internal bus of the portable terminal 100. The touch screen panel keeps a watch on the contact result, and if there is a touch input, sends the corresponding signals to the touch screen panel controller. The touch screen panel controller processes the signals, and then transmits corresponding data to the controller 180 so that the controller 180 can determine whether the touch input has been made and which area of the touch screen has been touched.

The display unit 151 may be formed of an e-paper. Electronic paper (e-Paper) is a kind of reflective display, and has excellent visual characteristics such as high resolution, wide viewing angle and bright white background as conventional paper and ink. The electronic paper (e-paper) can be implemented on any substrate such as plastic, metal, paper, and the image is retained even after the power is shut off, and the battery life of the portable terminal 100 is long Can be maintained. As the electronic paper, a hemispherical twist ball filled with a telephone can be used, or an electrophoresis method and a microcapsule can be used.

In addition, the display unit 151 may include a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display. It may include at least one of (3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, the external display unit (not shown) and the internal display unit (not shown) may be simultaneously provided in the portable terminal 100.

The audio output module 153 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 153 outputs sound signals related to functions performed in the portable terminal 100, for example, a call signal reception tone, a message reception tone, and the like. The sound output module 153 may include a speaker, a buzzer, and the like.

The alarm unit 155 outputs a signal for notifying the occurrence of an event of the portable terminal 100. Examples of events occurring in the portable terminal 100 include call signal reception, message reception, and key signal input. The alarm unit 155 outputs a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. The alarm unit 155 can output a signal to notify when a call signal is received or a message is received. Also. When the key signal is input, the alarm unit 155 may output the signal as a feedback to the key signal input. The user may recognize the occurrence of an event through the signal output from the alarm unit 155. The signal for notifying the event occurrence in the mobile terminal 100 may also be output through the display unit 151 or the sound output module 153.

The haptic module 157 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 157 is a vibration effect. When the haptic module 157 generates vibration with a haptic effect, the intensity and pattern of the vibration generated by the haptic module 157 can be converted, and the different vibrations can be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 157 may be provided with a function of stimulating by a pin arrangement vertically moving with respect to the contact skin surface, an effect of stimulating air through the injection or suction force of the air through the injection port or the suction port, The effect of stimulation through contact of an electrode (eletrode), the effect of stimulation by electrostatic force, and the effect of reproducing a cold sensation using a device capable of endothermic or exothermic can be generated. The haptic module 157 can be implemented not only to transmit the tactile effect through direct contact but also to feel the tactile effect through the muscular sense of the user's finger or arm. Two or more haptic modules 157 may be provided according to a configuration aspect of the mobile terminal 100.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input or output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM , And a ROM. ≪ / RTI > In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the portable terminal 100. Examples of the external device connected to the portable terminal 100 include a wired / wireless headset, an external charger, a wired / wireless data port, a memory card, a SIM (Subscriber Identification Module) card, a UIM An audio input / output (I / O) terminal, a video I / O (input / output) terminal, and an earphone. The interface unit 170 receives data from the external device or receives power from the external device and transmits the data to each component in the portable terminal 100 so that data in the portable terminal 100 can be transmitted to the external device .

The interface unit 170 is a path through which power from the cradle connected when the portable terminal 100 is connected to the external cradle is supplied to the portable terminal 100 or various command signals inputted from the cradle by the user are carried And may be a passage to be transmitted to the terminal 100.

The controller 180 typically controls the operation of the respective units to control the overall operation of the mobile terminal 100. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia playback module 181 for multimedia playback. The multimedia playback module 181 may be configured in hardware in the controller 180 or may be configured in software separately from the controller 180.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The mobile terminal 100 having such a configuration can be configured to be operable in a communication system capable of transmitting data through a frame or a packet, including a wired / wireless communication system and a satellite-based communication system. have.

The portable terminal related to the present invention has been described above in terms of components according to functions. Hereinafter, referring to FIGS. 2 and 3, the portable terminal related to the present invention will be further described in terms of components according to appearance. Hereinafter, for convenience of description, a bar type portable terminal having a front touch screen among various types of portable terminals such as a folder type, a bar type, a swing type, a slider type, and the like will be described as an example. However, the present invention is not limited to a bar-type portable terminal, but can be applied to all types of portable terminals including the above-mentioned type.

2 is a perspective view of a portable terminal according to an embodiment of the present invention viewed from the front. Referring to FIG. 2, the case forming the exterior of the portable terminal 100 is formed by the front case 100A-1 and the rear case 100A-2. Various electronic components are embedded in the space formed by the front case 100A-1 and the rear case 100A-2. At least one intermediate case may be further disposed between the front case 100A-1 and the rear case 100A-2. Such cases may be formed by injecting synthetic resin, or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the first sound output module 153a, the first camera 121a, and the first user input unit 130a may be disposed in the main body, specifically, the first front case 100A-1. In addition, a second user input unit 130b, a third user input unit 130, and a microphone 123 may be disposed on the side surface of the rear case 100A-2.

The display unit 151 includes a liquid crystal display (LCD), organic light emitting diodes (OLED), and the like, which visually express information. Since the touch pad is superimposed on the display unit 151 in a layer structure, the display unit 151 may operate as a touch screen, and may be configured to input information by a user's touch.

The first acoustic output module 153a may be implemented in the form of a receiver or a speaker. The first camera 121a may be implemented to be suitable for capturing an image or a video of a user or the like. The microphone 123 may be implemented in a form suitable for receiving a user's voice, other sounds, and the like.

The first to third user input units 130a, 130b, and 130c may be collectively referred to as the user input unit 130, and may be employed in any manner as long as the user is operating in a tactile manner.

For example, the user input unit 130 may be implemented as a dome switch or a touch pad that may receive a command or information by a user's push or touch operation, or may be operated as a wheel or jog or a joystick that rotates a key. Or the like. In the functional aspect, the first user input unit 130a is for inputting a command such as a start, end, call, etc., and the second user input unit 130b is for inputting a selection of an operation mode. In addition, the third user input unit 130c may operate as a hot-key for activating a special function in the portable terminal 100.

When the user's finger approaches the display unit 151, the proximity sensor 141 installed on the first body detects this and outputs a proximity signal. At this time, it may be configured to output different proximity signals according to the distance the user's finger is located. In general, the distance between the detection object approaching the proximity sensor and outputting the proximity signal is called the detection distance. When the proximity signal output from each proximity sensor is compared using a plurality of proximity sensors having different detection distances, the detection object is detected. You can see how close is.

In addition, a plurality of proximity sensors having different detection areas are arranged to determine which proximity sensor is output from among the proximity sensors to determine which area of the display unit 151 the proximity object approaches, and the detection object. May be determined to move closer to the display unit 151. Accordingly, the controller 180 may control to output a corresponding vibration signal by selecting a touch key corresponding to a close position of a user's finger or the like.

When the user tilts or shakes the portable terminal, the gyro sensor 142 detects the movement of the portable terminal. The gyro sensor 142 generates a signal corresponding to the movement of the portable terminal and outputs the signal to the controller 180. The controller 180 detects information related to the movement, such as the direction, angle, speed, intensity, and current position of the portable terminal, from the signal generated by the gyro sensor 142.

The controller 180 tracks the movement of the portable terminal through the information detected from the signal generated by the gyro sensor 142. Depending on the components constituting the gyro sensor 142 may be different information that can be detected from the signal generated by the gyro sensor. The portable terminal is provided with a gyro sensor 142 according to the motion information of the portable terminal to be detected. At least one gyro sensor 142 may be provided in the portable terminal. The controller 180 may control the gyro sensor 142 to operate only when executing a specific application in response to the information to be detected.

3 is a rear perspective view of the portable terminal shown in FIG. 2. Referring to FIG. 3, a fourth user input unit 130d, a fifth user input unit 130d, and an interface unit 170 may be disposed on the side of the rear case 100A-2, and the rear case 100A-2 may be disposed. The second camera 121b may be additionally mounted on the rear side of the).

The second camera 121b may have a photographing direction substantially opposite to that of the first camera 121a, and may have different pixels from the first camera 121a. For example, in the case of a video call or the like, the first camera 121a has a low pixel so that the user's face is photographed and transmitted to the other party, and the second camera 121b photographs a general subject and does not transmit it immediately. In many cases, it is desirable to have a high pixel.

The mirror 125 and the flash 126 may be further disposed adjacent to the second camera 121b. The mirror 125 allows the user to see his / her own face or the like when the user wants to photograph (self-photograph) the second camera 121b. The flash 126 illuminates the subject when the subject is photographed by the second camera 121b.

A second sound output module (not shown) may be further disposed on the rear case 100A-2. The second sound output module may implement the stereo function together with the first sound output module 153a, and may be used for talking in the speakerphone mode.

In addition, an antenna (not shown) for receiving a broadcast signal may be disposed on one side of the second rear case 100A-2 in addition to an antenna for a call. The antenna may be installed to be pulled out of the rear case 100A-2.

The interface unit 170 serves as a path for allowing the portable terminal 100 to exchange data with an external device. For example, the interface unit 170 may be at least one of a wired or wireless connection terminal for connecting to the earphone, a port for short-range communication, or a power supply terminal for supplying power to the mobile terminal 100. have. The interface unit 170 may be a card socket that accommodates an external card such as a subscriber identification module (SIM) or a user identity module (UIM) and a memory card for storing information.

The rear case 100A-2 is provided with a power supply unit 190 for supplying power to the portable terminal. The power supply unit 190 is, for example, a rechargeable battery, and may be detachably coupled to the rear case 100A-2 for charging.

In the above description, the second camera 121b and the like are disposed on the rear case 100A-2, but the present invention is not limited thereto. In addition, even if the second camera 12b is not separately provided, the first camera 121a may be rotatably formed to be able to shoot up to the photographing direction of the second camera 121b.

The following embodiments may be more easily implemented when the display unit 151 is configured as a touch screen. Hereinafter, the display unit 151 is assumed to be a touch screen. Reference numeral 400 of the touch screen is used.

4 to 6 are flowcharts illustrating a service provided in a person mode of a portable terminal according to an embodiment of the present invention.

Specifically, FIG. 4 is a flowchart illustrating a method of providing information about a person in a preview image according to an embodiment of the present invention.

As shown in FIG. 4, the preview image of the camera is displayed on the touch screen 400 of the portable terminal (S410). Specifically, when the user selects the shooting mode through a manual of the portable terminal, the touch screen 400 is a preview image for previewing the image to be taken by the camera (hereinafter referred to as a 'camera preview image' or 'preview image' ) Is displayed.

In response to the user command, the controller 180 sets the mode of the portable terminal to the person mode (S420). The user may input a command to change the mode of the portable terminal from the shooting mode to the portrait mode when the user wants to implement augmented reality that provides additional information about the person in the preview image while viewing the preview image. The person mode may be implemented by selecting a person icon displayed on the touch screen 400. When set to the person mode, the portable terminal can provide a service related to the person.

In the person mode, the controller 180 determines whether a specific person is selected (S430). The user may input a command for selecting a specific person by touching an area where a specific person is displayed or dragging an outline of the specific person.

Then, the controller 180 generates a person image corresponding to the selected specific person (S440). In detail, the controller 180 identifies an area in which the person is displayed in the preview image and generates a person image in which an image of an area other than the area in which the person is displayed is cropped. When a specific person is selected by dragging the outline of the specific person, the controller 180 identifies an area where the person is displayed based on the degree of change of the pixel value in the predetermined area including the dragged area. Alternatively, when the user touches an area where a specific person is displayed, whether or not a face exists in a predetermined area based on the touched area, that is, face recognition is performed and the area where the person is displayed is identified through color data. In addition, there are various methods of identifying the area in which the person is displayed, but the description is omitted for simplicity of the specification.

The controller 180 enlarges the person image and displays the enlarged person image in one region of the display unit 151 (S450). The enlarged person image is preferably displayed in an area where a person is displayed and an adjacent area of the display area of the display unit 151.

In this embodiment, although the person image is generated and enlarged as the cropped image, the present invention is not limited thereto. In the preview image, it is possible to distinguish between a person image representing a person and a background image representing a background, delete a background image adjacent to the person image, and enlarge the person image to the deleted area to display an enlarged person image. have. By expanding and displaying the person image of the selected specific person as described above, the user can identify the selected person.

The controller 180 determines whether a person information display command is input (S460). The touch screen 400 may display a service provided in the person mode as an icon. When the user wants to receive the person information, the user may input a person information display command by selecting an information icon displayed on the touch screen 400.

Then, the controller 180 performs a person recognition centering on a face among the selected person images (S465). In detail, the controller 180 extracts feature points such as eyes, nose, and mouth from the face of the selected person image. The extracted feature point is used to calculate the similarity between the face of the selected person image and the face of the person image stored in the person database, and based on the calculated similarity value, whether the person in the preview image is identical to a specific person included in the person database. Recognize figures as

The person database is preferably stored in the memory 160 of the portable terminal. However, when the person database is not pre-stored in the memory 160 of the portable terminal, the controller 180 can use the person database of the external device by connecting to an external device of the person database through the wireless communication unit 110. . In the above-mentioned person database, the person image and the person information may be matched and stored.

Here, the person information is information that characterizes the person, and preferably includes the person's name, telephone number, SNS (Social Network Site) site information, and the like. In particular, the phonebook database built into the mobile terminal may be used as a person database.

The controller 180 obtains person information of the recognized person from the person database (S470), and displays the obtained person information in one region of the preview image (S475). In displaying the person information, it is preferable to display the person information in an area adjacent to the area where the person is displayed, and may also display a notification indicator indicating that the person information of the specific person is displayed.

It is determined whether a storage command of a person image corresponding to a specific person in the preview image is input. The user may input a person image saving command by selecting an area where a specific person is displayed in the preview image and then selecting an area where a storage icon is displayed.

When it is determined that a person image storage command is input, the controller 180 generates a still image corresponding to the separated person image and stores the still image in the person database or the still image file.

In particular, when storing the person image in the person database, the controller 180, if a person image (hereinafter referred to as an existing person image) representing the same person as the person of the person image to be stored is already stored in the person database, the controller ( 180 may store the person image in the person database by updating the person image, that is, deleting the existing person image and storing the current person image in the person database, or adding the person image to the person image. In this case, the controller 180 may generate notification information indicating that a person image of the same person is pre-stored in the person database and display the same on the touch screen 400. If a person image of the same person is not stored in the person database, the controller 180 displays a screen corresponding to the person database. The screen includes an area in which a person image is displayed and an area in which person information can be input.

5 is a flowchart illustrating a method of adding a person image to a preview image according to an embodiment of the present invention.

As shown in FIG. 5, the controller 180 determines whether a person image addition command is input (S510). When a user wants to add a specific person to the preview image, the user may input a person image adding command by selecting an additional icon.

If it is determined that a person image adding command is input (S510-Y), the controller 180 reads out at least one person image (hereinafter, referred to as a candidate person image) to be added to a person database and is stored on a separate screen. Alternatively, the candidate person image is displayed on one region of the touch screen 400 in which the preview image is displayed (S520). When the candidate person image cannot be displayed on one screen, the controller 180 displays some of the read candidate person images on the touch screen 400 and scrolls the notification information indicating that there are more candidate person images that are not displayed. Alternatively, the display may be displayed using a progress bar or an arrow.

When a command for selecting an added person image from among candidate person images is input through the user input unit (S530-Y), the controller 180 displays the selected person image in the form of a pop-up window in one region of the preview image (S540).

When a drag is input after the multi-touch (S550-Y), the controller 180 changes the size of the person image in response to the multi-touch and displays the drag (S560). In detail, when the user touches an area where the person image is displayed with at least two fingers, at least two touch points are generated. When the user drags the touch points in a direction away from each other, the controller 180 enlarges and displays the size of the person image in response to the dragged distance. If the touch points are dragged in a direction in which the touch points are close to each other after being multi-touched, the controller 180 reduces the size of the person image in response to the dragged distance.

In addition, when a drag is input after a single touch (S570-Y), the controller 180 changes and displays the position of the person image in response to the drag after the single touch (S580). In detail, when one finger touches an area where a specific person image is displayed, one touch point is generated. When the user ends the touch after dragging the touch point, the controller 180 moves and displays the person image to the area where the touch is terminated.

When a photographing command is input (S590), the controller 180 generates one still image in which the preview image and the person image are combined and stores the same in the memory 160 (S595). Preferably, the memory is a person database.

6 is a flowchart illustrating a method of uploading a preview image according to an embodiment of the present invention.

The controller 180 determines whether an image upload command is input (S610). In detail, the user may input an image upload command by selecting an upload icon displayed on the touch screen 400.

In operation S620, a still image corresponding to the preview image is generated. That is, the resolution of the preview image is changed to the resolution of the still image, and the still image is generated by adjusting the preview image according to the changed resolution.

In addition, the controller 180 performs person recognition on the selected person image (S630). In detail, the controller 180 detects a face of the selected person image and extracts feature points such as eyes, nose, and mouth from the detected face. The extracted feature points are used to calculate the similarity between the faces of the selected person image and the faces stored in the person database, and based on the calculated similarity values, the person in the preview image is identified with the same person as the specific person included in the person database. Recognize.

The controller 180 obtains information on a social network service (SNS) site to which a person selected from the person database subscribes based on the person recognition result (S640). The information on the SNS site is preferably an address for the SNS site.

The order of S620, which is a process of generating still images, and S630 and S640, which are processes of acquiring SNS site information, may be changed.

The controller 180 accesses the SNS site using the SNS site information, and uploads a still image to the SNS site (S650). In addition, when uploading the preview image, the controller 180 may display a web page corresponding to the SNS site on the touch screen 400 by driving a web browser and display an image upload process. Here, SNS is a site that enables a social network, and includes a blog, cyworld, youtube, twitter and the like.

In the present embodiment, the preview image is uploaded to the SNS site of the selected person, but the present invention is not limited thereto. By selecting a person in the preview image, you can talk to the person or send a message.

Hereinafter, an operation method of the mobile terminal in the person mode will be described in detail with reference to the accompanying drawings.

7 is a reference diagram illustrating a process of converting from a shooting mode to a person mode in a portable terminal according to an embodiment of the present invention.

As shown in FIG. 7A, menu icons together with a preview image may be displayed on the touch screen 400 in a shooting mode. When the user selects the 'person' icon 710 to convert the mode of the portable terminal to the person mode, the controller 180 converts the mode of the mobile device to the person mode and is shown in FIG. As described above, icons 721, 722, 723, and 724 corresponding to a menu executable in the person mode are displayed on one region of the touch screen 400. When the icons cannot be displayed on one screen, the first arrow icon 725 may also be displayed. When the first arrow icon 725 is selected, the previously displayed icons 721, 722, 723, and 724 disappear. The remaining icons 727 and 728 are displayed as shown in (c) of FIG. 7. As shown in FIG. 7C, when the second arrow icon 726 is displayed and the second arrow icon 726 is selected, the icons 721 and 722 as shown in FIG. 7B are displayed. , 723, 724 are indicated.

8 is a reference diagram illustrating a process of displaying person information in a person mode according to an embodiment of the present invention.

As shown in (a) of FIG. 8, when the user touches an area where a person is displayed to receive the person information of the person displayed on the preview image, the controller 180 identifies the person image of the selected person, and FIG. 8. As shown in (b) of FIG. 1, an enlarged person image 810 is displayed. When the user selects the information icon 721 to check the person information on the selected person, the controller 180 performs person recognition on the person image and then acquires person information on the person selected from the person database. As shown in FIG. 8C, the person information 820 is displayed.

9 is a reference diagram illustrating a process of storing a person image in a person mode according to an embodiment of the present invention. In the present invention, a case of storing a person image for convenience of description in a phone book database of a mobile terminal will be described as an example.

As shown in FIG. 9A, when a specific person in the preview image is selected, the enlarged person image 910 is displayed on the touch screen 400. When the storage icon 722 is selected, the controller 180 determines whether a person image representing the selected person is stored in the phone book database. If a person image of the same person is stored in the phonebook database, the controller 180 pops up a first inquiry message 920 that asks whether to add or replace the person image to the phonebook database. As shown, display. When the user selects the first indicator 922 for adding person information, as shown in FIG. 9C, the person image 910 is displayed together with the person image 930 previously stored in the address book screen corresponding to the phonebook database. Will be displayed. The selected person image 910, the pre-stored person image 930, and the person information 940 are matched and stored in the phone book.

However, if the second indicator 922 is selected to replace person information, as shown in FIG. 9D, the person image previously stored in the address book screen is deleted and only the selected person image 910 is displayed. The selected person image 910 and person information 940 are matched and stored in the phonebook database.

10 is a reference diagram illustrating a process of storing a person image in a person mode according to another embodiment of the present invention.

As shown in FIG. 10A, when a specific person in the preview image is selected, the enlarged person image 1010 is displayed on the touch screen 400. When the storage icon 722 is selected, the controller 180 determines whether a person image representing the selected person is stored in the phone book database. If the selected person image 1010 is not stored in the phonebook database, as shown in FIG. 10B, the controller 180 may store the person image by matching the stored person information (add to the existing person information). 2) or a second inquiry message 1020 inquiring whether to store the person image together with the person information input (adding the person information) is displayed on the touch screen 400 as a pop-up window.

When the user selects the third indicator 1022 to add to the existing person information, the previously stored person information list 1030 is displayed as shown in FIG. 10C. The person information list 1030 may be configured with a name of a person. A person image may be added to the phone book database by selecting person information to be added from the person information list 1030.

In addition, when the user selects the fourth indicator 1024 that also adds person information, a new address book screen 1040 including the selected person image is displayed, and a person image field 1042 of the new address book screen 1040 is displayed. Image 1010 is displayed. The person image may be added to the phone book database by writing the person information in the person information field 1040.

11 is a reference diagram illustrating a process of adding a person image in a person mode according to an embodiment of the present invention.

In the preview image of the camera as shown in (a) of FIG. 11, when the user touches a region where the additional icon 723 is displayed, candidate person images 1110 to be added as shown in (b) of FIG. 11. ) Is displayed on one area of the touch screen 400. The user selects a person image to be added by touching an area in which the person image 1112 to be added is displayed among the candidate person images 1110. Then, as illustrated in FIG. 11C, the selected person image 1114 is superimposed on the preview image. The user may change the position of the person image by dragging after a single touch (see FIG. 11D). In addition, the size of the person image may be changed by multi-touching and dragging (see FIG. 11 (d)). In addition, when the preview image is to be stored together with the added person image, the photographing icon of FIG. If 1120 is selected, one still image combining the person image and the preview image is generated and stored.

12 is a reference diagram illustrating a process of uploading a preview image in a person mode according to an embodiment of the present invention.

As shown in FIG. 12A, when a person is selected, the enlarged person image 1210 is displayed on the touch screen. When the upload icon 724 is selected, the controller 180 determines whether the SNS site address of the selected person image is stored in the person database. If the SNS site address of the person image is stored in the person database, the controller 180 generates a still image from the preview image and uploads a still image corresponding to the preview image to the SNS site of the selected person. A message 1220 is generated and displayed as shown in (b) of FIG. 12. When the upload is completed, as shown in FIG. 12C, a second notification message 1230 indicating that the upload is completed is generated and displayed.

FIG. 13 is a reference diagram illustrating a process of attempting a call with a specific person in a person mode according to an embodiment of the present invention. FIG.

As shown in FIG. 13A, when a specific person is selected in the preview image, the enlarged person image is displayed on the touch screen 400. When the first arrow icon 725 is selected, the displayed icons 721, 722, 723, and 724 disappear, and the undisplayed icons 727 and 728 are displayed as shown in FIG. 13B. do. When the user touches the call icon 727, the controller 180 acquires the phone number of the person selected from the person database and attempts to call a specific person selected through the phone number, as shown in FIG. Likewise, the call waiting screen 1320 is displayed on the touch screen 400 as a popup window.

14 is a reference diagram illustrating a process of creating a message to be transmitted to a specific person in a person mode according to an embodiment of the present invention.

As shown in FIG. 14A, when a specific person is touched in the preview image, the enlarged person image 1410 is displayed on the touch screen 400. When the user touches the message icon 728, the controller 180 obtains the number of the mobile terminal of the specific person from the person database, and as shown in FIG. 14B, the number 1422 of the mobile terminal is displayed. Display the included message sending window 1420. Thus, the user can send a message to a specific person only by writing a message without having to search for the number of the mobile terminal of the specific person.

In the present embodiment, the menu executed in the person mode of the portable terminal may display person information of the selected person image, store the person image, add another person image, upload a preview image to the SNS site of the selected person, attempt a call to the selected person, and select the person. Although there is a message transmission, but this is only an embodiment, there may be a menu associated with the person.

In addition, although the controller 180 may display all menus that can be provided regardless of the person in the person mode, the controller 180 may display a menu that can be provided according to the person information of the selected person. In this case, when the user selects a person in the preview image, the controller 180 may activate and display a menu that can be provided based on the person information of the selected person as a ratio. For example, when the user does not select a specific person in the person mode, only an additional icon is displayed on one region of the touch screen, and when the user touches an area where the specific person is displayed, the controller 180 recognizes the specific person. To obtain the person information. If the person information of a particular person includes only a name and a phone number, the controller 180 can display the person information, store the person's image, call, and transmit a message, so that only the icon corresponding to the service is provided on one region of the touch screen. Mark on.

Meanwhile, the present invention can be embodied as processor readable codes on a processor readable recording medium provided in a mobile terminal such as a mobile station modem (MSM). The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may be implemented in the form of a carrier wave such as transmission through the Internet. The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

110: wireless communication unit
120: A / V input section
130: user input unit
140: sensing unit
150: output unit
151: display unit
160: memory
170: interface unit
180:

Claims (12)

Displaying a preview image of a camera on a display unit;
When a person in the preview image is selected, enlarging and displaying a person image corresponding to the person; And
And providing a service related to the person by using the person information corresponding to the person. The method of claim 1,
The portrait image,
And an image obtained by cropping an image of an area excluding the area where the person is displayed from the preview image. The method of claim 1,
The service related to the person,
And displaying the person information on one area of the display unit. The method of claim 1,
The service related to the person,
And matching the person image with the person information and storing the person image in a person database. The method of claim 1,
The service related to the person,
And uploading the preview image to an SNS (Social Network Service) site to which the person subscribes. The method of claim 1,
The person information,
And at least one of a name, an address, a contact, and an SNS site address of the person. The method of claim 1,
And the person information is included in a phone book database of the mobile terminal. Displaying a preview image of a camera on a display unit;
If a person image display command is input, displaying a person image pre-stored in a person database on an area of the display unit; And
And generating and storing a still image in which the preview image and the person image are combined, when a photographing command is input. The method of claim 8
At least one of the size and position of the person image is changeable. The method of claim 8,
The size of the person image is changeable in response to a user command of dragging after multi-touch. The method of claim 8,
The position of the person image is changeable in response to a user command of dragging after a single touch. camera;
A touch screen displaying a preview image of the camera; And
And a controller configured to enlarge and display a person image corresponding to the person when the person in the preview image is selected.

Images