KR20150106243A - Mobile terminal and method for controlling the same - Google Patents

Abstract

The present invention relates to a mobile terminal and a method for controlling the same. The mobile terminal comprises: a memory unit which stores pre-stored photographs; a display unit which displays information of the photographs; and a control unit which groups the pre-stored photographs, selects representative photographs for each group, and displays the representative photographs, which are selected based on the number of the photographs for each group, to be different from each other on the display unit. The control unit generates at least one grouping criterion based on the result of analysis on each pre-stored photograph, groups the pre-stored photographs based on the generated grouping criterion, and displays a graphic object related to the grouping criterion of a corresponding group in an area of the display unit on which at least one of the representative photographs are displayed.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a mobile terminal and a control method thereof.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal.

The functions of such mobile terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, modern mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

Such a mobile terminal can have a larger capacity memory due to the development of the present technology. As a result, current mobile terminals can store fewer than a few thousands of pictures. As a result, research is actively under way to enable users to search for desired pictures more easily and quickly.

The present invention is directed to solving the above-mentioned problems and other problems.

It is another object of the present invention to provide a mobile terminal capable of grouping pictures based not only on photographing time but also on a place where a photograph was taken or an event related to a photograph, and a control method thereof.

The present invention also provides a mobile terminal and a control method thereof, which can select any one of photographs that can represent grouped pictures and enable information related to the grouped pictures to be displayed on the selected representative picture For that purpose.

According to an aspect of the present invention, there is provided a mobile terminal including a memory unit for storing previously stored pictures, a display unit for displaying the image information, And a control unit for grouping the pictures, selecting representative pictures for each group, and displaying the selected representative pictures differently according to the number of pictures per group on the display unit, And a grouping step of grouping the previously stored photographs based on the generated grouping criterion, wherein at least one of the representative photographs is displayed in an area on the display unit on which at least one of the representative photographs is displayed, And a graphical object associated with the grouping criterion of the corresponding group is displayed .

In one embodiment, the control unit may infer the place or situation where the photographs were taken based on the analysis result of the image objects constituting each of the previously stored photographs, and generate the grouping criterion based on the inference result .

In one embodiment, the memory includes feature information on a predetermined plurality of specific places, and the control unit analyzes a foreground image and a background image of each of the pre-stored pictures And estimates a place where the photographs are taken in any one of the predetermined plurality of specific places based on a result of comparing the feature information with the feature points of the background image and the foreground image.

In one embodiment, the controller divides the image objects constituting each of the previously stored photographs into a portrait or an object, extracts a portrait or a portrait of the person included in the photograph from the image object divided by the portrait, Based on at least one of a result of recognition of at least one of the images and a result of recognizing the feature of the object included in the photograph from the image object classified by the object, .

In one embodiment, in the case of photographs including image objects recognizable as characters among the previously stored photographs, the control unit deduces an event related to the photographs based on the character recognition result of the image objects .

In one embodiment, the apparatus may further include a line of sight sensing unit for sensing a line of sight of the user, wherein the control unit displays the at least one of the representative photographs in an area on the display unit, And displaying the graphic object.

In one embodiment, the graphical object further includes map information on a location of the specific place when the grouping criterion is related to a specific place.

According to an aspect of the present invention, there is provided a method of controlling a mobile terminal, the method comprising: generating at least one grouping criterion based on analysis results of previously stored pictures; Grouping the pre-stored pictures based on the generated grouping criterion, selecting a representative picture for each group, displaying the selected representative picture according to the number of pictures included in the corresponding group, And displaying at least one graphical object related to a grouping reference of the corresponding group in an area on the display unit where at least one representative photograph of each group is displayed.

In one embodiment, the step of generating the grouping criterion may include the steps of analyzing the pre-stored photographs to deduce an event related to a place where each of the pictures is photographed or each of the photographs, And generating at least one grouping criterion from the previously stored photographs.

In one embodiment, the step of inferring may include analyzing feature points by classifying each of the pre-stored pictures into a background image and a foreground image based on whether or not the focus is matched, Further comprising the step of inferring a place where the photographs are taken based on a result of comparing feature points of the image and feature information of a predetermined plurality of specific places.

In one embodiment, the inferring may include: recognizing, for each of the pre-stored photographs, a characteristic shape of a person and an object included in each photograph; and determining at least one of the characteristic shapes of the person and the object And inferring an event related to each of the previously stored pictures using one of the plurality of pictures.

In one embodiment, the characteristic shape of the character is at least one of the posture, expression, age, and attire of the characters included in the respective photographs.

In one embodiment of the present invention, the step of displaying at least one graphic object includes the steps of recognizing the user's gaze, selecting one of the representative images of each group corresponding to the recognized user's gaze, And displaying the graphic object on any one of the selected representative photographs.

Effects of the mobile terminal and the control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, the present invention provides a method and apparatus for grouping photographs into a plurality of groups based on a result of analyzing image objects included in photographs, There is an advantage to be able to do.

In addition, according to at least one embodiment of the present invention, the present invention can be realized by grouping previously stored pictures by event, by photographing places, or by person, It is advantageous that only photographs or photographs of a specific person can be found more easily.

1 is a block diagram illustrating a mobile terminal according to the present invention.
2 is a flowchart illustrating an operation procedure of a mobile terminal according to the present invention.
FIG. 3 is a flowchart illustrating in greater detail an operation of creating a grouping criterion and grouping pictures during the operation shown in FIG.
Fig. 4 is a conceptual view showing a grouped example of the photographs in Fig. 3. Fig.
5 is a flowchart showing in more detail an operation procedure of selecting a representative picture in each group in the operation process shown in FIG.
FIG. 6 is a flowchart illustrating an operation of displaying a graphical object related to a grouping criterion on a representative picture in an operation process shown in FIG. 2. FIG.
FIG. 7 is an exemplary view illustrating an example in which grouped pictures are displayed in a mobile terminal according to the present invention.
8 is an exemplary diagram illustrating an example of a screen in which grouping criteria are set in a mobile terminal according to the present invention.
FIG. 9 is an exemplary view showing another example in which pictures grouped in the mobile terminal related to the present invention are displayed.
10A and 10B are views illustrating an example in which pictures are grouped according to a grouping criterion generated based on an analysis result of an image object in a mobile terminal according to the present invention.
11 is an exemplary view showing an example in which, in a mobile terminal related to the present invention, graphic objects related to grouping criteria are displayed in a representative picture.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like or similar elements are denoted by the same or similar reference numerals, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may be applied to fixed terminals such as a digital TV, a desktop computer, a digital signage, and the like, .

1A to 1C are block diagrams for explaining a mobile terminal according to the present invention, and FIGS. 1B and 1C are conceptual diagrams showing an example of a mobile terminal according to the present invention in different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, ), And the like. The components shown in FIG. 1A are not essential for implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than the components listed above.

The wireless communication unit 110 may be connected between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and another mobile terminal 100 or between the mobile terminal 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include at least one sensor for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic tip module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user and may provide an output interface between the mobile terminal 100 and a user.

The interface unit 160 serves as a path to various types of external devices connected to the mobile terminal 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the mobile terminal 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a plurality of application programs or applications running on the mobile terminal 100, data for operation of the mobile terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Also, at least a part of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions (e.g., telephone call receiving function, message receiving function, and calling function) of the mobile terminal 100. Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and may be operated by the control unit 180 to perform the operation (or function) of the mobile terminal.

In addition to the operations related to the application program, the control unit 180 typically controls the overall operation of the mobile terminal 100. The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 170. FIG. In addition, the controller 180 may operate at least two of the components included in the mobile terminal 100 in combination with each other for driving the application program.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the components included in the mobile terminal 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170. [

Hereinafter, the various components of the mobile terminal 100 will be described in detail with reference to FIG. 1A.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The mobile communication module 112 may be a mobile communication module or a mobile communication module such as a mobile communication module or a mobile communication module that uses technology standards or a communication method (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution And an external terminal, or a server on a mobile communication network established according to a long term evolution (AR), a long term evolution (AR), or the like.

The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. The wireless Internet module 113 is configured to transmit and receive a wireless signal in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, wireless LAN (WLAN), wireless fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, DLNA (Digital Living Network Alliance), WiBro Interoperability for Microwave Access, High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) and Long Term Evolution-Advanced (LTE-A) 113 transmit and receive data according to at least one wireless Internet technology, including Internet technologies not listed above.

The wireless Internet module 113 for performing a wireless Internet connection through the mobile communication network can be used for wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE or LTE- May be understood as a kind of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short-range communication module 114 is connected to the mobile terminal 100 and the wireless communication system through the wireless area networks, between the mobile terminal 100 and another mobile terminal 100, or between the mobile terminal 100 ) And the other mobile terminal 100 (or the external server). The short-range wireless communication network may be a short-range wireless personal area network.

Here, the other mobile terminal 100 may be a wearable device (e.g., a smartwatch, a smart glass, etc.) capable of interchanging data with the mobile terminal 100 according to the present invention (smart glass), HMD (head mounted display)). The short range communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100. [ If the detected wearable device is a device authenticated to communicate with the mobile terminal 100 according to the present invention, the control unit 180 may transmit at least a part of the data processed by the mobile terminal 100 to the short- 114 to the wearable device. Therefore, the user of the wearable device can use the data processed by the mobile terminal 100 through the wearable device. For example, according to this, when a telephone is received in the mobile terminal 100, the user performs a telephone conversation via the wearable device, or when a message is received in the mobile terminal 100, It is possible to check the message.

The position information module 115 is a module for obtaining the position (or current position) of the mobile terminal, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, when the mobile terminal utilizes the GPS module, it can acquire the position of the mobile terminal by using a signal transmitted from the GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, it can acquire the position of the mobile terminal based on information of a wireless access point (AP) that transmits or receives the wireless signal with the Wi-Fi module. Optionally, the location information module 115 may perform any of the other functions of the wireless communication unit 110 to obtain data relating to the location of the mobile terminal, in addition or alternatively. The location information module 115 is a module used to obtain the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or obtains the location of the mobile terminal.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, Or a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. [ A plurality of cameras 121 provided in the mobile terminal 100 may be arranged to have a matrix structure and various angles or foci may be provided to the mobile terminal 100 through the camera 121 having the matrix structure A plurality of pieces of image information can be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to a function (or a running application program) being executed in the mobile terminal 100. Meanwhile, the microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 123 is for receiving information from a user and when the information is inputted through the user input unit 123, the control unit 180 can control the operation of the mobile terminal 100 to correspond to the input information . The user input unit 123 may include a mechanical input unit or a mechanical key such as a button located on the rear or side of the mobile terminal 100, a dome switch, a jog wheel, Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, And a touch key disposed on the touch panel. Meanwhile, the virtual key or the visual key can be displayed on a touch screen having various forms, for example, a graphic, a text, an icon, a video, As shown in FIG.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a corresponding sensing signal. The control unit 180 may control the driving or operation of the mobile terminal 100 or may perform data processing, function or operation related to the application program installed in the mobile terminal 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface, without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner area of the mobile terminal or in proximity to the touch screen, which is covered by the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 141 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as "proximity touch & The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 141 can detect a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. Meanwhile, the control unit 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Furthermore, the control unit 180 can control the mobile terminal 100 such that different operations or data (or information) are processed according to whether the touch to the same point on the touch screen is a proximity touch or a touch touch .

The touch sensor senses a touch (or touch input) applied to the touch screen (or the display unit 151) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, do.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like. Here, the touch controller may be a separate component from the control unit 180, and may be the control unit 180 itself.

On the other hand, the control unit 180 may perform different controls or perform the same control according to the type of the touch object touching the touch screen (or a touch key provided on the touch screen). Whether to perform different controls or to perform the same control according to the type of the touch object may be determined according to the current state of the mobile terminal 100 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 180 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 121 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to sense a touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 153 may be vibration. The intensity and pattern of the vibration generated in the haptic module 153 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 153 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 153 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the user through the muscles of the finger or arm. The haptic module 153 may include two or more haptic modules 153 according to the configuration of the mobile terminal 100.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 100. Examples of events that occur in the mobile terminal 100 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the mobile terminal detecting the event confirmation of the user.

The interface unit 160 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data in the mobile terminal 100 to an external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, an audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 160.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the interface unit 160. [

The interface unit 160 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, And various command signals may be transmitted to the mobile terminal 100. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs the storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls the operations related to the application program and the general operation of the mobile terminal 100. [ For example, when the state of the mobile terminal meets a set condition, the control unit 180 can execute or release a lock state for restricting input of a user's control command to applications.

In addition, the control unit 180 performs control and processing related to voice communication, data communication, video call, or the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the controller 180 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the mobile terminal 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of an interface 160 through which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

Meanwhile, the mobile terminal 100 according to the embodiment of the present invention can clusters and group previously stored pictures in various ways so that the user can find the desired photo more easily and quickly. For example, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention can store the information about the time at which each photograph was taken corresponding to each photograph, and group the photographs using the time information have. For example, the control unit 180 may group previously stored pictures based on time information corresponding to each picture for a predetermined period of time, for example, month or year, according to the user's selection. As a result, the user can easily and quickly search for a desired image.

However, grouping the photographs according to the photographing time information has a limitation in selecting only the photographs desired by the user. For example, when photographs are grouped according to only the photographing time, the photographs include all the photographs photographed within a certain period selected by the user as one group. Accordingly, even when the user wants to enjoy the travel photograph taken in a specific period, other photographs taken in the same period can be included in the same group as the travel photographs. Therefore, the user can enjoy not only travel photographs but also other photographs (for example, business photographs).

In order to solve such a problem, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention can store not only the time information but also the position information at the time when the pictures are taken, corresponding to each picture. Therefore, the control unit 180 can group photos taken at the same position on the basis of the position information corresponding to each photograph. Accordingly, the user can enjoy the pictures by selecting only the group of the pictures taken at the specific place.

In addition, in the mobile terminal 100 according to the embodiment of the present invention, it is also possible to group pictures photographed at a specific place by each time zone using the information about the photographing time. In this case, the control unit 180 groups the pictures photographed during a predetermined period selected by the user according to the place where the pictures are taken, thereby making it easier for the user to find the pictures to be appreciated, .

In addition, it is needless to say that the controller 180 of the mobile terminal 100 according to the embodiment of the present invention can analyze the image itself and group the photographs based on the analyzed result. For example, the control unit 180 may extract a background image from the photographed photographs, and may infer the place where the photographed photograph was photographed using the extracted background image. In other words, the controller 180 recognizes the feature portion of the extracted background image and determines whether the place where the photograph was photographed is recognized as 'acid' or 'sea', or 'coastal' or 'riverside' Can be deduced. The control unit 180 may group the pictures having the same reasoning result into the same group, using the reasoning result about the place where the pictures were taken. Of course, the control unit 180 may use a foreground image instead of a background image to infer the photographing place of the photograph.

In addition, the controller 180 of the mobile terminal 100 according to the exemplary embodiment of the present invention extracts image objects constituting an image of each photograph, and groups the photographs using a result of analyzing the extracted image objects . For example, the control unit 180 extracts an image of a person or objects included in a photograph into respective image objects, and deduces an event related to the photograph according to the characteristics of each person image object or object image object It is possible.

For example, the control unit 180 can analyze the image of the person included in the photograph, and can recognize the posture or the expression, or the age range, the dress, etc., taken by the person. For example, the control unit 180 can analyze whether the person is jumping, standing, or standing, from the shape of each joint in the image of the person. Also, the controller 180 may analyze the facial expression of the person by analyzing the eyes, the mouth, and the like from the image of the face portion of the person. In addition, the control unit 180 may determine whether the image of the person includes a characteristic point of the clothes of the person, for example, a wrinkle or an ornament with many clothes, a short-sleeved or short- Necktie) to determine whether the garment is a suit.

The control unit 180 can deduce an event related to the photograph using the analysis result of the person image. For example, the control unit 180 may be able to infer that the faces of the characters included in the photograph are happy faces, and that when the clothes the characters wear are suits or wedding dresses, the photographs are photographed in 'wedding' have. Alternatively, if the characters included in the photograph have a tense look and the posture of the characters is running or running, and the clothes of the characters are shorts or short-sleeved clothes, It can be deduced that it is a photograph. The control unit 180 may group photos having the same reasoning result into the same group based on the result of the speculation.

Meanwhile, the controller 180 can deduce not only a portrait image object but also an event related to the photograph using an object included in the photograph. For example, the control unit 180 can analyze an image of an object of a predetermined size or more from the image of the photograph to deduce an event related to a place or a photograph where the photograph was taken from the characteristic point of the object. Alternatively, if the object includes a character recognizable image, the control unit 180 may perform character recognition on the image, and may infer the event related to the photograph based on the character recognition result. In addition, the controller 180 can deduce an event related to the place where the photograph was taken or the photograph, using a web search for the minutiae points of the image of the object.

The control unit 180 of the mobile terminal 100 according to the embodiment of the present invention includes an image analysis unit 181 and a reasoning unit 182 in order to deduce an event or a shooting place based on the image analysis result . The image analyzing unit 181 and the reasoning unit 182 may be implemented in the control unit 180 or may be implemented separately from the control unit 180 outside the control unit 180. [

Meanwhile, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention may select one of the grouped pictures as a representative picture of each group so that the user can easily and quickly find a desired picture, The representative photograph may be displayed on the display unit 151 based on the characteristics of the grouped photographs. For example, the control unit 180 can display a representative photograph of a group including many photographs on the display unit 151 in a relatively larger size than a representative photograph of a group that does not include the representative photograph. Further, the control unit 180 may display a graphic object for displaying information related to a grouping criterion such as a grouping criterion, that is, a grouping criterion such as the above-mentioned event, It may be displayed on at least a part of the area on the display unit 151 on which the photograph is displayed. Here, the graphic object may be text information for the grouping criterion, or map or navigation information indicating a specific place.

The control unit 180 may display such a graphic object in a representative photograph of a certain size or more, that is, a representative photograph of a group having a predetermined number of or more than the number of the photographs included in the corresponding group. In addition, the controller 180 may display the graphic object on only one representative photograph based on the user's gaze. For this, the sensing unit 140 of the mobile terminal 100 according to the embodiment of the present invention may include a visual recognition sensor 143 for sensing the user's gaze, and the sensing unit 140 may include a visual recognition The movement of the user's line of sight and line of sight can be sensed through the sensor 143. [ Accordingly, the control unit 180 may display the graphic object only on one representative photograph corresponding to the sight line of the sensed user, among the representative photographs displayed on the display unit 151 at this time. Also, in this case, the control unit 180 may display the graphic object only when the representative picture corresponding to the user's gaze has a predetermined size or greater.

In the above description, the controller 180 classifies pictures having the same reasoning result into the same group. However, the controller 180 may perform the classification of the reasoning result and the photographing time and the photographing location together Of course it is. For example, when there is a picture having a clear inference result, that is, a picture having a certain degree of certainty, the control unit 180 generates a grouping criterion based on the inference result of the picture, It is also possible to group other photographs taken during a time within a predetermined range of time, that is, photographs taken at the same time, or photographs taken at a place where the reasoning result is clear, into groups according to the grouping criterion. In this case, the control unit 180 can clearly group only the pictures photographed in the specific event into the photographs related to the specific event, without performing the image analysis on all the previously stored photographs.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

Hereinafter, embodiments related to a control method that can be implemented in a mobile terminal configured as above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

2 is a flowchart illustrating an operation procedure of the mobile terminal according to the present invention.

2, the controller 180 of the mobile terminal 100 according to the exemplary embodiment of the present invention may generate a grouping criterion for previously stored pictures and group the pictures based on the generated grouping criterion (S200). Here, as described above, the control unit 180 may generate the grouping criterion based on the information about the photographing time or the photographing place stored corresponding to each photograph. Alternatively, as described above, the grouping criterion may be generated based on an analysis of images of each photograph, and based on the analyzed result, an event related to the photographs or a result of inferring the place where the photograph was photographed. Hereinafter, an operation of analyzing images of each photograph and generating a grouping criterion according to the analysis result and grouping the photographs according to the generated grouping criterion will be described in more detail with reference to FIG.

In addition, the controller 180 sets a predetermined classification criterion such as time, place, event, or person, classifies each photograph according to each classification criterion, and then displays the photographing time, Such as an event related to the event.

When the grouping of the pictures is completed (S200), the controller 180 selects a representative picture among the pictures included in each group (S202). For example, the control unit 180 may select the first photographed image or the latest photographed image in the group as the representative photographed image.

Alternatively, the control unit 180 may select a photograph having the most specific inference result as the representative photograph in the case of photographs grouped according to a result of inferring a photograph-related event or a place where the photograph was photographed. For example, in the case of the photos grouped according to the reasoning inferred as 'wedding', the control unit 180 may select a photograph including a person who is recognized as wearing a 'wedding dress' as a representative photograph. In addition, the control unit 180 may select a photograph having the most definite character recognition result as a representative photograph when the photographs of the group are photographs grouped based on the character recognition result of the image recognizable by the character.

In addition, the control unit 180 may display a representative image of a specific region or place, which is searched from the web, among photographs of the group, based on a result of performing a web search on the object image, A photograph including an object image having the most similar characteristics to the representative image of the group may be selected as the representative photograph of the group. The operation of the controller 180 for selecting representative photographs in various ways will be described in more detail with reference to the drawings in FIG.

The control unit 180 may display the representative pictures of the respective groups based on the number of pictures included in each group (S204). That is, the control unit 180 may display the representative pictures on the display unit 151 differently based on the number of pictures, and a graphic object indicating the number of pictures may be displayed on the display unit 151 In the vicinity of the area on the screen.

The control unit 180 may display at least one graphic object related to the grouping reference of the group corresponding to at least one of the displayed representative pictures (S206). For example, the control unit 180 may display the graphic object only on at least one representative photograph displayed in a size larger than a predetermined size among the representative photographs. The graphic object may be for displaying information related to a common feature of the photographs included in the corresponding group, that is, the grouping criterion, as described above, and may be text information or a location where the pictures of the corresponding group are photographed It may be map information to be displayed.

Meanwhile, FIG. 3 is a detailed view illustrating an operation of creating a grouping criterion and grouping pictures in the operation procedure shown in FIG. 2, as described above.

Referring to FIG. 3, the control unit 180 analyzes an image of previously stored pictures (S300). For example, the control unit 180 may divide the image of each photograph into a background image and a foreground image according to whether the focal points match each other. That is, the control unit 180 divides an image of a portion corresponding to the photographed focus, for example, a contour or a portion of a color with a predetermined level or more in color, into a foreground image, The image of the outline or faded part can be divided into the background image.

The control unit 180 analyzes the divided background image or foreground image to deduce a place or an event in which the image is captured (S302). For example, the control unit 180 analyzes the background image or the foreground image, and compares the feature point portion of the background image or the foreground image with feature information of a plurality of places previously stored in the memory 170, Can be deduced. That is, if the background image includes an image of a ridged shape, or if the foreground image includes image objects recognizable as' trees', the control unit 180 determines whether the place where the photo was taken is' Mountain '. Or if the background image includes a horizontally-shaped image, or if the foreground image includes image objects that can be recognized as a 'ship' or a 'sandy beach', the control unit 180 determines It can be deduced that it is a 'beach'.

Here, the characteristics of the image that can be recognized as the 'ridiculous' image or 'tree' and the information about the characteristics of the image that can be recognized as 'horizontal line' or 'boat' or 'sandy beach' Quot; feature information of the places ". The feature information may be stored in the memory 170 or downloaded from the manufacturer or the web of the mobile terminal 100.

Alternatively, the control unit 180 may divide the image objects constituting the images of the photographs into a portrait image and a subject image, as described above, and may infer the place or event where the photographs are taken according to the minutiae points of the respective image objects . For example, in the case of a portrait image, the controller 180 can recognize the posture taken by the person based on the image of the joint part of the image of the person, as described above. Also, the controller 180 can analyze the facial expression of the person based on the image of the facial part of the person, and can determine the emotional state of the person based on the analysis result of the facial expression. For example, in the case of a person whose facial expression is smiling, the control unit 180 can determine that the emotional state of the person is a joy state. In the case of a person with a solid facial expression or bowing his head and pointing his gaze in a specific direction, It may be determined that the emotional state of the person is in a state of tension. In addition, the control unit 180 may determine the age range of the person based on the analysis result of the face image of the person.

Further, the control unit 180 can further analyze 'clothes' worn by the characters from the images of the characters included in the photograph. For example, the controller 180 may analyze the length of the sleeves, the length of the trousers, etc. of the clothes the characters wear, and determine whether the characters are wearing shorts or short-sleeved clothes. In addition, the controller 180 can determine the clothes the characters are wearing from the characteristic parts such as the color of the clothes and the ties.

The control unit 180 can deduce a place or an event where the corresponding photograph is photographed based on the determination result. For example, if the age group includes many young characters, the attitude of the characters is running or running, the emotional state of the characters is strained, and the clothes the characters wear are shorts or short- And the garment having little ornaments, the control unit 180 can infer that the photograph is photographed in the 'athletic meet' of the children. Alternatively, the controller 180 may include a plurality of middle-aged persons, a person in a standing posture, a person with a smiling expression, and a person wearing a formal or wedding dress including a tie or the like It can be inferred that the photograph was taken at the 'wedding'.

Also, in the case of the object image included in the photograph, the control unit 180 can deduce from the shape of the object image, etc., the place or event where the photograph was taken. For example, the control unit 180 may perform a web search for an object image, and may infer a place where the photograph was taken based on a web search result. For example, in the case of a photograph in which a specific cultural property (for example, 'Dabotop') is photographed, the control unit 180 can obtain a web search result for the image of the cultural property from the photograph, (E.g., " Gyeongju Bulguksa Temple ") can be deduced as the place where the photograph was taken.

Alternatively, when the object image included in the photograph includes an image recognizable as a character, the controller 180 may use the character recognition result. For example, when the control unit 180 has a phrase "celebrate birthday" in an image of an object, for example, a banner included in a photograph, the controller 180 performs character recognition on the phrase , It may be judged that the photograph was taken at the "birthday party".

Also, in the above description, it has been explained that a place or an event in which a photograph is photographed is inferred according to each case, and the controller 180 comprehensively judges such cases, It is also possible to deduce a place or an event.

For example, the control unit 180 can infer an event related to the pictures or a place where the pictures were taken according to a result of the inferred result of the object image, have. For example, the controller 180 may determine that the inference result (for example, character recognition result) from the image of the person's dress or object rather than the face or the attitude of the person has a higher degree of clarity of reasoning. In this case, even if the facial expression or attitude of the characters included in the photograph does not match the inferred result based on the outfit or character recognition result of the characters, Or on the basis of the inferred result according to the character recognition result.

In step S304, the controller 180 may generate a plurality of grouping criteria based on the inferred result in step S302. For example, the grouping criterion may be an inferred result based on the 'feature information of a plurality of previously stored places' or an inferred result based on the image of the person and the object image, It may be information. Then, the controller 180 groups the previously stored photographs based on the generated grouping criteria (S306). Accordingly, the pictures having the same reasoning result can be a group. Accordingly, the user can easily search a desired picture by selecting a desired group, or can accurately select only a specific event or a picture taken at a specific place.

Meanwhile, in the above description, it has been explained that the location or the event where the photograph was photographed was inferred using the analysis results of the previously stored photographs, but it is needless to say that it is not necessary to analyze all the photographs. For example, the control unit 180 may extract previously displayed photographs of specific events or places from pre-stored pictures. Here, pictures with specific events or places clearly marked are those that contain images that clearly show where the photos were taken or where the events were to be inferred.

Here, the 'clear image' may be a specific clothing or a specific phrase. For example, in the case of a photograph including a clothing image such as a 'wedding dress', the 'wedding dress' image may be an obvious basis for the inference result that the photograph is a 'wedding'. In this case, the control unit 180 may group the photographs taken at the same place in the same time zone as the photographs including the 'wedding dress' image into the photographs taken at the 'wedding'.

Also, for example, in the case of a photograph including an image including a phrase "Celebrate a birthday ", the above phrase may be a clear basis for the inference result that the photograph is a birthday party. In this case, the control unit 180 may group the pictures photographed at the same place in the same time zone as the photographs including the phrase, into the pictures photographed at the 'birthday party'.

Meanwhile, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention may classify the grouping criteria generated in step S302 according to a plurality of predefined classification items according to the attributes. When the user selects a specific category item, the grouped photos are displayed on the display unit 151 according to the grouping criterion corresponding to the attribute of the category item, It is easier and faster to find.

For example, the control unit 180 may have a plurality of predetermined classification criteria, such as a time line, a place, an event, and the like. In this case, the control unit 180 may classify the grouping criteria by time, place, and event attribute. For example, if the grouping criterion generated in step S302 is a name of 'mountain', 'sea', or a specific scenic spot or a specific place, the controller 180 may classify the grouping criterion as a 'place' item. Alternatively, when the grouping criterion is 'wedding', 'birthday feast', or 'athletic meet', the control unit 180 may classify the grouping criteria as an 'event' item. Here, the grouping criteria may be automatically classified into any one of the classification items according to an attribute of the user, but the attribute may be determined as a specific classification item according to the user's selection.

Meanwhile, FIG. 4 is a conceptual diagram illustrating an example in which pictures are grouped based on grouping criteria classified according to a predetermined classification criterion in FIG.

Referring to FIG. 4, the controller 180 classifies the reasoning results of step S302 into time 400, place 404, event 406, and the like. In addition, FIG. 4 shows that the category 406 includes the character 408. This is because, in the case of the mobile terminal 100 according to the embodiment of the present invention, the image of the face portion of the person included in the specific photograph is analyzed , And photographs including the same face image may be grouped based on the image of the face portion.

As shown in FIG. 4, when the classification items including the person 408 are set, the control unit 180 classifies the currently generated grouping criteria by classification items based on the respective attributes. For example, the classification item 'time 400' may include grouping criteria 410 and 412 for a certain period, and the category item 'place 404' may include grouping criteria 420, 422, 430, and 432 may be included. In the category item 'place (404)', position information 420 or 422 obtained as a result of web search for the feature point of the object image included in the photograph or the image information stored in correspondence with the photograph at the time of photographing is included as a grouping reference Or the result 430 or 432 derived from the background image or the foreground image of the photograph may be included as a grouping criterion.

In addition, the classification item 'event' may include inferred results 450 and 452 according to the result of recognizing from the clothing, the attitude expression of the person included in the photograph, the phrase of the object, and the like. In the category item 'character 408', the face images 460 and 462 of a specific person may be included as a grouping reference.

On the other hand, when the grouping of the pictures is completed, the controller 180 selects one of the grouped pictures as a representative picture in order to make it easier for the user to know what pictures the grouped pictures are related to, It may be displayed on the display unit 151 instead of the grouped pictures. If the grouping of photographs is performed according to the inference result of any one of the well-defined photographs as described above, it is also possible to select a photograph with the clear grounds as the representative photograph.

However, it is needless to say that the control unit 180 may select a photograph that satisfies a specific condition among photographs included in the same group as a representative photograph.

FIG. 5 is a detailed view illustrating an operation of selecting a photograph satisfying a specific condition as a representative photograph.

Referring to FIG. 5, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention determines whether or not the foreground region of the photos included in the same group and the saturation and brightness are higher than a predetermined level Pictures can be selected (S500). Here, the foreground region refers to a region of a portion corresponding to the focus from a photographic image. The wider the foreground region, the clearer the picture is, and the narrower the foreground region is, It can be judged that it is a cloudy photograph. This is to prevent images that are basically out of focus, that is, photographs that are erroneously photographed by the user, or images that are too blurry or dark to identify persons or objects as representative images. Here, the predetermined level may be increased or decreased according to the user's selection.

On the other hand, if the photographer has a certain level of saturation and brightness over the predetermined level and the photographer selects a photograph with a certain level or higher, the controller 180 may select representative photographs according to a predetermined criterion for classification item destruction or grouping criteria (S502).

For example, the control unit 180 may have a different representative picture selection criterion for each classification item. That is, in the case of the groups in which the classification item is included in the 'person', the control unit 180 determines that the 'face' image of the person who is the grouping reference among the pictures of each group is a photograph having a certain size or more and / 'The image of the expression can be selected as the representative picture of the expression of the smiling expression. Or if the category item is a 'place', the control unit 180 performs a web search for the place, and may select a photograph having the most similar characteristics to the representative image of the corresponding place searched from the web, as the representative photograph.

Or the control unit 180 may have different criteria for each grouping criterion. For example, when the reason for reasoning the grouping criterion is a clothing worn by a specific person included in the photograph, the control unit 180 determines that the focus is on the person wearing the garment or the garment and the sharpest, that is, May select the highest picture as the representative picture of the group. Similarly, when the reason for reasoning the grouping criterion is a character image included in a specific photograph, the control unit 180 may select the sharpest photograph among the photographs including the character image as a representative photograph of the group have.

In addition, the control unit 180 may select a photograph that is determined to have the best color feeling as a representative photograph. For example, when the place where the photograph was photographed is outdoor and the time at which the photograph was photographed is evening, the control unit 180 displays a photograph having high color saturation such as a color having cool color, for example, blue or white as a representative photograph It may be elected. On the other hand, when the photographed time is the daytime, a photograph having high color saturation such as red or yellow may be selected as the representative photograph. That is, the control unit 180 can select a preferred color based on the location where the photographs included in the group are photographed, the photographed time or the event, and select a specific photograph as a representative photograph of the group according to the selected color It is possible.

Meanwhile, in the mobile terminal 100 according to the embodiment of the present invention, the selected representative photographs are displayed differently according to the number of pictures included in the corresponding group so that the user can easily and quickly find desired pictures I have stated earlier. A graphical object for displaying information related to the grouping criterion may be displayed on at least a part of the area on the display unit 151 on which the representative picture is displayed.

FIG. 6 shows an operation of displaying a graphic object related to the grouping criteria on the representative picture in more detail.

6, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention displays representative pictures selected for each group in step S204, based on the number of pictures included in each group, If the representative photographs are displayed on the display unit 151, representative photographs having a size larger than a predetermined size displayed on the display unit 151 are selected (S600).

In step S602, the controller 180 generates graphic objects for displaying information related to each grouping criterion of groups corresponding to the selected representative photographs. Here, the grouping criterion is a result of inference of a place or an event where photographs belonging to each group are photographed, as described above, and may be a common point of the photographs included in the group. Therefore, the grouping criterion can be a feature of the group.

Also, graphic objects for displaying information related to the grouping criteria may be various. For example, the control unit 180 may generate a graphic object in the form of a text including information related to the grouping criteria. Alternatively, the control unit 180 may generate the graphic object with an image related to the grouping reference of the group so that the user can intuitively recognize the grouping reference of the photos included in the group. For example, the control unit 180 may generate an icon image corresponding to the inference result of the group as the graphic object, or, if the inference result is for a specific place, A thumbnail map image corresponding to a position on the map of the thumbnail image may be generated as the graphic object. Here, the control unit 180 may generate at least one graphic object.

In step S604, the control unit 180 displays the generated at least one graphic object on the display unit 151 on which at least one representative picture is displayed. For example, the controller 180 displays at least one graphic object corresponding to each of the representative photographs selected in step S600, that is, representative photographs displayed on the display unit 151 at a predetermined level or more in size, Or a graphic object corresponding to the corresponding representative photograph may be displayed in only one representative photograph based on a predetermined selection criterion.

For example, the control unit 180 selects at least one representative photograph displayed on the display unit 151 according to a preset size (for example, from top to bottom, from left to right) The corresponding graphic object may be displayed. Alternatively, the control unit 180 may display only one representative picture selected by the user, such as a touch input, among the representative pictures displayed at a predetermined size or more, and display the corresponding graphic object on the display unit 151 And may be displayed on at least a part of the area.

Alternatively, the control unit 180 can select any of the representative photographs displayed on the display unit 151 based on the sight line of the user sensed through the sensing unit 140. The control unit 180 may display a graphic object corresponding to the selected representative picture in an area on the display unit 151 where the selected representative picture is displayed. In this case, when the movement of the user's gaze is detected, the control unit 180 selects another representative picture corresponding to the moved visual line, and the graphic object corresponding to the selected representative picture is displayed on the display unit 151 may be displayed on at least some areas. An example of at least one graphic object displayed in response to the user's gaze movement will now be described in detail with reference to FIG.

In the above description, the operation flow of the mobile terminal 100 according to the embodiment of the present invention has been described in detail.

Hereinafter, an example of grouping pictures based on the operation procedure described in FIGS. 2 to 6 in the mobile terminal 100 according to the embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 7 illustrates an example in which pictures grouped in the mobile terminal related to the present invention are displayed.

7 (a) and 7 (b) show an example in which pictures are grouped on the basis of respective grouping criteria by a predetermined period (month). 7 (a) is a diagram illustrating photographs taken in August 2013 according to the place where the photograph was taken, the person included in the photograph, and the event related to the photograph. Respectively.

7 (a), the controller 180 can display the size of representative pictures differently based on the number of pictures included in each group. Accordingly, the controller 180 can display the pictures included in the group 1 700 The number of pictures included in group 2 702 and group 3 704 is similar and the number of pictures included in groups 4, 5 and 6 706, 708, and 710 is the most .

Also, as shown in the representative photographs of the respective groups, the groups 700, 702, 704, 706, 708, and 710 are grouped into different grouping criteria. For example, group 1 700 is a grouping of photographs where the place where the photograph was taken is inferred as 'Seashore', as can be seen from graphical object 710. The group 2 702 and the group 4 706 are grouped based on the face image of a specific person as shown in the elected representative picture and the group 3 704 and the groups 5 and 6 708 and 710 It is shown that each event is grouped based on grouping.

If another time period is selected by the user, the mobile terminal 100 according to the embodiment of the present invention analyzes images of photographs photographed in different selected time periods and groups them based on the analyzed results. FIG. 7B shows such an example, and the group 7 750 shows an example of grouping the photographs in which the event related to the photograph is inferred as 'Wedding'. That is, in the mobile terminal 100 according to the embodiment of the present invention, the image of the photographs photographed during the period selected by the user (December, 2013) is analyzed, and an event related to the photographs or a place where the photographs are taken is inferred , And group photos based on the inference results (grouping criteria).

As described above, in the mobile terminal 100 according to the embodiment of the present invention, it is possible to generate pictures having the same reasoning result in the same group, using the inferred results based on the analysis results of images as a grouping reference. Therefore, the present invention analyzes the image itself of the photograph, not the location information of the mobile terminal at the time of photographing or photographing, and groups the photographs according to contents included in the image of the photograph, Allows you to search for images you want to search or watch.

It is needless to say that the user may directly designate grouping of photographs according to the grouping criterion.

FIG. 8 shows an example of a screen in which a user directly sets a grouping criterion.

For example, as shown in FIG. 8 (a), the user may set different grouping criteria for each hour, place, and person, or may set specific conditions for grouping the pictures.

For example, when the user selects' time 900 'in the setting shown in FIG. 8 (a), the controller 180 of the mobile terminal 100 according to the embodiment of the present invention selects the' the setting screen shown in Fig. 6B can be displayed on the display unit 151. [ Then, the user can selectively change a predetermined period or the like that can be set as the grouping reference in the grouping reference setting by time shown in (b) of FIG.

Alternatively, the user may set a specific time, and when a predetermined number or more of photographs are photographed at a set time, they may be set to be generated in one group. For example, as shown in FIG. 8B, the user selects 'duration 810' and 'number of pictures 812', and for a predetermined time (for example, 3 hours) , Or when a predetermined number (for example, 30 or more) of photographs are photographed, a specific group may be generated.

In addition, it is needless to say that the user can directly set the group size in the setting screen shown in FIG. 8 (b). For example, as shown in the group setting 814 shown in FIG. 8B, the user may select a small group if the number of pictures is 1 or more, a middle group if the number of pictures is 10 or more and less than 30, If it is more than 30, it can be set to group into large groups.

8 (a), the controller 180 of the mobile terminal 100 according to the embodiment of the present invention selects the 'location 802' from the setting screen shown in FIG. 8 (a) c) on the display unit 151 as shown in FIG. In addition, the user can set the groups of the photographs taken at a specific place to be grouped into a specific group or not to group the pictures taken at a specific place in the grouping reference setting of each place shown in (c) of FIG.

For example, when the user selects 'Exception Place (820)' on the setting screen shown in (c) of FIG. 8, the control unit 180 displays Grouping may not be performed. Or 'Special Place', the control unit 180 may group pictures taken at a place designated by the user into a specific group. Here, the control unit 180 may receive an address for a specific place from the user, or may display a map related to the location designation on the display unit 151, and may receive position information on the specific place from the map. Here, the control unit 180 displays a graphical object related to the movement of the mobile terminal 100 on the map, thereby displaying a place where the user mainly stays on the map, You may also be able to enter information.

On the other hand, when the user selects the 'person 804' on the setting screen shown in FIG. 8A, the control unit 180 displays a setting screen as shown in FIG. 8D on the display unit 151 ). ≪ / RTI > For example, the user may set the photographs including the face of a specific person to be grouped separately, or may not group them for a specific person, similar to the one shown in FIG. 8B.

For example, in the setting screen shown in FIG. 8D, the user can register a face image of a specific person in 'Except People 830'. In this case, the controller 180 may analyze the face image of the person registered in the 'non-person 830' to prevent the pictures from being grouped on the basis of the person, or may be grouped according to another grouping criterion The photographs including the persons registered in the 'non-person 830' may be excluded from the photographs. Also, in the case of a person registered as 'Special People 840', the control unit 180 analyzes the face image of the person registered in the 'specific person 840' They may be grouped into specific groups.

Here, the control unit 180 may display predetermined pictures on the display unit 151 in order to register a face image of a specific person from the user. Here, the predetermined pictures may include pictures downloaded from a social network service (SNS) or pictures downloaded from the web or an external server.

The control unit 180 may extract the face images of the persons included in the specific photo selected from the user and display the extracted face images on the display unit 151. [ If at least one of the face images is selected, the control unit 180 may register the selected face image as the 'non-person 830' or the 'specific person 840'.

Meanwhile, according to the above description, the mobile terminal according to the embodiment of the present invention has mentioned that each grouping criterion generated at present can be classified according to a plurality of predetermined classification items according to the attribute.

FIG. 9 is an exemplary view showing another example in which pictures grouped in the mobile terminal related to the present invention are displayed in this case.

8, (a), (b), (c), and (d) of FIG. 8 illustrate an example of a classification of a category item by a time line, a place, an event, Are set. In this case, the control unit 180 may classify the grouping criteria into one of the category items based on the attribute of the grouping criteria generated according to the result of analyzing the image of the currently stored pictures.

Referring to FIG. 9 (a), FIG. 9 (a) shows an example in which the classification item 'time 900' is selected. If the category item 'time 900' is selected as described above, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention may search for the photographing time information of each photograph. Each photograph is classified according to a predetermined period. As described above, a representative picture is selected for each group, and the size of the representative picture displayed on the display unit 151 is determined according to the number of pictures included in the group. For a group having a size of the representative picture of a certain level or higher, information related to the grouping criterion of the group, that is, information related to the time at which the pictures included in the group are photographed, can be displayed using a graphic object. As described above, in the mobile terminal 100 according to the embodiment of the present invention, the photographs can be grouped according to the photographing time by predetermined period.

On the other hand, FIG. 9B shows an example in which 'place 910' is selected as a classification item. When the category item 'place 910' is selected as described above, the control unit 180 groups each photograph based on the inferred 'photographing place' based on the result of analyzing the images of the respective photographs. Here, the control unit 180 can deduce the location where the photographs were taken according to the location information of the mobile terminal 100 recorded at the time the photographs were taken, or analyze the images of the photographs, You can also infer where the photos were taken.

Accordingly, the pictures grouped in FIG. 9B may be similar photographs as the result of analyzing the images of photographs, such as 'beachfront 912', or may be similar photographs such as 'Gyeongju Bulguksa' or 'Busan Haeundae' As shown in FIG. As described above, when the 'place 910' is selected as the category item, the control unit 180 can display the photos grouped according to the grouping reference related to 'place' on the display unit 151.

Similarly, FIG. 9 (c) shows an example in which 'event 920' is selected as the classification item. If 'event 920' is selected as the category item, the control unit 180 groups each picture based on the inferred 'event' based on the result of analyzing the images of the respective pictures. For example, the control unit 180 can infer events related to photographs by analyzing the facial expression, posture, age of a person, clothing of a person, and the like, from a portrait image included in each photograph. Alternatively, the control unit 180 may analyze the shape of the object, characters, or the like from the images of the objects included in the photographs to deduce an event related to the photographs.

Then, the controller 180 groups the pictures based on the event reasoning result of each picture. Accordingly, the control unit 180 may group the pictures into grouping criteria such as 'wedding', 'athletic meeting', or 'birthday party', and display and group representative pictures according to the number of pictures included in each group You can display graphical objects related to the criteria. FIG. 9C shows an example in which when the classification item 'event 920' is selected, the reasoning results are grouped according to the grouping criteria related to 'event'.

Also, in the mobile terminal 100 according to the embodiment of the present invention, the photographs may be grouped based on the 'person 930'. For example, when the user selects the classification item 'person 930' as shown in FIG. 9D, the control unit 180 analyzes the 'face' portion of the person image included in the photograph, Photographs containing the same people can be grouped into the same group. When the category item 'person 930' is selected as described above, the control unit 180 can display the photos grouped on the basis of the face of each photograph on the display unit 151. The representative photographs of the respective groups can be the photographs in which the face of the person as the grouping reference of each group is best displayed. For example, the control unit 180 can select, from the pictures included in the group, a picture whose saturation or lightness is equal to or higher than a certain level and whose focal point matches the face of the person as a grouping criterion, A photograph in which a face of a person as a grouping reference is photographed in a larger size may be selected as the representative photograph.

10A and 10B are views illustrating an example in which pictures are grouped according to a grouping criterion generated based on an analysis result of an image object in a mobile terminal according to the present invention.

According to the above description, the mobile terminal 100 according to the embodiment of the present invention has been described in which, when an image that can be recognized as a character is contained in a photo image, it is possible to deduce an event related to the photo using the image . For example, as shown in (a) of FIG. 10A, in the case of a photograph including a character recognizable image 1000, the controller 180 performs character recognition on the image 1000, You can infer relevant events. Accordingly, if the recognized character is 'Birthday' as a result of recognizing the image 1000 as shown in FIG. 10A, the control unit 180 determines that the photo is related to the 'birthday party' Can be deduced. In this case, the control unit 180 inferred according to the result of analyzing the images of other previously stored pictures, and as a result, the event related to the corresponding photo is the 'birthday party' and the pictures shown in FIG. They can be grouped into one group.

The mobile terminal 100 according to the embodiment of the present invention may be configured such that, when there is a picture that can clearly deduce a specific event or a place where the picture is taken, For example, a group of such photographs as the clearly speculable photographs. 10A, if there is a picture that can be clearly deduced that it is a 'birthday party', the control unit 180 may display the same place in the same time zone as the picture shown in FIG. 10A (a) May be grouped into a group, that is, a group called 'birthday party 1002'.

In addition, according to the above description, the mobile terminal 100 according to the embodiment of the present invention can deduce an event related to the photograph based on the posture, facial expression, clothes, etc. of the person. Accordingly, in the case of the photograph shown in (a) of FIG. 10B, the controller 180 determines that the posture of the characters is in a large posture, for example, 'motion', based on the joint shape of the characters included in the photographic image It can be judged that it is doing. Then, the controller 180 analyzes the clothes of the characters, and if the characters are wearing sleeves and / or short skirts or short skirts, the photographer can judge the photograph as a photograph in which the characters act. In this case, the control unit 180 may group pictures having an analysis result similar to the above photographs into a group called 'Sports day (1050)'.

10B, when the controller 180 determines that the picture in which the characters are moving is clear, as in the case of FIG. 10A, Of course, the pictures photographed at the same place in the same time zone as the photograph may be grouped into one group, i.e., a group called 'athletic meeting (1050)'.

According to the above description, the mobile terminal 100 according to the embodiment of the present invention includes a graphic object for displaying information related to a grouping criterion of each group, At least in part. The graphical object can be of various forms.

FIG. 11 illustrates an example in which a graphic object related to a grouping reference is displayed in a representative picture in the mobile terminal according to the present invention.

The controller 180 of the mobile terminal 100 according to the embodiment of the present invention may display information related to grouping criteria of each group in various manners. For example, the control unit 180 may display information related to the grouping criteria of each group as graphic objects in the form of images such as graphic objects or icons. Alternatively, if the grouping criterion is position information of a specific place, information related to the grouping criterion may be displayed using a graphic object such as a map corresponding to the position information as shown in FIG. 11 (c).

In addition, the controller 180 may display graphic objects for displaying the information on all or at least one representative pictures displayed on the display unit 151. [ For example, as shown in FIG. 11A, the control unit 180 may display the graphic object only for one group, and the control unit 180 may display any group in which the graphic object is displayed It can be selected according to preset conditions.

For example, the controller 180 may select a group in which the graphic object is to be displayed according to the size of the representative picture representative of each group. Since the size of the representative picture may be determined according to the number of pictures belonging to the corresponding group, the controller 180 may display the graphic object only for a group having a predetermined number of pictures or more.

Accordingly, in the screen shown in FIG. 11, only the 'coastline' group 1100 and the 'birthday party' group 1102 can be selected as a group in which the graphic object can be displayed. In this state, the controller 180 may cause the graphic object to be displayed in all of the selected groups, or may display only one specific group of graphics objects, such as shown in FIGS. 11A and 11B, May be displayed.

Here, the controller 180 may select any one of the groups in which the graphic object is to be displayed in various manners. For example, the control unit 180 can sequentially change the group to display the graphic object at predetermined time intervals, or select one group based on the user's selection such as touch input. Alternatively, the control unit 180 may sense the user's gaze and select only one group corresponding to the user's gaze, as described above.

For example, the control unit 180 may sense the direction of the user's eyes, that is, the line of sight, based on the detection result of the sight recognition sensor 143 of the sensing unit 140. One point on the display unit 151 corresponding to the user's line of sight can be recognized, and a group corresponding to the recognized point can be selected as the group in which the graphic object is displayed. In this case, the control unit 180 may generate graphic objects corresponding to the information related to the grouping criteria, that is, information such as 'birthday party', 'wedding', 'mountain', and 'sea' . For example, the graphic object may be text information corresponding to the speculation result, or may be an image of an icon corresponding to the information.

The control unit 180 may display the graphic object in an area on the display unit 151 on which representative photographs of the currently selected group are displayed. Accordingly, when the group corresponding to the user's gaze is the 'coastal area' group 1100, the control unit 180 displays 'Seashore' in the 'coastal area' group 1100 as shown in FIG. Can be displayed as a graphic object.

If the user's gaze is moved to the area on the display unit 151 where the 'birthday party' group 1102 is displayed, the control unit 180 displays 'birthday party' as shown in FIG. 11 (b) A graphic object can be displayed in an area on the display unit 151 where a representative picture of the group 1102 is displayed. And the graphic object displayed here is for displaying information related to the grouping criteria of the birthday party 'group 1102. [ Accordingly, as shown in FIG. 11 (b), text information called 'Birthday Party' can be displayed as a graphic object.

Meanwhile, in the present invention described above, it has been explained that events and photographs related to photographs are photographed using the analysis result of the image itself of the photograph, but it is also possible to perform grouping according to specific photographs designated by the user Of course it is. For example, a specific grouping criterion may be generated according to a result of analyzing an image of a specific photograph specified by a user, or the other photographs and the corresponding photographs may be created as a group based on the generated grouping criterion. Here, the specific photograph designated by the user may be a photograph directly selected by the user with a touch input, or a photograph in which the user directly sets the title of the photograph.

That is, when there is a picture having a title designated by the user among the pre-stored pictures, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention can generate the title content on a grouping basis, And photographs taken at the same place in the same time zone as the corresponding photograph may be generated as a group according to the generated grouping criterion. Therefore, if the user sets a title for a person's name on a photograph of a specific person, the photographs including the same person as the person included in the photograph may be created as a group, The name of the person set by the user may be displayed.

The present invention described above can be implemented as computer readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). In addition, the computer may include a control unit 180 of the terminal. The foregoing detailed description, therefore, should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (13)

A memory unit for storing previously stored pictures;
A display unit for displaying image information; And
And a control unit for grouping previously stored pictures, selecting representative pictures for each group, and displaying the selected representative pictures differently according to the number of pictures per group on the display unit,
Wherein,
Generating at least one grouping criterion based on an analysis result of each of the previously stored photographs, and grouping the previously stored photographs based on the generated grouping criterion, and displaying at least one of the representative photographs on the display unit Wherein the display unit displays a graphical object related to the grouping reference of the corresponding group in the area on the screen.
The apparatus of claim 1,
Wherein the grouping criterion is generated based on an analysis result of the image objects constituting each of the previously stored photographs, and the grouping criterion is generated based on the inference result.
3. The method of claim 2,
The memory comprising:
And includes characteristic information for a plurality of predetermined specific places,
Wherein,
A background image or a foreground image of each of the pre-stored photographs is analyzed, and based on a result of comparing the feature information with a feature point of the background image or the foreground image, Wherein the location of the photographed image is inferred from any one of the photographs.
3. The apparatus of claim 2,
The image objects constituting each of the previously stored photographs are classified into figures or objects,
A result of recognizing at least one of a portrait, an expression, an age, or a dress of the person included in the photograph from the image object divided by the person, and a result of recognizing the feature of the object included in the photograph from the image object divided by the object And infer the event associated with each of the pictures based on the at least one picture.
3. The apparatus of claim 2,
Wherein in case of pictures including image objects recognizable as characters among the pre-stored pictures, an event related to the pictures is inferred based on a character recognition result of the image objects.
The method according to claim 1,
And a line of sight detection unit for detecting a line of sight of the user,
Wherein,
Wherein the at least one graphic object is displayed in an area on the display unit in which any one of the representative photographs corresponding to the user's gaze is displayed.
The method of claim 1,
Further comprising map information on a location of the specific location when the grouping criterion is related to a specific location.
Generating at least one grouping criterion based on an analysis result of each of the previously stored photographs;
Grouping the pre-stored pictures based on the generated grouping criteria;
Selecting representative photographs for each group and displaying the selected representative photographs according to the number of pictures included in the corresponding group; And
And displaying at least one graphic object related to a grouping reference of a corresponding group in an area on a display unit on which at least one representative photograph of each group is displayed.
9. The method of claim 8, wherein generating the grouping criterion comprises:
Analyzing the pre-stored photographs to deduce an event associated with each photograph or the place where each photograph was taken; And
And generating at least one grouping criterion from the pre-stored pictures based on the inference result.
10. The method of claim 9,
Separating each of the previously stored photographs into a background image and a foreground image and analyzing the feature points based on whether or not the focus is matched; And
Further comprising the step of inferring the location where the photographs were taken based on a result of comparing the feature points of the divided background image or the foreground image with feature information of a predetermined plurality of specific places, / RTI >
10. The method of claim 9,
Recognizing, for each of the previously stored photographs, a characteristic shape of a person and an object included in each photograph; And
Further comprising the step of inferring an event related to each of the previously stored pictures using at least one of the characteristic shapes of the person and the object.
12. The method of claim 11,
Wherein each of the photographs includes at least one of an attitude, a facial expression, an age range, and a dress of a person included in each of the photographs.
9. The method of claim 8, wherein displaying at least one graphic object comprises:
Recognizing the user's gaze;
Selecting any one of the representative images for each group corresponding to the line of sight of the recognized user; And
Further comprising the step of displaying the graphic object on any one of the selected representative photographs.

Images