KR20170060473A - Mobile terminal and method for controlling the same - Google Patents

Abstract

The present invention relates to a mobile terminal having a hologram output unit and a control method thereof, and more particularly, to a mobile terminal having a hologram output unit, a display unit for displaying a plurality of graphic objects, a hologram output unit for outputting a hologram object in a predetermined space, The hologram object generating unit generates a hologram object corresponding to any one of the plurality of graphic objects displayed on the display unit according to a first user input, And a controller for controlling the hologram output unit to display the generated hologram object in one area within the predetermined space spaced from the predetermined space.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a mobile terminal having a hologram output unit and a control method thereof.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal.

The functions of mobile terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, modern mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal. It is conceivable to further include a hologram output section capable of outputting the hologram as one of the structural changes and improvements.

On the other hand, such a hologram is a three-dimensional image formed of a three-dimensional image, and can be generated using the principle of holography. By using the hologram, various image information related to various functions of the mobile terminal can be displayed as a stereoscopic image, thereby providing a greater fun to the user.

In addition, a method of using various functions available in a mobile terminal more easily and conveniently by using the hologram is under active research.

It is an object of the present invention to solve the above-mentioned problems and other problems, and it is an object of the present invention to provide a mobile terminal that enables a user to more easily and conveniently control various functions executable in a mobile terminal using a hologram, And the like.

It is another object of the present invention to provide a mobile terminal and a control method thereof that enable a user to control various functions that can be executed in a mobile terminal without directly touching the mobile terminal.

According to an aspect of the present invention, there is provided a mobile terminal including a display unit displaying a plurality of graphic objects, a hologram output unit outputting a hologram object in a predetermined space, And generating a hologram object corresponding to one of a plurality of graphic objects displayed on the display unit in accordance with a first user input applied on the display unit, And a controller for controlling the hologram output unit to display the generated hologram object in one area within the predetermined space spaced apart from the display unit in accordance with the movement of the hologram.

In one embodiment, the apparatus further includes a sensing unit configured to sense a posture and a motion of a human body located in a space in which the hologram object is output, wherein the controller uses the sensing information obtained through the sensing unit, And a posture and a motion of the human body are processed as user input to the hologram object.

In one embodiment, the one graphic object includes a plurality of control functions related to functions executed in the mobile terminal, and the control unit controls the posture and the movement of the human body sensed through the sensing unit, A function of detecting a hologram object corresponding to one of the graphic objects as an input of a user and controlling a function executed in the mobile terminal according to one of the plurality of control functions according to an input of the detected user .

In one embodiment, the control unit generates a new portion of the hologram object detected by the second user input as a new hologram object when a second user input applied to the hologram object is sensed, And controls the hologram output unit to display the new hologram object in another area within the predetermined space spaced from the hologram object according to movement of a part of the user's body that has applied the user input.

In one embodiment, the controller may be configured to display the hologram object in a predetermined area of the predetermined space where the third user input is sensed when a third user input made by the user's body part is sensed, And the display of the hologram object is fixed.

In one embodiment, the control unit terminates the display of the specific hologram object when motion of a part of the user's body passing through the predetermined area of the predetermined space displayed at a speed higher than a predetermined speed is detected .

In one embodiment, when the body part of the user passes through one area in the preset space in which the specific hologram object is displayed at a speed of the predetermined speed or more, the body part of the user is moved in the predetermined space , And terminates the display of the specific hologram object according to the detection result.

In one embodiment, when the user body part is in contact with the display part through an area in the predetermined space in which a plurality of hologram objects are displayed, and at least a part of the display part is covered by the contact, The display of all of the plurality of hologram objects is ended.

In one embodiment, the controller changes a position where a specific hologram object is displayed according to a movement of the sensed user's body part, and the specific hologram object is a result of recognizing a posture taken by the user body part Accordingly, the hologram object is detected as being in contact with a specific part of the user's body part.

In one embodiment, when the position of the specific hologram object is changed, the control unit controls the display position of the other hologram object associated with the specific hologram object to be moved and displayed together with the movement of the specific hologram object And controls the hologram output unit.

In one embodiment, the control unit controls the hologram output unit such that when the object associated with a specific hologram object moves, the specific hologram object is moved and displayed together with the movement of the object, The object is located within a predetermined distance from one region of the predetermined space in which the specific hologram object is displayed.

In one embodiment, when the predetermined fourth user input is detected for the first hologram object output in the predetermined space, the controller may display any one of the plurality of graphic objects displayed on the display unit The second hologram object corresponding to the graphic object is displayed around the position where the first hologram object is displayed.

In one embodiment of the present invention, the controller may select any one of the plurality of hologram objects corresponding to one graphic object according to the first user input to one region in the preset space according to the movement of the user's body part And the plurality of hologram objects are different in shape and shape from each other.

In one embodiment, the graphic object is any one of a plurality of photographic images pre-stored in the mobile terminal, and the control unit may convert a plurality of holographic images corresponding to any one of the plurality of photographic images into the hologram object And displaying one of the plurality of holographic images in which one of the photographic images is edited according to a user input.

In one embodiment, the control unit further displays alarm information for indicating an event occurring in the mobile terminal, as an alarm hologram object, and a position at which the alarm hologram object is displayed is determined based on a predetermined criterion, And is determined according to the result of determining the importance.

In one embodiment, the control unit controls the hologram output unit such that the higher the importance of the generated event is, the more the alarm hologram object corresponding to the event is output to an area within the predetermined space corresponding to the front direction of the user's face. And a control unit.

In an exemplary embodiment, the alarm hologram object is different in size according to a position displayed in the predetermined space, and the position where the alarm hologram object is displayed is displayed smaller as the front face of the user's face is closer .

In one embodiment, the control unit determines the importance based on at least one of the type of the generated event and the information related to the generated event, and the information related to the event may include at least one of a reception of the message, The caller information of the message or the telephone or the text information included in the received message when the call is an incoming call.

In one embodiment, the apparatus further includes an output unit configured to output a feedback signal, wherein the control unit detects an approach of the human body to the hologram object based on a relative position between the hologram object and the human body And controls the output unit to transmit the feedback signal to the human body adjacent to or close to the hologram object based on the detection result.

According to an aspect of the present invention, there is provided a method of controlling a mobile terminal having a hologram output unit for outputting a hologram object in a predetermined space, the method comprising: The method comprising the steps of: sensing a first user input applied to a display unit of a terminal; generating a hologram object corresponding to one of a plurality of graphic objects displayed on the display unit according to the first user input sensed And displaying the generated hologram object in one area within the predetermined space separated from the display unit according to movement of the user's body part.

Effects of the mobile terminal and the control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, the present invention provides a method and apparatus for outputting various graphic objects related to a specific function of a mobile terminal to a hologram according to a user's selection, So that the user can more easily and conveniently control the mobile terminal.

In addition, according to at least one embodiment of the present invention, a hologram image related to a specific function of a mobile terminal is outputted at a desired position by a user, And to control the function.

1A is a block diagram illustrating a mobile terminal according to the present invention.
FIGS. 1B and 1C are conceptual diagrams illustrating an example of a mobile terminal according to the present invention in different directions.
FIGS. 2A and 2B are conceptual diagrams illustrating the implementation of the hologram image through the hologram output unit.
3A and 3B are conceptual diagrams for explaining the principle of the hologram.
4A to 4C are conceptual diagrams for explaining the transmission type hologram method.
5A to 5C are conceptual diagrams for explaining the reflective hologram method.
6 illustrates an example of providing a feedback signal to a human body approaching a hologram object.
7 is a flowchart illustrating an operation of outputting a hologram object in a mobile terminal according to an exemplary embodiment of the present invention.
8 is a conceptual diagram for explaining the operation process of FIG.
9 is a flowchart illustrating an operation of displaying a part of a currently generated hologram object as another hologram object in a mobile terminal according to an embodiment of the present invention.
10 is a conceptual diagram for explaining the operation process of FIG.
11 is a conceptual diagram for explaining a process of deleting any one of the hologram objects in the mobile terminal according to the embodiment of the present invention for displaying a plurality of hologram objects.
12 is a conceptual diagram for explaining a gesture of a user ending a hologram output in a mobile terminal according to an embodiment of the present invention for displaying at least one hologram object.
FIG. 13 is a flowchart illustrating an operation of displaying a new hologram object at a specific location where a hologram object is displayed in a mobile terminal according to an exemplary embodiment of the present invention.
14 is a conceptual diagram for explaining the operation process of FIG.
15 is a flowchart illustrating an operation of moving a position of one hologram object according to a gesture of a user in a mobile terminal according to an embodiment of the present invention for displaying at least one hologram object.
16 is a conceptual diagram for explaining the operation procedure of FIG.
17 is a flowchart illustrating an operation of displaying a pre-stored image using a holographic image in a mobile terminal according to an embodiment of the present invention.
18 is a conceptual diagram for explaining the operation process of FIG.
19 is an exemplary view showing an example in which a mobile terminal according to an embodiment of the present invention installed in a vehicle outputs a hologram object.
20 is an exemplary view illustrating an example of displaying a part of a hologram object currently generated in another mobile terminal according to an embodiment of the present invention as another hologram object.
21 is an exemplary diagram illustrating an example of changing the position of hologram objects currently generated according to the movement of a vehicle handle in a mobile terminal according to an embodiment of the present invention.
22 is a diagram illustrating an example in which the display of any one of the plurality of hologram objects displayed in the mobile terminal according to the embodiment of the present invention is completed.
23 is a diagram illustrating an example of outputting a hologram object corresponding to a specific graphic object according to a user's selection in a mobile terminal according to an exemplary embodiment of the present invention.
24 is an exemplary view illustrating an example of outputting additional information on a specific graphic object according to a user's selection as a hologram object in a mobile terminal according to an embodiment of the present invention.
25 is an exemplary view illustrating an example of changing the size of additional information about a specific graphic object displayed as a hologram object according to a gesture of a user in a mobile terminal according to an embodiment of the present invention.
26 is a diagram illustrating an example of displaying alarm information for notifying an event occurring in a mobile terminal according to a viewing angle of a user differently in a mobile terminal according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like or similar components are denoted by the same reference numerals, and redundant explanations thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

However, it will be appreciated by those skilled in the art that the configuration according to the embodiments described herein may be applied to fixed terminals such as a digital TV, a desktop computer, a digital signage, and the like, will be.

1A to 1C are block diagrams for explaining a mobile terminal according to the present invention, and FIGS. 1B and 1C are conceptual diagrams showing an example of a mobile terminal according to the present invention in different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, ), And the like. The components shown in FIG. 1A are not essential for implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than the components listed above.

The wireless communication unit 110 may be connected between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and another mobile terminal 100 or between the mobile terminal 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

In the mobile terminal according to the present invention, the camera 121 may include a 3D camera or a depth camera. In the present invention, it is possible to detect a human body located in an arbitrary space (or a predetermined space) through which a hologram image (or a hologram object) is output through an image sensor provided in a 3D camera have. Such an image sensor may be configured to sense a user's human body while sensing a user's gesture in the arbitrary space. In the mobile terminal according to the present invention, a 3D image (or a three-dimensional (3D) image) having a three-dimensional effect corresponding to a target object (or a human body) approaching the hologram image can be acquired through the 3D camera.

The sensing unit 140 may include at least one sensor for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 150 includes a display unit 151, an acoustic output unit 152, a haptrip module 153, a light output unit 154, and a hologram output unit (not shown) Or a holography module 155). The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user and may provide an output interface between the mobile terminal 100 and a user.

The interface unit 160 serves as a path to various types of external devices connected to the mobile terminal 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the mobile terminal 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a plurality of application programs or applications running on the mobile terminal 100, data for operation of the mobile terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Also, at least a part of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions (e.g., telephone call receiving function, message receiving function, and calling function) of the mobile terminal 100. Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and may be operated by the control unit 180 to perform the operation (or function) of the mobile terminal.

In addition to the operations related to the application program, the control unit 180 typically controls the overall operation of the mobile terminal 100. The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 170. FIG. In addition, the controller 180 may operate at least two of the components included in the mobile terminal 100 in combination with each other for driving the application program.

Further, the memory 170 may store information on the hologram interference pattern to support the projection of the hologram image of the hologram output unit 155. [ The controller 180 can output information visually output from the mobile terminal as a hologram image through the hologram output unit 155 using the information stored in the memory 170. [

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the components included in the mobile terminal 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170. [

Hereinafter, the various components of the mobile terminal 100 will be described in detail with reference to FIG. 1A.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The mobile communication module 112 may be a mobile communication module or a mobile communication module such as a mobile communication module or a mobile communication module that uses technology standards or a communication method (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution And an external terminal, or a server on a mobile communication network established according to a long term evolution (AR), a long term evolution (AR), or the like.

The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. The wireless Internet module 113 is configured to transmit and receive a wireless signal in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, wireless LAN (WLAN), wireless fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, DLNA (Digital Living Network Alliance), WiBro Interoperability for Microwave Access, High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) and Long Term Evolution-Advanced (LTE-A) 113 transmit and receive data according to at least one wireless Internet technology, including Internet technologies not listed above.

The wireless Internet module 113 for performing a wireless Internet connection through the mobile communication network can be used for wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE or LTE- May be understood as a kind of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short-range communication module 114 is connected to the mobile terminal 100 and the wireless communication system through the wireless area networks, between the mobile terminal 100 and another mobile terminal 100, or between the mobile terminal 100 ) And the other mobile terminal 100 (or the external server). The short-range wireless communication network may be a short-range wireless personal area network.

Here, the other mobile terminal 100 may be a wearable device (e.g., a smartwatch, a smart glass, etc.) capable of interchanging data with the mobile terminal 100 according to the present invention (smart glass), HMD (head mounted display)). The short range communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100. [ If the detected wearable device is a device authenticated to communicate with the mobile terminal 100 according to the present invention, the control unit 180 may transmit at least a part of the data processed by the mobile terminal 100 to the short- 114 to the wearable device. Therefore, the user of the wearable device can use the data processed by the mobile terminal 100 through the wearable device. For example, according to this, when a telephone is received in the mobile terminal 100, the user performs a telephone conversation via the wearable device, or when a message is received in the mobile terminal 100, It is possible to check the message.

The position information module 115 is a module for obtaining the position (or current position) of the mobile terminal, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, when the mobile terminal utilizes the GPS module, it can acquire the position of the mobile terminal by using a signal transmitted from the GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, it can acquire the position of the mobile terminal based on information of a wireless access point (AP) that transmits or receives the wireless signal with the Wi-Fi module. Optionally, the location information module 115 may perform any of the other functions of the wireless communication unit 110 to obtain data relating to the location of the mobile terminal, in addition or alternatively. The location information module 115 is a module used to obtain the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or obtains the location of the mobile terminal.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, Or a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. [ A plurality of cameras 121 provided in the mobile terminal 100 may be arranged to have a matrix structure and various angles or foci may be provided to the mobile terminal 100 through the camera 121 having the matrix structure A plurality of pieces of image information can be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to a function (or a running application program) being executed in the mobile terminal 100. Meanwhile, the microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 123 is for receiving information from a user and when the information is inputted through the user input unit 123, the control unit 180 can control the operation of the mobile terminal 100 to correspond to the input information . The user input unit 123 may include a mechanical input means (or a mechanical key such as a button located on the front, rear or side of the mobile terminal 100, a dome switch, a jog wheel, Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, And a touch key disposed on the touch panel. Meanwhile, the virtual key or the visual key can be displayed on a touch screen having various forms, for example, a graphic, a text, an icon, a video, As shown in FIG.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a corresponding sensing signal. The control unit 180 may control the driving or operation of the mobile terminal 100 or may perform data processing, function or operation related to the application program installed in the mobile terminal 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface, without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner area of the mobile terminal or in proximity to the touch screen, which is covered by the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 141 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as "proximity touch & The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 141 can detect a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. Meanwhile, the control unit 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Furthermore, the control unit 180 can control the mobile terminal 100 such that different operations or data (or information) are processed according to whether the touch to the same point on the touch screen is a proximity touch or a touch touch .

The touch sensor senses a touch (or touch input) applied to the touch screen (or the display unit 151) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, do.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like. Here, the touch controller may be a separate component from the control unit 180, and may be the control unit 180 itself.

On the other hand, the control unit 180 may perform different controls or perform the same control according to the type of the touch object touching the touch screen (or a touch key provided on the touch screen). Whether to perform different controls or to perform the same control according to the type of the touch object may be determined according to the current state of the mobile terminal 100 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 180 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 121 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to sense a touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

Also, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 153 may be vibration. The intensity and pattern of the vibration generated in the haptic module 153 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 153 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 153 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the user through the muscles of the finger or arm. The haptic module 153 may include two or more haptic modules 153 according to the configuration of the mobile terminal 100.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 100. Examples of events that occur in the mobile terminal 100 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the mobile terminal detecting the event confirmation of the user.

Next, the hologram output unit (or holography module 155) is configured to output the hologram image in a predetermined three-dimensional space. The structure of the holographic module 155 and the projection method of the holographic image using the same will be described in detail with reference to FIGS. 2A to 5B.

The interface unit 160 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data in the mobile terminal 100 to an external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, an audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 160.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the interface unit 160. [

The interface unit 160 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, And various command signals may be transmitted to the mobile terminal 100. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs the storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls the operations related to the application program and the general operation of the mobile terminal 100. [ For example, when the state of the mobile terminal meets a set condition, the control unit 180 can execute or release a lock state for restricting input of a user's control command to applications.

In addition, the control unit 180 performs control and processing related to voice communication, data communication, video call, or the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the controller 180 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the mobile terminal 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of an interface 160 through which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

Referring to FIGS. 1B and 1C, the disclosed mobile terminal 100 includes a bar-shaped terminal body. However, the present invention is not limited thereto and can be applied to various structures such as a folder type, a flip type, a slide type, a swing type, and a swivel type in which a watch type, a clip type, a glass type or two or more bodies are relatively movably coupled . A description of a particular type of mobile terminal, although relevant to a particular type of mobile terminal, is generally applicable to other types of mobile terminals.

Here, the terminal body can be understood as a concept of referring to the mobile terminal 100 as at least one aggregate.

The mobile terminal 100 includes a case (for example, a frame, a housing, a cover, and the like) that forms an appearance. As shown, the mobile terminal 100 may include a front case 101 and a rear case 102. Various electronic components are disposed in the inner space formed by the combination of the front case 101 and the rear case 102. At least one middle case may be additionally disposed between the front case 101 and the rear case 102.

A display unit 151 is disposed on a front surface of the terminal body to output information. The window 151a of the display unit 151 may be mounted on the front case 101 to form a front surface of the terminal body together with the front case 101. [

In some cases, electronic components may also be mounted on the rear case 102. Electronic parts that can be mounted on the rear case 102 include detachable batteries, an identification module, a memory card, and the like. In this case, a rear cover 103 for covering the mounted electronic components can be detachably coupled to the rear case 102. Therefore, when the rear cover 103 is separated from the rear case 102, the electronic parts mounted on the rear case 102 are exposed to the outside.

As shown, when the rear cover 103 is coupled to the rear case 102, a side portion of the rear case 102 can be exposed. In some cases, the rear case 102 may be completely covered by the rear cover 103 during the engagement. Meanwhile, the rear cover 103 may be provided with an opening for exposing the camera 121b and the sound output unit 152b to the outside.

These cases 101, 102, and 103 may be formed by injection molding of synthetic resin or may be formed of metal such as stainless steel (STS), aluminum (Al), titanium (Ti), or the like.

The mobile terminal 100 may be configured such that one case provides the internal space, unlike the above example in which a plurality of cases provide an internal space for accommodating various electronic components. In this case, a universal mobile terminal 100 in which a synthetic resin or metal is connected from the side to the rear side can be realized.

Meanwhile, the mobile terminal 100 may include a waterproof unit (not shown) for preventing water from penetrating into the terminal body. For example, the waterproof portion is provided between the window 151a and the front case 101, between the front case 101 and the rear case 102, or between the rear case 102 and the rear cover 103, And a waterproof member for sealing the inside space of the oven.

The mobile terminal 100 is provided with a display unit 151, first and second sound output units 152a and 152b, a proximity sensor 141, an illuminance sensor 142, a light output unit 154, Cameras 121a and 121b, first and second operation units 123a and 123b, a microphone 122, an interface unit 160, and the like.

1B and 1C, a display unit 151, a first sound output unit 152a, a proximity sensor 141, an illuminance sensor 142, an optical output unit (not shown) A second operation unit 123b, a microphone 122 and an interface unit 160 are disposed on a side surface of the terminal body, And a mobile terminal 100 having a second sound output unit 152b and a second camera 121b disposed on a rear surface thereof.

However, these configurations are not limited to this arrangement. These configurations may be excluded or replaced as needed, or placed on different planes. For example, the first operation unit 123a may not be provided on the front surface of the terminal body, and the second sound output unit 152b may be provided on the side surface of the terminal body rather than the rear surface of the terminal body.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

In addition, the display unit 151 may exist in two or more depending on the embodiment of the mobile terminal 100. In this case, the mobile terminal 100 may be provided with a plurality of display portions spaced apart from each other or disposed integrally with one another, or may be disposed on different surfaces, respectively.

The display unit 151 may include a touch sensor that senses a touch with respect to the display unit 151 so that a control command can be received by a touch method. When a touch is made to the display unit 151, the touch sensor senses the touch, and the control unit 180 generates a control command corresponding to the touch based on the touch. The content input by the touch method may be a letter or a number, an instruction in various modes, a menu item which can be designated, and the like.

The touch sensor may be a film having a touch pattern and disposed between the window 151a and a display (not shown) on the rear surface of the window 151a, or may be a metal wire . Alternatively, the touch sensor may be formed integrally with the display. For example, the touch sensor may be disposed on a substrate of the display or inside the display.

In this way, the display unit 151 can form a touch screen together with the touch sensor. In this case, the touch screen can function as a user input unit 123 (see FIG. 1A). In some cases, the touch screen may replace at least some functions of the first operation unit 123a.

The first sound output unit 152a may be implemented as a receiver for transmitting a call sound to a user's ear and the second sound output unit 152b may be implemented as a loud speaker for outputting various alarm sounds or multimedia playback sounds. ). ≪ / RTI >

The window 151a of the display unit 151 may be provided with an acoustic hole for emitting the sound generated from the first acoustic output unit 152a. However, the present invention is not limited to this, and the sound may be configured to be emitted along an assembly gap (for example, a gap between the window 151a and the front case 101) between the structures. In this case, the appearance of the mobile terminal 100 can be made more simple because the hole formed independently for the apparent acoustic output is hidden or hidden.

The optical output unit 154 is configured to output light for notifying the occurrence of an event. Examples of the event include a message reception, a call signal reception, a missed call, an alarm, a schedule notification, an email reception, and reception of information through an application. The control unit 180 may control the light output unit 154 to terminate the light output when the event confirmation of the user is detected.

Furthermore, a hologram output unit 155 may be provided in the terminal body. The hollogram output unit 155 may be configured to output the hologram image 155 '(see FIG. 2A) to the three-dimensional space on the front surface of the terminal body, for example, the display unit 151. In this figure, it is illustrated that the hologram output unit 155 is provided in one area of the display unit 151. In the mobile terminal according to the present invention, as shown in the drawing, the hologram output unit 155 may be arranged not only in one area of the display unit 151 but also in an arbitrary area of the main body of the terminal, The position can be changed variously according to the design.

The first camera 121a processes an image frame of a still image or a moving image obtained by the image sensor in the photographing mode or the video communication mode. The processed image frame can be displayed on the display unit 151 and can be stored in the memory 170. [

The first and second operation units 123a and 123b may be collectively referred to as a manipulating portion as an example of a user input unit 123 operated to receive a command for controlling the operation of the mobile terminal 100 have. The first and second operation units 123a and 123b can be employed in any manner as long as the user is in a tactile manner such as touch, push, scroll, or the like. In addition, the first and second operation units 123a and 123b may be employed in a manner that the user operates the apparatus without touching the user through a proximity touch, a hovering touch, or the like.

In this figure, the first operation unit 123a is a touch key, but the present invention is not limited thereto. For example, the first operation unit 123a may be a mechanical key, or a combination of a touch key and a touch key.

The contents input by the first and second operation units 123a and 123b can be variously set. For example, the first operation unit 123a receives a command such as a menu, a home key, a cancellation, a search, and the like, and the second operation unit 123b receives a command from the first or second sound output unit 152a or 152b The size of the sound, and the change of the display unit 151 to the touch recognition mode.

On the other hand, a rear input unit (not shown) may be provided on the rear surface of the terminal body as another example of the user input unit 123. The rear input unit is operated to receive a command for controlling the operation of the mobile terminal 100, and input contents may be variously set. For example, commands such as power on / off, start, end, scrolling, and the like, the size adjustment of the sound output from the first and second sound output units 152a and 152b, And the like can be inputted. The rear input unit may be implemented as a touch input, a push input, or a combination thereof.

The rear input unit may be disposed so as to overlap with the front display unit 151 in the thickness direction of the terminal body. For example, the rear input unit may be disposed at the rear upper end of the terminal body such that when the user holds the terminal body with one hand, the rear input unit can be easily manipulated using the index finger. However, the present invention is not limited thereto, and the position of the rear input unit may be changed.

When a rear input unit is provided on the rear surface of the terminal body, a new type of user interface using the rear input unit can be realized. When the first operation unit 123a is not disposed on the front surface of the terminal body in place of at least a part of the functions of the first operation unit 123a provided on the front surface of the terminal body, The display unit 151 may be configured as a larger screen.

Meanwhile, the mobile terminal 100 may be provided with a fingerprint recognition sensor for recognizing the fingerprint of the user, and the controller 180 may use the fingerprint information sensed through the fingerprint recognition sensor as authentication means. The fingerprint recognition sensor may be embedded in the display unit 151 or the user input unit 123.

The microphone 122 is configured to receive the user's voice, other sounds, and the like. The microphone 122 may be provided at a plurality of locations to receive stereophonic sound.

On the other hand, the hologram output unit 155 may be provided on the rear surface of the terminal body to output the hologram image 155 '(see FIG. 2B) in a three-dimensional space on the rear surface, as shown in the figure. In the mobile terminal according to the present invention, the hologram output unit 155 may be provided on at least one of the front surface, the side surface, and the rear surface of the main body.

The interface unit 160 is a path through which the mobile terminal 100 can be connected to an external device. For example, the interface unit 160 may include a connection terminal for connection with another device (for example, an earphone or an external speaker), a port for short-range communication (for example, an infrared port (IrDA Port), a Bluetooth port A wireless LAN port, or the like), or a power supply terminal for supplying power to the mobile terminal 100. The interface unit 160 may be implemented as a socket for receiving an external card such as a SIM (Subscriber Identification Module) or a UIM (User Identity Module) or a memory card for storing information.

And a second camera 121b may be disposed on a rear surface of the terminal body. In this case, the second camera 121b has a photographing direction which is substantially opposite to that of the first camera 121a.

The second camera 121b may include a plurality of lenses arranged along at least one line. The plurality of lenses may be arranged in a matrix form. Such a camera can be named an 'array camera'. When the second camera 121b is configured as an array camera, images can be taken in various ways using a plurality of lenses, and a better quality image can be obtained.

The flash 124 may be disposed adjacent to the second camera 121b. The flash 124 shines light toward the subject when the subject is photographed by the second camera 121b.

And a second sound output unit 152b may be additionally disposed in the terminal body. The second sound output unit 152b may implement a stereo function together with the first sound output unit 152a and may be used for implementing a speakerphone mode in a call.

The terminal body may be provided with at least one antenna for wireless communication. The antenna may be embedded in the terminal body or formed in the case. For example, an antenna constituting a part of the broadcast receiving module 111 (see FIG. 1A) may be configured to be able to be drawn out from the terminal body. Alternatively, the antenna may be formed in a film type and attached to the inner surface of the rear cover 103, or a case including a conductive material may be configured to function as an antenna.

The terminal body is provided with a power supply unit 190 (see FIG. 1A) for supplying power to the mobile terminal 100. The power supply unit 190 may include a battery 191 built in the terminal body or detachable from the outside of the terminal body.

The battery 191 may be configured to receive power through a power cable connected to the interface unit 160. In addition, the battery 191 may be configured to be wirelessly chargeable through a wireless charger. The wireless charging may be implemented by a magnetic induction method or a resonance method (magnetic resonance method).

The rear cover 103 is configured to be coupled to the rear case 102 so as to cover the battery 191 to restrict the release of the battery 191 and to protect the battery 191 from external impact and foreign matter . When the battery 191 is detachably attached to the terminal body, the rear cover 103 may be detachably coupled to the rear case 102.

The mobile terminal 100 may be provided with an accessory that protects the appearance or supports or expands the function of the mobile terminal 100. [ One example of such an accessory is a cover or pouch that covers or accommodates at least one side of the mobile terminal 100. [ The cover or pouch may be configured to interlock with the display unit 151 to expand the function of the mobile terminal 100. Another example of an accessory is a touch pen for supplementing or extending a touch input to the touch screen.

Hereinafter, the principle and method of outputting the hologram image through the hologram output unit 155 will be further described with reference to the accompanying drawings. FIGS. 2A and 2B are conceptual diagrams illustrating the implementation of the hologram image through the hologram output unit, and FIGS. 3A and 3B are conceptual diagrams illustrating the principle of the hologram. 4A to 4C are conceptual diagrams for explaining a transmission type hologram system, and FIGS. 5A to 5C are conceptual diagrams for illustrating a reflection type hologram system.

As described above, the mobile terminal according to the present invention includes the hologram output unit 155 and outputs the hologram image through the hologram output unit 155. In order to support the projection of the hologram image of the hologram output unit 155, the memory 170 may store information on the hologram interference pattern. The controller 180 can output information visually output from the mobile terminal as a hologram image through the hologram output unit 155 using the information stored in the memory 170. [

The hologram output unit 155 may be provided on at least one of a front surface, a rear surface, and a side surface of the terminal main body so as to output the hologram image 155 'in a three-dimensional space on the rear surface.

FIGS. 2A and 2B are conceptual diagrams showing a hologram image 155 'implemented through the hologram output unit 155 shown in FIGS. 1B and 1C.

Referring to FIG. 2A, as an example, the hologram output unit 155 may be disposed on the front surface of the mobile terminal 100. The hologram output unit 155 can be driven independently from the display unit 151. The hologram output unit 155 can output the hologram image 155 even when the display unit 151 does not display visual information, (155 '). Meanwhile, the hologram output unit 155 may be mounted on a bezel portion for enclosing the display unit 151.

Further, as shown in the drawing, an output section (or haptic module 153) for transmitting a feedback signal is disposed close to the position where the hologram output section 155 is disposed, and is output through the hologram output section 155 In response to the user's gesture with respect to the hologram object, a feedback signal can be delivered to the user.

On the other hand, as shown in the drawing, the light source unit for outputting light (or light ()) for outputting the hologram object and the haptic module 153 for outputting the feedback signal can be disposed independently of each other.

Meanwhile, although not shown, the light source for outputting the hologram object and the haptic module for outputting the feedback signal may be composed of the same light source. At this time, the controller 180 controls the light source for outputting the hologram object By adjusting the intensity of the laser, the feedback signal can be realized with the intensity of the laser.

2B, the hologram output unit 155 may be disposed on the rear surface of the mobile terminal 100 to output a hologram image 155 'in a predetermined three-dimensional space on the rear surface.

Dimensional space in which the hologram image output unit 155 is mounted and the hologram image 155 'is output are not limited to the above examples. The hologram output unit 155 may be configured to be rotated or pop-up, or may be detachably attached to the terminal body as a separate device. The hologram image 155 'may be output in a space independent of the installation direction of the hologram output unit 155 through tilting, a separate reflection structure, or the like.

The hologram image 155 ', which can be implemented through the hologram output unit 155, may include both a two-dimensional (2D) plane image and a three-dimensional (3D) stereoscopic image.

The planar imaging system is a monoscopic method of providing the same image in both eyes, and it is possible to arrange a polyhedron generated through one or more points, lines, faces, or a combination thereof on a virtual three-dimensional space, Thereby displaying an image.

Next, the stereoscopic method is a method of providing different images in both eyes (stereo scopic), and is a method using a principle of feeling a stereoscopic feeling when a human is looking at an object with the naked eye. In other words, the two eyes of a person see different planar images when they see the same object by the distance between them. These different planar images are transmitted to the brain through the retina, and the brain fuses them to feel the depth and reality of the stereoscopic image. Therefore, binocular disparity caused by the distance between the two eyes is a three-dimensional feeling, and there is a slight difference in each person.

The hologram image 155 'generated through the hologram output unit 155 to be described later may include both the planar image and the stereoscopic image. Hereinafter, for the sake of convenience of description, what is expressed by the planar imaging system includes a case where the stereographic imaging system is expressed.

Hereinafter, a method of representing the hologram image 155 'and a structure for implementing the hologram image 155' will be described in detail.

The image outputted through the display unit 151 records only the distribution of the light and dark planes of the object whereas the hologram image 155 'is the image that simultaneously accumulates and reproduces all the information of the wave, that is, Can be understood.

3A and 3B are conceptual diagrams for explaining the principle of the hologram.

Referring to FIG. 3A, the coherent light from the laser light source 201 is divided into two parts by a splitter 202.

One of the light beams illuminates the object 207, and the diffused light from the surface of the object 207 reaches the holographic photosensitive material 205. Hereinafter, this ray is referred to as an object wave. Mirrors 203 and 204 may be used for path switching of the light beam.

The other one of the rays is switched to the mirror 208 and diffused through the lens 206 to reach the front of the holographic photosensitive material 205 directly. Hereinafter, this light beam is referred to as a reference wave.

The object wave and the reference wave interfere with each other on the holographic photosensitive material 205 to produce a very delicate and complex interference pattern of about 500 to 1,500 per mm. The holographic storage medium on which such interference fringes are recorded is called a hologram.

3B, the interference fringe serves as a diffraction grating, so that light is diffracted at a position different from the direction in which the reference wave is incident when the interference fringe acts as a diffraction grating Diffracted, and diffracted light is gathered and formed as light generated by reflection from the initial object, thereby projecting the hologram image 209. That is, the first object light can be reproduced through the hologram and can be expressed as the hologram image 209. [

At this time, when you look inside the reproduced wavefront, the first object appears, but the object appears to be inside. When you move the viewing point, the position of the object changes and it looks as though you are viewing a stereoscopic image. In addition, since the original wavefront of the object is reproduced, it can also interfere with the wavefront from an object that is slightly deformed.

The hologram image 209 may be classified into a transmission type hologram image display method and a reflection type polarographic image display method according to a reproduction method. Figs. 4A to 4C are conceptual diagrams for explaining a transmission holographic method, and Figs. 5A to 5C are conceptual diagrams for explaining a reflection holographic method.

The transmission type hologram image representation method is a method in which the light emerging from the back of the hologram is viewed and viewed from the front of the hologram. In the transmission type hologram image representation method, the object wave and the reference wave are exposed to the holographic photosensitive material in the same direction at the time of manufacture, and the generated hologram image is characterized in that the color is clear and bright.

Referring to FIG. 4A, the light emitted from the laser light source 301 passes through a spatial filter 302 and then spreads into a smooth spherical wave. In the beam splitter 305, one of the spherical waves divided into two beams is illuminated by the object 308 to produce an object wave, and the other is illuminated to the holographic photosensitive material 307 to produce a reference wave. The illuminated object wave on the object 308 is illuminated onto the holographic photosensitive material 307. [ Mirrors 304, 306, and 309 may be used to switch the light path.

At this time, the illuminated object wave and the reference wave interfere with each other in the holographic photosensitive material 307 to form an interference fringe, and the interference fringe is recorded in the holographic photosensitive material 307.

That is, as shown in FIG. 4B, the object wave and the reference wave are projected together on the same side of the holographic photosensitive material 307 to generate an interference fringe.

Then, as shown in Fig. 4C, when a reproduction wave which is the same as the reference wave is projected onto the holographic photosensitive material 307, the object wave is transmitted in the direction opposite to the plane of the previous object wave and the reference wave incident to generate a hologram image .

Next, the reflection type hologram image display system is constructed such that the object wave and the reference wave are incident on the holographic photosensitive material in mutually opposite directions in such a manner that an image reflected by the light in front of the hologram can be observed in front of the hologram. The hologram image generated by the reflection type hologram image display method is characterized by excellent stereoscopic effect.

Referring to FIG. 5A, light emitted from a laser light source 401 passes through a spatial filter 402 and then spreads as a smooth spherical wave, and is transmitted through a beam splitter 405, One illuminated by the object 408 to produce an object wave, and the other illuminated to the holographic photosensitive material 407 to produce a reference wave. Mirrors 404, 406, and 409 may be used to switch the light path. However, unlike FIG. 4A, the reference wave and the object wave are illuminated with the holographic photosensitive material 407 at positions opposite to each other.

5B, the reference wave is projected onto the left surface of the holographic photosensitive material 407, and the object wave is projected through the right upper surface of the holographic photosensitive material 407. [ Thereafter, as shown in FIG. 5C, when the reproduction wave which is the same as the reference wave is projected onto the holographic photosensitive material 407, the object wave is transmitted in the opposite direction to generate a hologram image.

Meanwhile, the hologram image may be displayed according to the set holography pattern in accordance with an embodiment of the present invention. The holographic pattern means that the hologram image projected through the hologram output unit 155 changes in a predetermined pattern with time and is provided to the user.

The holographic pattern can be variously set in the manner described below.

First, the holographic pattern can be set by changing the distance difference between the hologram output portion and the hologram image with time. According to the above configuration, since the hologram image projected through the hologram output unit 155 can be moved up and down, a predetermined holographic pattern can be set.

Next, the holographic pattern can be set by changing the shape of the hologram image projected by the hologram output unit 155 with time. For example, the control unit 180 may control the hologram image projected from the hologram output unit 155 to have a circular shape at first, and then control the shape of the hologram image to change into a rectangular shape with time.

Also, a method of moving or rotating the projected hologram image through the hologram output unit 155 may be applied. That is, the holographic image can be set by rotating the hologram image while moving the hologram image horizontally, horizontally, or horizontally while maintaining the same distance difference between the hologram output unit 155 and the hologram image.

Further, the hologram pattern can be set by modifying the color or size of the projected hologram image with time or adjusting the hologram image to blink. Furthermore, it is possible to set the holographic pattern through projection brightness, refresh rate, illumination, vibration feedback, sound insertion, image insertion, and repeated projection.

In the above description, it is assumed that a holographic pattern is set by an individual factor, but a holographic pattern may be set by a plurality of elements. For example, it is also possible to set the holographic pattern so that the projected hologram image moves left and right while changing the distance between the hologram output unit 155 and the hologram image according to time.

In the above description, it is assumed that a holographic pattern is set for the whole hologram image. However, this is merely an example, and it is also possible that a holographic pattern is applied to only a part of the hologram image.

Although the transmission type holographic image display method and the reflection type holographic image display method have been described above, the method of outputting the hologram image may be various.

On the other hand, holograms can be classified into three types according to the method of generating and reproducing holograms. As described above, i) analog holography method such as transmission type holographic image display method and reflection type holographic image display method, and 2) 3-dimensional digital data photographed with an image sensor such as CCD camera or CMOS sensor are acquired to generate a hologram Ii) a digital holography system having a form capable of storing, processing, and editing; iii) a technique of generating a hologram effect by photographing a super-resolution stereoscopic image though not a complete hologram, (Iii) a pseudo hologram (pseudo hologram) method in which the hologram image effect is projected to imitate the hologram image effect. Examples of similar hologram systems are preformat, Peppers ghost, and Leia display system.

Examples of the 3D holography method include a Project Vermeer, a voixe box, and a 3D midair plasma display q method.

Meanwhile, in the mobile terminal according to the present invention, a hologram image can be implemented through any of the various schemes described above or in a manner other than the schemes described above.

On the other hand, such a hologram image may not only be a three-dimensional image output in an arbitrary space but also provide a user experience such that the user actually touches an object corresponding to the hologram image or the hologram image.

Hereinafter, the hologram image output through the hologram output unit 155 is collectively referred to as a 'hologram object'.

The hologram object is output through the hologram output unit 155 and may be output on a predetermined space (or a predetermined three-dimensional space). Here, the predetermined space may correspond to the projection range of the hologram output unit 155. [ Then, in a state in which the hologram object is output, a process of sensing a human body (e.g., 'hand' of a user) located in the predetermined space may be performed. Here, it is possible not to sense a human body located in the predetermined space, but to sense a human body located in a region where the hologram object is output and a peripheral region.

On the other hand, the placement position of the sensing unit 140 may be set so that the predetermined space corresponding to the projection range in which the hologram output unit 155 projects the hologram object is sensed.

Here, there may be various sensing units (or sensing means) for sensing a human body located in the predetermined space. For example, the sensing unit may be a 3D camera.

More specifically, the camera 121 may include a 3D camera or a depth camera. For example, a human body located in an arbitrary space (or a predetermined space) through which a hologram image (or a hologram object) is output through an image sensor provided in a 3D camera (depth camera) . Such an image sensor may be configured to sense a user's human body while sensing a user's gesture in the arbitrary space. (Or 3D) image corresponding to a target object (or a human body) located near the hologram object or a hologram object (or a user human body) approaching the hologram object is acquired through the 3D camera .

The coordinate information of the hologram object can be compared with the coordinate information of the human body from the image obtained through the image sensor. The control unit 180 can determine the relative position between the hologram object and the human body by comparing coordinate information between them. If the coordinate information of the human body is equal to or close to the coordinate information of the hologram object as a result of the comparison, the human body i) approaches (ii) touches, or (iii) passes through the hologram object It can be judged.

Here, the coordinate information of the hologram object and the coordinate information of the human body may be obtained through modeling for the hologram object and the human body, respectively. When the human body approach to the hologram object is detected as described above, the controller 180 controls the hologram object so that the feedback signal is transmitted to the human body adjacent to or close to the hologram object based on the detection result The feedback output unit 156 can be controlled.

Here, various examples of the feedback signal may exist. As an example, the feedback output section 156 may include at least one of a laser output module and a sonic output module. The feedback signal may include at least one of a laser output through the laser output module and a sound wave output through the sound output module. The laser or sound wave has a constant intensity (e.g., intensity or amplitude), a period (e.g., frequency), a waveform (e.g., pulse wave, sine wave, etc.). The laser or sound wave outputted through the feedback output unit 156 is transmitted to the human body through which the user can obtain an experience of actually touching the hologram object. This feedback signal may also be referred to as a " haptic signal ". It is also possible that the feedback output unit 156 comprises the haptic module 153.

FIG. 6 illustrates an example in which the feedback output unit 156 transmits a feedback signal according to the approach of the human body to the hologram object.

The control unit 180 transmits the feedback signal 620 output to the feedback output unit 156 based on the coordinate information obtained as a result of modeling the human body 610 approaching or adjacent to the hologram object 600 The coordinate information of the target point to be obtained can be obtained. The control unit 180 may use the modeled human coordinate information so that the feedback signal 620 is transmitted to the human body 610. In this case, the control unit 180 does not transmit the feedback signal 620 to all of the areas where the hologram object 600 is output, but may transmit the feedback signal 620 only at a position where the human body 610 is located, (620).

On the other hand, the hologram output unit 155 and the feedback output unit 156 may have different configurations. That is, the hologram output unit 155 can be driven independently of the feedback output unit 156. [ In this case, the hologram output unit 155 and the feedback output unit 156 may be physically separated from each other. That is, the hologram output unit 155 may be a light source such as an LED or a laser in the visible light region, and the feedback output unit 156 may include a laser output module provided separately from the hologram output unit 155, An output module, an ultrasonic output module, a wind output module, and a mist output module.

On the other hand, when the feedback output unit 156 is deactivated, the control unit 180 outputs the feedback signal 620 only when the proximity or proximity of the human body 610 to the hologram object 600 is detected The feedback output unit 156 may be activated or deactivated. In other words, the feedback output unit 156 can maintain the deactivation state at all times, and can be switched to the active state only when the feedback signal 620 is to be output, and output the feedback signal 620. Here, the deactivated state may mean an off state, and the activated state may mean an on state.

Accordingly, only when the user's access to the hologram object 600 is detected, the feedback output unit 156 controls the feedback signal 620 to be output, thereby further reducing the power consumption. In addition, by using the coordinate information of the human body 610 adjacent to or close to the hologram object 600, not only the entire region of the hologram object 600 but also the feedback signal 620 By controlling the feedback output unit 156 so as to output the feedback signal 620 to all the regions.

Hereinafter, embodiments related to a control method that can be implemented in the mobile terminal 100 configured as above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

7 is a flowchart illustrating an operation of outputting a hologram object in the mobile terminal 100 according to an embodiment of the present invention. And FIG. 8 is a conceptual diagram for explaining the operation process shown in FIG.

7, the controller 180 of the mobile terminal 100 according to the exemplary embodiment of the present invention executes a function of the mobile terminal 100 according to an input of a user, Can be displayed on the display unit 151. In this state, the controller 180 may detect whether there is an object corresponding to a predetermined user input (hereinafter, referred to as a first user input) among graphic objects displayed on the display unit 151 .

For example, the first user input may be a touch input of a predetermined user. For example, when the user applies a touch input to a point on the display unit 151 on which a specific graphic object is displayed for a preset time or longer, that is, the touch input held for a predetermined time or longer If so, it can be detected that the first user input is present.

If it is detected that the first user input is received, the controller 180 determines whether the first user input, for example, a specific graphic object displayed at the point where the touch input held for a predetermined time or longer is detected, As shown in FIG. In operation S700, the controller 180 may display the graphic object corresponding to the first user input separately from the other graphic objects. The display in step S700 may allow the user to identify that the graphic object is currently selected.

If a specific graphic object according to the first user input is selected, the controller 180 may generate a hologram object corresponding to the specific graphic object (S702). For example, the hologram object may have a shape similar to the size and shape of the specific graphic object. Alternatively, the hologram object may have a shape different from the shape of the specific graphic object. Here, the controller 180 may select a hologram object having a shape similar to that of the currently selected specific graphic object or a hologram object having a different shape from the user.

If the hologram object corresponding to the graphic object corresponding to the first user input is generated in step S702, the controller 180 determines whether the hologram object corresponding to the first user input It can detect movement. If the touch object is moving, the moved position can be tracked (S704). In the following description, it is assumed that the touch object is a human body, that is, a finger of a user.

The movement detection and position tracking of the user's finger may be performed through various sensing means. That is, as described above, the image can be sensed using an image sensed through a 3D camera or a depth camera, or can be sensed using a shape of the user's finger based on a time when infrared rays or light are reflected on the user's finger, And three-dimensional spatial coordinate information of the user's fingertip can be sensed.

When the coordinate information of the user's fingertip is sensed, the controller 180 can output the hologram object generated in step S702 to one point in the three-dimensional space corresponding to the coordinate information of the user's fingertip sensed (S706). When a first user input is applied to a specific graphic object displayed on the display unit 151 and a finger of a user who has applied the first user input is moved, A hologram object may be displayed.

Meanwhile, the specific graphic object represented by the hologram object may include a plurality of graphic objects corresponding to various control functions for controlling functions currently performed in the mobile terminal 100. [ In this case, the hologram object may be generated in a form including portions corresponding to each of the plurality of graphic objects, and may be generated at a position in the three-dimensional space corresponding to the fingertip of the user. In this case, the controller 180 may detect a part of the hologram object corresponding to the position of the user's body part (for example, the finger of the user) among the respective parts included in the hologram object, The mobile terminal 100 can control the function currently being executed according to the control function corresponding to the part.

Here, each part of the hologram object corresponding to each of the plurality of objects can be generated and output as a different partial hologram object. In this case, the control unit 100 detects a user's input to one of the partial hologram objects, and controls the current mobile terminal 100 (100) according to a control function corresponding to any one partial hologram object ) Can be controlled. Herein, the input of the user may be such that the user's finger is detected in any one of the three-dimensional spatial areas in which the partial hologram objects are displayed.

When the body part of the user is detected in one area of the three-dimensional space in which the hologram object is displayed, the controller 100 controls the feedback output unit 156 to allow the user to identify the hologram object with a tactile sense. To provide a feedback signal to a part of the body of the detected user.

FIG. 8 shows an example in which a hologram object is displayed according to the operation procedure.

8, the first drawing of FIG. 8 shows a case where the user's finger 810 is positioned at a point where the first graphic object 800 displayed on the display unit 151 of the mobile terminal 100 is displayed And the touch input is applied. In this case, the controller 180 may check whether the touch input corresponds to the first user input. If the touch input corresponds to a predetermined first user input, a hologram object 850 corresponding to the first graphic object 800 may be generated as in the description of FIG.

Meanwhile, in this state, when the user's finger 810 moves, that is, as shown in the second drawing of Fig. 8, the user's finger 810 moves from the display portion 151 to the distance d1 If so, the control unit 180 may sense the movement of the user's finger 810 and sense the moved position, as in the description of FIG. 7 described above. The generated hologram object 850 can be displayed at the sensed position.

8, the hologram object 850 is moved by d1 from the display unit 151 according to the position of the user's finger 810 spaced from the display unit 151 by d1 It can be displayed at a spaced position.

If the hologram object is displayed according to the position of the finger of the user, the controller 180 may detect whether the second user input has been applied (S708). Here, unlike the first user input for selecting any graphic object to be displayed as a hologram object, among the graphic objects displayed on the display unit 151, the second user input is a position of the currently displayed hologram object As shown in Fig. For example, the control unit 180 may detect that the user has applied the second user input when the moved touch object, that is, the user's finger does not move for more than a predetermined time.

If the second user input is detected as a result of step S708, the control unit 180 may fix the display position of the hologram object at step S710. Here, the fixed position of the hologram object is fixed, which means that the position at which the hologram object is output is fixed by the coordinates at the time when the second user input is detected. Accordingly, if the display position of the hologram object is fixed in step S710, the hologram object can be displayed in a specific three-dimensional space (the position at which the second user input is sensed) regardless of the movement of the user's finger.

If it is determined in step S708 that the second user input is not detected, the controller 180 may detect the movement of the user's finger in step S704. And can track the location of the moving user's finger. Then, in step S706, the hologram object generated in step S702 may be output to one point on the three-dimensional space corresponding to the moved user's fingertip. Then, the process proceeds to step S708, where it is detected whether or not there is a second user input.

Accordingly, until the second user input is detected, the controller 180 continuously detects the position of the user's finger and displays the generated hologram object at one point on the three-dimensional space corresponding to the user's fingertip sensed have. Therefore, in the mobile terminal 100 according to the embodiment of the present invention, the position at which the hologram object is displayed can be changed in accordance with the movement of the finger that applies the user input. Accordingly, the user can display the hologram object at a desired position by moving his / her finger to a specific position to display the hologram object.

Here, the first graphic object 800 may include a plurality of control functions for controlling functions performed in the current mobile terminal 100, as shown in FIG. 8, the hologram object 850 corresponding to the first graphic object 800 may be divided into a plurality of parts for each control function included in the first graphic object 800, . ≪ / RTI > Alternatively, the control unit 180 may generate and display a hologram object (partial hologram object) for each of the control functions. In this case, the hologram object 810 may be composed of the partial hologram objects.

When the user's finger is sensed in one area of the three-dimensional space in which the hologram object 850 is displayed, the controller 180 displays a portion of the hologram object 810 displayed at the position where the user's finger is sensed, It is possible to control a function currently executed in the mobile terminal 100 according to a control function corresponding to an object.

Meanwhile, the controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention may divide a part of the currently displayed hologram object into a new hologram object, and display the hologram object at another position desired by the user.

9 is a flowchart illustrating an operation of displaying a part of a currently generated hologram object as another hologram object in the mobile terminal 100 according to the embodiment of the present invention. FIG. 10 is a conceptual diagram for explaining the operation procedure of FIG.

9, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention can detect a user input that is input again to the displayed hologram object in a state in which the hologram object is displayed (S900). For the sake of understanding, the hologram object 850 shown in FIG. 8 will now be described.

For example, the user input displayed on the hologram object may be the first user input. That is, the first user input used to select a specific graphic object to be generated as a hologram object on the display unit 151 is used again to generate a new hologram object (hereinafter referred to as a second hologram object) in the currently generated hologram object It is possible.

That is, the first user input may be a specific graphic object displayed on the display unit 151 for generating a new hologram object, or a user input for selecting a part of the hologram object currently displayed. However, the present invention is not limited thereto. That is, the input of a user for selecting a specific graphic object on the display unit 151 and the input of a user for selecting some of the currently displayed hologram objects may be different from each other. However, in the following description, it is assumed that the user's input for selecting a specific graphic object on the display unit 151 and the user's input for selecting some of the currently displayed hologram objects are the same first user input Assumption will be described below.

If the input of the first user to the part of the currently displayed hologram object is sensed again in step S900, the controller 180 converts a part of the hologram object corresponding to the currently sensed first user input into another part of the hologram object (S902). ≪ / RTI > For example, assuming that a hologram object 850 is displayed, as shown in the first drawing of FIG. 10, the controller 180 determines whether the hologram object 850 is displayed in the center of the hologram object 850 (E.g., when the user's finger 810 is detected for a predetermined time or longer in the portion where the object 1000 is displayed). In this case, the control unit 180 divides the graphic object 1000 into a central portion (an object 1000) of the hologram object 850 as shown in the first drawing of FIG. 10, and another portion of the hologram object 850 As shown in FIG. Here, the object 1000 may be a partial hologram object constituting the hologram object 850 or the hologram object 850 may be a plurality of pieces of control information included in the first graphic object 800 It may be part of the part.

When a part of the hologram object (for example, the object 1000 in the center part) is selected, the controller 180 can generate a new hologram object corresponding to the selected part (hereinafter, referred to as a second hologram object) (S904). Then, the user's finger movement and the position of the moved finger can be sensed (S906). In operation S908, the second hologram object generated in step S904 may be displayed at one point on the three-dimensional space corresponding to the position of the moved user finger.

10, when the finger 810 of the user who has applied the first user input to the part (the object 1000) of the hologram object 850 moves, The second hologram object 1010 corresponding to the object 1000 selected by the first user input may be displayed at one point on the three-dimensional space according to the movement of the finger 810. [

In this state, the control unit 180 may again detect the user's second user input (S910). Here, the second user input may be for fixing the position of the currently displayed second hologram object 1010 similar to that described with reference to FIG. Accordingly, when the second user input is applied again, the controller 180 sets the coordinates of the second hologram object 1010 displayed on the second hologram object 1010 when the second user input is applied again, Can be fixed to the coordinates on the three-dimensional space displayed. The second hologram object 1010 can be displayed at a position on the specific three-dimensional space (the position at which the second user input is sensed) regardless of the movement of the user's finger 810. [

On the other hand, if it is determined in step S910 that the second user input is not sensed, the controller 180 may detect the movement of the user's finger in step S906. And can track the location of the moving user's finger. In step S908, the second hologram object 1010 generated in step S904 may be output to one point on the three-dimensional space corresponding to the moved user's fingertip. Then, the process proceeds to step S910 to detect whether there is a second user input.

Accordingly, until the second user input is sensed, the controller 180 continuously detects the position of the user's finger and displays the second hologram object 1010 at one point in the three-dimensional space corresponding to the user's fingertip sensed Can be displayed. Therefore, in the mobile terminal 100 according to the embodiment of the present invention, only a part of the currently displayed hologram object can be generated as a new hologram object, and displayed at a desired position by the user.

9 to 10, a portion of the currently displayed hologram object (hereinafter, referred to as a first hologram object) is displayed as a new hologram object (second hologram object). Here, a part of the first hologram object generated by the new hologram object (second hologram object) can be variously determined. For example, when the first hologram object includes a menu corresponding to a plurality of different functions, at least one menu according to the first user input may be generated as the second hologram object. Or a part of the image information represented by the first hologram object may be generated as the second hologram object. Alternatively, a part of the hologram image constituting the first hologram object may be generated as the second hologram object (for example, when the first hologram object is divided into a plurality of parts according to the user's selection).

On the other hand, the display of the hologram objects may be terminated or the display of the entire hologram objects may be terminated according to the user's selection. The input of the user for terminating the display of any one of the hologram objects and the input of the user for ending the display of the entire hologram objects may be different from each other. 11 and 12 are intended to illustrate the input of such a user.

11 is a conceptual diagram for explaining a process of deleting one hologram object in the mobile terminal 100 according to the embodiment of the present invention for displaying a plurality of hologram objects.

Referring to FIG. 11, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention can selectively terminate the display of any one of the hologram objects in a state in which a plurality of hologram objects are displayed. For example, as shown in the first drawing of FIG. 11, in a state in which the first hologram object 1100 and the second hologram object 1110 are displayed, a predetermined user's gesture is displayed for which hologram object The display of any one of the hologram objects may be selectively terminated.

Here, the gesture of the predetermined user may be a specific hologram object passing at a speed higher than a predetermined speed. That is, as shown in the first diagram of FIG. 11, when the user's hand passes through one of the hologram objects (the first hologram object 1120) at a speed higher than a predetermined speed, And can detect the input of the user to terminate the display of the first hologram object. Accordingly, the controller 180 may terminate the display of the first hologram object 1120 and display only the second hologram object 1110, as shown in the second diagram of FIG.

On the other hand, unlike what is shown in Fig. 11, the gesture of a particular user may be to end the display of the entire hologram objects. FIG. 12 shows an example in which the display of the holograms displayed on the mobile terminal 100 is completed according to the gesture of the user.

For example, the user's gesture for terminating the display of the entire hologram objects may be a gesture that the user covers the mobile terminal 100 with the palm of his hand. 12, when the user's hand 1200 passes through the plurality of holographic objects 1210 and 1220 while a plurality of hologram objects 1210 and 1220 are being displayed, (When the user's hand touches the display unit 151 and the user's hand covers at least a part of the display unit 151), the control unit 180 may sense the hologram objects 1210 and 1220 as a gesture of the user to terminate the display of the entire hologram objects 1210 and 1220.

Here, the controller 180 can detect the position of the user's hand and the distance between the mobile terminals 100 using a 3D camera, a depth camera, an infrared sensor, or a laser sensor provided in the mobile terminal 100 And may sense that the user's hand approaches the mobile terminal 100 according to the change of the detected position and distance. Also, the controller 180 can sense the palm of the user covering the display unit 151 using an illuminance sensor, a proximity sensor, and the like provided in the mobile terminal 100.

If the gesture of the user that covers the display unit 151 of the mobile terminal 100 is detected through the plurality of holographic objects 1210 and 1220 as described above, May be detected as a gesture of the user to terminate the display of the entire objects 1210 and 1220. Accordingly, as shown in the second diagram of FIG. 12, the controller 180 may terminate all the display of the hologram objects 1210 and 1220.

13 is a flowchart illustrating an operation of displaying a new hologram object at a specific position of a hologram object in the mobile terminal 100 according to an embodiment of the present invention. And FIG. 14 is a conceptual diagram for explaining the operation process of FIG.

Referring to FIG. 13, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention displays the input of a user for one hologram object in a state in which at least one hologram object is displayed Can be detected. Here, the input of the user to any one of the hologram objects may be performed by a gesture different from the touch input or the gesture corresponding to the first and second user inputs. Accordingly, in order to distinguish the input of a user applied to one of the hologram objects from the first and second user inputs, it is referred to as a third user input.

For example, the third user input may be a gesture in which a specific hologram object contacts a predetermined number of times or more within a predetermined time. Or the third user input may be a gesture in which the angle at which the user's finger touches the specific hologram object changes to an angle in a predetermined direction. That is, when the finger of the user is sensed in one region in which the specific hologram object is displayed, or in a boundary of one region in which the hologram object is displayed, the controller 180 determines whether a user touches the specific hologram object As a result. When the angle between the tip of the detected finger and the finger is changed to a predetermined angle (for example, a predetermined angle in the upward direction (the direction opposite to the direction in which the gravitational force is directed) or the downward direction (the direction in which the gravitational force is directed) The user may sense that the third user input is applied to the specific hologram object.

The controller 180 may display the specific hologram object sensed as being applied with the third user input to be distinguished from other hologram objects so that the user can recognize that the third user input is applied (S1300).

Meanwhile, the control unit 180 may sense an input of a user applied on the display unit 151 of the mobile terminal 100 (S1302). The input of the user sensed in step S1302 may be a user input such as the first user touch input. That is, when the touch input is applied to one point on the display unit 151 and the state in which the touch input is maintained is continuously maintained for a predetermined time or more, the control unit 180 determines that there is an input from the user in step S1302 Can be detected. The controller 180 determines whether any one of the graphic objects corresponding to the input of the user detected in step S1302 among the graphic objects displayed on the display unit 151 of the mobile terminal 100 is the user It can be recognized as being selected by the input. In this case, it is needless to say that the controller 180 can display any one of the graphic objects separately from the other graphic objects.

Then, the controller 180 may generate a hologram object (third hologram object) corresponding to the currently selected graphic object (S1304). In operation S1306, the generated third hologram object may be displayed at a position adjacent to a position where the specific hologram object sensed as the third user input is displayed. For example, the controller 180 may display the third hologram object around the specific hologram object, or display the third hologram object at a position where the specific hologram object is displayed. When the third hologram object is displayed at the position where the specific hologram object is displayed, the controller 180 displays only the third hologram object, so that the third hologram object is displayed at a position where the specific hologram object is displayed It may indicate that it is being displayed.

14, the first diagram of FIG. 14 illustrates a first graphic object 1400 displayed on the display unit 151 of the mobile terminal 100 and a second graphic object 1400 displayed on the display unit 151 of the mobile terminal 100. [ 2 graphic object 1450 is displayed, a first hologram object 1410 corresponding to the first graphic object 1400 is displayed at a position in a specific space. In this state, the controller 180 may sense a user input (third user input) applied to the first hologram object 1410, as shown in the second diagram of FIG. In this case, the first hologram object 1410 may be the 'specific hologram object' of FIG.

In this state, as shown in the second drawing of FIG. 14, the control unit 180 can sense the touch input of the user applied on the display unit 151. And a time when the touch input of the user is continued at a point on the display unit 151 on which the second graphic object 1450 is displayed is longer than a preset time. The controller 180 may recognize that the second graphic object 1450 is selected by the user. In this case, the controller 180 may display the second graphic object 1450 as a color or a separate graphic object to indicate that the second graphic object 1450 is currently selected.

If the second graphic object 1450 is selected as described above, the controller 180 may generate a third hologram object 1460 corresponding to the second graphic object 1450. The third hologram object 1460 may be displayed at a position adjacent to the position where the specific hologram object 1410 is displayed. Here, the adjacent position may be a position on the same three-dimensional spatial coordinate, or only one of three axes (X, Y, Z axis) constituting the three-dimensional spatial coordinate may be different. That is, the third hologram object 1460 may be displayed only at a position where the hologram object is displayed from the display unit 151 at a position where the specific hologram object 1410 is displayed. The third diagram in Fig. 14 shows this example.

13, when the third hologram object is displayed at a position adjacent to the specific hologram object, the controller 180 determines whether the hologram object corresponding to the third user input (i.e., the specific hologram object 1410) ) Or the third hologram object 1460 (step S1308). The input of the user may include a gesture of a user contacting the specific hologram object 1410 or the third hologram object 1460 for a predetermined time or more. That is, when a predetermined time has elapsed in a state where the user has touched a finger with either the specific hologram object 1410 or the third hologram object 1460, the controller 180 determines whether the specific The hologram object 1410 or the third hologram object 1460 is input by the user.

In step S1310, the controller 180 may move the display position of the specific hologram object 1410 or the third hologram object 1460. That is, the controller 180 may change the position of any one of the hologram objects detected by the user in step S1308 to a position corresponding to the movement of the user's finger applied with the input of the user.

14, in the state where the third hologram object 1460 is displayed at the position where the first hologram object 1410 is displayed, when the user presses the third hologram object 1460, The control unit 180 may determine that the third hologram object 1460 is selected by the user when applying the user input in step S1308. In this case, the controller 180 may display the third hologram object 1460 as a separate hologram object or color to indicate that the third hologram object 1460 is currently selected.

If the user moves the finger while the third hologram object 1460 is selected by a finger for inputting a user input, the controller 180 controls the third hologram object 1460 according to the movement of the finger You can also change the displayed location. 14, the third hologram object 1460 can be displayed at the position where the finger is moved.

Meanwhile, when a predetermined user's gesture is sensed, the controller 180 of the mobile terminal 100 according to the exemplary embodiment of the present invention displays a position where at least one hologram object corresponding to the sensed user's gesture is displayed, Based on the gesture of the user.

FIG. 15 is a flowchart illustrating an operation procedure of moving a position of one hologram object according to a gesture of a user in a mobile terminal 100 according to an embodiment of the present invention, which displays at least one hologram object. And FIG. 16 is a conceptual diagram for explaining the operation process of FIG.

Referring to FIG. 15, the controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention can detect whether a portion of a user's body detected is in a specific posture. If the body part of the user takes a predetermined specific posture, it can be detected that the input of the user for changing the display position of the hologram object (hereinafter referred to as the fourth user input) is applied.

When the fourth user input is sensed, the controller 180 may select at least one hologram object corresponding to the fourth user input among currently displayed hologram objects. For example, the hologram object corresponding to the fourth user input may be determined according to a specific posture of a part of the user's body corresponding to the fourth user input. For example, when the user takes a posture for picking up an object with at least two fingers, the control unit 180 can recognize the posture taken by the body part of the user with reference to the predetermined modeling data. When the posture is recognized, the corresponding hologram object can be selected according to the recognized posture. For example, the controller 180 may determine that a hologram object in contact with a specific part of a part of the user's body (e.g., a user's fingertip) that has taken the recognized attitude is selected by the user. In addition, the selected hologram object can be displayed separately according to the recognized posture (S1500)

That is, as described above, when the recognized posture is 'posture of picking up objects with at least two fingers', the controller 180 determines that the hologram object corresponding to the 'at least two fingers' As shown in FIG.

16, in the state where the hologram object 1410 is being displayed, as shown in the first drawing of Fig. 16, when the user inserts the hologram object 1410 into two If the posture is the same as that of picking up a finger, the control unit 180 can recognize the posture taken by the user's hand 1600, that is, the posture of picking up the object with two fingers. And a specific hologram object may be selected based on the two fingertips recognized in the user's hand 1600 according to the recognized posture.

16, when the user's hand 1600 takes the same attitude as picking up an object with two fingers, the control unit 180 controls the position of the fingertip in the position corresponding to the two fingertips Based on this, a hologram object can be selected. 16, when the hologram object 1410 is displayed at a position corresponding to the fingertip of the user's hand 1600, the controller 180 determines whether the hologram object 1410 is displayed in the It can be recognized that the user's hand 1600 is selected by the posture taken.

Then, the controller 180 may display to the user that the hologram object 1410 has been selected. For example, as shown in the third drawing of FIG. 16, the controller 180 displays the hologram object 1410 in a specific color or displays a separate hologram object to indicate that the hologram object 1410 is selected Can be displayed.

If a specific hologram object is selected according to the attitude (fourth user input) of a portion of the user's body, the controller 180 detects a body part of the user who has applied the fourth user input, i.e., can do. If the movement of the user's hand is detected, the moved position may be sensed (S1502).

Meanwhile, if the movement of the user's hand is detected in step S1502, the controller 180 may move the currently selected hologram object according to the movement of the user's hand. That is, the controller 180 may display the selected hologram object at one point on the three-dimensional space corresponding to the position of the moved user's body part (the part of the body applying the fourth user input) (S1504).

In this state, the controller 180 may detect whether there is a user input for fixing the display of the hologram object (S1506). For example, the user's input for fixing the display of the hologram object 1410 may be such that a portion of the user's body does not move for a certain period of time during movement, such as the second user input described above. Accordingly, if it is determined in step S1506 that the hand 1600 of the user is not moving for a predetermined period of time, the controller 180 can detect that the second user input is applied. In this case, the display position of the selected hologram object 1410 may be fixed according to the attitude of the user's hand 1600, and may be displayed at a specific position regardless of whether the user's hand 1600 is moved or not have.

On the other hand, if it is determined that the second user input is not detected as a result of step S1506, the controller 180 proceeds to step S1504 to detect movement of the user's hand 1600. [ The selected hologram object 1410 can be displayed according to the moved position. Then, the process proceeds to step S1506 to detect whether there is a second user input. Accordingly, until the second user input is detected, the controller 180 continuously detects the position of the user's hand 1600, and detects the position of the user's hand 1600 1410).

The third diagram of FIG. 16 and the fourth diagram of FIG. 16 show an example in which the selected hologram object 1410 is moved as described above.

16, when the user presses the hand 1600 in a state in which the hologram object 1410 is selected according to the attitude of the user's hand 1600, which seems to pick up the hologram object 1410, The controller 180 may display the currently selected hologram object 1410 at the moved position according to the movement of the user's hand 1600. [

The display position of the hologram object 1410 may be fixed if the user does not move for a predetermined period of time or more at a position desired by the user. Accordingly, the selected hologram object 1410 may be displayed separately from the user's hand 1600, as shown in the fourth diagram of FIG.

15, the input of the user for fixing the display of the hologram object, that is, the second user input, is a state in which the user's hand 1600 has not moved for a predetermined time or longer However, it goes without saying that the input for fixing the display of the hologram object may have various other inputs. For example, when the user takes a posture for picking up the hologram object, that is, a posture for picking up an object with two fingers, the controller 180 fixes the display of the hologram object when the posture is released The user's input is detected. Here, the 'attitude is solved' may mean that the attitude of the user's hand is no longer identified as an attitude similar to the attitude input for selecting the hologram object. Or when the user's hand takes a predetermined other posture, it may be detected that there is an input for fixing the display of the hologram object.

In the above description, a specific graphic object displayed on the display unit 151 is displayed as a hologram object. However, the mobile terminal 100 according to the embodiment of the present invention can display the pre- Of course, be displayed. In the following description, a hologram object that displays a specific graphic object as a stereoscopic image and a hologram image in which the pre-stored image is displayed on a three-dimensional space in order to distinguish the pre- Quot;

17 is a flowchart illustrating an operation of displaying a previously stored image using a holographic image in the mobile terminal 100 according to an embodiment of the present invention. 18 is a conceptual diagram for explaining the operation process of FIG.

17, the controller 180 of the mobile terminal 100 according to the exemplary embodiment of the present invention may generate a holographic image of any one of images stored in advance according to the user's selection S1700). The generated holographic image may be displayed in a three-dimensional space on the front or rear surface of the predetermined mobile terminal 100 or a specific position according to the user's selection (for example, in a region where the user's specific gesture is sensed) ).

Here, the 'holographic image' may be an image in which the pre-stored image is displayed on a three-dimensional space. That is, if the pre-stored image is an image such as a photograph displayed on the display unit 151, the holographic image may display the photographic image in a specific area on the three-dimensional space.

Meanwhile, when the user edits a photographic image, the controller 180 of the mobile terminal 100 according to an embodiment of the present invention can compare the photographic image edited using the holographic image with the original image . For this, the controller 180 may generate a plurality of holography images corresponding to any one of the photographic images selected by the user in step S1700. In step S1702, the plurality of generated holography images may be displayed. In the following description, it is assumed that a plurality of holographic images are generated and displayed for one selected image.

Accordingly, in step S1702, the controller 180 may generate and display a plurality of holographic images corresponding to any one of the photographic images selected by the user. In this case, the control unit 180 generates hologram objects corresponding to the functions (e.g., functions in the 'editing tool') capable of editing the image (hereinafter, Objects) (S17040) in order to allow a user to edit any one of the images using the editing tool objects.

In step S1706, the controller 180 may edit the second holographic image based on the editing tool objects and a user's input to one of the holographic images (hereinafter referred to as a second holographic image). For example, in step S1706, when one of the editing tool objects is selected by the user, based on the function corresponding to the selected editing tool object and the gesture of the user applied to the second holographic image, You can edit the holography image and display the edited state.

Accordingly, the user can confirm the state of the current image edited by comparing the second holographic image with another holographic image through the holographic image displayed on the three-dimensional space. If there is an input of a preset user, the control unit 180 may store the edited image or replace the original image with the edited image (S1710).

FIG. 18 illustrates an example in which a selected image is edited by a user using a holographic image according to the operation procedure of FIG.

First, FIG. 18 shows an example in which a plurality of holographic images corresponding to any one image selected by the user are displayed in step S1702. For the sake of convenience, the lower holographic image 1800 of the plurality of holographic images is referred to as a first holographic image 1800, and the holographic image displayed above the first holographic image 1830 is referred to as a second holographic image 1810. In the vicinity of the second holographic image 1810 out of the first and second holographic images 1800 and 1810, hologram objects corresponding to different functions for editing the image, i.e. editing tool objects 1850, 1852, 1854) is displayed.

For example, the editing tool objects 1850, 1852, and 1854 may include a cut function (a first editing tool object 1850), a memo input function (a second editing tool object 1852), a color change function Third editing tool object 1854). In this state, if the user touches the user's finger with the first editing tool object 1850, as shown in the second drawing of FIG. 18, the control unit 180 determines that the detected three- And the first editing tool object 1850, the user may sense that the first editing tool object 1850 is selected according to a result of comparing the areas of the three-dimensional space in which the first editing tool object 1850 is displayed. In this case, as shown in the second diagram of FIG. 18, the controller 180 may change the color or the like of the first editing tool object 1850 to indicate the selected state by the current user.

18, when a specific editing tool object is selected, the control unit 180 controls the editing function corresponding to the selected editing tool object such that the user who is applied to the second holographic image 1810 Based on the gesture of the user. 18, if the currently selected editing tool function is the cropping function, the controller 180 determines whether the second holographic image 1810 is displayed in a region on the three-dimensional space where the second holographic image 1810 is displayed The area for the cropping function can be set based on the gesture of the user.

That is, as shown in the third drawing of FIG. 18, when the first editing tool object 1850 corresponding to the 'cut function' is selected and the user selects the third holographic image 1810 If a gesture 1860 corresponding to a pinch-out is taken for an area on the dimensional space, the control unit 180 determines whether the gesture 1860 of the user corresponding to the pinch- It can be detected by the user's input for setting the area. Thus, as shown in the third diagram of FIG. 18, a region 1862 corresponding to the pinch-out gesture 1860 can be displayed in the second holographic image 1810.

In this state, when the user completes the user input, the controller 180 edits the image according to the currently set editing function, and displays the state of the edited image as a holographic image. That is, when the area 1862 according to the user's pinch-out gesture 1860 is set, as shown in the third drawing of FIG. 18, the control unit 180 controls the currently selected editing tool object 1850), the second holographic image 1810 can be edited. That is, the controller 180 may 'cut' only a portion of the second holographic image 1810 corresponding to the set area 1862.

The controller 180 of the mobile terminal 100 according to the embodiment of the present invention can display the edited holographic image. That is, as shown in the fourth diagram of FIG. 18, only the portion corresponding to the set region 1862, which is 'cut out' from the second holographic image 1810, is displayed as the second holographic image 1810 . Accordingly, the mobile terminal 100 according to the embodiment of the present invention displays the selected original image (pre-edited image) through the first holographic image 1800, and displays the edited image through the second holographic image 1810 The user can compare the original image and the edited image using a plurality of holographic images.

Meanwhile, in the above description, the controller 180 may detect whether the user has completed the input in various ways in a state where the area corresponding to the gesture of the user is displayed on the second holographic image. For example, the control unit 180 can detect that the input of the user is completed when the movement of the user's finger taking the gesture is stopped for a predetermined time or more. Or the control unit 180 may detect that the user's input is completed when the user's finger is no longer detected in the area on the three-dimensional space where the second holographic image is displayed. Alternatively, the control unit 180 may detect that the user's input is completed when there is another gesture of the predetermined user (for example, a gesture for touching the other finger with respect to the set area 1862).

In the above description, the mobile terminal 100 according to the embodiment of the present invention generates a hologram object according to a user's selection and displays the generated hologram object at a position desired by the user, or displays the generated hologram object The terminating operations have been described in detail with reference to a plurality of flowcharts and conceptual diagrams.

In the following description, the mobile terminal 100 according to the embodiment of the present invention is implemented in the form of navigation provided in the vehicle, thereby generating the hologram object, changing the position where the generated hologram object is displayed Examples of terminating the display of the hologram object will be described in more detail with reference to the drawings.

19 shows an example in which the mobile terminal 100 according to the embodiment of the present invention provided in a vehicle outputs a hologram object according to a user's selection.

Referring to FIG. 19, the controller 180 of the mobile terminal 100 may sense a touch input of a user on the display unit 151 in a state where the controller 180 of the mobile terminal 100 is installed in the vehicle. 7 and 8, the graphic object 1900 corresponding to the touch input among the graphic objects currently displayed on the display unit 151 is displayed separately in accordance with the touch input can do. This may be to indicate to the user that the selected graphic object is the current hologram object.

In this state, the controller 180 may generate a hologram object 1910 corresponding to the graphic object 1900 corresponding to the touch input of the user. When the finger 1920 of the user who has applied the touch input moves, the generated hologram object 1910 may be output to the coordinates on the three-dimensional space corresponding to the end of the user finger 1920 being moved. 19, when the user moves the finger 1920 around the handle 1950 of the vehicle, the hologram object 1910 moves along the moved finger 1920 to the handle (not shown) 1950). ≪ / RTI >

In this case, the controller 180 may change the size of the hologram object 1910 according to the position of the hologram object 1910. That is, the controller 180 may output the hologram object 1910 so that the distance between the mobile terminal 100 and the position at which the hologram object 1910 is displayed becomes longer. Accordingly, as shown in the second drawing of FIG. 19, the size of the hologram object 1910 generated around the handle may be displayed larger than when the mobile terminal 100 is created.

It should be noted that the size of the hologram object 1910 may be changed according to other gestures of the user. For example, when the user takes a pinch-out gesture using at least two fingers in the region of the three-dimensional space in which the hologram object 1910 is displayed, the controller 180 detects the size of the hologram object 1910, It can be big. On the contrary, when the user takes a pinch-in gesture using at least two fingers in the region of the three-dimensional space in which the hologram object 1910 is displayed, the controller 180 calculates the size of the hologram object 1910 .

FIG. 20 illustrates an example of displaying a part of a currently generated hologram object as another hologram object in the mobile terminal 100 according to the embodiment of the present invention.

As described above, the controller 180 can generate a new hologram object corresponding to a part of the currently displayed hologram object based on the input of the user. Then, the generated new hologram object can be displayed at a position according to the user's selection.

First, referring to the first drawing of FIG. 20, the first drawing of FIG. 20 shows an example in which a specific hologram object (hereinafter referred to as a first hologram object 2000) is generated around the handle according to the user's selection. In this state, the controller 180 may sense a user input (e.g., the first user input in FIG. 7) applied to the first hologram object 2000.

That is, when the user moves the finger 2050 in a three-dimensional space in which the first hologram object 2000 currently displayed by the user is displayed, for example, as shown in the second drawing of FIG. 20, 180 may detect that the user's finger 2050 is in contact with the first hologram object 2000, i.e., the user is touching the first hologram object 2000. In this case, the control unit 180 controls the feedback output unit 156 to output a feedback signal at a position on the three-dimensional space of the sensed user's finger 2050, It is possible to make the user feel tactile.

In this state, if the time when the user's finger 2000 is in contact with a part of the first hologram object 2000 is equal to or longer than a preset time, the control unit 180 outputs the input of the user to the first hologram object 2000 2000) to generate a new hologram object (second hologram object). Here, the second hologram object may be a part of the first hologram object 2000 corresponding to the finger 2050 of the user.

Accordingly, the control unit 180 divides a part of the first hologram object 2000 corresponding to the input of the currently-applied user (the contact of the finger 2050 for a predetermined time or more) to another part of the first hologram object 2000 As shown in FIG. For example, the control unit 180 may display a part of the first hologram object 2000 in a different color or may display another hologram object. The third diagram in FIG. 20 shows an example in which a part 2010 of the first hologram object 2000 is displayed separately. Accordingly, the user can recognize that a portion of the first hologram object 2000 is selected.

Meanwhile, the controller 180 may detect whether the position of the finger 2050 of the user who selected a portion of the first hologram object 2000 is moved. 20, when a user's finger 2050 moves to another position around the handle, a new hologram object corresponding to a part 2010 of the first hologram object 2000, that is, The second hologram object 2060 can be displayed at a position where the user's finger 2050 is moved. In this state, the controller 180 may fix the display position of the second hologram object 2060 according to the user's selection (e.g., when the second user input of FIG. 7 is detected). 20, a second hologram object 2060 corresponding to a part of the first hologram object 2000 as well as the first hologram object 2000 are arranged in different three-dimensional Can be displayed in space.

On the other hand, the displayed hologram objects can be changed in any position according to the gesture of the user similar to that shown in Figs. 15 and 16 above. For example, the control unit 180 can identify a part of the body of the user who takes a specific posture (posture in which the object is picked up), similarly to the case described with reference to Figs. 15 and 16 above. At least one hologram object selected according to the specific posture may be displayed in a different position in accordance with movement of a part of the user body taking the specific posture.

As described above, the position of the hologram objects can be changed according to movement or movement of not only a user's gesture but also other related hologram objects or related specific objects. Here, the associated other hologram objects may be preset to be associated with each other according to a corresponding function or a user's selection. Or the association may be determined according to the position at which the hologram object is displayed. That is, a specific object or another hologram object located in the vicinity of the hologram object may be the related specific object or another related hologram object.

FIG. 21 is an exemplary diagram illustrating an example in which the position of the hologram objects generated according to the movement of the vehicle handle is changed in the mobile terminal 100 according to the embodiment of the present invention installed in the vehicle.

21, the first diagram of FIG. 21 illustrates a case where the hologram objects 2110 and 2120 are disposed around a specific object, i.e., the vehicle handle 2100, as shown in FIGS. 19 to 20, That is, a state in which a part of the vehicle is in contact with the vehicle handle 2100. In this case, the vehicle handle 2100 may be an object located within a certain distance from the area of the three-dimensional space in which the hologram objects 2110 and 2120 are displayed, and may be the related specific object.

In this case, as shown in FIGS. 19 and 20, the controller 180 displays the hologram objects 2110 and 2120 at positions shown in the first drawing of FIG. 21 according to the moved user's finger position .

In this state, the controller 180 can sense the movement of the vehicle handle 2100. [ For example, the control unit 180 can detect the rotation of the handle 2100 in accordance with the rotation of the cam shaft connected to the vehicle handle 2100 from a control unit (not shown) of the vehicle. Alternatively, the controller 180 may detect the movement of the handle 2100, that is, the rotation of the handle 2100, using a camera provided in the mobile terminal 100.

When the rotation of the handle 2100 is sensed, the controller 180 controls the hologram objects 2110 and 2120, which are displayed in contact with the handle 2100, It can be displayed at the moved position. That is, the controller 180 may display the hologram objects 2110 and 2120 at positions shifted according to the rotated angle of the handle 2100.

For example, when the handle 2100 is rotated, the controller 180 may change the display position of the current hologram objects 2110 and 2120 along the rotated handle 2100. That is, the control unit 180 detects the positions of the areas where the current hologram objects 2110 and 2120 are displayed on the handle 2100 and detects the position of the hologram objects 2110 and 2120 on the handle 2100 in a manner similar to the above- It is possible to track the position of the moved regions according to the rotation. For example, the control unit 180 may use the image of the handle 2100 sensed through the camera of the mobile terminal 100 to track the position. That is, the control unit 180 may track the positions of the regions in which the positions of the regions are shifted by the angle at which the handle 2100 is rotated along the periphery of the handle 2100. The position where the hologram objects 2110 and 2120 are displayed can be changed according to the tracked position.

Accordingly, the mobile terminal 100 according to the embodiment of the present invention may be configured such that when the handle 2100 is rotated, as shown in the second drawing of FIG. 21, the hologram objects 2110, and 2120 can be moved. Therefore, even if the handle 2100 is rotated, the state in which the hologram objects 2110 and 2120 are displayed in a specific area of the handle 2100 can be maintained.

21, the hologram objects 2110 and 2120 are moved together with movement of a specific object located in a three-dimensional space. However, It goes without saying that the objects may be moved together according to the movement of not only a specific object but also other related hologram objects.

For example, the controller 180 may be configured to move the hologram objects 2110 and 2120 together according to the user's selection. In this case, if any one of the hologram objects 2110 and 2120 is moved according to the gesture of the user similar to that described with reference to Figs. 15 and 16, another hologram object may be displayed on any one of the moved hologram objects So that they can be moved together.

22 shows an example in which, in the mobile terminal 100 according to the embodiment of the present invention, the display of one of the plurality of hologram objects currently displayed is finished according to the gesture of the user It is.

22 is a block diagram illustrating a configuration of a mobile terminal 100 according to an exemplary embodiment of the present invention. Referring to FIG. 22, a first hologram object 2210, a second hologram object 2220, ) Is shown. In this case, the hologram objects 2210 and 2220 may be generated in the hologram output area 2200 generated around the hologram output unit 155 of the mobile terminal 100.

In this state, the controller 180 may apply a user input for terminating the display of any one of the hologram objects. For example, the input of such a user may be a gesture such that a part of a user's body (e.g., a user's hand) passes through a specific hologram object at a speed of at least a certain speed, as described with reference to FIG.

Accordingly, as shown in the second diagram of FIG. 22, if the first hologram object 2210 detects movement of the user's hand passing through one area of the three-dimensional space displayed at a speed higher than a predetermined speed , The controller 180 may terminate the display of the first hologram object 2210 according to the movement of the user's hand. In this case, as shown in the third diagram of FIG. 22, the controller 180 can output only the second hologram object 2220.

On the other hand, the controller 180 may further use the hologram output area 2200 to more clearly detect the gesture of the user who ends the display of the hologram object.

That is, when the motion of the user's hand passing through a region of the three-dimensional space in which the specific hologram object is displayed at a speed higher than a predetermined speed is detected, the control unit 180 moves the user's hand out of the predetermined hologram output region 2200 It can detect more whether or not it is. In a case where a hand of a user who has passed through a region of a three-dimensional space in which the specific hologram object is displayed at a speed of a predetermined speed or more deviates from the preset hologram output region 2200, May be terminated.

This is because when the user controls the hologram object and controls such as passing through another hologram object or moving the control object hologram object at a speed higher than a certain speed, it is necessary to control the gesture of the user for controlling the hologram object, And to distinguish a user's gesture for ending the display of the hologram object.

Meanwhile, in the explanation of the above-mentioned 22, the display of the hologram object is terminated according to the gesture of the user. However, the display of all the currently displayed hologram objects may be terminated according to the user's specific gesture Of course. That is, as shown in FIG. 12, when a gesture of a user who passes through a plurality of hologram objects and covers at least a part of the display unit 151 by hand is detected, the control unit 180 controls the display unit 151 to display all hologram objects It can be detected by the user's input. Accordingly, the controller 180 may simultaneously display the second hologram object 2220 as well as the first hologram object 2120 based on the gesture of the user.

Meanwhile, according to the above description, the mobile terminal 100 according to the embodiment of the present invention may determine the type of the hologram object to be displayed according to the user's selection. FIG. 23 is an exemplary view illustrating an example of outputting a hologram object corresponding to a specific graphic object according to a user's selection in the mobile terminal 100 according to the embodiment of the present invention.

23 shows an example in which a graphic object to be generated as a hologram object is selected from the graphic objects displayed on the display unit 151 of the mobile terminal 100. As shown in the first drawing of FIG. 23, when the user's touch input is applied on the display unit 151 for a predetermined time or longer (for example, the first user input in FIG. 7) It can be determined that the user has selected to display the graphic object 2300 as a hologram object based on the touch input of the user.

23, the control unit 180 displays graphic objects 2310 and 2320 indicating the shapes of pre-stored hologram objects corresponding to the graphic object 2300 on the display unit 151, Can be displayed on the screen. The user can recognize which type of hologram object has been selected based on the user's touch input to the area on the display unit 151 on which the graphic objects 2310 and 2320 are displayed. 23, when a touch input is detected in an area on the display unit 151 corresponding to the graphic object 2310 having a stick shape among the graphic objects 2310 and 2320, 180 may recognize that the user has selected a hologram object corresponding to the stick-shaped graphic object 2310. [

Then, the controller 180 may display the hologram object 2350 corresponding to the selected graphic object 2310 on the three-dimensional space, as shown in the third drawing of FIG. In this case, the position on the three-dimensional space where the hologram object 2350 is displayed may be determined according to the movement of the user's finger, similarly to the above.

On the other hand, the controller 180 can display graphic objects corresponding to different shapes of hologram objects on the display unit 151 as shown in the second drawing of FIG. . For example, the predetermined graphic object may correspond to a plurality of different types of hologram objects by a user or according to a predetermined setting.

Alternatively, when a specific graphic object is selected by the preset touch input, the controller 180 may detect that there is a user's input for selecting the shape of the hologram object. For example, the predetermined touch input may be a rubbing of a point where the touch input is applied in a specific direction. In this case, when the user touches the touch input point at a predetermined time while applying the touch input over a predetermined time on the display unit 151, , Graphic objects corresponding to hologram objects having different shapes may be displayed on the display unit 151. [

In the above description, only a specific graphic object displayed on the display unit 151 is displayed as a hologram object. Alternatively, the display unit 151 may display various graphics related to a specific graphic object displayed on the display unit 151 The additional information may be displayed in the form of the hologram object. For example, as described above, when the mobile terminal 100 according to the embodiment of the present invention is implemented in the form of navigation and provided in a vehicle, various additional information represented by the hologram object may be related to a specific feature It can be various information. 24 to 25 illustrate examples of such cases.

24, an example in which additional information on a specific graphic object according to a user's selection is outputted as a hologram object in the mobile terminal 100 according to an embodiment of the present invention, It is.

As described above, when the mobile terminal 100 according to the embodiment of the present invention is implemented in the form of navigation and provided in the vehicle, the display unit 151 displays map information such as that shown in the first drawing of Fig. Can be displayed. 24, when the user touches one point on the display unit 151, the controller 180 displays the map information corresponding to one point at which the touch input is sensed The feature item 2400 can be sensed as being selected according to the touch input.

According to the selection of the user, the controller 180 may display various additional information as a hologram object. For example, as shown in the second drawing of FIG. 24, the controller 180 may generate and display a hologram object 2410 including address information of the selected feature item 2400 according to the touch input of the user have. In this case, the hologram object 2410 may be displayed in a three-dimensional space in front of or behind the predetermined mobile terminal 100. Alternatively, the hologram object 2410 may be displayed at a position designated by a user, for example, a position corresponding to a user's finger or a body part of a user taking a specific posture.

On the other hand, the additional information may include not only address information but also various information. For example, the controller 180 may display the image object 2420 associated with the feature 2400 as additional information related to the feature 2400, as shown in the third diagram of FIG. 24, as a hologram have.

Alternatively, the control unit 180 may provide a menu that provides various convenient functions related to the currently selected feature item 2400 as a hologram object. That is, as shown in the fourth diagram of FIG. 24, the control unit 180 may include a telephone function 2436 that can perform telephone communication with a telephone number corresponding to the feature item 2400, A menu object 2430 including a payment function 2434 capable of performing a related payment function may be generated as a hologram and displayed. 24, the menu object 2430 includes a 'favorites' function 2432 for storing information on the feature item 2400 selected by the current user as a favorite place of the user, as shown in the fourth drawing of FIG. ).

Meanwhile, the additional information shown in the fourth drawing in the second drawing of FIG. 24 is an example according to the embodiment of the present invention, but the additional information provided in the present invention is not limited thereto. For example, when a specific building is selected by the user, the controller 180 may provide various other additional information corresponding to the particular building, in addition to the above-mentioned information.

On the other hand, as shown in any one of the second to fourth figures in FIG. 24, if any one of the additional information about the feature selected by the user is displayed as a hologram object, The displayed additional information may be changed to other information. For example, when a user takes a gesture, for example, a gesture that pushes or hits the currently displayed hologram object in the left or right direction, for the hologram object containing the additional information (for example, A swipe gesture), and may detect other gestures and display other additional information.

That is, when the hologram object (address information hologram object 2410) shown in the second drawing of FIG. 24 is displayed and the swipe gesture applied in the right direction is detected, A hologram object (image hologram object: 2420) shown in the drawing can be displayed. If the swipe gesture applied in the left direction is detected in this state, the hologram object (address information hologram object 2410) shown in the second drawing of FIG. 24 can be displayed again.

On the other hand, if a swipe gesture applied in the right direction is detected while the hologram object (image hologram object 2420) shown in the third drawing of FIG. 24 is displayed, the hologram object Menu hologram object: 2430). If the swipe gesture applied in the right direction is detected again in this state, the controller 180 may display the hologram object (address information hologram object 2410) shown in the second drawing of FIG. 24 again.

Meanwhile, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention not only displays different information according to the gesture of the user, but also may change the size of the hologram object currently displayed according to the gesture of the user . FIG. 25 shows an example in which, in the mobile terminal 100 according to the embodiment of the present invention, a size of additional information about a specific graphic object displayed as a hologram object is changed according to a gesture of a user.

As described above, when the mobile terminal 100 according to the embodiment of the present invention is driven by navigation, the display unit 151 may display the map information as shown in the first drawing of FIG. 25, when the user touches one point on the display unit 151, the controller 180 displays the map information corresponding to one point at which the touch input is sensed The feature item 2400 can be sensed as being selected according to the touch input. Then, the controller 180 may display the additional information related to the currently selected feature item 2400 as a hologram object. The second diagram of FIG. 25 shows an example in which the image of the selected feature item 2400 is displayed as a hologram object 2420 as an example of such additional information.

In this state, the controller 180 may detect a user's gesture for changing the size of the hologram object 2420. [ For example, such a user's gesture may be a pinch-in or pinch-out gesture of the user applied on the three-dimensional space in which the hologram object 2420 is displayed.

25, in the area on the three-dimensional space where the hologram object 2420 is displayed, when the user takes a pinch-out gesture, the control unit 180 determines that the gesture of this user is a gesture, The size of the hologram object 2420 can be sensed for enlarging. Accordingly, the controller 180 can enlarge the size of the hologram object 2420 based on the applied pinch-out gesture as shown in the fourth diagram of FIG.

Conversely, a pinch-in gesture may be to reduce the size of the currently displayed hologram object. Accordingly, when the user's pinch-in gesture is sensed in the specific area of the three-dimensional space where the hologram object is currently displayed, the controller 180 can reduce the size of the hologram object currently displayed.

Meanwhile, the controller 180 of the mobile terminal 100 according to the embodiment of the present invention may determine the size of the hologram object according to the position of the hologram object, as well as the predetermined gesture of the user. For example, if the hologram object is displayed at a position where the hologram object can be more easily recognized by the user, the controller 180 may display the hologram object at a position that can not be easily recognized by the user, Can be displayed in a larger size. The position that the user can more easily recognize or the position that can not be easily recognized can be determined according to the field of view (FOV) of the user.

The viewing angle, that is, the FOV, refers to a range of a visual field that the user can observe with the naked eye, and may have a range of about 140 degrees for each of the eyes of the user. Accordingly, the front portions of the user's eyes are overlapped with the viewing angles of the both eyes, so that the front portion can be a viewable range of view in both eyes of the user. Thus, the frontal portion of the user may be a field of view in which the user can more easily perceive the change.

Conversely, a portion of the left or right portion of the user may be a viewable range of view only with the left or right eye. This is because the binocular field of view does not completely coincide with the distances between the eyes of the user and the position of the eyes. Accordingly, the portion that can be observed only with the left eye or the right eye can be a view range in which it is difficult for the user to easily recognize the change.

Accordingly, the entire viewing angle that can be observed by the user in both eyes is determined by the viewing range (the portion where the viewing angles of the two eyes overlap each other) easily recognizable by the user and the viewing range that the user can not easily recognize A non-overlapping portion).

FIG. 26 is an exemplary view illustrating an example in which alarm information for notifying an event occurring in the mobile terminal 100 is displayed differently according to the viewing angle of the user in the mobile terminal 100 according to an embodiment of the present invention.

For example, the mobile terminal 100 according to an exemplary embodiment of the present invention may display alarm information for notifying the occurrence of a preset event to the hologram object (hereinafter, referred to as an alarm hologram object) can do. However, in this case, as described above, the alarm hologram object is not generated according to the user's selection, and thus the display position may not be determined. Accordingly, the control unit 180 can display the alarm hologram object at a predetermined position. The predetermined position may be a position in the three-dimensional space included in the viewing angle of the user.

That is, when the alarm information is generated, the controller 180 can acquire the user's facial image using the camera of the mobile terminal 100 or the like. Using the acquired facial projection, the direction of the face of the user can be detected, and the range of the three-dimensional space included in the viewing angle of the user can be calculated from the sensed direction.

The first diagram of FIG. 26 shows a range of the three-dimensional space included in the viewing angle of the user calculated through this process. Here, the first area 2630 including the front direction of the user may be an area where the viewing angles of the user's eyes overlap each other. On the other hand, the second area 2610 may be an area included only in the viewing angle of the left eye of the user's binocular. In addition, the third region 2630 may be an area in which only the right eye is included in the viewing angle of the user's binocular. Accordingly, the first area 2630 may be a region where the user can more easily recognize a change occurring in the corresponding area, and the second area 2610 and the third area 2620 may be areas where the user can recognize It may be an area where it is difficult to easily recognize a change that occurs.

Meanwhile, the control unit 180 of the mobile terminal 100 according to the embodiment of the present invention may display the alarm hologram object using the difference of the user's perception angle according to the viewing angle. For example, when an event that is determined to be important according to a preset reference is generated, the controller 180 may output the alarm hologram object corresponding to the event to the first region 2630, have. On the other hand, when it is determined that the event generated in the mobile terminal 100 is not important according to a predetermined criterion, the second region 2610 or the third region 2610, in which it is difficult for the user to easily recognize the alarm hologram object corresponding to the event, And outputs it to the memory 2620.

Here, the criterion for determining whether the event is important may be variously determined. For example, the controller 180 can determine whether the event is important or not according to the type of the event. That is, the control unit 180 can determine that an important event has occurred only when the type of the generated event is important.

Alternatively, the determination of the importance may be determined according to the information related to the generated event. For example, when a message or a call incoming event occurs, the control unit 180 may determine whether it is an important event based on the sender information of the message or the call. That is, the control unit 180 can determine that an important event has occurred only when the message is a message received from a predetermined sender or an incoming call.

Alternatively, the control unit 180 may determine, in the case of message reception, whether the reception of the message corresponds to an important event from the text of the received message. That is, if the received message includes a predetermined string such as 'important', 'contact', 'emergency', etc., the controller 180 determines that reception of the received message corresponds to an important event can do.

Meanwhile, the controller 180 may change the sizes of the alarm hologram objects according to their positions. For example, the control unit 180 may display the alarm hologram object displayed in the first area 2630 including the front direction of the user, rather than the alarm hologram object displayed in the second area 2610 or the third area 2620 The size can be made smaller. 26, the alarm hologram object 2650a displayed in the second area 2610 or the third area 2620 displays its size in a larger size, and the first area (the second area) The size of the alarm hologram object 2650a displayed on the display unit 2630 can be reduced. This is because the first area 2630 includes the front direction of the user, and thus there is a danger of obscuring the front view of the user.

In the above description, the user's body part (e.g., a finger) touches the hologram object as a user's input to the hologram object, and the controller (e.g., 180 detects the hologram object to which the user's input is applied.

When a user's body part is in contact with the hologram object, the controller 180 may output a feedback signal to a part of the body of the user to interact with the hologram object, to be. That is, in the above description, when the user's finger is detected in the area where the hologram object is displayed, or in the vicinity of the area where the hologram object is displayed, the feedback output unit 156 outputs the tactile sense corresponding to the hologram object It is possible to output a specific feedback signal corresponding to the hologram object. Accordingly, when the user touches a specific hologram object or takes a specific gesture (for example, a gesture as if picking up with a finger), the controller 180 controls the hologram object Of course, it is also possible to feel the tactile sense of touching the object directly.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). Also, the computer may include a control unit 180 of the terminal. The foregoing detailed description, therefore, should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (20)

A display unit displaying a plurality of graphic objects;
A hologram output unit configured to output a hologram object in a predetermined space;
Generating a hologram object corresponding to one of a plurality of graphic objects displayed on the display unit according to a first user input applied on the display unit,
And a controller for controlling the hologram output unit to display the generated hologram object in one area of the predetermined space spaced from the display unit in accordance with movement of a part of the user's body applying the first user input The mobile terminal comprising: 3. The method of claim 2,
Further comprising a sensing unit configured to sense a posture and a motion of a human body located in a space in which the hologram object is output,
Wherein,
And processing the human body posture and movement as user input to the hologram object using the sensing information obtained through the sensing unit. 3. The method of claim 2,
The graphical object of any one of the preceding claims,
A plurality of control functions related to functions executed in the mobile terminal,
Wherein,
Wherein the control unit senses the posture and movement of the human body sensed through the sensing unit as a user's input to the hologram object corresponding to any one of the graphic objects,
And controls functions executed in the mobile terminal according to any one of the plurality of control functions according to an input of the detected user. The apparatus of claim 3,
When a second user input applied to the hologram object is sensed, a part of the hologram object in which the second user input is sensed is generated as a new hologram object,
And controls the hologram output unit to display the new hologram object in another area in the predetermined space spaced apart from the hologram object according to movement of a part of the user's body applying the second user input. . 3. The apparatus of claim 2,
And fixing the display of the hologram object so that the hologram object is displayed in one area within the predetermined space in which the third user input is sensed when a third user input made by the user's body part is detected . 3. The apparatus of claim 2,
Wherein the display of the specific hologram object is terminated when movement of a part of the user's body passing through a predetermined area of the predetermined space in which the specific hologram object is displayed is detected at a speed higher than a predetermined rate. 7. The apparatus of claim 6,
Further detecting whether a part of the body of the user is out of the predetermined space when a part of the body of the user passes through one area in the predetermined space in which the specific hologram object is displayed at a speed of the predetermined speed or more, And terminates display of the specific hologram object according to the detection result. 3. The apparatus of claim 2,
When a part of the user's body passes through an area in the predetermined space in which a plurality of hologram objects are displayed and contacts the display part and at least a part of the display part is covered by the contact, the display of all of the plurality of hologram objects is ended Wherein the mobile terminal is a mobile terminal. 3. The apparatus of claim 2,
Changing a position at which the specific hologram object is displayed according to the movement of the sensed user's body part,
The specific hologram object may include:
Wherein the hologram object is a hologram object that is detected as being in contact with a specific part of the user's body part according to a result of recognizing a posture taken by the user body part. 10. The apparatus according to claim 9,
And controls the hologram output unit so that the display position of another hologram object associated with the specific hologram object is moved and displayed together with the movement of the specific hologram object when the position of the specific hologram object is changed, Mobile terminal. The apparatus of claim 1,
When the object associated with a specific hologram object moves, the hologram output unit is controlled so that the specific hologram object is moved and displayed together with the movement of the object,
The associated object may be,
Wherein the predetermined hologram object is located within a predetermined distance from one area of the predetermined space in which the specific hologram object is displayed. The apparatus of claim 1,
When a predetermined fourth user input is detected for the first hologram object output in the preset space,
Wherein the display unit displays a second hologram object corresponding to one of the plurality of graphic objects displayed on the display unit around a position where the first hologram object is displayed. The apparatus of claim 1,
Displaying one of a plurality of hologram objects corresponding to one graphic object according to the first user input in one area within the predetermined space according to movement of a part of the user's body,
Wherein the plurality of hologram objects include:
Wherein the shape and shape of the mobile terminal are different from each other. The method according to claim 1,
Wherein the graphic object comprises:
Wherein the image data is one of a plurality of photographic images previously stored in the mobile terminal,
Wherein,
Displaying a plurality of holographic images corresponding to any one of the plurality of photographic images as the hologram object,
Wherein one of the plurality of holography images displays a state in which any one of the photographic images is edited according to a user input. The apparatus of claim 1,
Further displaying alarm information for indicating an event occurring in the mobile terminal as an alarm hologram object,
The location where the alarm hologram object is displayed,
Wherein the determination is made according to a result of determining the importance of the event according to a predetermined criterion. 16. The apparatus of claim 15,
Wherein the control unit controls the hologram output unit such that an alarm hologram object corresponding to the generated event is output to an area within the predetermined space corresponding to a front direction of the user's face. 17. The apparatus of claim 16, wherein the alarm hologram object comprises:
The sizes are different according to positions displayed in the preset space,
Wherein the position where the alarm hologram object is displayed is displayed smaller as the position of the alarm hologram object is closer to the front direction of the user's face. 17. The apparatus of claim 16,
Determines the importance based on at least one of the type of the generated event and the information related to the generated event,
The information related to the event may include,
Wherein the event information includes at least one of caller information of the message or the telephone or text information included in the received message when the event is a reception of a message or an incoming call of a telephone. The method according to claim 1,
Further comprising an output section configured to output a feedback signal,
Wherein,
Detecting an approach of the human body to the hologram object based on a relative position between the hologram object and the human body,
And controls the output unit so that the feedback signal is transmitted to the human body adjacent to or close to the hologram object based on the detection result. And a hologram output unit outputting a hologram object in a predetermined space,
Sensing a first user input applied on a display portion of the mobile terminal by a body part of the user;
Generating a hologram object corresponding to one of the plurality of graphic objects displayed on the display unit according to the sensed first user input; And
And displaying the generated hologram object in one area within the predetermined space spaced apart from the display unit according to movement of a part of the user's body.

Images