KR101667722B1 - Information displaying apparatus and method thereof - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information display apparatus and method for a mobile terminal capable of effectively displaying a 3D augmented reality image or a 2D augmented reality image. An information display apparatus of a mobile terminal according to embodiments of the present invention includes a camera; A display unit; When a 2D content display request is received while displaying a 3D (three-dimensional) augmented reality image including a real image photographed by the camera and a virtual image or information on a single display area of the display unit, And a controller for converting the 2D augmented reality image into a two-dimensional augmented reality image, dividing a single display area of the display unit into a multi-display area, and displaying the 2D augmented reality image and the 2D content together in the multi- have.

Description

TECHNICAL FIELD [0001] The present invention relates to an information display apparatus and a method thereof,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information display apparatus and method for a mobile terminal.

Generally, an information display device of a mobile terminal according to the related art has taken an image or a moving image and displayed the photographed image or moving image on a display.

It is an object of the present invention to provide an information display apparatus and method for a mobile terminal capable of effectively displaying a three-dimensional (3D) augmented reality image or a two-dimensional augmented reality image.

An information display apparatus of a mobile terminal according to embodiments of the present invention includes a camera; A display unit; When a 2D content display request is received while displaying a 3D (three-dimensional) augmented reality image including a real image photographed by the camera and a virtual image or information on a single display area of the display unit, And a controller for converting the 2D augmented reality image into a two-dimensional augmented reality image, dividing a single display area of the display unit into a multi-display area, and displaying the 2D augmented reality image and the 2D content together in the multi- have.

In one embodiment of the present invention, the controller divides a single display area of the display unit into a first display area and a second display area, displays the 2D augmented reality image in the first display area, Area to display the 2D content.

According to an embodiment of the present invention, when the input requesting the 3D augmented reality image is received while displaying the 2D augmented reality image and the 2D content in the multi display area, And displays the 3D augmented reality image in a single display area of the display unit.

As an example of the present invention, the controller may convert the 3D augmented reality image into the 2D augmented reality image according to the direction of the mobile terminal.

As an example related to the present invention, when the direction of the mobile terminal is switched to the second direction during the display of the 3D augmented reality image in the first direction of the mobile terminal, And displays the converted 2D augmented reality image on the display unit in a second direction of the mobile terminal.

According to an embodiment of the present invention, when the direction of the mobile terminal is switched to the second direction while displaying the 2D augmented reality image in the first direction of the mobile terminal, And displays the converted 3D augmented reality image on the display unit in a second direction of the mobile terminal.

As an example related to the present invention, the controller may display the 2D augmented reality image on the display unit while the direction of the mobile terminal is switched.

According to an embodiment of the present invention, when the direction of the mobile terminal is being switched to the second direction while the 3D augmented reality image is displayed in the first direction of the mobile terminal, the controller displays the 2D augmented reality image on the display unit Can be displayed.

In one embodiment of the present invention, the controller adjusts the size of the 3D augmented reality image to match the second direction of the mobile terminal when the direction of the mobile terminal is switched to the second direction, And display the adjusted 3D augmented reality image on the display unit in a second direction of the mobile terminal.

A method of displaying information on a mobile terminal according to embodiments of the present invention includes displaying a three-dimensional (3D) augmented reality image including a real image photographed by a camera and a virtual image or information on a single display area of a display unit Wow; Converting the 3D augmented reality image into a 2D (two-dimensional) augmented reality image when a 2D content display request is received while displaying the 3D augmented reality image in the single display area; Dividing a single display area of the display into a multi-display area; And displaying the 2D augmented reality image and the 2D content together in the multi-display area.

The apparatus and method for displaying information of a mobile terminal according to embodiments of the present invention can effectively remove fatigue due to a difference between 3D and 2D by simultaneously displaying a 3D augmented reality image and a 2D content in a multi- It is effective.

The apparatus and method for displaying information of a mobile terminal according to embodiments of the present invention may convert a 3D augmented reality image into a 2D augmented reality image according to a direction of the mobile terminal and display the converted augmented reality image, It is possible to quickly display an augmented reality image in accordance with the direction of the image.

An apparatus and method for displaying information of a mobile terminal according to embodiments of the present invention include a step of performing 2D enhancement when a direction of the mobile terminal is being changed in a different direction (for example, By displaying the real image on the display unit 403, there is an effect that the augmented reality image can be continuously displayed on the display unit 403 even when the direction of the mobile terminal is changed.

1 is a block diagram of a mobile communication terminal to which an information display apparatus according to embodiments of the present invention is applied.
2 is a configuration diagram illustrating an information display apparatus of a mobile terminal according to a first embodiment of the present invention.
3 is a flowchart illustrating a method of displaying information of a mobile terminal according to a first embodiment of the present invention.
4 is a view illustrating an example of a 3D augmented reality image displayed on a display unit according to the first embodiment of the present invention.
5 is a diagram illustrating an example of 2D augmented reality images and 2D contents displayed on a display unit according to the first embodiment of the present invention.
6 is a flowchart illustrating a method of displaying information of a mobile terminal according to a second embodiment of the present invention.
FIG. 7 is an exemplary view illustrating a 3D augmented reality image displayed on a display unit according to a second embodiment of the present invention.
8 is a diagram illustrating an example of a 2D augmented reality image displayed on a display unit according to a second embodiment of the present invention.
9 is a flowchart illustrating a method of displaying information of a mobile terminal according to a third embodiment of the present invention.
10 is an exemplary view illustrating a 3D augmented reality image and a 2D augmented reality image displayed on a display unit according to a third embodiment of the present invention.

Hereinafter, an information display apparatus and method of a mobile terminal capable of effectively displaying a 3D augmented reality image or a 2D augmented reality image will be described with reference to Figs. 1 to 10. Fig.

FIG. 1 is a configuration diagram illustrating a configuration of a mobile communication terminal 100 to which an information display device according to embodiments of the present invention is applied. The mobile communication terminal (mobile phone) 100 may be implemented in various forms. For example, the mobile communication terminal 100 includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), and a PMP (Portable Multimedia Player).

1, the mobile communication terminal 100 includes a wireless communication unit 110, an A / V (audio / video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150 A memory 160, an interface unit 170, a control unit 180, a power supply unit 190, and the like. All of the components of the mobile communication terminal 100 shown in FIG. 1 are not essential components, and the mobile communication terminal 100 may be implemented by more components than the components shown in FIG. 1, The mobile communication terminal 100 may be implemented by a component.

The wireless communication unit 110 may include at least one component for performing wireless communication between the mobile communication terminal 100 and the wireless communication system or wireless communication between the mobile communication terminal 100 and a network on which the mobile communication terminal 100 is located . For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114 and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may be a server for generating and transmitting broadcast signals and / or broadcast-related information, or a server for receiving previously generated broadcast signals and / or broadcast-related information and transmitting the broadcast signals and / or broadcast- related information to the mobile communication terminal 100 . The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112. The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above. The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data related to transmission and reception of a voice call signal, a video call signal, and / or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile communication terminal 100 or externally. Here, the wireless Internet technology includes a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a WiMAX, a high speed downlink packet access (HSDPA) Can be used.

The short-range communication module 114 means a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, etc. may be used as the short distance communication technology.

The position information module 115 is a module for confirming or obtaining the position of the mobile terminal (capable of checking the position of the vehicle when the mobile terminal is mounted on the vehicle). One example is the GPS (Global Position System) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information indicated by latitude and longitude. For example, the GPS module can accurately calculate the current position according to the triangle method by measuring the precise time and distance from three or more satellites and measuring three different distances. A method of obtaining distance and time information from three satellites and correcting the error by one satellite may be used. In particular, the GPS module can acquire latitude, longitude and altitude as well as three-dimensional velocity information and accurate time from the position information received from the satellite. A Wi-Fi Positioning System and / or a Hybrid Positioning System may be applied as the location information module 115.

The position information module 115 may further include a geomagnetic sensor and / or a gravity sensor for detecting a direction. For example, the position information module 115 detects directions (e.g., north, south, east, and west) of a mobile communication terminal for implementing navigation using augmented reality through the geomagnetic sensor (electronic compass). The position information module 115 detects in which direction gravity acts through the gravity sensor (G sensor). When the user holds the mobile communication terminal vertically, the position information module 115 displays a vertical screen, , The screen is automatically rotated 90 degrees to show a wide screen. Also, when the user watches the moving image, the position information module 115 rotates the screen according to the direction in which the user holds the mobile communication terminal through the gravity sensor (G sensor) so that the user can easily view the image.

The A / V input unit 120 is for inputting an audio signal or a video signal. The A / V input unit 120 includes a camera 121 and a microphone 122, . The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be composed of two or more according to the configuration of the mobile terminal.

The microphone 122 receives an external sound signal by a microphone in a communication mode, a recording mode, a voice recognition mode, and the like, and processes it as electrical voice data. In the call mode, the processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the mobile terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. Particularly, when the touch pad has a mutual layer structure with the display unit 151, it can be called a touch screen.

The sensing unit 140 senses the position of the mobile communication terminal 100 in the open / close state, the position of the mobile communication terminal 100, the presence of the user, the orientation of the mobile communication terminal 100, The mobile communication terminal 100 senses the current state of the mobile communication terminal 100 and generates a sensing signal for controlling the operation of the mobile communication terminal 100. For example, when the mobile communication terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. The sensing unit 140 senses whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The interface unit 170 serves as an interface with all external devices connected to the mobile communication terminal 100. For example, the interface unit 170 may include a headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, An audio input / output (I / O) port, a video I / O (input / output) port, and an earphone port. Here, the identification module is a chip that stores various information for authenticating the usage right of the mobile communication terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) ), A Universal Subscriber Identity Module (" USIM "), and the like. In addition, an apparatus (hereinafter, referred to as 'identification device') having the identification module may be manufactured in a smart card format. Accordingly, the identification module can be connected to the mobile communication terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device and transmits the received data to each component in the mobile communication terminal 100 or transmits data in the mobile communication terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 includes a display unit 151, an audio output module 152, an alarm unit 153, May be included.

The display unit 151 displays information to be processed by the mobile communication terminal 100 and outputs the information. For example, when the mobile communication terminal 100 is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile communication terminal 100 is in the video communication mode or the photographing mode, the displayed image or the UI or GUI is displayed.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display (3D display). There may be two or more display units 151 according to the embodiment of the mobile communication terminal 100. For example, the mobile communication terminal 100 may be provided with an external display unit (not shown) and an internal display unit (not shown) at the same time.

(Hereinafter, referred to as a 'touch screen') has a mutual layer structure with the display unit 151 and a sensor (hereinafter, referred to as 'touch sensor' It can also be used as an input device in addition to the device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch. If there is a touch input to the touch sensor, the corresponding signal (s) is sent to a touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the control unit 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

In the present invention, the proximity-touch means that the pointer is not actually touched on the screen but approaches a predetermined distance from the screen.

The proximity sensor 141 may be disposed in an inner region of the mobile communication terminal 100 or in the vicinity of the touch screen, which is surrounded by the touch screen. The proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface or an object existing in the vicinity thereof without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 has a longer lifetime and higher utilization than the contact-type sensor.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch" The act of actually touching the pointer on the screen is called "Contact Touch ". A position where the pointer is proximally touched on the touch screen means a position where the pointer corresponds to the touch screen vertically when the pointer is touched.

Further, the proximity sensor 141 detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, do. Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sensing unit 140 may include an acceleration sensor 142. The acceleration sensor 142 is a device that converts an acceleration change in one direction into an electric signal and is widely used along with the development of MEMS (micro-electromechanical systems) technology. Since the acceleration sensor 142 measures the acceleration of a large value that is built in the airbag system of an automobile and used to detect a collision, the acceleration sensor 142 recognizes the minute motion of a human hand and calculates a minute value acceleration There are various kinds of things to measure. The acceleration sensor 142 is usually configured by mounting two or three axes in one package. In some cases, only one axis of the Z axis is required depending on the use environment. Therefore, when the acceleration sensor in the X-axis direction or the Y-axis direction is used instead of the Z-axis direction for some reason, the acceleration sensor may be mounted on the main substrate by using a separate piece substrate.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, and a broadcast reception mode. The sound output module 152 outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile communication terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile communication terminal 100. Examples of events that occur in the mobile terminal include receiving a call signal, receiving a message, and inputting a key signal. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When the call signal is received or the message is received, the alarm unit 153 can vibrate the mobile terminal through the vibration means to inform it. Alternatively, when the key signal is input, the alarm unit 153 may vibrate the mobile communication terminal 100 through the vibration means with the feedback of the key signal input. Through the above-described vibration, the user can recognize the occurrence of the event. Of course, a signal for notifying the occurrence of an event may also be output through the display unit 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of air through the injection port or the suction port, a spit on the skin surface, contact with an electrode, And various effects of tactile effect such as effect of reproducing the cold sensation using the heat absorbing or heatable element can be generated.

The haptic module 154 can transmit a tactile effect through direct contact, and the tactile effect can be felt by a user through a muscular sense such as a finger or an arm. At least two haptic modules 154 may be provided according to the configuration of the telematics terminal. The haptic module 154 may be provided in a place where frequent contact with a user is frequent in the vehicle. For example, a steering wheel, a shift gear lever, a seat, and the like.

The memory 160 may store a program for processing and controlling the controller 180 and temporarily store the input / output data (e.g., map data, a phone book, a message, a still image, For example.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) A disk, and / or an optical disk. In addition, the mobile communication terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, the interface unit 170 may include a headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, An audio input / output (I / O) port, a video I / O (input / output) port, and an earphone port. Here, the identification module is a chip that stores various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) A Universal Subscriber Identity Module (USIM), and the like. In addition, the device having the identification module (hereinafter, referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification module can be connected to the mobile terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device and transmits the received data to each component in the mobile terminal 100 or transmits data in the mobile terminal 100 to an external device.

When the mobile terminal 100 is connected to an external cradle, the interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100, or various commands A signal may be transmitted to the mobile terminal 100. The various command signals or the power source inputted from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile communication terminal 100. For example, the control unit 180 performs related control and processing for voice call, data communication, video call, and the like. In addition, the controller 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180.

The control unit 180 can perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 and supplies power required for operation of the respective components.

The functions of the components applied to the mobile communication terminal 100 may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. According to a hardware implementation, an application specific integrated circuit (ASIC), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs) controllers may be implemented using at least one of micro-controllers, microprocessors, and electrical units for performing functions. In some cases, such embodiments may be implemented by the controller 180 In accordance with a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be stored in a software application written in a suitable programming language The software code may be stored in the memory 160, May also be performed by the < / RTI >

The voice recognition module 182 recognizes a voice uttered by a user and performs a corresponding function according to the recognized voice signal.

A navigation session 300 applied to the mobile communication terminal 100 displays a traveling route on the map data.

Meanwhile, an information display device applied to the mobile communication terminal 100 according to the embodiments of the present invention includes a camera; A display unit; When a 2D content display request is received while displaying a 3D (three-dimensional) augmented reality image including a real image photographed by the camera and a virtual image or information on a single display area of the display unit, And a controller for converting the 2D augmented reality image into a two-dimensional augmented reality image, dividing a single display area of the display unit into multi-display areas, and displaying the 2D augmented reality image and the 2D content together in the multi- have.

Details of the components of the information display device applied to the mobile communication terminal 100 according to the embodiments of the present invention will be described with reference to FIG. 2 to FIG.

Hereinafter, an information display apparatus and method of a mobile terminal according to a first embodiment of the present invention will be described with reference to Figs. 2 to 5. Fig. The information display apparatus and method according to embodiments of the present invention can be applied not only to a mobile terminal such as the mobile communication terminal 100, the telematics terminal, and a navigation device, but also to a smart phone, a notebook computer, , A PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a video device, and the like.

2 is a configuration diagram illustrating an information display apparatus of a mobile terminal according to a first embodiment of the present invention.

As shown in FIG. 2, the information display apparatus of the mobile terminal according to the first embodiment of the present invention includes a camera 405; A display unit 403; A three-dimensional augmented reality (AR) image including an actual image photographed through the camera 405 and a virtual image (or information) is displayed on a single display area ), A 2D augmented reality image is converted into a two-dimensional augmented reality image, and the display area of the display unit 403 is converted to a multi-display area (for example, 1 display area and a second display area), and displays the 2D augmented reality image and the 2D content together in the multi-display area.

The storage unit 404 stores a virtual image (or information), content, data, a program, and the like tagged in the actual image photographed through the camera 405.

The position information module 402 includes a Global Position System (GPS) module for detecting position information and a geomagnetic sensor and / or a gravity sensor for detecting a direction. For example, the position information module 402 detects a photographing position of the image through a GPS sensor. The location information module 402 detects direction information (for example, east, west, north, south) of the mobile communication terminal for augmented reality through the geomagnetic sensor (electronic compass). The position information module 402 detects the direction in which the gravity acts through the gravity sensor (G sensor) to detect the direction information (attitude information) (for example, the sky direction, the front direction, and the earth direction) .

The control unit 401 receives point of interest information corresponding to each object (facility, building, etc.) included in a real image (photographed image or photographed moving picture) from a server through a communication network, May be displayed on the real image.

The 3D augmented reality image may be generated by continuously photographing a subject with a single camera, or by simultaneously photographing a subject with two cameras. The 2D augmented reality image may be generated by photographing a subject by operating only one camera when the two augmented reality images are mobile terminals equipped with two cameras.

3 is a flowchart illustrating a method of displaying information of a mobile terminal according to a first embodiment of the present invention.

First, the camera 405 photographs a subject (actual image) and outputs the photographed actual image to the control unit 401.

The control unit 401 receives a three-dimensional augmented reality (AR) image including an actual image photographed through the camera 405 and a virtual image (or information) stored in the storage unit 404, Is displayed in a single display area (entire display area) of the display unit 403 (S11). A method of implementing the augmented reality itself is disclosed in U.S. Published Patent Application No. 20060241792, and a detailed description thereof will be omitted.

4 is a view illustrating an example of a 3D augmented reality image displayed on a display unit according to the first embodiment of the present invention.

4, the control unit 401 performs 3D enhancement including an actual image photographed through the camera 405 and a virtual image (or information) stored in the storage unit 404 in response to a user's request The real image 4-1 is displayed in a single display area (entire display area) of the display part 403. [

The control unit 401 determines whether a 2D content display request is received while displaying the 3D augmented reality image in a single display area (entire display area) of the display unit 403 (S12). For example, the control unit 401 may display the 2D content or the 2D content stored in the storage unit 404 while displaying the 3D augmented reality image in a single display area (entire display area) of the display unit 403 Lt; / RTI > is selected by the user. The 2D content may be an image, a moving image, and map data for navigation stored in the storage unit 404. The application for displaying the 2D content may include an application for displaying an image stored in the storage unit 404, an application for outputting a television broadcast program or a moving image stored in the storage unit 404, and map data for navigation And the like.

The controller 401 controls the 3D augmented reality image to be 2D (two-dimensional) when receiving a 2D content display request while displaying the 3D augmented reality image in a single display area (entire display area) ), And divides the display area of the display unit 403 into a multi-display area (for example, a first display area and a second display area) (S13).

The controller 401 displays the 2D augmented reality image and the 2D content together in the multi-display area (S14). For example, the controller 401 displays the 2D augmented reality image in the first display area and displays the 2D content in the second display area through multitasking.

5 is a diagram illustrating an example of 2D augmented reality images and 2D contents displayed on a display unit according to the first embodiment of the present invention.

5, when the 2D content display request is received while displaying the 3D augmented reality image in a single display area (entire display area) of the display unit 403, Converts the real image into a two-dimensional augmented reality image, and divides the entire display area of the display unit 403 into a first display area and a second display area. The controller 401 displays the 2D augmented reality image 5-1 in the first display area and the 2D content 5-2 in the second display area.

Meanwhile, the controller 401 displays the 2D augmented reality image 5-1 in the first display area and displays the 3D augmented reality image 5-2 in the second display area while displaying the 2D augmented reality image 5-1 in the first display area, Converts the 2D augmented reality image (5-1) into the 3D augmented reality image (4-1) when an input (for example, when an icon or a key is selected) And the 3D augmented reality image 4-1 may be displayed in the entire display area of the display unit 403. [

Accordingly, the information display apparatus and method of the mobile terminal according to the first embodiment of the present invention can display the 3D augmented reality image and the 2D content together in the multi display area of the display unit 403, (E.g., dizziness, vomiting, etc.) can be effectively removed. For example, when the 3D augmented reality image and the 2D content are displayed together in the multi-display area of the display unit 403, the user feels the blood pressure (for example, dizziness, vomiting, etc.) It is possible to remove the fatigue by displaying the 2D augmented reality image and the 2D content together with the multi-display area of the display unit 403 instead of the 3D augmented reality image.

Hereinafter, an information display apparatus and method of a mobile terminal according to a second embodiment of the present invention will be described with reference to Figs. 2 and 6 to 8. Fig.

6 is a flowchart illustrating a method of displaying information of a mobile terminal according to a second embodiment of the present invention.

First, the controller 401 displays the 3D augmented reality image on the display unit 403 when the direction of the mobile terminal is the first direction (for example, the horizontal direction) (S21). For example, the control unit 401 detects the direction of the mobile terminal through the gravity sensor (G sensor) in the position information module 115, and determines whether the direction of the mobile terminal is 90 degrees Direction) or 270 degrees (the horizontal direction), the 3D augmented reality image is displayed on the display unit 403.

FIG. 7 is an exemplary view illustrating a 3D augmented reality image displayed on a display unit according to a second embodiment of the present invention.

7, when the direction of the mobile terminal is the first direction (for example, the horizontal direction), the control unit 401 outputs the 3D augmented reality image 7-1 to the direction Is displayed on the display unit 403 in one direction (e.g., in the horizontal direction) (S21).

The control unit 401 determines whether the direction of the mobile terminal is rotated in the second direction (e.g., the vertical direction) while displaying the 3D augmented reality image on the display unit 403 (S22). For example, the control unit 401 detects the direction of the mobile terminal through the gravity sensor (G sensor) in the position information module 115, and determines whether the direction of the mobile terminal is 180 degrees Direction).

When the direction of the mobile terminal is rotated in a second direction (for example, the vertical direction) while the 3D augmented reality image is displayed on the display unit 403, the controller 401 controls the 3D augmented reality image to be displayed on the 2D Into an augmented reality image (S23). That is, in the case of a mobile terminal displaying a 3D augmented reality image only in a specific direction (for example, the horizontal direction), when the direction of the mobile terminal is rotated in the second direction (for example, The 3D augmented reality image is displayed only in a specific direction (for example, the horizontal direction), but in the present invention, the 3D augmented reality image is displayed only in a specific direction (for example, In the case of a mobile terminal, when the direction of the mobile terminal is rotated in a second direction (for example, longitudinal direction or horizontal direction), the 3D ARF image is converted into a 2D ARF image, On the display unit 403 in a second direction (e.g., longitudinal direction or horizontal direction) of the terminal.

When the direction of the mobile terminal is rotated in a second direction (for example, the longitudinal direction), the controller 401 controls the direction of the mobile terminal in a second direction (for example, Direction) on the display unit 403 (S24).

8 is a diagram illustrating an example of a 2D augmented reality image displayed on a display unit according to a second embodiment of the present invention.

8, when the direction of the mobile terminal is rotated in the second direction (e.g., the vertical direction) in the first direction (e.g., the horizontal direction) Converts the 2D augmented reality image into the 2D augmented reality image 8-1, and outputs the converted augmented reality image to the display unit 8-1 in the second direction (e.g., the vertical direction) 403).

Meanwhile, the controller 401 displays the 3D augmented reality image on the display unit 403 when the direction of the mobile terminal is the second direction (for example, the longitudinal direction), and displays the 3D augmented reality image on the display unit 403), when the direction of the mobile terminal is rotated in a first direction (e.g., a horizontal direction), the 3D augmented reality image is converted into the 2D augmented reality image, The direction of the mobile terminal may be displayed on the display unit 403 in a first direction (e.g., a horizontal direction).

The controller 401 displays the 2D augmented reality image on the display unit 403 when the direction of the mobile terminal is the first direction (for example, the horizontal direction), and displays the 2D augmented reality image on the display unit 403), when the direction of the mobile terminal is rotated in a second direction (e.g., vertical direction), the 2D ARF image is converted into the 3D ARP image, and the converted 3D ARP image The direction of the mobile terminal may be displayed on the display unit 403 in the second direction (e.g., the longitudinal direction).

When converting the 3D augmented reality image into the 2D augmented reality image, the controller 401 outputs the left augmented image or the intermediate image of the left eye image and the right eye image of the 3D augmented reality image as the 2D augmented reality image You may.

 Accordingly, the information display apparatus and method of a mobile terminal according to the second embodiment of the present invention can convert a 3D augmented reality image into a 2D augmented reality image according to the direction of the mobile terminal, and display the converted 2D augmented reality image So that the augmented reality image can be quickly displayed according to the direction of the mobile terminal.

Hereinafter, an information display apparatus and method of a mobile terminal according to a third embodiment of the present invention will be described with reference to Figs. 2 and 9 to 10. Fig.

9 is a flowchart illustrating a method of displaying information of a mobile terminal according to a third embodiment of the present invention.

First, the controller 401 displays the 3D augmented reality image on the display unit 403 when the direction of the mobile terminal is the first direction (for example, the longitudinal direction) (S31). For example, the control unit 401 detects the direction (or tilt angle) of the mobile terminal through the gravity sensor (G sensor) in the position information module 115 and assumes that the opposite direction of the gravity is 0 degrees And displays the 3D augmented reality image on the display unit 403 when the direction of the mobile terminal is within 0 to 10 degrees.

The control unit 401 determines whether the direction of the mobile terminal is switched (rotated) in the second direction (e.g., the horizontal direction) while displaying the 3D augmented reality image on the display unit 403 (S32) . For example, the control unit 401 detects the direction of the mobile terminal through the gravity sensor (G sensor) in the position information module 115, and when it is assumed that the opposite direction of the gravity is 0 degrees, Is within the range of 11 degrees to 80 degrees.

When the direction of the mobile terminal is switched (rotated) in a second direction (for example, in the horizontal direction) while the 3D augmented reality image is displayed on the display unit 403, the controller 401 displays the 2D augmented reality image Is displayed on the display unit (403). For example, the control unit 401 detects the direction of the mobile terminal through the gravity sensor (G sensor) in the position information module 115, and when it is assumed that the opposite direction of the gravity is 0 degrees, Is within 11 degrees to 80 degrees, the 2D augmented reality image is displayed on the display unit (403).

The control unit 401 determines whether the direction of the mobile terminal is switched (rotated) in the second direction (e.g., the horizontal direction) while displaying the 2D augmented reality image on the display unit 403 (S34) . For example, the control unit 401 detects the direction of the mobile terminal through the gravity sensor (G sensor) in the position information module 115, and when it is assumed that the opposite direction of the gravity is 0 degrees, Is within the range of 81 degrees to 90 degrees.

The control unit 401 displays the 3D augmented reality image on the display unit 403 when the direction of the mobile terminal is switched (rotated) in a second direction (for example, the horizontal direction) (S35). For example, while the controller 401 displays the 3D augmented reality image in a specific direction (e.g., a horizontal direction) of the mobile terminal, And displays the 2D augmented reality image on the display unit 403 when the direction of the mobile terminal is switched to another direction (for example, the vertical direction) It is possible to continuously display the augmented reality image on the display unit 403 even when the direction of the mobile terminal is changed by displaying the direction of the mobile terminal on the display unit 403 in a different direction (for example, the longitudinal direction). That is, while the direction of the mobile terminal is changed, it is difficult to display the 3D augmented reality image on the display unit 403 promptly due to deterioration of the readability of the 3D augmented reality image. In the present invention, The user can quickly identify the augmented reality image even when the direction of the mobile terminal is changed by displaying the 2D augmented reality image on the display unit 403 quickly.

In addition, while the controller 401 displays the 3D augmented reality image in a specific direction (e.g., portrait) of the mobile terminal, the controller 401 switches the direction of the mobile terminal to another direction And displays the 2D augmented reality image on the display unit 403. When the direction of the mobile terminal is switched to another direction (for example, the horizontal direction), the 3D augmented reality image is displayed in the direction of the mobile terminal It is possible to continuously display the augmented reality image on the display unit 403 even when the direction of the mobile terminal is changed by displaying the augmented reality image on the display unit 403 in another direction (for example, the horizontal direction).

10 is an exemplary view illustrating a 3D augmented reality image and a 2D augmented reality image displayed on a display unit according to a third embodiment of the present invention.

10, when the direction of the mobile terminal is different while the 3D augmented reality image 10-1 is displayed in a specific direction (e.g., portrait direction) of the mobile terminal, The 2D augmented reality image 10-2 is displayed on the display unit 403 while the direction of the mobile terminal is switched to another direction (for example, the horizontal direction) When the conversion is completed, the size of the 3D augmented reality image 10-1 is adjusted to match the other direction (e.g., the horizontal direction) of the mobile terminal, and the size of the 3D augmented reality image 10-3 Can be displayed on the display unit 403 in a different direction (e.g., the horizontal direction) of the mobile terminal, thereby continuously displaying the augmented reality image on the display unit 403 even when the direction of the mobile terminal is changed .

Accordingly, the information display apparatus and method of a mobile terminal according to the third exemplary embodiment of the present invention can be applied to a case where the direction of the mobile terminal is being switched to another direction (for example, the horizontal direction) while displaying the 3D augmented reality image By displaying the 2D augmented reality image on the display unit 403, the augmented reality image can be continuously displayed on the display unit 403 even when the direction of the mobile terminal is changed.

As described above, the information display apparatus and method of a mobile terminal according to embodiments of the present invention display 3D augmented reality images and 2D contents together in a multi-display area of a display unit, (E. G., Dizziness, vomiting, etc.).

The apparatus and method for displaying information of a mobile terminal according to embodiments of the present invention may convert a 3D augmented reality image into a 2D augmented reality image according to a direction of the mobile terminal and display the converted augmented reality image, The augmented reality image can be quickly displayed according to the direction of the augmented reality image.

An apparatus and method for displaying information of a mobile terminal according to embodiments of the present invention include a step of performing 2D enhancement when a direction of the mobile terminal is being changed in a different direction (for example, By displaying the real image on the display unit 403, it is possible to continuously display the augmented reality image on the display unit 403 even when the direction of the mobile terminal is changed.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

401: control unit 402: position information module
403: Display section 404:
405: Camera

Claims (18)

A camera;
A display unit;
When a 2D content display request is received while displaying a 3D (three-dimensional) augmented reality image including a real image photographed by the camera and a virtual image or information on a single display area of the display unit, And a controller for converting the 2D augmented reality image into a 2D augmented reality image, dividing a single display area of the display unit into multi display areas, and displaying the 2D augmented reality image and the 2D content together in the multi display area,
Wherein when the mobile terminal is rotating in the second direction from the first direction while the 3D augmented reality image is displayed in the first direction of the mobile terminal, Wherein the 2D augmented reality image is displayed on the display unit instead of the 3D augmented reality image and the 3D augmented reality image is displayed on the display unit instead of the 2D augmented reality image when the mobile terminal is rotated in the second direction. An information display device of a terminal. The apparatus of claim 1,
The 2D display device displays the 2D augmented reality image in the first display area and displays the 2D content in the second display area by dividing a single display area of the display part into a first display area and a second display area, To the information display device of the mobile terminal. The apparatus of claim 1,
Wherein when the input requesting the 3D augmented reality image is received while displaying the 2D augmented reality image and the 2D content in the multi display area, the 2D augmented reality image is converted into the 3D augmented reality image, And displays the 3D augmented reality image in a display area. delete delete delete delete delete The apparatus of claim 1,
Adjusting a size of the 3D augmented reality image to match the second direction of the mobile terminal when the direction of the mobile terminal is switched to the second direction, And displays the information on the display unit in a second direction. delete delete delete delete delete delete delete delete delete

Images