KR100864725B1 - Mobile terminal having hot keys for operating contents and interaction method thereof - Google Patents

Abstract

A mobile terminal having hot keys for contents operation and an interaction method thereof are provided to operate contents exhibited in an exhibition hall variously on virtual reality or mixed reality, thereby enabling a user to operate the contents more conveniently and instinctively during movement. A mobile terminal(100) having hot keys for contents operation comprises a camera(110), an RF(Radio Frequency) processor(120), a keypad(130), a control unit(170), a storage unit(150), a display unit(160), and a hot key pad(140). The camera photographs contents. The hot key pad operates the contents photographed by the camera. The display unit displays the contents photographed by the camera. The control unit operates the contents displayed on the display unit according to a control command inputted through the hot key pad on virtual reality or mixed reality.

Description

Mobile terminal having hot keys for operating contents and interaction method thereof

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile terminal and an interaction method thereof, and more particularly, to a mobile terminal and an interaction method thereof capable of variously manipulating contents displayed in an exhibition hall on a virtual reality or mixed reality using hot keys.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. [Task Management No .: 2007-S-018-01, Title: Mixed reality-based u-experience type Development of content operation platform].

As for the operation method of the mobile terminal, the text input based on the keypad and the small QWERTY keyboard is the mainstream, but the portable information provided for the experience service system of exhibition halls and museums using the virtual reality or mixed reality technology. When applied to the device, there are many restrictions on interaction with the experience elements, which results in a disadvantage in that the user does not have a sufficient level of service benefits.

In the case of exhibition halls or museums in which various artifacts and materials are arranged, in recent years, IT technology has been used to deliver information on exhibits more accurately to visitors, and further, virtually manipulate the exhibits, thereby providing various functions of the original exhibits. Efforts have been made to make the experience more realistic. As part of the effort to make these efforts possible, it is recommended that visitors have a simple portable information device when entering the exhibition hall, and in the process of viewing each exhibition, provide various content services related to the exhibition through the mobile device. There is an example. However, most of these services merely provide video or audio information using radio frequency identification (RFID), and it is virtually difficult to deliver a wide range of information related to the exhibition to users by adding empirical elements.

The present invention is to solve the above problems, an object of the present invention is to provide a mobile terminal and its interaction method that can be used to manipulate the content displayed in the exhibition hall, etc. in virtual reality or mixed reality using hot keys, etc. have.

Another object of the present invention is to attach a hotkey that can be operated more conveniently and intuitively while the user is on the move, and a mobile terminal to smoothly enjoy a variety of experience content services based on virtual reality or mixed reality technology in the ubiquitous environment It is to provide a method of interaction thereof.

Another object of the present invention is to manipulate the content displayed on the virtual reality or mixed reality using hot keys, so that the information on the exhibits displayed on the mobile terminal can be grasped more effectively or realistically, and various types of interactions can be realized. In addition, the present invention provides a mobile terminal and a method of interacting with the mobile terminal, so that the user can directly experience it.

The mobile terminal of the present invention for achieving the above object, the camera to shoot the content; A hotkey pad for manipulating content captured by the camera; A display unit for displaying the content photographed by the camera; And a controller configured to manipulate the content displayed on the display unit in the virtual reality or the mixed reality according to a control command input through the hot key pad.

The present invention provides a method of interacting with a mobile terminal having hot keys for manipulating content in virtual reality or mixed reality, the method comprising: photographing content; Displaying the photographed content; And manipulating the displayed content on a virtual reality or a mixed reality according to a control command input through the hot keys.

According to the present invention having the above configuration, the user can operate the contents displayed on the exhibition hall or the like in various ways on the virtual reality or mixed reality by using hot keys, so that the user can operate more conveniently and intuitively while on the move, thereby enabling the virtual reality under the ubiquitous environment. Or, it can smoothly enjoy various experience contents services based on mixed reality technology.

The present invention manipulates content displayed in virtual reality or mixed reality using hotkeys to more effectively or realistically grasp information on exhibitions displayed on a mobile terminal, and applies various types of interactions to provide direct experience. Let's do it.

In front of each pavilion, while looking at portable information devices, especially mobile terminals such as the Ultra-Mobile Person Computer (UMPC), which is rather easy to provide visual information, while getting an overview of the exhibits, a more detailed description of the exhibits is virtual. If the virtual reality or mixed reality content provided for at the same time can be provided through a mobile terminal such as UMPC, it can be a great help for the user to understand the exhibits. As an example, when viewing a turtle ship exhibit through a screen of a mobile terminal such as a UMPC, if the foreground views of the inside are projected, users who have only been familiar with the appearance of the turtle ship can examine the internal structure of the turtle ship of the Joseon Dynasty. I will have a chance. In this case, the most important thing is the interaction between the content provided by the service and the user. That is, the present invention rotates the contents of the exhibits virtually provided through the operation method of hotkeys of the mobile terminal, which is simply implemented in order to more effectively or realistically grasp the information on the exhibits shown on the mobile terminal such as UMPC. It is intended to give you a hands-on experience by applying various types of interactions, such as watching or taking special actions.

In addition, there is a limitation in providing additional information of exhibits displayed in museums or the like due to spatial constraints. Therefore, recent museums and exhibition halls provide additional information on exhibits through mobile terminals such as UMPC. The existing additional information is limited to text, an image, a video, and the like, but may include mixed reality-based image information that provides additional image information by being superimposed on image information obtained by a camera in the future. For example, the present invention goes beyond showing only the appearance of the turtle ship appearance obtained by the camera in the situation where the user views the turtle ship exhibition through a mobile terminal such as a UMPC, and superimposes the contents of the inside of the turtle ship made of virtual reality technology on the acquired image by overlapping the acquired image. By presenting to them, you can see the whole inside and outside of the turtle ship. That is, the present invention obtains an actual image using a camera attached to a mobile terminal, and provides additional information by superimposing a virtual image by a mixed reality technique. In this case, if the user is looking at the side of the turtle ship, he may want to look closely at the stern, deck or crown of the turtle ship. In some cases, there may be a desire to reproduce the flames from the crown. This is an experiential factor in terms of some kind of virtual or mixed reality technology, where hotkeys can be placed on the rear panel of the mobile terminal to easily perform these functions on the mobile terminal and manipulate these hotkeys in each case. By doing so, the user can have various experiences with the contents displayed in the virtual reality or mixed reality.

In order to realize such a method, the present invention proposes a method of attaching a rear panel to a rear surface opposite to the front panel of a mobile terminal and arranging hotkeys for content manipulation on the rear panel. Of course, the number of hotkeys and the functions that each hotkey is responsible for may vary depending on the content. These hotkeys are intended to fulfill some of the most basic functions required for the manipulation of content, each responsible for zoom-in, zoom-out, rotation, scroll up / down, and other specific action functions associated with the content. For a user holding a mobile terminal in his or her hand, the thumb of both hands is usually placed on the front panel and the remaining eight fingers are placed on the rear panel to hold the mobile terminal. At this time, the two thumbs are usually involved in text input. The recently released UMPC has a QWERTY-type keyboard attached to the front panel, which makes it easy to enter text using two thumbs, though it is inconvenient. However, manipulating virtual contents related to exhibits through text input can be difficult in mobile environment. That is, a new method is required for the interaction between the user and the mobile terminal. There are many such new methods, but in the present invention, a thin panel having the same size as the front panel is attached to the rear of the mobile terminal, and hotkeys for content manipulation are attached to the rear panel.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is a block diagram of a mobile terminal having hot keys for content manipulation according to an embodiment of the present invention.

Referring to FIG. 1, the mobile terminal 100 of the present invention includes a camera 110 for capturing an image, a radio frequency processor 120 for transmitting and receiving radio frequency signals, and a keypad for inputting a user command. 130, a hotkey pad 140 for manipulating content photographed by the camera 110, a storage unit 150 for storing an execution program of the hotkey pad 140, and a photographing by the camera 110. The display unit 160 for displaying the content to be displayed and the content photographed by the camera 110 is displayed on the display unit 160 and displayed on the display unit 160 according to a control command input through the hot key pad 140. The control unit 170 includes a control unit 170 for operating the content on the virtual reality or mixed reality.

The camera 110 captures an image captured in front under the control of the controller 170. For example, when the mobile terminal 100 is located in an exhibition hall such as a museum, the camera 110 may photograph content displayed in the exhibition hall, and the captured content is displayed on the display unit 160 as shown in FIG. 2. Is displayed. Here, FIG. 2 shows a case where a turtle ship of an exhibition hall is photographed as content.

The radio frequency processing unit 120 converts the baseband signal transmitted from the control unit 170 into a radio frequency signal and transmits it to the outside, and also converts the high frequency signal received from the outside into a baseband signal and transmits it to the control unit 170. . The radio frequency processor 120 receives information about the content displayed in the exhibition hall and transmits the information to the controller 170. For example, as shown in FIG. 2, the turtle ship of the exhibition hall is captured by the camera 110 as the content. If there is, the RF processor 120 receives the information on the turtle ship and transmits it to the controller 170.

The keypad 130 is attached to the front panel of the mobile terminal 100 and receives various user commands and transmits them to the controller 170. For example, the user instructs photographing of the camera 110 using the keypad 130.

The hotkey pad 140 is attached to the rear panel of the mobile terminal 100, and a plurality of hotkeys 141 to 146 are arranged on the hotkey pad 140 as shown in FIG. 3. As shown in Figure 4, hotkey 141 is used to reduce the size of the content as a zoom-in hotkey, and hotkey 142 is used to increase the size of the content as a zoom-out hotkey. Hotkey 143 is a rotating hotkey that is used to rotate the content at various angles, and hotkey 144 is an action hotkey that is used to perform a specific action of the content. Hotkey 145 is then used to move the content upwards as a scroll up hotkey, and hotkey 146 is used to move the content downwards as a scrolldown hotkey. Meanwhile, the present invention arranges six hot keys 141 to 146 on the hot key pad 140, but the present invention is not limited thereto. The number of hot keys may be 6 or less or more depending on the type of content. have.

In addition, the keypad 130 and the hotkey pad 140 may allow a user holding the mobile terminal 100 in his hand to place the thumb of both hands on the front panel, and place the remaining eight fingers on the rear panel to hold the mobile terminal 100. It is preferable to implement. In this case, the two thumbs of the user are implemented to engage in text input.

The storage unit 150 may be implemented as a memory device such as an EEPROM, a NAND memory, a flash memory, or the like, and stores an execution program of the hot key pad 140.

The display unit 160 is implemented as a display device such as a liquid crystal display (LCD) or an organic light emitting diode display (OLED). The content captured by the camera 110 under the control of the controller 170 is illustrated in FIG. 2. Display as shown.

The controller 170 displays the content captured by the camera 110 on the display unit 160 and displays the content displayed on the display unit 160 according to a control command input through the hot key pad 140 on the virtual or mixed reality. Operation is as described below.

For example, in a state where a turtle ship displayed in an exhibition hall is displayed on the display unit 160 as shown in FIG. 2, a user uses hot keys 141 to 146 to display a turtle ship as a content in virtual reality or mixed reality. Operation is as follows.

When the user inputs a zoom-in control command using the hotkey 141, the controller 170 reduces the size of the turtle ship in virtual reality or mixed reality according to the zoom-in control command input through the hotkey 141.

When the user inputs a zoom out control command using the hot key 142, the controller 170 increases the size of the turtle ship in virtual reality or mixed reality according to the zoom out control command input through the hot key 142.

When the user inputs a rotation control command using the hotkey 143, the controller 170 rotates the turtle ship according to the rotation control command input through the hotkey 143.

When the user inputs an action control command using the hotkey 144, the controller 170 operates the turtle ship according to the action control command input through the hotkey 144, in which case the turtle ship emits fire by mouth.

When the user inputs a scroll up control command using the hot key 145, the controller 170 moves the turtle line upward according to the scroll up control command input through the hot key 145.

When the user inputs a scroll down control command using the hot key 146, the controller 170 moves the turtle line downward according to the scroll down control command input through the hot key 145.

In this way, the user interacts with the mobile terminal 100 using hot keys to manipulate various contents displayed in the exhibition hall on the virtual reality or mixed reality, so that various experience contents services can be smoothly enjoyed.

The controller 170 may change the default function of the hot keys 141 to 146 according to the type of the content. In this case, the controller 170 may generate the keys generated by the hot keys 141 to 146. Recognize signals differently from key signals configured by default.

Although the technical spirit of the present invention has been described in detail according to the above-described preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

1 is a block diagram of a mobile terminal having hotkeys for content operation according to an embodiment of the present invention.

2 is a view showing a state in which content is displayed on the front panel of a mobile terminal according to the present invention.

3 is a view showing an arrangement of hotkeys on a rear panel of a mobile terminal according to the present invention;

4 is an exemplary view for explaining the function of the hotkeys shown in FIG.

Claims (14)

A camera for photographing content; A hotkey pad for manipulating content captured by the camera; A display unit for displaying the content photographed by the camera; And And a controller for manipulating the content displayed on the display unit in the virtual reality or the mixed reality according to a control command input through the hotkey pad. The method of claim 1, wherein the hot key pad, And a hot key for content manipulation, which is attached to a rear panel of the mobile terminal. The method of claim 2, wherein on the hotkey pad, Zoom-in hotkeys used to reduce the size of the content, Zoom-out hotkeys used to increase the size of the content, Rotation hotkeys used to rotate the content at different angles, Action hotkeys used to perform specific actions of the content, Move content upward And a scroll up hot key used for moving the content downward, and a scroll down hot key used for moving the content downward. The method of claim 3, wherein the control unit, And a control command is input through the zoom-in hotkey to reduce the size of the content. The method of claim 3, wherein the control unit, And a control command is inputted through the zoom-out key to increase the size of the content. The method of claim 3, wherein the control unit, And a control command is inputted through the rotation hotkey to rotate the content. The method of claim 3, wherein the control unit, And a control command is input through the action hotkey to perform a specific operation of the content. The method of claim 3, wherein the control unit, And a control command is input through the scroll-up hot key to move the content upward. The method of claim 3, wherein the control unit, And a control command is inputted through the scroll down hotkey to move the content downward. The method of claim 3, wherein Mobile hot keys for content operation characterized in that the user holding the mobile terminal in the hand to place the thumb of both hands on the front panel and the remaining eight fingers on the rear panel to hold the mobile terminal terminal. The method of claim 3, wherein And a radio frequency processor configured to receive information about the content. In the interaction method of a mobile terminal having hot keys for manipulating content in virtual reality or mixed reality, Shooting content; Displaying the photographed content; And And manipulating the displayed content on a virtual reality or mixed reality according to a control command inputted through the hotkeys. The method of claim 12, After the step of shooting the content, Receiving information about the content; interaction method of a mobile terminal further comprising. The method of claim 12, wherein manipulating the displayed content comprises: Reducing the size of the content; Increasing the size of the content; Rotating the content; Moving the content upward; And Moving the content downward; Containing at least one of Interaction method of a mobile terminal, characterized in that.

Images