KR20230015799A - Method of providing movable hologram contents in vehicle such as a bus and a system thereof - Google Patents

Abstract

A 3D interactive hologram content providing system that provides the fusion of virtual reality content (first content) using a hologram and augmented reality content (second content) using a hologram of a real object in a vehicle such as a bus according to the present invention includes: a hologram display device that projects the virtual reality content (first content) using a hologram stored in a content storage device and displays the virtual reality content (first content) on a screen; a first content display control unit that receives first time information by the playback of the first content displayed on the screen; a real object image photographing device that photographs an image of a real object; an augmented reality content conversion unit that converts the photographed image of the real object into the augmented reality content (second content) using a hologram; a content transmission device that transmits the second content to a content display device; a second content display control unit that receives second time information by the playback of the second content displayed on the content display device; and a content fusion device that synchronizes the second content displayed on the content display device with the first content displayed on the screen using the first time information and the second time information. The content transmission device is connected to a mobile device through wireless or wired communication, and the content display device is composed of a head-mounted display (HMD) device arranged for each seat of the vehicle. The HMD device is configured to be connected to the mobile device through wireless communication and to communicate with the content transmission device connected wirelessly or by wire. Therefore, the present invention can provide content so that students can experience experiences that they could not experience due to a lack of equipment in an underprivileged area.

Description

METHOD OF PROVIDING MOVABLE HOLOGRAM CONTENTS IN VEHICLE SUCH AS A BUS AND A SYSTEM THEREOF

The present invention relates to a method for providing mobile hologram contents and a system for providing the same.

As the low birth rate and population aging intensify throughout our society, the population of the region is flocking to the metropolitan area and large cities, and the region is facing the crisis of regional extinction due to population decline. In particular, provincial areas are at risk of collapsing childbirth, childrearing, childcare, education, and economic foundations as tax revenues decrease and competitiveness declines due to population decline. In terms of education, there has been a demand for a method of providing educational contents to elementary, middle, high school, and out-of-school youth in so-called marginalized areas, such as rural areas and old city centers, where opportunities for educational experiences are relatively scarce compared to urban centers.

However, as a social contribution project to bridge the educational experience opportunity gap, there is a limit to the content in providing various experiences to elementary, middle, high school, and out-of-school youth in underprivileged areas.

Therefore, there is a need for a method and a providing system for providing various contents even in remote areas such as rural areas.

Accordingly, interest in a method of providing educational contents using a mobile hologram technology is growing in order to provide various contents in an isolated area.

A hologram refers to a three-dimensional image created using the principle of holography, and represents a medium on which interference fringes that reproduce a three-dimensional image are recorded. The principle of holography is to split the light beam from the laser into two so that one light directly illuminates the screen and the other light illuminates the object we are trying to see. At this time, the light that directly illuminates the screen is called a reference beam, and the light that illuminates the object is called an object beam. Since object light is reflected from each surface of an object, the phase difference (distance from the surface of the object to the screen) is different depending on the surface of the object. At this time, the untransformed reference light causes interference with the object light, and the interference pattern at this time is stored on the screen. The film in which these interference fringes are stored is called a hologram.

On the other hand, in order to reproduce the stored image again, the light beam used during recording must be re-directed to the screen plate. The light beam used for reproduction must exactly match the reference light used for recording because only waves with the same frequency as when recording are reproduced in three dimensions, and waves with different wavelengths and phases pass through the stored hologram without any effect. do.

On the other hand, these holograms can give a sense of liveliness and impression equivalent to that without directly participating in or visiting various cultural, educational performances, or historical sites. Recently, holograms have been in the limelight to improve accessibility to cultural media in these marginalized areas.

In particular, there is a growing need for a technology capable of improving accessibility to hologram contents only by moving a vehicle regardless of distance or location by installing a mobile hologram on a transportation medium such as a bus.

However, in order to install a mobile hologram in a vehicle such as a bus, the hologram providing system must be configured as simplified as possible for convenient movement, and the hologram production system has a structure that can give a three-dimensional effect and a sense of vitality even though it is installed in a relatively narrow space called a bus. should be designed

That is, when the interior space of the bus is converted into a theater space that projects holograms and hologram contents are provided outdoors such as school grounds, at least one side of the bus must be opened so that the audience can see the hologram image projected on the screen. , there is a problem in that the light and sound of the hologram content are dispersed to the open side, thereby impairing the liveliness and three-dimensional effect.

In addition, the hologram technology for providing such educational and cultural contents may be provided by combining virtual reality technology and augmented reality technology, and since the virtual reality contents are viewed while wearing an HMD (Head-Mounted Display) device, cyber sickness There is a limit to the improvement of immersion to induce, so various approaches are needed to improve it.

In addition, there is a need for a specific and new approach to increase student participation while providing mixed reality content that combines existing virtual reality content and augmented reality content in a vehicle such as a bus.

Korean Patent Registration No. 10-1910878 Korea Patent Registration No. 10-1665363

The present invention is an invention made to solve the above problems, and an object of the present invention is to provide a method and system capable of providing mobile hologram contents capable of providing various educational contents to students in remote areas. .

In addition, the present invention is to provide a method and system for providing hologram contents in which hologram, virtual reality (VR), and augmented reality (AR) technologies are converged, which can improve the liveliness and three-dimensional effect of holograms provided within a limited space of a vehicle. .

In addition, it is intended to provide a method and system for providing hologram content implemented as a video image that can interact with students in order to increase the concentration of viewing educational content by arousing students' interest.

In addition, it is intended to provide a method and system for providing hologram contents capable of adjusting the brightness of an image of provided contents according to the change in brightness of light in a vehicle.

In addition, the advantage of providing bright content rather causes a problem of causing unpleasant glare to the user, and light damages the retina and is the main cause of macular degeneration resulting in visual acuity loss. Therefore, the present invention provides a content transmission device and mobile device capable of controlling screen luminance according to the luminance of the surrounding environment in consideration of human visual characteristics in order to reduce eye strain caused by glare and to prevent deterioration of eyesight by lowering the intensity of light. It is intended to provide a method and system for providing hologram content including a device.

In addition, the present invention provides a method for providing hologram content that can protect the user's eyesight by displaying the screen of the user's mobile device in an eye protection mode according to user input, content providing time, and user's eye fatigue state, and the like. We want to provide a system.

In addition, since the present invention is characterized by being performed in a vehicle, there is a limitation in its limited power. Accordingly, an object of the present invention is to provide a method and system for providing hologram content including a content transmission device capable of saving power consumption by controlling screen luminance so that glare does not occur according to ambient luminance.

In addition, the present invention uses an HMD device, and since the HMD provides high-definition content and maintains high brightness, when students' eyes are exposed to bright light of such high definition, the eyesight of growing students is easily adversely affected. Therefore, it is intended to provide a method and system for providing hologram content that limits the time of using the HMD device for students.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

A method for providing 3D interactive hologram contents by converging virtual reality contents (first contents) using holograms and augmented reality contents (second contents) using holograms of real objects in a vehicle such as a bus according to the present invention provides hologram display Projecting, by a device, virtual reality content (first content) using a hologram stored in a content storage device and displaying it on a screen; receiving, by a first content display controller, first time information according to reproduction of the first content displayed on the screen; Taking an image of a real object by a real object image capturing device; converting, by an augmented reality content conversion unit, the captured image of the real object into augmented reality content (second content) using a hologram; transmitting, by a content transmission device, the second content to a content display device; receiving, by a second content display controller, second time information according to reproduction of the second content displayed on the content display device; and synchronizing, by a content convergence device, second content displayed on a content display device with the first content displayed on the screen by using the first time information and the second time information; The content transmission device is connected to a mobile device through wireless or wired communication, and the content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle, and the HMD device is connected to the mobile device through wireless communication. connected, and is configured to communicate with the content transmission device connected wirelessly or wired thereto.

In the 3D interactive hologram content providing method according to the present invention, the content transmission device may be selected from a group including a desktop, a laptop, a mobile terminal, and a smart phone.

In the method for providing 3D interactive hologram contents according to the present invention, the real object may be located on the screen.

In the method for providing 3D interactive hologram contents according to the present invention, the first contents may be contents for student education.

In the 3D interactive hologram content providing method according to the present invention, the real object may be a user who enjoys the first content using the HMD device.

A system for providing 3D interactive hologram contents by converging virtual reality contents (first contents) using holograms and augmented reality contents (second contents) using holograms of real objects in a vehicle such as a bus according to the present invention provides content storage A hologram display device that projects virtual reality content (first content) using a hologram stored in the device and displays it on a screen; a first content display controller receiving first time information according to reproduction of the first content displayed on the screen; A real object image capturing device for capturing an image of a real object; an augmented reality content conversion unit that converts the captured image of the real object into augmented reality content (second content) using a hologram; a content transmission device that transmits the second content to a content display device; A second content display control unit that receives second time information according to reproduction of the second content displayed on the content display device, and a second content display controller that is displayed on the content display device using the first time information and the second time information. and a content convergence device that synchronizes two content to the first content displayed on the screen; The content transmission device is connected to a mobile device through wireless or wired communication, and the content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle, and the HMD device is connected to the mobile device through wireless communication. connected, and is configured to communicate with the content transmission device connected wirelessly or wired thereto.

In the 3D interactive hologram content providing system according to the present invention, the content transmission device may be selected from a group including a desktop, a lap top, a mobile terminal, and a smart phone.

In the system for providing 3D interactive hologram contents according to the present invention, the real object may be located on the screen.

In the system for providing 3D interactive hologram contents according to the present invention, the first contents may be contents for student education.

In the 3D interactive holographic content providing system according to the present invention, the real object may be a user who enjoys the first content using the HMD device.

As another embodiment, a method for providing 3D interactive content by converging virtual reality content (first content) and augmented reality content (second content) of a real object in a vehicle such as a bus according to the present invention includes a content storage device Transmitting the virtual reality content (first content) stored in the content convergence device; Receiving, by a first content display controller, first time information according to reproduction of the first content; Taking an image of a real object by a real object image capturing device; converting the captured image of the real object into augmented reality content (second content) by an augmented reality content conversion unit and transmitting the image to the content convergence device; receiving, by a second content display controller, second time information according to reproduction of the second content; A content transmission device receiving the first content and the second content from the content convergence device and transmitting them to a content display device, and the content convergence device using the first time information and the second time information to Synchronizing the first content and the second content displayed on a content display device; The content transmission device is connected to a mobile device through wireless or wired communication, and the content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle, and the HMD device is connected to the mobile device through wireless communication. connected and configured to communicate with the content transmission device connected wirelessly or wired thereto; The 3D interactive content providing method may further include: causing the content transmission apparatus to measure ambient luminance of the mobile device in the mobile device, and transmitting the measured environmental luminance to the contents transmission apparatus; The content transmission device calculates a threshold screen luminance at which glare starts to occur in the mobile device using the measured environmental luminance and a preset glare index value, and the average luminance level of the contents corresponding to the calculated threshold screen luminance. deriving a threshold; and the content transmission device calculates an average luminance level of the content using the luminance of each pixel constituting a frame of the content, and the calculated average luminance level of the content exceeds a threshold value of the average luminance level of the content. and adjusting the luminance of the light emitting module of the mobile device so that the screen luminance of the mobile device does not exceed the threshold screen luminance when the screen luminance of the mobile device is selected.

As another embodiment, in the 3D interactive content providing method according to the present invention, the content transmission device may be selected from a group including a desktop, a laptop, a mobile terminal, and a smartphone.

As another embodiment, in the 3D interactive content providing method according to the present invention, the real object may be located on the screen.

As another embodiment, in the method for providing 3D interactive contents according to the present invention, the first contents may be contents for student education.

As another embodiment, in the 3D interactive content providing method according to the present invention, the real object may be a user who enjoys the first content using the HMD device.

As an additional embodiment, a system for providing 3D interactive contents by fusing virtual reality contents (first contents) and augmented reality contents (second contents) of real objects in a vehicle such as a bus according to the present invention provides virtual reality contents ( a content storage device that stores first content); a first content display controller receiving first time information according to reproduction of the first content; A real object image capturing device for capturing an image of a real object; an augmented reality content converting unit that converts the captured image of the real object into augmented reality content (second content); a content convergence device that receives and fuses the first content and the second content; a second content display control unit receiving second time information according to reproduction of the second content; and a content transmission device receiving the first content and the second content from the content convergence device and transmitting the received content to a content display device; the content convergence device is configured to synchronize the first content and the second content displayed on the content display device using the first time information and the second time information; The content transmission device is connected to the mobile device through wireless or wired communication; The content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle; The HMD device is configured to be connected to the mobile device through wireless communication, and to communicate with the content transmission device connected wirelessly or wired thereto; The mobile device may include an environmental luminance measuring unit configured to measure ambient luminance of the mobile device; a terminal control unit to transmit the measured environmental luminance to the content transmission device; A critical screen luminance of the mobile device at which glare starts to occur is calculated using a preset glare index value and the environmental luminance measured by the environmental luminance measurement unit, and an average luminance level threshold of content corresponding to the calculated critical screen luminance a threshold determining unit for deriving; an average luminance level calculation unit that calculates an average luminance level of the contents using the luminance of each pixel constituting a frame of the input contents; and when the average luminance level of the contents calculated by the average luminance level calculation unit exceeds the average luminance level of the contents derived by the threshold determination unit, light is emitted so that the screen luminance of the mobile device does not exceed the threshold screen luminance. It includes a luminance control unit for adjusting the luminance of the light emitted from the module.

As a further embodiment, in the 3D interactive content providing system according to the present invention, the content transmission device may be selected from a group including a desktop, a laptop, a mobile terminal, and a smart phone.

As an additional embodiment, in the 3D interactive content providing system according to the present invention, the real object may be located on the screen.

As an additional embodiment, in the 3D interactive content providing system according to the present invention, the first content may be content for student education.

As an additional embodiment, in the 3D interactive content providing system according to the present invention, the real object may be a user who enjoys the first content using the HMD device.

Other specific details of the invention are included in the detailed description and drawings.

The method and system for providing mobile hologram contents according to the present invention have the following effects.

Since the present invention can provide hologram contents using a vehicle such as a bus, it is possible to provide various educational contents to students in underprivileged areas. You can provide content so that you can experience this.

Since the present invention can provide a holographic image that combines virtual reality and augmented reality, it is possible to increase the liveliness and three-dimensional effect even if an image is provided in a vehicle in a limited space such as a bus.

Since the present invention can converge and provide virtual reality contents and augmented reality contents, it is possible to provide “realistic video contents” for realistic visualization and “interaction contents” to naturally interact with users through virtual holograms. .

In addition, according to the present invention, virtual reality content using holograms and augmented reality content using holograms can be synchronized using playback or display time information or reproduced with a time difference as necessary, thereby increasing not only the liveliness of images but also the fun. there is.

In addition, the present invention converts the movement itself of a student user watching virtual reality content using holograms into augmented reality content and then fuses and reproduces them, thereby increasing students' interest in the image and at the same time in passive virtual reality content. In this way, students' indirect experiential learning effect can be added.

In addition, according to the content transmission apparatus 100 of the present invention, by using the ambient luminance of the mobile device 200 and the glare index to adjust the luminance of the light emitting module 270, glare occurs in the screen luminance of the mobile device 200. It is possible to maintain an appropriate luminance that does not occur. Accordingly, it is possible to reduce eye fatigue of a user using a mobile device.

In addition, according to the present invention, since the light emitting module 270 automatically adjusts the luminance according to the luminance of the surrounding environment using the luminance of the surrounding environment and the glare index of the mobile device 200, power consumption can be saved.

In addition, since students who are expected to be users of the present invention have sensitive eyesight to the surrounding environment, the content is provided by limiting the length of the content, so that vision deterioration that may occur when the virtual reality device is worn for a long time can be reduced.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram illustrating a system for providing 3D interactive content according to an embodiment of the present invention.
2 is a diagram illustrating a method of providing 3D interactive content according to an embodiment of the present invention.
3 is a block diagram schematically showing the configuration of a content transmission device 100 according to an embodiment of the present invention.
4 is a block diagram schematically illustrating the configuration of a mobile device 200 according to an embodiment of the present invention.
5 is a block diagram schematically illustrating the configuration of an HMD device 300 according to an embodiment of the present invention.
6 is a block diagram showing configurations of a content transmission apparatus 100 and a mobile device 200 involved in luminance control according to an embodiment of the present invention.
7 is a diagram for explaining the content transmission device 100 involved in luminance control according to an embodiment of the present invention.
8 is a flowchart illustrating a method for adjusting luminance according to an embodiment of the present invention.
9 is a flowchart illustrating a method of providing content when a distance between a mobile device and a user is less than or equal to a threshold value according to an embodiment of the present invention.
10 is a flowchart illustrating a method for providing an eye protection mode in the content transmission device 100 according to an embodiment of the present invention.
11 is a diagram illustrating a system for providing 3D interactive contents according to another embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in a variety of different forms, only these embodiments make the disclosure of the present invention complete, and fully inform those skilled in the art of the scope of the invention. It is provided to note that the invention is only defined by the scope of the claims.

When one element is referred to as "connected to" another element, it includes both a case where it is directly connected to another element or another element intervenes. On the other hand, when one component is referred to as “directly connected to” another component, it indicates that another component is not intervened.

Like reference numerals designate like elements throughout this specification.

In this specification, “and/or” includes each and every combination of one or more of the recited items.

Although first, second, etc. are used to describe various components, modules and/or sections, it is needless to say that components, modules and/or sections are not limited by these terms. These terms are only used to distinguish one component, module or section from another component, module or section. Accordingly, a first component, a first module, or a first section referred to below may be a second component, a second module, or a second section within the technical spirit of the present invention.

In this specification, the singular form also includes the plural form unless otherwise specified in the phrase.

As used herein, "comprises" and/or "comprising" refers to the presence or addition of one or more other components, modules and/or sections other than the recited components, modules and/or sections. do not rule out

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with meanings commonly understood by those skilled in the art. In addition, the terms defined in the dictionary are not interpreted ideally or excessively unless explicitly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a system for providing 3D interactive content according to an embodiment of the present invention.

The hologram content of the present invention is a 3D interactive hologram in which virtual reality content and augmented reality content by interaction with a real object such as a user are combined.

Referring to FIG. 1, a 3D interactive fusion (synthesis) of virtual reality content (first content) using a hologram and augmented reality content (second content) using a hologram of a real object in a vehicle such as a bus according to the present invention is provided. The hologram content providing system includes a content transmission device 100, a mobile device 200, an HMD device 300, a virtual reality content providing device 400 using a hologram, an augmented reality content providing device 500 using a hologram, and The content convergence devices 600 are connected to each other through a network.

A content transmission device 100, a mobile device 200, an HMD device 300, a virtual reality content providing device 400 using a hologram, an augmented reality content providing device using a hologram 500, and a content convergence device 600 may transmit and receive data and/or information to and from each other through a network.

The network may be composed of networks of various sizes, such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and the like. The network may consist of a wired or wireless network.

The virtual reality screen 700 is a screen on which virtual reality content (first content) is projected, and a separate network connection may not be required.

The device for providing virtual reality contents using holograms 400 includes a content storage device 410 storing virtual reality content using holograms and virtual reality content (first content) using holograms stored in the content storage device 410. It includes a hologram display device 420 that projects and displays on the screen 700, and a first content display controller 430 that receives first time information according to reproduction of the first content displayed on the screen 700. .

The augmented reality content providing device 500 using the hologram includes a real object image capturing device 510 that captures an image of a real object, and converts the captured image of the real object into augmented reality content (second content) using a hologram. It includes an augmented reality content conversion unit 520 that converts, and a second content display controller 530 that receives second time information according to reproduction of the second content displayed on the content display device 300 .

The second content is transmitted by the content transmission device 100 to the content display device 300 such as an HMD device.

Meanwhile, in the present invention, virtual reality content using a hologram is directly projected on the virtual reality screen 700, and augmented reality content using a hologram is transmitted to the user's content display device 300 through the content transmission device 100, and the user can be seen through the eyes of

At this time, when the user wearing the HMD device 300 looks at the virtual reality screen 700, the actual image constituting the augmented reality content is the virtual reality content using holograms projected on the virtual reality screen 700 (first content) and/or an image of a real object on the virtual reality screen 700 . On the virtual reality screen 700, another user may be photographed as a real object by the real object image capture device 510, as will be described later. In this case, the user on the virtual reality screen 700 can watch the second content including the user's image together with the first content through the HMD device 300 . In particular, when both the first content and the second content are produced using a hologram, the interaction between the first content and the second content and the user can be strengthened. The user's enjoyment of watching can be remarkably increased.

Through this, problems such as dizziness of a user who watches virtual reality content or augmented reality content while wearing an HMD can be partially resolved due to increased immersion or concentration of the user.

In addition, in the 3D interactive holographic content providing system according to the present invention, since the first content and the second content are simultaneously viewed, and the first content may be used as a background image of the second content, the first content and the second content Temporal synchronization between them is very important. If temporal synchronization is not appropriate, not only the user's immersion is reduced, but also the liveliness or three-dimensional effect may be greatly reduced due to unnecessary temporal separation.

To this end, in the 3D interactive holographic content providing system of the present invention, the first time information according to the reproduction of the first content and the second time information according to the reproduction of the second content are displayed on the content display device 300 A content convergence device 600 that synchronizes the second content to the first content displayed on the screen 700 is employed.

For example, when both virtual reality and augmented reality are reproduced simultaneously, the content display device 300 requires high-resolution and immersive image rendering through a high-performance graphic processing unit (GPU). However, depending on the quality of the image, the performance of the content display device 300, or network conditions, a time delay may occur in which the virtual reality content or the augmented reality content does not respond as expected, and in particular, the first content and the second content When a difference in playback speed occurs, the user's sense of immersion is inevitably lowered, and apart from the decrease in the fun of content, dizziness due to visual confusion, so-called digital motion sickness, may greatly occur. In order to solve this problem, the content convergence device 600 of the present invention controls the playback speed of the other content to slow down when a time delay occurs in any one of the first content and the second content to reproduce both contents. can be synchronized.

To this end, the content convergence device 600 may control the number of frames per second (FPS) of the first content or the second content. By the way, in the case of the first content, it is reproduced on the screen 700 by the hologram display device 420 while being produced in advance and stored in the content storage device 410, and the graphic performance of the hologram display device 420 suitable for this However, when network performance is secured, it is highly likely that a time delay due to a loading speed delay of the second content directly filmed in the field occurs rather than an unexpected time delay occurring in the first content. Therefore, in a preferred embodiment of the present invention, the first content displayed on the screen 700 using the first time information according to the reproduction of the first content and the second time information according to the reproduction of the second content is synchronized with the second content displayed on the content display device 300 . That is, when the second content is delayed, the first content is synchronized with it.

Conversely, in the present invention, the first time information and the second time information may not be synchronized and a specific time distance may occur so that the first content and the second content are reproduced at a time interval to increase the user's fun. can make it

In particular, since the second content can be provided by photographing the movement of a user watching the first content, the second content can be photographed against the background of the first content, and the playback time of the second content is less than that of the first content. When a temporal distance between the playback time and the second content occurs, a different sense of fun can be given to a user who enjoys content in which the first content and the second content are synthesized (fused). In particular, if the first content is content about a vehicle with a sense of speed, such as an airplane or train, or conversely, if it is a video about a very slow-moving animal such as a turtle or small insect, the person moving on the first content is actually can cause a unique temporal fun by inducing a temporal delay between the first time information and the second time information based on the video frames of the first content and the second content to be shown several seconds later in the second content, which is augmented reality content. .

The content transmission device 100 is connected to a mobile device through wireless or wired communication, and the content display device 300 is composed of Head-Mounted Display (HMD) devices arranged for each seat of the vehicle, and the HMD device 300 ) may be configured to be connected to the mobile device through wireless communication and communicate with the content transmission device 100 connected again wirelessly or wired.

In the present invention, both virtual reality contents and augmented reality contents can be implemented as holograms, and in reality, especially in small provincial cities, experiences on cultural or educational contents that are difficult to directly implement on-site are provided with virtual reality technology, while at the same time providing users, that is, By providing additional images as augmented reality contents that can induce interactions of students mainly for education, students can increase their interest through direct participation.

Virtual reality content viewed through the HMD device 300 is inherently limited in user experience, and in particular, when watching while wearing the HMD device 300 on the head, students complain of dizziness and lose their concentration significantly. do.

In order to fundamentally improve this problem, in the present invention, augmented reality content (second content) using a hologram is provided together so that the augmented reality content can be experienced together with the virtual reality content, but the augmented reality content itself can be used by students themselves Since the video is taken and provided as a hologram, the concentration of students, who are the subjects of education, can be greatly increased. Therefore, rather than simply providing the so-called mixed reality content that combines virtual reality content and augmented reality content to the HMD device 300, the HMD device 300 takes a real image of a real object (user) and augments it using a converted hologram. Real content is provided, and virtual reality content using a hologram is projected onto a separate screen 700 and provided so that users can view it through the HMD device 300 .

The screen 700 also serves as a screen on which virtual reality contents related to cultural heritage sites or famous art performances are projected, but on the other hand, users can be located on the screen and interact with the virtual reality contents, and these real things An image of the object is captured and provided to the user again as augmented reality content.

The screen may be installed on one side of a vehicle in which the system for providing mobile hologram contents according to the present invention is installed, or may be separately installed outside the vehicle since the space of the vehicle is limited. For example, it can be installed in a school playground where the vehicle enters and stops.

Therefore, in the case of using the 3D interactive hologram content providing system of the present invention, the user as a pure spectator and the user who is a shooting target for augmented reality content (referred to as “actor user” in this specification) while watching virtual reality content come into existence

Such an actor user can view himself/herself as augmented reality content as if they are being filmed together with the virtual reality content they are watching, interacting with the virtual reality content.

Conventional augmented reality content overlaps a real image and a three-dimensional virtual image to show it as one image. In the present invention, the real image itself includes another virtual image, virtual reality content (first content). do. Therefore, the augmented reality content (second content) of the present invention is an image of pure reality (ie, an image of an actor user), virtual reality content (first content), and a virtual image added from augmented reality content (third content). ) is composed of three things.

The third content is a lower concept of the second content and may be designed to be suitable for the type of the first content.

For example, if the first content is virtual reality content for famous cultural sites (for example, Mireuksaji Stone Pagoda, Bulguksa Temple, and Cheomseongdae), the third content is the first content, Mireuksaji Stone Pagoda, Bulguksa Temple, and Cheomseongdae. It may be a sign, a user interface capable of reproducing various announcements, or an image indicating remaining viewing time, distance to a final destination, and the like. If the user is a foreigner, a virtual image of a user interface capable of reproducing an announcement in a foreign language such as English or Chinese may be added.

As another example, if the first content is a madangnori performance such as Simcheongjeon, the third content may be a script of lines of the madangnori performance or a virtual image of performance time or bio information of an actor.

Alternatively, a user interface for adjusting the playback speed or volume of the first content may be provided as a virtual image of the third content.

In addition, it is possible to provide third content of virtual images that can provide tactile or olfactory experiences in addition to sight or hearing among the human user's five senses. and feel the clothes worn by the characters in Madangnori of Simcheongjeon or the tactile sensations of the surrounding environment (sea, field, etc.).

Contents using virtual reality and content using augmented reality of the present invention are preferably produced as holograms in order to improve the user's interaction experience, but are not necessarily limited thereto.

Augmented reality contents using holograms are not ordinary augmented reality images that are directly implemented in real objects, but images can be implemented as if they are floating in space based on the principles of light interference (constructive interference), refraction, and diffraction. There is a difference.

The content transmission device 100 is installed inside the vehicle and provides content or an effect control signal according to the content to the mobile device 200. In this regard, the content transmission device 100 transmits the content to the mobile device 200. It can function as a terminal that

The content transmission device 100 can be controlled together with a vehicle providing the 3D interactive hologram content of the present invention.

The first content stored in the content storage device 410 and transmitted to the screen 700 by the hologram display device 420 includes, for example, educational games, educational travel contents, educational movies/dramas, educational music videos, and educational animations. , new experiences, vocational education programs, technical education programs, or subject-related programs such as math and science.

In addition, while viewing the content, in particular, the virtual reality content, the user's personal characteristics such as the user's age, gender, weight, height, and eye fatigue, or the type of content, the resolution of the content, and the saturation of the content. Eye fatigue or cybersickness may be felt differently depending on the characteristics of content. For example, younger users feel eye fatigue more quickly than adult users, and when the luminance of content is high, eye fatigue is increased and cybersickness is amplified.

On the other hand, if the luminance of the content is lowered, the resolution is lowered, preventing the user from being immersed in the virtual reality content. Therefore, it is necessary to reduce the resolution of the content to reduce eye fatigue and avoid cybersickness, while maintaining a certain resolution so as to be immersed in the content.

To this end, the present invention provides content by lowering the resolution, but cannot provide content below a certain level of resolution for immersion, so the length of the content is limited according to the characteristics of the user and the content as described above. In the present invention, virtual reality content (first content ) is desirable to limit the length of

The length of the virtual reality content may be controlled by cutting off the power provided to the content storage device 410 or the hologram display device 420, or by cutting off the power provided to the HMD device 300 worn by the user. there is. Even if the power provided to the HMD device 300 is cut off, the user will be able to watch the first content provided on the screen 700, but if the HMD device 300 does not operate, the virtual reality content (first content) and augmentation Since the degree of immersion due to the combination of real content (second content) is lowered, the user may also stop viewing the first content. So, if the first content causing cybersickness is not directly provided to the HMD device 300, but the content is provided over a limited time in consideration of the characteristics of the user wearing the HMD device 300 and the characteristics of the content , It may also be effective to terminate the content by cutting off the power provided to the HMD device 300 .

In the present invention, a color-corrected 3D image may be provided by increasing color saturation compared to a general 2D image to give a three-dimensional effect. In addition, convergence contents can be provided by synthesizing images using 3D text and particle effects.

In addition, the screen 700 can be configured to secure space for interactive content in which a person moves along with an image in front or behind the screen. On the other hand, it is possible to sufficiently secure the space behind the mesh screen and prevent the view of the image spreading on the ceiling and both sides of the vehicle in which the screen is installed.

Additionally, in order to simplify installation after moving a vehicle such as a bus that provides content according to the present invention, the front frame is assembled using a butterfly nut and is simply attached using a tent pole and a zipper at the rear screen. It may be desirable to

By doing this, it is possible to solve the existing problem of forming images on both sides of the ceiling behind the screen, and it is easy to check the possibility of interacting with a real person.

On the other hand, in this specification, the description that the content transmission device 100 transmits or provides augmented reality content to the HMD device 300 or the mobile device 200 indicates that the content transmission device 100 is the HMD device 300 or the mobile device 200 means transmitting the above content, which should be understood as the same as providing the corresponding content to the user of the HMD device 300 or mobile device 200 disposed on a seat in a vehicle. In addition, the mobile device 200 may perform a function of controlling content playback and display of the HMD device 300 between the content transmission device 100 and the HMD device 300 .

Likewise, in the present specification, actions of the HMD device 300, the mobile device 200, or the content transmission device 100 may be understood as actions performed by a user's manipulation of using or controlling them.

The content transmission device 100 may be, for example, a personal computer (PC) such as a desktop (desk top) or laptop (lap top), a mobile terminal, and a smartphone (smartphone). The content transmission device 100 may be another computing device not illustrated.

The HMD device 300 is connected to the mobile device 200 and receives and displays augmented reality content transmitted to the mobile device 200 by the content transmission device 100 from the mobile device 200 . The HMD device 300 provides an augmented reality effect when displaying content. For example, when the first content is content for viewing the inside of Bulguksa Temple, virtual images for various information on viewing courses while viewing the inside of Bulguksa Temple are provided to users wearing the HMD device 300 .

The HMD device 300 provides time information according to content reproduction to the second content display controller 530 in real time. This can be achieved when the HMD device 300 transmits the time information to the content transmission device 100 and the content transmission device 100 provides it to the second content display control unit 530 . Then, the second content display controller 530 may cause the content convergence device 600 to synchronize playback of the first content and the second content using the time information.

Alternatively, referring to FIG. 11 showing a system for providing 3D interactive content according to another embodiment of the present invention, the content convergence device 600 is directly connected to the virtual reality content providing device 400 or its internal content storage device. The first content may be transmitted from 410, and the image taken by the real object image capturing device 510 may be directly transmitted by converting the augmented reality content conversion unit 520 into an augmented reality image. At this time, through the first time information received from the first content display control unit 430 and the second time information received from the second content playback control unit 530, the content fusion device 600 displays the first content and the second content. may be converged and immediately provided to the content transmission device 100. In this case, the user wearing the HMD device 300 can view virtual reality content and augmented reality content that the content convergence device 600 has converged with. At this time, it may not be necessary to transmit separate content to the virtual reality screen 700 . In this case, the virtual reality screen 700 serves as a blue screen for capturing a real object such as a user for a chroma key. In addition, in this case, the first content and the second content may be provided as a virtual image other than a hologram method, so the hologram display device 420 of the virtual reality content providing device 400 may not be needed. The augmented reality content conversion unit 520 of the augmented reality content providing device 500 also combines the real object image taken by the real object image capturing device 510 with a virtual image other than a hologram to provide to the content transmission device 100. can

The HMD device 300 is arranged for each seat or person in the vehicle, and when notified of wearing the HMD device, users wear the HMD device 300 on their heads.

The HMD device 300 may have, for example, a communication function and a data processing function, and may be worn on a user's head, and the HMD device 300 worn by the user may display a screen display content on it. In this case, the HMD device 300 may display content in a preset size at a preset location. Also, the HMD device 300 may have a shape of glasses. The HMD device 300 may be temporarily fixed to the user's head by a frame or the like.

The HMD device 300 has a first type having a CPU and a display itself, and a second type that can be used by mounting the mobile device 200 to the HMD device 300 . In the case of the second type, the mobile device 200 serves to control the HMD device 300 to display content.

The first type HMD device 300 receives and displays content from the content transmission device 100 .

The second type HMD device 300 is connected to the mobile device 200 through wired or wireless communication. The mobile device 200 receives content from the content transmission device 100 , and the HMD device 300 receives and displays the content from the mobile device 200 .

1 of the present specification describes the second type as an example.

In the case of the second type, the HMD device 300 may be communicatively connected to the mobile device 200, and the HMD device 300 is designed to output content transmitted from the mobile device 200. At this time, the HMD device 300 receives content from the mobile device 200 through a mirrorcast technology or the like, and displays the received content on a display.

The mobile device 200 connected to the HMD device 300 includes a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a global positioning system (GPS) device, and an electronic device. It may be a book terminal, a digital broadcasting terminal, a navigation device, a kiosk, an MP3 player, a digital camera, and other mobile computing devices, but is not limited thereto. In addition, the mobile device 200 may be a watch or a band having a communication function and a data processing function, but is not limited thereto.

As such, the mobile device 200 and the HMD device 300 are connected to each other through wired or wireless communication. For data pairing between the HMD device 300 and the mobile device 200, a mirrorcast technology or a wifi-direct technology may be applied.

2 is a diagram illustrating a method of providing mobile hologram contents according to an embodiment of the present invention.

Referring to FIG. 2, a method of providing 3D interactive hologram contents by converging virtual reality contents (first contents) using holograms and augmented reality contents (second contents) using holograms of real objects in a vehicle such as a bus, A hologram display device projects virtual reality content (first content) using a hologram stored in a content storage device and displays it on a screen (S1210), and a first content display control unit plays back the first content displayed on the screen. Receiving the first time information according to the step (S1220), the step of the real object image recording device taking an image of the real object (S1230), the augmented reality content conversion unit converts the captured image of the real object into augmented reality content using a hologram Converting to (second content) (S1240), the content transmission device transmitting the second content to the content display device (S1250), the second content display control unit converting the second content displayed on the content display device Receiving second time information according to reproduction of (S1260), and the content fusion device displays second content displayed on the content display device on the screen using the first time information and the second time information. and synchronizing with the first content (S1270).

In the method for providing mobile holographic content according to the present invention, the content transmission device 100 is connected to the mobile device 200 through wireless or wired communication, and the content display device 300 is an HMD disposed for each vehicle seat. It is configured as a (Head-Mounted Display) device, and the HMD device 300 is connected to the mobile device 200 through wireless communication, so that it can communicate with the content transmission device 100 connected wirelessly or wired. It consists of

3 is a block diagram schematically showing the configuration of a content transmission device 100 according to an embodiment of the present invention.

Referring to FIG. 3, the content transmission device 100 includes a database 110, a communication unit 120, a content service unit 130, a start signal transmission unit 140, a mobile device control unit 150, and a time information reception unit 160. ), and an effect controller 170, and a terminal controller 180.

Information on augmented reality content (second content) is stored in the database 110, which is provided from the augmented reality content providing device 500 using a hologram.

The communication unit 120 is a component for transmitting and receiving data with various electronic devices. Specifically, the communication unit 120 includes the content transmission device 100 as a mobile device 200, an HMD device 300, a virtual reality content providing device 400 using a hologram, and an augmented reality content providing device 500 using a hologram. , Content convergence device 600, or a separate content providing server (not shown) is connected through a wired communication network or a wireless communication network.

The content service unit 130 extracts the second content from the database 110 and transmits it to the mobile device 200 or the HMD device 300 .

When the communication unit 120 receives a content provision request from the content transmission device 100, the content service unit 130 provides content to the content transmission device 100 in response to the received content provision request.

The start signal transmitter 140 transmits the content reproduction start signal to the mobile device 200 .

The mobile device controller 150 transmits the content to the HMD device 300 through the mobile device 200 and causes the HMD device 300 to display the content.

Instead of displaying the first content on the virtual reality screen 700, as described above, when the first content and the second content are fused in the content convergence device 600 and directly transmitted to the content transmission device 100, The content transmission device 100 is separate from the first content display controller 430 of the virtual reality content providing device 400 using a hologram or the second content display controller 530 of the augmented reality content providing device 500 using a hologram. A time information receiving unit 160 of may be provided.

The time information receiving unit 160 receives time information according to content reproduction, and the effect controller 170 receives time information according to content reproduction from the HMD device 300, and the content convergence device 600 receives the time information according to the time information. ), it is possible to request synchronization of playback times of the first content and the second content, and transmit an effect control signal to be generated at the corresponding time to the HMD device 300 through the content transmission device 100, so that the effect control signal The HMD device 300 synchronizes with the content.

Here, the effect control signal may be a signal for realizing one or more of effects of motion, vibration, snow, rain, wind, dew, fog, scent, and light.

Meanwhile, each of the content service unit 130 and the effect control unit 170 may be implemented by a processor required to execute a program on a computing device. As such, each of the content service unit 130 and the effect control unit 170 may be implemented as a physically independent component or may be implemented in a functionally differentiated form within one processor.

The terminal controller 180 includes a database 110, a communication unit 120, a content service unit 130, a start signal transmitter 140, a mobile device controller 150, a time information receiver 160, and an effect controller 170. ) It is a configuration for controlling the operation of various components of the content transmission device 100 including. The terminal control unit 180 may include at least one arithmetic unit, wherein the arithmetic unit includes a general-purpose central processing unit (CPU), a programmable device element (CPLD, FPGA) implemented to suit a specific purpose, and an application-specific semiconductor. It can be an arithmetic unit (ASIC) or a microcontroller chip.

The terminal controller 180 may control to display content in an eye protection mode. That is, a reference time length is stored inside the content transmission device 100 so that the content is automatically displayed in an eye protection mode when the content time exceeds this value, or is initially stored in the content transmission device 100 or selected by the user. When a program using the eyesight protection mode is executed, content may be displayed in the eyesight protection mode.

In addition, the terminal controller 180 receives the user's biometric information from the mobile device 200, and when it is confirmed that the user is in a tired state, providing content in an eye protection mode according to a command of the terminal controller 180. can

In addition, the terminal controller 180 may control to end the eye protection mode and display the screen in the normal mode. The eye protection mode end command may be defined in various ways, such as clicking or touching an end icon or ending using a separate button. This end icon is provided as third content of an augmented reality image in the HMD device 300. It can be.

4 is a block diagram schematically illustrating the configuration of a mobile device 200 according to an embodiment of the present invention.

In FIG. 4 , the mobile device 200 includes a processor 210, a display unit 220, a communication unit 230, a proximity sensor unit 240, a memory unit 250, an environmental luminance measuring unit 260, and a light emitting module ( 270), the mobile device 200 may include other electronic components, a USB terminal, a Mobile High-Definition Link (MHL) terminal, and the like.

The mobile device 200 receives content from the content transmission device 100 through the communication unit 230 in the first type and transmits the content to the HMD device 300, and in the second type, the processor 210 directly displays the content. A user's input is received from the unit 220 and content is transmitted to the HMD device 300 .

The processor 210 controls the display unit 220 . The processor 210 determines a screen to be displayed on the display unit 220. When the processor 210 identifies a connection with the HMD device 300, it can display video content lists on the display unit 220.

In addition, if the processor 210 identifies a connection with the HMD device 300, it can execute a program or application related to the HMD device 300, and the processor 210 can access a site providing content.

The user can select any one of the displayed contents list, and the processor 210 transmits the selected contents to the HMD device 300 . Content may be downloaded through the Internet or may be stored in the memory unit 250 . A content list for selecting such content may be provided as third content of an augmented reality image in the HMD device 300 .

The processor 210 receives a user's input from the display unit 220 . The processor 210 may change the state of the display unit 220 or transmit data to the HMD device 300 according to a user's input.

The processor 210 may transmit content to the HMD device 300 based on the signal received from the proximity sensor unit 240 . The proximity sensor unit 240 measures the distance between the mobile device 200 and the user and outputs the measured distance to the processor 210 . The processor 210 may determine whether to transmit content to the HMD device 300 based on the measured distance. For example, the processor 210 transmits content to the HMD device 300 when the measured distance is less than or equal to a threshold value.

The environmental luminance measurement unit 260 measures the environmental luminance of the screen portion when the mobile device 200 is connected to the HMD device 300 and worn by the user. This is described based on the second type in which the mobile device 200 is mounted and used on the HMD device 300, and when the HMD device 300 is the first type, the environmental luminance measuring unit 260 of the mobile device 200 etc. may be provided in the HMD device 300 .

The light emitting module 270 controls luminance of the mobile device by including a light emitting unit.

5 is a block diagram schematically illustrating the configuration of an HMD device 300 according to an embodiment of the present invention.

Referring to FIG. 5 , the HMD device 300 includes a communication unit 310, a display unit 320, a controller 330, and a memory unit 340.

The communication unit 310 is a component for transmitting and receiving data with various electronic devices. Specifically, the communication unit 310 is a component for transmitting and receiving information necessary for the HMD device 300 to display content and change the displayed content with the content transmission device 100 or the mobile device 200.

The display unit 320 displays information processed by the HMD device 300 . In particular, the display unit 320 can display content.

The content output through the display unit 320 may show both virtual reality content and augmented reality content, and the augmented reality content is a virtual image of the image captured by the real object image capturing device 510 and the augmented reality content. The three contents may be seen overlapping.

The memory unit 340 stores information necessary for the HMD device 300 to display content and change the displayed content.

The controller 330 controls overall operations of the HMD device 300 in general. For example, the controller 330 may control the communication unit 310 and the display unit 320 by executing programs stored in the memory unit 340 .

The controller 330 controls the content received through the communication unit 310 to be displayed on the display unit 320 .

In addition, the controller 330 may provide time information according to content reproduction to the content transmission device 100 in real time.

The HMD device 300 may be connected to at least one of the content transmission device 100 and the mobile device 200, and receive content from at least one of the content transmission device 100 and the mobile device 200, so that the HMD device Contents can be displayed on the screen of (300).

The controller 330 is a component that controls the operation of various components of the HMD device 300 including the communication unit 310 , the display unit 320 , and the memory unit 340 . The control unit 330 may include at least one arithmetic unit, wherein the arithmetic unit includes a general-purpose central processing unit (CPU), a programmable device device (CPLD, FPGA) implemented for a specific purpose, and an application-specific semiconductor operation. It can be a device (ASIC) or a microcontroller chip.

6 is a block diagram showing configurations of a content transmission apparatus 100 and a mobile device 200 involved in luminance control according to an embodiment of the present invention, and FIG. 7 is a luminance control according to an embodiment of the present invention. It is a diagram for explaining the involved content transmission device 100.

Referring to FIG. 6 , the content transmission device 100 includes an average luminance level calculator 181, a threshold value determiner 182, a storage 183, and a luminance controller 184, and a mobile device 200 ) includes an environmental luminance measuring unit 260 and a light emitting module 270 .

The terminal control unit 180 causes the mobile device 200 to measure the ambient luminance of the mobile device 200 by the environmental luminance measuring unit 260 of the mobile device. The environmental luminance measurement unit 260 is preferably installed to measure the environmental luminance of the screen portion when the mobile device 200 is connected to the HMD device 300 and worn by the user.

The threshold determiner 182 of the content transmission device 100 uses the environmental luminance measured by the environmental luminance measurer 260 and a preset glare index value to determine whether the mobile device 200 or contents where a glare phenomenon begins to occur. A threshold screen luminance of the display device 300 is calculated. The unified glare rating (UGR) is proposed as a practical glare rating system based on general parameters affecting unpleasant glare by the CIE technical committee 3-31. Such a glare index is obtained through Equation 1 below, and looking at the relationship between the glare index and subjective feeling, when the glare index value is "10" or more, you start to feel glare, and when the glare index value is "16" or more When the glare index value is "22" or more, you start to feel uncomfortable, and when the glare index value is "28" or more, you start to feel it too much. Accordingly, an object of the present invention is to provide a mobile device that adjusts the light emission luminance of the light emitting module 270 so that the glare index value may be "10" or less.

Equation 1

Here, Lb is the luminance of the surrounding environment, Ls is the luminance of the light emitting part of the light source in the observer's line of sight, ω is the solid angle of the light emitting part of the light source in the observer's line of sight, and p is the position index of Guth of each light source in the observer's line of sight. The units of Lb (ambient luminance) and Ls (emission luminance) are [cd/m 2 ].

That is, the threshold determiner 182 calculates Ls by substituting the environmental luminance measured by the environmental luminance measurer 260 and the glare index value of “10” into Equation 1, where Ls starts to cause glare. becomes the critical screen luminance of the mobile device 200 or the content display device 300 that

In addition, the threshold determining unit 182 derives a threshold average luminance level corresponding to the threshold light emission luminance, which is derived through a value stored by measuring the screen luminance of the mobile device 200 for each value of the average luminance level. Accordingly, the threshold determination unit 182 further includes a storage unit 183 to store data for determining the threshold emission luminance and the threshold average luminance level.

An average picture level (APL) calculation unit 181 calculates an average brightness level of content based on the brightness of each pixel constituting a frame of input content. The input content is a content signal processed by the content service unit 130 or a content signal input to the content service unit 130 .

The luminance control unit 184 is configured to perform the mobile device ( The luminance of the light emitting module 270 is controlled so that the screen luminance of 200 does not exceed the threshold screen luminance. The light emitting unit provided in the light emitting module 270 is an LED or CCFL, and when the light emitting module 270 is a liquid crystal display device, it is preferable that it is a backlight module.

Since glare can be prevented when the screen luminance of the mobile device 200 or content display device 300 does not exceed the critical screen luminance, when the average luminance level of the content exceeds the critical average luminance level, the light emitting module 270 ) is controlled so that the screen luminance of the mobile device 200 does not exceed the threshold screen luminance.

That is, the luminance control unit 184 is a light emitting module when the average luminance level of the contents calculated by the average luminance level calculator 181 exceeds the threshold average luminance level of the contents derived by the threshold value determiner 182. 270 reduces the luminance of light emitted from the light emitting module 270. For example, as the value is closer to the average luminance level, the degree of reducing the luminance of light emitting module 270 decreases.

In addition, the luminance control unit 184 determines that the average luminance level of the contents calculated by the average luminance level calculation unit 181 is the same as the average luminance level of the contents calculated by the average luminance level calculation unit 181 while the luminance of the light emitting module 270 is reduced by a certain value from the maximum value capable of emitting light. When the average luminance level of the contents derived by the threshold value determiner 182 is less than the threshold average luminance level, the light emission luminance of the light emitting module 270 is increased. Through this, a clear and bright screen luminance is provided.

As a result, the content transmission device 100 of the present invention configured as described above maintains the screen luminance of the mobile device 200 or the content display device 100 properly according to the environmental luminance so as not to cause glare, It provides bright screen luminance so that viewers can comfortably enjoy content.

In addition, since the content transmission apparatus 100 of the present invention configured as described above reduces power consumption by preventing the screen luminance from exceeding the threshold screen luminance, the power of the mobile device 200 can be maintained for a long time. .

8 is a flowchart showing a luminance control method according to an embodiment of the present invention.

In the content transmission device 100, the mobile device 200 measures the luminance of the surrounding environment and transmits the measured environmental luminance to the content transmission device 100 (S910). The critical screen luminance of the mobile device 200 or content display device 100 where glare starts to occur is calculated using the luminance and the preset glare index value (S920). The glare index value is a value determined by the CIE technical committee 3-31, and "10", which starts to feel glare, is applied.

Then, an Average Picture Level (APL) threshold of the contents corresponding to the calculated threshold screen luminance is derived (S930). This is derived through a value stored by measuring the screen luminance of the mobile device 200 for each value of the average luminance level in advance.

Then, the content transmission device 100 calculates the average luminance level of the content using the luminance of each pixel constituting the frame of the input content (S940). The calculated average luminance level of the content is the average luminance level of the content. When the threshold is exceeded (S950), the luminance of the light emitting module 370 is adjusted so that the screen luminance of the mobile device 200 does not exceed the threshold screen luminance (S960). The input content is a content signal processed by the content service unit 130 or a content signal input to the content service unit 130 .

9 is a flowchart illustrating a method of providing content when a distance between a mobile device and a user is less than or equal to a threshold value according to an embodiment of the present invention.

In step S1010, the mobile device 200 identifies a connection between the mobile device 200 and the HMD device 300. The mobile device 200 may execute a program or application when the mobile device 200 and the HMD device 300 are connected.

In step S1020, the mobile device 200 displays a content list on the HMD device 300 when the mobile device 200 and the HMD device 300 are connected. The list may be content provided from the Internet or content stored in the memory unit 250 .

In step S1030, the mobile device 200 receives an input for selecting any one content from the list from the user.

In steps S1040 and S1050, the mobile device 200 may measure a distance between the mobile device 200 and the user, and determine whether to transmit content to the HMD device 300 based on the measured distance. For example, the processor 210 transmits content to the HMD device 300 when the measured distance is less than or equal to a threshold value.

If the measured distance is less than or equal to the threshold value, it may be a case where the user brings the HMD device 300 to his/her face to view content. If the measured distance is equal to or greater than the threshold value, since the user is not viewing content, the mobile device 200 does not transmit the content.

In step S1040 and step S1050, in addition to the distance between the user and the mobile device 200, the mobile device 200 includes means for recognizing that the user wears the mobile device 200. For example, it includes dimming ambient brightness, biometrics such as iris recognition and face recognition.

In step S1060, the mobile device 200 transmits content to the HMD device 300.

Each of the steps described in FIG. 9 may be stored in the memory unit 250 in the form of a program or executed by the processor 210 .

10 is a flowchart illustrating a method for providing an eye protection mode in the content transmission device 100 according to an embodiment of the present invention.

The method for providing the eye protection mode of the content transmission device 100 according to an embodiment of the present invention shown in FIG. 10 may be performed by the content transmission device 100 described above with reference to FIG. 3 . That is, the method of providing the eye protection mode of the content transmission device 100 may be performed by an application program such as application software stored in the database 110 and executed by the terminal control unit 180 .

Referring to FIG. 10 , a method for providing an eyesight protection mode of a content transmission device according to an embodiment of the present invention is performed when an eye protection mode use program initially stored in the content transmission device 100 or selected by a user is executed. A step of displaying the content in a protected mode may be included.

For example, when the vision protection mode use program includes a virtual reality/augmented reality content providing program, a web browser program, an e-book program, etc., when the user executes the web browser program or the e-book program, the terminal control unit 180 may automatically control the content transmission device 100 to display the screen in eye protection mode. The list of programs using the eyesight protection mode may be stored in the database 110 of the content transmission device 100, and may be initially stored in the content transmission device 100, and the user selects a specific program to use the eyesight protection mode. You can also add them to your program list.

When the eye protection mode using program is executed, displaying the screen in the eyesight protection mode may be performed at various times. As an example, the step of adjusting the luminance of the provided content by the content transmission device 100 involved in adjusting the luminance of the present invention and the step of displaying the screen in the eye protection mode may occur simultaneously. For example, while the luminance of the content of the mobile device is adjusted according to the ambient luminance by the content transmission apparatus 100, the luminance, contrast ratio, and color temperature of the mobile device 200 can be adjusted according to the time of the content or the user's condition. .

As an example, the eye protection mode sets the luminance of the content transmitted from the content transmission device 100 to 180 to 185 nits, and sets the contrast ratio (C / R) to 30: 1 to 32: 1, The color temperature may be set to 5000 to 6000K, preferably 5600K. That is, in general, content is displayed with a luminance of 500 nits, a contrast ratio of 1000:1, and a color temperature of 6000 to 7000 K. will be. In addition, compared to displaying a screen with a color gamut of about 72% in general, eyesight protection mode can display a screen with a color gamut of about 25%. In this way, the eyesight protection mode provides a screen similar to that of actual paper by lowering the luminance, contrast ratio, and color temperature, so that the user can comfortably view the screen.

A method for providing an eye protection mode for a content transmission device according to an embodiment of the present invention may include determining, by the terminal controller 180, the length of time that the user uses the content transmission device 100 (S1110).

In a method for providing an eye protection mode for a content transmission device according to an embodiment of the present invention, when the length of use time of the content transmission device 100 has a value greater than the reference time length, according to a command of the terminal control unit 180 It may include providing content in an eye protection mode (S1120). Otherwise, the content transmission apparatus 100 may further include receiving the user's biometric information from the biometric information measuring unit of the mobile device 200 (S1130). The reference time length may have an initial value stored in the content transmission device 100, and may be changed by a user's selection.

In addition, in the method for providing the vision protection mode of the content transmission device according to an embodiment of the present invention, the biometric information measurement unit is included in the mobile device 200, and the user's biometric information can be measured and transmitted to the terminal control unit 180, , The biometric information of the user may include, for example, at least one of the user's blood pressure, body temperature, heart rate, activity level, electrocardiogram, body fat mass, water content, stress response, blood sugar, and blood oxygen saturation.

In addition, in the method for providing the eyesight protection mode of the content transmission device according to an embodiment of the present invention, when the content transmission device 100 receives user biometric information from the mobile device 200 and confirms that the user is in a tired state, the terminal control unit It may include providing content in an eye protection mode according to the command of (180) (S1140, S1150).

In addition, the method for providing the eyesight protection mode of the content transmission device 100 according to an embodiment of the present invention may include displaying a screen in a normal mode according to a user's command to end the eyesight protection mode.

That is, when the user inputs an eye protection mode end command to the content transmission device 100, the terminal controller 180 can control the eyesight protection mode to end and the screen to be displayed in the normal mode. The vision care mode end command may be defined in various ways, such as clicking or touching an end icon or ending using a separate button.

In the method for providing virtual reality and/or augmented reality contents in a vehicle such as a bus of the present invention, the vehicle may include a content transmission device 100, a mobile device 200, an HMD device 300, and seats. In order to form an internal space, seats other than the driver's seat of the vehicle may be removed, and to form or utilize a wide space, it may be custom-made according to a user's request.

In the space, it is possible to watch virtual reality/augmented reality images, provide a comfortable resting environment with air conditioning and heating facilities, and have at least one seat. In addition, convenience facilities, a refrigerator, a sink, a toilet, a sofa and a table may be included on one side.

Since the vehicle is equipped with a means for providing virtual reality/augmented reality contents, it can provide a variety of contents, while being equipped with mobility, the virtual reality/augmented reality contents can be delivered directly to a place where consumers have poor access to virtual reality/augmented reality contents. can be found

Steps of a method or algorithm described in relation to the embodiments of the present invention may be directly implemented as a hardware module, a software module, or a combination of the two executed by a processor. A software module may be RAM (Random Access Memory), ROM, EPROM, EEPROM, flash memory, registers, hard disk, removable disk, CD-ROM, or any other form of computer readable data well known in the art. It may reside on a recording medium. An exemplary recording medium is coupled to a processor, and the processor can derive information from and write information to the recording medium. Alternatively, the recording medium may be integral with the processor. The processor and recording medium may reside within an application specific integrated circuit (ASIC). An ASIC may reside within a mobile device. Alternatively, the processor and recording medium may reside as separate components within the mobile device.

Although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art will understand that the present invention may be embodied in other specific forms without changing its technical spirit or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. In addition, it is natural that cross-combination of the above embodiments also falls within the scope of the present invention.

In addition, it is natural that embodiments in which the components and steps of the server or method of the present invention described above are selectively combined also fall within the scope of the present invention.

100: content transmission device
110: database
120: communication department
130: content service department
140: start signal transmission unit
150: mobile device control unit
160: time information receiver
170: effect control
180: terminal control unit
181: average luminance level calculation unit
182: threshold determining unit
183: storage unit
184: luminance control unit
200: mobile device
210: processor
220: display unit
230: communication department
240: proximity sensor unit
250: memory unit
260: environmental luminance measuring unit
270: light emitting module
300: content display device (HMD device)
310: communication department
320: display unit
330: control unit
340: memory unit
400: virtual reality content providing device
410: content storage device
420: hologram display device
430: first content display control unit
500: augmented reality content providing device
510: real object video recording device
520: augmented reality content conversion unit
530: second content display control unit
600: content convergence device
700: virtual reality screen

Claims (10)

In a method for providing 3D interactive hologram contents by fusing virtual reality contents (first contents) using holograms in a vehicle such as a bus and augmented reality contents (second contents) using holograms of real objects,
Projecting, by a hologram display device, virtual reality content (first content) using a hologram stored in a content storage device and displaying it on a screen;
Receiving, by a first content display controller, first time information according to reproduction of the first content displayed on the screen;
The step of taking an image of a real object by a real object image capturing device;
Converting, by an augmented reality content conversion unit, the captured image of the real object into augmented reality content (second content) using a hologram;
Transmitting, by a content transmission device, the second content to a content display device;
Receiving, by a second content display controller, second time information according to reproduction of the second content displayed on the content display device; and
Synchronizing, by a content convergence device, second content displayed on a content display device with the first content displayed on the screen using the first time information and the second time information;
The content transmission device is connected to the mobile device through wireless or wired communication,
The content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle,
The method of providing 3D interactive hologram contents, characterized in that the HMD device is configured to be connected to the mobile device through wireless communication and communicate with the content transmission device connected thereto wirelessly or wired. According to claim 1,
The method of providing 3D interactive hologram contents, characterized in that the content transmission device is selected from a group including a desktop, a lap top, a mobile terminal, and a smartphone. According to claim 1,
3D interactive hologram content providing method, characterized in that the real object is located on the screen. According to claim 1,
The first content is a 3D interactive hologram content providing method, characterized in that the content for student education. According to claim 3,
The method of providing 3D interactive hologram content, characterized in that the real object is a user who enjoys the first content using the HMD device. In a 3D interactive hologram content providing system that fuses virtual reality content (first content) using holograms and augmented reality content (second content) using holograms of real objects in a vehicle such as a bus,
A hologram display device that projects virtual reality content (first content) using a hologram stored in a content storage device and displays it on a screen;
A first content display controller for receiving first time information according to reproduction of the first content displayed on the screen;
A real object video recording device for taking an image of a real object,
An augmented reality content conversion unit that converts the captured image of the real object into augmented reality content (second content) using a hologram;
A content transmission device for transmitting the second content to a content display device;
A second content display controller configured to receive second time information according to reproduction of the second content displayed on the content display device; and
A content convergence device for synchronizing second content displayed on a content display device with the first content displayed on the screen by using the first time information and the second time information;
The content transmission device is connected to the mobile device through wireless or wired communication,
The content display device is composed of a Head-Mounted Display (HMD) device arranged for each seat of the vehicle,
The HMD device is connected to the mobile device through wireless communication, and is configured to communicate with the content transmission device connected wirelessly or wired. According to claim 1,
The content transmission device is a 3D interactive hologram content providing system, characterized in that selected from a group including a desktop (desk top), lap top (lap top), mobile terminal, smart phone. According to claim 1,
The real object is a 3D interactive hologram content providing system, characterized in that located on the screen. According to claim 1,
The first content is a 3D interactive hologram content providing system, characterized in that the content for student education. According to claim 3,
The real object is a 3D interactive hologram content providing system, characterized in that the user who enjoys the first content using the HMD device.

Images