KR102437155B1 - Realistic Real-Time Learning System Using Hologram Display Device And Method Thereof - Google Patents

Abstract

The present invention relates to a realistic real-time remote learning system and method using a hologram display device. The realistic real-time remote learning system of the present invention comprises: a studio unit for transmitting a lecture video and environmental data together; a server unit for generating a holographic instructor image; and a display unit for controlling a plurality of lighting modules. According to the present invention, it is possible to provide a realistic online lecture video as if a user participates in an offline lecture.

Description

Realistic Real-Time Learning System Using Hologram Display Device And Method Thereof

The present invention is a technology related to a system for allowing lectures to be taken in the form of a realistic instructor through hologram display devices located at different remote locations. That is, the present invention is a technology related to distance learning.

With the development of communication technology, many people can watch the video provided by the server in various places. In addition, remote classes, which are non-face-to-face classes, are spreading due to infectious diseases caused by viruses and the like. Non-face-to-face remote classes have become the only way to hold classes while preventing the spread of the virus.

Until now, many remote classes have been conducted in such a way that, when a lecture is recorded with a camera located at a remote location, a learner connected to a server connected to the camera and a network watches the lecture video. The remote class is a method of conducting a class while sharing lecture videos with learners in real time using separate hardware and software devices, or storing pre-recorded lecture videos on a video streaming server, etc., and delivering them to learners in non-real time.

Such remote classes have the advantage of being able to lecture and learn anytime, anywhere, because there is no need for face-to-face meetings from the point of view of instructors or learners, and there are no time and space restrictions.

However, remote classes using such lecture videos have a problem in that the instructor's sense of reality is lacking and realistic, compared to the face-to-face class, so that the learner cannot be focused for a long time.

In order to solve this problem, on the one hand developing technology for distance learning systems, realistic images based on VR (Virtual Realty), AR (Augmented Realty), and MR (Mixed Realty) can be used to deliver vivid lecture images. Attempts to make it are actively being made.

However, in order to produce a lecture video in such a immersive media method, it is necessary to separately configure an image production apparatus and equipment such as a special professional studio equipped with expensive hardware and software equipment. Accordingly, there is a problem in that high cost is required to produce an image.

Republic of Korea Patent Publication No. 10-2001-0096152 (published on November 07, 2001)

The present invention solves the problem of high cost in providing realistic online lecture video and solves the problem of online class, that is, the problem of lack of concentration in class participation.

The problem to be solved by the present invention is not limited to the problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

The immersive real-time remote learning system using the hologram display device of the present invention for achieving the above object is a studio unit that captures a lecture image, extracts environmental data from the captured lecture image, and transmits it together with the lecture image, A server unit that receives lecture images and environmental data from the studio unit, extracts the lecturer image from the received lecture image, and synthesizes the lecturer image extracted with the background image, shadow image, and reflection image with hologram effect to generate a hologram lecturer image. and a display unit including a plurality of lighting modules, receiving the hologram instructor image and environment data from the server unit, and calculating the hologram instructor image and environment data to control the lighting module. Here, the studio unit includes a camera module for recording lectures conducted in the studio, an environment information input module capable of inputting photographing environment information including information on a light source direction, illuminance, and color tone of lighting in the studio, and a camera module for shooting It includes a lecture video information transmission module that transmits the lecture video and shooting environment information to the server unit in real time.

In addition, the server unit connects to the studio unit through a network and receives an image receiving module for receiving a lecture video, an environment data receiving module for receiving environmental data from the studio unit, and the color tone between the setting object and the background in the lecture image received from the studio unit An instructor image extraction module that extracts a setting object through the difference, an instructor image storage module that stores the extracted instructor image, and a data management module that creates and stores a background image, a shadow image, and a reflection image having a plurality of hologram effects class,

An image synthesis module that generates an instructor composite image by synthesizing a background image, a shadow image, and a reflection image having a hologram effect managed by the data management module with the instructor image extracted from the instructor image extraction module, and the instructor synthesis created by the image synthesis module An instructor image transmission module that transmits an image to the display unit, a lighting control data generation module that extracts lighting control data from environmental data received from the studio unit, and a lighting that transmits lighting control data from the lighting control data generation module to the display unit It may include a control information transmission module.

The display unit includes an instructor image receiving module for receiving the instructor synthesized image from the server unit, an instructor image playback module for reproducing and outputting the received instructor composite image, and a lighting control data receiving module for receiving the lighting control data transmitted by the server; It may include a lighting control module for controlling the chromaticity and illuminance of the lighting of the plurality of lighting modules based on the lighting control data.

The display unit may include a body module having an LCD panel installed on the front surface and having a space therein, and an LED module installed at the top, side, rear and bottom of the internal space of the body module.

In addition, the body module is attached to one surface to open and close, and may further include an eye-shielding plate that can cover the entire or edge of the LCD panel. And the LCD panel may be formed of a transparent panel.

The immersive real-time distance learning method using the hologram display device of the present invention for achieving the above object is (a) a studio filming a lecture image, extracting environmental data from the captured lecture image, and transmitting it together with the lecture image. step; (b) The server unit receives the lecture image and environmental data from the remote studio unit, extracts the lecturer image from the received lecture image, and has a holographic effect background image (B), shadow image (C) and reflection image (D) generating a holographic instructor image by synthesizing the extracted instructor image; (c) the display unit receives the hologram instructor image and environment data from the server unit, and outputs the hologram instructor image while controlling the lighting module by calculating the hologram instructor image and environment data.

Step (b) is (b-1) the image receiving module included in the server unit 20 connects to the studio unit through the network to receive the lecture video, (b-2) environmental data included in the server unit Step of the receiving module receiving environmental data from the studio unit, (b-3) extracting the instructor image through the color difference between the setting object and the background from the lecture image received from the studio unit by the instructor image extraction module included in the server unit step; (b-4) storing the lecturer image from which the lecturer image storage module included in the server unit is extracted; (b-5) the data management module included in the server unit generating a background image, a shadow image, and a reflection image having a plurality of hologram effects, and storing the generated image; (b-6) The image synthesis module included in the server unit synthesizes the instructor image extracted from the instructor image extraction module with a background image, a shadow image, and a reflection image having a hologram effect managed by the data management module to create an instructor composite image to do; (b-7) transmitting, by the instructor image transmission module included in the server unit, the instructor synthesized image generated by the image synthesis module to the display unit; (b-8) extracting lighting control data from the environment data received from the studio unit by the lighting control data generation module included in the server unit; (b-9) the lighting control information transmitting module included in the server unit may include the step of transmitting the lighting control data generated by the lighting control data generating module to the display unit.

In addition, step (c) is (c-1) the instructor image receiving module included in the display unit receives the instructor synthesized image from the server unit, (c-2) the instructor image playback module included in the display unit is received Reproducing and outputting the instructor synthesized image, (c-3) receiving the lighting control data transmitted by the server by the lighting control data receiving module included in the display part, (c-4) lighting control included in the display part The module may include controlling the lighting chromaticity and illuminance of the plurality of lighting modules based on the lighting control data.

(a) step (a-1) the camera module included in the studio unit to take a lecture in the studio; (a-2) enabling the environment information input module included in the studio unit to input photographing environment information including information on a light source direction, illuminance, and color tone of lighting in the studio; (a-3) the lecture image information transmitting module included in the studio unit may include a step of transmitting the lecture video and shooting environment information to the server unit in real time.

The present invention makes it feel like participating in an offline lecture while participating in an online lecture in consideration of the economic situation and the level of lecture video in providing a realistic lecture video.

1 is an overall configuration diagram of an immersive real-time distance learning system using a hologram display device according to an embodiment of the present invention.
FIG. 2 is a basic configuration diagram of a studio unit according to an embodiment of the present invention of FIG. 1 .
3 and 4 are diagrams illustrating a web interface formed in the studio unit.
5 is a basic configuration diagram of a server according to an embodiment of the present invention of FIG. 1 .
6, 7, and 8 are diagrams illustrating examples of a hologram effect image set according to an embodiment of the present invention.
9 is a configuration diagram of a display unit according to an embodiment of the present invention of FIG. 1 .
10 is a perspective view of a display unit according to an embodiment of the present invention of FIG. 1 .
11 and 12 are diagrams illustrating an operating state of the display unit of FIG. 1 according to an embodiment of the present invention.
13 is a view illustrating an instructor output from a display unit after synthesizing a hologram of an instructor image according to an embodiment of the present invention.
14 is a view showing a display unit according to another embodiment in which the eye shield is installed.
15 is a view showing a display unit of another embodiment in which the gaze control module is installed.
16 is a diagram illustrating an instructor output from the display unit before synthesizing a hologram of an instructor image according to an embodiment of the present invention.
17 is a flowchart of an immersive real-time distance learning method of the present invention according to an embodiment of the present invention.

Advantages and features of the present invention and an apparatus for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below. The present invention can be embodied in a variety of different forms, and only this embodiment is provided to complete the disclosure of the present invention and to completely inform those of ordinary skill in the art to the scope of the invention. will be.

The claims of the present invention may be defined by the claims. In addition, like reference numerals used throughout this specification refer to like elements.

Hereinafter, realistic real-time distance learning using a hologram display device will be described in detail with reference to FIGS. 1 to 17 .

First, an immersive real-time distance learning system using a hologram display device will be described in detail with reference to FIG. 1 .

1 is an overall configuration diagram according to an embodiment of the present invention.

The immersive real-time distance learning system 1 using a holographic display device captures lecture images at lecture sites in different remote locations, and extracts instructor images from the captured lecture images. Then, after synthesizing the extracted instructor image into a holographic image, it is transmitted to the display unit 30 at a different remote location so that it can be reproduced.

The immersive real-time distance learning system 1 using such a hologram display device includes a studio unit 10, a server unit 20, and a display unit 30 as components, such as an instructor image and lighting in a place where the instructor is located. By extracting the environmental factors of the teacher and providing it as a hologram instructor video that can be reproduced again, it makes use of the advantages of remote classes and high online effects can be expected. Here, the studio unit 10, the server unit 20, and the display unit 30 are each component for implementing the immersive real-time distance learning system 1 using a hologram display device, and are generally connected and connected through a network. do.

Hereinafter, components constituting an immersive real-time distance learning system using a hologram display device will be described in detail with reference to FIGS. 2 to 10 .

FIG. 2 is a basic configuration diagram of a studio unit according to an embodiment of the present invention of FIG. 1 , and FIGS. 3 and 4 are diagrams illustrating a web interface formed in the studio unit. And Figure 5 is a basic configuration diagram of a server according to an embodiment of the present invention of Figure 1, Figures 6, 7, 8 is a view showing an example of a hologram effect image set according to an embodiment of the present invention, 9 is a configuration diagram of a display unit according to an embodiment of the present invention of FIG. 1 . And FIG. 10 is a perspective view of the display unit of FIG. 1 according to an embodiment of the present invention.

The studio unit 10 shoots a lecture image, receives environmental data for the captured lecture image, and transmits it to the server unit 20 together with the lecture image. Such a studio unit 10 is a place where a plurality of lights, chroma key screens, computers, etc. are installed, and the lecturer shoots lectures. In this case, there may be a plurality of studio units 10 located at different positions. The studio unit 10 captures the lecture video in real time, and captures the instructor from the captured video. In addition, the captured image may be transmitted to the server unit 20 connected through a wired or wireless network. Moreover, the studio unit 10 is loaded with an Internet network program such as a web browser that connects to the server unit 20 to exchange information in real time. Here, the studio unit 10 may be collectively referred to as a lecture image capture studio.

The studio unit 10 includes a camera module 101 , an environment information input module 102 , and a lecture image information transmission module 103 . In addition, the studio unit 10 may further include various facilities such as a recording module 104 , a lighting module 105 , a screen module 106 and a soundproofing device module 107 as components.

The camera module 101 may be a camera for photographing an instructor conducting a lecture. Such a camera module 101 may be one computer or a program module installed in one computer, and may be installed in one computer with the environment information input module 102 and the lecture image information transmission module 103 .

The environment information input module 102 allows the user to input various information about lighting in the studio. The environment information input module 102 is installed in a computer to display a screen as shown in FIG. 3 . The environment information input module 102 allows the user to input the direction, illuminance, and color tone of the light source, and the direction and illuminance of the light source illuminating the instructor when captured by the camera module 101 and the environment photographed by the camera module 101 . and color tones. Here, the color tone of the light source is an example of an object set in a lecture image captured by the server unit 20 to be described later, and may be an element capable of extracting an instructor as a difference in color tone.

The lecture video information transmission module 103 may be a module that enables data exchange between terminals and servers. Such lecture video information transmission module 103 transmits data such as Wi-Fi, Bluetooth, Internet, LAN (Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), LTE and 5G. It may include a network module that enables transmission and reception.

The recording module 104 records the instructor's voice. The recording module 104 is connected to a microphone attached to the instructor's clothes, and may be formed as a device for recording the instructor's voice. The recording module 104 transmits the recorded audio data to the lecture image information transmission module 103 so that it can be transmitted to the server unit 20 .

The lighting module 105 may be a lighting that irradiates light toward the screen module 106 of the studio unit 10 . The intensity and direction of illuminating the light of the lighting module 105 are controlled by the input value of the computer, so that the lighting module 105 may emit light of various intensity toward the chroma key screen in various directions.

The screen module 106 may be formed as a chroma key screen on which various background screens can be applied.

The sound insulation module 107 is installed outside the studio so that the lecture sound of the instructor emitted from the inside of the studio to the outside does not go out. This soundproofing device module 107 may be a sound absorbing material attached to the inside and outside of the studio.

The server unit 20 is a device for processing data transmitted from the studio unit 10 and transmitting it to the display unit 30 . As an example, it may be a computer capable of data communication. In this case, the server unit 20 may be composed of a plurality of modules capable of performing each function so as to exert a plurality of functions. The server unit 20 is connected to the studio unit 10 through the network, receives the lecture image through the lecture image information transmission module 103, and the lecture image capture environment transmitted through the environment information input module 102 Receive information, and extract an instructor image from the received lecture image. Then, a hologram effect image set to be synthesized with the extracted instructor image is created or selected, and the hologram instructor image is synthesized and generated, and the generated hologram instructor image is transmitted to the display unit 30 .

In other words, the server unit 20 receives the lecture image from the studio unit 10 through the network, extracts the lecturer image from the received lecture image, synthesizes and generates a hologram lecturer image, and then displays it to the display unit 30 . to be able to send Such a server unit 20 includes an image receiving module 201, an environmental data receiving module 202, an instructor image extraction module 203, an instructor image storage module 204, and a data management module 205 as components. do. And it may further include an image synthesis module 206 , an instructor image transmission module 207 , a lighting control data generation module 208 , and a lighting control information transmission module 209 . Here, the image receiving module 201 may be connected to the studio unit 10 through a network to receive the captured lecture image. And the environment data receiving module 202 may receive the environment data from the studio unit (10).

The instructor image extraction module 203 extracts the setting object from the lecture image received from the studio unit 10 through the color difference between the setting object and the background, that is, chroma-key. Through this, the instructor image extraction module 203 allows only the instructor image to be extracted using the difference in color tone.

The instructor image storage module 204 is a device for storing the extracted instructor image. That is, the instructor image storage module 204 may be formed as a memory device for storing data.

The data management module 205 may generate and store a background image (B), a shadow image (C), and a reflection image (D) having a hologram effect on the instructor image extracted from the instructor image extraction module 203 . And the stored images are managed so that they can be combined with the extracted instructor video. Here, the background image (B) having the hologram effect stored in the data management module 205 is, as shown in FIG. 6 , the upper left of the blackboard, the lower left of the blackboard, the center of the blackboard, the lower center of the blackboard, and the blackboard. It can be a background image with a holographic effect, such as the upper right of the , the lower right of the blackboard. And, as shown in FIG. 7, the shadow image (C) is an image in which a shadow is located in the upper-left direction of the instructor, an image in which a shadow is located in the lower-left direction of the instructor, an image in which a shadow is located under the instructor, and below the instructor. As an example of the direction, it may be an image with a shadow positioned at 6 o'clock, an image with a shadow positioned at 1 o'clock as an example of the upper right of the instructor, and an image with a shadow positioned at the lower right of the instructor, for example, at 5 o'clock.

In addition, as shown in FIG. 8 , the reflected image (D) may be a reflected image based on the instructor.

The image synthesis module 206 is a background image (B), a shadow image (C) and a reflection image (D) having a hologram effect managed by the data management module 205 on the instructor image extracted from the instructor image extraction module 203 . by synthesizing the instructor composite image.

The data management module 205 may select a background image, select a shadow image, select a reflection image, and then create a hologram effect image set according to the extracted instructor image and environment data. The data management module 205 may select an effect image that already exists from among data stored according to the extracted instructor image and environment data, that is, lecture image capture environment information, and display the hologram effect. Here, the background image, shadow image, and reflection image for generating the hologram effect image set are virtual images for realizing the hologram effect. It doesn't matter if you don't consider it.

The background image, shadow image, and reflection image shown in FIGS. 6, 7, and 8 are merely for realizing a more vivid hologram effect while synthesizing a flat instructor image into a three-dimensional hologram instructor image, a background image, a shadow image, Reflective images are not limited thereto.

The instructor image transmission module 207 transmits the instructor synthesized image generated by the image synthesis module 206 to the display unit 30 .

The lighting control data generation module 208 extracts lighting control data from the environment data received from the studio unit 10 . The lighting control information transmission module 209 transmits the lighting control data generated by the lighting control data generation module 208 to the display unit 30 . That is, the lighting control information transmission module 209 becomes a transmission module for transmitting data to the display unit 30 .

The display unit 30 includes a plurality of lighting modules 105, receives the hologram instructor image and lighting control data from the server unit 20, reproduces the hologram instructor image, and calculates the lighting control data to operate the lighting module 105. control

Such a display unit 30 may be connected to the server unit 20 through a network to receive, store, reproduce, and output the generated hologram instructor image. At this time, the display unit 30 receives the lecture environment data transmitted by the studio unit 10 to the server unit 20, and controls the operation of a plurality of LED modules, such as the intensity of light output from the lighting and the operation sequence of the lighting. The lighting is operated while controlling the display unit 30 using the available lighting control data. Here, as shown in FIG. 9 , the display unit 30 includes a body module 310 , an LCD panel 320 , an LED module 330 , a blindfold plate 340 , an instructor image receiving module 301 , and an instructor. It includes an image reproduction module 302 , a lighting control data receiving module 303 , and a lighting control module 304 . For example, the display unit 30 has the body module 310 formed as shown in FIG. 10 as a basic skeleton, the LCD panel 320 is installed on the front side of the body module 310, the body module 310 The LED module 330 may be installed at the top, both sides, back, and bottom of the space formed inside the . Here, the LCD panel 320 may be formed of a transparent panel, so that the inside of the body module 310 can be seen from the outside. In this case, the LCD panel may be transformed into a panel such as an LED panel or an OLED panel. Such an LCD panel may be formed by variously deforming only the structure that is a transparent panel.

Hereinafter, an operating state of the display unit will be described in detail with reference to FIGS. 11 to 14 .

11 and 12 are views showing the operating state of the display unit according to an embodiment of the present invention of FIG. 1 , and FIG. 13 is a view of an instructor outputted from the display unit after synthesizing a hologram of an image according to an embodiment of the present invention. is a diagram showing

The LED module 330 may be installed on the top, bottom, side, and rear of the body module 310 as shown in FIG. 11 . At this time, the LED module 330 is through the lighting control module 304 installed inside the body module 310, the LED module 330 installed on the rear side, the LED module 330 installed on the side, and the LED module installed on the bottom ( 330) may be turned on individually or simultaneously. Through this, as shown in FIG. 12 , the display unit 30 allows a gradation to appear inside the body module 310 to create a three-dimensional environment.

In such an environment, the display unit 30 may receive the hologram instructor image from the server unit 20 and output the hologram instructor image as shown in FIG. 13 .

Hereinafter, structures of various display units will be described in detail with reference to FIGS. 14 and 15 .

14 is a view showing a display unit installed with an eye-shielding plate according to another embodiment, and FIG. 15 is a view showing a display unit according to another embodiment in which a gaze control panel is installed.

As shown in FIG. 12 , the display unit 30 has a front surface in the first display unit 30-1 and the body module 310 in which the eye-shielding plate 340 is installed on the front side of the body module 310 in addition to the structure, as shown in FIG. It may be formed as the second display unit 30-2 in which the gaze control module 350 is formed toward the direction. Here, the first display unit 30 - 1 and the second display unit 30 - 2 are all included in the display unit 30 , the instructor image receiving module 301 , the instructor image reproducing module 302 , and lighting control data. It may include a receiving module 303 , a lighting control module 304 , a body module 310 , and an LCD panel 320 and an LED module 330 .

First, as shown in FIG. 14 , the first display unit 30 - 1 may include a rectangular thin-plate eye-shielding plate 340 . The eye shield plate 340 may be detachably installed on one side of the body module 310 . The eye shield plate 340 opens and closes to protect the front or edge of the transparent LCD panel, and to block the eyes of the audience from the side of the screen output to the transparent LCD panel. In addition, as shown in FIG. 15 , the second display unit 30 - 2 may include a gaze control module 350 protruding from left and right sides and up and down inside the body module on one surface of the body module. This gaze control module 350 prevents the light coming from the side from flowing into the LCD panel 320 and likewise blocks the gaze from the side of the audience.

Hereinafter, the appearance of the instructor output from the display unit before hologram synthesis will be described with reference to FIG. 16 .

16 is a diagram illustrating an instructor output from the display unit before synthesizing a hologram of an image according to an embodiment of the present invention.

When a background image having a hologram effect, a shadow image, and a reflection image are not synthesized in the instructor's image, as shown in FIG. 16 , a flat image may be output to the display unit.

As described above, based on the components constituting the immersive real-time remote learning system using the hologram display device, a immersive real-time remote learning method using the hologram display device will be described in detail.

In the description of the immersive real-time distance learning method using the hologram display device, the subject executing each step may be a component constituting the immersive real-time distance learning system using the hologram display device.

Accordingly, in order to make the description of the immersive real-time distance learning method using the hologram display device concise and clear, the description of the subject of the method overlapping with the immersive real-time remote learning system using the hologram display device will be omitted.

Hereinafter, an immersive real-time remote learning method using a hologram display device will be described in detail with reference to FIG. 17 .

17 is a flowchart of an immersive real-time distance learning method using the hologram display apparatus of the present invention according to an embodiment of the present invention.

In the immersive real-time distance learning method using a hologram display device, the studio unit 10 takes a lecture image, extracts the captured lecture image and environmental data, and transmits it together with the lecture image (a), the server unit 20 Receives lecture images and environmental data from the remote studio unit 10, extracts lecture images from the received lecture images, and has a background image (B), a shadow image (C) and a reflection image ( It proceeds to step (b) of synthesizing the instructor image extracted in D) and synthesizing the hologram instructor image. After that, the display unit 30 receives the hologram instructor image and environment data from the server unit 20, calculates the hologram instructor image and environment data, and controls the lighting module 105 while controlling the lighting module 105 (c) step of outputting the hologram instructor image can proceed with Here, in step (a), the camera module 101 included in the studio unit 10 records a lecture being conducted in the studio (a-1) step (S602), environmental information included in the studio unit 10 (a-2) step (S603) in which the input module 102 can input shooting environment information including information on the light source direction, illuminance and color tone of the lighting in the studio, lecture image information included in the studio unit 10 The transmitting module 103 may transmit the captured image to the server unit 20 in real time (S604), and (a-3) step (S605) of transmitting the shooting environment information to the server unit 20 in real time may include Step (a) may further include a step (S601) of accessing the web interface of the studio in progress before step (a-1) proceeds. And in step (b), the image receiving module 201 included in the server unit 20 connects to the studio unit 10 through the network and receives the lecture video image (b-1) (S606) and the server unit The environment data receiving module 202 included in (20) receives the shooting environment data from the studio unit 10 (b-2) step (S607), the instructor image extraction module 203 included in the server unit 20 ) extracting the instructor through the chroma-key from the lecture image received from the studio unit 10 (B-3) step (S608), the instructor image storage module 204 included in the server unit 20 ) of (b-4) storing the extracted instructor image (S609) and the data management module 205 included in the server unit 20 generates a background image, a shadow image, and a reflection image having a plurality of hologram effects (S611), storing the generated image (S612), and may include a step (b-5) of reflecting this to the extracted instructor image. And the image synthesis module 206 included in the server unit 20 has a background image, a shadow image and a reflection image having a hologram effect managed by the data management module 205 on the instructor image extracted from the instructor image extraction module 203 . (b-6) step (S614) of synthesizing and generating a holographic instructor synthesized image, the instructor image transmission module 207 included in the server unit 20 transmits the holographic instructor synthesized image generated by the image synthesis module 206. The lighting control data generation module 208 included in the (b-7) step (S615) and the server unit 20 transmits the lighting control data from the environment data received from the studio unit 10 to the display unit 30. Extracting (b-8) step (S610) and the lighting control information transmission module 209 included in the server unit 20 transmits the lighting control data from the lighting control data generation module 208 to the display unit 30 (b-9) step (S616) may be included. And in step (c), the instructor image receiving module 301 included in the display unit 30 receives the instructor synthesized image, that is, the hologram instructor image, from the server unit 20, (c-1) step (S617), the display (c-2) step (S621, S622) in which the instructor image reproducing module 302 included in the unit 30 reproduces and outputs the received instructor synthesized image (S621, S622), the lighting control data receiving module included in the display unit 30 The lighting control module 304 included in the (c-3) step (S619) and the display unit 30 of receiving the lighting control data transmitted by the server unit 20 received at 303 is based on the lighting control data to control the chromaticity and illuminance of the illumination of the plurality of illumination modules 105 as (c-4) step (S620).

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing the technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

1: Realistic real-time distance learning system using hologram display device
10: studio
101: camera module
102: environment information input module 103: lecture video information transmission module
104: recording module 105: lighting module
106: screen module 107: sound insulation module
20: server unit
201: image receiving module 202: environmental data receiving module
203: instructor image extraction module 204: instructor image storage module
205: data management module 206: image synthesis module
207: instructor image transmission module 208: lighting control data generation module
209: lighting control information transmission module
30: display unit
301: instructor video receiving module 302: instructor video playback module
303: lighting control data receiving module 304: lighting control module
310: body module 320: LCD panel
330: LED module 340: eye shield plate
350: eye control module
A: Web interface
B: Background image with hologram effect
C: shadow image
D: Reflected image

Claims (10)

a studio unit 10 for taking a lecture image, and transmitting environmental data along with the lecture image when environmental data for the captured lecture image is input;
The lecture image and environmental data are received from the remote studio unit 10, the instructor image is extracted from the received lecture image, and the background image (B), the shadow image (C) and the reflection image (D) having a hologram effect are extracted. A server unit 20 that generates a hologram instructor image by synthesizing the instructor images and
It includes a plurality of lighting modules 105, and receives the hologram instructor image and environmental data from the server unit 20, and calculates the hologram instructor image and environmental data to control the lighting module 105. do,
The studio unit 10,
A camera module 101 for recording lectures in the studio, and
An environment information input module 102 capable of inputting photographing environment information including information on the light source direction, illuminance, and color tone of lighting in the studio;
and a lecture image information transmission module 103 for transmitting the lecture image and shooting environment information photographed by the camera module 101 to the server unit 20 in real time,
The server unit 20,
An image receiving module 201 that connects to the studio unit 10 through a network to receive a lecture video, and
an environment data receiving module 202 for receiving environmental data from the studio unit 10;
Instructor image extraction module 203 for extracting a setting object through the difference in color between the setting object and the background from the lecture image received from the studio unit 10;
Instructor image storage module 204 for storing the extracted instructor image, and
A data management module 205 for generating and storing a background image (B) having a plurality of hologram effects, a plurality of shadow images (C), and a plurality of reflection images (D);
An image synthesis module 206 for generating an instructor composite image by synthesizing a background image, a shadow image, and a reflection image having a hologram effect managed by the data management module 205 with the instructor image extracted from the instructor image extraction module 203 and ,
Instructor image transmission module 207 for transmitting the instructor synthesized image generated by the image synthesis module 206 to the display unit 30;
A lighting control data generation module 208 for extracting lighting control data from the environmental data received from the studio unit 10;
and a lighting control information transmission module 209 for transmitting the lighting control data generated by the lighting control data generation module 208 to the display unit 30,
The display unit 30,
Instructor image receiving module 301 for receiving the instructor composite image from the server unit 20;
Instructor image playback module 302 that reproduces and outputs the received instructor composite image;
A lighting control data receiving module 303 for receiving the lighting control data transmitted by the server unit 20;
A lighting control module 304 for controlling the chromaticity and illuminance of the lighting of the plurality of lighting modules 105 based on the lighting control data;
A body module 310 having a space formed therein by installing an LCD panel 320 formed of a transparent panel on the front side;
An LED module 330 installed at the top, side, back and bottom of the inner space of the body module 310, and
It includes a gaze control module 350 that is formed to protrude from one side of the body module toward the front so that light flowing from the side does not flow into the LCD panel 320,
The background image (B) having a plurality of hologram effects,
A background image having a plurality of hologram effects in the upper left of the blackboard, a background image having a plurality of hologram effects in the lower left of the blackboard, a background image having a plurality of hologram effects in the center of the blackboard, and a plurality of hologram effects in the lower center of the blackboard a background image having a background image, a background image having a plurality of hologram effects on the upper right of the blackboard, and a background image having a plurality of hologram effects on the lower right side of the blackboard,
A plurality of shadow images (C),
An image with a shadow positioned in the upper-left direction of the instructor, an image with a shadow positioned in the lower-left direction of the instructor, an image with a shadow positioned downward of the instructor, an image with a shadow positioned in the upper-right of the instructor, and a shadow positioned in the lower-right of the instructor. contain images;
A plurality of reflection images (D),
An image with a reflective image positioned toward the upper left of the instructor, an image with a reflective image positioned toward the lower left of the instructor, an image with a reflective image positioned downward of the instructor, an image with a reflective image positioned on the upper right of the instructor, and lower right of the instructor An immersive real-time distance learning system using a hologram display device, including an image on which a reflection image is located. delete delete delete delete delete (a) the studio unit 10 taking a lecture image, extracting environmental data from the captured lecture image and transmitting it together with the lecture image;
(b) the server unit 20 receives the lecture image and environmental data from the remote studio unit 10, extracts the lecturer image from the received lecture image, and a background image (B) and a shadow image (C) having a hologram effect ) and generating a holographic instructor image by synthesizing the extracted instructor image with the reflected image (D);
(c) the display unit 30 receives the hologram instructor image and environmental data from the server unit 20, calculates the hologram instructor image and the environmental data, and outputs the hologram instructor image while controlling the lighting module 105 do,
(a) step,
(a-1) the camera module 101 included in the studio unit 10 filming a lecture being conducted in the studio;
(a-2) step of the environment information input module 102 included in the studio unit 10 to input shooting environment information including information on the light source direction, illuminance, and color tone of lighting in the studio;
(a-3) the lecture video information transmitting module 103 included in the studio unit 10 transmits the lecture video and shooting environment information to the server unit 20 in real time,
(b) step,
(b-1) receiving, by the image receiving module 201 included in the server unit 20, a lecture-captured image by connecting to the studio unit 10 through a network;
(b-2) receiving, by the environment data receiving module 202 included in the server unit 20, the environment data from the studio unit 10;
(b-3) extracting, by the lecturer image extraction module 203 included in the server unit 20, extracting the lecturer image through the color difference between the setting object and the background from the lecture image received from the studio unit 10;
(b-4) storing the extracted instructor image by the instructor image storage module 204 included in the server unit 20;
(b-5) the data management module 205 included in the server unit 20 generating a background image having a plurality of hologram effects, a plurality of shadow images, and a plurality of reflection images, and storing the generated image;
(b-6) the image synthesis module 206 included in the server unit 20 has a hologram effect managed by the data management module 205 on the instructor image extracted from the instructor image extraction module 203, a shadow generating an instructor composite image by synthesizing the image and the reflected image;
(b-7) transmitting, by the instructor image transmission module 207 included in the server unit 20, the instructor synthesized image generated by the image synthesis module 206 to the display unit 30;
(b-8) extracting lighting control data from the environment data received from the studio unit 10 by the lighting control data generation module 208 included in the server unit 20;
(b-9) the lighting control information transmission module 209 included in the server unit 20 transmits the lighting control data generated by the lighting control data generation module 208 to the display unit 30,
(c) step,
(c-1) receiving, by the instructor image receiving module 301 included in the display unit 30, the instructor synthesized image from the server unit 20;
(c-2) the instructor image reproducing module 302 included in the display unit 30 reproduces and outputs the received instructor composite image;
(c-3) receiving, by the lighting control data receiving module 303 included in the display unit 30, the lighting control data transmitted by the server unit 20;
(c-4) the lighting control module 304 included in the display unit 30 controls the chromaticity and illuminance of the lighting of the plurality of lighting modules 105 based on the lighting control data,
The display unit 30 has a gaze control module 350 protruding from one side to block the inflow of light from the side of the body module to the LCD panel 320,
The background image (B) having a plurality of hologram effects,
A background image having a plurality of hologram effects in the upper left of the blackboard, a background image having a plurality of hologram effects in the lower left of the blackboard, a background image having a plurality of hologram effects in the center of the blackboard, and a plurality of hologram effects in the lower center of the blackboard a background image having a background image, a background image having a plurality of hologram effects on the upper right of the blackboard, and a background image having a plurality of hologram effects on the lower right side of the blackboard,
A plurality of shadow images (C),
An image with a shadow positioned in the upper-left direction of the instructor, an image with a shadow positioned in the lower-left direction of the instructor, an image with a shadow positioned downward of the instructor, an image with a shadow positioned in the upper-right of the instructor, and a shadow positioned in the lower-right of the instructor. contain images;
A plurality of reflection images (D),
An image with a reflective image positioned toward the upper left of the instructor, an image with a reflective image positioned toward the lower left of the instructor, an image with a reflective image positioned downward of the instructor, an image with a reflective image positioned on the upper right of the instructor, and lower right of the instructor An immersive real-time distance learning method using a hologram display device, including an image in which a reflection image is located. delete delete delete

Images