KR20010067612A - Method and system for virtual reality based on internet tele-lecturing - Google Patents

Abstract

PURPOSE: An Internet remote lecture system based on a vertical reality and a method thereof are provided for a lecturer to offer a streaming of video lecture data for multi-casting audio/video data having a high compression rate based on an MPEP-4 one by one and a navigation through an avatar in a cyber lecture room to a plurality of learners located in remote places through a multi cast network. CONSTITUTION: An LOD(lecture on demand) server(11) has a conference manager and a session manager for performing one-way 1:N group communication and comprises a media server playing a role of an audio/video streaming server, a lecture managing DB and a chatting server module. In addition, the LOD server(11) stores VR/MPEG/Presentation for an Internet lecture. A lecturer client(22) registers and stores moving picture/audio data, word processor as the MS-word and a teaching material as a HTML web document in the LOD server(11) for a cyber lecture through a web camera/speaker and may check a lecturing process. A learner client(33) takes an Internet remote lecture by receiving a channel searching browser function for searching a multi cast channel wanted by a user, a VRML media browsing, a MPEP-4 media browsing, a lecture presentation browsing, an avatar and an automation function capable of performing an animation supplying a voice and a sound, and lecture data as a VR/MPEG/Presentation from the LOD server(11) by providing a synchronization between multi media.

Description

Virtual Reality-based Internet Lecture System and Method {METHOD AND SYSTEM FOR VIRTUAL REALITY BASED ON INTERNET TELE-LECTURING}

The present invention relates to a Virtual Reality (VR) based Internet Tele-Lecturing system and method, and in particular, lecturers and a large number of participants geographically separated from the Internet through MPEG-4 based The present invention relates to a virtual reality-based internet remote lecture system and method for providing realistic navigation through avatars in a remote video lecture and a cyber classroom that transmits video / audio data having high compression ratio to 1: N multicasting.

In general, with the development of the Internet and communication technology, Internet services of cyber universities including digital libraries, distance learning systems, and VOD systems are increasing in a high-speed communication network environment.

1 is a block diagram of a conventional Internet distance education system.

Conventional distance education software products include lecture texts edited by MS-Power Point, MS-Word and HWP through internet and PC communication using lecturer's voice, Mpeg, avi, etc. Attach class auxiliary materials such as videos, images such as Flash, and URL links, etc., and save them to an Internet remote lecture server (Lecture On Demand (LOD) server). Through the client, the lecturer's video captured by the PC-Camera can be listened to in the cyber space without a separate web camera or other processing device.

This distance education software is based on the Internet-based virtual education, Internet School (Internet School), supplementary and home school classes of kindergarten, elementary, middle and high schools, reinforcement and virtual university lectures, various seminars, corporate training centers and the public You can use it in a course.

In addition, the video captured from the lecturer's computer's web camera is MPEG-2 based, with 1: N multicast streaming to multiple students through a streaming video server in a remote lecture server on a multicast network. It is provided as a video and slides together with the lecturer's voice.

It is known that broadcasting quality of streaming service of MPEG-1 video / audio is good on 10BaseT Ethernet, and broadcasting quality of MPEG-2 video / audio data is good at 100Mbps.

However, the conventional MPEG-2 based remote lecture system requires high bandwidth and low compression rate due to the connection of more than 100 users, so packet loss and packet delay are low. Quality of Service (QoS) may be deteriorated, and there is a problem that the reality is lowered by taking a lecture in a two-dimensional environment (Web2D).

SUMMARY OF THE INVENTION The present invention has been proposed to solve the problems of the prior art, and an object of the present invention is to provide video / audio data having a high compression rate based on MPEG-4 through a multicast network. Application technology to provide 1: N multicasting video lecture data streaming and realistic navigation through avatars in cyber classrooms {Avatar technology that supports voice and sound in virtual environment, media integrated browsing technology (VRML Media Browsing, MPEG-4 Media Browsing, Presentation Browsing, Avatar Auto Animation) and Multimedia DB Technology to Store and Process VR / MPEG-4 / Presentation Media}, System Technology (Agent Engine) Agent that defines the synchronization and communication interface between the browser and the VR browser Technology, session management technology that handles mapping between selected channels and multicast group addresses through the channel search browser, and withdrawal from group subscription, VR / MPEG / Presentation media synchronization technology), and network including Internet multicast routing protocol for group communication. It is to provide a virtual reality based internet remote lecture system using technology.

Another object of the present invention is to provide a method for constructing a virtual reality-based Internet remote lecture system that is particularly suitable for building the Internet remote lecture system.

1 is a block diagram of a conventional Internet distance learning system.

2 is a diagram illustrating a system configuration of a virtual remote lecture system based on virtual reality according to the present invention.

3 is a bottom protocol stack structure of a VR-based Internet lecture system for implementing FIG.

4A-4B illustrate a client user interface for learners in accordance with the present invention.

5 is a configuration diagram of a multimedia codec showing data transmission and reception of MFEG-4 video / audio.

6 is a flowchart illustrating a virtual reality-based Internet remote lecture method according to an embodiment of the present invention.

<Description of the code | symbol about the principal part of drawing>

11: Remote Lecture Server

22: lecturer client 33: lecturer client

LOD: Lecture On Demand

SDP: Session Description Protocol SAP: Session Announcement Protocol

SDQ: Session Discovery Query SIP: Session Initiation Protocol

ATP: Agent Transport Protocol

In order to achieve the above object of the present invention, the present invention provides a remote education system on an Internet multicast network: a remote lecture server for storing and processing VR / MPEG / Presentation which is a large amount of multimedia information for internet lecture (LOD Server). ); A lecturer client for a lecturer to register and store lecture texts such as video / audio data, word documents and HTML web documents on the remote lecture server through a web camera / speaker for cyber lectures; And channel search browser capabilities to find and select the desired multicast channels, VRML media browsing, MPEG-4 media browsing, presentation browsing, animated avatars and audio with voice and sound Virtual reality, characterized in that it includes a client for attending a course on the Internet remotely by receiving the lecture data, such as VR / MPEG / Presentation from the remote lecture server through the Internet by providing an animation function, synchronization between multimedia (Synchronization) Provide a remote Internet-based lecture system.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 shows an Internet lecture system based on a virtual reality according to the present invention, which is composed of a remote lecture server 11, a lecturer client 22, and a lecturer client 33.

The remote lecture server (LOD Server) 11 has a conference manager and a session manager to enable one-way 1: N group communication, and is a media server and lecture that serves as an audio / video streaming server. It consists of management DB and chat server module to store and provide VR / MPEG / Presentation, which is a large amount of multimedia information for internet lecture.

The lecturer client 22 provides lecturers with lecture texts such as video / audio data and word documents such as MS-Word and HTML web documents through the web camera / speaker for cyber lectures to the remote lecture server 11. You can register, save, and check the course.

The participant client 33 has a channel search browser function for searching and selecting a multicast channel desired by a user, VRML media browsing, MPEG-4 media browsing, presentation browsing, animation supporting voice and sound, and the like. Avatar, auto animation function, and synchronization between multimedia are provided to receive lecture data such as VR / MPEG / Presentation from the remote lecture server 11 through the Internet and take a remote lecture on the Internet.

The lecturer edits lecture texts edited with MS-Power Point, MS-Word, and HWP based on Web3D virtual reality through a high-speed internet using a web camera and the client client 22, such as Mpeg and avi. After attaching class materials such as video, Flash image, URL link, etc., to internet remote lecture server (Lecture On Demand (LOD) server), a large number of students and lecturers who have membership can use Through the client 33, a lecturer's moving picture captured by the PC-Camera can be viewed in the cyber space without a separate processing device such as a web camera.

FIG. 3 illustrates a bottom protocol stack structure of a VR-based Internet lecture system for implementing FIG. 2.

Referring to FIG. 4, the multimedia browser layer displayed on the client user interface screen according to the present invention includes a channel browser ①, an MPEG-4 media browser ②, and lecture materials displayed as slides. It consists of Presentation window (③) and Web3D VR Media Browser (④).

The MPEG-4 media browser provides audio / video and virtual reality media based on Real Time Protocol / Real Time Control Protocol (RTP / RTCP) to enable 1: N multicasting on a UDP layer in real time over an IPv4 or IPv6 multicast network. AV / VR media) is multicasted and transmitted to display media data.

Session Announcement Protocol (SAP) and SIP on TCP / UDP for mapping processing, group join, and withdrawal processing technology selected from the session manager of the remote lecture server 11 and channel selected through the channel search browser and multicast group address (Session Initiation Protocol), SDP (Session Description Protocol) that transmits the information on the media stream of the multimedia session on top of the Simple Mail Transport Protocol (SMTP) base for sending mail between lecturers and participants, and session management information for the database and query. The Session Discovery Query (SDS) protocol is implemented to build a Session Initation and Discovery Application, which is linked with the Multimedia Browser Layer with the integrated function of MPEG-4 / VRML / Presentation / Search.

VR-based Internet remote lecture system according to the present invention is applied technology (avatar technology, channel search technology, media integrated browsing technology, multimedia DB technology), system technology (agent technology, session management technology, media synchronization, media server), network Technology (multicast routing technology, IPv6 technology, QoS provisioning technology, high-speed IP transmission technology).

Application technology

Avatar technology: Avatar technology that enables animation of voices and sounds that can interact with users in a virtual environment.

Channel search technology: Based on RSVP (ReSource reserVation Protocol), the user can search and select the desired multicast channel using channel search browser through Channel Resource Control, and the SDP (Session Description Protocol) protocol is web-based as a multicast application technology. Is converted to.

• Integrated Media Browsing Technology: Includes VR / MPEG / Presentation / Avatar's auto-animated browser capabilities for integrated media browsing within a web browser.

-VRML media browsing

-MPEG-4 media browsing

-Browsing presentation materials

-Avatar auto animation feature

-Synchronization between multimedia

Multimedia DB technology: DB construction technology that can store and process a large amount of multimedia (VR / MPEG / Presentation) .It builds a high-performance database using MS SQL Enterprise Server and the correlation between VR / MPEG-4 / Presentation media. Is defined.

System technology

Agent technology: Gives avatars intelligence and interactivity.

-Define communication interface between Agent Engine and VR Browser

-Synchronization between Agent and Avatar should be done.

Agent Transport Protocol (ATP) is defined for the transfer of agents from the remote lecture server 11 on top of the Transport Control Protocol (TCP) layer.

Session management technology: Mapping process between the selected channel and multicast group address through the channel search browser, group join, and withdrawal processing technology

-Implementation of Session Discription Protocol (SDP) and Session Discovry Query (SDS) protocols

-Define interface with browser

Media synchronization: Synchronization process between VR / MPEG / Presentation media (e.g. SMIL, RTP / RTCP)

Synchronized Multimedia Integration Language (SMIL) must be applied to MS Internet Explorer 5.5.

MPEG-4 requires RTP encoding

Synchronize other media to MPEG-4 information.

· Media Server: It is a server that can process a large amount of streaming video / audio data without interruption and utilizes NT Server-based MS Media Server.

■ Network technology

Multicast routing technology: The only network solution that can minimize the load on the network and servers during multi-party communication such as Distance Vector Multicast Routing Protocol (DVMRP)

IPv6 technology:

-Supports IPv6 multicast with the next generation IP protocol standard that can extend the depleted IPv4 address space to 2 ¹²⁸ , making it the only Internet remote lecture solution that can support the creation of numerous multicast channels.

In the IPv6 standard of IETF (Internet Engineering Task Force), a specific IP address space is allocated as a multicast-only space, and in the present invention, a transport channel is configured using the IPv6 multicast address space.

QoS offering technology:

-Ensure stable media stream transmission to IntServ (Integrated Service) and DiffServ (Differentiated Service) recommended by IETF to provide quality of service (QoS).

High speed IP transmission technology:

-With the introduction of MPLS, Gigabit routers and high-speed switches, it is possible to use the packet transmission service with the minimum transmission delay, so it can be used as the next generation Internet application that occupies a high bandwidth of 10Mbps or more.

5 is a configuration diagram of a multimedia codec showing data transmission / reception of MPEG-4 video / audio.

The transmitter of the multimedia codec transmits the video of the instructor received from the web camera to the multiplexer through the video input unit (VideoInputControl) and the video encoder (VideoEncoder). The audio encoder is transmitted to the multiplexer through an audio encoder, and the multiplexer transmits the AV media data to the receiver of the multimedia codec in real time on the RTP / RTCP layer.

The receiver of the multimedia codec receives AV media data in real time from a demultiplexer and outputs a video through a video decoder and a video decoder, and an audio decoder and an audio output unit. Audio is output through AudioRenderer. The arrow indicates a direction of data transmission as a buffer for storing temporary data.

6 is a flowchart illustrating a virtual reality-based Internet remote lecture method according to an embodiment of the present invention.

In an Internet distance learning system having a remote lecture server 11, a lecturer client 22, and a lecturer client 33, the instructor downloads the lecturer client 22 by file and downloads a word, web document, and web. A teacher lecture material, which is MPEG-4 video / audio data, is produced through a camera and a speaker and stored in the remote lecture server 11 (step S1).

If a person who wants a cyber lecture registers for the internet course (step S2), pays a predetermined fee, and confirms the payment of the fee through internet banking (step S3), and downloads the client client 33 from the remote lecture website. If the user checks whether the user is an authorized user (step S4), if the user is not authenticated, the process is retried.

Search for the multicast lecture channel through the channel browser and enter the classroom (step S5). An avatar capable of voice and sound support over the Agent Transport Protocol (ATP) protocol and an intelligent agent based on Java. In step S6, the education medium is synchronized with the 3D virtual reality (VR).

Video / voice for MPEG-4 lecture based on RTP (Realtime Transport Protocol) based on User Datagram Protocol (UDP) layer using IPv6 / IP Multicast / RSVP / Diffserv or IPv4 / IP Multicast network technology from the remote lecture server 11 The streaming data is displayed on the MPEG-4 media screen of the student client in real time (step S7).

PowerPoint (PPT) and word material lectures are displayed in a slide format, and video and audio of lecturers are played back (step S8). Multi-chat is performed (step S9).

If the homework is presented through the notice and the professor and the lecturer submit the report by email until the designated time (S10), the lecturer ends the lecture by notifying the bulletin board on the bulletin board by the lecturer who received the report that is the result of the assignment (step S11). ).

Therefore, using IPv6 multicasting as well as IPv4 multicasting technology, it provides MPEG-4 based MPEG-4 based high compression rate streaming video / audio data transmission over TCP / RTP layer and avatar in cyber classroom. It can increase the effectiveness of education by providing realistic Web3D navigation through.

As described above, the virtual reality-based Internet remote lecture system and method according to the present invention is not only the existing IPv4 multicasting but also the next-generation Internet utilizing IPv6 multicasting that requires a high bandwidth of 10Mbps or more. Improved quality of service (QoS) by 1: N multicasting MPEG-4 based high compression rate streaming video / audio data, and realistic 3D virtual reality Web3D navigation through avatar in cyber classroom or classroom To increase the effectiveness of education.

In addition, we presented the Internet multicast routing protocol for group communication in virtual environment, animated avatar technology supporting voice and sound, VRML media browsing, MPEG-4 media browsing, presentation data browsing, and Avatar's Auto. VR / MPEG-4 / Presentation media browser integration function such as animation, agent engine and agent to define the interface and communication interface between VR browser, mapping process between selected channel and multicast group address through channel search browser It can be used for high-quality multimedia distance learning system that requires high bandwidth in next generation Internet such as high speed communication network by enabling session management function and VR / MPEG / Persentation media synchronization.

Therefore, VR based Internet virtual education, kindergarten, supplementary and home schooling of Internet School of elementary, middle and high school, reinforcement of university and virtual university lecture, various seminars, corporate training institute and general public It can be used for internet remote courses.

Claims (4)

In distance education systems on Internet multicast networks: A remote lecture server (LOD Server) 11 for storing and processing VR / MPEG / Presentation which is a large amount of multimedia information for Internet lecture; A lecturer client 22 for a lecturer to register and store lecture texts such as video / audio data and word documents and HTML web documents in the remote lecture server 11 through a web camera / speaker for cyber lectures; And Channel search browser function to find and select the desired multicast channels, VRML media browsing, MPEG-4 media browsing, presentation browsing, animated avatars and voice-supporting audio and sound It provides a function, multimedia synchronization (Synchronization) to receive lecture data, such as VR / MPEG / Presentation from the remote lecture server 11 over the Internet to include a student client 33 for attending the Internet remote lecture Virtual Reality-based Internet remote lecture system. In an Internet distance learning system having a remote lecture server 11, a lecturer client 22, and a learner client 33: (a) downloading the file of the lecturer client and creating a teacher lecture material, which is MPEG-4 video / audio data, through a word and a web document, a web camera, and a speaker; (b) A person who wants to take a cyber course registers for the Internet course (S2), pays a predetermined fee, and confirms the payment of the course fee through Internet banking (S3) by downloading and executing the client for the student from a remote course web site. Checking whether the user is an authenticated user (S4), if the user is not authenticated, retrying the process, and entering the cyber classroom if the person who registers and pays the course fee; (c) Search for the multicast lecture channel through the channel browser and enter the classroom (S5). Avatar and Java based animations with voice and sound support on top of the Agent Transport Protocol (ATP) protocol. Synchronizing with the intelligent agent to browse the educational medium in a 3D virtual reality (S6); (d) displaying streaming data on the MPEG-4 media screen of the client client in real time on the user datagram protocol (UDP) layer based on the Realtime Transport Protocol (RTP) based MPEG-4 lecture from the remote lecture server (S7) ; (e) PowerPoint (PPT) and word material lectures are displayed in a slide format, and video and audio of lecturers are played back (S8), and interactive multi-chat is conducted between geographically separated cyber lecturers ( S9) step; And (f) Present homework through announcements and submit a report by email by professors and lecturers until the designated time (S10). Virtual reality-based Internet remote lecture method comprising the step (S11). The method of claim 2, The VR-based Internet remote lecture system Geographically separated lecturers and multiple attendees stream video lecture data 1: N multicasting MPEG-4-based high compression video / audio data over a multicast network, and avatars in cyber classrooms Application technologies to provide realistic navigation {animated avatar technology to support voice and sound in virtual environment, media integrated browsing technology (VRML media browsing, MPEG-4 media browsing, presentation browsing, and avatar) Multi-media DB technology to store and process VR / MPEG-4 / Presentation media), system technology (agent technology to define synchronization and communication interface between Agent engine and VR browser) Mapping processing between cast group addresses and group Session management technology to handle the withdrawal, VR / MPEG / Presentation media synchronization technology), network technology including Internet multicast routing protocol for group communication (multicast routing technology, IPv6 technology, QoS provisioning technology, high speed IP transmission technology) Virtual reality-based Internet remote lecture method characterized in that to provide. The method of claim 2, (A), (b), (c), and (c) for a virtual reality (VR) based internet remote lecture system having a remote lecture server, a lecturer client, and a lecturer client on a computer. d) a computer-readable recording medium having recorded thereon a function, said (e) function and a program for realizing said (f) function.