JP3212833U - Interactive education support system - Google Patents

Abstract

An interactive education support system that generates a 3D scene that can be operated via an operation interface and synchronizes the 3D scene displayed on each operation interface.
An interactive education support system includes a server for logging in a plurality of users through a network connection. The server forms an operation interface that displays the same 3D scene that can be operated according to each user, and each operation interface corresponds to a different user, and an operation interface for educators and at least one operation for learners Form an interface. The operation interface for educators or the operation interface for learners can receive an operation command for a 3D scene, and after corresponding video processing is performed on the 3D scene according to the operation command, the 3D scene originally displayed on each operation interface is synchronized. Update. [Selection] Figure 2

Description

  The present invention relates to an interactive educational support system, and more particularly to an interactive educational support system that combines 3D synchronized video.

At present, when a learner wants to learn an arbitrary language, there is a method of learning via a network in addition to a method of learning by participating in a language course of a school or a cram school other than a method of reading a book. Learners can learn languages by using only a terminal that can be connected to the network, regardless of time or place. For busy modern people, the convenience and practicality of learning are greatly improved. . However, the learner has a method of watching a video recorded in advance by a language education educator or a method of learning while conversing with an educator through software having a video conversation function. There is a problem that the learning effect by is limited and the interactivity between the learner and the educator is not good.

In addition, 3D video processing technology has become a hot topic in recent years. For example, virtual reality (VR) or 3D scene playback has been widely applied in exhibition halls for electronic games or dynamic exhibitions. If a combination of 3D video processing technology and language learning can be realized, it is thought that the motivation of learners will be enhanced by dynamic 3D video. However, it is a research-worthy issue to realize a technique for synchronously reproducing 3D video when a plurality of users perform bi-directional or multi-directional learning using an operation interface.

An object of the present invention is to provide an interactive education support system, and achieves an effect of generating a 3D scene operable through an operation interface and synchronizing the 3D scene displayed on each operation interface.

In order to achieve the above object, the interactive education support system according to the present invention enables a plurality of users to connect to a server through a network. The server includes a 3D video module, an interface generation module, and a synchronization module. The 3D video module generates a 3D scene that can be operated, and the interface generation module is electrically connected to the 3D video module to form an operation interface that can display the same 3D scene to each user. The operation interface generates an educator operation interface and at least one learner operation interface according to different users, and either the educator operation interface or the learner operation interface outputs a 3D scene operation instruction command. The 3D video module is notified to perform the corresponding video processing in response to the operation instruction command, and the synchronization module is electrically connected to the 3D video module. Synchronize the originally displayed 3D scene.

In one embodiment according to the present invention, the control interface and transfer right of the 3D scene are pre-assigned in the operation interface for educators, and the educator who has the transfer right transfers and collects the control right to the operation interface for learners. As a result, the educator operation interface or the learner operation interface having the control right can receive the operation instruction command and perform the corresponding video processing.

In one embodiment according to the present invention, one of the educator operation interface and any one of the learner operation interfaces has control right at the same time.

In one embodiment according to the present invention, the operation interface generates an operation interface for an educator or an operation interface for a learner according to personal settings of different users.

In one embodiment according to the present invention, the interactive educational support system further includes a plurality of user terminals, and each user terminal is connected to a server by a single user via a network, and an individual setting value is set. Is stored in the user terminal and the corresponding operation interface is displayed for operation from the user terminal.

In one embodiment according to the present invention, each user terminal includes a corresponding synchronization module, and the synchronization module of the corresponding server and the synchronization module of the server maintain bidirectional transmission according to a communication protocol to synchronize and update data.

In one embodiment according to the present invention, the corresponding synchronization module is an application or a browser having a material synchronization function.

In one embodiment according to the present invention, the synchronization module includes an interface data synchronization module, and is used to synchronize / update video streaming data, graphics / document data, or user information displayed on each operation interface.

In one embodiment according to the present invention, the corresponding video processing is 3D scene transition, 3D scene movement, 3D scene viewing angle change, object state change in 3D scene, movement, rotation, or 3D electronic game interface. Includes generation.

In one embodiment according to the present invention, the corresponding video processing includes audio data, and synchronizes / updates the 3D scene displayed on each operation interface and outputs the audio data according to the video processing.

As a result, the interactive education support system according to the present invention provides an operation interface, and a plurality of users conduct interactive language education and combine 3D video applications through the operation interface, and simultaneously 3D video data and non-3D video. Synchronize and update data.

It is a figure which shows the system of the interactive education support system which concerns on this invention. It is a figure which shows the structure of the server and user terminal of the interactive education support system which concern on this invention. It is a figure which shows the connection state of the interactive education support system which concerns on this invention. It is a figure which shows the operation interface of the interactive education support system which concerns on this invention. It is a figure which shows the 3D scene displayed on the operation interface of the interactive education support system which concerns on this invention. It is a figure which shows 1st Embodiment of control-right transfer of the 3D scene of the interactive education support system which concerns on this invention. It is a figure which shows 2nd Embodiment of control-right transfer of the 3D scene of the interactive education support system which concerns on this invention. It is a figure which shows 3rd Embodiment of control-right transfer of the 3D scene of the interactive education support system which concerns on this invention.

  Each embodiment shows the example which concerns on this invention, and is not limited to this. It will be understood that other embodiments than the embodiments of the present invention can be reached without departing from the scope of the present invention after reading the specification of the present invention. The features and advantages of the embodiment can be recognized based on the following detailed description and the scope of the utility model registration request.

In this specification, the terms “including”, “having” or other similar terms are meant to be inclusive and not exclusive. For example, “a unit or structure containing a plurality of requirements” includes not only the components listed in this specification but also a normal component (unit or structure) other than the listed components. To do.

(See FIG. 1) FIG. 1 is a diagram showing a system of an interactive education support system 1 according to the present invention. As shown in FIG. 1, an interactive education support system 1 according to the present invention includes a server 10 for a plurality of users to log in through a network connection. Each user uses a user terminal 20 having a network connection function, logs in by connecting to the server 10 via the Internet 30, and then uses the interactive education support system 1 according to the present invention to conduct interactive education. I do. The server 10 may be composed of a single computer terminal that executes the integration function or a plurality of computer terminals that separately execute the aforementioned integration function at least partially. The user terminal 20 may be a desktop computer, a tablet computer, a smartphone, or a terminal similar to other computers that can be connected to a network.

The server 10 includes a processor 11 and a storage medium 12, and the storage medium 12 is electrically connected to the processor 11. The storage medium 12 includes an operating system (not shown) and a software program 12a. The operating system is loaded by the processor 11, and the software program 12a is executed in the operating system environment. The processor 11 here may be a central processing unit of a computer, a microprocessor or other processor having a similar processing function, and the storage medium 12 is a general storage medium, a hard disk or a storage unit having a similar storage function. There may be. In this embodiment, since the software program 12a is configured using the flash language, each user executes a browser-related application preinstalled in the user terminal 20, and the present invention is made via the Internet 30. By logging in to the server terminal 10 of the interactive education support system 1, a related operation interface is displayed on the user terminal 20 and can be operated by each user. This eliminates the need for the user to additionally install other software on the user terminal 20, allows connection / login to the server 10, and increases convenience for the user. In addition, this invention is not limited to embodiment mentioned above. For example, when the user terminal 20 is a smartphone, the user terminal 20 is installed by preinstalling a mobile application (application, APP) that is connected to the server 10 and forms a related operation interface after being executed. After the mobile application is executed, the next interactive operation can be performed through the operation interface.

FIG. 2 is a diagram showing the internal configuration of the server 10 and the user terminal 20 of the interactive education support system according to the present invention, and FIG. 3 is an interactive education support system according to the present invention. It is a figure which shows the connection state of. As shown in FIG. 2 and FIG. 3, user information corresponding to each user terminal 20 is stored, and when an arbitrary user a logs in to the server 10 through the user terminal 20, the software program 12a is The user information is acquired through 20, the user a is assigned to the education group A determined by the course division data, and the course division data is preloaded in the storage medium 12. The number, level, or status of the logged-in user a, each user a is distributed, and at least one education group A is formed. Each education group A consists of a plurality of users a, and each education group A includes an educator a1 and at least one learner a2. The related technology for distributing the user a based on the user information described above is not mentioned here because the applicant of the present invention is disclosed in Taiwan Patent Publication No. I249112.

The software program 12a includes a 3D video module 121, an interface generation module 122, and a synchronization module 123 that are electrically connected to each other. It is used to form a 3D scene that can be operated by the 3D video module 121. The 3D scene described here is a normal 3D stereoscopic image, and the 3D scene is displayed on the operation interface formed through the browser of the user terminal 20 by applying WebGL technology to each user. 3D images are 3D interactive images formed with virtual reality (VR, virtual reality), augmented reality (AR) or mixed reality (MR) technology. It is viewed on a peripheral device (for example, a head-mounted display or other auxiliary display device) connected to the person terminal 20 and causes an interactive reaction.

As shown in FIGS. 2 and 3, the interface generation module 122 generates the operation interface 200 by each education group A, and each education group A performs interactive education. The above-described user information of each user a includes personal setting values for the user a, and after generating the operation interface 200, the operation interface 200 reads the individual setting values of each user a. The personalized operation interface 200a is generated and displayed on the user terminal 20 according to the difference of the read individual setting values, and each user a is identical through the personalized operation interface 200a. Education group A can perform exchange learning with other users. Further, the interface generation module 122 divides the generated personalized operation interface 200a into an educator operation interface or a learner operation interface depending on the difference of the read individual setting values, for example, the status of the corresponding different users. Permission to use different interfaces is granted.

The interface generation module 122 can be combined with the 3D video module 121 and displays the same 3D scene on the personalized operation interface 200a generated according to each user of the same education group A. The user a can input an operation command for the 3D scene through the user terminal 20, and the educator operation interface or the learner operation interface receives the operation command and communicates with the 3D video module 121 in response to the operation command. Perform the appropriate video processing. The operation command described here indicates that the user inputs through various input devices (for example, touch panel, mouse, keyboard, remote control device, voice input device, etc.) of the user terminal 20, The invention is not limited to this. For example, a user's movement or gesture is detected through a sensing device (for example, an optical sensor, a dynamic sensor, an accelerometer, etc.) electrically connected to the user terminal 20. Input of an operation command. The type of operation command may be directly click, drag, circle selection, function key operation, voice control or perceivable gesture or movement on the 3D scene.

The synchronization module 123 synchronously updates the 3D scene originally displayed on each operation interface with the corresponding 3D scene after video processing, and can also apply WebGL technology. The 3D video module 121 described above performs video processing of the 3D scene in accordance with an input operation command, and then the 3D scene is changed by different video processing. At this time, the synchronization module 123 changes the changed 3D scene. Is updated synchronously in each operation interface, and the change of the 3D scene is displayed to all users of the same educational group. The video processing described here includes 3D scene changes, 3D scene movements, 3D scene viewing angle changes, object state changes in 3D scenes, object movements, object rotations, or 3D electronic game interface generation. Etc.

The aforementioned 3D scene change, 3D scene movement, or 3D scene viewing angle change refers to the overall change of the 3D scene. The change of the 3D scene is switching between different 3D scenes according to an operation command, for example, switching from indoor to outdoor. The movement of the 3D scene is a movement in a different direction in the 3D scene according to the operation command. For example, the movement to the front, rear, left, and right in the current 3D scene can be given. Changing the viewing angle of the 3D scene includes adjusting or changing the viewing angle in the 3D scene, for example, adjusting the viewing angle in different directions, such as up, down, left, and right.

The object state change, object movement, or object rotation in the 3D scene described above is a single or partial object change in the 3D scene. The change in the state of the object in the 3D scene is a change in the state of the existing object in the 3D scene according to the operation command, and examples thereof include light emission, explosion, deformation, and object conversion of the object. The movement of the object is a movement of the position of the object in the 3D scene according to the operation command. For example, the object is moved from the left to the right of the 3D scene. The rotation of an object is to rotate an existing object in a 3D scene in accordance with an operation command. For example, the object in the 3D scene is rotated from the back to the front. Also, the 3D video module 121 generates a 3D electronic game interface according to the input operation command and is displayed in synchronization with each operation interface, and the user can enjoy learning while experiencing the contents of the electronic game through the 3D electronic game interface. Can taste.

The video processing may be combined with audio data, and the 3D scene displayed on each operation interface is synchronously updated according to the video processing, and the audio data can be output, resulting in a visual and auditory double effect. For example, when the video processing is a movement of a 3D scene, a footstep is output in accordance with the moving scene, and the user enhances the sense of presence of the movement in the 3D scene, or the video processing is an object state in the 3D scene. In the case of a change (for example, light emission of an object), a bee sound or other sound can be output in accordance with the change in the state of the object, giving a different feeling when the user operates the interface.

In addition to the function of synchronously updating the 3D scene displayed on each operation interface, the synchronization module 123 described above can also perform synchronous update of other data on the operation interface. In one embodiment according to the present invention, the synchronization module 123 includes an interface data synchronization module 123a, which applies flash technology to synchronously update data other than the 3D scene displayed on each operation interface, for example, Video streaming data (for example, video image data), graphic / document data (for example, educational materials) or user information (for example, current user status) at each operation interface may be used.

As shown in FIG. 2, in order to match the synchronization module 123 of the server 10, each user terminal 20 includes the synchronization module 21, and the synchronization module 21 and the synchronization module 123 of the server 10 are bidirectional depending on the communication protocol. The data is transferred and the materials are updated synchronously. Among them, the synchronization module 21 includes an application installed on a smartphone or a browser having a data synchronization function. For example, in one embodiment according to the present invention, the synchronization module 21 includes a video streaming unit 211 and a 3D video data processing unit 212. The video streaming unit 211 performs video streaming processing with the server 10, For example, adopt the Flash Media Server architecture. Perform 3D video data synchronization processing with the server 10 by the 3D video data processing unit 212, for example, adopt WebGL technology that supports 3D scenes, and adopt the WebSocket that supports WebGL as the communication protocol described above Can do. As a result, the user terminal 20 can update the video streaming data and the 3D scene synchronously through the synchronization module 21, but the architecture of the corresponding synchronization module 21 is not particularly limited.

(See FIGS. 4 and 5) FIG. 4 is a diagram showing a personalized operation interface 200a of the interactive education support system 1 according to the present invention. FIG. 5 is a diagram showing a 3D scene displayed by the interactive education support system according to the present invention via the personalized operation interface 200a. As shown in FIGS. 2 to 4, after the user a logs in to the server 10 via the user terminal 20, the personal setting value of the user a is read and customized based on the personal setting value. A personalized operation interface 200a is formed, and interactive education is performed in an interface mode familiar to the user a. The command input authority in the personalized operation interface 200a of the user a varies depending on the personal setting value read when logging into the server 10. For example, when user a is an educator a1, in order to perform efficient education, the system sets the authority setting for each item of command input authority to ON, and the user a is a learner a2. The partial command input authority is set to the OFF state in advance. The above-mentioned command input authority setting adjusts the default value according to the difference in the system, but the present invention is not limited to this.

As shown in FIGS. 2 and 4, each personalized operation interface 200a generated by the interface generation module 122 includes a status display area 210, an instant messaging area 220, a video display area 230, an interface function setting area 240, and an interactive function. Display area 250. In the status display area 210, the connection status of each user and the status information of each user such as the identity recognition status or the input authority status are displayed. The status information displayed in the status display area 210 includes the items described above. It is not limited to. The instant messaging area 220 is used to display text data input by each user in the same educational group, and performs both or many text conversations. The video data acquired from the educator is displayed in the video display area 230. For example, a real-time video of the educator acquired using a network camera can be mentioned. The video display area 230 includes a video / audio toolbar 231 having functions such as opening / closing and adjusting video / audio. The interface function setting area 240 provides instant selection and display of partial interface functions, and includes a time message display 241, an interface function option 242, an auxiliary communication function option 243, and the like.

The interactive display area 250 provides an electronic whiteboard function, and can display educational display information (eg, document file, web page material, audiovisual material, etc.) via the personalized operation interface 200a and is displayed in the interactive display area 250. The Education exhibition information is usually provided by an educator with interactive input authority, and selection and operation of education exhibition information (for example, text input, annotation or line drawing on education exhibition information, etc.) is performed through the toolbar 251 in the interactive display area 250. A learner who is used for auxiliary teaching materials or explanations when teaching and whose interactive input authority is set to ON can use the related function of the interactive display area 250. Education exhibition information corresponding to each education group can be loaded in advance into the interactive display area 250 of the personalized operation interface 200a through system settings, and can be prepared by the user who logs in first. It should be noted that the related interface display material in the personalized operation interface 200a described above can be synchronized and updated via the interface data synchronization module 123a of the synchronization module 123.

In the interactive education support system 1 according to the present invention, the user performs a 3D scene opening / closing function through the interactive display area 250 of the personalized operation interface 200a. As shown in FIG. 4, in one embodiment according to the present invention, the toolbar 251 provided in the interactive display area 250 of the personalized operation interface 200a includes a 3D scene switching key 251a, which allows the user to perform a mouse operation or a touch panel. When the switch button 251a is pressed, the 3D scene (as shown in FIG. 5) is displayed in the interactive display area 250 or the 3D scene is closed. Similarly, the user can perform pre-learning by displaying the 3D scene via the switching button 251a of the interactive display area 250 before the interactive education starts. The user inputs an operation command for the 3D scene through the personalized operation interface 200a (for example, clicking on the 3D scene article or toolbar displayed in the interactive display area 250 or dragging with the mouse, or pressing the corresponding function key on the keyboard). Then, the 3D video module 121 performs the above-described arbitrary corresponding video processing according to the operation command to change the 3D scene. At this time, the 3D scene changed via the synchronization module 123 is displayed on each operation interface. Update synchronously. By combining 3D scene changes and language learning related functions, users can use the interactive educational support system 1 according to the present invention to increase their interest in language learning and produce better learning outcomes.

In the interactive educational support system 1 according to the present invention, when an operation command for a 3D scene is input through an arbitrary operation interface and the 3D scene is controlled, the operation interface has a control right for the 3D scene. The operation interface has an authority to receive an operation command for the 3D scene. In one embodiment according to the present invention, the control right and transfer right in the 3D scene are given in advance to the educator operation interface, and the transfer right here is a control right for any learner. It refers to returning to the operation interface for educators from one of the operation interfaces for learners. The educator operation interface or the learner operation interface having the control right can receive an operation command for the 3D scene and perform corresponding video processing.

(See FIGS. 6A to 6C) FIGS. 6A to 6C are diagrams showing some embodiments of transfer of control rights of the 3D scene of the interactive educational support system 1 according to the present invention. In the interactive education support system 1 according to the present invention, the control right and transfer right of the 3D scene are given in advance to the operation interface of the user terminal 20 whose identity is an educator, and the educator enters the 3D scene in a normal situation. By inputting the operation command, the 3D scene can be controlled through the educator operation interface. As shown in FIG. 6A, in the first embodiment, the educator sends a control right transfer command to the server 10 via the operation interface for educators at the user terminal 20 as necessary. The control right is transferred to the user terminal 20 corresponding to one of the learners selected by the educator according to the transfer instruction, and the learner's operation interface for the learner has the right to control the 3D scene. A 3D scene operation command is input through the learner's operation interface to control the 3D scene.

As shown in FIG. 6B, in the second embodiment, when the educator determines that the learner has completed the control for the 3D scene, the educator returns to the educator operation interface of the user terminal 20 again. The control right is transmitted to the server 10, and the server 10 transmits the control right to the educator's user terminal 20 according to the transfer instruction, and the educator's operation interface controls the 3D scene. Can be recovered. Alternatively, as shown in FIG. 6C, in the third embodiment, the server 10 transfers the control right to the user terminal 20 corresponding to another learner selected by the educator according to the transfer command. The learner's operation interface of the learner possesses the right to control the 3D scene, and controls the 3D scene by inputting operation instructions for the 3D scene via the learner's operation interface.

The above-mentioned control right is possessed by one of the educator operation interface and any of the learner operation interfaces at the same time. This is a setting for avoiding the possibility of an error occurring in 3D video synchronization due to the 3D video module performing a plurality of corresponding video processes simultaneously when control rights exist in a plurality of operation interfaces.

As described above, the interactive education support system 1 according to the present invention can form the operation interface according to different users, and the operation interface and the 3D scene are combined and displayed. Through the synchronization module 123 of the server 10 and the synchronization module 21 of each user terminal 20, the 3D scene displayed on each operation interface of each user terminal 20 can be updated synchronously. On the other hand, the interface data synchronization module 123a of the synchronization module 123 of the server 10 is combined with the synchronization module 21 of each user terminal 20 to interface data other than the 3D scene in each operation interface of each user terminal 20 (for example, Sync video streaming materials). As a result, the interactive educational support system 1 according to the present invention can simultaneously update 3D video materials and video materials according to the operation interface, improve the synchronization effect of the materials on the operation interface, and encourage users to learn. Is expected to increase.

The above-described embodiments are for explaining the present invention, and are not limited to the embodiments mentioned in the present invention or applications such as the embodiments. In addition, although at least one example has been described in the above-described embodiment, it should be understood that there are a plurality of patterns in the present invention. It should also be understood that the embodiments described herein do not limit the scope of the claims. Instead, the embodiments according to the present invention are for providing teaching and suggestion to engineers in the technical field. Furthermore, the change of the function / array of the configuration is a change that does not depart from the gist of the claims of the present invention, and the mode realized by arbitrarily combining the components and functions in each embodiment is also possible. It is within the scope of the present invention.

1 Interactive education support system
10 servers
11 processor
12 Storage media
12a software program
121 3D video module
122 Interface generation module
123 synchronization module
123a interface data synchronization module
20 User terminal
21 Compatible synchronization modules
211 video streaming unit
212 3D video data processing unit
30 Internet
200 Operation interface
200a Personalized operation interface
210 Status display area
220 Instant Messaging Area
230 Video display area
231 Audio / Video toolbar
240 Interface function setting area
241 hour message display
242 Interface function options
243 Auxiliary communication function option
250 Interactive display area
251 Toolbar
251a button
A Education group
a User
a1 educator
a2 Learner

Claims (10)

An interactive educational support system that displays 3D synchronized video in combination,
The interactive education support system includes:
Including a server where multiple users log in through a network connection,
The server includes a 3D video module, an interface generation module, and a synchronization module;
The 3D video module generates an operable 3D scene,
The interface generation module is electrically connected to the 3D video module, and forms an operation interface capable of displaying the same previous 3D scene according to each user, and the operation interface is provided according to a different user. A learner operation interface and at least one learner operation interface, wherein the educator operation interface or the learner operation interface can receive an operation command for the 3D scene and corresponds to the operation command. To notify the 3D video module to execute the video processing to be performed,
The synchronization module is electrically connected to the 3D video module and the interface generation module, and synchronously updates the originally displayed 3D scene in each operation interface with the 3D scene after the video processing. An interactive educational support system. A control right and a transfer right of the 3D scene are given to the educator operation interface in advance,
The transfer right is transferred by transferring the control right to the learner operation interface or to the educator operation interface.
The interactive education according to claim 1, wherein the educator operation interface or the learner operation interface having the control right receives the operation command and performs the corresponding video processing. Support system. 3. The interactive educational support according to claim 2, wherein the control right is possessed by any one of the educator operation interface and any one of the learner operation interfaces. system. The interactive operation support system according to claim 1, wherein the operation interface forms the educator operation interface or the learner operation interface according to different setting values of individual users. It further includes a plurality of user terminals,
Each of the user terminals is connected to the server by a single user via a network,
5. The interactive educational support system according to claim 4, wherein the personal setting value is stored in the user terminal and the corresponding operation interface for the user terminal to operate is displayed. . Each of the user terminals includes a corresponding synchronization module;
6. The interactive education support system according to claim 5, wherein the corresponding synchronization module and the synchronization module of the server perform synchronous update of data while maintaining bidirectional transmission according to a communication protocol. The interactive education support system according to claim 6, wherein the corresponding synchronization module is an application or a browser having a data synchronization function. The synchronization module includes an interface data synchronization module;
2. The interactive educational support system according to claim 1, wherein flat video streaming data, graphics / document data, or user information displayed on each operation interface is updated synchronously. The corresponding video processing includes 3D scene change, 3D scene movement, 3D scene viewing angle change, object state change in 3D scene, object movement, object rotation, or 3D electronic game interface generation. The interactive educational support system according to claim 1, wherein: In the corresponding video processing, by combining audio data, the 3D scene displayed on each of the operation interfaces is synchronously updated according to the corresponding video processing, and the audio data is output. The interactive education support system according to claim 1, wherein the interactive education support system is characterized.