TW202224422A - System and method for making audio visual teaching materials based on augmented reality - Google Patents

Abstract

A system for making audio-visual teaching materials based on augmented reality includes a server and first augmented reality glasses. The server generates a recording event definition file according to a first operation of a user. The first augmented reality glasses is communicatively coupled to the server and record a first video, wherein the first augmented reality glasses receives the recording event definition file from the server, and in response to a first event in the first video matching the recording event definition file, the first augmented reality glasses generates a first event label corresponding to the first event, and transmits the first video and the first event label to the server, wherein the server generates the audio-visual teaching material according to the first video and the first event label.

Description

System and method for producing augmented reality-based audio-visual teaching materials

The present disclosure relates to a system and method for producing an audio-visual teaching material based on augmented reality (AR).

In the process of recording the augmented reality audio-visual teaching material, the recorder wearing the augmented reality glasses needs to use voice commands to divide the audio-visual teaching material into different teaching steps. In addition to affecting the fluency of recording audio-visual teaching materials, it may also increase the recording time due to repeated confirmation. In addition, when playing the recorded audio-visual teaching materials, the augmented reality glasses cannot effectively confirm whether the learners wearing the augmented reality glasses have completed the content according to the audio-visual teaching materials.

The present disclosure provides a system and method for an audio-visual teaching material based on augmented reality, which can increase the fluency when recording the audio-visual teaching material, and can automatically determine whether the learner has completed the learning task in learning the audio-visual teaching material.

The system for producing an augmented reality-based audio-visual teaching material of the present disclosure includes a server and a first augmented reality glasses. The server generates a video event definition file according to the user's first operation. The first augmented reality glasses are communicatively connected to the server and record a first video, wherein the first augmented reality glasses receive a recording event definition file from the server, in response to the first event in the first video and the video recording event definition The files are matched to generate a first event tag corresponding to the first event, and the first video and the first event tag are sent to the server, wherein the server generates the audio-visual teaching material according to the first video and the first event tag.

The method for producing an augmented reality-based audio-visual teaching material of the present disclosure includes: generating a video event definition file by a server according to a user's first operation; recording a first video by the first augmented reality glasses; An augmented reality glasses receive a recording event definition file from the server; the first augmented reality glasses generate a first event corresponding to the first event in response to the first event in the first video matching the recording event definition file tag, and transmit the first video and the first event tag to the server; and the server generates the audio-visual teaching material according to the first video and the first event tag.

Based on the above, the system and method for an augmented reality-based audio-visual teaching material of the present disclosure can enable augmented reality glasses to generate event tags according to the video event definition file. Based on this, the server can automatically generate the audio-visual teaching material according to the event tag, thereby increasing the convenience of recording the audio-visual teaching material.

In order to make the above-mentioned features and advantages of the present disclosure more obvious and easy to understand, the following embodiments are given and described in detail in conjunction with the accompanying drawings as follows.

FIG. 1 is a schematic diagram of a system 100 for producing an augmented reality-based audio-visual teaching material according to an embodiment of the present disclosure. The system 100 may include a server 110 , a first augmented reality glasses 120 and a second augmented reality glasses 130 .

The server 110 may have a processing unit (eg, but not limited to, a processor), a communication unit (eg, various communication chips, mobile communication chips, Bluetooth chips, WiFi chips, etc., but not limited thereto), and a storage unit (eg: Removable random access memory, flash memory, hard disk, etc., but not limited thereto, are necessary components to run the server 110 . In addition, the server 110 may also have an input device (eg, a keyboard, a mouse or a touch screen). The server 110 may include devices with IP addresses or networking capabilities such as mobile stations, advanced mobile stations (AMS), clients, desktop computers, notebook computers, personal digital assistants, or workstations.

The first augmented reality glasses 120 (or the second augmented reality glasses 130 ) may have a processing unit (eg, but not limited to, a processor), a communication unit (eg, various communication chips, mobile communication chips, Bluetooth Chips, WiFi chips, etc., but not limited thereto) and storage units (such as removable random access memory, flash memory, hard disk, etc., but not limited to), etc., run the first augmented reality glasses 120 (or the first augmented reality glasses 120). 2. A necessary component of augmented reality glasses 130). The first augmented reality glasses 120 (or the second augmented reality glasses 130 ) can be communicatively connected to the server 110 . The first augmented reality glasses 120 and the second augmented reality glasses 130 may have the same function.

The server 110 can receive the user's operation through the input device to generate the video event definition file. The user can be the producer of audio-visual teaching materials. For example, if the user wants the learning task of the audio-visual teaching material to include confirming whether a specific object is recognized, the user can set a recognition event related to the specific object through an operation. The server 110 can generate a video event definition file including the identification event of the specific object according to the user's operation. If the user wants the content of the audio-visual teaching material to include confirming whether the value of the sensor is normal, the user can set the sensor threshold event by operation. The server 110 can generate a video event definition file including the sensor threshold event according to the user's operation. The present disclosure does not limit the content of the video event definition file.

In response to the server 110 generating the recording event definition file, the first augmented reality glasses 120 may receive the recording event definition file generated by the server 110 from the server 110 .

The first augmented reality glasses 120 can record the first video. In this embodiment, the first movie may be a movie to be used to make an audio-visual teaching material. The first augmented reality glasses 120 can be worn by the recorder who records the audio-visual teaching material.

In one embodiment, the first augmented reality glasses 120 can identify the object in the first video, and determine that the first event occurs in response to identifying the object matching the video event definition file. For example, assume that the video event definition file includes an identification event for a specific object. If the first augmented reality glasses 120 identify that the object in the first video matches the specific object in the recording event definition file, the first augmented reality glasses 120 can determine that the identification event occurs.

In one embodiment, the first augmented reality glasses 120 can recognize the action of the object in the first video, and determine that the first event occurs in response to the action matching the video event definition file. For example, it is assumed that the recording event definition file includes the event that the value of the pressure sensor returns to normal. If the first augmented reality glasses 120 recognize that the value of the pressure sensor in the first video is within a normal range, the first augmented reality glasses 120 can determine that the value returns to normal and an event occurs.

In one embodiment, the first augmented reality glasses 120 can identify the first event in the first video according to the first machine learning model. Machine learning models can be machine learning models such as Linear Regression, Decision Tree, or Convolutional Neural Network (CNN). The present disclosure does not limit the kind of the first machine learning model.

FIG. 2 is a schematic diagram illustrating an event tag of a video according to an embodiment of the present disclosure. The first augmented reality glasses 120 may generate an event tag corresponding to the first event in response to the first event in the first video being matched with the recording event definition file. For example, it is assumed that the recording event definition file includes an identification event for a specific object. The first augmented reality glasses 120 can determine whether there is a first event matching the content of the recording event definition file (ie, the identification event of a specific object) among one or more events of the recorded first video. If there is a first event matching the recording event definition file in the first video, the first augmented reality glasses 120 can generate a first event tag corresponding to the first event. For another example, it is assumed that the recording event definition file includes the event that the value of the pressure sensor returns to normal. The first augmented reality glasses 120 can determine whether there is a first event matching the content of the recording event definition file (ie, the event that the value of the pressure sensor returns to normal) in one or more events of the recorded first video. If there is a first event matching the recording event definition file in the first video, the first augmented reality glasses 120 can generate a second event tag corresponding to the first event. In one embodiment, the event tag may include a time stamp. For example, the first event tag may be "time 00:30: specific object recognized", and the second event tag may be "time 10:18: pressure returns to normal".

In response to generating the first event tag, the first augmented reality glasses 120 may transmit the first video and the first event tag to the server 110 .

FIG. 3 is a schematic diagram illustrating an audio-visual teaching material according to an embodiment of the present disclosure. The server 110 may generate an audio-visual teaching material according to the first video and the first event tag, wherein the audio-visual teaching material may include a first augmented reality image corresponding to the first event tag or a second augmented reality image corresponding to the second event tag Reality image. In one embodiment, the server 110 may edit the first video according to the user's operation to generate an edited video, and may generate an audio-visual teaching material according to the edited video and the first event tag (or the second event tag), wherein the audio-visual teaching material is Augmented reality imagery associated with the edited video may be included. For example, the server 110 may generate a first augmented reality image corresponding to the first event tag "time 00:30: a specific object is recognized", wherein the first augmented reality image may include, for example, a description of "please Find a specific object" text box. For another example, the server 110 may generate a second augmented reality image corresponding to the second event label "time 10:18: pressure returns to normal", wherein the second augmented reality image may include, for example, a description of " Please observe the text box for Pressure Sensor. In one embodiment, the first augmented reality image (or the second augmented reality image) may be automatically generated by the server 110 according to the first video (or the edited video). In one embodiment, the server 110 may receive the user's operation through the input device to generate the first augmented reality image (or the second augmented reality image).

The second augmented reality glasses 130 can be worn by learners of audio-visual teaching materials. In one embodiment, the second augmented reality glasses 130 can obtain audio-visual teaching materials from the server 110 , and the second augmented reality glasses 130 can play videos and augmented reality images in the audio-visual teaching materials.

The second augmented reality glasses 130 can record a second video. The second video may be a video that will be used to check the learner's learning outcomes.

In one embodiment, the second augmented reality glasses 130 can identify the second event in the second video according to the second machine learning model. Machine learning models can be machine learning models such as linear regression, decision trees, or convolutional neural networks. The present disclosure does not limit the kind of the second machine learning model.

The second augmented reality glasses 130 may stop playing the first augmented reality image in response to the second event in the second video matching the first event tag. For example, when the second augmented reality glasses 130 record the second video, the second augmented reality glasses 130 may play the first augmented reality image "Please find a specific object". The second augmented reality glasses 130 can determine whether there is a second event matching the first event tag "video time 00:30: a specific object is recognized" in one or more events of the recorded second video. If there is a second event matching the first event tag in the second video, the second augmented reality glasses 130 can stop playing the first augmented reality image. In other words, when the second augmented reality glasses 130 determines that there is a second event matching the first event tag in the second video, the second augmented reality glasses 130 can determine that the learner of the audio-visual teaching material has completed the first event. The learning task "Please find a specific object" indicated by the augmented reality image.

The second augmented reality glasses 130 may start playing the second augmented reality image in response to the second event in the second video matching the first event tag. For example, if the second augmented reality glasses 130 determines that the learner of the audio-visual teaching material has completed the learning task "please find a specific object" indicated by the first augmented reality image, the second augmented reality glasses 130 may Start playing the second augmented reality image to require the learners of the audio-visual teaching material to continue to follow the instructions of the second augmented reality image to perform the next step of the learning task "please observe the pressure sensor".

It should be noted that the first augmented reality glasses 120 and the second augmented reality glasses 130 in the embodiment of the present disclosure may be different augmented reality glasses, but may also be the same augmented reality glasses. This disclosure does not limit this.

FIG. 4 is a flowchart illustrating a method for producing an augmented reality-based audio-visual teaching material according to an embodiment of the present disclosure, wherein the method can be implemented by the system 100 shown in FIG. 1 .

In step S401, a video recording event definition file is generated by the server according to the first operation of the user.

In step S402, a first video is recorded by the first augmented reality glasses.

In step S403, a video event definition file is received from the server by the first augmented reality glasses.

In step S404, a first event tag corresponding to the first event is generated by the first augmented reality glasses in response to the match between the first event in the first video and the recording event definition file, and the first video and the first video are transmitted Event labels to the server.

In step S405, an audio-visual teaching material is generated by the server according to the first video and the first event tag.

To sum up, the system and method for an audio-visual teaching material based on augmented reality of the present disclosure can enable augmented reality glasses to generate event tags according to the video event definition file. The server can automatically generate audio-visual teaching materials according to the event tags. In this way, the fluency of recording audio-visual teaching materials can be increased. Besides, the augmented reality glasses can judge whether the learner has completed the learning task in the audio-visual teaching material by playing the augmented reality image and judging whether the event label matches the event.

Although the present disclosure has been disclosed above with examples, it is not intended to limit the present disclosure. Anyone with ordinary knowledge in the technical field may make some changes and modifications without departing from the spirit and scope of the present disclosure. The scope of protection of the present disclosure shall be determined by the scope of the appended patent application.

100: System 110: Server 120: First Augmented Reality Glasses 130: Second Augmented Reality Glasses S401, S402, S403, S404, S405: Steps

FIG. 1 is a schematic diagram of a system for producing an audio-visual teaching material based on augmented reality according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram illustrating an event tag of a video according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram illustrating an audio-visual teaching material according to an embodiment of the present disclosure. FIG. 4 is a flowchart illustrating a method for producing an audio-visual teaching material based on augmented reality according to an embodiment of the present disclosure.

100: System

110: Server

120: First Augmented Reality Glasses

130: Second Augmented Reality Glasses

Claims (8)

A system for producing augmented reality-based audio-visual teaching materials, comprising: a server, which generates a video event definition file according to the first operation of the user; and The first augmented reality glasses are communicatively connected to the server and record a first video, wherein the first augmented reality glasses receive the video event definition file from the server, in response to the first The first event in the video is matched with the recording event definition file to generate a first event tag corresponding to the first event, and the first video and the first event tag are transmitted to the server, wherein The server generates the audio-visual teaching material according to the first video and the first event tag. The system of claim 1, wherein The server edits the first video according to the second operation of the user to generate an edited video, and generates the audio-visual teaching material according to the edited video and the first event tag, wherein the audio-visual teaching material Augmented reality imagery associated with the edited video is included. The system of claim 1, wherein the audio-visual teaching material comprises a first augmented reality image corresponding to the first event tag, wherein the system further comprises: The second augmented reality glasses are communicatively connected to the server and record a second video, wherein the second augmented reality glasses obtain the audio-visual teaching material from the server and play the first augmented reality and stop playing the first augmented reality image in response to a second event in the second movie matching the first event tag. The system of claim 3, wherein The audio-visual teaching material further includes a second augmented reality image corresponding to the second event label, wherein The second augmented reality glasses start playing the second augmented reality image in response to the second event in the second video matching the first event tag. The system of claim 3, wherein The first augmented reality glasses identify the first event in the first video according to a first machine learning model, wherein The second augmented reality glasses identify the second event in the second video according to a second machine learning model. The system of claim 1, wherein The first augmented reality glasses identify an object in the first video, and determine that the first event occurs in response to identifying the object matching the video event definition file. The system of claim 1, wherein The first augmented reality glasses recognize the action of the object in the first video, and determine that the first event occurs in response to the action matching the video event definition file. A method for producing an audio-visual teaching material based on augmented reality, comprising: A video event definition file is generated by the server according to the user's first operation; recording the first video by the first augmented reality glasses; receiving the recording event definition file from the server by the first augmented reality glasses; A first event tag corresponding to the first event is generated by the first augmented reality glasses in response to the first event in the first video being matched with the recording event definition file, and the first event tag is transmitted. a video and the first event tag to the server; and The audio-visual teaching material is generated by the server according to the first video and the first event tag.

Images