CN111242962A - Method, device and equipment for generating remote training video and storage medium - Google Patents
Method, device and equipment for generating remote training video and storage medium Download PDFInfo
- Publication number
- CN111242962A CN111242962A CN202010043553.0A CN202010043553A CN111242962A CN 111242962 A CN111242962 A CN 111242962A CN 202010043553 A CN202010043553 A CN 202010043553A CN 111242962 A CN111242962 A CN 111242962A
- Authority
- CN
- China
- Prior art keywords
- video
- picture
- independent
- portrait
- teaching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/958—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
- H04N23/959—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Signal Processing (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Electrically Operated Instructional Devices (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, and discloses a method for generating a remote training video. The invention also provides a device for generating the remote training video, training equipment and a computer readable storage medium. The invention realizes that both sides can observe the states of teaching and listening in real time, improves the interactivity and can quickly and accurately convey the training content.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method, a device and equipment for generating a remote training video and a computer readable storage medium.
Background
Multimedia (Multimedia) is a composite of multiple media, generally including multiple media forms such as text, sound, and images. Multimedia is the embodiment of modern informatization and the trend of social development, and particularly in the field of education, multimedia education also belongs to a part of modern informatization, so that the multimedia education is greatly promoted to become the trend of educational development, meanwhile, the defects in the traditional teaching are overcome, and the teaching can be simultaneously carried out in different regions.
In order to solve the problem of regional limitation of the traditional education mode, in the prior art, remote teaching of multimedia technology is realized by using the internet, but in the training of remote teaching and the like at present, a teacher needs to share courseware (PPT and/or other documents) and a computer desktop with students, but the students can not see expressions, actions and body languages of the teacher when watching a computer, so that body expression information of the teacher can be omitted, and besides, the interactivity and the interestingness are poor.
Disclosure of Invention
The invention mainly aims to provide a method, a device and equipment for generating a remote training video and a computer readable storage medium, and aims to solve the technical problems that the existing remote training video is too single in display and lacks interactivity and interestingness.
In order to achieve the purpose, the invention provides a method for generating a remote training video, which is applied to a remote training platform and comprises the following steps:
acquiring a video stream in a training process, wherein the video stream comprises: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
performing segmentation processing on the video stream through a preset image segmentation model to obtain an independent picture, wherein the independent picture comprises one or more of an independent portrait picture, an independent teaching courseware picture and an independent student picture of the lecturer;
extracting element content in the independent picture and position information of the element content in the picture;
constructing a picture frame of the virtual lecture hall according to the position information, wherein the picture frame is a picture layout for accommodating the independent picture;
and adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
Optionally, after the step of acquiring a video stream in a training process, the method further includes:
detecting whether a teaching video in the video stream is a mixed video, wherein the mixed video comprises a portrait video of the lecturer and a teaching courseware video;
if the teaching video is a mixed video, extracting an independent portrait picture of a lecturer in the face video by using a portrait extraction algorithm, and extracting courseware information currently used by the lecturer in the teaching courseware video by using a character detection algorithm to obtain the independent teaching courseware picture;
and if the teaching video is a non-mixed video, executing a step of segmenting the video stream through a preset image segmentation model to obtain an independent picture.
Optionally, the step of segmenting the video stream by using a preset image segmentation model to obtain an independent picture includes:
if the video stream is the lecture video of the lecturer, calculating depth values of the depth of field of each picture in the lecture video according to a preset depth of field formula;
identifying a foreground area and a background area of a picture in the teaching video according to the depth value, wherein the foreground area comprises a portrait picture;
extracting the foreground region from the teaching video by using an image matting algorithm to obtain a foreground video picture, and extracting the background region from the teaching video to obtain an independent teaching courseware picture;
and identifying a portrait picture in the foreground area according to a preset portrait identification algorithm, and extracting the portrait picture from the foreground area to form the independent portrait picture.
Optionally, the step of segmenting the video stream by using a preset image segmentation model to obtain an independent picture includes:
if the video stream is a lesson listening interactive video of the student, identifying whether the lesson listening interactive video contains the student meeting classroom interaction postures by using a face identification technology, wherein the classroom interaction postures comprise standing and raising hands;
if the answer exists, carrying out human body scanning processing on the student through a camera to obtain a portrait outline of the student, and calculating a depth of field value of the portrait outline in the lecture attending interactive video according to a preset depth of field formula;
and cutting all pictures positioned on the critical point in the lecture interactive video by taking the depth of field value as a picture cutting critical point to form the picture of the independent student.
Optionally, the step of extracting element contents in the independent portrait screen, the independent teaching courseware screen and/or the independent trainee screen, and the position information of the element contents in the screens includes:
creating a canvas according to the length and the width of the independent picture, selecting any one corner point in the canvas as a coordinate origin, and establishing a two-dimensional coordinate system;
calculating coordinate information of the portrait of the lecturer or the portrait of the student in the independent picture and calculating coordinate information of the teaching content of the teaching courseware in the independent picture based on the two-dimensional coordinate system;
and extracting the portrait and courseware contents from the independent picture according to the coordinate information.
Optionally, the step of constructing a frame of a virtual lecture hall according to the location information includes:
taking the independent teaching courseware picture as a background canvas of an AI lecture, and constructing a coordinate system on the background canvas;
according to the coordinate information, a portrait filling area with the same shape as the portrait is sketched on the background canvas to obtain the picture frame;
the step of adding the element content to the corresponding position of the picture frame to obtain the training video of the AI lecture hall comprises the following steps:
and filling the extracted portrait into the corresponding portrait filling area, and fusing the portrait with the background canvas through a boundary interpolation background fusion algorithm to obtain the training video.
Optionally, the depth of field calculation formula is:
wherein δ is allowed to be a circle of confusion diameter, F lens focal length, F lens photographing aperture value, and L focus distance.
In order to solve the above problem, the present invention also provides a remote training video generation apparatus, including:
the acquisition module is used for acquiring a video stream in a training process, wherein the video stream comprises: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
the segmentation module is used for segmenting the video stream through a preset image segmentation model to obtain an independent picture, wherein the independent picture comprises one or more of an independent portrait picture, an independent teaching courseware picture and an independent student picture of the lecturer;
the extraction module is used for extracting the element content in the independent picture and the position information of the element content in the picture;
the synthesis module is used for constructing a picture frame of the virtual lecture hall according to the position information, and the picture frame is a picture layout used for accommodating the independent picture; and adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
Optionally, the device for generating the remote training video further includes a detection module, configured to detect whether a teaching video in the video stream is a mixed video, where the mixed video includes a portrait video of the lecturer and a courseware video;
if the teaching video is a mixed video, the segmentation module extracts an independent portrait picture of a lecturer in the face video by using a portrait extraction algorithm, and extracts courseware information currently used by the lecturer in the teaching courseware video by using a character detection algorithm to obtain the independent teaching courseware picture;
and if the teaching video is a non-mixed video, controlling the segmentation module to execute a preset image segmentation model to segment the video stream to obtain an independent picture.
Optionally, the segmentation module comprises a first calculation unit, an identification unit and a first cutting unit, wherein:
the first calculating unit is used for calculating depth values of the depth of field of each picture in the teaching video according to a preset depth-of-field formula;
the identification unit is used for identifying a foreground area and a background area of a picture in the teaching video according to the depth value, wherein the foreground area comprises a portrait picture;
the first cutting unit is used for extracting the foreground region from the teaching video by using an image matting algorithm to obtain a foreground video picture, and extracting the background region from the teaching video to obtain the independent teaching courseware picture; and identifying a portrait picture in the foreground area according to a preset portrait identification algorithm, and extracting the portrait picture from the foreground area to form the independent portrait picture.
Optionally, the segmentation module includes a face recognition unit, a second calculation unit, and a second cutting unit, wherein:
the face recognition unit is used for recognizing whether students meeting classroom interaction postures exist in the lecture listening interaction video or not by using a face recognition technology, wherein the classroom interaction postures comprise standing and raising hands;
if the answer exists, the second calculating unit carries out human body scanning processing on the student through the camera to obtain a portrait outline of the student, and calculates a depth of field value of the portrait outline in the lecture listening interactive video according to a preset depth of field formula;
the second cutting unit is used for cutting all pictures on the critical point in the lecture attending interactive video by taking the depth of field value as a picture cutting critical point to form the independent student picture.
Optionally, the extraction module is configured to create a canvas according to the length and the width of the independent picture, select any one corner point in the canvas as a coordinate origin, and establish a two-dimensional coordinate system; calculating coordinate information of the portrait of the lecturer or the portrait of the student in the independent picture and calculating coordinate information of the teaching content of the teaching courseware in the independent picture based on the two-dimensional coordinate system; and extracting the portrait and courseware contents from the independent picture according to the coordinate information.
Optionally, the synthesis module is configured to use the independent teaching courseware picture as a background canvas of an AI lecture, and construct a coordinate system on the background canvas; according to the coordinate information, a portrait filling area with the same shape as the portrait is sketched on the background canvas to obtain the picture frame; the step of adding the element content to the corresponding position of the picture frame to obtain the training video of the AI lecture hall comprises the following steps: and filling the extracted portrait into the corresponding portrait filling area, and fusing the portrait with the background canvas through a boundary interpolation background fusion algorithm to obtain the training video.
Optionally, the depth of field calculation formula is:
wherein δ is allowed to be a circle of confusion diameter, F lens focal length, F lens photographing aperture value, and L focus distance.
In addition, to achieve the above object, the present invention also provides a training apparatus including: a memory, a processor, and a remote training video generation program stored on the memory and executable on the processor, the remote training video generation program when executed by the processor implementing the steps of the remote training video generation method as in any one of the above.
Further, to achieve the above object, the present invention provides a computer-readable storage medium having stored thereon a program for generating a remote training video, which when executed by the processor, implements the steps of the method for generating a remote training video according to any one of the above.
The invention provides a method for generating a remote training video, which mainly comprises the steps of collecting a teaching video of a lecturer and a training video of a student in real time, carrying out video segmentation based on an image segmentation model, extracting an individual picture, extracting element content and position information of the element content in the original picture from the picture, constructing a picture frame of a composite video according to the position information, and synthesizing a video stream into a new training video based on the picture frame to obtain an AI lecture video.
Drawings
FIG. 1 is a schematic diagram of an operating environment of a remote training platform according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart diagram illustrating an embodiment of a method for generating a remote training video according to the present invention;
FIG. 3 is a schematic flow chart diagram illustrating a method for generating a remote training video according to another embodiment of the present invention;
FIG. 4 is a schematic flow chart of element content extraction provided by the present invention;
FIG. 5 is a schematic view illustrating a flow chart of a classroom screen layout according to the present invention;
FIG. 6 is a schematic flow chart diagram illustrating a method for generating a remote training video according to yet another embodiment of the present invention;
fig. 7 is a functional block diagram of an embodiment of a device for generating a remote training video according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a remote training platform, and referring to fig. 1, fig. 1 is a schematic structural diagram of an operating environment of the remote training platform according to an embodiment of the invention.
As shown in fig. 1, the remote training platform includes: a processor 101, e.g. a CPU, a communication bus 102, a user interface 103, a network interface 104, a memory 105. Wherein the communication bus 102 is used for enabling connection communication between these components. The user interface 103 may comprise a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the network interface 104 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface). The memory 105 may be a high-speed RAM memory or a non-volatile memory (e.g., a disk memory). The memory 105 may optionally also be a storage device separate from the aforementioned processor 101.
Those skilled in the art will appreciate that the hardware configuration of the remote training platform shown in fig. 1 does not constitute a limitation of the remote training video generation apparatus and training device of the present invention, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, the memory 105, which is a type of computer-readable storage medium, may include therein an operating system, a network communication program module, a user interface program module, and a generation program for implementing a remote training video. The operating system schedules communication among modules in the remote training platform and executes a generating program of a remote training video stored in a memory so as to realize synthesis of the training video, wherein the synthesis comprises synthesis of a portrait of a lecturer, a teaching courseware and an interactive picture between the lecturer and a student, so that the real-time performance and the interestingness of the training video can be greatly improved, and the on-site experience of remote teaching is realized.
In the hardware configuration of the remote training platform shown in fig. 1, the network interface 104 is mainly used for accessing a network; the user interface 103 is mainly used for monitoring a real-time teaching picture of a lecturer end and a real-time teaching listening picture of a student end, pictures at two ends are obtained through monitoring of the user interface 103, then the control processor 101 calls a remote training video generation program stored in the memory 105 to synthesize the monitored pictures in real time and update the synthesized pictures to the two parties, so that the two parties can observe the pictures of the other party in real time, the integrity and the on-site experience of the teaching pictures can be reflected, the interestingness of the teaching pictures is enhanced, and the operation of each embodiment of the remote training video generation method provided below is specifically realized.
Based on the hardware structure of the remote training platform, embodiments of the method for generating the remote training video are provided, and of course, the remote training platform listed here is only one implementation device for executing the method for generating the remote training video provided by the embodiments of the present invention, and in practical application, the implementation device may also be a training robot, and the training robot may be an AP or VR device, and the method is executed to implement remote lecture, so that the on-site experience of a training video picture is enhanced, and the interactive training picture of both parties can be embodied.
Referring to fig. 2, fig. 2 is a flowchart of a method for generating a remote training video according to an embodiment of the present invention. The method is essentially an AI lecture room for realizing remote training, and specifically comprises the steps of obtaining a lecture sample set of a lecturer in a training process, a lecture listening interaction sample set and a lecture courseware sample set of a student in the training process, and constructing a virtual AI lecture room according to the lecture sample set, the lecture listening interaction sample set, the lecture courseware sample set, a combined image segmentation model and a video synthesis model.
In the training process, specific personnel to be trained are determined through face recognition, then video streams of a lecturer in the teaching process are collected, video streams of a student in the training process are collected, video streams of courseware playing are collected, the video stream comprises a teaching sample set in the teaching process of the lecturer and a lecture listening interaction sample set in the teaching process of the student, the video stream is uploaded to the image segmentation model for image segmentation processing to obtain a lecturer teaching independent sample, a student lecture listening interaction independent sample and a courseware sample, a remote training AI lecture hall is generated according to the lecturer teaching independent sample, the student attending interactive independent sample and the courseware sample input video synthesis model, therefore, the regional limitation of education is solved, interactive experience between the lessee and the lessee can be realized, and the method for generating the remote training video specifically comprises the following steps:
step S210, obtaining a video stream in a training process, wherein the video stream comprises: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
in this step, the video stream is to be understood as a generic term for video data generated from the instructor side and the trainee side, and the video stream may include only video data at one end or video data at both ends, and specifically may be acquired in real time through the internet, or may be read from a database of a transfer platform for remote training.
If the video is connected to the intermediate control platform, the method in the embodiment should be applied to the intermediate control platform, and after the video is finally synthesized, the synthesized video is synchronized to the two ends in real time to be played and displayed; if the two ends are connected with each other, the instructor end is selected as the main operation end, namely the method provided by the embodiment is applied to the training equipment of the instructor end, the training equipment of the instructor end monitors the video stream of the student end, the video stream of the student end is divided, key information is extracted and synthesized into the video stream of the instructor end, and the key information is synchronously played for the student end.
In practical application, a depth camera is adopted to collect videos given by lecturers in a remote training process, color and depth information of each frame in the videos are extracted, areas where images are located are marked to obtain corresponding labels, a convolutional neural network is trained through the color and depth information, color and spatial continuity of the images in a three-dimensional world exist, the convolutional neural network can be used for predicting the areas which are possibly the images in the video frames, and the video frames of original images of the lecturers are obtained through segmentation.
Step S220, carrying out segmentation processing on the video stream through a preset image segmentation model to obtain independent pictures, wherein the independent pictures comprise an independent portrait picture, an independent teaching courseware picture and an independent student picture of teaching by the lecturer;
in practical applications, the image segmentation model may be any one of the following algorithms, which are a threshold segmentation algorithm, an edge segmentation algorithm, a region segmentation algorithm, a cluster analysis image segmentation algorithm, and an artificial neural network segmentation algorithm, and training image segmentation is performed based on these algorithms to obtain a final model.
In this embodiment, a preferred segmentation algorithm for selecting an artificial neural network has a basic idea that a linear decision function is obtained by training multi-layer perception, and then pixels are classified by the decision function to achieve the purpose of segmentation. The neural network has huge connection, and spatial information is easy to introduce, so that the problems of noise and unevenness in images are further solved. In practical application, as to which network structure should be selected as the image segmentation model, the selection is specifically made according to the practical requirements of remote training.
Step S230, extracting element contents in the independent portrait picture, the independent teaching courseware picture and/or the independent student picture, and position information of the element contents in the picture;
in this embodiment, when the obtained video stream is a video stream of a lecturer, a lecturer teaching picture and a courseware picture in the video stream are divided and separated, specifically, a portrait area and a courseware area are identified and distinguished through a portrait identification algorithm and a character identification algorithm, based on the distinction, video pictures in the two areas are extracted, and certainly, the portrait in the picture can also be extracted through a video matting technology, and the background color of the courseware area is filled, so that the separation of the lecturer teaching picture and the courseware picture is realized.
In practical application, for the two kinds of picture segmentation, the segmentation can be extracted by a region labeling mode, that is, the segmentation is firstly identified, then the regions are traced by adopting different marks, the moving condition of the regions is monitored and tracked in real time, and meanwhile, the position information of the regions is calculated, wherein the position information is specific to the coverage area of the picture.
Step S240, constructing a picture frame of a virtual lecture hall according to the position information, wherein the picture frame is a picture layout for simultaneously accommodating the independent portrait picture, the independent teaching courseware picture and the independent student picture;
in this step, the frame refers to a blank background frame, in which a fixed wandering area is formed according to the position information of the contents of different elements, or a connection relationship between the area and the portrait in the frame is established, and the area moves along with the portrait on the frame.
In practical application, the area corresponding to the teaching courseware image in the image frame is fixedly arranged, based on the fixed area, the motion relation between the lecturer portrait area and the trainee portrait area in the fixed area is established, and meanwhile, the association between the motion area and the video image is also established, so that the final image frame is obtained.
And S250, adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
In this step, when adding element content, automatic filling can be performed by presetting a mode of filling fixed type elements in each position in the frame, that is, a corresponding relationship between the positions and the elements is firstly established, the corresponding element content can be obtained by monitoring the corresponding frame through a corresponding recognition technology, and the element content is mapped to a corresponding area in the frame, so that a training video frame of any action and information of both sides can be seen at the same time.
In practical application, for the mapping of the portrait, only the movement relationship can be mapped, the actual portrait picture does not need to be mapped, the mapped picture can be a preset figurine of the system, and certainly, in order to improve interactivity and interestingness, the real portrait can be directly mapped.
In this embodiment, there are two situations for the video stream of the instructor side, that is, the lecture picture and the courseware picture may be separately recorded and acquired, or may be separately recorded and acquired, if the instructor side uses projection teaching, it is possible that the instructor and the courseware are recorded and acquired together, and when the instructor only uses a computer to teach, the instructor does not record the video pictures of the lecture side and the courseware at the same time, but separately records the video pictures.
In practical application, a lecture video for a lecturer is generally obtained by dividing into two parts, one part is a lecture video for the lecturer, and the other part is a courseware video, and for the two cases, there is a difference in ID when a camera records, so in order to facilitate the synthesis of the video, in this embodiment, after a video stream is obtained, the video stream needs to be detected, and the specific process is as follows:
detecting whether a teaching video in the video stream is a mixed video, wherein the mixed video comprises a portrait video of the lecturer and a teaching courseware video;
if the teaching video is a mixed video, extracting an independent portrait picture of a lecturer in the face video by using a portrait extraction algorithm, and extracting courseware information currently used by the lecturer in the teaching courseware video by using a character detection algorithm to obtain the independent teaching courseware picture;
and if the teaching video is not a mixed video, executing a step of segmenting the video stream through a preset image segmentation model to obtain an independent picture.
In practical applications, it may be specifically determined whether the lecture video is a mixed video by detecting whether a capture track of the lecture video is a single track, if so, it is determined that the lecture video is not a mixed video, and further, it may be determined by detecting a video source in the lecture video, when it is detected that the lecture video does not belong to the mixed video, that is, a video collected from only one video source in a video stream is detected, the step S220 is skipped to perform the frame segmentation process, of course, before the frame segmentation process, it may also be detected whether a human-image frame exists in the video stream, if so, the step S220 is executed, otherwise, the step S230 is directly skipped.
If it is detected that the video stream belongs to a mixed video, that is, when video materials collected by more than two video sources exist in the video stream, extracting the pictures of the video stream through different extraction algorithms, wherein a specific processing process is shown in fig. 3.
For the acquisition of the mixed video stream, the extraction of the picture can be specifically carried out in the following way:
s301, storing a first source video and a second source video into a first video frame queue and a second video frame queue respectively by taking a frame as a unit;
s302, extracting a frame of image of a first source video from a first video frame queue, processing the image, extracting an interested moving target and acquiring a foreground image;
s303, extracting a frame of image of a second source video from the second video frame queue, and respectively storing the obtained foreground image and the frame of image of the second source video extracted from the second video frame queue into different pictures;
and S304, repeating the steps S302-S303 until all the images in the first video frame queue and the second video frame queue are processed, finally synthesizing the extracted images into a new image set, and then synthesizing the images extracted from the student side into the image set.
In this embodiment, in order to reduce data processing of the remote training platform, specifically, when a teaching video is divided, a video at the instructor end and a video at the student end may be separately processed, and preferably, the videos may be separately placed on corresponding recording devices for processing, where the processing at the instructor end specifically includes:
if the video stream is the lecture video of the lecturer, the step of segmenting the video stream through a preset image segmentation model to obtain an independent picture comprises the following steps:
calculating depth values of the depth of field of each picture in the teaching video according to a preset depth-of-field formula;
identifying a foreground area and a background area of a picture in the teaching video according to the depth value, wherein the foreground area comprises a portrait picture;
extracting the foreground region from the teaching video by using an image matting algorithm to obtain a foreground video picture, and extracting the background region from the teaching video to obtain an independent teaching courseware picture;
and identifying a portrait picture in the foreground area according to a preset portrait identification algorithm, and extracting the portrait picture from the foreground area to form the independent portrait picture.
In this embodiment, if the video stream is a lesson-listening interactive video of the student, the step of performing segmentation processing on the video stream through a preset image segmentation model to obtain an independent picture includes:
identifying whether students meeting classroom interaction postures exist in the lecture listening interaction video or not by using a face identification technology, wherein the classroom interaction postures comprise standing and raising hands;
if the answer exists, carrying out human body scanning processing on the student through a camera to obtain a portrait outline of the student, and calculating a depth of field value of the portrait outline in the lecture attending interactive video according to a preset depth of field formula;
and cutting all pictures positioned on the critical point in the lecture interactive video by taking the depth of field value as a picture cutting critical point to form the picture of the independent student.
In this embodiment, the depth of field calculation formula is obtained as follows:
wherein, the delta L depth of field, the delta allowable circle diameter of diffusion, the F lens focal length, the shooting aperture value of the F lens, the L focusing distance and the delta L1 front depth of field.
In practical application, generally, the video stream collected from the student end is only the portrait video of the student, so that only the portrait needs to be extracted, and certainly, the pictures of the teaching courseware collected at the same time are not excluded.
In this embodiment, when there is an interaction with the student, it is necessary to record the interaction information of the student, that is, it is necessary to extract various motion information and question information of the student by tracking the motion.
After each depth of field in the video stream is calculated based on the above calculation formula, the independent portrait picture, the independent lecture courseware picture and the independent student picture of the lecturer are extracted based on the depth of field, and in practical application, the independent portrait picture and the independent lecture courseware picture are generally obtained based on the video stream of the lecturer side, that is, the depth of field where the portrait of the lecturer is located is a foreground area, the portrait picture of the lecturer is obtained from the foreground area, and the background area with a relatively increased depth of field value is used as a background area, from which courseware information is extracted.
In order to improve the quality of the training video, in this embodiment, after the extracted corresponding picture, important information of teaching and listening is extracted from the corresponding picture through a matting and character extraction technology, and a new video picture is constructed based on the important information, that is, the element contents in the independent portrait picture, the independent teaching courseware picture and/or the independent student picture, and the position information of the element contents in the picture are extracted.
In practical application, the specific implementation process of the extraction step is shown in fig. 4:
step S401, creating a canvas according to the length and the width of the independent picture, selecting any one corner point in the canvas as a coordinate origin, and establishing a two-dimensional coordinate system;
step S402, calculating coordinate information of the portrait of the lecturer or the portrait of the student in the independent picture and calculating coordinate information of the teaching content of the teaching courseware in the independent picture based on the two-dimensional coordinate system;
and S403, extracting the portrait and courseware content from the independent picture according to the coordinate information.
In practical applications, when creating a two-dimensional coordinate system, the two-dimensional coordinate system is usually constructed by using a display device of a picture, and different display sizes can be converted according to resolution to obtain corresponding position information.
In this embodiment, for the extraction of the elements, it may also be to perform the distinguishing extraction according to colors, for example, to identify a background color, extract all elements in the background color as the courseware information, extract elements of non-background colors as the portrait elements, and of course, the extraction of the portrait elements may also be performed by determining whether the elements are moving, and if the elements are moving, the elements are regarded as the portrait elements, and then perform the drawing extraction of the outlines.
Further, before constructing a new training video frame based on the extracted elements, a frame layout of a classroom needs to be constructed, and the specific construction steps are shown in fig. 5:
step S501, taking the independent teaching courseware picture as a background canvas of an AI lecture, and constructing a coordinate system on the background canvas;
step S502, according to the coordinate information, a portrait filling area with the same shape as the portrait is sketched on the background canvas to obtain the picture frame;
based on the constructed picture frame, correspondingly filling the extracted elements into corresponding positions, specifically adding the element contents to the corresponding positions of the picture frame, and obtaining a training video of the AI lecture hall, the method comprises the following steps:
and filling the extracted portrait into the corresponding portrait filling area, and fusing the portrait with the background canvas through a boundary interpolation background fusion algorithm to obtain the training video.
In practical application, in the process of synthesizing the extracted elements into the training video, the video synthesis model can be realized, specifically, after the elements in the picture frame are synthesized into a complete video, the video synthesis model also performs the coloring treatment on the filled training video, namely, the edges in the picture frame are mutually fused by means of the treatments of drying, feathering and the like, so that the seamless connection of the video materials is realized.
In summary, the training video generated by the method provided by the embodiment can improve the participation sense of the students and the interaction effect of the students of the lecturer in the remote training scene, and finally assist schools, institutions and the like to improve the training effect and the learning achievement of the students in the links of remote training, teaching and the like.
The following describes in detail the implementation of the method for generating a training video according to the present invention, taking a lecture video at a lecturer end as an example, as shown in fig. 6.
Step S601, a depth camera is adopted to collect a video of lectures given by a lecturer in a remote training process;
the video comprises the portrait of the instructor and courseware information, and preferably, the courseware information can be read directly from the training equipment, and of course, the courseware information can also be obtained by recording from a projection screen of the training video by using a depth camera.
Step S602, extracting color and depth information of each frame in the video, and labeling the area where the image is located to obtain a corresponding label;
in this embodiment, when extracting a video frame, a convolutional neural network may be specifically trained through color and depth information, the color and depth information in the video is extracted based on the trained neural network, and continuity in a three-dimensional space between the color and depth information and a label is also established, further, continuity in a space with a human figure of a lecturer may be added, an area that may be a human figure in the video frame may be predicted by using the convolutional neural network, and a video frame of an original image of the lecturer is obtained by segmentation.
Step S603, overlapping the obtained lecturer and the set background image, and carrying out interpolation and denoising on the boundary to obtain a generated video frame;
and step S604, combining according to the time sequence of the video frames to obtain the AI lecture hall.
The remote training AI lecture room based on image segmentation and video synthesis extracts action videos of lecturer teaching videos, student interaction and the like in an image segmentation mode, and then combines courseware videos in a real-time online video generation mode to generate the AI lecture room, so that the student participation sense of a remote training scene and the interaction effect of lecturers and students can be greatly improved, and finally schools, institutions and the like are assisted to improve the training effect and the learning performance of students in remote training, teaching and other links.
In this embodiment, if there is an interactive video of the student, the real-time lecture listening video information of the student can be added to the synthesized AI classroom video, and the implementation process thereof is as follows:
firstly, determining specific personnel to be trained through face recognition in a training process, acquiring video streams of a lecturer in a teaching process, acquiring video streams of a student in the training process, and acquiring video streams played by courseware, wherein the video streams comprise a teaching sample set in the lecturer teaching process and a lecture listening interaction sample set in the student training process;
then, uploading the video stream to the image segmentation model for image segmentation processing to obtain an independent lecture teaching sample of a lecturer, an independent lecture listening interaction sample of a student and a courseware sample;
and inputting the lecturer teaching independent sample, the student attending interactive independent sample and the courseware sample into a video synthesis model to generate a remote training AI lecture.
At this moment, in the process of synthesizing the video by the video synthesis model, the teacher teaching independent sample, the student attending teaching interactive independent sample, the courseware sample, the teacher teaching independent sample and the courseware sample from the teacher end or the teacher teaching independent sample and the courseware sample need to be compared to remove the duplication, and different video frames are synthesized into the video, so that the simplicity and the accuracy of the video and the real-time performance of the video are ensured.
In summary, the remote training AI lecture hall based on image segmentation and video synthesis uses the image segmentation model and the video synthesis model to analyze and process the student participation process and the lecture giving process, so as to generate the remote training AI lecture hall, which can improve the student participation sense and the lecturer student interaction effect in a remote training scene, and finally assist schools, institutions and the like in remote training, lecture giving and other links to improve the training effect.
In order to solve the above problem, an embodiment of the present invention further provides a device for generating a remote training video, as shown in fig. 7, the device for generating a remote training video includes:
an acquisition module 71, configured to acquire a video stream in a training process, where the video stream includes: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
a segmentation module 72, configured to perform segmentation processing on the video stream through a preset image segmentation model to obtain an independent picture, where the independent picture includes an independent portrait picture, an independent lesson courseware picture, and an independent trainee picture of the lecturer;
an extracting module 73, configured to extract element contents in the independent portrait screen, the independent teaching courseware screen, and/or the independent trainee screen, and position information of the element contents in the screen;
a composition module 74, configured to construct a frame of a virtual lecture hall according to the position information, where the frame is a frame layout for accommodating the independent teaching picture, the independent teaching courseware picture, and the independent student picture at the same time; and adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
The execution function and the execution flow corresponding to the function based on the device are the same as the contents described in the above embodiment of the method for generating a remote training video according to the embodiment of the present invention, and therefore, the contents of the embodiment of the device for generating a remote training video are not described in detail in this embodiment.
In addition, an embodiment of the present invention further provides a training apparatus, including: the method for generating the remote training video is realized by referring to various embodiments of the method for generating the remote training video provided by the invention when the program for generating the remote training video is executed by the processor, and therefore, redundant description is not repeated.
The invention also provides a computer readable storage medium.
In this embodiment, the computer-readable storage medium stores a program for generating a remote training video, and the method implemented when the program for generating a remote training video is executed by the processor may refer to each embodiment of the method for generating a remote training video of the present invention, and therefore, description thereof is not repeated.
The method and the device provided by the embodiment of the invention mainly analyze and process the student participation process and the lecture giving process by using the image segmentation model and the video synthesis model to generate the remote training AI lecture hall, thereby improving the student participation sense in the remote training scene and the interaction effect of the lecturer and the student.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM), and includes instructions for causing a terminal (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The embodiments of the present invention have been described in connection with the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are only illustrative and not restrictive, and those skilled in the art can make many changes and modifications without departing from the spirit and scope of the invention as claimed, and all changes and modifications that come within the meaning and range of equivalency of the claims are intended to be embraced therein.
Claims (10)
1. A method for generating a remote training video is applied to a remote training platform and is characterized by comprising the following steps:
acquiring a video stream in a training process, wherein the video stream comprises: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
performing segmentation processing on the video stream through a preset image segmentation model to obtain an independent picture, wherein the independent picture comprises one or more of an independent portrait picture, an independent teaching courseware picture and an independent student picture of the lecturer;
extracting element content in the independent picture and position information of the element content in the picture;
constructing a picture frame of the virtual lecture hall according to the position information, wherein the picture frame is a picture layout for accommodating the independent picture;
and adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
2. The method for generating a remote training video as set forth in claim 1, wherein after said step of obtaining a video stream during training, further comprising:
detecting whether a teaching video in the video stream is a mixed video, wherein the mixed video comprises a portrait video of the lecturer and a teaching courseware video;
if the teaching video is a mixed video, extracting an independent portrait picture of a teacher in the face video by using a portrait extraction algorithm, extracting courseware information currently used by the teacher in the teaching courseware video by using a character detection algorithm, and synthesizing the independent portrait picture and the courseware information into an independent teaching courseware picture;
and if the teaching video is a non-mixed video, executing a step of segmenting the video stream through a preset image segmentation model to obtain an independent picture.
3. The method for generating a remote training video according to claim 2, wherein the step of segmenting the video stream by a preset image segmentation model to obtain an independent picture comprises:
when the video stream is the lecture video of the lecturer, calculating depth values of the depth of field of each picture in the lecture video according to a preset depth of field formula;
identifying a foreground area and a background area of a picture in the teaching video according to the depth value, wherein the foreground area comprises a portrait picture;
extracting the foreground region from the teaching video by using an image matting algorithm to obtain a foreground video picture, and extracting the background region from the teaching video to obtain an independent teaching courseware picture;
and identifying a portrait picture in the foreground area according to a preset portrait identification algorithm, and extracting the portrait picture from the foreground area to form the independent portrait picture.
4. The method for generating a remote training video according to claim 2, wherein the step of segmenting the video stream by a preset image segmentation model to obtain an independent picture comprises:
when the video stream is a lesson listening interactive video of the student, identifying whether the lesson listening interactive video contains the student meeting classroom interaction postures by using a face identification technology, wherein the classroom interaction postures comprise standing and raising hands;
if the answer exists, carrying out human body scanning processing on the student through a camera to obtain a portrait outline of the student, and calculating a depth of field value of the portrait outline in the lecture attending interactive video according to a preset depth of field formula;
and cutting all pictures positioned on the critical point in the lecture interactive video by taking the depth of field value as a picture cutting critical point to form the picture of the independent student.
5. The method for generating a remote training video according to any one of claims 1 to 4, wherein the step of extracting the element content in the independent screen and the position information of the element content in the screen includes:
creating a canvas according to the length and the width of the independent picture, selecting any one corner point in the canvas as a coordinate origin, and establishing a two-dimensional coordinate system;
calculating coordinate information of the portrait of the lecturer or the portrait of the student in the independent picture and calculating coordinate information of the teaching content of the teaching courseware in the independent picture based on the two-dimensional coordinate system;
and extracting the portrait and courseware contents from the independent picture according to the coordinate information.
6. The method for generating a remote training video according to claim 5, wherein the step of constructing a frame of a virtual lecture hall based on the position information includes:
taking the independent teaching courseware picture as a background canvas of an AI lecture, and constructing a coordinate system on the background canvas;
according to the coordinate information, a portrait filling area with the same shape as the portrait is sketched on the background canvas to obtain the picture frame;
the step of adding the element content to the corresponding position of the picture frame to obtain the training video of the AI lecture hall comprises the following steps:
and filling the extracted portrait into the corresponding portrait filling area, and fusing the portrait with the background canvas through a boundary interpolation background fusion algorithm to obtain the training video.
7. The method for generating a remote training video according to claim 6, wherein the depth of field calculation formula is:
wherein δ is allowed to be a circle of confusion diameter, F lens focal length, F lens photographing aperture value, and L focus distance.
8. A device for generating a remote training video, the device comprising:
the acquisition module is used for acquiring a video stream in a training process, wherein the video stream comprises: a teaching video when a lecturer gives a lecture and/or a lecture listening interactive video when a student participates in the lecture;
the segmentation module is used for segmenting the video stream through a preset image segmentation model to obtain an independent picture, wherein the independent picture comprises one or more of an independent portrait picture, an independent teaching courseware picture and an independent student picture of the lecturer;
the extraction module is used for extracting the position information in the independent picture;
the synthesis module is used for constructing a picture frame of the virtual lecture hall according to the position information, and the picture frame is a picture layout used for accommodating the independent picture; and adding the element content to the corresponding position of the picture frame to obtain a training video of the AI lecture hall.
9. A training apparatus, characterized in that the training apparatus comprises: a memory, a processor, and a remote training video generation program stored on the memory and executable on the processor, the remote training video generation program when executed by the processor implementing the steps of the method of generating a remote training video of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a generation program of a remote training video, which when executed by the processor, implements the steps of the generation method of a remote training video according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010043553.0A CN111242962A (en) | 2020-01-15 | 2020-01-15 | Method, device and equipment for generating remote training video and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010043553.0A CN111242962A (en) | 2020-01-15 | 2020-01-15 | Method, device and equipment for generating remote training video and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111242962A true CN111242962A (en) | 2020-06-05 |
Family
ID=70879554
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010043553.0A Pending CN111242962A (en) | 2020-01-15 | 2020-01-15 | Method, device and equipment for generating remote training video and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111242962A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111757140A (en) * | 2020-07-07 | 2020-10-09 | 南京百家云科技有限公司 | Teaching method and device based on live classroom |
| CN111914760A (en) * | 2020-08-04 | 2020-11-10 | 华中师范大学 | Online course video resource composition analysis method and system |
| CN111954024A (en) * | 2020-08-27 | 2020-11-17 | 顾建亮 | Course recording live broadcasting method and system |
| CN112040185A (en) * | 2020-08-28 | 2020-12-04 | 深圳市融讯视通科技有限公司 | Method and system for improving remote education courseware sharing |
| CN112232166A (en) * | 2020-10-10 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Artificial intelligence-based lecturer dynamic evaluation method and device, and computer equipment |
| CN112330579A (en) * | 2020-10-30 | 2021-02-05 | 中国平安人寿保险股份有限公司 | Video background replacing method and device, computer equipment and computer readable medium |
| CN112351291A (en) * | 2020-09-30 | 2021-02-09 | 深圳点猫科技有限公司 | Teaching interaction method, device and equipment based on AI portrait segmentation |
| CN112804516A (en) * | 2021-04-08 | 2021-05-14 | 北京世纪好未来教育科技有限公司 | Video playing method and device, readable storage medium and electronic equipment |
| CN112929688A (en) * | 2021-02-09 | 2021-06-08 | 歌尔科技有限公司 | Live video recording method, projector and live video system |
| CN113099254A (en) * | 2021-03-31 | 2021-07-09 | 上海平安智慧教育科技有限公司 | Online teaching method, system, equipment and storage medium with regional variable resolution |
| CN113486709A (en) * | 2021-05-26 | 2021-10-08 | 南京泛智信息技术有限公司 | Intelligent education platform and method based on virtual reality multi-source deep interaction |
| CN115002343A (en) * | 2022-05-06 | 2022-09-02 | 重庆工程学院 | Method and system for objectively evaluating classroom performance of student based on machine vision |
| CN116110080A (en) * | 2023-04-04 | 2023-05-12 | 成都新希望金融信息有限公司 | Switching method of real facial mask and virtual facial mask |
-
2020
- 2020-01-15 CN CN202010043553.0A patent/CN111242962A/en active Pending
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111757140A (en) * | 2020-07-07 | 2020-10-09 | 南京百家云科技有限公司 | Teaching method and device based on live classroom |
| CN111914760A (en) * | 2020-08-04 | 2020-11-10 | 华中师范大学 | Online course video resource composition analysis method and system |
| CN111914760B (en) * | 2020-08-04 | 2021-03-30 | 华中师范大学 | Online course video resource composition analysis method and system |
| CN111954024A (en) * | 2020-08-27 | 2020-11-17 | 顾建亮 | Course recording live broadcasting method and system |
| CN112040185A (en) * | 2020-08-28 | 2020-12-04 | 深圳市融讯视通科技有限公司 | Method and system for improving remote education courseware sharing |
| CN112351291A (en) * | 2020-09-30 | 2021-02-09 | 深圳点猫科技有限公司 | Teaching interaction method, device and equipment based on AI portrait segmentation |
| CN112232166A (en) * | 2020-10-10 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Artificial intelligence-based lecturer dynamic evaluation method and device, and computer equipment |
| CN112232166B (en) * | 2020-10-10 | 2023-12-01 | 中国平安人寿保险股份有限公司 | Lecturer dynamic evaluation method and device based on artificial intelligence and computer equipment |
| CN112330579A (en) * | 2020-10-30 | 2021-02-05 | 中国平安人寿保险股份有限公司 | Video background replacing method and device, computer equipment and computer readable medium |
| CN112929688A (en) * | 2021-02-09 | 2021-06-08 | 歌尔科技有限公司 | Live video recording method, projector and live video system |
| CN113099254A (en) * | 2021-03-31 | 2021-07-09 | 上海平安智慧教育科技有限公司 | Online teaching method, system, equipment and storage medium with regional variable resolution |
| CN112804516A (en) * | 2021-04-08 | 2021-05-14 | 北京世纪好未来教育科技有限公司 | Video playing method and device, readable storage medium and electronic equipment |
| CN112804516B (en) * | 2021-04-08 | 2021-07-06 | 北京世纪好未来教育科技有限公司 | Video playing method and device, readable storage medium and electronic equipment |
| CN113486709A (en) * | 2021-05-26 | 2021-10-08 | 南京泛智信息技术有限公司 | Intelligent education platform and method based on virtual reality multi-source deep interaction |
| CN115002343A (en) * | 2022-05-06 | 2022-09-02 | 重庆工程学院 | Method and system for objectively evaluating classroom performance of student based on machine vision |
| CN116110080A (en) * | 2023-04-04 | 2023-05-12 | 成都新希望金融信息有限公司 | Switching method of real facial mask and virtual facial mask |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111242962A (en) | Method, device and equipment for generating remote training video and storage medium | |
| CN109271945B (en) | Method and system for realizing job correction on line | |
| CN110334610B (en) | Multi-dimensional classroom quantification system and method based on computer vision | |
| CN107909022B (en) | Video processing method and device, terminal equipment and storage medium | |
| CN111144356B (en) | Teacher sight following method and device for remote teaching | |
| CN110175534A (en) | Teaching assisting system based on multitask concatenated convolutional neural network | |
| CN114638732A (en) | Artificial intelligence intelligent education platform and application thereof | |
| CN111325853B (en) | Remote coaching system and method based on augmented reality glasses | |
| CN110765827A (en) | Teaching quality monitoring system and method | |
| CN112331001A (en) | Teaching system based on virtual reality technology | |
| CN113705510A (en) | Target identification tracking method, device, equipment and storage medium | |
| Pope et al. | The latest in immersive telepresence to support shared engineering education | |
| CN114267213A (en) | Real-time demonstration method, device, equipment and storage medium for practical training | |
| Qianqian et al. | Research on behavior analysis of real-time online teaching for college students based on head gesture recognition | |
| CN117115917A (en) | Teacher behavior recognition method, device and medium based on multi-modal feature fusion | |
| CN112675527A (en) | Family education game system and method based on VR technology | |
| US20230353702A1 (en) | Processing device, system and method for board writing display | |
| Zhao et al. | Implementation of online teaching behavior analysis system | |
| CN116434253A (en) | Image processing method, device, equipment, storage medium and product | |
| Wang et al. | Lecture video enhancement and editing by integrating posture, gesture, and text | |
| TWI726233B (en) | Smart recordable interactive classroom system and operation method thereof | |
| CN113207008A (en) | AR-based tele-immersive simulation classroom and control method thereof | |
| CN112200739A (en) | Video processing method and device, readable storage medium and electronic equipment | |
| Guan et al. | Evaluation of classroom teaching quality based on video processing technology | |
| KR102590652B1 (en) | Method for providing online learning, certificating and issuing management using kit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |