WO2023097684A1

WO2023097684A1 - Virtual laboratory teaching implementation method and system based on 3d simulation, and medium and device

Info

Publication number: WO2023097684A1
Application number: PCT/CN2021/135495
Authority: WO
Inventors: 冯彧; 赵昕; 王�华; 崔红; 吴锐; 王涛; 李其岭
Original assignee: 华特数字科技有限公司
Priority date: 2021-12-03
Filing date: 2021-12-03
Publication date: 2023-06-08

Abstract

Provided are a virtual laboratory teaching implementation method and system based on 3D simulation. A built virtual reality laboratory system is used; a virtualized experiment system and a cloud computing mode are used; a scene virtual model is built by means of collecting articles required by an experimental teaching task; an associated experiment task is established in a virtual reality laboratory scene; and scene simulation is performed by means of login selection of a student, such that learning tracking and real-time feedback for the student and virtual interaction during an experiment process are realized, and the process of each implementation task is monitored.

Description

A method, system, medium and equipment for realizing virtual laboratory teaching based on 3D simulation

technical field

The invention belongs to the field of virtual reality simulation, in particular to a method for realizing virtual laboratory teaching based on 3D simulation.

Background technique

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

With the continuous development of science and technology, the level of education and teaching has been continuously improved, and the quality of education is becoming more and more important. At the same time, the needs of students in teaching methods and teaching content are also increasing, and the way of virtual experiments has gradually become a kind of A new way of learning, or use the mouse to click and drag to assemble various instruments or meters on the microcomputer into a complete experimental system according to the experimental requirements and process. The virtual laboratory is the carrier of virtual experiment operation and is a virtual experiment It is an experimental platform that can work collaboratively in a network environment, realize virtual experiments and design, and fully realize resource sharing.

However, the current Internet-based virtual distance teaching industry has great disadvantages, that is, the teaching environment is relatively single, some teaching links that require special equipment cannot be carried out, and students cannot fully understand it just by watching teachers demonstrate, and cannot carry out immersive learning and Experience, teachers and students do not have a more realistic and diverse teaching environment, unable to realize the leap from mastering to application of knowledge, lacking an experimental teaching system and method based on virtual reality technology, and on the premise that students do not carry out practical experiments, There is no specific perception of the specific experimental process, the lack of specific interaction in the experimental process, and the inability to simulate the experimental scene and experimental objects, resulting in low learning efficiency and the disadvantages of reducing students' interest in experimental teaching. Second, for some dangerous and impossible experiments, the implementation of the course cannot be carried out, which affects the learning process of students. Third, the teaching management of the laboratory has not yet been fully informatized, and the usual tracking of students' records cannot be effectively carried out. However, the roll call of classroom learning cannot be fed back in real time, and real-time headcount, classroom quizzes, and monitoring cannot be performed.

Contents of the invention

In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method for implementing virtual laboratory teaching based on 3D simulation, which aims to realize For students' learning tracking, real-time feedback and virtual interaction in the experiment process, monitor the progress of each implementation task.

In order to achieve the above object, the present invention adopts the following technical solutions:

The first aspect of the present invention provides a method for implementing virtual laboratory teaching based on 3D simulation, which is characterized in that it includes:

Step (1): Collect and record the experimental items needed in the laboratory teaching, use all the collected experimental items to build an item library, collect all the laboratory student information, and establish an information database.

Step (2): Use the camera to collect the student's face image, and display the collected student's face image;

Step (3): Segment all faces from the displayed student face image, compare the segmented faces with the data in the information database in step (1), and identify the student identity information;

Step (4): According to the results of the recognized face image, student identity information and item information in the item library, create the required model and establish a virtual reality laboratory scene system;

Step (5): Establish tasks associated with the established virtual reality laboratory scene to realize laboratory teaching, learning and testing.

Further, in the established virtual reality laboratory scene, the student's log-in identity is recorded according to the student's identity information, and the recorded student's log-in identity is uploaded to the data processing center for classified storage.

Specifically, students use the stored log-in information to enter the virtual reality laboratory scene, select associated tasks, start the learning mode for experimental learning and laboratory teaching and testing.

Further, the laboratory student information includes student information, laboratory courses corresponding to each student, and experimental course tasks corresponding to the students, and the information corresponding to the above students is uploaded to the data processing center, and the data processing center can Communicate with the created virtual reality lab system that can retrieve all lab student information.

Further, the task of establishing the association is specifically:

Collect all the experimental teaching tasks required by the virtual reality laboratory, create the required experimental teaching tasks into the virtual reality laboratory, associate the created laboratory teaching tasks with all laboratory student information, and When each laboratory teaching task is started, it can track and monitor the students who should be studying under the task, monitor whether the student is studying under the task, and give feedback on the learning situation of the students, and upload the feedback information to Data processing center, and the beating display of student avatars.

Further, the specific virtual reality experimental teaching methods are as follows:

Start a laboratory teaching task associated with it, and time the start time of the experiment according to the course information in the student information. Students need to log in to the associated teaching task in the virtual display laboratory system before this time. , if all students have entered, the experimental teaching task will start, if there are students who have not entered the teaching task of the virtual reality laboratory system before this time, then count all the students who have not entered the student information and upload it to the data processing center, and the feedback display reminds the administrator to urge offline.

Further, after the laboratory teaching task is started, the virtual reality laboratory performs the experimental teaching task, calls the experimental item model used in the experimental teaching task according to the established model, and transfers the experimental item models according to the experiment itself. Processes are associated to realize the entire process of a certain experimental teaching and complete the virtual reality experimental teaching task.

Further, the specific face segmentation and comparison process includes:

The face data is collected by the camera, and the image of the face is segmented by image segmentation, the key point information with the segmented face area in the image is calculated and extracted, and the face to be segmented is delineated according to the key point information In the edge area, determine the semantic prior layer information corresponding to the face area to be segmented, and input the semantic prior layer information and the face image into the pre-trained network model to obtain the segmentation result;

The process of comparing is: comparing the segmented face with the identity information in the student information in the database to determine the identity of the student.

The second aspect of the present invention provides a system for realizing virtual laboratory teaching based on 3D simulation, which is characterized in that it includes:

The data processing center realizes the storage and processing of all information, which includes PCs to display student information and feedback information;

Camera to realize the collection of face images;

The image face segmentation module realizes the segmentation and extraction of the face from the face image;

The calculation module realizes the calculation and extraction of the key point information with the segmented face area in the image.

A third aspect of the present invention provides a computer readable storage medium.

A kind of computer-readable storage medium, is stored with computer program on it, is characterized in that, when this program is executed by processor, realizes the step in the described a kind of teaching implementation method based on 3D simulation virtual laboratory.

A fourth aspect of the present invention provides a computer device.

A computer device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements a virtual laboratory teaching based on 3D simulation when executing the program steps in the method.

Compared with prior art, the beneficial effect of the present invention is:

Using cloud computing technology and virtual reality simulation to integrate system resources, it is convenient to realize online testing for students and generate experimental trajectories. At the same time, teachers can also evaluate the experimental results in real time. Students can achieve a thorough understanding of experimental tasks and perform immersive Learning and experience, teachers and students have a relatively real and diverse teaching environment, and realize the leap from mastering to applying knowledge.

It can realize the specific perception of the experimental process, realize the interactivity in the experimental process, simulate the experimental scene and experimental objects, and improve the learning efficiency and students' interest in experimental teaching.

It can realize the implementation of some dangerous and impossible experimental courses, speed up the learning process of students; realize the informatization of teaching management in the laboratory, and effectively carry out the students' usual tracking of student records, and the roll call of classroom learning in real time Feedback, real-time headcounts, classroom quizzes, and monitoring.

Advantages of additional aspects of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention.

Fig. 1 is the flow chart that the laboratory teaching method of the present invention realizes;

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It should be noted that the terminology used here is only for describing specific embodiments, and is not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and/or combinations thereof.

It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in a flowchart or a block diagram may represent a module, a program segment, or a part of a code, and the module, a program segment, or a part of a code may include one or more An executable instruction for a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, can be implemented using a dedicated hardware-based system that performs the specified functions or operations , or can be implemented using a combination of dedicated hardware and computer instructions.

Embodiment one

As shown in Figure 1, this embodiment provides a method for implementing virtual laboratory teaching based on 3D simulation. In this embodiment, the method is applied to a server for illustration. It can be understood that this method can also be applied to a terminal. It can be applied to a terminal, a server and a system, and is realized through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, intermediate Cloud servers for basic cloud computing services such as software services, domain name services, security service CDN, and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be connected directly or indirectly through wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

A method for realizing virtual laboratory teaching based on 3D simulation, comprising:

Specifically, in each step,

Specifically, the collection of experimental items required in laboratory teaching mainly refers to the items used in a certain simulation experiment project, which may be chemical reagents and various models in physical experiments. Collect and build an item library of all items, which are used to build item models in the virtual reality laboratory, and are used to create the entire laboratory virtual reality scene.

In addition, collecting all laboratory student information mainly refers to the student information of students who need to log in to study in a certain experimental teaching task, and store it, which can be used for identity verification, tracking student learning process, and learning Manage the students, including the courses of the students' learning experiments, the class time, and the corresponding curriculum, which are used to realize the statistics of the students' class realization and track and monitor the students' learning process.

Specifically, in the established virtual reality laboratory scene, the student's log-in identity is recorded according to the student's identity information, and the recorded student's log-in identity is uploaded to the data processing center for classified storage.

This role is to obtain more realistic student information, so as to maximize the mapping of real students' learning in the laboratory, that is, to construct virtual characters and scenes that are closest to students and laboratory tasks in the virtual environment.

The specific face segmentation and comparison process includes:

Specifically, obtain the face image including the face area to be segmented; extract the key point information of the face area to be segmented; Segment the semantic prior layer information corresponding to the face area; wherein, the semantic prior layer information represents the restricted segmentation area corresponding to each segmentation object in the face area to be segmented; the semantic prior layer information and the face image are input to the pre- The trained network model obtains the segmentation results corresponding to each segmentation object.

In the process of face segmentation in this embodiment, while inputting the face image including the face area to be segmented into the network model, the semantic prior layer information representing the edge of the face area to be segmented is also input, and the semantic prior layer information is passed through the semantic prior layer The information can limit the edge of the segmented face area, avoid wrong segmentation, and improve the accuracy of segmentation.

The virtual reality laboratory scene includes the completed virtual scene, including the items needed for the experiment, the experimental process, and the function of the virtual reality scene module is to select the item model for the experiment in the laboratory virtual item library. In the student information Select the characters in the system to construct the scene required by the system. This scene is a complete scene, including laboratories, objects and virtual students.

The test includes the managers who conduct the test. The managers can use their own real data to map to the virtual reality laboratory scene to use their own real information, or use the roles assigned by the system. Managers connect to the system through virtual reality devices, including virtual reality headsets, bipedal motion controllers and other virtual reality devices, so that testers can control their characters in the virtual reality world and operate freely in the virtual reality scene. And interact with everything in the scene in virtual reality. Then managers can conduct tests according to the experimental plan prepared in advance. There can be many or few managers. Due to the limitation of the virtual reality laboratory scene, managers can choose only the current test personnel in the whole scenario. In the case of many testers, you can choose to add student avatars to the system, and let the system simulate and control other virtual people in the scene, or without the participation of testers, the virtual scene only includes virtual students controlled by the system for immersive experimental experience.

Specifically, students use the stored log-on information to enter the virtual reality laboratory scene, select associated tasks, start the learning mode for experimental learning and laboratory teaching and testing.

Specifically, the laboratory student information includes student information, laboratory courses corresponding to each student, and experimental course tasks corresponding to the students, and the information corresponding to the above students is uploaded to the data processing center, and the data processing center can Communicate with the created virtual reality lab system that can retrieve all lab student information.

In addition, the task of establishing the association is specifically:

The specific virtual reality experimental teaching methods are as follows:

In addition, after the laboratory teaching task is started, the virtual reality laboratory performs the experimental teaching task, calls the experimental item model used in the experimental teaching task according to the established model, and transfers the experimental item models according to the flow of the experiment itself. Associated, so as to realize the entire process of a certain experimental teaching, and complete the experimental teaching task of the virtual reality.

Specifically, when an error occurs in the task execution selection established by the association, the task will be removed from the task queue without any processing. When the task ends in this cycle and enters the next cycle, it will re-enter the task queue in the scanning phase. To prevent the infinite loop of error tasks, the same task will not enter the task queue when the same error type reaches 3 times, and it will only enter the task queue after the administrator executes and fixes this type of error. During this period, the individual operations on the teacher's monitoring page will not be affected. In order to avoid entering the "fake death" state due to unexpected factors during execution, every time a new script is triggered by a scheduled task, it will first check whether the task being executed has timed out, and the timeout is limited to 10 minutes. If the task times out, it will be executed Resets the task queue operation; if the task has not timed out, the completion check does nothing.

Embodiment two

This embodiment provides a system for realizing teaching in a virtual laboratory based on 3D simulation, including:

Camera to realize the collection of face images;

Specifically, using the above-mentioned system to implement a method for implementing virtual laboratory teaching based on 3D simulation, including:

Embodiment Three

This embodiment provides a computer-readable storage medium and a computer device, on which a computer program is stored, which is characterized in that, when the program is executed by a processor, a 3D simulation-based Steps in the implementation of virtual laboratory teaching.

A computer device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the processor implements one of the above-mentioned embodiment 1 when executing the program. Based on the 3D simulation virtual laboratory teaching to realize the steps in the method.

The beneficial effect of the present invention is that, by using cloud computing technology and virtual reality simulation, system resources are integrated, online testing of students can be easily realized and experimental tracks can be generated, and at the same time, teachers can also evaluate the experimental results in real time, and students can realize the experimental tasks Through immersion learning and experience, teachers and students have a more realistic and diverse teaching environment to realize the leap from mastering knowledge to applying knowledge.

It should be noted here that the above-mentioned acquisition and preprocessing module, input data set building module, output data set building module, model training module and inversion module correspond to steps S101 to S105 in Embodiment 1, and the above-mentioned modules correspond to the corresponding The examples and application scenarios implemented by the steps are the same, but are not limited to the content disclosed in the first embodiment above. It should be noted that, as a part of the system, the above-mentioned modules can be executed in a computer system such as a set of computer-executable instructions.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the programs can be stored in a computer-readable storage medium. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random AccessMemory, RAM), etc.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

A method for realizing virtual laboratory teaching based on 3D simulation, characterized in that it includes:

Step (1): Collect and record the experimental items needed in the laboratory teaching, use all the collected experimental items to establish an item library, collect all the information of laboratory students, and establish an information database;

Step (2): Use the camera to collect the student's face image, and display the collected student's face image;

Step (3): Segment all faces from the displayed student face image, compare the segmented faces with the data in the information database in step (1), and identify the student identity information;

Step (4): According to the results of the recognized face image, student identity information and item information in the item library, create the required model and establish a virtual reality laboratory scene system;

Step (5): Establish tasks associated with the established virtual reality laboratory scene to realize laboratory teaching, learning and testing.
A method for implementing virtual laboratory teaching based on 3D simulation according to claim 1, wherein, specifically, in the established virtual reality laboratory scene, record the student's log-in identity according to the student's identity information, and The recorded students can log in and upload to the data processing center for classified storage;

Preferably, students use the stored log-in information to enter the virtual reality laboratory scene, select associated tasks, and start the learning mode for experimental learning and laboratory teaching tests.
A method for realizing virtual laboratory teaching based on 3D simulation as claimed in claim 1, wherein said laboratory student information includes student information, laboratory courses corresponding to each student, and experimental course tasks corresponding to students , and upload the information corresponding to the above students to the data processing center, the data processing center can communicate with the created virtual reality laboratory system, and the virtual reality laboratory system can retrieve all laboratory student information.
A method for realizing virtual laboratory teaching based on 3D simulation as claimed in claim 1, wherein the task of establishing the association is specifically:

Collect all the experimental teaching tasks required by the virtual reality laboratory, create the required experimental teaching tasks into the virtual reality laboratory, associate the created laboratory teaching tasks with all laboratory student information, and When each laboratory teaching task is started, it can track and monitor the students who should be studying under the task, monitor whether the student is studying under the task, and give feedback on the learning situation of the students, and upload the feedback information to Data processing center, and the beating display of student avatars.
A method for realizing virtual laboratory teaching based on 3D simulation as claimed in claim 1, wherein the specific method of virtual reality experimental teaching is:

Start a laboratory teaching task associated with it, and time the start time of the experiment according to the course information in the student information. Students need to log in to the associated teaching task in the virtual display laboratory system before this time. , if all the students have entered, the experimental teaching task will start, if there are students who have not entered the teaching task of the virtual reality laboratory system before this time, then count all the students who have not entered the student information and upload it to the data processing center, and the feedback display reminds the administrator to urge offline.
A method for realizing teaching in a virtual laboratory based on 3D simulation according to claim 5, wherein, after starting the laboratory teaching task, the virtual reality laboratory performs the experimental teaching task, and calls the The experimental item models used in the experimental teaching tasks are associated with the experimental item models according to the process of the experiment itself, so as to realize the entire process of a certain experimental teaching and complete the virtual reality experimental teaching task.
A kind of teaching realization method based on 3D simulation virtual laboratory as claimed in claim 1, is characterized in that, concrete face segmentation and comparison process comprise:

The face data is collected by the camera, and the image of the face is segmented by image segmentation, the key point information with the segmented face area in the image is calculated and extracted, and the face to be segmented is delineated according to the key point information In the edge area, determine the semantic prior layer information corresponding to the face area to be segmented, and input the semantic prior layer information and the face image into the pre-trained network model to obtain the segmentation result;

The process of comparing is: comparing the segmented face with the identity information in the student information in the database to determine the identity of the student.
A teaching implementation system based on 3D simulation virtual laboratory, characterized in that it includes:

The data processing center realizes the storage and processing of all information, which includes PCs to display student information and feedback information;

Camera to realize the collection of face images;

The image face segmentation module realizes the segmentation and extraction of the face from the face image;

The calculation module realizes the calculation and extraction of the key point information with the segmented face area in the image.
A computer-readable storage medium, on which a computer program is stored, characterized in that, when the program is executed by a processor, it realizes a kind of teaching realization based on 3D simulation virtual laboratory as described in any one of claims 1-7 steps in the method.
A computer device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements any one of claims 1-7 when executing the program The steps in the implementation method of a virtual laboratory teaching based on 3D simulation.