CN111311995A - Remote teaching system and teaching method based on augmented reality technology - Google Patents
Remote teaching system and teaching method based on augmented reality technology Download PDFInfo
- Publication number
- CN111311995A CN111311995A CN202010205746.1A CN202010205746A CN111311995A CN 111311995 A CN111311995 A CN 111311995A CN 202010205746 A CN202010205746 A CN 202010205746A CN 111311995 A CN111311995 A CN 111311995A
- Authority
- CN
- China
- Prior art keywords
- student
- teaching
- teacher
- eye movement
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 69
- 238000005516 engineering process Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 33
- 239000011521 glass Substances 0.000 claims abstract description 47
- 230000005540 biological transmission Effects 0.000 claims abstract description 24
- 230000003993 interaction Effects 0.000 claims abstract description 15
- 230000004424 eye movement Effects 0.000 claims description 101
- 238000011156 evaluation Methods 0.000 claims description 84
- 238000004458 analytical method Methods 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 20
- 230000009471 action Effects 0.000 claims description 13
- 238000007726 management method Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 12
- 210000003128 head Anatomy 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 210000001747 pupil Anatomy 0.000 claims description 6
- 230000001133 acceleration Effects 0.000 claims description 4
- 230000004886 head movement Effects 0.000 claims description 2
- 230000033001 locomotion Effects 0.000 claims 2
- 230000002452 interceptive effect Effects 0.000 abstract description 4
- 230000009466 transformation Effects 0.000 description 8
- 230000007547 defect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 241000711573 Coronaviridae Species 0.000 description 1
- 206010035664 Pneumonia Diseases 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 208000002173 dizziness Diseases 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
- G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- General Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a remote teaching system and a teaching method based on an augmented reality technology, which belong to the technical field of digital teaching, wherein the remote teaching system based on the augmented reality technology at least comprises a remote teaching platform; the system comprises a teacher department, a student department and a network transmission part for realizing data interaction between a first data acquisition terminal and a second data acquisition terminal; the teacher section includes: the system comprises a first data acquisition terminal for acquiring teaching situations of teachers and a first display terminal for receiving and displaying student data; this student portion includes: the second display terminal is used for receiving and displaying the teacher data, and the second data acquisition terminal is used for acquiring the student attendance situation; the method is characterized in that: the second display terminal is augmented reality glasses. By adopting the technical scheme, the invention utilizes the augmented reality technology to simulate classroom teaching, and can realize that students can carry out immersive learning at home. The teacher can carry out real-time interdynamic with the student, improves interactive experience.
Description
Technical Field
The invention belongs to the technical field of digital teaching, and particularly relates to a remote teaching system and a teaching method based on an augmented reality technology.
Background
As is well known, the traditional classroom teaching mode has space limitation and time limitation objectively, that is, students and teachers must arrive at a specific place at a specific time to carry out teaching work, in recent years, along with the rapid development of the internet of things technology, in order to overcome the above defects of the traditional classroom teaching mode, the online teaching system comes up, and at present, the existing online teaching system mainly comprises three parts: two information recording parts for obtaining audio and video information of teacher or student and a network transmission part for transmitting data. Under the condition of 'stopping a lesson and not stopping a study' during the period of novel coronavirus pneumonia epidemic situation prevention and control and delayed start of study of the department of education, network teaching is used in a large number, and the defects of the existing network teaching system exist in the using process:
firstly, the learning experience of students is poor;
the teacher department usually uses the tools such as a mobile phone, a notebook computer, a writing pad and the like to carry out live broadcasting teaching, and has the defects of flat picture quality, low definition, small shooting and recording visual angle, poor writing effect and the like; if the Virtual Reality (VR) remote teaching is adopted, students cannot meet the requirement of watching videos and recording notes; if the device is in a pure Virtual Reality (VR) environment for a long time, discomfort such as dizziness can be caused.
Secondly, the participation degree of students is low;
the student part generally uses a mobile phone, a tablet personal computer and a computer for learning, so that the defects of small visual field, flat picture and lack of immersion feeling of the picture exist, the picture can only be interacted by clicking a mouse or a touch screen, and the students are difficult to obtain the participation feeling in the real topic environment.
Thirdly, effective management and control are difficult;
even teacher's portion gathers student's facial image through student's portion camera, nevertheless under the condition that tens or even hundreds of students were in class simultaneously, be difficult to judge student's study state on the tens student's head portrait that reduces from teacher's portion management platform, be difficult to carry out timely, effectual management and control.
Fourthly, the attention of the students is hard to concentrate; since the learning place of the student is not in the classroom, for example, the student is at home, and at this time, the family may be watching tv or doing something else, the attention of the student is likely to be distracted, which greatly reduces the quality and effect of teaching.
Disclosure of Invention
The invention provides a remote teaching system and a teaching method based on an augmented reality technology for solving the technical problems in the known technology, and the teaching system and the teaching method can simulate classroom teaching by the augmented reality technology and can realize that students can carry out immersive learning at home. The teacher can carry out real-time interdynamic with the student, improves interactive experience. Meanwhile, the attention of the students is detected in real time by using an eye movement tracking technology, the conditions of attention transfer and the like of the students are found in real time, and effective attention management and control are realized. By mining eye movement data of students in a certain period, the quality evaluation of courses and the psychological and hobby evaluation of the students can be carried out.
A first object of the present invention is to provide a remote teaching system based on augmented reality technology, at least comprising;
a teacher department; the teacher section includes: the system comprises a first data acquisition terminal for acquiring teaching situations of teachers and a first display terminal for receiving and displaying student data;
a student department; this student portion includes: the second display terminal is used for receiving and displaying the teacher data, and the second data acquisition terminal is used for acquiring the student attendance situation;
the network transmission part is used for realizing data interaction between the first data acquisition terminal and the second data acquisition terminal;
the second display terminal is augmented reality glasses.
And (5) recording the video in real time by one key. The student end is provided with a teaching video one-key recording module, and when the student has storage and review requirements on the teaching video, the student can record the teaching video one-key. When a student clicks to record, in order to record a relatively complete teaching video, the recording module sets the video recording starting point to one minute ahead, so that the video content before missing is avoided, and relatively complete video recording is realized. The cloud server provides a certain capacity of video storage space for students to store the recorded interesting videos, and the searching and playback are convenient.
Further, the second data acquisition terminal comprises a three-axis acceleration sensor and a three-axis gyroscope sensor which are installed on the augmented reality glasses, and the user head action data processing module based on the three-axis acceleration sensor and the three-axis gyroscope sensor; the head action data processing module measures the head action of a user in real time and adjusts the display position of a teaching video; when the user heads down to write and note, the video does not appear in the visual field range of the augmented reality glasses; when the user is looking straight ahead, the video appears in the field of view of the augmented reality glasses; when the user takes the head shaking or nodding posture, the head movement data processing module can upload data to the teacher management platform.
Furthermore, the second data acquisition terminal comprises a camera installed on the augmented reality glasses, and the student hands-up action recognition module based on the camera; the student hand-lifting action recognition module recognizes the student hand-lifting action according to the preset action.
Furthermore, the second data acquisition terminal comprises an eye movement signal sensor installed on the augmented reality glasses, and the remote teaching system based on the augmented reality technology further comprises an eye movement signal analysis module; the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving the concentration evaluation value Pi and forwarding the concentration evaluation value Pi to the teacher department.
Furthermore, the first data acquisition terminal comprises a 3D panoramic camera or a 4K high-definition pan-tilt camera, and the first display terminal comprises a notebook computer or a desktop computer.
Furthermore, the remote teaching system based on the augmented reality technology further comprises a data storage part and a teaching management platform.
Furthermore, the eye movement signal analysis module comprises a concentration evaluation value analysis module for solving the concentration evaluation values Pi of each subject through the pupil diameter and the eye movement state.
The second purpose of the present invention is to provide a teaching method of a remote teaching system based on the augmented reality technology, which includes the following steps:
s101, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s102, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part;
and S103, the student part outputs the received teaching data through the augmented reality glasses for the students to study.
The third purpose of the invention is to provide a teaching method of a remote teaching system based on augmented reality technology, which comprises the following steps:
s201, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s202, a teacher part collects teaching data of teaching situations of a teacher and sends the teaching data to a student part through a network transmission part;
s203, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s204, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s205, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
and S206, the teacher part displays the attention evaluation value Pi through the first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value Pi of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part.
Further, the processing and analyzing process in S205 is:
let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipAnd judging that the student attention evaluation value does not meet the model requirement.
The fourth purpose of the invention is to provide a teaching method of a remote teaching system based on augmented reality technology, which comprises the following steps:
s301, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s302, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part;
s303, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s304, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s305, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
s306, the teacher part displays the attention evaluation value Pi through a first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part;
and S307, the eye movement signal analysis module solves concentration evaluation value Pi curves of all the subjects through the pupil diameter and the eye movement state, and the teacher adjusts the later stage teaching plan through the concentration evaluation value Pi curves.
Further, the processing and analyzing process in S305 is:
let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipAnd judging that the student attention evaluation value does not meet the model requirement.
The invention has the advantages and positive effects that:
firstly, the learning experience of students is good;
the teacher adopts a 3D panoramic camera or a 4K high-definition pan-tilt camera to shoot, so that pictures of a classroom blackboard and a teacher can be shot clearly and completely, a complete and high-definition teaching video is obtained, and teaching information can be transmitted to the student completely and infallibly; and meanwhile, the teaching video is watched, and meanwhile, notes can be synchronously handwritten and recorded.
Secondly, the participation degree of students is high;
student portion adopts augmented reality glasses, and the student can obtain high definition, depth of field or panorama teaching video, has to immerse sense and participate in the sense, combines the action of raising one's hands recognition module of augmented reality glasses simultaneously, can simulate real classroom scene and teacher portion and carry out operations such as the interaction of raising one's hands.
Thirdly, effective management and control are facilitated;
through carrying on eye movement sensor on augmented reality glasses, obtain the eye movement information of student study teaching video in-process, but the effective analysis goes out student's the condition of being absorbed in effort, realizes teacher's real-time feedback and warning.
Fourthly, the attention of the students is concentrated; because the augmented reality technology is adopted, the students can well enter the immersive teaching process, the influence of the external environment is abandoned, the students can conveniently put into teaching work all over the body, the mechanical learning teaching content is intensively noticed, and the quality and the effect of the teaching are greatly improved.
Drawings
FIG. 1 is a schematic diagram of a preferred embodiment of the present invention;
FIG. 2 is a block diagram of the architecture of the preferred embodiment of the present invention;
FIG. 3 is a block diagram of an eye movement signal analysis module according to the present invention;
fig. 4 is a graph of concentration evaluation values Pi in the present invention.
Detailed Description
In order to further understand the contents, features and effects of the present invention, the following embodiments are illustrated and described in detail with reference to the accompanying drawings:
referring to fig. 1 to 4, the first preferred embodiment: a remote teaching system based on augmented reality technology comprises;
a teacher department 1; the teacher section includes: the system comprises a first data acquisition terminal for acquiring teaching situations of teachers and a first display terminal for receiving and displaying student data; preferably, the method comprises the following steps: the first data acquisition terminal selects a 4K high-definition pan-tilt camera with a mature technology, and the first display terminal selects a notebook computer or a desktop computer with a mature technology.
A student section 2; this student portion includes: the second display terminal is used for receiving and displaying the teacher data, and the second data acquisition terminal is used for acquiring the student attendance situation; in order to realize the immersive teaching mode, the second display terminal is augmented reality glasses.
And a network transmission part 3 for realizing data interaction between the first data acquisition terminal and the second data acquisition terminal;
in order to facilitate the storage and playback of historical data, a teaching video one-key recording module is additionally arranged in the student part, when a student needs to store and review teaching videos, the teaching video one-key recording module can be started through a recording trigger key, when the student clicks and records, in order to record a complete teaching video, the recording module sets a video recording starting point to be one minute before, the video content before the mistake is avoided, and the complete video recording is realized. The cloud server provides a certain capacity of video storage space for students to store the recorded interesting videos, and the searching and playback are convenient.
The teaching method of the remote teaching system of the augmented reality technology comprises the following steps:
s101, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s102, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part; the teaching data mainly comprises video information and audio information;
and S103, the student part outputs the received teaching data through the augmented reality glasses for the students to study.
In order to realize the monitoring of the learning process of the students and further carry out quantitative assessment on the attention of the students, on the basis of the above preferred embodiment,
second preferred embodiment: the second data acquisition terminal comprises an eye movement signal sensor arranged on augmented reality glasses, and the remote teaching system based on the augmented reality technology further comprises an eye movement signal analysis module; the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving the concentration evaluation value Pi and forwarding the concentration evaluation value Pi to the teacher department.
The teaching method of the remote teaching system based on the augmented reality technology in the preferred embodiment comprises the following steps:
s201, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s202, a teacher part collects teaching data of teaching situations of a teacher and sends the teaching data to a student part through a network transmission part;
s203, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s204, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s205, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
and S206, the teacher part displays the attention evaluation value Pi through the first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value Pi of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part.
Preferably, on the basis of the above preferred embodiment, the remote teaching system based on augmented reality technology further includes a data storage unit and a teaching management platform.
The teaching management platform comprises attendance, examination and absence prompts (for guardians) and interactive question answering. In addition, the student concentration evaluation value Pi returned after the eye movement data analysis is judged, so that the functions of automatic judgment, effective real-time judgment and the like of the teacher on whether the student attention and the attention point move along with the teaching content of the teacher are realized;
the eye movement sensor is carried on the augmented reality glasses of the student end, the direct relation between the eye state and the display content of the equipment is observed through the sensor for observing the glasses in the augmented reality or mixed reality equipment (and an effective corresponding coordinate system is established),
the specific process comprises the following steps: the position coordinate obtained by the eye movement sensor is a two-dimensional coordinate system, and the visual field range of the augmented reality glasses can also be regarded as a two-dimensional coordinate system. Since the direction of the eye movement sensor is opposite to the direction of the visual field of the glasses, and the oblique cutting angle exists, the coordinates obtained by the eye movement sensor are required to be related to the coordinates of the visual field of the glasses by designing an affine transformation algorithm. The correlation of two independent coordinates, at first take the minimum external rectangle of the glasses field of vision, take glasses left eye and left eye movement sensor as an example, the lower left corner and the lower right corner of the rectangle are respectively set as the origin of the glasses field of vision, and the origin of the eye movement sensor, therefore, the reflection transformation is firstly carried out on the eye movement sensor coordinates (x, y):
because there is a bevel angle, the resulting coordinates are again scaled and beveled:
the (x2, y2) is the coordinate transformation of the eye movement sensor on the visual field of the glasses, and the variable of the coefficient a1-a6 can be obtained according to the installation angle of the eye movement sensor and the coordinate calibration.
After the coordinate transformation, a time-series-based track of the fixation point of the student on the video image is obtained, for example, a probability Pi (an embodiment of a preset eye movement state model Mi) of whether the near 5 minutes exceeds the effective range of the remote teaching video of the teacher is analyzed, and a graph is shown, so that the state of the student is judged on the basis, then a state signal is transmitted to the teacher end in real time, and the student is prompted (by sound and light) to wake up in a manual or automatic mode.
Let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipAnd judging that the student attention evaluation value does not meet the model requirement.
Third preferred embodiment: a remote teaching system based on augmented reality technology comprises;
a teacher department 1; the teacher section includes: the system comprises a first data acquisition terminal for acquiring teaching situations of teachers and a first display terminal for receiving and displaying student data; preferably, the method comprises the following steps: the first data acquisition terminal selects a 4K high-definition pan-tilt camera with a mature technology, and the first display terminal selects a notebook computer or a desktop computer with a mature technology.
A student section 2; this student portion includes: the second display terminal is used for receiving and displaying the teacher data, and the second data acquisition terminal is used for acquiring the student attendance situation; in order to realize the immersive teaching mode, the second display terminal is augmented reality glasses.
And a network transmission part 3 for realizing data interaction between the first data acquisition terminal and the second data acquisition terminal;
the second data acquisition terminal comprises an eye movement signal sensor arranged on augmented reality glasses, and the remote teaching system based on the augmented reality technology further comprises an eye movement signal analysis module 4; the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving the concentration evaluation value Pi and forwarding the concentration evaluation value Pi to the teacher department.
The eye movement signal analysis module comprises a concentration evaluation value analysis module, and concentration evaluation values Pi of all disciplines are solved through pupil diameters and eye movement states.
The teaching method of the third preferred embodiment comprises the following steps:
s301, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s302, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part;
s303, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s304, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s305, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
s306, the teacher part displays the attention evaluation value Pi through a first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part;
and S307, the eye movement signal analysis module solves concentration evaluation value Pi curves of all the subjects through the pupil diameter and the eye movement state, and the teacher adjusts the later stage teaching plan through the concentration evaluation value Pi curves.
Preferably, on the basis of the above preferred embodiment, the remote teaching system based on augmented reality technology further includes a data storage unit and a teaching management platform.
The teaching management platform comprises attendance, examination and absence prompts (for guardians) and interactive question answering. In addition, the student concentration evaluation value Pi returned after the eye movement data analysis is judged, so that the functions of automatic judgment, effective real-time judgment and the like of the teacher on whether the student attention and the attention point move along with the teaching content of the teacher are realized;
the eye movement sensor is carried on the augmented reality glasses of the student end, the direct relation between the eye state and the display content of the equipment is observed through the sensor for observing the glasses in the augmented reality or mixed reality equipment (and an effective corresponding coordinate system is established),
the specific process comprises the following steps: the position coordinate obtained by the eye movement sensor is a two-dimensional coordinate system, and the visual field range of the augmented reality glasses can also be regarded as a two-dimensional coordinate system. Since the direction of the eye movement sensor is opposite to the direction of the visual field of the glasses, and the oblique cutting angle exists, the coordinates obtained by the eye movement sensor are required to be related to the coordinates of the visual field of the glasses by designing an affine transformation algorithm. The correlation of two independent coordinates, at first take the minimum external rectangle of the glasses field of vision, take glasses left eye and left eye movement sensor as an example, the lower left corner and the lower right corner of the rectangle are respectively set as the origin of the glasses field of vision, and the origin of the eye movement sensor, therefore, the reflection transformation is firstly carried out on the eye movement sensor coordinates (x, y):
because there is a bevel angle, the resulting coordinates are again scaled and beveled:
the (x2, y2) is the coordinate transformation of the eye movement sensor on the visual field of the glasses, and the variable of the coefficient a1-a6 can be obtained according to the installation angle of the eye movement sensor and the coordinate calibration.
After the coordinate transformation, a time-series-based track of the fixation point of the student on the video image is obtained, for example, a probability Pi (an embodiment of a preset eye movement state model Mi) of whether the near 5 minutes exceeds the effective range of the remote teaching video of the teacher is analyzed, and a graph is shown, so that the state of the student is judged on the basis, then a state signal is transmitted to the teacher end in real time, and the student is prompted (by sound and light) to wake up in a manual or automatic mode.
Let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipAnd judging that the student attention evaluation value does not meet the model requirement.
The Pi values within a certain time length are averaged, and can be fed back and distinguished to a certain extent, such as functional application with different interest points of the Wenck thought, the rational thought and the like.
The first method is as follows: under the condition of not considering the weight coefficient, solving the Pi average value in a certain time length by adopting a simple average value method:
the second method comprises the following steps: because the concentration differences of the students in the subjects at the initial stage of the course are not obvious, the concentration differences gradually amplify the trend along with the progress of the course, and under the condition of considering the weight coefficient, a weighted average method is adopted to solve the Pi average value within a certain time length:
The above-mentioned embodiments are only for illustrating the technical ideas and features of the present invention, and the purpose thereof is to enable those skilled in the art to understand the contents of the present invention and to carry out the same, and the present invention shall not be limited to the embodiments, i.e. the equivalent changes or modifications made within the spirit of the present invention shall fall within the scope of the present invention.
Claims (10)
1. A remote teaching system based on augmented reality technology; at least comprises the following components;
a teacher department; the teacher section includes: the system comprises a first data acquisition terminal for acquiring teaching situations of teachers and a first display terminal for receiving and displaying student data;
a student department; this student portion includes: the second display terminal is used for receiving and displaying the teacher data, and the second data acquisition terminal is used for acquiring the student attendance situation;
the network transmission part is used for realizing data interaction between the first data acquisition terminal and the second data acquisition terminal; the method is characterized in that:
the second display terminal is augmented reality glasses.
2. The augmented reality technology-based distance teaching system according to claim 1, wherein the second data acquisition terminal includes an eye movement signal sensor mounted on augmented reality glasses, the augmented reality technology-based distance teaching system further includes an eye movement signal analysis module; the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving the concentration evaluation value Pi and forwarding the concentration evaluation value Pi to the teacher department.
3. The augmented reality technology-based distance teaching system of claim 1, wherein the second data acquisition terminal comprises a three-axis acceleration sensor and a three-axis gyroscope sensor mounted on augmented reality glasses, and a user head motion data processing module based on the three-axis acceleration sensor and the three-axis gyroscope sensor; the head action data processing module measures the head action of a user in real time and adjusts the display position of a teaching video; when the user heads down to write and note, the video does not appear in the visual field range of the augmented reality glasses; when the user is looking straight ahead, the video appears in the field of view of the augmented reality glasses; when the user takes the head shaking or nodding posture, the head movement data processing module can upload data to the teacher management platform.
4. The augmented reality technology-based distance teaching system of claim 1 wherein the second data acquisition terminal comprises a camera mounted on augmented reality glasses, the camera-based student hands-up motion recognition module; the student hand-lifting action recognition module recognizes the student hand-lifting action according to a preset action.
5. The augmented reality technology-based distance teaching system of claim 1 or 2, wherein the first data acquisition terminal comprises a 3D panoramic camera or a 4K high definition pan-tilt camera, and the first display terminal comprises a notebook computer or a desktop computer.
6. The augmented reality technology-based distance teaching system of claim 3 further comprising a data store and a teaching management platform.
7. The augmented reality technology-based distance learning system of claim 2, wherein the eye movement signal analysis module comprises a concentration evaluation value analysis module for solving concentration evaluation values Pi of each subject through pupil diameter and eye movement state.
8. A teaching method based on the augmented reality technology-based distance teaching system of claim 1, comprising the steps of:
s101, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s102, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part;
and S103, the student part outputs the received teaching data through the augmented reality glasses for the students to study.
9. A teaching method based on the augmented reality technology-based distance teaching system of claim 2, characterized by comprising the steps of:
s201, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s202, a teacher part collects teaching data of teaching situations of a teacher and sends the teaching data to a student part through a network transmission part;
s203, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s204, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s205, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
the process of processing and analyzing comprises the following steps:
let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipJudging that the student attention evaluation value does not meet the model requirement;
and S206, the teacher part displays the attention evaluation value Pi through the first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value Pi of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part.
10. A teaching method based on the augmented reality technology-based distance teaching system of claim 7, comprising the steps of:
s301, establishing a data interaction relationship by the teacher department and the student department through the network transmission department;
s302, the teacher part collects teaching data of teaching situations of the teacher and sends the teaching data to the student part through the network transmission part;
s303, the second data acquisition terminal sends the received teaching data to augmented reality glasses for output, so that students can learn;
s304, the eye movement signal sensor collects eye movement information and sends the eye movement information to the eye movement signal analysis module;
s305, the eye movement signal analysis module is used for processing and analyzing the returned eye movement sensor data Si and the preset eye movement state model Mi, solving a concentration evaluation value Pi and forwarding the concentration evaluation value Pi to a teacher department;
the process of processing and analyzing comprises the following steps:
let NiThe total number of sampling points in the latest t duration;
Miefor the model M in the t periodiThe number of matched sampling points;
Miepas model MiConcentration state evaluation value baseline threshold of (1);
f is a concentration evaluation function;
the concentration evaluation value p of the student in the period ti＝f(Mie,Ni)；
If p isi≥MipJudging that the student attention evaluation value meets the model requirement;
if p isi<MipJudging that the student attention evaluation value does not meet the model requirement;
s306, the teacher part displays the attention evaluation value Pi through a first display terminal, the teacher judges the state of the student according to the attention evaluation value Pi, and when the judgment structure is that the attention evaluation value of the student does not meet the model requirement, a reminding signal is sent to the student part through the teacher part;
and S307, the eye movement signal analysis module solves concentration evaluation value Pi curves of all the subjects through the pupil diameter and the eye movement state, and the teacher adjusts the later stage teaching plan through the concentration evaluation value Pi curves.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205746.1A CN111311995A (en) | 2020-03-23 | 2020-03-23 | Remote teaching system and teaching method based on augmented reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205746.1A CN111311995A (en) | 2020-03-23 | 2020-03-23 | Remote teaching system and teaching method based on augmented reality technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111311995A true CN111311995A (en) | 2020-06-19 |
Family
ID=71160708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010205746.1A Pending CN111311995A (en) | 2020-03-23 | 2020-03-23 | Remote teaching system and teaching method based on augmented reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311995A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255431A (en) * | 2021-04-02 | 2021-08-13 | 青岛小鸟看看科技有限公司 | Reminding method and device for remote teaching and head-mounted display equipment |
| CN113808447A (en) * | 2021-08-27 | 2021-12-17 | 深圳市北辰教育科技发展有限公司 | Network teaching method and system based on AI analysis |
| CN114119932A (en) * | 2021-10-14 | 2022-03-01 | 中国科学院自动化研究所 | VR teaching method, apparatus, electronic device, storage medium and program product |
| CN114582185A (en) * | 2022-03-14 | 2022-06-03 | 广州容溢教育科技有限公司 | Intelligent teaching system based on VR technique |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349119A (en) * | 2013-07-24 | 2015-02-11 | 浙江大华技术股份有限公司 | Monitoring video recording method and device |
| CN106056092A (en) * | 2016-06-08 | 2016-10-26 | 华南理工大学 | Gaze estimation method for head-mounted device based on iris and pupil |
| CN107411762A (en) * | 2017-08-31 | 2017-12-01 | 杭州极智医疗科技有限公司 | Cognitive disorder diagnostic equipment |
| CN107515677A (en) * | 2017-08-31 | 2017-12-26 | 杭州极智医疗科技有限公司 | Notice detection method, device and storage medium |
| CN108881861A (en) * | 2018-08-08 | 2018-11-23 | 湖北统讯智能科技有限公司 | A kind of students ' behavior identification and management system |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| US20190122437A1 (en) * | 2017-10-20 | 2019-04-25 | Raytheon Company | Field of View (FOV) and Key Code limited Augmented Reality to Enforce Data Capture and Transmission Compliance |
| CN109949187A (en) * | 2019-03-01 | 2019-06-28 | 北华大学 | A kind of novel Internet of Things teleeducation system and control method |
| CN110135380A (en) * | 2019-05-22 | 2019-08-16 | 东北大学 | A kind of classroom focus knowledge method for distinguishing based on Face datection |
| CN110147163A (en) * | 2019-05-20 | 2019-08-20 | 浙江工业大学 | The eye-tracking method and system of the multi-model fusion driving of facing mobile apparatus |
| CN110233841A (en) * | 2019-06-11 | 2019-09-13 | 上海文景信息科技有限公司 | A kind of long-distance education data interaction system and method based on AR holographic glasses |
| US20190295325A1 (en) * | 2015-04-06 | 2019-09-26 | Scope Technologies Us Inc. | Methods for augmented reality applications |
| CN110428678A (en) * | 2019-08-12 | 2019-11-08 | 重庆工业职业技术学院 | A kind of computer online teaching management system |
| CN110730377A (en) * | 2019-10-30 | 2020-01-24 | 阔地教育科技有限公司 | Short video recording method and device based on interactive classroom |
-
2020
- 2020-03-23 CN CN202010205746.1A patent/CN111311995A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349119A (en) * | 2013-07-24 | 2015-02-11 | 浙江大华技术股份有限公司 | Monitoring video recording method and device |
| US20190295325A1 (en) * | 2015-04-06 | 2019-09-26 | Scope Technologies Us Inc. | Methods for augmented reality applications |
| CN106056092A (en) * | 2016-06-08 | 2016-10-26 | 华南理工大学 | Gaze estimation method for head-mounted device based on iris and pupil |
| CN107411762A (en) * | 2017-08-31 | 2017-12-01 | 杭州极智医疗科技有限公司 | Cognitive disorder diagnostic equipment |
| CN107515677A (en) * | 2017-08-31 | 2017-12-26 | 杭州极智医疗科技有限公司 | Notice detection method, device and storage medium |
| US20190122437A1 (en) * | 2017-10-20 | 2019-04-25 | Raytheon Company | Field of View (FOV) and Key Code limited Augmented Reality to Enforce Data Capture and Transmission Compliance |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| CN108881861A (en) * | 2018-08-08 | 2018-11-23 | 湖北统讯智能科技有限公司 | A kind of students ' behavior identification and management system |
| CN109949187A (en) * | 2019-03-01 | 2019-06-28 | 北华大学 | A kind of novel Internet of Things teleeducation system and control method |
| CN110147163A (en) * | 2019-05-20 | 2019-08-20 | 浙江工业大学 | The eye-tracking method and system of the multi-model fusion driving of facing mobile apparatus |
| CN110135380A (en) * | 2019-05-22 | 2019-08-16 | 东北大学 | A kind of classroom focus knowledge method for distinguishing based on Face datection |
| CN110233841A (en) * | 2019-06-11 | 2019-09-13 | 上海文景信息科技有限公司 | A kind of long-distance education data interaction system and method based on AR holographic glasses |
| CN110428678A (en) * | 2019-08-12 | 2019-11-08 | 重庆工业职业技术学院 | A kind of computer online teaching management system |
| CN110730377A (en) * | 2019-10-30 | 2020-01-24 | 阔地教育科技有限公司 | Short video recording method and device based on interactive classroom |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255431A (en) * | 2021-04-02 | 2021-08-13 | 青岛小鸟看看科技有限公司 | Reminding method and device for remote teaching and head-mounted display equipment |
| CN113808447A (en) * | 2021-08-27 | 2021-12-17 | 深圳市北辰教育科技发展有限公司 | Network teaching method and system based on AI analysis |
| CN114119932A (en) * | 2021-10-14 | 2022-03-01 | 中国科学院自动化研究所 | VR teaching method, apparatus, electronic device, storage medium and program product |
| CN114582185A (en) * | 2022-03-14 | 2022-06-03 | 广州容溢教育科技有限公司 | Intelligent teaching system based on VR technique |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110334610B (en) | Multi-dimensional classroom quantification system and method based on computer vision | |
| CN107945592B (en) | Synchronous mutual-aid classroom teaching system | |
| CN111311995A (en) | Remote teaching system and teaching method based on augmented reality technology | |
| CN108010394B (en) | Virtual teaching method, control terminal and virtual teaching system based on VR | |
| US11694565B2 (en) | Virtual and augmented reality instruction system | |
| CN111242962A (en) | Method, device and equipment for generating remote training video and storage medium | |
| KR102308443B1 (en) | Smart advanced lecture and recoding system | |
| CN211787619U (en) | Remote education and teaching system | |
| TWI674553B (en) | System and method for monitoring qualities of teaching and learning | |
| CN111429316A (en) | Online learning special attention detection system and method based on augmented reality glasses | |
| CN111325853B (en) | Remote coaching system and method based on augmented reality glasses | |
| CN112652200A (en) | Man-machine interaction system, man-machine interaction method, server, interaction control device and storage medium | |
| KR101757420B1 (en) | The system for remote video communication and lecture based on interaction using transparent display | |
| CN104575138A (en) | Scene interaction type multimedia system and application thereof | |
| CN116343311A (en) | Virtual person teaching auxiliary system based on facial expression recognition | |
| CN110933350A (en) | Electronic cloud mirror recording and broadcasting system, method and device | |
| CN116781847A (en) | Blackboard writing guiding and broadcasting method, device, equipment and storage medium | |
| JP2005234368A (en) | Remote lecture system | |
| KR102590186B1 (en) | Online learning system with improved interaction between instructor and student | |
| CN112634096A (en) | Classroom management method and system based on intelligent blackboard | |
| CN117440117A (en) | Remote teaching eye movement switching system and method for network intelligent training room | |
| CN209748700U (en) | Remote video conference system | |
| CN112423035A (en) | Method for automatically extracting visual attention points of user when watching panoramic video in VR head display | |
| CN209343559U (en) | A kind of connection classroom system | |
| Ly et al. | 360TourGuiding: Towards virtual reality training for tour guiding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200619 |
|
| RJ01 | Rejection of invention patent application after publication |