CN111325853B - Remote coaching system and method based on augmented reality glasses - Google Patents
Remote coaching system and method based on augmented reality glasses Download PDFInfo
- Publication number
- CN111325853B CN111325853B CN202010205824.8A CN202010205824A CN111325853B CN 111325853 B CN111325853 B CN 111325853B CN 202010205824 A CN202010205824 A CN 202010205824A CN 111325853 B CN111325853 B CN 111325853B
- Authority
- CN
- China
- Prior art keywords
- augmented reality
- student
- reality glasses
- teacher
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 62
- 239000011521 glass Substances 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000003993 interaction Effects 0.000 claims abstract description 23
- 230000001360 synchronised effect Effects 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
- G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20061—Hough transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Business, Economics & Management (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The invention discloses a remote coaching system and method based on augmented reality glasses, which belong to the technical field of digital teaching and at least comprise the following steps of; the teacher end is used for collecting teaching situations of a teacher, receiving student end data and displaying the student end data; the student end is used for receiving and displaying teacher end data and collecting the student class listening situation; the cloud server is used for realizing data interaction between the teacher end and the student end; the method is characterized in that: the teacher end and the student end comprise augmented reality glasses; the student end augmented reality glasses are used for shooting videos at a first person view angle and displaying video and voice information returned from a teacher end; the augmented reality glasses at the teacher end are used for shooting videos at a first person view angle and then sending the videos and the voice information to the student end, or playing teaching courseware and the videos to be synchronized to the student end, or displaying the video and the voice information returned from the student end.
Description
Technical Field
The invention belongs to the technical field of digital teaching, and particularly relates to a remote coaching system and method based on augmented reality glasses.
Background
As is well known, the conventional classroom teaching mode objectively has space limitation and time limitation, that is, students and teachers must arrive at a specific place to perform teaching work at a specific time, and in recent years, with rapid development of internet of things technology, in order to overcome the above-mentioned defects of the conventional classroom teaching mode, a network teaching system has been developed, and currently, the existing network teaching system mainly includes three parts: the system comprises two information recording parts for acquiring audio and video information of teachers or students and a cloud server for transmitting data. The existing network teaching system has the following defects:
in the conventional teaching process of classroom or extracurricular coaching, a teacher usually requires students to solve questions on a blackboard or on manuscript paper, and the students observe the mastering condition of knowledge points to see whether the questions are solved or not clearly, so that targeted teaching is developed. However, the remote education system on the market at present mainly uses live broadcast or recorded broadcast of the video of the lesson of the teacher, lacks interaction with students of the participants in the classroom, and is particularly difficult to realize that the teacher wants to check the problem solving situation of the students at the remote place in real time so as to achieve the purpose of knowing the learning degree of the students.
Disclosure of Invention
The invention provides a remote coaching system and a remote coaching method based on augmented reality glasses for solving the technical problems in the prior art, which can simulate classroom teaching through the augmented reality technology and realize the immersive learning of students at home. The teacher can interact with the students in real time, and interactive experience is improved. Meanwhile, the eye tracking technology is utilized to detect the attention of the students in real time, the attention transfer and other conditions of the students are found in real time, and effective attention management and control are realized. By mining eye movement data of students in a certain period, course quality assessment and student psychological and hobby assessment work can be carried out.
A first object of the present invention is to provide a remote coaching system based on augmented reality glasses, comprising at least;
the teacher end is used for collecting teaching situations of a teacher, receiving student end data and displaying the student end data;
the student end is used for receiving and displaying teacher end data and collecting the student class listening situation;
the cloud server is used for realizing data interaction between the teacher end and the student end; the method is characterized in that:
the teacher end and the student end comprise augmented reality glasses;
the student end augmented reality glasses are used for shooting videos at a first person view angle and displaying video and voice information returned from a teacher end; the augmented reality glasses at the teacher end are used for shooting videos at a first person view angle and then sending the videos and the voice information to the student end, or playing teaching courseware and the videos to be synchronized to the student end, or displaying the video and the voice information returned from the student end.
Further, the student end and the teacher end are respectively provided with a front camera, and the cameras are used for shooting videos in the handwriting process of the paper surface, and simultaneously recognizing gestures of the student or the teacher and recognizing content variation written on the paper.
Further, the augmented reality glasses of teacher end and student end all carry miniature speaker and microphone that the pronunciation interacted.
Further, the remote guidance system based on the augmented reality glasses comprises a courseware playing and/or video module and a real-time guidance and/or post-hoc guidance module.
Further, the student end comprises a manual recording module of the teaching video.
Further, the remote guidance system based on the augmented reality glasses further comprises an image processing module, when the remote guidance system is used for performing after-the-fact guidance, the augmented reality glasses at the student end record the problem solving process of the student and upload the problem solving process to the cloud server for storage, the augmented reality glasses grasp the problem solving video screenshot of the student at regular time, the digital image processing module is used for identifying the content variation of the student writing on paper, when the content of the next capture is compared with the content of the previous capture, the data of the next capture is associated with a time stamp, and the data of the next capture is uploaded to the cloud server for storage in a mode of replacing the video with a picture.
Further, the cloud server stores the video clips in a time stamp-based manner.
A second object of the present invention is to provide a method for a remote teaching system based on the above-mentioned augmented reality technology, comprising the steps of:
s101, establishing a data interaction relationship between a teacher end and a student end through a cloud server;
s102, a teacher side collects teaching data of teaching situations of the teacher and sends the teaching data to a cloud server;
s103, the student side acquires real-time coaching or post-coaching data through the cloud server, and then enters a corresponding coaching course;
when students select real-time coaching, the augmented reality glasses of the teacher end and the student end are synchronously opened, and locally acquired data are sent to the other party; the two parties can read the data of the other party in real time through the augmented reality glasses;
when students choose to conduct after-the-fact coaching, the augmented reality glasses at the students end record the problem solving process of the students and upload the problem solving process to the cloud server for storage, the augmented reality glasses capture the problem solving video screenshot of the students at regular time, the digital image processing module is used for identifying the content variation quantity written by the students on paper, when the content of the next capture is compared with the content of the previous capture, the content of the next capture exceeds a preset percentage, the data of the next capture is associated with a timestamp, and the data of the next capture is uploaded to the cloud server for storage and stored in a mode that the video is replaced by pictures.
Further, the digital image processing module identifies the specific process of the content change amount written on the paper by the student as follows:
s201, a timing screenshot;
s202, preprocessing an image;
s203, calculating the variable quantity of two adjacent screenshots, comparing the variable quantity with a threshold value, returning to the step S201 when the variable quantity does not exceed the threshold value, and executing the step S204 when the variable quantity exceeds the threshold value;
s204, associating the screenshot with a timestamp;
s205, sending the data to a cloud server for storage.
Further, the image preprocessing specifically includes:
s2021, identifying the paper contour of an original image; the method comprises the following steps:
marking the paper surface outline in the screenshot by using a differential operator canny algorithm; firstly, smoothing an image by adopting a Gaussian filter to remove image noise; then, respectively calculating gradient components in the x-direction and the y-direction and gradient directions by using a sobel operator; then, suppressing the place where the gradient value is not the maximum value, excluding non-edge pixels, and only retaining some thin lines; finally, connecting the communication points on the graph by using the double threshold values;
the sobel operator is used for a discrete difference operator of edge detection, and a certain pixel and a neighborhood thereof in the image are expressed as follows:
wherein: i is the neighborhood of the pixel point f (x, y) and 3*3;
respectively deriving the x direction and the y direction, and when the kernel size is 3:
horizontal conversion
Vertical transform->
Gradient magnitude and direction were calculated using the following formula:
gradient amplitude
Direction angle->
S2022, binarizing the outline image;
s2023, calculating the inclination angle and the position of the contour by Hough transformation;
s2024, image rotation and stretching correction.
The invention has the advantages and positive effects that:
the invention can simulate one-to-one or one-to-many extracurricular coaching by using the augmented reality glasses, and can realize remote coaching of students at home. The teacher can obtain the visual angle of the student and interact with the student in real time, so that the interactive experience is improved; the cloud server can be used for consulting the stored visual angle images of the students after the events, and analyzing the learning state of the students. The method comprises the following steps:
when the teacher starts to play courseware or teaching video, the courseware and the teaching video are synchronously played in the student-side augmented reality glasses.
The student end has the manual module of recording of teaching video, and when the student had the storage to the synchronous teaching video of teacher end and review the demand, can record manually. When a student clicks to record, in order to record a more complete teaching video, the recording module sets the video recording starting point to be one minute before the recording module records the more complete video. The cloud server provides a video storage space with a certain capacity for students to store recorded interesting videos, and is convenient to search and play back.
When the real-time tutoring is carried out, the student-side augmented reality glasses shoot the problem solving process of the students in real time, and the real-time video stream is transmitted to the teacher display side through the cloud management server to carry out real-time interaction; when a plurality of students participate in the tutoring, the teacher display terminal can display the problem solving videos of the plurality of students at the same time, and the teacher can select one-to-one interaction tutoring with the students or simultaneously select interaction tutoring with the plurality of students.
Drawings
FIG. 1 is a block diagram of the preferred embodiment of the present invention;
FIG. 2 is a schematic view of the mounting position of the front camera in the preferred embodiment of the present invention;
FIG. 3 is a flow chart of identifying the amount of change in the content written on paper by a student in a preferred embodiment of the invention;
fig. 4 is a flow chart of image preprocessing in a preferred embodiment of the present invention.
Detailed Description
For a further understanding of the invention, its features and advantages, reference is now made to the following examples, which are illustrated in the accompanying drawings in which:
referring to fig. 1 to 4, a first preferred embodiment: a remote coaching system based on augmented reality glasses, comprising; the system comprises a student part, a teacher part, a network transmission part and a cloud management server part, wherein the network transmission part is used for realizing data interaction between a first data acquisition terminal and a second data acquisition terminal; the student department includes: the first data acquisition terminal is used for acquiring handwriting video of the student, and the first display terminal is used for receiving teacher data and displaying the teacher data; the teacher section includes: the second display terminal is used for receiving student part data and displaying the student part data, and the second data acquisition terminal is used for acquiring teacher handwriting video; the first data acquisition and display terminal and the second data acquisition and display terminal are augmented reality glasses. By adopting the technical scheme, the invention can simulate one-to-one or one-to-many extracurricular coaching by using the augmented reality glasses, and can realize remote coaching of students at home. The teacher can obtain the visual angle of the student and interact with the student in real time, so that the interactive experience is improved; the cloud server can be used for consulting the stored visual angle images of the students after the events, and analyzing the learning state of the students.
Wherein: the augmented reality glasses at the student end are used for shooting videos at a first person viewing angle and displaying video and voice information returned from the teacher end; the augmented reality glasses at the teacher end are used for shooting videos at the first person view angle and then sending the videos and the voice information to the student end, or playing teaching courseware and the videos to be synchronized to the student end, or displaying the video and the voice information returned from the student end.
Student's end and teacher's end augmented reality glasses carry on leading camera, like fig. 2, shoot the video with student or teacher's first person's visual angle, mainly used shoots the video at paper handwriting process, discerns student or teacher's gesture simultaneously and discerns the content variation who writes on paper.
Platform authorities mainly divide three roles of system administrators, teachers and students. The system administrator may perform any processing of platform dependent rights on the platform. Any user can create a virtual classroom and set a classroom password, the user enters the classroom according to the classroom password, the user defaults to a student role when entering the virtual classroom, and a virtual classroom creator can designate a member in the classroom as a teacher role. When the augmented reality glasses at the student end are connected to the platform, the video content presented by the screen is controlled by the teacher of the classroom until the classroom manager gives up control or the student exits the virtual classroom.
The voice interaction of teacher end and student end is realized through miniature speaker and microphone that augmented reality glasses carried.
The teacher end system is provided with two functional modules of courseware playing, video playing, real-time coaching and post coaching.
When the teacher starts to play courseware or teaching video, the courseware and the teaching video are synchronously played in the student-side augmented reality glasses.
The student end has the manual module of recording of teaching video, and when the student had the storage to the synchronous teaching video of teacher end and review the demand, can record manually. When a student clicks to record, in order to record a more complete teaching video, the recording module sets the video recording starting point to be one minute before the recording module records the more complete video. The cloud server provides a video storage space with a certain capacity for students to store recorded interesting videos, and is convenient to search and play back.
When the real-time tutoring is carried out, the student-side augmented reality glasses shoot the problem solving process of the students in real time, and the real-time video stream is transmitted to the teacher display side through the cloud management server to carry out real-time interaction; when a plurality of students participate in the tutoring, the teacher display terminal can display the problem solving videos of the plurality of students at the same time, and the teacher can select one-to-one interaction tutoring with the students or simultaneously select interaction tutoring with the plurality of students.
When the teacher end adopts intelligent display equipment such as a mobile phone or a tablet, interaction comprises voice interaction and touch handwriting input interaction, the handwriting input interaction comprises images such as marks and characters, and the images are synchronously sent to the augmented reality glasses of students through a network for display.
When the teacher end adopts the augmented reality glasses equipment, the interaction comprises voice interaction, gesture input interaction and paper handwriting interaction, and the real-time video of the teacher end with handwriting on paper is synchronously sent to the augmented reality glasses of students through the network for display.
When the post-customization tutoring is carried out, the student-side augmented reality glasses record the problem solving process of the students and upload the problem solving process to the cloud server for storage, so that the storage capacity of the cloud server is reduced, and the problem solving process is stored in a video screenshot mode based on a time stamp. The augmented reality glasses capture the problem solving video screenshot of the student at regular time, identify the content variation of the student writing on paper by utilizing a digital image processing algorithm, when the content of the next capture exceeds a preset percentage compared with the content of the previous capture, correlate the data of the next capture with a time stamp, upload the data to a cloud server and store the data, and greatly reduce the storage capacity of the cloud server by using a storage mode of replacing the video with a picture.
When the first person views the handwriting of the student, the paper is usually laid on the desktop, the optical axis of the eyes is generally 45-90 degrees with the paper surface, and the optical axis of the camera is not perpendicular to the image, which causes perspective deformation of the image. And the video shot by the same student at each moment has angle change, which is unfavorable for identifying the change amount of the front and rear contents. Therefore, when the digital image processing algorithm is used for identifying the content variation written on paper, firstly, the picture obtained by the video screenshot needs to be preprocessed, and then, the handwriting content variation is detected.
The preprocessing algorithm mainly comprises paper contour recognition, binarization, hough transformation recognition, straight line and tangential correction of an original image.
Marking the paper surface outline in the screenshot by using a differential operator canny algorithm; firstly, smoothing an image by adopting a Gaussian filter to remove image noise; secondly, respectively calculating gradient components in the x-direction and the y-direction and gradient directions by using a sobel operator; the third step suppresses the place where the gradient value is not the maximum value, and is used for eliminating non-edge pixels, and only some thin lines are reserved; and finally, connecting the communication points on the graph by using the double threshold values.
The sobel operator is a discrete difference operator mainly used for edge detection, and a certain pixel and a neighborhood thereof in an image are expressed as follows:
wherein: i is the neighborhood of the pixel point f (x, y) and 3*3;
respectively deriving the x direction and the y direction, and when the kernel size is 3:
horizontal conversion
Vertical transformation
Gradient magnitude and direction were calculated using the following formula:
gradient amplitude
Direction angle->
After the paper surface contour is obtained through detection of the paper surface, binarization processing is carried out on the paper surface, and the black-white contour of the paper surface is remained while background interference is removed. And then carrying out Hough transformation to fit a paper surface contour straight line, and calculating the inclination angle and the position of the contour in the screenshot.
The hough transform is an algorithm for detecting a straight line, and considering the correspondence between points and lines, a straight line passing through a point (x 1, y 1) can be expressed as: y1=kx1+b, the variables and parameters are interchanged, knowing a point (x 1, y 1), the linear cluster passing through this point can be expressed as b= (-x 1) k+y1. Points lying on the same straight line have the same slope and intercept, and are reflected on the parameter space, namely the straight lines meet the same point (k, b), so that straight lines in the picture are found. The inclination angle of the paper surface can be obtained by the straight line of the fitted paper surface contour through Hough transformation.
After pretreatment, a paper surface binarized black-and-white image with the same paper surface size and inclination angle is obtained. The detection algorithm of the handwriting content variation comprises two parts, namely expansion and image subtraction.
The expansion of the image is a convolution operation, and is used for strengthening and highlighting the paper surface characters, so that the change of the front picture and the rear picture can be conveniently identified subsequently. The convolution operation is very simple, and for each pixel of the image, a certain neighborhood is taken, and the maximum value is calculated as the pixel value of the corresponding pixel position of the new image.
And finally, calculating the content variation of the students writing on paper in the front and back video screenshot by utilizing digital image subtraction operation, and when the content of the next screenshot exceeds the content of the previous screenshot by a preset percentage, correlating the data of the next screenshot with a time stamp, and uploading the data to a cloud server for storage.
The above-described embodiments are only for illustrating the technical spirit and features of the present invention, and it is intended to enable those skilled in the art to understand the content of the present invention and to implement it accordingly, and the scope of the present invention is not limited to the embodiments, i.e. equivalent changes or modifications to the spirit of the present invention are still within the scope of the present invention.
Claims (9)
1. Remote coaching system based on augmented reality glasses; at least comprises;
the teacher end is used for collecting teaching situations of a teacher, receiving student end data and displaying the student end data;
the student end is used for receiving and displaying teacher end data and collecting the student class listening situation;
the cloud server is used for realizing data interaction between the teacher end and the student end; the method is characterized in that:
the teacher end and the student end comprise augmented reality glasses;
the remote guidance system based on the augmented reality glasses further comprises an image processing module, when the remote guidance system is used for performing the post-hoc guidance, the augmented reality glasses at the student end record the problem solving process of the student and upload the problem solving process to the cloud server for storage, the augmented reality glasses grasp the problem solving video screenshot of the student at regular time, the digital image processing module is used for identifying the content variation of the student writing on paper, when the content of the next capture is compared with the content of the previous capture, the data of the next capture is associated with a time stamp, and the data of the next capture is uploaded to the cloud server for storage in a mode of replacing the video with a picture;
the student end augmented reality glasses are used for shooting videos at a first person view angle and displaying video and voice information returned from a teacher end; the augmented reality glasses at the teacher end are used for shooting videos at a first person view angle and then sending the videos and the voice information to the student end, or playing teaching courseware and the videos to be synchronized to the student end, or displaying the video and the voice information returned from the student end.
2. The augmented reality glasses-based remote coaching system according to claim 1, wherein the student side and the teacher side are each equipped with a front camera for capturing video of a handwriting process on a paper surface, and recognizing gestures of the student or the teacher and recognizing a content variation written on the paper.
3. The augmented reality glasses-based remote coaching system according to claim 1, wherein the teacher side and student side augmented reality glasses are each equipped with a micro speaker and microphone for voice interaction.
4. The augmented reality glasses-based remote coaching system according to claim 1, wherein the augmented reality glasses-based remote coaching system comprises a play courseware and/or video module, a real-time coaching and/or post-hoc coaching module.
5. The augmented reality glasses-based remote coaching system according to claim 1, wherein the student side includes a manual recording module of teaching video.
6. The augmented reality glasses-based remote coaching system according to claim 1, wherein the cloud server stores in the form of a time-stamp-based video screenshot.
7. A method of augmented reality glasses-based remote coaching system according to claim 1, comprising the steps of:
s101, establishing a data interaction relationship between a teacher end and a student end through a cloud server;
s102, a teacher side collects teaching data of teaching situations of the teacher and sends the teaching data to a cloud server;
s103, the student side acquires real-time coaching or post-coaching data through the cloud server, and then enters a corresponding coaching course;
when students select real-time coaching, the augmented reality glasses of the teacher end and the student end are synchronously opened, and locally acquired data are sent to the other party; the two parties can read the data of the other party in real time through the augmented reality glasses;
when students choose to conduct after-the-fact coaching, the augmented reality glasses at the students end record the problem solving process of the students and upload the problem solving process to the cloud server for storage, the augmented reality glasses capture the problem solving video screenshot of the students at regular time, the digital image processing module is used for identifying the content variation quantity written by the students on paper, when the content of the next capture is compared with the content of the previous capture, the content of the next capture exceeds a preset percentage, the data of the next capture is associated with a timestamp, and the data of the next capture is uploaded to the cloud server for storage and stored in a mode that the video is replaced by pictures.
8. The method of augmented reality glasses-based remote coaching system according to claim 7, wherein the digital image processing module identifies the content variation specific process of students writing on paper is:
s201, a timing screenshot;
s202, preprocessing an image;
s203, calculating the variable quantity of two adjacent screenshots, comparing the variable quantity with a threshold value, returning to the step S201 when the variable quantity does not exceed the threshold value, and executing the step S204 when the variable quantity exceeds the threshold value;
s204, associating the screenshot with a timestamp;
s205, sending the data to a cloud server for storage.
9. The method of augmented reality glasses-based remote coaching system according to claim 8, wherein the image preprocessing is specifically:
s2021, identifying the paper contour of an original image; the method comprises the following steps:
marking the paper surface outline in the screenshot by using a differential operator canny algorithm; firstly, smoothing an image by adopting a Gaussian filter to remove image noise; then, respectively calculating gradient components in the x-direction and the y-direction and gradient directions by using a sobel operator; then, suppressing the place where the gradient value is not the maximum value, excluding non-edge pixels, and only retaining some thin lines; finally, connecting the communication points on the graph by using the double threshold values;
the sobel operator is used for a discrete difference operator of edge detection, and a certain pixel and a neighborhood thereof in the image are expressed as follows:
wherein: i is the neighborhood of the pixel point f (x, y) and 3*3;
respectively deriving the x direction and the y direction, and when the kernel size is 3: horizontal conversion
Vertical transformation
Gradient magnitude and direction were calculated using the following formula:
gradient amplitude
Angle of direction
S2022, binarizing the outline image;
s2023, calculating the inclination angle and the position of the contour by Hough transformation;
s2024, image rotation and stretching correction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205824.8A CN111325853B (en) | 2020-03-23 | 2020-03-23 | Remote coaching system and method based on augmented reality glasses |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205824.8A CN111325853B (en) | 2020-03-23 | 2020-03-23 | Remote coaching system and method based on augmented reality glasses |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111325853A CN111325853A (en) | 2020-06-23 |
| CN111325853B true CN111325853B (en) | 2023-04-28 |
Family
ID=71165869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010205824.8A Active CN111325853B (en) | 2020-03-23 | 2020-03-23 | Remote coaching system and method based on augmented reality glasses |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111325853B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671875B (en) * | 2020-12-17 | 2021-09-03 | 广州宏途教育网络科技有限公司 | Remote psychological coaching system and method for primary and secondary schools |
| CN113507599B (en) * | 2021-07-08 | 2022-07-08 | 四川纵横六合科技股份有限公司 | Education cloud service platform based on big data analysis |
| CN114283639A (en) * | 2022-01-10 | 2022-04-05 | 立人实业(广东)集团有限公司 | Remote education control system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228982A (en) * | 2016-07-27 | 2016-12-14 | 华南理工大学 | A kind of interactive learning system based on education services robot and exchange method |
| CN107240317A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of long-distance education |
| JP6227816B1 (en) * | 2017-03-27 | 2017-11-08 | 株式会社ドワンゴ | Class system, class method, correspondence class server and correspondence class program |
| CN109102727A (en) * | 2018-09-26 | 2018-12-28 | 宋飞 | A kind of virtual reality teaching equipment |
| CN110233841A (en) * | 2019-06-11 | 2019-09-13 | 上海文景信息科技有限公司 | A kind of long-distance education data interaction system and method based on AR holographic glasses |
| KR102012316B1 (en) * | 2019-02-12 | 2019-10-21 | 주식회사 이에프엠 | After-school class contents provision system based on augmented reality |
-
2020
- 2020-03-23 CN CN202010205824.8A patent/CN111325853B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228982A (en) * | 2016-07-27 | 2016-12-14 | 华南理工大学 | A kind of interactive learning system based on education services robot and exchange method |
| JP6227816B1 (en) * | 2017-03-27 | 2017-11-08 | 株式会社ドワンゴ | Class system, class method, correspondence class server and correspondence class program |
| CN107240317A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of long-distance education |
| CN109102727A (en) * | 2018-09-26 | 2018-12-28 | 宋飞 | A kind of virtual reality teaching equipment |
| KR102012316B1 (en) * | 2019-02-12 | 2019-10-21 | 주식회사 이에프엠 | After-school class contents provision system based on augmented reality |
| CN110233841A (en) * | 2019-06-11 | 2019-09-13 | 上海文景信息科技有限公司 | A kind of long-distance education data interaction system and method based on AR holographic glasses |
Non-Patent Citations (1)
| Title |
|---|
| 马少斌等.一种基于AR的可视化交互教学平台.《兰州文理学院学报(自然科学版)》.2019,第33卷(第1期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111325853A (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107945592B (en) | Synchronous mutual-aid classroom teaching system | |
| CN111325853B (en) | Remote coaching system and method based on augmented reality glasses | |
| CN111242962A (en) | Method, device and equipment for generating remote training video and storage medium | |
| CN104575142A (en) | Experiential digitalized multi-screen seamless cross-media interactive opening teaching laboratory | |
| WO2019019403A1 (en) | Interactive situational teaching system for use in k12 stage | |
| CN107135333A (en) | A kind of teaching writing/playing system | |
| CN103297688A (en) | System and method for multi-media panorama recording | |
| CN112652200A (en) | Man-machine interaction system, man-machine interaction method, server, interaction control device and storage medium | |
| CN203588489U (en) | A situational teaching device | |
| CN111179650A (en) | Platform system for automatic documenting of paper writing board writing and explanation | |
| CN111311995A (en) | Remote teaching system and teaching method based on augmented reality technology | |
| CN106056997A (en) | Computer multimedia remote education training teaching device | |
| CN110378261B (en) | Student identification method and device | |
| CN110933350A (en) | Electronic cloud mirror recording and broadcasting system, method and device | |
| CN116226821B (en) | Teaching data center management system | |
| CN205792941U (en) | A kind of teaching management recording and broadcasting system | |
| CN111050111A (en) | Online interactive learning communication platform and learning device thereof | |
| CN209748700U (en) | Remote video conference system | |
| CN217543870U (en) | Interactive teaching classroom system | |
| CN105357476A (en) | Video photographing system and method | |
| CN111327943B (en) | Information management method, device, system, computer equipment and storage medium | |
| CN112270264A (en) | Multi-party interactive teaching system | |
| CN106131505B (en) | System for realizing cloud board image splicing and recording method | |
| CN111369400A (en) | Middle school student learning process supervision method based on image data processing | |
| WO2024001560A1 (en) | Augmented reality-based drawing teaching method and system, display terminal, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |