WO2022183423A1 - 基于视线跟踪的线上教学实现方法、装置及存储介质 - Google Patents

基于视线跟踪的线上教学实现方法、装置及存储介质 Download PDF

Info

Publication number
WO2022183423A1
WO2022183423A1 PCT/CN2021/079034 CN2021079034W WO2022183423A1 WO 2022183423 A1 WO2022183423 A1 WO 2022183423A1 CN 2021079034 W CN2021079034 W CN 2021079034W WO 2022183423 A1 WO2022183423 A1 WO 2022183423A1
Authority
WO
WIPO (PCT)
Prior art keywords
teaching
data
strategy
learner
gaze tracking
Prior art date
Application number
PCT/CN2021/079034
Other languages
English (en)
French (fr)
Inventor
王化
Original Assignee
深圳技术大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳技术大学 filed Critical 深圳技术大学
Priority to PCT/CN2021/079034 priority Critical patent/WO2022183423A1/zh
Publication of WO2022183423A1 publication Critical patent/WO2022183423A1/zh

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
    • G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication

Definitions

  • the invention relates to the field of electronic technology, and in particular, to a method, device and storage medium for realizing online teaching based on sight tracking.
  • the main purpose of the embodiments of the present invention is to provide an online teaching implementation method, device and storage medium based on line of sight tracking, which can at least solve the problem that the teacher role agent in the related art cannot make flexible changes in teaching strategies according to the performance of learners, resulting in The teaching effect is more limited.
  • a first aspect of the embodiments of the present invention provides an online teaching implementation method based on gaze tracking, the method comprising:
  • the teacher role agent is controlled to execute the teaching strategy in real time.
  • a second aspect of the embodiments of the present invention provides an online teaching implementation device based on gaze tracking, the device comprising:
  • the acquisition module is used to collect the gaze tracking data of the learners through the sensor
  • a determining module for determining a corresponding teaching strategy based on the sight tracking data
  • the control module is used to control the teacher role agent to execute the teaching strategy in real time.
  • a third aspect of the embodiments of the present invention provides an electronic device, the electronic device includes: a processor, a memory, and a communication bus;
  • the communication bus is used to realize the connection communication between the processor and the memory
  • the processor is configured to execute one or more programs stored in the memory, so as to implement the steps of any one of the above-mentioned online teaching implementation methods.
  • a fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be processed by one or more to implement any of the steps of the above-mentioned online teaching implementation method.
  • the gaze tracking data of learners is collected through sensors; the corresponding teaching strategy is determined based on the gaze tracking data; the teacher role agent is controlled to execute the teaching strategy in real time .
  • the learning performance of the learner is determined according to the eye tracking data of the learner, and the teaching strategy is formulated accordingly, so that the teacher role agent can adjust the teaching strategy in real time, thereby effectively Improve the friendliness of teaching interaction, stimulate learners' interest in learning, and greatly improve the teaching effect.
  • FIG. 1 is a schematic flowchart of a basic flow of an online teaching implementation method provided by the first embodiment of the present invention
  • Fig. 2 is a kind of emotion recognition map provided by the first embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an online teaching display screen provided by the first embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a program module of an online teaching implementation device provided by a second embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
  • this embodiment proposes an online teaching implementation method based on gaze tracking, as shown in the figure 1 shows the basic flow chart of the online teaching implementation method provided by this embodiment.
  • the online teaching implementation method provided by this embodiment includes the following steps:
  • Step 101 Collect the gaze tracking data of the learner through the sensor.
  • the learner will respond to the current teaching content in terms of eye tracking behavior, and the learner's eye tracking behavior is related to the emotional response, the attention response, and the like.
  • the gaze tracking data may include pupil dilation data, eye movement data, etc.
  • the user's eye tracking may be performed by an image sensor (such as a camera) on a computer or mobile phone used by the learner for online learning. Then, according to the image recognition algorithm, multiple consecutive eye images are recognized to obtain the gaze tracking data.
  • Step 102 Determine a corresponding teaching strategy based on the gaze tracking data.
  • the gaze tracking data of the learner is related to his online learning performance, so this embodiment can adaptively adjust the teaching strategy in the online teaching process based on the gaze tracking data.
  • the specific implementation of determining the corresponding teaching strategy based on the gaze tracking data includes: determining the corresponding attention representation data and interest representation data based on the gaze tracking data; combining the attention representation data and the interest representation data to determine the corresponding teaching strategy.
  • this embodiment responds to the learner's sight line information in real time, and records the learner's eye movement data and pupil dilation data during the online learning process.
  • Eye movements can reveal learner interest and attention, enrich non-verbal information by detecting attention, interest, etc. from learners’ real-time eye movement data, and use pupil dilation data to facilitate emotion-related reasoning.
  • the learner shows less interest in the teaching content, the pupil dilation is smaller or the eye movement is smaller.
  • the method before determining the corresponding teaching strategy based on the gaze tracking data, the method further includes: acquiring the learner's biological information, and acquiring the learner's emotion recognition result.
  • determining the corresponding teaching strategy based on the gaze tracking data includes: determining the corresponding teaching strategy in combination with the gaze tracking data, biological information and emotion recognition results.
  • this embodiment can also perceive the learner's biological information, emotional information and other non-linguistic information, so as to perceive the learner's learning state from multiple perspectives, thereby improving the The accuracy of the learner's perception of the learning state.
  • the specific implementation manner of acquiring the learner's emotion recognition result includes: acquiring the learner's galvanic skin response data and muscle electrical response data; determining corresponding emotional arousal data based on the galvanic skin response data, And determine the corresponding emotional valence data based on the muscle electrical response data; combine the emotional arousal data and the emotional valence data to determine the corresponding emotional recognition result of the learner.
  • the galvanic skin response instrument is an instrument that records the changes of the galvanic skin response in the form of curve ripples, and can collect the muscle electrical response data through the electromyography instrument, and record the muscle bioelectricity graph. Therefore, the corresponding emotional arousal data can be determined based on the galvanic skin response data, and the corresponding emotional valence data can be determined based on the electrical muscle response data; then, the corresponding learner emotion recognition result can be determined by combining the emotional arousal data and the emotional valence data.
  • this embodiment uses the GSR sensor to detect the emotional arousal data, and the GSR sensor that measures the skin conductivity (SC) is connected to the middle finger of the learner's non-dominant hand. and index finger, and the signal can be recorded with the ProComp+ device. SC varied linearly with overall arousal levels and increased with anxiety and stress.
  • the emotional arousal data can be detected and obtained by analyzing the peaks and troughs of the GSR data.
  • Figure 2 shows an emotion recognition map provided by the present embodiment, wherein the horizontal axis represents emotional valence data, the vertical axis represents emotional arousal data, and the elements in each quadrant are different types of emotion recognition results, including excitement , relaxation, sadness, fear, etc.
  • the emotional valence data represents the emotional type of the learner
  • the negative valence emotions include fear, sadness, etc.
  • the positive valence emotions include happiness, excitement, etc.
  • the emotional arousal data represents learning The degree of emotional intensity of the person under the corresponding emotional type.
  • Step 103 Control the teacher role agent to execute the teaching strategy in real time.
  • the teacher role agent in this embodiment is a virtual teacher image simulated on the online teaching client.
  • the teacher role agent interacts with the learners to display emotions and teaching content, so that the learners can learn more Motivated to learn.
  • ESA Empathic The role agent in Software Agent
  • the role can be created using the Microsoft agent engine and the emotion, motion and control functions can be created using the MPML agent control language
  • ESA can use a two-dimensional graphics window to display the teacher role agent and educational content.
  • the gaze tracking data of learners can be stored and transmitted using XML files.
  • the serial data is transferred to an XML file using the Com port, and then JavaScript is used to send the information to the interface.
  • the teaching strategy includes: a teaching content interaction strategy and a teaching management interaction strategy.
  • the above step of controlling the teacher role agent to execute the teaching strategy in real time specifically includes: controlling the main teacher role agent to execute the teaching content interaction strategy in real time, and controlling the auxiliary teacher role agent to execute the teaching management interaction strategy in real time.
  • teacher role agents can be simulated on the display interface of the online teaching client, one of which is the main agent, which is used to interact with the learner's teaching content, communicate with the learner, and The teaching content is explained, while the other is an auxiliary agent that performs student monitoring management, managing interactions based on whether the learner is currently focused on or interested in the current teaching content.
  • the teaching strategy includes: emotional feedback form and/or teaching content.
  • the above-mentioned steps of controlling the teacher role agent to execute the teaching strategy in real time specifically include: controlling the teacher role agent to use the above emotional feedback form to conduct teaching interaction with the learner in real time; and/or, controlling the teacher role agent to carry out the teaching content in real time. exhibit.
  • the teacher role agent can interact with the learner by taking corresponding emotions according to the performance of the learner. For example, when it is determined according to the gaze tracking data that the learner is focused on the current teaching topic, the teacher role agent can express Emotions of happiness/satisfaction to motivate the learner, and, when the learner is distracted, the teacher role agent can show a slight anger to alert the learner.
  • the teacher role agent can also display the teaching content correspondingly according to the performance of the learner. For example, when it is determined according to the gaze tracking data that the learner is watching a specific teaching content for more than a certain period of time, the detailed subordinate content of the teaching content can be displayed. to show.
  • Figure 3 is a schematic diagram of an online teaching display screen provided by the present embodiment.
  • the picture enclosed by the circle is the picture that the learner is looking at. That is, the time the eye dwells exceeds the selected threshold, then the system treats it as a selection input and zooms in to give more details on the relevant topic.
  • the above-mentioned step of determining the corresponding teaching strategy based on the gaze tracking data specifically includes: acquiring the learner's historical learning performance data; combining the gaze tracking data and the historical learning performance data to determine the corresponding teaching strategy teaching strategies.
  • this embodiment can track the learner's previous learning performance, and then formulate a corresponding teaching strategy in combination with the previous learning performance and the current gaze tracking data. Effectiveness of teaching strategies.
  • the present invention can detect the learner's attention information and related information from non-linguistic data such as the learner's gaze tracking information, extract the learner's learning trajectory and characteristics, and provide feedback according to the different learning modes of each learner. Personalized learning interaction for learners, learners will be more motivated in the learning process of online education, and can improve the efficiency of learning.
  • the gaze tracking data of the learner is collected by the sensor; the corresponding teaching strategy is determined based on the gaze tracking data; the teacher role agent is controlled to execute the teaching strategy in real time.
  • the learning performance of the learner is determined according to the eye tracking data of the learner, and the teaching strategy is formulated accordingly, so that the teacher role agent can adjust the teaching strategy in real time, thereby effectively Improve the friendliness of teaching interaction, stimulate learners' interest in learning, and greatly improve the teaching effect.
  • this embodiment shows an online teaching implementation device based on gaze tracking.
  • the device for realizing online teaching in this embodiment includes:
  • the acquisition module 401 is used to collect the gaze tracking data of the learner through the sensor;
  • a determination module 402 configured to determine a corresponding teaching strategy based on the sight tracking data
  • the control module 403 is used to control the teacher role agent to execute the teaching strategy in real time.
  • the determining module is specifically configured to: determine the corresponding attention representation data and interest representation data based on the gaze tracking data; and determine the corresponding teaching strategy in combination with the attention representation data and the interest representation data.
  • the online teaching implementation device further includes: an acquisition module, configured to acquire the learner's biological information and acquire the learner's emotion recognition before determining the corresponding teaching strategy based on the gaze tracking data result.
  • the determining module is specifically used for: determining the corresponding teaching strategy in combination with the sight tracking data, the biological information and the emotion recognition result.
  • the acquisition module when the acquisition module performs the above-mentioned function of acquiring the emotion recognition result of the learner, it is specifically used to: acquire the galvanic skin response data and the electrical muscle response data of the learner; The response data determines the corresponding emotional arousal data, and the corresponding emotional valence data is determined based on the muscle electrical response data; the learner's corresponding emotional recognition result is determined by combining the emotional arousal data and the emotional valence data.
  • the determining module is specifically configured to: acquire the historical learning performance data of the learner; and determine the corresponding teaching strategy in combination with the sight tracking data and the historical learning performance data.
  • the teaching strategy includes: emotional feedback form and/or teaching content.
  • the control module is specifically used to: control the teacher role agent to conduct teaching interaction with the learner in real time using the above emotional feedback form; and/or control the teacher role agent to display the above teaching content in real time.
  • the teaching strategy includes: a teaching content interaction strategy and a teaching management interaction strategy.
  • the control module is specifically used to: control the main teacher role agent to execute the teaching content interaction strategy in real time, and control the auxiliary teacher role agent to execute the teaching management interaction strategy in real time.
  • the online teaching implementation method based on line of sight tracking in the foregoing embodiments can be implemented based on the online teaching implementation device based on line of sight tracking provided in this embodiment, and those of ordinary skill in the art can clearly understand that For the convenience and brevity of description, for the specific working process of the online teaching implementation device described in this embodiment, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.
  • the learner's gaze tracking data is collected through sensors; the corresponding teaching strategy is determined based on the gaze tracking data; the teacher role agent is controlled to execute the teaching strategy in real time.
  • the learning performance of the learner is determined according to the eye tracking data of the learner, and the teaching strategy is formulated accordingly, so that the teacher role agent can adjust the teaching strategy in real time, thereby effectively Improve the friendliness of teaching interaction, stimulate learners' interest in learning, and greatly improve the teaching effect.
  • This embodiment provides an electronic device, as shown in FIG. 5 , which includes a processor 501, a memory 502, and a communication bus 503, wherein: the communication bus 503 is used to implement connection and communication between the processor 501 and the memory 502; processing The device 501 is configured to execute one or more computer programs stored in the memory 502 to implement at least one step of the online teaching implementation method in the first embodiment.
  • the present embodiments also provide a computer-readable storage medium embodied in any method or technology for storing information, such as computer-readable instructions, data structures, computer program modules, or other data volatile or nonvolatile, removable or non-removable media.
  • Computer-readable storage media include but are not limited to RAM (Random Access Memory, random access memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory, electrically erasable programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), Digital Versatile Disc (DVD) or other optical disk storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or may be used to store desired information and be accessible by a computer any other medium.
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • EEPROM Electrically Erasable Programmable read only memory
  • flash memory or other memory technology
  • CD-ROM Compact Disc Read-Only Memory
  • the computer-readable storage medium in this embodiment may be used to store one or more computer programs, and the stored one or more computer programs may be executed by a processor to implement at least one step of the method in the first embodiment.
  • This embodiment also provides a computer program, which can be distributed on a computer-readable medium and executed by a computer-readable device to implement at least one step of the method in the above-mentioned first embodiment; and in some cases , at least one of the steps shown or described may be performed in an order different from that described in the above embodiments.
  • This embodiment also provides a computer program product, including a computer-readable device, where the computer program as shown above is stored on the computer-readable device.
  • the computer-readable device may include the computer-readable storage medium as described above.
  • the functional modules/units in the system and the device can be implemented as software (which can be implemented by computer program codes executable by a computing device) ), firmware, hardware, and their appropriate combination.
  • the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components Components execute cooperatively.
  • Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit .
  • communication media typically embodies computer readable instructions, data structures, computer program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery, as is well known to those of ordinary skill in the art medium. Therefore, the present invention is not limited to any particular combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

本发明公开了一种基于视线跟踪的线上教学实现方法、装置及存储介质，通过传感器采集学习者的视线跟踪数据；基于视线跟踪数据确定对应的教学策略；控制教师角色代理实时执行教学策略。通过本发明的实施，在线上教学过程中，根据学习者的视线跟踪数据确定学习者的学习表现，并以此对应制定教学策略，以供教师角色代理实时适应性调整教学策略，由此可有效提高教学交互的友好性，激发学习者的学习兴趣，大大提升了教学效果。

Description

基于视线跟踪的线上教学实现方法、装置及存储介质 技术领域
本发明涉及电子技术领域，尤其涉及一种基于视线跟踪的线上教学实现方法、装置及存储介质。
 背景技术
随着互联网技术的不断发展，线上教学已经成为世界范围内的一种重要的教学工具，在现有的线上教学实现中，通常是由虚拟化的教师角色代理与学习者进行教学交互。
在目前的线上教学实现方式中，针对教师角色代理通常预置有固定的教学策略，从而在实际教学过程中，不能针对学习者的表现作出灵活应对，导致教学效果较为局限。
 技术问题
本发明实施例的主要目的在于提供一种基于视线跟踪的线上教学实现方法、装置及存储介质，至少能够解决相关技术中教师角色代理无法针对学习者的表现作出教学策略的灵活变化，所导致的教学效果较为局限的问题。
 技术解决方案
为实现上述目的，本发明实施例第一方面提供了一种基于视线跟踪的线上教学实现方法，该方法包括：
通过传感器采集学习者的视线跟踪数据；
基于所述视线跟踪数据确定对应的教学策略；
控制教师角色代理实时执行所述教学策略。
为实现上述目的，本发明实施例第二方面提供了一种基于视线跟踪的线上教学实现装置，该装置包括：
采集模块，用于通过传感器采集学习者的视线跟踪数据；
确定模块，用于基于所述视线跟踪数据确定对应的教学策略；
控制模块，用于控制教师角色代理实时执行所述教学策略。
为实现上述目的，本发明实施例第三方面提供了一种电子装置，该电子装置包括：处理器、存储器和通信总线；
所述通信总线用于实现所述处理器和存储器之间的连接通信；
所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现上述任意一种线上教学实现方法的步骤。
为实现上述目的，本发明实施例第四方面提供了一种计算机可读存储介质，该计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现上述任意一种线上教学实现方法的步骤。
 有益效果
根据本发明实施例提供的基于视线跟踪的线上教学实现方法、装置及存储介质，通过传感器采集学习者的视线跟踪数据；基于视线跟踪数据确定对应的教学策略；控制教师角色代理实时执行教学策略。通过本发明的实施，在线上教学过程中，根据学习者的视线跟踪数据确定学习者的学习表现，并以此对应制定教学策略，以供教师角色代理实时适应性调整教学策略，由此可有效提高教学交互的友好性，激发学习者的学习兴趣，大大提升了教学效果。
本发明其他特征和相应的效果在说明书的后面部分进行阐述说明，且应当理解，至少部分效果从本发明说明书中的记载变的显而易见。
 附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明第一实施例提供的线上教学实现方法的基本流程示意图；
图2为本发明第一实施例提供的一种情绪识别图谱；
图3为本发明第一实施例提供的一种线上教学显示画面的示意图；
图4为本发明第二实施例提供的线上教学实现装置的程序模块示意图；
图5为本发明第三实施例提供的电子装置的结构示意图。
 本发明的实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
第一实施例：
为了解决相关技术中教师角色代理无法针对学习者的表现作出教学策略的灵活变化，所导致的教学效果较为局限的问题，本实施例提出了一种基于视线跟踪的线上教学实现方法，如图1所示为本实施例提供的线上教学实现方法的基本流程示意图，本实施例提出的线上教学实现方法包括以下的步骤：
步骤101、通过传感器采集学习者的视线跟踪数据。
具体的，在实际教学过程中，学习者会针对当前教学内容作出视线跟踪行为方面的反应，学习者的视线跟踪行为关联于情绪方面的反应、注意力方面的反应等。
在实际应用中，当学习者进行线上学习的过程中可以连接特定传感器，传感器进行数字采样，而数据捕获可以通过使用Visual C++编写的模块来实现。
在本实施例中，视线跟踪数据可以包括瞳孔扩张数据、眼动数据等，在实际应用中可以通过学习者用于线上学习使用的电脑或手机上的图像传感器（例如摄像头）来进行用户眼部图像采集，然后根据图像识别算法识别连续多张眼部图像来获取视线跟踪数据。
步骤102、基于视线跟踪数据确定对应的教学策略。
具体的，学习者的视线跟踪数据与其线上学习表现相关，从而本实施例可以基于其视线跟踪数据适应性调整线上教学过程中的教学策略。
在本实施例中，基于视线跟踪数据确定对应的教学策略的具体实现方式包括：基于视线跟踪数据确定相应的注意力表征数据和兴趣表征数据；结合注意力表征数据和兴趣表征数据，确定对应的教学策略。
具体的，本实施例对学习者的视线信息作出实时反应，在线上学习过程中记录学习者的眼动数据和瞳孔扩张数据。眼动可以显示出学习者的兴趣和注意力，通过从学习者的实时眼动数据中检测注意力、兴趣等信息来丰富非语言信息，使用瞳孔扩张数据来促进情绪相关的推理。在实际应用中，如果学习者对教学内容表现出较少的兴趣，则其瞳孔的扩张量较小或眼球的活动量较少。
在本实施例的另一种可选的实施方式中，基于视线跟踪数据确定对应的教学策略之前，还包括：获取学习者的生体信息，以及获取学习者的情绪识别结果。相对应的，基于所述视线跟踪数据确定对应的教学策略，包括：结合视线跟踪数据、生体信息以及情绪识别结果，确定对应的教学策略。
具体的，除了对学习者的视线跟踪数据进行感知，本实施例还可以对学习者的生体信息、情绪信息等非语言信息进行感知，以从多方位感知学习者的学习状态，由此可以提高学习者的学习状态感知的准确性。
进一步地，在本实施例中，获取所述学习者的情绪识别结果的具体实现方式包括：获取学习者的皮肤电反应数据和肌肉电反应数据；基于皮肤电反应数据确定相应的情绪唤醒数据，以及基于肌肉电反应数据确定相应的情绪效价数据；结合情绪唤醒数据和情绪效价数据，确定学习者相应的情绪识别结果。
应当理解的是，在实际应用中，学习者情绪识别结果可以基于皮肤电反应数据和肌肉电反应数据进行分析实现，其中，通过皮肤电反应仪（GSR,Galvanic Skin Responseapparatus）来进行皮肤电反应数据的采集，皮肤电反应仪是以曲线波纹形式记录皮肤电反应变化的仪器，并可通过肌电仪来进行肌肉电反应数据的采集，对肌肉生物电图形进行记录。从而，可以基于皮肤电反应数据确定相应的情绪唤醒数据，以及基于肌肉电反应数据确定相应的情绪效价数据；然后，结合情绪唤醒数据和情绪效价数据，确定相应的学习者情绪识别结果。
应当说明的是，由于GSR数据相对于BVP数据的噪声信号较少，从而本实施例使用GSR传感器来检测情绪唤醒数据，测量皮肤电导率（SC）的GSR传感器连接在学习者非支配手的中指和食指上，并可用ProComp+装置记录信号。SC随整体唤醒水平线性变化，随焦虑和压力增加而增加。在实施例中，通过分析GSR数据的波峰和波谷即可检测得到情绪唤醒数据。
如图2所示为本实施例提供的一种情绪识别图谱，其中，横轴代表情绪效价数据，纵轴代表情绪唤醒数据，各象限中的元素则为不同类型的情绪识别结果，包括兴奋、放松、悲伤、恐惧等，在本实施例中，情绪效价数据表示学习者的情绪类型，负价情绪包括恐惧、悲伤等，正价情绪包括快乐、兴奋等，而情绪唤醒数据则表示学习者在相应情绪类型下的情绪强弱程度。
步骤103、控制教师角色代理实时执行教学策略。
具体的，本实施例的教师角色代理为线上教学客户端上所模拟的虚拟教师形象，在线上教学过程中通过教师角色代理与学习者进行交互，展示情感和教学内容，以让学习者更有学习动力。
应当说明的是，对于人机交互的线上教学接口（ESA，Empathic Software Agent）中的角色代理可以采用不同方式实现，例如可使用Microsoft代理引擎创建角色，并使用MPML代理控制语言创建情感、动作和控制功能，ESA可以使用一个二维图形窗口来显示教师角色代理和教育内容。在实际应用中，学习者的视线跟踪数据可以使用XML文件存储和传输。在这里，串行数据使用Com端口传输到XML文件，然后使用JavaScript将信息发送到接口。
在本实施例的一种可选的实施方式中，教学策略包括：教学内容交互策略和教学管理交互策略。相对应的，上述控制教师角色代理实时执行教学策略的步骤，具体包括：控制主教师角色代理实时执行教学内容交互策略，以及控制辅教师角色代理实时执行教学管理交互策略。
具体的，在实际应用中，可以在线上教学客户端的显示界面上模拟出两个不同类型的教师角色代理，其中一个是主代理，用于执行与学习者的教学内容交互，与学习者对话以及解释教学内容，而另一个则是辅助代理，用于执行学生监控管理，根据学习者当前是否专注于当前教学内容或是否对当前教学内容感兴趣来进行管理交互。
在本实施例的另一种可选的实施方式中，教学策略包括：情绪反馈形式和/或教学内容。相对应的，上述控制教师角色代理实时执行教学策略的步骤，具体包括：控制教师角色代理实时采用上述情绪反馈形式与学习者进行教学交互；和/或，控制教师角色代理实时对上述教学内容进行展示。
具体的，在一种实现中，教师角色代理可以根据学习者的表现采取对应的情绪与之进行交互，例如在根据视线跟踪数据确定学习者专注于当前的教学主题时，教师角色代理可以表现出快乐/满意情绪以激励学习者，以及，在学习者注意力不集中时，教师角色代理可以表现出轻微愤怒情绪以提醒学习者。在另一种实现中，教师角色代理还可以根据学习者的表现对应展示教学内容，例如在根据视线跟踪数据确定学习者正在注视特定教学内容超过一定时长时，可以对该教学内容的详细下级内容进行展示。如图3所示为本实施例提供的一种线上教学显示画面的示意图，图中有三张教学图片，圆圈所圈定的图片为学习者正在注视的图片，如果学习者“凝视”该面板，即眼睛停留时间超过所选阈值的时间，则系统将其视为选择输入，并放大给出相关主题的更多细节。
在本实施例的一种可选的实施方式中，上述基于视线跟踪数据确定对应的教学策略的步骤，具体包括：获取学习者的历史学习表现数据；结合视线跟踪数据以及历史学习表现数据确定对应的教学策略。
具体的，本实施例可以跟踪学习者以往的学习表现情况，然后结合以往的学习表现情况以及当前的视线跟踪数据来制定对应教学策略，由此，通过多维度的数据来制定教学策略，可以提升教学策略的有效性。
应当理解的是，线上教学往往需要在教师角色代理和学习者之间建立更有效的互动，特别是，通过激发学习者的兴趣来增加学习动机是很重要的。本发明可以通过从学习者的视线跟踪信息等非语言数据中检测学习者的注意力信息以及相关信息，并提取学习者的学习的轨迹与特点，并根据每个学习者的不同学习模式，反馈给学习者的个性化学习互动，学习者在线上教育的学习过程中将更有动力，并能提高学习的效率。
根据本发明实施例提供的基于视线跟踪的线上教学实现方法，通过传感器采集学习者的视线跟踪数据；基于视线跟踪数据确定对应的教学策略；控制教师角色代理实时执行教学策略。通过本发明的实施，在线上教学过程中，根据学习者的视线跟踪数据确定学习者的学习表现，并以此对应制定教学策略，以供教师角色代理实时适应性调整教学策略，由此可有效提高教学交互的友好性，激发学习者的学习兴趣，大大提升了教学效果。
第二实施例：
为了解决相关技术中教师角色代理无法针对学习者的表现作出教学策略的灵活变化，所导致的教学效果较为局限的问题，本实施例示出了一种基于视线跟踪的线上教学实现装置，具体请参见图4，本实施例的线上教学实现装置包括：
采集模块401，用于通过传感器采集学习者的视线跟踪数据；
确定模块402，用于基于视线跟踪数据确定对应的教学策略；
控制模块403，用于控制教师角色代理实时执行教学策略。
在本实施例的一些实施方式中，确定模块具体用于：基于视线跟踪数据确定相应的注意力表征数据和兴趣表征数据；结合注意力表征数据和兴趣表征数据，确定对应的教学策略。
在本实施例的另一些实施方式中，线上教学实现装置还包括：获取模块，用于在基于视线跟踪数据确定对应的教学策略之前，获取学习者的生体信息，以及获取学习者的情绪识别结果。相应的，确定模块具体用于：结合视线跟踪数据、生体信息以及情绪识别结果，确定对应的教学策略。
进一步地，在本实施例的一些实施方式中，获取模块在执行上述获取学习者的情绪识别结果的功能时，具体用于：获取学习者的皮肤电反应数据和肌肉电反应数据；基于皮肤电反应数据确定相应的情绪唤醒数据，以及基于肌肉电反应数据确定相应的情绪效价数据；结合情绪唤醒数据和情绪效价数据，确定学习者相应的情绪识别结果。
在本实施例的又一些实施方式中，确定模块具体用于：获取学习者的历史学习表现数据；结合视线跟踪数据以及历史学习表现数据确定对应的教学策略。
在本实施例的一些实施方式中，教学策略包括：情绪反馈形式和/或教学内容。相应的，控制模块具体用于：控制教师角色代理实时采用上述情绪反馈形式与学习者进行教学交互；和/或，控制教师角色代理实时对上述教学内容进行展示。
在本实施例的一些实施方式中，教学策略包括：教学内容交互策略和教学管理交互策略。相应的，控制模块具体用于：控制主教师角色代理实时执行教学内容交互策略，以及控制辅教师角色代理实时执行教学管理交互策略。
应当说明的是，前述实施例中的基于视线跟踪的线上教学实现方法，均可基于本实施例提供的基于视线跟踪的线上教学实现装置实现，所属领域的普通技术人员可以清楚的了解到，为描述的方便和简洁，本实施例中所描述的线上教学实现装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
采用本实施例提供的基于视线跟踪的线上教学实现装置，通过传感器采集学习者的视线跟踪数据；基于视线跟踪数据确定对应的教学策略；控制教师角色代理实时执行教学策略。通过本发明的实施，在线上教学过程中，根据学习者的视线跟踪数据确定学习者的学习表现，并以此对应制定教学策略，以供教师角色代理实时适应性调整教学策略，由此可有效提高教学交互的友好性，激发学习者的学习兴趣，大大提升了教学效果。
第三实施例：
本实施例提供了一种电子装置，参见图5所示，其包括处理器501、存储器502及通信总线503，其中：通信总线503用于实现处理器501和存储器502之间的连接通信；处理器501用于执行存储器502中存储的一个或者多个计算机程序，以实现上述实施例一中的线上教学实现方法中的至少一个步骤。
本实施例还提供了一种计算机可读存储介质，该计算机可读存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、计算机程序模块或其他数据)的任何方法或技术中实施的易失性或非易失性、可移除或不可移除的介质。计算机可读存储介质包括但不限于RAM（Random Access Memory，随机存取存储器）, ROM（Read-Only Memory，只读存储器）, EEPROM（Electrically Erasable Programmable read only memory，带电可擦可编程只读存储器）、闪存或其他存储器技术、CD-ROM（Compact Disc Read-Only Memory，光盘只读存储器），数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。
本实施例中的计算机可读存储介质可用于存储一个或者多个计算机程序，其存储的一个或者多个计算机程序可被处理器执行，以实现上述实施例一中的方法的至少一个步骤。
本实施例还提供了一种计算机程序，该计算机程序可以分布在计算机可读介质上，由可计算装置来执行，以实现上述实施例一中的方法的至少一个步骤；并且在某些情况下，可以采用不同于上述实施例所描述的顺序执行所示出或描述的至少一个步骤。
本实施例还提供了一种计算机程序产品，包括计算机可读装置，该计算机可读装置上存储有如上所示的计算机程序。本实施例中该计算机可读装置可包括如上所示的计算机可读存储介质。
可见，本领域的技术人员应该明白，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件（可以用计算装置可执行的计算机程序代码来实现）、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。
此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、计算机程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。所以，本发明不限制于任何特定的硬件和软件结合。
以上内容是结合具体的实施方式对本发明实施例所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种基于视线跟踪的线上教学实现方法，其特征在于，包括：
    通过传感器采集学习者的视线跟踪数据；
    基于所述视线跟踪数据确定对应的教学策略；
    控制教师角色代理实时执行所述教学策略。
  2. 如权利要求1所述的线上教学实现方法，其特征在于，所述基于所述视线跟踪数据确定对应的教学策略，包括：
    基于所述视线跟踪数据确定相应的注意力表征数据和兴趣表征数据；
    结合所述注意力表征数据和所述兴趣表征数据，确定对应的教学策略。
  3. 如权利要求1所述的线上教学实现方法，其特征在于，所述基于所述视线跟踪数据确定对应的教学策略之前，还包括：
    获取所述学习者的生体信息，以及获取所述学习者的情绪识别结果；
    所述基于所述视线跟踪数据确定对应的教学策略，包括：
    结合所述视线跟踪数据、所述生体信息以及所述情绪识别结果，确定对应的教学策略。
  4. 如权利要求3所述的线上教学实现方法，其特征在于，所述获取所述学习者的情绪识别结果，包括：
    获取所述学习者的皮肤电反应数据和肌肉电反应数据；
    基于所述皮肤电反应数据确定相应的情绪唤醒数据，以及基于所述肌肉电反应数据确定相应的情绪效价数据；
    结合所述情绪唤醒数据和所述情绪效价数据，确定所述学习者相应的情绪识别结果。
  5. 如权利要求1所述的线上教学实现方法，其特征在于，所述基于所述视线跟踪数据确定对应的教学策略，包括：
    获取所述学习者的历史学习表现数据；
    结合所述视线跟踪数据以及所述历史学习表现数据确定对应的教学策略。
  6. 如权利要求1至5中任意一项所述的线上教学实现方法，其特征在于，所述教学策略包括：情绪反馈形式和/或教学内容；
    所述控制教师角色代理实时执行所述教学策略，包括：
    控制教师角色代理实时采用所述情绪反馈形式与所述学习者进行教学交互；
    和/或，控制所述教师角色代理实时对所述教学内容进行展示。
  7. 如权利要求1至5中任意一项所述的线上教学实现方法，其特征在于，所述教学策略包括：教学内容交互策略和教学管理交互策略；
    所述控制教师角色代理实时执行所述教学策略，包括：
    控制主教师角色代理实时执行所述教学内容交互策略，以及控制辅教师角色代理实时执行所述教学管理交互策略。
  8. 一种基于视线跟踪的线上教学实现装置，其特征在于，包括：
    采集模块，用于通过传感器采集学习者的视线跟踪数据；
    确定模块，用于基于所述视线跟踪数据确定对应的教学策略；
    控制模块，用于控制教师角色代理实时执行所述教学策略。
  9. 一种电子装置，其特征在于，包括：处理器、存储器和通信总线；
    所述通信总线用于实现所述处理器和存储器之间的连接通信；
    所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现如权利要求1至7中任意一项所述的线上教学实现方法的步骤。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至7中任意一项所述的线上教学实现方法的步骤。
PCT/CN2021/079034 2021-03-04 2021-03-04 基于视线跟踪的线上教学实现方法、装置及存储介质 WO2022183423A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/079034 WO2022183423A1 (zh) 2021-03-04 2021-03-04 基于视线跟踪的线上教学实现方法、装置及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/079034 WO2022183423A1 (zh) 2021-03-04 2021-03-04 基于视线跟踪的线上教学实现方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2022183423A1 true WO2022183423A1 (zh) 2022-09-09

Family

ID=83153905

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079034 WO2022183423A1 (zh) 2021-03-04 2021-03-04 基于视线跟踪的线上教学实现方法、装置及存储介质

Country Status (1)

Country Link
WO (1) WO2022183423A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116130062A (zh) * 2022-12-29 2023-05-16 华南师范大学 基于眼动特征的学习者注意力水平评估分析方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108074203A (zh) * 2016-11-10 2018-05-25 中国移动通信集团公司 一种教学调整方法和装置
US20190102722A1 (en) * 2017-10-03 2019-04-04 International Business Machines Corporation System and method enabling dynamic teacher support capabilities
CN110992222A (zh) * 2019-11-05 2020-04-10 深圳追一科技有限公司 教学交互方法、装置、终端设备及存储介质
CN111610862A (zh) * 2020-06-22 2020-09-01 江苏开放大学（江苏城市职业学院） 一种基于眼动信号的在线教学模式切换方法
CN112016829A (zh) * 2020-08-27 2020-12-01 江苏开放大学（江苏城市职业学院） 一种基于眼动信息的在线课程学习评估方法
CN112530218A (zh) * 2020-11-19 2021-03-19 深圳市木愚科技有限公司 多对一陪伴智能教学系统及教学方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108074203A (zh) * 2016-11-10 2018-05-25 中国移动通信集团公司 一种教学调整方法和装置
US20190102722A1 (en) * 2017-10-03 2019-04-04 International Business Machines Corporation System and method enabling dynamic teacher support capabilities
CN110992222A (zh) * 2019-11-05 2020-04-10 深圳追一科技有限公司 教学交互方法、装置、终端设备及存储介质
CN111610862A (zh) * 2020-06-22 2020-09-01 江苏开放大学（江苏城市职业学院） 一种基于眼动信号的在线教学模式切换方法
CN112016829A (zh) * 2020-08-27 2020-12-01 江苏开放大学（江苏城市职业学院） 一种基于眼动信息的在线课程学习评估方法
CN112530218A (zh) * 2020-11-19 2021-03-19 深圳市木愚科技有限公司 多对一陪伴智能教学系统及教学方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116130062A (zh) * 2022-12-29 2023-05-16 华南师范大学 基于眼动特征的学习者注意力水平评估分析方法
CN116130062B (zh) * 2022-12-29 2024-01-16 华南师范大学 基于眼动特征的学习者注意力水平评估分析方法

Similar Documents

Publication Publication Date Title
Goldberg et al. Attentive or not? Toward a machine learning approach to assessing students’ visible engagement in classroom instruction
Sharma et al. Student engagement detection using emotion analysis, eye tracking and head movement with machine learning
Zaletelj et al. Predicting students’ attention in the classroom from Kinect facial and body features
Dewan et al. Engagement detection in online learning: a review
JP3224675U (ja) 瞳孔反応、顔追跡、及び感情検出を使用した対話型・適応型学習、神経認知障害診断、並びに非追随性検出のシステム
Mangaroska et al. Exploring students' cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity
JP4631014B2 (ja) 電子教材学習支援装置と電子教材学習支援システム及び電子教材学習支援方法並びに電子学習支援プログラム
CN107909037B (zh) 信息输出方法和装置
CN112546390A (zh) 一种注意力训练方法、装置、计算机设备及存储介质
Kirsal Ever et al. The effectiveness of a new classification system in higher education as a new e-learning tool
CN111428686A (zh) 一种学生兴趣偏好评估方法、装置及系统
Buono et al. Assessing student engagement from facial behavior in on-line learning
Pise et al. Estimation of learning affects experienced by learners: an approach using relational reasoning and adaptive mapping
WO2022183423A1 (zh) 基于视线跟踪的线上教学实现方法、装置及存储介质
Wang et al. Multimodal motivation modelling and computing towards motivationally intelligent E-learning systems
Zhang et al. Review of gaze tracking and its application in intelligent education
Alhasan et al. An experimental study of learning behaviour in an elearning environment
Cannon et al. Action anticipation and interference: A test of prospective gaze
CN112908066A (zh) 基于视线跟踪的线上教学实现方法、装置及存储介质
US20230127335A1 (en) Intelligent and adaptive measurement system for remote education
Kasperiuniene et al. Affective engagement to virtual and live lectures
Miao et al. Analysis of facial expressions for the estimation of concentration on online lectures
Koshti et al. AttenQ-Attention Span Detection Tool for Online Learning
CN115690867A (zh) 课堂专注度检测方法、装置、设备及存储介质
Bernard et al. Minimizing cognitive overload in cybersecurity learning materials: an experimental study using eye-tracking

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21928510

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21928510

Country of ref document: EP

Kind code of ref document: A1