CN117743509A

CN117743509A - Auxiliary learning method and server based on combination of AR glasses analysis and transducer

Info

Publication number: CN117743509A
Application number: CN202311647280.0A
Authority: CN
Inventors: 曹晋; 崔海涛; 李星
Original assignee: Goolton Technology Co ltd
Current assignee: Goolton Technology Co ltd
Priority date: 2023-12-04
Filing date: 2023-12-04
Publication date: 2024-03-22

Abstract

An AR glasses analysis and transducer combined auxiliary learning method and server are applied to the education field, and the method comprises the following steps: acquiring a question text and a answer text in the test paper through AR glasses; vectorizing the topic text to obtain a topic vector; comparing the similarity of the topic vector with the knowledge vector in a preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector; determining target knowledge corresponding to the target knowledge vector; respectively inputting the target knowledge and the answer text into a transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; and comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct. The learning evaluation method and the learning evaluation device have the effect of improving learning evaluation accuracy.

Description

Auxiliary learning method and server based on combination of AR glasses analysis and transducer

Technical Field

The application relates to the field of education, in particular to an auxiliary learning method and server based on combination of AR glasses analysis and a transducer.

Background

With the continuous development of technology, learning assistance techniques and products are also being continuously improved and innovated. In the related art, learning auxiliary products, such as an online course platform and the like, provide a brand new learning mode and rich learning resources by utilizing an information technology, digitize the learning resources through the Internet, enable a learner to learn at any time and any place, and break the time and space limitation of the traditional education.

However, online curriculum platforms also have certain limitations. For example, in evaluating learning effects of learners, it is often only possible to examine selection questions and judgment questions, and it is difficult to examine some questions, because an automatic scoring system of an online lesson platform usually scores based on a preset answer library, which often contains only standard answers or limited options. For open questions, there may be multiple correct answers or solutions, so it is difficult to find a completely matching answer in a preset answer library, thereby affecting the accuracy of the evaluation.

Disclosure of Invention

The application provides an AR glasses analysis and transducer combination-based auxiliary learning method and server, which are used for improving the accuracy of learning evaluation.

In a first aspect, the present application provides an AR glasses analysis and transducer combined assisted learning method, the method comprising: acquiring a question text and a answer text in the test paper through AR glasses; vectorizing the topic text to obtain a topic vector; comparing the similarity of the topic vector with the knowledge vector in a preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector; determining target knowledge corresponding to the target knowledge vector; respectively inputting the target knowledge and the answer text into a transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; and comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct.

In the above embodiment, the question text and the answer text of the student in the test paper are extracted, the question text is vectorized, the text content is converted into the numerical representation, the question vector is obtained, the similarity comparison is performed between the question vector and the knowledge vector in the preset knowledge base, and the target knowledge vector with the highest similarity with the question vector is found, so that the target knowledge most relevant to the question is determined. And respectively inputting the target knowledge and the answer text into a transducer model for semantic analysis, and obtaining a semantic analysis result of the target knowledge and a semantic analysis result of the answer text through processing the model. And comparing the semantic analysis result of the target knowledge with the semantic analysis result of the answer text to determine the correctness of the answer text. If the matching degree of the two is high, the answer text can be judged to be correct; otherwise, errors may exist. The method is beneficial to realizing semantic analysis and correctness judgment of the answer text and improving the accuracy of learning evaluation. Through the AR glasses, students can accurately see the positions of the test questions and answers, the understanding of learners on the questions and the accuracy of answering the questions are improved, and a learning mode with better interactivity and interestingness is provided for the students

With reference to some embodiments of the first aspect, in some embodiments, before the step of acquiring the question text and the answer text in the test paper through the AR glasses, the method further includes: determining a question area and a answer area in the paper area; generating an AR test paper according to the selection of a user, wherein the AR test paper is a virtual test paper generated by adopting an enhanced display technology and comprises questions extracted from a preset question bank; and displaying the AR test paper in the paper area, and in the question area and the answer area in the paper area through the display screen of the AR glasses.

In the embodiment, through determining the paper area and the question area and the answer area in the paper area, students do not need to carry a large amount of paper test papers, and the test papers can be answered anytime and anywhere only by wearing the AR glasses, so that virtual questions and answer areas are seen on virtual paper, the sense of reality and immersion of the test papers are enhanced, and more specific visual experience is provided. The generation and display of the test paper can be adjusted and modified according to the requirements, a more flexible use mode is provided, the AR test paper can be connected with the system, and the functions of real-time feedback and automatic assessment are realized. Students can obtain answer results and evaluation in real time, and personalized learning guidance and advice are provided.

With reference to some embodiments of the first aspect, in some embodiments, before the step of acquiring the question text and the answer text in the test paper through the AR glasses, the method further includes: inputting knowledge text in a preset knowledge base into a preset word embedding model to obtain a knowledge word vector; and calculating the average value of the knowledge word vectors to obtain the knowledge vectors in the preset knowledge base.

In the above embodiment, each knowledge word may be converted into a vector representation by inputting the knowledge text into the preset word embedding model, which may capture semantic relationships and context information between words such that each word has a corresponding vector representation. By calculating the average value of the knowledge word vectors, a plurality of knowledge word vectors in a preset knowledge base can be integrated into one knowledge vector. This knowledge vector can be regarded as an overall representation in the knowledge base, representing the main features and semantic information in the knowledge base. When the similarity comparison is carried out between the question vector and the knowledge vector in the preset knowledge base, the knowledge vector in the preset knowledge base is used as a reference, so that the target knowledge most relevant to the question can be found. The matching accuracy is improved, and the correlation between the selected target knowledge and the questions is high.

With reference to some embodiments of the first aspect, in some embodiments, vectorizing the topic text to obtain a topic vector specifically includes: inputting the topic text into a preset word embedding model to obtain a topic word vector; and calculating the average value of the topic word vectors to obtain the topic vector.

In the above embodiment, in the topic vectorization process, the preset word embedding model may capture the semantic feature of each word in the topic text, so that the topic vector can express the similarity and the difference of the topics in terms of semantics. Calculating the average value of the topic word vectors can integrate a plurality of word vectors in the topic text into one vector, the dimension of the vector can be reduced through the operation of reducing the dimension and the integration, and simultaneously, a plurality of semantic features of the topic are integrated into one integral feature to better represent the semantic meaning of the topic. Through topic vectorization, similarity comparison can be carried out on the topic vector and other vectors so as to find out the most relevant target knowledge of the topic, and matching accuracy is improved.

With reference to some embodiments of the first aspect, in some embodiments, after the step of comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct, the method further includes: under the condition that the answer text is correct, comparing the answer text semantic analysis result of the next question with the target knowledge semantic analysis result; and in the case that the answer text is incorrect, marking the corresponding question as a wrong question.

In the above embodiment, after judging whether the answer text of the current question is correct, the answer text of the next question is continuously evaluated whether to be consistent with or relevant to the target knowledge, so as to check the answer situation of the student in real time and provide more comprehensive feedback. In the case that the answer text is judged to be incorrect, the corresponding questions are marked as wrong questions, so that the students can record errors and conduct targeted subsequent teaching and coaching. By combining wrong topic labeling with continuous assessment, more personalized and accurate learning coaching and feedback can be provided, students can be helped to learn own errors and shortages, and knowledge supplementing and strengthening exercises can be performed in a targeted manner.

With reference to some embodiments of the first aspect, in some embodiments, after the step of comparing the text semantic analysis result of the next question with the target knowledge semantic analysis result, the method further includes: after determining that all questions in the test paper are compared with the semantic analysis results, obtaining a modification suggestion of the answer text according to the target knowledge semantic analysis results of the questions; evaluating all questions and answering conditions of students in the test paper to obtain difficulty evaluation of the test paper and level evaluation of the students; and displaying the modification advice, the difficulty evaluation and the level evaluation through a display screen of the AR glasses.

In the above-described embodiments, by providing modification suggestions for answer text, students are purposefully assisted in correcting errors and enhancing knowledge applications to provide personalized learning guidance, helping students better understand and master targeted knowledge. Through the evaluation of the test paper and the students, the difficulty evaluation of the test paper and the level evaluation of the students can be obtained, the overall learning condition of the students and the quality of the test paper can be known, and references are provided for subsequent teaching and improvement. Through the display screen application of the AR glasses, modification suggestions, difficulty evaluation and level evaluation can be displayed to students in real time. So that students can directly obtain feedback and guidance in the process of making questions, and more flexible and convenient learning experience is provided.

With reference to some embodiments of the first aspect, in some embodiments, after the step of marking the corresponding question as a wrong question in the case that the answer text is incorrect, the method further includes: extracting word stems or restoring word shapes of the wrong text to obtain a standardized wrong text; determining a wrong topic vocabulary according to the standardized wrong topic text; and inputting the wrong topic vocabulary into a transformer model to obtain the topics with the same types of wrong topics.

In the above embodiment, the wrong question text is standardized through operations such as stem extraction or morphological reduction, so as to reduce the influence of vocabulary variation and extract the core meaning of the wrong question. And generating a wrong question embedding vector through a preset word embedding model, and converting the wrong questions into semantic representations, so that subsequent semantic analysis and processing are facilitated. By utilizing the semantic understanding and generating capability of the transducer model, the topics of the same type as the wrong questions are generated, thereby helping students to review and consolidate the wrong questions in a targeted manner.

In a second aspect, embodiments of the present application provide a server, including: the acquisition module is used for acquiring the question text and the answer text in the test paper through the AR glasses; the vectorization module is used for vectorizing the topic text to obtain a topic vector; the comparison module is used for comparing the similarity between the topic vector and the knowledge vector in the preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector; the first determining module is used for determining target knowledge corresponding to the target knowledge vector; the second determining module is used for respectively inputting the target knowledge and the answer text into the transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; and the third determining module is used for comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct.

In a third aspect, embodiments of the present application provide a server, including: one or more processors and memory; the memory is coupled to the one or more processors, the memory for storing computer program code comprising computer instructions that the one or more processors call to cause the server to perform the method as described in the first aspect and any possible implementation of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a server, cause the server to perform a method as described in the first aspect and any possible implementation of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium comprising instructions that, when executed on a server, cause the server to perform a method as described in the first aspect and any possible implementation of the first aspect.

It will be appreciated that the servers provided in the second aspect, the third aspect, the computer program product provided in the fourth aspect, and the computer storage medium provided in the fifth aspect are each configured to perform the method provided in the embodiment of the present application. Therefore, the advantages achieved by the method can be referred to as the advantages of the corresponding method, and will not be described herein.

One or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:

1. because the AR glasses analysis technology is adopted, a learner can directly display virtual test questions on the test paper in an augmented reality mode, and virtual learning experience in a real environment is realized. In traditional education, students often need to read questions and write answers on paper test paper, which may present problems of understanding bias and difficult operation. Through AR glasses analysis, students can accurately see the positions of the test questions and the answers, and the understanding of learners on the questions and the accuracy of answering the questions are improved. The technical innovation not only improves the learning effect, but also provides a learning mode with interactivity and interestingness for students.

2. By applying the transducer model in the learning auxiliary field, the invention can carry out deep semantic analysis on target knowledge and answer texts, accurately evaluate the knowledge mastering degree and thinking ability development of students, effectively solve the problem that the traditional learning evaluation method usually only pays attention to the correctness of the answers, neglects the understanding of the students on the knowledge and the cultivation of the application ability, and further realizes that the students are provided with targeted learning suggestions through the personalized evaluation mode, thereby helping the students to understand and master the knowledge better.

2. By adopting the method combining the preset knowledge base and the similarity comparison and comparing the similarity between the question vector and the knowledge vector in the preset knowledge base, the accurate positioning of target knowledge is realized, the problem that students possibly face a large amount of knowledge points and information in traditional learning and are difficult to determine learning key points and core content is effectively solved, and further learning of helping students focus on key knowledge points is realized, and learning efficiency and learning results are improved.

Drawings

FIG. 1 is a flow chart of an assisted learning method based on AR glasses analysis combined with a transducer in an embodiment of the present application;

FIG. 2 is another flow chart of an assisted learning method based on AR glasses analysis combined with a transducer in an embodiment of the present application;

FIG. 3 is a schematic diagram of a functional module of a server according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a physical device of a server according to an embodiment of the present application.

Detailed Description

The terminology used in the following embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates to the contrary. It should also be understood that the term "and/or" as used in this application is intended to encompass any or all possible combinations of one or more of the listed items.

The terms "first," "second," and the like, are used below for descriptive purposes only and are not to be construed as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature, and in the description of embodiments of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.

For easy understanding, the application scenario of the embodiments of the present application is described below. In the field of education, more and more students select an online learning platform for learning. However, the existing online learning platform has some problems, such as that students often lack personalized learning guidance and feedback, resulting in poor learning effect. In addition, students may understand knowledge points to varying degrees during the learning process, and the platform may not provide accurate assessment and assistance.

To solve this problem, we propose an assisted learning method based on AR glasses analysis combined with a transducer. The method utilizes the AR glasses technology to present the virtual test paper in front of students in an enhanced display mode, and combines a transducer model to carry out semantic analysis, so as to provide more accurate and personalized learning assistance.

AR glasses (Augmented Reality Glasses) are intelligent glasses devices that combine augmented reality (Augmented Reality) technology and the functionality of the glasses. Augmented reality is a technology that fuses virtual information with the real world, providing a rich interactive and information presentation experience for users by superimposing virtual images, text, video or other digital content in the user's field of view.

The transducer model is a neural network architecture based on a self-attention mechanism for processing sequence data.

In the related art, on a conventional online learning platform, students learn through computers or mobile devices, and mainly rely on course contents and online tests provided by the platform. However, there are some drawbacks to this approach. First, platforms often fail to accurately assess the understanding of knowledge points by students because they lack automated semantic analysis and personalized learning assistance functionality. Secondly, students may lack timely feedback and guidance, failing to correct errors and promote learning effects in time.

By adopting the auxiliary learning method based on the combination of AR glasses analysis and the transducer in the embodiment of the application, students wear AR glasses and see the questions on the virtual test paper in an enhanced display mode. The AR glasses acquire test paper images through the built-in camera and transmit the test paper images to the server for processing. The server uses a transducer model to carry out semantic analysis on the question text and the student answer text, and compares the semantic analysis with knowledge vectors in a preset knowledge base, so as to determine target knowledge and judge the correctness of the answer text.

Through the scheme, students can obtain the semantic analysis result of the answer text in real time and compare the semantic analysis result with target knowledge. Thus, students can timely learn about errors and defects of the students and obtain corresponding modification suggestions and guidance. Meanwhile, the server can provide personalized learning recommendation and assistance according to the learning condition and semantic analysis result of the students. The personalized learning assistance helps students to better understand and master knowledge points, and improves learning effect.

For ease of understanding, the method provided in this embodiment is described in the following in conjunction with the above scenario. Fig. 1 is a schematic flow chart of an auxiliary learning method based on AR glasses analysis and transducer combination in the embodiment of the application.

S101, acquiring a question text and a response text in a test paper through AR glasses;

firstly, capturing images on a virtual test paper through a camera of the AR glasses, and then extracting text information in the images by using computer vision and optical character recognition technology. The optical character recognition algorithm can recognize the question text and answer text on the test paper and convert them into processable text data.

After the topic text and answer text of the optical character recognition are obtained, text processing and analysis are performed, including cleaning, normalizing and segmenting the text to extract specific content of each topic and corresponding answer text, and text data is processed and analyzed by using natural language processing technology for subsequent operation and analysis.

S102, vectorizing the topic text to obtain a topic vector;

dividing or cutting each word in the topic text to obtain a word sequence, and converting each word into a corresponding word vector by using a selected word embedding model; and averaging or weighted averaging is carried out on word vectors of all words to obtain a vector representation of the topic text, namely the topic vector.

Alternatively, in general, the vectorizing the topic text to obtain the topic vector may be implemented as follows:

inputting the topic text into a preset word embedding model to obtain a topic word vector; and calculating the average value of each topic word vector to obtain the topic vector.

First, a suitable Word embedding model, such as Word2Vec, gloVe or BERT, is determined. Word2Vec, gloVe, and BERT are all Word vector representation methods or models commonly used in the field of natural language processing (Natural Language Processing, NLP).

Word2Vec (Word to Vector) is a neural network based Word vector representation model that learns the distributed representation of words by training a shallow neural network.

GloVe (Global Vectors for Word Representation) is a word vector representation model based on global vocabulary statistics. GloVe builds a global vocabulary co-occurrence matrix by analyzing statistics of word co-occurrence in context. Then, by performing matrix decomposition on the matrix, a word vector representation of each word is obtained. The GloVe model is able to capture semantic relationships and co-occurrence information between words.

BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on a transducer model that learns rich semantic and grammatical representations by self-supervised learning on large scale unlabeled text data. Unlike conventional left-to-right or right-to-left language models, BERT employs a bi-directional encoder and utilizes bi-directional context information. The BERT learns the context-dependent word vector representation in a pre-training stage by tasks such as filling, masking, and next sentence prediction. In many NLP tasks, the pretrained model of BERT can be subjected to transfer learning through fine tuning, and excellent performance is achieved.

These models can map words to fixed length vector representations; then, word segmentation or cutting is carried out on each word in the topic text to obtain a word sequence, and the word sequence is converted into a corresponding word vector by using a selected word embedding model for each word; and averaging or weighted averaging is carried out on word vectors of all words to obtain a vector representation of the topic text, namely the topic vector.

S103, comparing the similarity of the topic vector with knowledge vectors in a preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector;

The topic vector is compared with each knowledge vector in the knowledge base using a suitable similarity measure, such as cosine similarity, euclidean distance, manhattan distance, etc., and a similarity score between the topic vector and each knowledge vector is calculated.

And sorting according to the similarity score to obtain the target knowledge vector with the highest similarity with the topic vector.

S104, determining target knowledge corresponding to the target knowledge vector;

and selecting the target knowledge vector with the highest similarity score from the sorting result as the most similar knowledge point with the topic vector.

S105, determining target knowledge corresponding to the target knowledge vector;

S106, comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct.

The subject matter or core content of the semantic analysis results of the target knowledge and the answer text are compared to determine whether they are conceptually and logically identical. If so, determining that the answer text is correct.

In the above embodiment, by introducing the auxiliary learning method combining AR glasses analysis and Transformer, the results of question text and answer text acquisition, vectorization, similarity comparison and semantic analysis are achieved to judge the answer results of students, the evaluation accuracy of auxiliary learning is improved, and in practical application, the method can be further optimized.

The following supplements the scenario of the present embodiment.

Student's elementary school is using an AR glasses-based auxiliary learning system for mathematical exercises. He wears AR glasses and the title is presented in front of his eyes in a manner of enhanced display. The Xiaoming is solving a complex mathematical topic, but he makes an error.

On the display screen of the AR glasses, the small answer is marked as wrong and resolution of the correct answer is displayed. The Xiaoming finds out that he is wrong in the process of solving the problems by observing and analyzing. He notices that he has missed a critical intermediate step in the calculation process, leading to erroneous results.

The server displays a prompt message on the display screen of the AR glasses, wherein the prompt message indicates the step of the Ming's possibly making mistakes in solving the problems, and provides related problem solving skills and methods. The min re-thinks about the solution idea through the prompt information and makes a second attempt on the AR glasses.

The method is used for solving the problems carefully according to the prompt information. He completed the calculation process through the AR glasses and gave the correct answer. The display screen on the AR glasses again marks his answer as correct.

With the aid of the AR glasses, the Xiaoming finds and corrects own errors in time in the process of solving the problems. AR glasses provide accurate error markers and analysis cues that help a small person to understand the cause of the error and guide him to re-think about the process of solving the problem. The instant feedback and individuation assistance enable the Ming to better understand and master the mathematical knowledge and improve the problem solving capability of the Ming.

In addition to this correction of errors, AR glasses also record a small and clear process of solving the problem and related data. These data may be transmitted to a server for analysis for generating learning reports and personalized learning recommendations. By analyzing the questions solving data of the small scale, the server can know the strong points and weak points of the server in mathematical learning and provide specific learning resources and suggestions for the server so as to further improve the learning effect of the server.

In combination with the above scenario, a further more specific flow of the method provided in this embodiment will be described below. Fig. 2 is a schematic flow chart of an auxiliary learning method based on AR glasses analysis and transducer combination in the embodiment of the application.

S201, determining a paper area, and a question area and a question answering area in the paper area;

Real-time images of a scene are acquired using cameras built into AR glasses or external sensors. The position and pose of the camera relative to the environment is tracked by applying a visual tracking algorithm, such as time-lapse localization and mapping (Simultaneous Localization and Mapping, SLAM) or feature point-based tracking.

In the real-time image, the paper is detected using a computer vision method. This may be accomplished by edge detection, color filtering, shape matching, etc. techniques to find the location and boundaries of the paper in the image.

Once a sheet is detected, the rotation and tilt angles of the sheet may be determined using a pose estimation algorithm. Based on the position and posture information of the paper, a computer vision method is used for positioning the questions and the answer areas, and the positions and boundaries of the questions and the answer areas in the image are found.

S202, generating an AR test paper according to the selection of a user;

the AR test paper is a virtual test paper generated by adopting an enhanced display technology and comprises questions extracted from a preset question bank.

According to the selection of the user, whether the test questions are real test paper or virtual test paper is determined, and the types of the test questions are selected questions, filled questions, judgment questions and the like.

And generating contents in the AR test paper by using a corresponding algorithm or template according to the selected test question types and the number.

S203, displaying the AR test paper in the paper area and the question area and the answer area in the paper area through a display screen of the AR glasses;

the generated AR test paper content is displayed on the display screen of the AR glasses through the AR development tool and the frame, and is positioned according to the positions and boundaries of the question area and the answer area, so that the AR test paper is ensured to be aligned with the paper area and correctly displayed.

S204, inputting knowledge texts in a preset knowledge base into a preset word embedding model to obtain a knowledge word vector;

firstly, preprocessing a knowledge text in a preset knowledge base, including operations such as text cleaning, word segmentation, stop word removal and the like; then, a vocabulary is constructed from the preprocessed text data. A vocabulary is a collection of unique words contained in all text data, ensuring that each word is mapped to a unique index; then, using a preset Word embedding model (such as Word2Vec, gloVe, etc.), the preprocessed text data is input into the model for training, word vectors of each Word are generated, and during the training process, the model learns the semantics and the context relation between the words.

And obtaining a corresponding word vector of each word in the preset knowledge base by inquiring the vocabulary table by using the trained word embedding model.

S205, calculating the average value of each knowledge word vector to obtain a knowledge vector in a preset knowledge base;

the average value of the knowledge word vectors may be obtained by performing an array operation using a vector operation, for example using a numpy library, summing all knowledge word vectors, and dividing the result by the total number of knowledge words. The average value of the knowledge word vectors is the knowledge vector in the preset knowledge base.

NumPy (Numerical Python) is a base library for scientific calculations in the Python language. It provides high performance multi-dimensional array objects (ndarray) and functions for manipulating arrays, as well as tools for mathematical, logical, statistical, and linear algebraic operations on arrays.

S206, under the condition that the answer text is correct, comparing the answer text semantic analysis result of the next question with the target knowledge semantic analysis result;

the text and target knowledge for the next question is semantically analyzed using natural language processing techniques and related tools and libraries, such as word embedding models, text classification algorithms, semantic similarity calculations, and the like.

Semantic analysis is performed to obtain semantic representations of the next question answer text and target knowledge, wherein the semantic representations of the answer text and the target knowledge can be word vectors, sentence vectors or other representation forms.

The semantic representation of the answer text of the next question is compared with the semantic representation of the target knowledge using a similarity calculation method. Common similarity calculation methods include cosine similarity, euclidean distance, manhattan distance, and the like.

And judging whether the answer of the next question is correct or not by comparing the results of the similarity. If the similarity is higher, the answer is considered to be correct when the answer is semantically matched with the target knowledge, and the text semantic analysis result of the next question is compared with the target knowledge semantic analysis result.

S207, marking the corresponding questions as wrong questions when the answer text is incorrect;

and judging whether the answer of the next question is correct or not by comparing the results of the similarity. If the similarity is low, which may mean that there is a large difference in semantics between the answer and the target knowledge, the answer may be considered incorrect, and the corresponding question is marked as a wrong question.

S208, after the comparison of the semantic analysis results is completed for all the questions in the test paper, obtaining a modification suggestion of the answer text according to the target knowledge semantic analysis results of the questions;

by comparing the semantic representation of the answer text to the semantic representation of the target knowledge by calculating a similarity, distance, or other semantic matching index, possible errors or inconsistencies in the answer text are identified.

The detected errors or inconsistencies are classified in order to provide corresponding modification suggestions for different types of errors. For example, errors may include conceptual errors, logical errors, fact errors, and the like. By classifying errors, a corresponding suggestion may be provided for each error type more accurately.

According to the error classification result, providing corresponding modification suggestions for each error type, wherein the modification suggestions comprise concepts for correcting errors, providing correct arguments or evidences, logical reasoning for correcting errors and the like, and the modification suggestions can be personalized according to specific error types and knowledge fields.

S209, evaluating all questions and answering situations of students in the test paper to obtain difficulty evaluation of the test paper and level evaluation of the students;

calculating the average score, the score rate (the proportion of the number of scored persons to the total number of scored persons) and the like of each question, wherein a lower score rate indicates that the questions are relatively difficult, and a higher score rate indicates that the questions are relatively easy. The types of the questions (selection questions, filling questions, solving questions and the like) and the difficulty level of the related knowledge points are analyzed to consider the difficulty difference between different types and knowledge points, and the overall difficulty of the test paper is comprehensively evaluated.

The average score, score rate, etc. of the student population is calculated. A higher average score and score indicates a higher overall student level, and a lower score indicates a lower overall student level. The scoring of students on different question types and knowledge points is analyzed to help determine the level of the students' mastery in different areas, thereby more accurately assessing their level.

S210, displaying the modification suggestion, the difficulty evaluation and the level evaluation through a display screen of the AR glasses;

and establishing connection with the AR glasses, and displaying the modification suggestion, the difficulty evaluation and the level evaluation on a display screen of the AR glasses through a remote control function.

S211, extracting word stems or restoring word shapes of the wrong text to obtain a standardized wrong text;

stem extraction (Stemming) is a process of converting words into their basic stem or root form, typically by deleting the suffix of the word, to obtain a shared stem.

For example, for the verb "running," stem extraction may convert it to "run"; for the noun "cat," stem extraction may convert it to "cat.

Morphological reduction (Lemmatization) is the process of reducing words to their original morphology or root. Unlike stem extraction, the result of morphological reduction is a true word form, taking into account the word context and parts of speech to ensure that the resulting word is semantically correct.

For example, for the verb "running," the morphological reduction may reduce it to "run"; for the noun "cat", the morphological reduction may reduce it to "cat".

And obtaining the standardized wrong text through stem extraction and morphological reduction of the wrong text.

S212, determining a wrong topic vocabulary according to the standardized wrong topic text;

firstly, word segmentation is carried out on standardized wrong texts, and the texts are divided into single words; then, a wrong topic vocabulary can be constructed according to the word segmentation result.

S213, inputting the wrong topic vocabulary into a transducer model to obtain topics with the same types of wrong topics.

The wrong vocabulary is formed into a text sequence according to a certain rule or order, and space or other separators can be used for separating the vocabularies. The text sequence is provided as input data to a transducer model and the generation process is run. The specific generation method and parameter settings will vary depending on the transducer model used. One common approach is to use an autoregressive (autoregressive) approach to generate the next vocabulary from a given input text, building up a new sequence of topics.

In the embodiment of the application, the AR glasses record the small problem solving process and related data, so that the data can be transmitted to the server for analysis, and are used for generating a learning report and personalized learning recommendation, so that the problem of pertinence learning and coaching is effectively solved, and the learning effect is further improved.

The server in the embodiment of the present application is described below from the viewpoint of a module. Fig. 3 is a schematic structural diagram of a functional module of a server according to an embodiment of the present application.

The server includes: the acquisition module (301) is used for acquiring the question text and the answer text in the test paper through the AR glasses; the vectorization module (302) is used for vectorizing the topic text to obtain a topic vector; the comparison module (303) is used for comparing the similarity between the topic vector and the knowledge vector in the preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector; a first determining module (304) configured to determine target knowledge corresponding to the target knowledge vector; the second determining module (305) is used for respectively inputting the target knowledge and the answer text into the transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; and a third determining module (306) for comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct.

In some embodiments, the server further includes a test paper determination module, specifically configured to:

determining a question area and a answer area in the paper area;

Generating an AR test paper according to the selection of a user, wherein the AR test paper is a virtual test paper generated by adopting an enhanced display technology and comprises questions extracted from a preset question bank;

and displaying the AR test paper in the paper area, and in the question area and the answer area in the paper area through the display screen of the AR glasses.

In some embodiments, the server further comprises a knowledge vectorization module, specifically configured to:

inputting knowledge text in a preset knowledge base into a preset word embedding model to obtain a knowledge word vector;

and calculating the average value of the knowledge word vectors to obtain the knowledge vectors in the preset knowledge base.

In some embodiments, the vectorization module is specifically configured to:

inputting the topic text into a preset word embedding model to obtain a topic word vector;

and calculating the average value of the topic word vectors to obtain the topic vector.

In some embodiments, the server further comprises a processing module, specifically configured to:

under the condition that the answer text is correct, comparing the answer text semantic analysis result of the next question with the target knowledge semantic analysis result;

and in the case that the answer text is incorrect, marking the corresponding question as a wrong question.

In some embodiments, the processing module is specifically configured to:

After determining that all questions in the test paper are compared with the semantic analysis results, obtaining a modification suggestion of the answer text according to the target knowledge semantic analysis results of the questions;

evaluating all questions and answering conditions of students in the test paper to obtain difficulty evaluation of the test paper and level evaluation of the students;

and displaying the modification advice, the difficulty evaluation and the level evaluation through a display screen of the AR glasses.

In some embodiments, the processing module is specifically further configured to:

extracting word stems or restoring word shapes of the wrong text to obtain a standardized wrong text;

determining a wrong topic vocabulary according to the standardized wrong topic text;

and inputting the wrong topic vocabulary into a transformer model to obtain the topics with the same types of wrong topics.

The server in the embodiment of the present application is described above from the point of view of the modularized functional entity, and the server in the embodiment of the present application is described below from the point of view of hardware processing, please refer to fig. 4, which is a schematic structural diagram of an entity device of the server in the embodiment of the present application.

It should be noted that the structure of the server shown in fig. 4 is only an example, and should not limit the functions and the application scope of the embodiments of the present invention.

As shown in fig. 4, the server includes a central processing unit (Central Processing Unit, CPU) 401 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 402 or a program loaded from a storage section 408 into a random access Memory (Random Access Memory, RAM) 403. In the RAM 403, various programs and data required for the system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other by a bus 404. An Input/Output (I/O) interface 405 is also connected to bus 404.

The following components are connected to the I/O interface 405: an input section 406 including a camera or the like; an output portion 407 including a liquid crystal display (Liquid Crystal Display, LCD), a display, and the like; a storage section 408 including a hard disk or the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. The drive 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

In particular, according to embodiments of the present invention, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 409 and/or installed from the removable medium 411. When executed by a Central Processing Unit (CPU) 401, the computer program performs various functions defined in the present invention.

It should be noted that, the computer readable medium shown in the embodiments of the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Specifically, the server of the present embodiment includes a processor and a memory, where the memory stores a computer program, and when the computer program is executed by the processor, the auxiliary learning method based on the combination of AR glasses analysis and transducer provided in the foregoing embodiment is implemented.

As another aspect, the present invention also provides a computer-readable storage medium, which may be contained in the server described in the above embodiment; or may exist alone without being assembled into the server. The storage medium carries one or more computer programs which, when executed by a processor of the server, cause the server to implement the methods provided in the embodiments described above.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

As used in the above embodiments, the term "when …" may be interpreted to mean "if …" or "after …" or "in response to determination …" or "in response to detection …" depending on the context. Similarly, the phrase "at the time of determination …" or "if detected (a stated condition or event)" may be interpreted to mean "if determined …" or "in response to determination …" or "at the time of detection (a stated condition or event)" or "in response to detection (a stated condition or event)" depending on the context.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, from a website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc.

Those of ordinary skill in the art will appreciate that implementing all or part of the above-described method embodiments may be accomplished by a computer program to instruct related hardware, the program may be stored in a computer readable storage medium, and the program may include the above-described method embodiments when executed. And the aforementioned storage medium includes: ROM or random access memory RAM, magnetic or optical disk, etc.

Claims

1. An AR glasses analysis and transducer combined assisted learning method, comprising:

acquiring a question text and a answer text in the test paper through AR glasses;

vectorizing the topic text to obtain a topic vector;

comparing the similarity of the topic vector with the knowledge vector in a preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector;

determining target knowledge corresponding to the target knowledge vector;

respectively inputting the target knowledge and the answer text into a transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result;

and comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct.

2. The method of claim 1, wherein prior to the step of capturing the question text and the answer text in the test paper via the AR glasses, the method further comprises:

determining a paper area, and a question area and a question answering area in the paper area;

and displaying the AR test paper in the paper area and the question area and the answer question area in the paper area through a display screen of the AR glasses.

3. The method of claim 1, wherein prior to the step of capturing the question text and the answer text in the test paper via the AR glasses, the method further comprises:

and calculating the average value of each knowledge word vector to obtain the knowledge vector in the preset knowledge base.

4. The method according to claim 2, wherein the vectorizing the topic text to obtain a topic vector specifically comprises:

5. The method of claim 1, wherein after the step of comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct, the method further comprises:

and marking the corresponding questions as wrong questions under the condition that the answer text is incorrect.

6. The method of claim 5, wherein after the step of comparing the text semantic analysis result of the next question with the target knowledge semantic analysis result, the method further comprises:

and displaying the modification suggestion, the difficulty evaluation and the level evaluation through a display screen of the AR glasses.

7. The method of claim 5, wherein, in the event that the answer text is incorrect, the method further comprises, after the step of marking the corresponding question as a wrong question:

8. A server, comprising:

the acquisition module is used for acquiring the question text and the answer text in the test paper through the AR glasses;

the vectorization module is used for vectorizing the topic text to obtain a topic vector;

the comparison module is used for comparing the similarity between the topic vector and the knowledge vector in the preset knowledge base to obtain a target knowledge vector with the highest similarity with the topic vector;

the first determining module is used for determining target knowledge corresponding to the target knowledge vector;

the second determining module is used for respectively inputting the target knowledge and the answer text into a transducer model to obtain a target knowledge semantic analysis result and an answer text semantic analysis result;

And the third determining module is used for comparing the target knowledge semantic analysis result with the answer text semantic analysis result to determine whether the answer text is correct or not.

9. A server, comprising: one or more processors and memory;

the memory is coupled to the one or more processors, the memory for storing computer program code comprising computer instructions that the one or more processors invoke to cause the server to perform the method of any of claims 1-7.

10. A computer readable storage medium comprising instructions which, when run on a server, cause the server to perform the method of any of claims 1-7.