WO2021073332A1

WO2021073332A1 - Method and apparatus for assisting maths word problem

Info

Publication number: WO2021073332A1
Application number: PCT/CN2020/115178
Authority: WO
Inventors: 何涛; 罗欢; 陈明权
Original assignee: 杭州大拿科技股份有限公司
Priority date: 2019-10-16
Filing date: 2020-09-15
Publication date: 2021-04-22
Also published as: CN110675677A

Abstract

A method for assisting a maths word problem and an apparatus (800) for assisting a maths word problem. The method comprises: identifying, from an image comprising an answered maths word problem, a first region comprising a question of the maths word problem and a second region comprising a given answer to the maths word problem (601); identifying the question on the basis of the first region, and identifying the given answer on the basis of the second region; performing equivalent conversion on the given answer and/or the standard answer to the maths word problem to convert the corresponding expressions in the given answer and the standard answer into the same character form, the equivalent conversion being performed according to a pre-established equivalent correspondence; matching the given answer with the standard answer; and if the matching result meets a condition, determining that the given answer is correct (604); if the matching result does not meet the condition, determining that the given answer is wrong (605).

Description

Method and device for assisting mathematical application problems

Technical field

The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for assisting mathematical application problems.

Background technique

In recent years, artificial intelligence has been used in daily teaching and learning. For example, using electronic devices such as smart terminals to correct test papers or homework questions.

Therefore, there is a demand for new technologies.

Summary of the invention

An object of the present disclosure is to provide a method and device for assisting mathematical application problems.

According to a first aspect of the present disclosure, there is provided a method for assisting math word problems, including: identifying a first area of a problem including the math word problem from an image including the answered math word problem, and The second area including the answer to the math application question; the question is identified based on the first area, and the answer answer is identified based on the second area; the answer to the answer and/or the math application The standard answer to the question is equivalently converted to convert the corresponding expression in the answer answer and the standard answer into the same character form, and the equivalent conversion is performed according to the pre-established equivalent correspondence; and The answer answer is matched with the standard answer. If the matching result meets the condition, it is determined that the answer answer is correct; if the matching result does not meet the condition, it is determined that the answer answer is wrong.

According to a second aspect of the present disclosure, there is provided an apparatus for assisting mathematical application problems, including: one or more processors; and one or more memories configured to store a series of Computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions, wherein, when the series of computer-executable instructions are executed by the one or more processors, cause The one or more processors perform the method as described above.

According to a third aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium, wherein a series of computer-executable instructions are stored on the non-transitory computer-readable storage medium. When the computer-executable instructions are executed by one or more computing devices, the one or more computing devices perform the methods described above.

Through the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, other features and advantages of the present disclosure will become clear.

Description of the drawings

The drawings constituting a part of the specification describe the embodiments of the present disclosure, and together with the specification, serve to explain the principle of the present disclosure.

With reference to the accompanying drawings, the present disclosure can be understood more clearly according to the following detailed description, in which:

FIG. 1A is a schematic diagram schematically showing an image suitable for a method for assisting mathematical word problems according to an embodiment of the present disclosure.

FIG. 1B is a schematic diagram schematically showing a screen displayed by the method for assisting mathematical application problems according to an embodiment of the present disclosure.

FIG. 2A is a diagram schematically showing an image suitable for a method for assisting mathematical word problems according to an embodiment of the present disclosure.

2B and 2C are schematic diagrams respectively schematically showing screens displayed by the method for assisting mathematical application problems according to an embodiment of the present disclosure.

Fig. 3 is a flowchart schematically showing at least a part of a method for assisting mathematical word problems according to an embodiment of the present disclosure.

Fig. 4 is a structural diagram schematically showing at least a part of an apparatus for assisting mathematical word problems according to an embodiment of the present disclosure.

FIG. 5 is a structural diagram schematically showing at least a part of an apparatus for assisting mathematical application problems according to an embodiment of the present disclosure.

Note that in the embodiments described below, the same reference numerals are sometimes used in common between different drawings to denote the same parts or parts having the same functions, and repeated descriptions thereof are omitted. In this specification, similar reference numerals and letters are used to indicate similar items. Therefore, once a certain item is defined in one figure, it does not need to be further discussed in subsequent figures.

Detailed ways

Hereinafter, various exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that unless specifically stated otherwise, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure. In the following description, in order to better explain the present disclosure, many details are set forth. However, it is understandable that the present disclosure can also be practiced without these details.

The following description of at least one exemplary embodiment is actually only illustrative, and in no way serves as any limitation to the present disclosure and its application or use. In all examples shown and discussed herein, any specific value should be interpreted as merely exemplary, rather than as a limitation.

The technologies, methods, and equipment known to those of ordinary skill in the relevant fields may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be regarded as part of the specification.

The present disclosure provides a method for assisting mathematical application problems, which can be used for teaching and learning, for example. The user can use the first electronic device with the image acquisition function to take photos or videos of the math application problem that needs assistance to obtain the image of the problem, as shown in Figure 1A, and then can use the second electronic device with the display function (the first The second electronic device can be the same device or a different device) to check whether the answer to the mathematical application question is correct, as shown in FIG. 1B. In addition, the second electronic device can also display the question of the mathematical application problem (it can display the recognized text and/or picture form of the question, and can also display the obtained image of the question), the answer, the standard answer, and the answer. Problem solving process, and standard problem solving process, etc. In some embodiments, the method can assist multiple mathematical word problems together. The user can use the first electronic device to shoot an image including multiple mathematical application questions, as shown in FIG. 2A, and then check whether the answers to the multiple mathematical application questions are correct on the second electronic device, as shown in FIG. 2B. In some embodiments, for a question whose answer is incorrect, the second electronic device may also display the standard answer of the question, as shown in FIG. 2C. The "mathematics application problem" referred to in this article refers to a mathematical problem in which known conditions are given by words and/or pictures to solve the answer. Those skilled in the art should understand that the answering form of the "mathematics application problem" referred to in this article is not limited to the narration of the answer in words as shown in Figures 1A to 2C. The answering form can also include the use of graphics/pictures, calculations/vertical expressions, etc. Give the answer, and answer with the code of the option (such as A, B, C, etc.).

The following describes a method for assisting mathematical application problems according to an embodiment of the present disclosure with reference to FIG. 3. A method for assisting mathematical word problems according to some embodiments includes a process 600, which includes steps 601 to 605.

Step 601: Identify the first area (for example, the part of the printed font in the image 100, and The picture part), and the second area (for example, the handwritten font part in the image 100) that includes the answer to the math word problem. Mathematical application problems can be presented on the first surface. The first surface can include paper (such as test papers, books or brochures, etc.), whiteboards, chalk boards, display screens (such as TV screens, computer screens, tablet screens or learning machines). Screen, etc.) or various other surfaces. The image 100 of the mathematical problem presented on the first surface can be obtained by the image acquisition device in the first electronic device. Images can include any form of visual presentation, such as photos or videos. The image acquisition device may include a camera, an imaging module, an image processing module, etc., and may also include a communication module for receiving or downloading images. Correspondingly, image acquisition by the image acquisition device may include taking photos or videos, receiving or downloading photos or videos, and so on. A computing device and a pre-trained region recognition model can be used to identify the first region including the question and the second region including the answer in the image 100. The first and second areas are image areas divided from the image 100. The region recognition model used to recognize the first region and the second region may be the same or different. The region recognition model can be obtained based on any known neural network training, such as deep residual network, recurrent neural network, and so on.

Step 602: Identify the question of the mathematical application problem based on the first region, and obtain the standard answer of the mathematical application problem based on the pre-established question bank. A computing device and a pre-trained character recognition model can be used to recognize the problem of the mathematical application problem from the first area, that is, to recognize the text and/or picture in the problem. For example, for the image 100, the text "4. Xiao Ming walked for 3 minutes is exactly half of the whole journey, how many meters is his home from the school? (6 points)" and the picture located at the lower right of the image 100 are recognized from the first area. It should be understood that the characters referred to in this article include text (including text, graphic text, letters, numbers, symbols, etc.) and pictures. The character recognition model can be obtained based on any known neural network training, such as a deep residual network, a recurrent neural network, and so on.

Obtaining the standard answer to the mathematical application question based on the pre-established question bank may include: extracting features of the question to generate a feature vector (such as a two-dimensional feature vector), searching for a matching vector matching the feature vector from the question bank, and according to the question bank The answer associated with the matching vector generates a standard answer. The feature vector can be generated using a vectorized model known in the art. For example, for the text in the title, known vectorized models such as word2vec, doc2vec, and sentence2vec can be used to generate the first feature vector. For the pictures in the title, for example, you can use the trained network model to perform feature extraction under the caffe framework, or you can use the vgg convolutional neural network based on the image data set ImageNet pre-trained model to perform feature extraction, etc., to generate the first Two eigenvectors. Join the first and second feature vectors to get the feature vector of the topic. Those skilled in the art should understand that in addition to the methods listed above, the first and second feature vectors can also be generated by other known vectorization models, or they can be generated based on known data sets and neural networks. A vectorized model obtained by a known training method is generated.

Searching from the question bank for a matching vector that matches the feature vector of the math word problem can be searched by vector approximation. For example, find the feature vector with the closest distance to the feature vector of the problem targeted by the method in the question bank. It is understandable that the method that can be used to measure the similarity between different vectors is to calculate the "distance" between the vectors, for example, Euclidean distance, Manhattan distance, angle cosine similarity, etc. The question bank can include the feature vector of each question, the content of the question (referring to the character form of the question), the answer corresponding to the question, the problem solving process corresponding to the question, and the identification of the test paper to which the question belongs. The above-mentioned data in the question bank can be stored in any form that is convenient for searching. For example, it can be grouped and stored according to the length of the feature vector of each topic (for example, the length of the feature vector of different length topics may also be different), and the length of the feature vector of the topics in each group is the same or all within the same range . Searching can first find a group with matching length in the question bank according to the length of the feature vector of the problem targeted; then find the feature vector that is closest to the feature vector of the problem targeted in this group. In this way, a matching vector that matches the feature vector of the problem can be searched more quickly.

You can search the question bank for the first vector with the highest similarity to the feature vector of the topic of the mathematical application problem ("the subject"), if the similarity between the first vector and the feature vector is not less than the preset first threshold (That is, the similarity meets the requirement, for example, it may be that the distance between the first vector and the feature vector is less than a certain preset threshold), then the first vector is determined as a matching vector that matches the feature vector of the subject. After the matching vector is obtained, the question corresponding to the matching vector (also referred to as the "reference question" in this article) can be obtained according to the question bank, and the standard answer is generated according to the answer corresponding to the reference question. The reference topic can be compared with the topic of the problem targeted by the method. For example, the comparison can be a character-level comparison (such as a text comparison performed by a known text comparison tool) to determine one or more of the reference topics Whether the quantity is the same as one or more corresponding quantities in the subject of the question. If they are the same, the answer to the reference question (that is, the answer associated with the matching vector) is directly used as the standard answer to the question; The corresponding one or more numbers in the question of the question are used to calculate the standard answer to the question.

If the similarity of the first vector with the highest similarity to the feature vector of the question searched from the question bank does not meet the requirements, that is, the similarity of all vectors in the question bank to the feature vector is less than the first threshold, then search for One or more similar questions for the question. For example, search for one or more candidate questions whose distance from the feature vector of the target question is not greater than the preset second threshold from the question bank; match the target question with one or more candidate questions respectively, and follow Sort one or more candidate questions from high to low matching degree to get one or more similar questions of the targeted questions. In view of the situation that no matching vector is found, in the subsequent step of displaying whether the answer to the mathematical application problem is correct on the second electronic device with a display function, it is not necessary to display whether the answer to the mathematical application problem is correct. For example, a special prompt can be displayed for the mathematical application problem (for example, the area of the mathematical application problem is framed by a box), when the relevant area of the mathematical application problem is operated (for example, the area framed by the box is operated by the user). Time), displays the problem-solving process and/or answers of one or more similar problems. It should be understood that the problem-solving process and/or answers for one or more similar questions come from the question bank.

Step 603: Identify the answer based on the second area and match the answer with the standard answer; Step 604: If the matching result meets the condition, it is determined that the answer answer is correct; Step 605: If the matching result does not meet the condition, it is determined as The answer is wrong. A computing device and a pre-trained character recognition model can be used to recognize the answer to the mathematical word problem from the second area, that is, to recognize the text and/or picture in the answer. For example, for the image 100, the characters "65×3=195 (meters)" and "195×2=390 (meters)" are recognized from the second area. The character recognition model can be obtained based on any known neural network training, such as a deep residual network, a recurrent neural network, and so on. It should be understood that the character recognition model used to recognize the answer from the second area and the aforementioned character recognition model used to recognize the question from the first area may be the same model, different models with the same structure/feature, or structure/ Different models with different characteristics. In some embodiments, considering that the font of the general question is printed, and the font of the answer is handwritten, the models used to identify the question and the answer may be different models that are trained separately. In some embodiments, recognizing the answer based on the second area may be that all characters recognized from the second area are used as the answer answer. For example, for the image 100, “65×3=195 (meters)” and “195×2=390 (meters)” can be recognized as answer answers. In some embodiments, the answer based on the recognition of the second area may be the following specific characters (for example, punctuation marks such as ":", relational symbols such as "=" and ">", etc.) recognized from the second area. All characters or part of characters are used as the answer. For example, for the image 100, "195 (meters)" and "390 (meters)" may be recognized as answer answers, or only "195" and "390" may be recognized as answer answers. The method described in the present disclosure can provide users with choices. For example, the user can choose to only check whether the quantity is correct, then only "195" and "390" can be recognized as answer answers, and in the subsequent steps, as long as the number of matches is correct, it is judged as answer The answer is correct; the user can also choose to check whether the quantity and unit are correct at the same time, then "195 (meter)" and "390 (meter)" can be recognized as answer answers, and the number and unit need to be matched in the subsequent steps are correct and there is no omission , It is judged that the answer is correct. It should be understood that how to identify the strategy of answering answers can also be determined to be compatible with the strategy of determining standard answers stored when the question bank is established.

According to the characteristics of the answers to the mathematical application questions, matching the answer answers with the standard answers can adopt different matching methods for different answer types. For example, if the standard answer contains one or more quantities, matching the answer answer with the standard answer can include finding all one or more quantities contained in the standard answer in the answer answer. If the answer includes one or more of these numbers, the matching result is considered to meet the conditions. This matching method can be aimed at questions whose answers are focused on calculation results. For example, for the image 100, in some embodiments, the standard answer includes two numbers "195" and "390", then these two numbers are matched in the answer answer. As long as the answer contains these two numbers, regardless of whether it contains other characters, the matching result is considered to meet the conditions. The expression "quantity" is used here instead of "number" because at least one of the two numbers "195" and "390" can also be expressed as words, such as "one hundred and ninety-five" or " "Three hundred and ninety" can also be expressed as a figure, such as a figure of the corresponding number, or a figure of the corresponding number in the hundreds, tens, and units digits. For these different expressions of quantity, the matching can be improved through the process of equivalent conversion as described below.

For another example, if the standard answer does not contain a quantity, matching the answer answer with the standard answer can include searching for keywords included in the standard answer in the answer answer, such as performing character-level matching. If a keyword is found in the answer, it is considered that the matching result meets the conditions. For example, if the standard answer is the character string "can be completed", then the keyword in the standard answer can be extracted as "able", and then the character string matching this keyword in the answer answer. If the answer contains this string, it is considered that the matching result meets the condition (even if it contains other characters). In addition, even if the entire string of keywords is matched, it is necessary to determine whether negative words, such as "no", "no", "no", "not", etc., are included before the keywords included in the standard answer. If it does not, it can be considered that the matching result meets the conditions. For example, the keywords in the standard answer are "can", "correct", "able", "reach", "complete", etc. It is also necessary to determine whether the answer contains negative words before these keywords, and only if it does not contain It is considered that the matching result meets the condition only in the case of negative words. For the case where the standard answer itself expresses a negative meaning, for example, the standard answer is a string of "cannot be completed", the keywords in the standard answer can be extracted as "unable", that is, the keywords should contain negative words, and then The string that matches this keyword in the answer. In this way, the matched answer that satisfies the conditions is also an answer that expresses a negative meaning. In addition, the aforementioned operation of judging whether negative words are included before the keywords included in the standard answer can be performed only when the standard answer does not contain negative words, or when the standard answer does not include negative words and contains Negative words are carried out in both cases.

For a question in a language expression that can have multiple forms of expression for a result, before matching the answer answer with the standard answer, you can also perform equivalent conversion of the answer answer and/or the text in the standard answer to combine the answer answer with the standard answer. The corresponding expression in the standard answer is converted to the same character form. The equivalent conversion can be performed according to the pre-established equivalent correspondence relationship. It is possible to establish equivalent correspondences for characters expressing the same or similar meanings, for example, “0.4”, “two-fifths”, “2/5” and

Between "≥" and "greater than or equal to", an equivalent corresponding relationship is established between "60 seconds", "60s", "1 minute" and "1min". The corresponding expressions in the answer answer and the standard answer (for example, "0.4" in the answer answer and "2/5" in the standard answer) can be converted to the same character form (for example, both are converted to "0.4"), and then Make a match.

In some cases, the standard answer may include multiple standard sub-answers, for example, when the question contains multiple questions. At this time, the answer may include one answer sub-answer, or it may include multiple answer sub-answers. In these cases, matching the answer answer with the standard answer may include: matching each answer sub-answer with the corresponding standard sub-answer. If all matching results meet the conditions, then it is also judged whether the number of answer sub-answers is less than If the number of standard sub-answers is less, it is judged that the answer is partially correct; otherwise, it is judged that the answer is correct. If the partial matching result meets the conditions, it is determined that the answer is partially correct. If none of the matching results meet the conditions, it is determined that the answer is wrong.

In some embodiments, determining whether the answer is correct can also consider whether the problem-solving process is correct. For example, in the above step 601, in addition to the answer to the answer, the identified second area may also include the process of solving the mathematical application problem. In the above step 603, the problem-solving process of the mathematical application problem is also identified based on the second area, and the computing device and/or model used in the problem-solving process can be identified with the computing device and/or the computing device and/or model used in the step 603 to identify the answer. The model is the same. For example, for the image 100, the identified problem-solving process can be "65×3" and "195×2". Although “65×3” and “195×2” are described as the problem-solving process here, it should be understood that in some embodiments, “65×3” and “195×2” can also be recognized as answering answers Part of the problem-solving process.

In the case where the matching result of the answer answer and the standard answer meets the conditions, it can also be judged whether the problem solving process is correct. For example, for image 100, if the answers "195 (meters)" and "390 (meters)" match the standard answers, it can also be judged whether the problem-solving process "65×3" and "195×2" are correct . If the problem-solving process is also correct, it is judged that the answer is correct; if the problem-solving process is wrong, it is judged that the answer is wrong. In some embodiments, the problem-solving process associated with the matching vector may be stored in the question bank. The standard problem-solving process may be obtained according to the process similar to the above-mentioned standard answer, and then the matching answer answer and the standard answer can be compared with the above-mentioned standard answer. The similar process matches the problem-solving process of answering and the standard problem-solving process to judge whether the process of problem-solving is correct. In some embodiments, a simpler method can be used to determine whether the problem-solving process is correct. For example, it is judged whether one or more numbers involved in the calculation included in the problem-solving process of the answer appear in the question. If at least one of the one or more numbers does not appear in the question, it is judged as an error in the problem-solving process. This situation may be the use of unrelated numbers in the problem-solving process. For another example, it is judged whether the result of the formula included in the problem-solving process is related to the answer. For another example, it is judged whether the calculation formula included in the process of solving the problem is correct (including the form of the listed formula, the operator and the calculation result), and if it is not correct, it is judged as an error in the process of solving the problem. These situations may be an arbitrarily fictitious process of solving problems. In this way, when judging whether the answer is correct or not, it is also considered whether the process of understanding the question is correct, so as to avoid judging the questions that do not really solve the problem but happen to write the correct answer as correct.

Whether the answer to the math word problem is correct can be displayed on the second electronic device with a display function, that is, the determination result is displayed in the relevant area of the math word problem in the image (for example, the display screen 200 shown in FIG. 1B). For example, "√" indicates that the answer is correct (for example, the mark 201 shown in FIG. 1B and the mark 401 shown in FIG. 2B), and "×" indicates that the answer is wrong (for example, the mark 402 shown in FIG. 2B), use

It means that the answer is partially correct (for example, the mark 403 shown in FIG. 2B). In some embodiments, when the result of the determination is that the answer is wrong, the standard answer may also be displayed. In some embodiments, only the result of the judgment is displayed in the screen 200, when the relevant area of the math word problem (for example, the area where the math word problem is located, the area around the math word problem, the mark indicating the judgment result, other specific marks, etc.) The standard answer and/or standard problem-solving process will only be displayed when it is operated (for example, tapping, tapping twice in a row, long-pressing, deep-pressing, swiping, etc.). For word questions with incorrect or partially correct answers, the method can automatically add them to the user's wrong question book, or provide an interface for the user to choose whether to add it to the wrong question book.

The method for assisting mathematical application problems according to an embodiment of the present disclosure can also assist multiple mathematical application problems together. When multiple mathematical application problems are assisted together, the process 600 described above can be used to assist each mathematical application problem. An image containing multiple mathematical application questions is acquired through the image acquisition device in the first electronic device, for example, it may be an image of all or part of a page of a test paper or a workbook. The acquired images can also include other content besides multiple mathematical application problems. Through the computing device and the area recognition model, the areas where the respective questions of the multiple mathematical application questions in the image are located and the areas where the respective answers are located are recognized. Through the computing device and the character recognition model, the characters in the above-mentioned regions are respectively recognized, so as to obtain the respective questions of the multiple mathematical application questions and the respective answers. Then, for each question's own question, search the matching vector in the question bank respectively. If a matching vector is found, a standard answer is generated based on the answer associated with the matching vector, so as to determine whether the answer to the question is correct. If the matching vector is not found in the question bank, one or more similar questions of the question are found, so that the user can present the problem-solving process and/or answers of the one or more similar questions.

Each of the above neural network models can be stored in one or more of the following items as a whole, or the first part can be stored in one or more of the following items. One storage medium, and the second part is stored on one or more storage media in any one of the following: the first and/or second electronic device, one or more remote servers, the aforementioned computing devices One or more of.

Any two of the computing devices that perform the above-mentioned steps may be the same computing device or different computing devices. Each computing device may include one or more processors, and one or more processors belonging to a computing device may be: all located in the physical housing of the first and/or second electronic device, all located in one or more In the physical housing of two remote servers, or the first part is located in the physical housing of the first and/or second electronic device and the second part is located in the physical housing of one or more remote servers. It should be understood that each of the computing devices may also include one or more memories to store instructions that can be executed by the one or more processors and data required to execute the instructions, such as the one or more neural network models described above. At least part of.

FIG. 4 is a structural diagram schematically showing at least a part of an apparatus 800 for assisting mathematical application problems according to an embodiment of the present disclosure. Those skilled in the art can understand that the device 800 is only an example and should not be considered as limiting the scope of the present disclosure or the features described herein. In this example, the apparatus 800 may include one or more neural network models 810, one or more electronic devices 820, one or more computing devices 830, one or more remote servers 840, and a network 850. Among them, one or more neural network models 810, one or more electronic devices 820, one or more computing devices 830, and one or more remote servers 840 may be connected to each other through a network 850. The network 850 may be any wired or wireless network, and may also include a cable. In addition, although one or more neural network models 810 in the device 800 are independent of one or more electronic devices 820, one or more computing devices 830, one or more remote servers 840, and a separate network 850 The box shows, it should be understood that one or more neural network models 810 may be actually stored on any one of the

other entities

820, 830, 840, 850 included in the device 800. For example, one or more computing devices may include server computing devices that operate as a load-balanced server farm. In addition, although some of the functions described above are indicated as occurring on a single computing device with a single processor, various aspects of the subject matter described herein can be implemented by multiple computing devices communicating with each other, for example, through a network.

Each of one or more electronic devices 820, one or more computing devices 830, and one or more remote servers 840 may be located at different nodes of the network 850, and can directly or indirectly communicate with other nodes of the network 850 Communication. Those skilled in the art can understand that the device 800 may also include other devices not shown in FIG. 4, where each different device is located at a different node of the network 850. Various protocols and systems can be used to interconnect the network 850 and the components in the devices described herein so that the network 850 can be part of the Internet, the World Wide Web, a specific intranet, a wide area network, or a local area network. The network 850 may utilize standard communication protocols such as Ethernet, WiFi, and HTTP, protocols that are proprietary to one or more companies, and various combinations of the foregoing protocols. Although certain advantages are obtained when transferring or receiving information as described above, the subject matter described herein is not limited to any specific information transfer method.

Each of one or more electronic devices 820, one or more computing devices 830, and one or more remote servers 840 may be configured to be similar to the device 900 shown in FIG. 5, that is, having one or more processors 910, one or more memories 920, and instructions and data. Each of the one or more electronic devices 820, the one or more computing devices 830, and the one or more remote servers 840 may be a personal computing device intended to be used by a user or a commercial computer device used by an enterprise, and has All components commonly used in combination with personal computing devices or commercial computer devices, such as a central processing unit (CPU), memory for storing data and instructions (e.g., RAM and internal hard drive), such as a display (e.g., a monitor with a screen, One or more I/O devices such as touch screens, projectors, televisions, or other devices operable to display information), mice, keyboards, touch screens, microphones, speakers, and/or network interface devices. The one or more electronic devices 820 may also include one or more cameras for capturing still images or recording video streams, and all components for connecting these elements to each other.

Although one or more electronic devices 820 may each include a full-size personal computing device, they may optionally include a mobile computing device capable of wirelessly exchanging data with a server through a network such as the Internet. For example, the one or more electronic devices 820 may be a mobile phone, or a device such as a PDA with wireless support, a tablet PC, or a netbook that can obtain information via the Internet. In another example, one or more electronic devices 820 may be wearable computing devices.

FIG. 5 is a structural diagram schematically showing at least a part of an apparatus 900 for assisting mathematical application problems according to an embodiment of the present disclosure. The device 900 includes one or more processors 910, one or more memories 920, and other components (not shown) usually present in devices such as computers. Each of the one or more memories 920 can store content that can be accessed by one or more processors 910, including instructions 921 that can be executed by one or more processors 910, and can be executed by one or more processors 910. Retrieved, manipulated, or stored data 922.

The instructions 921 may be any set of instructions to be directly executed by the one or more processors 910, such as machine code, or any set of instructions to be executed indirectly, such as a script. The terms "instruction", "application", "process", "step" and "program" in this article can be used interchangeably in this article. The instructions 921 may be stored in an object code format for direct processing by one or more processors 910, or stored in any other computer language, including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. Instructions 921 may include instructions that cause, for example, one or more processors 910 to act as various neural networks herein. The rest of this article explains the functions, methods, and routines of instruction 921 in more detail.

The one or more memories 920 may be any temporary or non-transitory computer-readable storage medium capable of storing content accessible by the one or more processors 910, such as a hard drive, memory card, ROM, RAM, DVD, CD, USB memory, writable memory and read-only memory, etc. One or more of the one or more memories 920 may include a distributed storage device, where the instructions 921 and/or data 922 may be stored on multiple different storage devices that may be physically located at the same or different geographic locations. One or more of the one or more memories 920 may be connected to the one or more processors 910 via a network, and/or may be directly connected to or incorporated into any one of the one or more processors 910.

One or more processors 910 may retrieve, store, or modify data 922 according to instructions 921. The data 922 stored in the one or more memories 920 may include the various images to be recognized, the question bank, and the parameters used for each neural network as described above. Other data not associated with images, question banks, or neural networks may also be stored in one or more memories 920. For example, although the subject described in this article is not restricted by any specific data structure, the data 922 may also be stored in a computer register (not shown), as a table or XML document with many different fields and records stored in the relationship Type database. The data 922 may be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. In addition, the data 922 may include any information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories such as other network locations, or used by functions to calculate correlations. Data information.

The one or more processors 910 may be any conventional processors, such as a central processing unit (CPU), a graphics processing unit (GPU), etc., which are commercially available. Alternatively, the one or more processors 910 may also be dedicated components, such as an application specific integrated circuit (ASIC) or other hardware-based processors. Although not required, one or more processors 910 may include dedicated hardware components to perform specific calculation processes faster or more efficiently, such as image processing on images.

Although one or more processors 910 and one or more memories 920 are schematically shown in the same frame in FIG. 5, the apparatus 900 may actually include multiple processors that may exist in the same physical housing or different Multiple processors or memories in a physical housing. For example, one of the one or more memories 920 may be a hard disk drive or other storage located in a housing different from the housing of each of the one or more computing devices (not shown) described above. medium. Therefore, references to processors, computers, computing devices, or memories should be understood to include references to collections of processors, computers, computing devices, or memories that may or may not operate in parallel.

The term "A or B" in the specification and claims includes "A and B" and "A or B", and does not exclusively include only "A" or only "B" unless specifically stated otherwise.

In the present disclosure, reference to "one embodiment" or "some embodiments" means that a feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment or at least some embodiments of the present disclosure. Therefore, the appearances of the phrases "in one embodiment" and "in some embodiments" in various places in this disclosure do not necessarily refer to the same or the same embodiments. In addition, in one or more embodiments, any suitable combination and/or sub-combination may be used to combine features, structures, or characteristics.

As used herein, the word "exemplary" means "serving as an example, instance, or illustration" and not as a "model" to be copied exactly. Any implementation described exemplarily herein is not necessarily construed as being preferred or advantageous over other implementations. Moreover, the present disclosure is not limited by any expressed or implied theory given in the above technical field, background art, summary of the invention, or specific embodiments.

As used herein, the word "substantially" means to include any minor changes caused by design or manufacturing defects, device or component tolerances, environmental influences, and/or other factors. The word "substantially" also allows for differences between the perfect or ideal situation caused by parasitic effects, noise, and other practical considerations that may be present in the actual implementation.

The foregoing description may indicate elements or nodes or features that are "connected" or "coupled" together. As used herein, unless expressly stated otherwise, "connected" means that one element/node/feature is electrically, mechanically, logically, or otherwise directly connected (or Direct communication). Similarly, unless expressly stated otherwise, "coupled" means that one element/node/feature can be directly or indirectly connected to another element/node/feature mechanically, electrically, logically, or in other ways. Interaction is allowed, even if the two features may not be directly connected. In other words, "coupled" intends to include direct connection and indirect connection of elements or other features, including the connection of one or more intermediate elements.

In addition, for reference purposes only, certain terminology may also be used in the following description, and thus is not intended to be limiting. For example, unless clearly indicated by the context, the words "first", "second" and other such numerical words referring to structures or elements do not imply an order or order.

It should also be understood that when the term "including/comprising" is used in this text, it indicates that the specified features, wholes, steps, operations, units and/or components are present, but it does not exclude the presence or addition of one or more other features, Whole, steps, operations, units and/or components and/or combinations thereof.

In the present disclosure, the terms "component" and "device" are intended to refer to an entity related to a computer, or hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, an object, an executable state, an execution thread, and/or a program running on the processor. By way of example, both an application program running on a server and the server may be one component. One or more components may exist within an executing process and/or thread, and a component may be located on one computer and/or distributed between two or more computers.

Those skilled in the art should realize that the boundaries between the above operations are merely illustrative. Multiple operations can be combined into a single operation, a single operation can be distributed in additional operations, and the operations can be executed at least partially overlapping in time. Also, alternative embodiments may include multiple instances of specific operations, and the order of operations may be changed in other various embodiments. However, other modifications, changes and replacements are also possible. Therefore, this specification and drawings should be regarded as illustrative rather than restrictive.

In addition, the embodiments of the present disclosure may also include the following examples:

1. A method for assisting mathematical application problems, including:

Identifying a first area including a question of the mathematical application problem and a second area including an answer to the mathematical application problem from the image including the mathematical application problem that has been answered;

Identifying the question based on the first area, and identifying the answer answer based on the second area;

An equivalent conversion is performed on the answer answer and/or the standard answer of the math word problem to convert the corresponding expression in the answer answer and the standard answer into the same character form, and the equivalent conversion is based on the previous The established equivalent correspondence relationship; and

Matching the answer answer with the standard answer, if the matching result meets the condition, it is determined that the answer answer is correct; if the matching result does not meet the condition, it is determined that the answer answer is wrong.

2. The method according to 1, characterized in that the standard answer includes a plurality of standard sub-answers, and the answer answer includes one or more answer sub-answers, wherein matching the answer answer with the standard answer includes : Match each of the answer sub-answers with the corresponding standard sub-answers,

If all matching results meet the conditions, it is also judged whether the number of answering sub-answers is less than the number of standard sub-answers, if it is less, it is judged that the answering answer is partially correct; otherwise, it is judged as said The answer is correct;

If the partial matching result meets the condition, it is determined that the answer is partially correct; and

If none of the matching results meet the conditions, it is determined that the answer is wrong.

3. The method according to 1, wherein if the standard answer contains one or more numbers, the matching includes searching for the one or more numbers in the answer answer, if the answer answer If the one or more quantities are included in the, the matching result satisfies the condition.

4. The method according to 1, characterized in that, if the standard answer does not include a quantity, the matching includes searching for keywords included in the standard answer in the answer answer, and if the answer is in the answer answer If the keyword is found in, the matching result meets the condition.

5. The method according to 4, characterized in that the matching further comprises: if the keyword is found in the answer answer, further searching for whether there is a negative word before the keyword in the answer answer , If there is no negative word, the matching result meets the condition.

6. The method according to 1, wherein the second area further includes a problem-solving process of the mathematical application problem, and the problem-solving process is also identified based on the second area, and the method further includes:

If the matching result satisfies the condition, it is also judged whether the problem-solving process is correct, if the problem-solving process is also correct, it is judged that the answer is correct; if the problem-solving process is wrong, it is judged as the answer The answer is wrong.

7. The method according to 6, characterized in that judging whether the problem-solving process is correct includes:

Determine whether one or more numbers involved in the calculation included in the problem-solving process appear in the question. If at least one of the one or more numbers does not appear in the question, it is judged as the The problem-solving process is wrong;

Determine whether the result of the formula included in the problem-solving process is related to the answer to the answer, if not, it is determined that the problem-solving process is wrong;

Determine whether the calculation formula included in the problem-solving process is correct, and if it is incorrect, it is determined that the problem-solving process is wrong;

Otherwise, it is judged that the problem-solving process is correct.

8. The method according to 1, wherein recognizing the answer answer based on the second area includes: recognizing all characters in the second area, and using all characters after a specific character as the answer answer .

9. The method according to 1, further comprising: displaying the result of the determination in a relevant area of the mathematical word problem in the image.

10. The method according to 9, further comprising: when the result of the determination is that the answer is wrong, further displaying the standard answer.

11. The method according to 9, characterized in that when the relevant area of the mathematical word problem is operated, the standard answer is displayed.

12. The method according to 11, characterized in that when the relevant area of the mathematical application problem is operated, the standard problem-solving process of the mathematical application problem is also displayed.

13. The method according to 1, characterized in that obtaining the standard answer to the mathematical application question based on a pre-established question bank includes:

Perform feature extraction on the question to generate a feature vector, search for a matching vector matching the feature vector from the question bank, and generate the standard answer based on the answer associated with the matching vector in the question bank .

14. The method according to 13, wherein searching for a matching vector that matches the feature vector from the question bank includes: calculating the similarity between all vectors in the question bank and the feature vector, if the first The similarity between the vector and the feature vector is not less than a preset first threshold, then the first vector is determined as the matching vector.

15. The method according to 14, further comprising: if the similarity between all vectors in the question bank and the feature vector is less than the first threshold, then:

Searching from the question bank for one or more candidate questions whose vector and the feature vector of the first question are not greater than a preset second threshold;

The first question is matched with the one or more candidate questions, and the one or more candidate questions are sorted according to the degree of matching from high to low to obtain one or more of the first questions. A similar topic; and

When the relevant area of the first question is operated, the problem-solving process and/or answer of the one or more similar questions are displayed, wherein the problem-solving process and/or answer of the one or more similar questions comes from The question bank.

16. A device for assisting mathematical application problems, including:

One or more processors; and

One or more memories configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions,

Wherein, when the series of computer-executable instructions are executed by the one or more processors, the one or more processors are caused to perform the method described in any one of 1-15.

17. A non-transitory computer-readable storage medium, wherein a series of computer-executable instructions are stored on the non-transitory computer-readable storage medium. When executed by multiple computing devices, the one or more computing devices are caused to perform the method described in any one of 1-15.

Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present disclosure. The various embodiments disclosed herein can be combined arbitrarily without departing from the spirit and scope of the present disclosure. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

A method used to assist mathematical word problems, including:

Identifying a first area including a question of the mathematical application problem and a second area including an answer to the mathematical application problem from the image including the mathematical application problem that has been answered;

Identifying the question based on the first area, and identifying the answer answer based on the second area;

An equivalent conversion is performed on the answer answer and/or the standard answer of the math word problem to convert the corresponding expression in the answer answer and the standard answer into the same character form, and the equivalent conversion is based on the previous The established equivalent correspondence relationship; and

Matching the answer answer with the standard answer, if the matching result meets the condition, it is determined that the answer answer is correct; if the matching result does not meet the condition, it is determined that the answer answer is wrong.
The method according to claim 1, wherein the standard answer includes a plurality of standard sub-answers, and the answer answer includes one or more sub-answers, wherein matching the answer answer with the standard answer includes : Match each of the answer sub-answers with the corresponding standard sub-answers,

If all matching results meet the conditions, it is also judged whether the number of answering sub-answers is less than the number of standard sub-answers, if it is less, it is judged that the answering answer is partially correct; otherwise, it is judged as said The answer is correct;

If the partial matching result meets the condition, it is determined that the answer is partially correct; and

If none of the matching results meet the conditions, it is determined that the answer is wrong.
The method according to claim 1, wherein if the standard answer contains one or more numbers, the matching includes searching for the one or more numbers in the answer answer, and if the answer answer is If the one or more quantities are included in the, the matching result satisfies the condition.
The method according to claim 1, wherein if the standard answer does not include a quantity, the matching includes searching for keywords included in the standard answer in the answer answer, and if the answer is in the answer answer If the keyword is found in, the matching result meets the condition.
The method according to claim 4, wherein the matching further comprises: if the keyword is found in the answer, searching for whether there is a negative word before the keyword in the answer , If there is no negative word, the matching result meets the condition.
The method according to claim 1, wherein the second area further comprises a problem-solving process of the mathematical application problem, and the problem-solving process is also identified based on the second area, and the method further comprises:

If the matching result satisfies the condition, it is also judged whether the problem-solving process is correct, if the problem-solving process is also correct, it is judged that the answer is correct; if the problem-solving process is wrong, it is judged as the answer The answer is wrong.
The method according to claim 6, wherein judging whether the problem-solving process is correct comprises:

Determine whether one or more numbers involved in the calculation included in the problem-solving process appear in the question. If at least one of the one or more numbers does not appear in the question, it is judged as the The problem-solving process is wrong;

Determine whether the result of the formula included in the problem-solving process is related to the answer to the answer, if not, it is determined that the problem-solving process is wrong;

Determine whether the calculation formula included in the problem-solving process is correct, and if it is incorrect, it is determined that the problem-solving process is wrong; and

Otherwise, it is judged that the problem-solving process is correct.
The method according to claim 1, wherein recognizing the answer answer based on the second area comprises: recognizing all characters in the second area, and using all characters after a specific character as the answer answer .
The method according to claim 1, further comprising: displaying the result of the determination in a relevant area of the mathematical word problem in the image.
The method according to claim 9, further comprising: when the result of the determination is that the answer is wrong, further displaying the standard answer.
The method according to claim 9, further comprising: displaying the standard answer when the relevant area of the mathematical word problem is operated.
The method according to claim 11, further comprising: displaying the standard problem-solving process of the mathematical application problem when the relevant area of the mathematical application problem is operated.
The method according to claim 1, wherein obtaining the standard answer to the mathematical application question based on a pre-established question bank comprises:

Perform feature extraction on the question to generate a feature vector, search for a matching vector matching the feature vector from the question bank, and generate the standard answer based on the answer associated with the matching vector in the question bank .
The method according to claim 13, wherein searching for a matching vector that matches the feature vector from the question bank comprises: calculating the similarity between all vectors in the question bank and the feature vector, if the first The similarity between the vector and the feature vector is not less than a preset first threshold, then the first vector is determined as the matching vector.
The method according to claim 14, further comprising: if the similarity between all vectors in the question bank and the feature vector is less than the first threshold, then:

Searching from the question bank for one or more candidate questions whose vector and the feature vector of the first question are not greater than a preset second threshold;

The first question is matched with the one or more candidate questions, and the one or more candidate questions are sorted according to the degree of matching from high to low to obtain one or more of the first questions. A similar topic; and

When the relevant area of the first question is operated, the problem-solving process and/or answer of the one or more similar questions are displayed, wherein the problem-solving process and/or answer of the one or more similar questions comes from The question bank.
A device for assisting mathematical application problems, including:

One or more processors; and

One or more memories configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions,

Wherein, when the series of computer-executable instructions are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1-15.
A non-transitory computer-readable storage medium, characterized in that a series of computer-executable instructions are stored on the non-transitory computer-readable storage medium, and when the series of computer-executable instructions are replaced by one or more When executed by the computing device, the one or more computing devices are caused to perform the method according to any one of claims 1-15.