CN117290487A - Automatic scrolling method based on large language model, electronic equipment and storage medium - Google Patents
Automatic scrolling method based on large language model, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117290487A CN117290487A CN202311404413.1A CN202311404413A CN117290487A CN 117290487 A CN117290487 A CN 117290487A CN 202311404413 A CN202311404413 A CN 202311404413A CN 117290487 A CN117290487 A CN 117290487A
- Authority
- CN
- China
- Prior art keywords
- question
- examination
- questions
- language model
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000006870 function Effects 0.000 claims abstract description 12
- 230000001172 regenerating effect Effects 0.000 claims abstract description 7
- 238000012805 post-processing Methods 0.000 claims description 56
- 238000012360 testing method Methods 0.000 claims description 42
- 238000012549 training Methods 0.000 claims description 26
- 230000015654 memory Effects 0.000 claims description 15
- 239000013598 vector Substances 0.000 claims description 13
- 238000012986 modification Methods 0.000 claims description 11
- 230000004048 modification Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000003058 natural language processing Methods 0.000 claims description 6
- 238000013459 approach Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000004804 winding Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012821 model calculation Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 241001416177 Vicugna pacos Species 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Educational Administration (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Tourism & Hospitality (AREA)
- Biomedical Technology (AREA)
- Educational Technology (AREA)
- Strategic Management (AREA)
- Human Computer Interaction (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- General Business, Economics & Management (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The embodiment of the invention discloses an automatic volume grouping method based on a large language model, electronic equipment and a storage medium. The method comprises the following steps: responding to the triggering operation of the automatic paper assembling function, displaying basic information about the paper and a first prompt setting entry of each default question type so as to guide a user to set the whole examination class of the paper and input a large language model, and automatically generating each question by the large language model; displaying the questions in a question-dividing manner, and displaying a second prompt setting entry aiming at the questions and the questions so as to guide a user to set adjustment requirements on the questions and/or the questions; meanwhile, a manual editing inlet aiming at each examination question is displayed and is used for manually modifying each examination question; responding to the triggering operation of each second prompt setting entry, and regenerating each new examination question by the large language model; responding to the triggering operation of each manual editing entry, and replacing each examination question with each new examination question which is manually modified; and forming the final examination paper by the examination questions.
Description
Technical Field
The embodiment of the invention relates to the technical field of automatic volume grouping, in particular to an automatic volume grouping method based on a large language model, electronic equipment and a storage medium.
Background
Examination is often carried out by examining knowledge and skills of a participant without leaving examination questions, and talent selection of various industries is often realized through examination questions in different fields. In the prior art, the task-setting and winding work is usually completed manually, which is time-consuming and labor-consuming.
Patent CN115688380a provides a method and a system for optimizing test paper generation based on a target course question bank, and patent CN115688380a provides a method and a system for optimizing test paper generation based on a target course question bank, but both methods need to rely on a question bank constructed in advance. With the coming of the national learning type society, new examination contents and personalized examination demands are more and more, and under the condition that users do not have question banks, how to quickly generate test papers aiming at different demands is a problem to be solved urgently.
Disclosure of Invention
The embodiment of the invention provides an automatic paper assembly method, electronic equipment and storage medium based on a large language model, which can quickly generate test paper according to personalized examination contents and examination demands without providing sample questions by a user.
In a first aspect, an embodiment of the present invention provides an automatic scrolling method based on a large language model, including:
Responding to the triggering operation of the automatic paper assembling function, displaying basic information about the paper and a first prompt setting entry of each default question type so as to guide a user to set the overall examination class of the paper; inputting the whole examination class into a large language model, and automatically generating each examination question by the large language model;
displaying the questions in a question-dividing manner, and displaying a second prompt setting entry aiming at the questions and the questions so as to guide a user to set adjustment requirements on the questions and/or the questions; meanwhile, a manual editing inlet aiming at each examination question is displayed and is used for manually modifying each examination question;
responding to the triggering operation of each second prompt setting entry, inputting the adjustment requirement of each question type and/or each examination question into a large language model, and regenerating each new examination question by the large language model; responding to the triggering operation of each manual editing entry, and replacing each examination question with each new examination question which is manually modified;
and forming the final examination papers by the examination questions, and optimizing the direction of the assembly papers of the large language model according to the examination question information and the examination paper information.
In a second aspect, an embodiment of the present invention provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the automated method of assembling volumes based on a large language model of any embodiment.
In a third aspect, embodiments of the present invention further provide a computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the automatic volume grouping method based on a large language model according to any of the embodiments.
The embodiment of the invention provides an automatic paper assembling method based on a large language model, which learns and understands the characteristics and requirements of questions by utilizing technologies such as deep learning, natural language processing and the like, so that a test paper meeting the requirements is automatically generated according to specific examination contents and question requirements, and a user does not need to construct a question library in advance. Firstly, a basic configuration interface is displayed for a user, the user is guided to upload examination content files aiming at examination basic information, examination outline is set through a first prompt entry, examination basic parameters (quantity and score) are configured aiming at questions, examination formats are set through another first prompt entry, the examination outline together forming a test paper is provided to the LLM, and initial examination questions are generated by the LLM. And then, displaying a test question configuration interface to the user, guiding the user to set adjustment requirements on specific questions or questions through a second prompt language inlet aiming at each question type and each question, and indicating the LLM to regenerate the questions, or guiding the user to manually modify the specific questions through a manual editing inlet aiming at each question so as to meet the personalized requirements of the user. Compared with repeated blind modification of questions which do not meet requirements in the prior art, the method and the device utilize the ultra-strong language understanding and processing capacity of the LLM, set different prompt entry for different objects at different stages of the group paper in a two-stage prompt mode, respectively guide a user to input accurate test paper requirements, question requirements and question requirements, facilitate the LLM to accurately understand the question characteristics and quickly generate test paper which meets the requirements; and the group paper direction of the LLM is optimized according to the examination question information and the examination paper information, so that the adaptability of the whole method to the demands of users is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an automatic scrolling method based on a large language model provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a basic configuration interface provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a prompt setting for basic information of a test paper according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a question-oriented reminder setting provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram of a test question configuration interface provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram of another alert setting for a question type provided by an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a post-processing module according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the invention, are within the scope of the invention.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should also be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
The embodiment of the invention provides an automatic scrolling method based on a large language model (Large Language Model, LLM). The method can quickly group the examination papers according to personalized examination contents and examination demands, and an examination organizer can quickly generate the examination papers by the large language module only by providing related data to be examined and question requirements for the large language model. Meanwhile, the examination organizer can evaluate the examination paper, adjust the examination paper in a regenerated or manually modified mode, and reversely optimize the direction of the examination paper of the large language model. To illustrate this method, a large language model is preferentially described.
The large language model is a deep learning model for obtaining output characters from input characters, is obtained through training a large amount of text data, and can generate natural language texts or understand the meaning of language texts. Traditional language models are often oriented to a certain type of natural language task, such as text classification, translation, question-answering and the like, and LLM enlarges the model scale and displays stronger natural language processing capability (such as learning through context). Illustratively, LLM of the present embodiment employs T0, chatGLM, alpaca, GPT (generating Pre-Trained Transformer), chatGPT, etc. The ChatGPT is changed into a dialogue form on the basis of LLM, and one LLM is used for completing multiple tasks in a mode of giving prompt instructions in the interaction process. More specifically, the language model may be regarded as a black box that accepts a token string as input (where the token may be a Chinese character, or an English word, etc.), and outputs a probability that the token string is a normal human sentence (or fragment). Mathematical formalization is as follows: given a sequence of tokens (u 1, u2,..un), the language model outputs a probability p (u 1, u2,.., un) that indicates the probability that the tokens compose a sentence (or fragment) in order. The following formula expresses the language model described above, expanding this probability into the form of a conditional probability: p (u 1, u2,) un) =p (u 1) pi p (ui|u1, u2,) ui-1. The language model can complete the task of text generation: giving a plurality of generated words in front, calculating the next word with the maximum sequence probability, and outputting the word as a prediction result; the model then adds the predicted word to the given sequence and repeats the process described above, continuing to predict the next word until the next word is predicted to be an end symbol or a desired length is reached.
In a specific embodiment, assuming that the text input into the large language model is text, the final output result is obtained after the following steps.
Step one, input pretreatment. Preprocessing the input text, including word segmentation, stop word removal, part-of-speech tagging and other operations, so as to obtain a preprocessed text sequence. Assuming that the input text is text, the text sequence obtained after preprocessing is token, wherein each token represents a word or a symbol, as shown in table 1:
TABLE 1
And step two, inputting codes. The input preprocessed text sequence is encoded into a vector of values for input into a neural network for computation. Each word may be mapped to a real vector using word embedding techniques and then the entire text sequence represented as a matrix, as shown in table 2.
TABLE 2
And thirdly, inputting the vectorized text sequence into a large language model neural network for calculation by model calculation to obtain an output vector, wherein the output vector is shown in a table 3. For example, a large language model may be calculated using a recurrent neural network or variant Transformer model, where model parameters have been trained through a large amount of text data during the training phase.
TABLE 3 Table 3
And step four, outputting decoding. And decoding the output vector obtained by the model calculation to obtain the final output word output_encoding. Illustratively, decoding may use an output layer to map the output vector into words or characters in a vocabulary, and then combine the words or characters into a segment of text output, as shown in Table 4.
TABLE 4 Table 4
And fifthly, outputting post-treatment. Post-processing the output text, including removing redundant spaces, punctuation, etc., and further text processing and analysis as needed, is shown in table 5. It should be noted that the above description is only for illustrating the technical principle of natural language processing using a large language model, and in practical application, a specific LLM structure may be developed according to application requirements, and a mature LLM algorithm (for example, LLM provided by hundred degrees, ali, signal flight, etc.) may be called through an API interface, which is not limited in this embodiment.
Based on the LLM technique described above, fig. 1 is a flowchart of an automatic scrolling method based on a large language model according to an embodiment of the present invention. The method is executed by the electronic equipment, can be an automatic scroll system running on the electronic equipment, and interacts with a user (such as an examination organizer) through a user interface, so that the personalized scroll requirement of the user is met. As shown in fig. 1, the method specifically includes:
S110, responding to the triggering operation of the automatic paper assembling function, displaying basic information about the paper and a first prompt setting entry of each default question type so as to guide a user to set the overall examination class of the paper; and inputting the whole examination class into a large language model, and automatically generating each examination question by the large language model.
When the user has the requirement of the paper, the automatic paper assembling function can be triggered through the system interface, and the system displays a basic configuration interface of the paper to the user, as shown in fig. 2, wherein the interface comprises two parts of basic information setting and question setting of the paper. The test paper basic information setting part is used for setting basic information such as test paper names, examination contents and the like, the test paper names can be directly input, the examination contents can be set by selecting a specific examination content file, and an exemplary file of 'development of a large language model will bring convenience to life' is selected in fig. 2 as an examination material, and related questions are presented according to the file in subsequent operation. The question type setting part comprises the number of default question types and an input window of each question score, and can directly input the required question types and scores as basic information of the test question group paper, and the default question types in fig. 2 comprise selection questions and answer questions by way of example.
In addition, the embodiment also provides a prompt setting entry for the basic information of the test paper and a prompt setting entry for each default question type, which are respectively used for inputting more detailed requirements for the basic information of the test paper and each default information.
In a specific embodiment, in response to the triggering operation of the prompt entry of the basic information about the test paper, the system automatically associates the examination content file and displays an editable examination outline, where the examination outline may include: examination purpose, default question type division, test purpose of each question type, examination difficulty, coverage field, reference answer requirement, and information of the examination content file, and the overall requirement of the examination paper is flexibly embodied from different angles. The examination purpose in fig. 3 is "the purpose of the examination paper is to test the product manager knowledge and skills of the recruiter", the default question division includes selection questions and questions, the test purpose of the selection questions is "test basic knowledge and theory", the test purpose of the questions is "understanding of certain knowledge", the examination difficulty is "the examination question difficulty should be moderate, the difficulty is divided into three levels of high, medium and low", the coverage neighborhood is "the core product manager field capable of covering product definition, user research, project management, data analysis and the like", the reference case requirement is "please provide a reference answer for each question", the information of the examination content file is "[ examination content ], and the examination content file automatically related to the uploading on the basic configuration interface" development of large language model will bring convenience for life ". Alternatively, the editable examination outline may be displayed on an upper layer (as shown in fig. 3) of the basic configuration interface, beside or on a new page, which is not particularly limited in this embodiment.
Furthermore, in the examination outline, except the examination content file selected by the user is automatically associated with the examination content, other contents can be edited according to the requirement. It should be noted that, the default question type division displayed in the examination outline corresponds to the default question type in the question type setting, if the default question type division in the examination outline is manually modified, the system will automatically switch to a new question type option in the question type setting part, and the prompt setting entry of each default question type will also be automatically replaced by the prompt setting entry of each new question type. For example, if the default question type portion in the examination outline of fig. 3 is modified to be a selection question, a blank question, and a question-answer question, the question type setting portion in the left side base configuration interface will present three lines of contents corresponding to the selection question, the blank question, and the question-answer question, respectively.
Continuing back to the basic configuration interface of FIG. 2, where the reminder setting entry for each question type is used to set the basic information and question format for each question type. In response to a triggering operation of a prompt setting entry for each question type, the system displays basic information of each question type and an editable question format, wherein the basic information is directly input by an automatic relation question type setting part (the number of questions) and (the question type) and the question format part comprises: stem, correct answer, question difficulty, question resolution, etc. For example, fig. 4 is a diagram showing the content of a prompt set by a user after the prompt for a selected question is set at an entry, where two items of "number of selected questions" and "number of questions" in the question type basic information are automatically associated with a question type setting part, and other contents can be modified according to the needs of the user.
After the examination outline and the examination question format are modified, the examination content file, the final examination outline, the number, the score and the final examination question format of each question form together form an overall examination outline of the examination paper, a large language model is input, and each examination question conforming to the overall examination outline is automatically generated by the large language model according to the examination content file.
S120, showing the questions in a question-dividing manner, and showing a second prompt setting entry aiming at the questions and the questions so as to guide a user to set adjustment requirements on the questions and/or the questions; meanwhile, a manual editing entry for each question is displayed and is used for manually modifying each question.
The questions generated by LLM may be presented in a question configuration interface, as shown in fig. 5. Because the LLM may not be able to generate all questions satisfied by the user at one time, the interface displays corresponding alert setting entries for each question type and each question respectively, the alert setting entries for each question type are used for guiding the user to set adjustment requirements for the questions, the alert setting entries for each question are used for guiding the user to set adjustment requirements for specific questions, and the LLM is convenient to regenerate new questions according to the adjustment requirements. For convenience of distinction and description, the prompt entry displayed on the basic configuration interface (fig. 2) in S110 is referred to as a first prompt entry, for the user to set the overall examination outline of the examination paper, and the prompt entry displayed on the examination paper configuration interface (fig. 5) in this step is referred to as a second prompt entry, for the user to set adjustment requirements about specific questions and topics. In addition, a manual editing entry for each question is also shown in the question configuration interface, such as the "edit" button in fig. 5.
S130, responding to the triggering operation of each second prompt setting entry, inputting the adjustment requirement of each question type and/or each question into a large language model, and regenerating each new question by the large language model; and responding to the triggering operation of each manual editing entry, and replacing each question with each new question which is manually modified.
In one embodiment, the system automatically displays the question format and the adjustment requirement that the question type can be edited in response to the triggering operation of the second prompt setting entry of any question type. The examination question format determined in the first prompt entry is displayed by default, and can be edited for the second time so as to guide a user to modify the whole examination question format; the adjustment requirements include at least one of knowledge points, test purposes, and emphasis directions to guide a user to input adjustment requirements in terms of knowledge points, test purposes, and emphasis directions. Illustratively, FIG. 6 presents a prompt that triggers the final editing completion after setting up the portal for the second prompt for the question type. After the prompt is submitted, the examination content file, the examination question format and the adjustment requirement are input into a large language model together, and all new examination questions under the examination question are automatically generated by the large language model.
Similarly, the system automatically displays the editable examination question format and adjustment requirements of any examination question in response to the triggering operation of the second prompt setting entry of the examination question. The question format defaults to display the latest question format of the current question type, and can be edited for a second time, and the adjustment requirement comprises at least one of knowledge points, test purposes and emphasis directions, so as to guide a user to modify the format of a single question and guide the user to input a specific adjustment requirement. And finally, after the second prompt language aiming at the questions is edited, the examination content file, the examination question format and the adjustment requirement are input into a large language model together, and a new examination question is automatically generated by the large language model.
In response to the triggering operation of the manual editing entry of any examination question, the user can modify any information of the questions at the editing interface, and the system automatically saves the content modified by the user as the final examination question.
S140, forming the final examination questions into examination papers, and optimizing the direction of the examination papers of the large language model according to the examination question information and the examination paper information.
After the winding is completed, the system extracts each pair of new questions and old questions which are independently modified aiming at the questions, wherein the conditions of regenerating a certain question through LLM or directly and manually modifying the certain question are included, but the conditions of regenerating all the questions under the whole question through LLM are not included; and the new and old test questions before and after modification are utilized to feed back the LLM, so that the group paper direction of the LLM is more in line with the actual needs. Specifically, according to the difference between the old and new questions, the present embodiment provides the following several optional embodiments:
In the first alternative implementation manner, feedback training is not needed for LLM, but a light post-processing module is added behind the LLM, and the post-processing module is trained according to each pair of old and new questions to learn the modification rule between the old and new questions; after training, the post-processing module is applied to the next automatic group volume, and the questions directly generated by the large language model are processed by the post-processing module and then used as questions finally generated by the large language model, wherein the post-processing module can be a neural network structure for natural language processing. The method is suitable for the situation that the difference types between the new and the old questions are based on the modification of the questions (such as pen errors, word order adjustment and the like, which are irrelevant to examination contents) or the difference degree between the new and the old questions is small, and samples are formed by the new questions and the old questions which meet the conditions, so that a sample library aiming at a post-processing module is constructed. After the sample library meets the scale requirement, each sample is used for training the post-processing module, so that after each old question is input into the post-processing module, the module output continuously approaches to the corresponding new question.
In a second alternative embodiment, feedback training is directly performed on the LLM, and basic rules between the overall examination class of the examination paper and the final examination paper are learned; after training, the new LLM model is applied to the next automatic group volume. The method is suitable for the situation that the difference types between new and old questions are complex or the difference is large, and a sample can be formed by the whole examination class of each group of the rolls and each final question, so that a sample library aiming at a large language model is constructed. After the sample library meets the scale requirement, training the large language model by utilizing each sample, so that after the whole test class of each sub-group volume is input into the large language model, the model output continuously approximates each final examination question of each sub-group volume.
The above two alternative embodiments may be executed separately, may be executed sequentially, or may be executed simultaneously, which is not limited by the embodiment. For convenience of distinguishing and description, the sample library for the post-processing module is referred to as a first sample library, and the sample library for the LLM is referred to as a second sample library, so in a specific embodiment, after each automatic scrolling is completed, the difference type of each pair of new and old questions is first determined, if the difference type between any new question and old question is a miswriting and/or a word order adjustment, a sample is formed by the new question and the old question, and the sample is added to the first sample library for the post-processing module of the large language model. Otherwise, the new examination question and the old examination question are vectorized respectively, the similarity of the two vectors is calculated, and the degree of difference between the two vectors is measured through the similarity. If the similarity is smaller than a set threshold value, forming a sample by the old examination questions and the new examination questions, and adding the sample to a first sample library of a post-processing module aiming at a large language model; if the similarity is greater than or equal to a set threshold, forming a sample by the whole examination class of the current set of volumes and the final examination questions, and adding the sample to a second sample library aiming at the large language model. After a first sample library meets the scale requirement, training the post-processing module by using the first sample library, so that after each old examination question is input into the post-processing module, the module output continuously approaches to the corresponding new examination question; after training, the post-processing module is applied to the subsequent automatic group coil. After the second sample library meets the scale requirement, training a large language model by using the second sample library, so that after the whole examination class of each secondary winding is input into the large language model, the model output continuously approximates to each final examination question of each secondary winding; after training, the new large language model is applied to the subsequent automatic group volume.
Further, in a specific embodiment, there may be a plurality of post-processing modules, where each question type corresponds to a different post-processing module, and each post-processing module corresponds to its own first sample library. After the new and old questions meeting the conditions are used for forming samples, the samples are added to a first sample library of a post-processing module corresponding to the current question type. Correspondingly, after a first sample library corresponding to any question type meets the scale requirement, training a post-processing module of the question type by using the sample library, applying the trained post-processing module to a subsequent automatic group volume, and inputting the question of the question type directly generated by a large language model into the post-processing module of the question type to obtain a final question.
Alternatively, the post-processing module may be implemented using a conventional RNN network, LSTM (Long Short-Term Memory network), GRU (Gate Recurrent Unit, gate cycle unit), attention mechanism network, or the like. These networks all have a well-established network structure and may generally include both an encoder and a decoder. After converting the old questions into vector matrix and inputting the vector matrix into an encoder, the encoder is used for extracting the depth language features of the whole or partial input questions, and the decoder is used for decoding the extracted depth language features into a new vector matrix and restoring the new questions. In one embodiment, when the same old question is modified into N new questions (for example, modified by different users into different new questions, where N is a natural number, including the case of regenerating a certain question by LLM or directly modifying a certain question manually), the difference type and the difference degree between the old and new questions can also be measured by the size of N. When N >1, the difference types between the new and old questions are considered to be more complex or have larger difference degree, and a post-processing module shown in fig. 7 is adopted to learn common language features and differentiated language features between the new and old questions and merge into another new question.
As shown in fig. 7, the post-processing module includes n+1 encoders and decoders, wherein a solid line box portion represents a basic structure of the post-processing module, and a dotted line box portion represents input and output of each structure. For example, taking an attention mechanism network as an example, each encoder and decoder can select a 5-layer multi-head self-attention neural network, each layer adopts 8 self-attention layers to form multi-head self-attention layers, a normalization layer is connected behind each layer, and a full connection layer is connected behind the last normalization layer for matching the specific size of the vector matrix. After finishing each winding, forming samples by the old questions and the N new questions, and adding the samples into a first sample library of a post-processing module aiming at a large language model; in the training process, the old examination questions and N new examination questions in the sample are respectively input into the N+1 encoders, and the common language characteristics of the old examination questions and the N new examination questions are extracted by each encoder through the minimization of the output difference of each encoder; and inputting the common language characteristic into the decoder, and generating another new question fused with the common language characteristic and the differentiated language characteristic of each new question by minimizing the difference between the output of the decoder and each new question.
Further, in connection with fig. 7, output difference minimization of each encoder can be achieved by the following loss function:
（1）
wherein,representing the depth language characteristics of the old questions output after passing through the corresponding encoders,/for the old questions>To->Respectively representing the depth language characteristics of the first to the Nth new questions output after passing through the corresponding encoders. By->The depth language characteristics output by any two encoders tend to be consistent, and the common language characteristics F of the old examination questions and the new examination questions are continuously approximated, so that the core parts which remain unchanged in the old examination questions and the new examination questions are reserved. At the same time->To->In case of a trend towards coincidence, the question of restoration after decoding by the decoder>To->And the two questions also tend to be consistent, and another new question Y which reflects common language characteristics is continuously approximated.
Meanwhile, the output of the decoder and the difference minimization of each new question can be realized through the following loss function:
（2）
wherein,representing each new question to which the old question is modified in the winding process.By->To minimize the final new question of the decoder output +.>And each new question in the sample->The difference is minimized, so that the difference characteristics of each new question in the sample are considered, and the difference language characteristics of each modification mode are reserved. In practical use, at +. >To->In the case of a trend towards coincidence, use can be made of +.>Implementation +.>Is calculated by the computer.
In summary, the total loss function can be constructed according to equation (1) and equation (2)The output difference minimization of each encoder is realized through the minimization of L, so that each encoder extracts the common language characteristics of the old questions and N new questions, and meanwhile, the difference minimization of the output of the decoder and each new question is realized, and the final new questions give consideration to the common language characteristics and the differentiated modification characteristics of each new question, thereby obtaining a more reasonable question. After the post-processing module is trained, the questions generated by LLM are input into the encoder and decoder corresponding to the old questions in the post-processing module in the next group volume, and the output of the decoder is +.>As a final question.It should be noted that, for each old question, the number of modified new questions may be different, the maximum value of the number of new questions modified by all the old questions may be taken as the value of N in the training process, the missing new questions are filled into any one or several existing new questions, and N encoder internal references are respectively input and calculated.
The embodiment provides an automatic paper assembling method based on a large language model, which utilizes deep learning, natural language processing and other technologies to learn and understand the characteristics and requirements of questions, so that test papers meeting the requirements are automatically generated according to specific examination contents and question requirements, and a user does not need to construct a question library in advance. Firstly, a basic configuration interface is displayed for a user, the user is guided to upload examination content files aiming at examination basic information, examination outline is set through a first prompt entry, examination basic parameters (quantity and score) are configured aiming at questions, examination formats are set through another first prompt entry, the examination outline together forming a test paper is provided to the LLM, and initial examination questions are generated by the LLM. And then, displaying a test question configuration interface to the user, guiding the user to set adjustment requirements on specific questions or questions through a second prompt language inlet aiming at each question type and each question, and indicating the LLM to regenerate the questions, or guiding the user to manually modify the specific questions through a manual editing inlet aiming at each question so as to meet the personalized requirements of the user. Compared with repeated blind modification of questions which do not meet requirements in the prior art, the method and the device utilize the ultra-strong language understanding and processing capacity of the LLM, set different prompt entry for different objects at different stages of the group paper in a two-stage prompt mode, respectively guide a user to input accurate test paper requirements, question requirements and question requirements, facilitate the LLM to accurately understand the question characteristics and quickly generate test paper which meets the requirements; and the group paper direction of the LLM is optimized according to the examination question information and the examination paper information, so that the adaptability of the whole method to the demands of users is improved.
In particular, for each question which is independently modified, the embodiment constructs different sample libraries according to the difference condition between the new test question and the old test question, is used for carrying out optimization training on the LLM or the post-processing module of the LLM, and is applied to the subsequent automatic group volume. The LLM training takes the overall examination paper as input, takes the final examination paper questions as output, learns the overall rule of examination paper generation, and is suitable for the situation that the difference between the new examination paper and the old examination paper is obvious and complex; the training of the post-processing module takes old questions as input and new questions as output, and learns the modification rule among the questions, so that the post-processing module is suitable for the condition that the difference among the new questions and the old questions is small or simple. In addition, when the same examination question is modified into a plurality of new examination questions by different users, a specific structure of a post-processing module can be constructed through a plurality of encoders and a decoder, the old examination questions and N new examination questions in a sample are respectively input into the N+1 encoders in the training process, and the common language characteristics of the old examination questions and the N new examination questions are extracted by the encoders through the minimization of the output difference of the encoders; and inputting the common language features into the decoder, and generating another new question which is fused with the common language features and the different language features of the new questions by minimizing the difference between the output of the decoder and the new questions, so as to fully learn the modification rule of each group of volumes.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 8, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device may be one or more, one processor 60 being taken as an example in fig. 8; the processor 60, the memory 61, the input means 62 and the output means 63 in the device may be connected by a bus or other means, in fig. 8 by way of example.
The memory 61 is a computer readable storage medium, and may be used to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the automatic volume grouping method based on a large language model in the embodiment of the present invention. The processor 60 executes various functional applications of the device and data processing by running software programs, instructions and modules stored in the memory 61, i.e., implements the above-described automatic scrolling method based on a large language model.
The memory 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 61 may further comprise memory remotely located relative to processor 60, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 62 may be used to receive entered numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output 63 may comprise a display device such as a display screen.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the automated method of assembling volumes based on large language models of any of the embodiments.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the C-programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.
Claims (10)
1. An automatic scrolling method based on a large language model, comprising:
responding to the triggering operation of the automatic paper assembling function, displaying basic information about the paper and a first prompt setting entry of each default question type so as to guide a user to set the overall examination class of the paper; inputting the whole examination class into a large language model, and automatically generating each examination question by the large language model;
displaying the questions in a question-dividing manner, and displaying a second prompt setting entry aiming at the questions and the questions so as to guide a user to set adjustment requirements on the questions and/or the questions; meanwhile, a manual editing inlet aiming at each examination question is displayed and is used for manually modifying each examination question;
Responding to the triggering operation of each second prompt setting entry, inputting the adjustment requirement of each question type and/or each examination question into a large language model, and regenerating each new examination question by the large language model; responding to the triggering operation of each manual editing entry, and replacing each examination question with each new examination question which is manually modified;
and forming the final examination papers by the examination questions, and optimizing the direction of the assembly papers of the large language model according to the examination question information and the examination paper information.
2. The method of claim 1, wherein the presenting a first prompt setup portal for basic information and question setup of the test paper in response to a triggering operation of the automatic paper assembly function to guide a user to set up an overall class of the test paper comprises:
responding to the triggering operation of the automatic paper assembling function, acquiring an examination content file, and displaying a first prompt setting entry of basic information about the test paper and a first prompt setting entry of each default question type;
responding to the triggering operation of the first prompt entry of the basic information about the test paper, automatically associating the examination content file and displaying an editable examination outline, wherein the examination outline comprises the following components: examination purpose, default question type division, test purpose of each question type, examination difficulty, coverage field, reference answer requirement and information of the examination content file;
In response to a manual modification operation of the default question type division, replacing the first prompt setting entry for each default question type with the first prompt setting entry for each new question type;
responsive to a trigger operation of a first prompt setup portal for each question type, displaying a question format editable for each question type, the question format including: stem, correct answer, question difficulty, question analysis.
3. The method of claim 2, wherein the inputting the overall class into a large language model, automatically generating questions from the large language model, comprises:
and the examination content file, the final examination outline, the number, the score and the final examination question format of each question form the overall examination outline input large language model of the examination paper, and each examination question conforming to the overall examination outline is automatically generated by the large language model according to the examination content file.
4. A method according to claim 1, wherein the responding to the triggering operation of the second prompt setting entry inputs the adjustment requirement of each question type and/or each question into a large language model, and automatically generates each new question from the large language model, including:
Responding to the triggering operation of a second prompt setup portal of any question type and/or question, and displaying the question format and adjustment requirement of the editable question type and/or question; the examination content file, the examination question format and the adjustment requirement are input into a large language model together, and a new examination question is automatically generated by the large language model;
wherein, the question format capable of being edited shows the latest question format of the question by default, and the adjustment requirement comprises: at least one of knowledge points, test purposes, and emphasis directions.
5. The method of claim 1, wherein optimizing the group directions of the large language model based on the question information and the test paper information comprises:
extracting each pair of new and old questions which are modified for the questions individually;
according to the difference type and the difference degree between each pair of new questions and old questions, respectively constructing a sample library aiming at a large language model and/or a post-processing module aiming at the large language model, and training the model and/or the module so that the output of the large language model and/or the post-processing module continuously approximates each new question;
after training, the large language model and/or the post-processing module is applied to the next automatic group volume, wherein the post-processing module is of a neural network structure for natural language processing, and when the post-processing module is applied, the questions directly generated by the large language model are processed by the post-processing module and then are used as questions finally generated by the large language model.
6. A method according to claim 5, wherein constructing a sample library for the large language model and/or the post-processing module for the large language model and training the model and/or the module according to the difference type and the difference degree between each pair of the new questions and the old questions, respectively, so that the output of the large language model and/or the post-processing module approaches each new question continuously, comprises:
if the difference type between any pair of new questions and old questions is pen error and/or word order adjustment, forming a sample by the old questions and the new questions, and adding the sample into a first sample library of a post-processing module aiming at a large language model;
otherwise, vectorizing any pair of new examination questions and old examination questions respectively, and calculating the similarity of the two vectors; if the similarity is smaller than a set threshold value, forming a sample by the old examination questions and the new examination questions, and adding the sample to a first sample library of a post-processing module aiming at a large language model; if the similarity is greater than or equal to a set threshold, forming a sample by the whole examination class of the current group of the coils and each final examination question, and adding the sample into a second sample library aiming at the large language model;
after the sample library meets the scale requirement, training the post-processing module by utilizing the first sample library, so that after each old examination question is input into the post-processing module, the module output continuously approaches to the corresponding new examination question; or training the large language model by using the second sample library, so that after the whole examination class of each sub-group volume is input into the large language model, the model output continuously approximates each final examination question of each sub-group volume.
7. The method of claim 6, wherein the plurality of post-processing modules are provided, and each question type corresponds to a different post-processing module;
the forming a sample by the old examination questions and the new examination questions, adding the sample to a first sample library of a post-processing module aiming at a large language model, and comprising: forming a sample by the old examination questions and the new examination questions, and adding the sample into a first sample library of a post-processing module corresponding to the question type of the old examination questions;
the question directly generated by the large language model is processed by the post-processing module and then is finally generated as the large language model, and the method comprises the following steps: any question directly generated by the large language model is processed by a post-processing module corresponding to the question type of the any question and then is used as the question finally generated by the large language model.
8. A method as claimed in claim 5, wherein in the case where there are N new questions to which the same old question is modified, the post-processing module includes n+1 encoders and decoders, where N is a natural number, and the type of difference and the degree of difference between the old and new questions are represented as N in size;
according to the difference type and the difference degree between each pair of new questions and old questions, respectively constructing a sample library of a large language model and/or a post-processing module of the large language model, and training the model and/or the module, so that the output of the large language model and/or the post-processing module continuously approaches each new question, and the method comprises the following steps:
For any group of old questions and N new questions with N >1, forming samples by the old questions and the N new questions, and adding the samples into a first sample library of a post-processing module for a large language model;
in the training process, the old questions and N new questions in any sample are respectively input into the N+1 encoders, and the common language characteristics of the old questions and the N new questions are extracted by each encoder through the minimization of the output difference of each encoder; and inputting the common language characteristic into the decoder, and generating another new question fused with the common language characteristic and the differentiated language characteristic of each new question by minimizing the difference between the output of the decoder and each new question.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the automated method of assembling volumes based on a large language model of any of claims 1-8.
10. A computer readable storage medium, having stored thereon a computer program which when executed by a processor implements the automated large language model based method of scrolling of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311404413.1A CN117290487B (en) | 2023-10-27 | Automatic scrolling method based on large language model, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311404413.1A CN117290487B (en) | 2023-10-27 | Automatic scrolling method based on large language model, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117290487A true CN117290487A (en) | 2023-12-26 |
| CN117290487B CN117290487B (en) | 2024-05-31 |
Family
ID=
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050075887A1 (en) * | 2003-10-07 | 2005-04-07 | Bernard Alexis P. | Automatic language independent triphone training using a phonetic table |
| CN110415571A (en) * | 2018-12-05 | 2019-11-05 | 漳州万利达科技有限公司 | A kind of intelligent Auto-generating Test Paper, the method for examination and system |
| CN111368925A (en) * | 2020-03-06 | 2020-07-03 | 深圳深知未来智能有限公司 | Homodyne model for solving homodyne problem of comprehensive visual reasoning test by little learning |
| CN111723870A (en) * | 2020-06-22 | 2020-09-29 | 中国平安人寿保险股份有限公司 | Data set acquisition method, device, equipment and medium based on artificial intelligence |
| CN112995419A (en) * | 2021-02-05 | 2021-06-18 | 支付宝(杭州)信息技术有限公司 | Voice conversation processing method and system |
| CN113610231A (en) * | 2021-08-19 | 2021-11-05 | 北京金山数字娱乐科技有限公司 | Language model training method and device and phrase recommendation method and device |
| CN114170248A (en) * | 2021-12-21 | 2022-03-11 | 上海微创医疗机器人（集团）股份有限公司 | Image processing method, data processing method, medical system, device, and medium |
| WO2022111244A1 (en) * | 2020-11-26 | 2022-06-02 | 腾讯科技（深圳）有限公司 | Data processing method and apparatus, electronic device and storage medium |
| CN114678014A (en) * | 2022-03-23 | 2022-06-28 | 平安普惠企业管理有限公司 | Intention recognition method, device, computer equipment and computer readable storage medium |
| CN115114395A (en) * | 2022-04-15 | 2022-09-27 | 腾讯科技（深圳）有限公司 | Content retrieval and model training method and device, electronic equipment and storage medium |
| CN115168543A (en) * | 2022-07-15 | 2022-10-11 | 南京云问网络技术有限公司 | Examination question automatic generation design method based on unstructured text |
| CN115730038A (en) * | 2022-11-16 | 2023-03-03 | 广西通信规划设计咨询有限公司 | Method and device for automatically generating test paper and examining test paper, electronic equipment and medium |
| CN115934908A (en) * | 2022-11-30 | 2023-04-07 | 浙江华为通信技术有限公司 | Method and system for automatically generating questions |
| CN116070599A (en) * | 2022-12-01 | 2023-05-05 | 国家电网有限公司高级培训中心 | Intelligent question bank generation and auxiliary management system |
| US20230140997A1 (en) * | 2022-05-12 | 2023-05-11 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method and Apparatus for Selecting Sample Corpus Used to Optimize Translation Model |
| CN116341562A (en) * | 2023-03-28 | 2023-06-27 | 桂林电子科技大学 | Similar problem generation method based on Unilm language model |
| CN116561260A (en) * | 2023-07-10 | 2023-08-08 | 北京十六进制科技有限公司 | Problem generation method, device and medium based on language model |
| CN116561538A (en) * | 2023-04-04 | 2023-08-08 | 厦门美柚股份有限公司 | Question-answer scoring method, question-answer scoring device, electronic equipment and storage medium |
| WO2023173560A1 (en) * | 2022-03-16 | 2023-09-21 | 来也科技(北京)有限公司 | Rpa and ai based text error correction method, training method and related device thereof |
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050075887A1 (en) * | 2003-10-07 | 2005-04-07 | Bernard Alexis P. | Automatic language independent triphone training using a phonetic table |
| CN110415571A (en) * | 2018-12-05 | 2019-11-05 | 漳州万利达科技有限公司 | A kind of intelligent Auto-generating Test Paper, the method for examination and system |
| CN111368925A (en) * | 2020-03-06 | 2020-07-03 | 深圳深知未来智能有限公司 | Homodyne model for solving homodyne problem of comprehensive visual reasoning test by little learning |
| CN111723870A (en) * | 2020-06-22 | 2020-09-29 | 中国平安人寿保险股份有限公司 | Data set acquisition method, device, equipment and medium based on artificial intelligence |
| WO2022111244A1 (en) * | 2020-11-26 | 2022-06-02 | 腾讯科技（深圳）有限公司 | Data processing method and apparatus, electronic device and storage medium |
| CN112995419A (en) * | 2021-02-05 | 2021-06-18 | 支付宝(杭州)信息技术有限公司 | Voice conversation processing method and system |
| CN113610231A (en) * | 2021-08-19 | 2021-11-05 | 北京金山数字娱乐科技有限公司 | Language model training method and device and phrase recommendation method and device |
| CN114170248A (en) * | 2021-12-21 | 2022-03-11 | 上海微创医疗机器人（集团）股份有限公司 | Image processing method, data processing method, medical system, device, and medium |
| WO2023173560A1 (en) * | 2022-03-16 | 2023-09-21 | 来也科技(北京)有限公司 | Rpa and ai based text error correction method, training method and related device thereof |
| CN114678014A (en) * | 2022-03-23 | 2022-06-28 | 平安普惠企业管理有限公司 | Intention recognition method, device, computer equipment and computer readable storage medium |
| CN115114395A (en) * | 2022-04-15 | 2022-09-27 | 腾讯科技（深圳）有限公司 | Content retrieval and model training method and device, electronic equipment and storage medium |
| US20230140997A1 (en) * | 2022-05-12 | 2023-05-11 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method and Apparatus for Selecting Sample Corpus Used to Optimize Translation Model |
| CN115168543A (en) * | 2022-07-15 | 2022-10-11 | 南京云问网络技术有限公司 | Examination question automatic generation design method based on unstructured text |
| CN115730038A (en) * | 2022-11-16 | 2023-03-03 | 广西通信规划设计咨询有限公司 | Method and device for automatically generating test paper and examining test paper, electronic equipment and medium |
| CN115934908A (en) * | 2022-11-30 | 2023-04-07 | 浙江华为通信技术有限公司 | Method and system for automatically generating questions |
| CN116070599A (en) * | 2022-12-01 | 2023-05-05 | 国家电网有限公司高级培训中心 | Intelligent question bank generation and auxiliary management system |
| CN116341562A (en) * | 2023-03-28 | 2023-06-27 | 桂林电子科技大学 | Similar problem generation method based on Unilm language model |
| CN116561538A (en) * | 2023-04-04 | 2023-08-08 | 厦门美柚股份有限公司 | Question-answer scoring method, question-answer scoring device, electronic equipment and storage medium |
| CN116561260A (en) * | 2023-07-10 | 2023-08-08 | 北京十六进制科技有限公司 | Problem generation method, device and medium based on language model |
Non-Patent Citations (6)
| Title |
|---|
| JONATHAN M FRAME 等: "Post-processing the national water model with long short-term memory networks for streamflow predictions and model diagnostics", JOURNAL OF THE AMERICAN WATER RESOURCES ASSOCIATION, vol. 57, no. 6, 15 November 2021 (2021-11-15), pages 885 - 905 * |
| YUAN BIN 等: "Design on Algorithm of Automatic Test papers Generation for Examination System of Electric Energy Measurement", 2012 INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND SERVICE SYSTEM, 13 August 2012 (2012-08-13), pages 1397 - 1400, XP032295338, DOI: 10.1109/CSSS.2012.352 * |
| 刘雅莉: "一种改进遗传算法的自动组卷系统优化研究", 《微型电脑应用》, vol. 35, no. 8, 20 August 2020 (2020-08-20), pages 28 - 30 * |
| 毛秉毅: "基于目标树的组卷算法的研究", 《计算机工程与应用》, no. 23, 1 December 2002 (2002-12-01), pages 245 - 247 * |
| 盛嘉祺;许鑫;: "融合主题相似度与合著网络的学者标签扩展方法研究", 《数据分析与知识发现》, no. 8, 8 May 2020 (2020-05-08), pages 75 - 85 * |
| 连哲 等: "基于深度学习的自然场景文本检测综述", 《计算机工程》, 30 August 2023 (2023-08-30), pages 16 - 27 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Uc-Cetina et al. | Survey on reinforcement learning for language processing | |
| CN1457041B (en) | System for automatically annotating training data for natural language understanding system | |
| CN111310440B (en) | Text error correction method, device and system | |
| CN112989796B (en) | Text naming entity information identification method based on syntactic guidance | |
| WO2023201975A1 (en) | Difference description sentence generation method and apparatus, and device and medium | |
| CN110309511B (en) | Shared representation-based multitask language analysis system and method | |
| CN111985243B (en) | Emotion model training method, emotion analysis device and storage medium | |
| CN114722839A (en) | Man-machine collaborative dialogue interaction system and method | |
| CN110377910B (en) | Processing method, device, equipment and storage medium for table description | |
| CN114880307A (en) | Structured modeling method for knowledge in open education field | |
| CN112599129B (en) | Speech recognition method, apparatus, device and storage medium | |
| CN113947072A (en) | Text error correction method and text error correction device | |
| CN113705207A (en) | Grammar error recognition method and device | |
| CN117290487B (en) | Automatic scrolling method based on large language model, electronic equipment and storage medium | |
| CN117290487A (en) | Automatic scrolling method based on large language model, electronic equipment and storage medium | |
| CN114625759A (en) | Model training method, intelligent question answering method, device, medium, and program product | |
| CN115630652A (en) | Customer service session emotion analysis system, method and computer system | |
| CN113377943B (en) | Multi-round intelligent question-answering data processing system | |
| CN113345409B (en) | Speech synthesis method, speech synthesis device, electronic equipment and computer-readable storage medium | |
| CN116167361A (en) | Text error correction method, apparatus, device, computer storage medium, and program product | |
| CN112784573A (en) | Text emotion content analysis method, device and equipment and storage medium | |
| CN113807053A (en) | Method, apparatus, computer device and medium for generating decision book | |
| CN113934450A (en) | Method, apparatus, computer device and medium for generating annotation information | |
| CN111782781A (en) | Semantic analysis method and device, computer equipment and storage medium | |
| CN117762499B (en) | Task instruction construction method and task processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |