CN116796857A - LLM model training method, device, equipment and storage medium thereof - Google Patents
LLM model training method, device, equipment and storage medium thereof Download PDFInfo
- Publication number
- CN116796857A CN116796857A CN202310799459.1A CN202310799459A CN116796857A CN 116796857 A CN116796857 A CN 116796857A CN 202310799459 A CN202310799459 A CN 202310799459A CN 116796857 A CN116796857 A CN 116796857A
- Authority
- CN
- China
- Prior art keywords
- dialogue
- nodes
- text
- llm
- time sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 91
- 238000012549 training Methods 0.000 title claims abstract description 77
- 238000010586 diagram Methods 0.000 claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 36
- 238000012216 screening Methods 0.000 claims description 44
- 238000004140 cleaning Methods 0.000 claims description 17
- 125000002015 acyclic group Chemical group 0.000 claims description 2
- 238000010276 construction Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 description 9
- 238000003058 natural language processing Methods 0.000 description 8
- 238000013473 artificial intelligence Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010195 expression analysis Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 208000013738 Sleep Initiation and Maintenance disease Diseases 0.000 description 1
- 241000282840 Vicugna vicugna Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 206010022437 insomnia Diseases 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/313—Selection or weighting of terms for indexing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
Abstract
The embodiment of the application belongs to the technical field of digital medical treatment, is applied to a digital medical doctor and patient dialogue consultation training scene, and relates to a LLM model training method, device and equipment and a storage medium thereof, wherein the method comprises the steps of obtaining a natural language corpus to be subjected to LLM model training; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; according to the dialogue main body of the dialogue text and the dialogue sequence, a text dialogue time sequence diagram is arranged, then each service node is screened out according to the word nature in the dialogue text, and an LLM model is trained according to the text dialogue time sequence diagram and each service node, so that LLM model training according to the global property, the intermediate expressive property and the multidimensional interactivity of the dialogue is ensured, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
Description
Technical Field
The application relates to the technical field of digital medical treatment, and is applied to a digital medical doctor and patient dialogue consultation training scene, in particular to a LLM model training method, device and equipment and a storage medium thereof.
Background
Along with the development of the computer industry and artificial intelligence and the coming of the big data age, the traditional medical mode is gradually converted into the digital medical mode. With the development of deep learning, a Language Model (LM) has made a great progress in the field of natural Language processing (Natural Language Processing, NLP) capable of exhibiting remarkable performance on various tasks. Through unsupervised or self-supervised learning on massive text data, rich language knowledge and general capability are obtained.
LLM (Large Language Model, LLM) typically uses an autoregressive or autocoded approach to generate words in an output sequence one by one in a left-to-right or center-to-outside order after a given input sequence. This approach, while simple and efficient, suffers from several drawbacks: lack of global, intermediate expressive, and multidimensional interactivity. Therefore, on the dialog system of the digital medical industry, the language model is still trained by adopting the autoregressive or autocoding mode, and the language model cannot well combine the global dialog content to carry out semantic expression analysis due to the lack of global property, intermediate expressive property and multidimensional interactivity, so that the semantic recognition result of the dialog system of the digital medical industry is not accurate enough.
Disclosure of Invention
The embodiment of the application aims to provide a LLM model training method, a device, equipment and a storage medium thereof, which are used for solving the problems that the prior art is lack of a LLM language model with global property, intermediate expressive property and multidimensional interactivity, semantic expression analysis can not be well carried out by combining global dialogue content, and the semantic recognition result of a dialogue system in the digital medical industry is inaccurate.
In order to solve the above technical problems, the embodiment of the present application provides a LLM model training method, which adopts the following technical scheme:
a LLM model training method comprising the steps of:
acquiring a natural language corpus to be trained by a LLM model, wherein the natural language corpus comprises dialogue texts of doctors and corresponding consultants which are arranged in advance from a target medical dialogue platform, and the dialogue texts carry dialogue time information;
performing non-focus data cleaning on the dialogue text according to a preset cleaning rule, and obtaining cleaned text to be classified;
carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information;
classifying time sequence numbers corresponding to all dialogue sentences in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers, wherein the time sequence numbers in the first ordered classification set correspond to dialogue sentences of consultants, and the time sequence numbers in the second ordered classification set correspond to dialogue sentences of respondents;
According to the sequence of the time sequence numbers, selecting target time sequence numbers from the first ordered classification set and the second ordered classification set alternately to form a time sequence number sequence;
selecting corresponding dialogue sentences according to the time sequence number sequence, taking the dialogue sentences corresponding to the consultants as first dialogue side sentences and taking the dialogue sentences corresponding to the respondents as second dialogue side sentences;
generating a directed acyclic graph based on the first dialogue side statement, the second dialogue side statement, the sequence of time sequence numbers and the time sequence numbers corresponding to each dialogue side statement respectively, and obtaining a dialogue text time sequence graph;
inputting the dialogue text time sequence diagram into a pre-constructed LLM model, and training the pre-constructed LLM model according to the dialogue text time sequence diagram to obtain a trained LLM model.
Further, the step of performing non-focus data cleaning on the dialogue text according to a preset cleaning rule to obtain cleaned text to be classified specifically includes:
acquiring a preset non-focus data form;
taking the dialogue text as a search field, and sequentially acquiring each piece of non-focus data in the non-focus data form as a search word;
Screening out non-focus data from the search domain according to the search term, and marking all the screened out non-focus data;
and deleting all non-focus data according to the marking processing result to obtain the cleaned text to be classified.
Further, the step of numbering each dialogue sentence in the text to be classified according to the dialogue time information specifically includes:
according to the dialogue time information corresponding to each dialogue sentence in the dialogue text, sorting out the sequence of each dialogue sentence in the dialogue text;
according to the sequence of each dialogue sentence in the dialogue text, carrying out distinguishing numbering treatment on each dialogue sentence in the text to be classified from small to large by adopting an Arabic digital numbering method, and obtaining a distinguishing numbering treatment result;
acquiring distinguishing identification information preset for different dialogue main bodies;
screening out each dialogue sentence in the text to be classified and each dialogue sentence in the text to be classified, which are respectively corresponding to different dialogue subjects, and the corresponding distinguishing number of each dialogue sentence in the text to be classified according to the dialogue text and the distinguishing number processing result;
and splicing the distinguishing identification information and the distinguishing number corresponding to different dialogue main bodies to obtain the time sequence number.
Further, before the step of inputting the dialog text timing diagram into a pre-built LLM model and training the pre-built LLM model according to the dialog text timing diagram, the method further includes:
acquiring node screening keywords which are deployed in the LLM model in advance, wherein the node screening keywords are all keywords which are arranged in advance and can screen all service nodes, and the all service nodes comprise input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes;
starting a node naming component which is deployed in advance in the LLM model, wherein the node naming component can carry out differential naming on different nodes;
the step of inputting the dialogue text time sequence diagram into a pre-constructed LLM model and training the pre-constructed LLM model according to the dialogue text time sequence diagram specifically comprises the following steps:
analyzing the dialogue text time sequence diagram to obtain ordered dialogue sentences in the dialogue text time sequence diagram;
screening out all nodes contained in the ordered dialogue statement according to the node screening keywords which are deployed in the LLM model in advance;
According to a node naming component which is deployed in advance in the LLM model, respectively carrying out differential naming processing on all nodes in the ordered dialogue statement to obtain a differential naming processing result;
identifying input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes in all service nodes according to the distinguishing naming processing result;
constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node and the operation node;
and training the pre-constructed LLM model according to the execution workflow and the ordered dialogue statement.
Further, the node screening keywords include key nouns corresponding to variable nodes respectively, and further include key verbs corresponding to input nodes, output nodes, return nodes, method calling nodes and operation nodes respectively, and the step of screening all nodes included in the ordered dialogue statement according to the node screening keywords deployed in the LLM model in advance specifically includes:
splitting the ordered dialogue statement according to part of speech to obtain all nouns and all verbs contained in the ordered dialogue statement;
Screening all key nouns from all nouns according to the node screening keywords, and screening all key verbs from all verbs;
respectively screening out corresponding variable nodes according to key nouns respectively corresponding to the variable nodes;
and respectively screening out corresponding input nodes, output nodes, return nodes, method calling nodes and operation nodes according to the key verbs respectively corresponding to the input nodes, the output nodes, the return nodes, the method calling nodes and the operation nodes.
Further, after the step of identifying the input node, the output node, the method call node, the variable node, the return node and the operation node in all the service nodes according to the differential naming processing result is executed, the method further includes:
setting execution priority for all the identified input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes according to the time sequence in the dialogue text time sequence diagram, and setting a data flow logic line;
the step of constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node and the operation node specifically comprises the following steps:
And constructing the execution workflow based on the execution priorities respectively corresponding to all the input nodes, the output nodes, the method calling nodes, the variable nodes, the return nodes and the operation nodes and the data flow logic line.
Further, the training the pre-built LLM model according to the execution workflow and the ordered dialogue statement specifically includes:
screening keywords input into input nodes from the ordered dialogue sentences;
inputting the keywords input into the input nodes into the corresponding input nodes through the execution workflow;
acquiring all returned results and output results in the execution process of the execution workflow according to the execution result of the execution workflow;
identifying execution accuracy of the return results and the output results according to a preset accuracy algorithm formula and a preset reference result, wherein the preset reference result comprises all correct return results and all correct output results;
and if the execution accuracy rates respectively corresponding to the returned result and the output result meet the corresponding accuracy rate threshold, completing the LLM model training.
In order to solve the above technical problems, the embodiment of the present application further provides an LLM model training apparatus, which adopts the following technical scheme:
a LLM model training apparatus comprising:
the training corpus acquisition module is used for acquiring a natural language corpus to be subjected to LLM model training, wherein the natural language corpus contains dialogue texts of doctors and corresponding consultants which are arranged in advance from a target medical dialogue platform, and the dialogue texts carry dialogue time information;
the dialogue text cleaning module is used for cleaning the non-focus data of the dialogue text according to a preset cleaning rule, and acquiring cleaned text to be classified;
the dialogue sentence time sequence numbering module is used for time sequence numbering of each dialogue sentence in the text to be classified according to the dialogue time information;
the dialogue sentence classification and arrangement module is used for classifying time sequence numbers corresponding to dialogue sentences in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers, wherein the time sequence numbers in the first ordered classification set correspond to dialogue sentences of consultants, and the time sequence numbers in the second ordered classification set correspond to dialogue sentences of respondents;
The time sequence number sequence forming module is used for alternately selecting target time sequence numbers from the first ordered classification set and the second ordered classification set according to the order of the time sequence numbers to form a time sequence number sequence;
the dialogue sentence classification selection module is used for selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence and taking the dialogue sentence corresponding to the respondent as a second dialogue party sentence;
the dialogue text timing diagram generation module is used for generating a directed acyclic diagram based on the first dialogue side statement and the second dialogue side statement, the timing sequence number sequence and the timing sequence number corresponding to each dialogue side statement respectively, and obtaining a dialogue text timing diagram;
and the LLM model training module is used for inputting the dialogue text time sequence diagram into a pre-constructed LLM model, and training the pre-constructed LLM model according to the dialogue text time sequence diagram to obtain a trained LLM model.
In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical schemes:
a computer device comprising a memory having stored therein computer readable instructions which when executed by the processor implement the steps of the LLM model training method described above.
In order to solve the above technical problems, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical schemes:
a computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the LLM model training method as described above.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
according to the LLM model training method, a natural language corpus to be subjected to LLM model training is obtained; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers; forming a time sequence number sequence according to the sequence of the time sequence numbers; and selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue party sentence. According to the method, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequence in dialogue texts, each service node is selected according to word dialogues in the dialogue texts, an LLM model is trained according to the text dialogue time sequence diagram and each service node, LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of dialogue is guaranteed, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
Drawings
In order to more clearly illustrate the solution of the present application, a brief description will be given below of the drawings required for the description of the embodiments of the present application, it being apparent that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a LLM model training method in accordance with the present application;
FIG. 3 is a flow chart of one embodiment of step 202 of FIG. 2;
FIG. 4 is a flow chart of one embodiment of step 203 shown in FIG. 2;
FIG. 5 is a flow chart of one embodiment of step 208 of FIG. 2;
FIG. 6 is a flow chart of one embodiment of step 502 shown in FIG. 5;
FIG. 7 is a flow chart of one embodiment of step 506 of FIG. 5;
FIG. 8 is a schematic diagram of an embodiment of a LLM model training device according to the present application;
FIG. 9 is a schematic diagram of an embodiment of a computer device in accordance with the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to make the person skilled in the art better understand the solution of the present application, the technical solution of the embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture ExpertsGroup Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving PictureExperts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the LLM model training method provided by the embodiment of the present application is generally executed by a server, and accordingly, the LLM model training device is generally disposed in the server.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow chart of one embodiment of a LLM model training method in accordance with the present application is shown. The LLM model training method comprises the following steps:
Step 201, a natural language corpus to be trained by the LLM model is obtained, wherein the natural language corpus contains dialogue texts of doctors and corresponding consultants which are arranged in advance from a target medical dialogue platform, and the dialogue texts carry dialogue time information.
In this embodiment, the dialog text of the doctor and the corresponding consultant, which is previously arranged from the target medical dialog platform, includes the symptom data content provided by the consultant, for example: pain sites, insomnia symptoms, etc., personal information contents such as age, sex, etc., as well as inquiry guidance sentences provided by a doctor, detection suggestions provided by a doctor, or diagnosis results provided by a doctor.
The LLM (Large Language Model, LLM) model, i.e., a large-scale language processing model, is often applied to the field of text correction for recognizing text with large data volumes.
Step 202, cleaning non-focus data of the dialogue text according to a preset cleaning rule, and obtaining cleaned text to be classified, wherein the non-focus data refers to a language meaning word and a title pronoun which frequently appear in a dialogue scene.
With continued reference to FIG. 3, FIG. 3 is a flow chart of one embodiment of step 202 shown in FIG. 2, comprising:
Step 301, acquiring a preset non-focus data form;
specifically, the preset non-focus data form includes language-qi auxiliary words, such as, for example, a person who has a good, known, etc., or a pronoun, such as, for example, me, she, father, etc., which often occur in a dialogue scene.
Step 302, taking the dialogue text as a search field, and sequentially acquiring each piece of non-focus data in the non-focus data form as a search word;
step 303, screening out non-focus data from the search domain according to the search term, and performing marking processing on all the screened out non-focus data;
and step 304, deleting all non-focus data according to the marking processing result to obtain the cleaned text to be classified.
In this embodiment, the non-focus data cleaning is performed on the dialog text, so as to clean redundant words in the dialog text that are not useful for model training.
And 203, carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information.
With continued reference to fig. 4, fig. 4 is a flow chart of one embodiment of step 203 shown in fig. 2, comprising:
step 401, according to the dialogue time information corresponding to each dialogue sentence in the dialogue text, sorting out the sequence of each dialogue sentence in the dialogue text;
Step 402, according to the sequence of each dialogue sentence in the dialogue text, performing differential numbering processing on each dialogue sentence in the text to be classified from small to large by adopting an Arabic number numbering method, and obtaining a differential numbering processing result;
step 403, obtaining distinguishing identification information preset for different dialogue main bodies;
in this embodiment, the distinguishing identifier information is respectively set as a factor and a buffer according to different dialogue subjects, such as a doctor and a patient.
Step 404, screening out each dialogue sentence in the text to be classified and each distinguishing number corresponding to each dialogue sentence in the text to be classified according to the dialogue text and the distinguishing number processing result;
and step 405, splicing the distinguishing identification information and the distinguishing number corresponding to different dialogue bodies to obtain the time sequence number.
In this embodiment, it is assumed that the difference numbers of the dialogue sentences corresponding to the patient are 1, 2, 4, and 6, respectively, and the difference numbers of the dialogue sentences corresponding to the doctor are 3, 5, 7, and 8, respectively. The timing number includes: the buffer 1, buffer 2, vector 3, buffer 4, vector 5, buffer 6, vector 7, and vector 8.
Step 204, classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue body of the dialogue text, and obtaining a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers, wherein the time sequence numbers in the first ordered classification set correspond to the dialogue sentences of the consultant, and the time sequence numbers in the second ordered classification set correspond to the dialogue sentences of the respondent.
In this embodiment, in order to avoid classifying a large data volume and constructing a set, an ordered set is directly constructed by the timing sequence number of each dialogue sentence. Wherein the respondent is a doctor.
Step 205, selecting a target sequence number from the first ordered classification set and the second ordered classification set alternately according to the sequence of the sequence numbers, so as to form a sequence of sequence numbers.
In this embodiment, the sequence of time series numbers is constructed for the dialogue sentences of the doctor and the patient/consultant in the order of the time series numbers, and therefore, the sequence of time series numbers corresponds to the sequence of dialogue sentences of the doctor and the patient/consultant.
Step 206, selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue side sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue side sentence.
Step 207, generating a directed acyclic graph based on the first dialogue side statement, the second dialogue side statement, the sequence of time sequence numbers and the time sequence numbers corresponding to each dialogue side statement respectively, and obtaining a dialogue text time sequence graph.
And step 208, inputting the dialogue text time sequence diagram into a pre-constructed LLM model, and training the pre-constructed LLM model according to the dialogue text time sequence diagram to obtain a trained LLM model.
In this embodiment, the pre-built LLM model is a Chinese dialogue model, where the Chinese dialogue model is a Chinese dialogue model that is tuned by using a Chinese data+lora scheme based on the Vicuna model.
In this embodiment, before the step of inputting the dialog text timing diagram into a pre-built LLM model and training the pre-built LLM model according to the dialog text timing diagram, the method further includes: acquiring node screening keywords which are deployed in the LLM model in advance, wherein the node screening keywords are all keywords which are arranged in advance and can screen all service nodes, and the all service nodes comprise input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes; and starting a node naming component which is deployed in advance in the LLM model, wherein the node naming component can name different nodes differently.
With continued reference to fig. 5, fig. 5 is a flow chart of one embodiment of step 208 of fig. 2, including:
step 501, analyzing the dialogue text time sequence diagram to obtain ordered dialogue sentences in the dialogue text time sequence diagram;
step 502, screening out all nodes contained in the ordered dialogue statement according to the node screening keywords deployed in the LLM model in advance;
in this embodiment, the node screening keywords include key nouns respectively corresponding to variable nodes, and further include key verbs respectively corresponding to input nodes, output nodes, return nodes, method calling nodes and operation nodes.
With continued reference to FIG. 6, FIG. 6 is a flow chart of one embodiment of step 502 shown in FIG. 5, comprising:
step 601, splitting the ordered dialogue statement according to part of speech to obtain all nouns and all verbs contained in the ordered dialogue statement;
in this embodiment, the splitting processing may be performed on the ordered dialogue statement according to parts of speech, and specifically, the splitting processing may be performed using a BERT-based NLP natural language processing model, where the NLP natural language processing model satisfies parts of speech that can identify different terms, for example: name, verb, adjective, mood aid, etc.
Step 602, screening all key nouns from all nouns according to the node screening keywords, and screening all key verbs from all verbs;
step 603, screening out corresponding variable nodes according to key nouns corresponding to the variable nodes respectively;
in this embodiment, the variable node, i.e. the object generation node, is often named noun because the variable is often related to the entity class. Therefore, the keyword nouns are preset as keywords for screening variable nodes.
Step 604, respectively screening out corresponding input nodes, output nodes, return nodes, method call nodes and operation nodes according to key verbs respectively corresponding to the input nodes, the output nodes, the return nodes, the method call nodes and the operation nodes.
In this embodiment, because input and output are commonly used in executing programs, input and output are also verbs in dialogue text; the return and return are also verbs, the method call is often corresponding to the verbs, and the accumulation, modulo, division and the like are often expressed during operation, and are also verbs, so that key verbs are preset as keywords for filtering input nodes, output nodes, return nodes, method call nodes and operation nodes
Step 503, according to node naming components deployed in advance in the LLM model, respectively performing differential naming processing on all nodes in the ordered dialogue sentence, and obtaining a differential naming processing result;
step 504, identifying input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes in all service nodes according to the distinguishing naming processing result;
in this embodiment, after executing the step of identifying the input node, the output node, the method call node, the variable node, the return node and the operation node in all the service nodes according to the result of the differential naming processing, the method further includes: and setting execution priority for all the identified input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes according to the time sequence in the dialogue text time sequence diagram, and setting a data flow logic line.
Step 505, constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node and the operation node;
in this embodiment, the step of constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node, and the operation node specifically includes: and constructing the execution workflow based on the execution priorities respectively corresponding to all the input nodes, the output nodes, the method calling nodes, the variable nodes, the return nodes and the operation nodes and the data flow logic line.
And step 506, training the pre-constructed LLM model according to the execution workflow and the ordered dialogue statement.
With continued reference to FIG. 7, FIG. 7 is a flow chart of one embodiment of step 506 of FIG. 5, including:
step 701, screening keywords input into input nodes from the ordered dialogue sentences;
step 702, inputting the keywords input into the input nodes into the corresponding input nodes through the execution workflow;
step 703, obtaining all returned results and output results in the execution process of the execution workflow according to the execution result of the execution workflow;
step 704, identifying execution accuracy of the returned results and the output results according to a preset accuracy algorithm formula and a preset reference result, wherein the preset reference result comprises all correct returned results and all correct output results;
step 705, if the execution accuracy rates respectively corresponding to the returned result and the output result meet the corresponding accuracy rate threshold, the LLM model training is completed.
According to the embodiment, the LLM model is trained by pre-training dialogue texts of doctors and corresponding consultants which are arranged from the target medical dialogue platform, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequences in the dialogue texts, each service node is screened according to word dialogues in the dialogue texts, the LLM model is trained according to the text dialogue time sequence diagram and each service node, the LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of the dialogue is guaranteed, the dialogue model which is more in line with medical scenes is obtained, the construction and the service processing of the auxiliary digital medical platform are facilitated, and the semantic recognition accuracy of the dialogue system in the digital medical industry is improved.
The method comprises the steps of obtaining a natural language corpus to be trained by a LLM model; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers; forming a time sequence number sequence according to the sequence of the time sequence numbers; and selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue party sentence. According to the method, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequence in dialogue texts, each service node is selected according to word dialogues in the dialogue texts, an LLM model is trained according to the text dialogue time sequence diagram and each service node, LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of dialogue is guaranteed, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In the embodiment of the application, the natural language corpus to be trained by the LLM model is obtained; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; according to the dialogue main body of the dialogue text and the dialogue sequence, a text dialogue time sequence diagram is arranged, then each service node is screened out according to the word nature in the dialogue text, and an LLM model is trained according to the text dialogue time sequence diagram and each service node, so that LLM model training according to the global property, the intermediate expressive property and the multidimensional interactivity of the dialogue is ensured, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
With further reference to fig. 8, as an implementation of the method shown in fig. 2, the present application provides an embodiment of an LLM model training apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 8, the LLM model training apparatus 800 according to the present embodiment includes: a training corpus acquisition module 801, a dialogue text cleansing module 802, a dialogue sentence timing numbering module 803, a dialogue sentence classification and sorting module 804, a timing numbering sequence constructing module 805, a dialogue sentence classification and selecting module 806, a dialogue text timing diagram generating module 807 and a LLM model training module 808. Wherein:
a training corpus acquisition module 801, configured to acquire a natural language corpus to be trained by the LLM model, where the natural language corpus includes dialogue texts of doctors and corresponding consultants that are previously arranged from a target medical dialogue platform, and the dialogue texts carry dialogue time information;
the dialogue text cleaning module 802 is configured to clean non-focus data of the dialogue text according to a preset cleaning rule, and obtain a cleaned text to be classified, where the non-focus data refers to a word-air assisted word and a name pronoun that often occur in a dialogue scene;
A dialogue sentence timing sequence numbering module 803, configured to perform timing sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information;
the dialogue sentence classification and arrangement module 804 is configured to classify, according to a dialogue main body of the dialogue text, time sequence numbers corresponding to dialogue sentences in the text to be classified, and obtain a first ordered classification set and a second ordered classification set that are formed by the time sequence numbers, where the time sequence numbers in the first ordered classification set correspond to dialogue sentences of a consultant, and the time sequence numbers in the second ordered classification set correspond to dialogue sentences of a respondent;
a sequence number sequence constructing module 805, configured to alternately select a target sequence number from the first ordered classification set and the second ordered classification set according to the sequence of the sequence numbers, to construct a sequence number sequence;
a dialogue sentence classification selection module 806, configured to select a corresponding dialogue sentence according to the time sequence number sequence, take the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and take the dialogue sentence corresponding to the respondent as a second dialogue party sentence;
a dialog text timing diagram generating module 807, configured to generate a directed acyclic graph based on the first dialog sentence and the second dialog sentence, the sequence of timing numbers, and the timing numbers corresponding to each dialog sentence, to obtain a dialog text timing diagram;
And the LLM model training module 808 is configured to input the dialog text timing diagram into a pre-constructed LLM model, and train the pre-constructed LLM model according to the dialog text timing diagram to obtain a trained LLM model.
The method comprises the steps of obtaining a natural language corpus to be trained by a LLM model; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers; forming a time sequence number sequence according to the sequence of the time sequence numbers; and selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue party sentence. According to the method, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequence in dialogue texts, each service node is selected according to word dialogues in the dialogue texts, an LLM model is trained according to the text dialogue time sequence diagram and each service node, LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of dialogue is guaranteed, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by computer readable instructions, stored on a computer readable storage medium, that the program when executed may comprise the steps of embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 9, fig. 9 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 9 comprises a memory 9a, a processor 9b, a network interface 9c communicatively connected to each other via a system bus. It should be noted that only a computer device 9 having components 9a-9c is shown in the figures, but it should be understood that not all of the illustrated components need be implemented, and that more or fewer components may alternatively be implemented. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 9a includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 9a may be an internal storage unit of the computer device 9, such as a hard disk or a memory of the computer device 9. In other embodiments, the memory 9a may also be an external storage device of the computer device 9, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 9. Of course, the memory 9a may also comprise both an internal memory unit of the computer device 9 and an external memory device. In this embodiment, the memory 9a is generally used to store an operating system and various application software installed on the computer device 9, such as computer readable instructions of a LLM model training method. Further, the memory 9a may be used to temporarily store various types of data that have been output or are to be output.
The processor 9b may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 9b is typically used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is configured to execute computer readable instructions stored in the memory 9a or process data, such as computer readable instructions for executing the LLM model training method.
The network interface 9c may comprise a wireless network interface or a wired network interface, which network interface 9c is typically used for establishing a communication connection between the computer device 9 and other electronic devices.
The computer equipment provided by the embodiment belongs to the technical field of digital medical treatment, and is applied to the digital medical doctor and patient dialogue consultation training scene. The method comprises the steps of obtaining a natural language corpus to be trained by a LLM model; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers; forming a time sequence number sequence according to the sequence of the time sequence numbers; and selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue party sentence. According to the method, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequence in dialogue texts, each service node is selected according to word dialogues in the dialogue texts, an LLM model is trained according to the text dialogue time sequence diagram and each service node, LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of dialogue is guaranteed, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
The present application also provides another embodiment, namely, a computer readable storage medium, where computer readable instructions are stored, where the computer readable instructions are executable by a processor to cause the processor to perform the steps of the LLM model training method as described above.
The computer readable storage medium provided by the embodiment belongs to the technical field of digital medical treatment, and is applied to the digital medical doctor and patient dialogue consultation training scene. The method comprises the steps of obtaining a natural language corpus to be trained by a LLM model; carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information; classifying time sequence numbers corresponding to each dialogue sentence in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers; forming a time sequence number sequence according to the sequence of the time sequence numbers; and selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence, and taking the dialogue sentence corresponding to the replier as a second dialogue party sentence. According to the method, a text dialogue time sequence diagram is arranged according to different dialogue main bodies and dialogue sequence in dialogue texts, each service node is selected according to word dialogues in the dialogue texts, an LLM model is trained according to the text dialogue time sequence diagram and each service node, LLM model training according to the globally, the intermediately expressive and the multidimensional interactivity of dialogue is guaranteed, a dialogue model which is more in line with medical scenes is obtained, the construction of a digital medical platform and consultation dialogue service processing are facilitated, and the semantic recognition accuracy of a dialogue system in the digital medical industry is improved.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.
Claims (10)
1. A LLM model training method, comprising the steps of:
acquiring a natural language corpus to be trained by a LLM model, wherein the natural language corpus comprises dialogue texts of doctors and corresponding consultants which are arranged in advance from a target medical dialogue platform, and the dialogue texts carry dialogue time information;
performing non-focus data cleaning on the dialogue text according to a preset cleaning rule, and obtaining cleaned text to be classified;
carrying out time sequence numbering on each dialogue sentence in the text to be classified according to the dialogue time information;
classifying time sequence numbers corresponding to all dialogue sentences in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers, wherein the time sequence numbers in the first ordered classification set correspond to dialogue sentences of consultants, and the time sequence numbers in the second ordered classification set correspond to dialogue sentences of respondents;
according to the sequence of the time sequence numbers, selecting target time sequence numbers from the first ordered classification set and the second ordered classification set alternately to form a time sequence number sequence;
Selecting corresponding dialogue sentences according to the time sequence number sequence, taking the dialogue sentences corresponding to the consultants as first dialogue side sentences and taking the dialogue sentences corresponding to the respondents as second dialogue side sentences;
generating a directed acyclic graph based on the first dialogue side statement, the second dialogue side statement, the sequence of time sequence numbers and the time sequence numbers corresponding to each dialogue side statement respectively, and obtaining a dialogue text time sequence graph;
inputting the dialogue text time sequence diagram into a pre-constructed LLM model, and training the pre-constructed LLM model according to the dialogue text time sequence diagram to obtain a trained LLM model.
2. The LLM model training method as set forth in claim 1, wherein the step of performing non-focus data cleansing on the dialog text according to a preset cleansing rule to obtain a cleansed text to be classified specifically includes:
acquiring a preset non-focus data form;
taking the dialogue text as a search field, and sequentially acquiring each piece of non-focus data in the non-focus data form as a search word;
screening out non-focus data from the search domain according to the search term, and marking all the screened out non-focus data;
And deleting all non-focus data according to the marking processing result to obtain the cleaned text to be classified.
3. The LLM model training method as set forth in claim 1, wherein the step of numbering each dialogue sentence in the text to be classified according to the dialogue time information includes:
according to the dialogue time information corresponding to each dialogue sentence in the dialogue text, sorting out the sequence of each dialogue sentence in the dialogue text;
according to the sequence of each dialogue sentence in the dialogue text, carrying out distinguishing numbering treatment on each dialogue sentence in the text to be classified from small to large by adopting an Arabic digital numbering method, and obtaining a distinguishing numbering treatment result;
acquiring distinguishing identification information preset for different dialogue main bodies;
screening out each dialogue sentence in the text to be classified and each dialogue sentence in the text to be classified, which are respectively corresponding to different dialogue subjects, and the corresponding distinguishing number of each dialogue sentence in the text to be classified according to the dialogue text and the distinguishing number processing result;
and splicing the distinguishing identification information and the distinguishing number corresponding to different dialogue main bodies to obtain the time sequence number.
4. The LLM model training method as set forth in claim 1, wherein prior to performing the step of inputting the dialog text timing diagram into a pre-built LLM model, training the pre-built LLM model in accordance with the dialog text timing diagram, the method further comprises:
Acquiring node screening keywords which are deployed in the LLM model in advance, wherein the node screening keywords are all keywords which are arranged in advance and can screen all service nodes, and the all service nodes comprise input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes;
starting a node naming component which is deployed in advance in the LLM model, wherein the node naming component can carry out differential naming on different nodes;
the step of inputting the dialogue text time sequence diagram into a pre-constructed LLM model and training the pre-constructed LLM model according to the dialogue text time sequence diagram specifically comprises the following steps:
analyzing the dialogue text time sequence diagram to obtain ordered dialogue sentences in the dialogue text time sequence diagram;
screening out all nodes contained in the ordered dialogue statement according to the node screening keywords which are deployed in the LLM model in advance;
according to a node naming component which is deployed in advance in the LLM model, respectively carrying out differential naming processing on all nodes in the ordered dialogue statement to obtain a differential naming processing result;
identifying input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes in all service nodes according to the distinguishing naming processing result;
Constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node and the operation node;
and training the pre-constructed LLM model according to the execution workflow and the ordered dialogue statement.
5. The LLM model training method of claim 4, wherein the node screening keywords comprise key nouns corresponding to variable nodes respectively, and further comprise key verbs corresponding to input nodes, output nodes, return nodes, method calling nodes and operation nodes respectively, and the step of screening all nodes included in the ordered dialogue statement according to the node screening keywords deployed in the LLM model in advance specifically comprises:
splitting the ordered dialogue statement according to part of speech to obtain all nouns and all verbs contained in the ordered dialogue statement;
screening all key nouns from all nouns according to the node screening keywords, and screening all key verbs from all verbs;
respectively screening out corresponding variable nodes according to key nouns respectively corresponding to the variable nodes;
And respectively screening out corresponding input nodes, output nodes, return nodes, method calling nodes and operation nodes according to the key verbs respectively corresponding to the input nodes, the output nodes, the return nodes, the method calling nodes and the operation nodes.
6. The LLM model training method as set forth in claim 4, wherein after performing the step of identifying an input node, an output node, a method call node, a variable node, a return node, and an operation node among all service nodes according to the differential naming process result, the method further comprises:
setting execution priority for all the identified input nodes, output nodes, method calling nodes, variable nodes, return nodes and operation nodes according to the time sequence in the dialogue text time sequence diagram, and setting a data flow logic line;
the step of constructing an execution workflow in the LLM model based on the input node, the output node, the method call node, the variable node, the return node and the operation node specifically comprises the following steps:
and constructing the execution workflow based on the execution priorities respectively corresponding to all the input nodes, the output nodes, the method calling nodes, the variable nodes, the return nodes and the operation nodes and the data flow logic line.
7. The LLM model training method as set forth in claim 4, wherein the training the pre-built LLM model based on the execution workflow and the ordered dialog statements comprises:
screening keywords input into input nodes from the ordered dialogue sentences;
inputting the keywords input into the input nodes into the corresponding input nodes through the execution workflow;
acquiring all returned results and output results in the execution process of the execution workflow according to the execution result of the execution workflow;
identifying execution accuracy of the return results and the output results according to a preset accuracy algorithm formula and a preset reference result, wherein the preset reference result comprises all correct return results and all correct output results;
and if the execution accuracy rates respectively corresponding to the returned result and the output result meet the corresponding accuracy rate threshold, completing the LLM model training.
8. A LLM model training apparatus, comprising:
the training corpus acquisition module is used for acquiring a natural language corpus to be subjected to LLM model training, wherein the natural language corpus contains dialogue texts of doctors and corresponding consultants which are arranged in advance from a target medical dialogue platform, and the dialogue texts carry dialogue time information;
The dialogue text cleaning module is used for cleaning the non-focus data of the dialogue text according to a preset cleaning rule, and acquiring cleaned text to be classified;
the dialogue sentence time sequence numbering module is used for time sequence numbering of each dialogue sentence in the text to be classified according to the dialogue time information;
the dialogue sentence classification and arrangement module is used for classifying time sequence numbers corresponding to dialogue sentences in the text to be classified according to the dialogue main body of the dialogue text to obtain a first ordered classification set and a second ordered classification set which are formed by the time sequence numbers, wherein the time sequence numbers in the first ordered classification set correspond to dialogue sentences of consultants, and the time sequence numbers in the second ordered classification set correspond to dialogue sentences of respondents;
the time sequence number sequence forming module is used for alternately selecting target time sequence numbers from the first ordered classification set and the second ordered classification set according to the order of the time sequence numbers to form a time sequence number sequence;
the dialogue sentence classification selection module is used for selecting a corresponding dialogue sentence according to the time sequence number sequence, taking the dialogue sentence corresponding to the consultant as a first dialogue party sentence and taking the dialogue sentence corresponding to the respondent as a second dialogue party sentence;
The dialogue text timing diagram generation module is used for generating a directed acyclic diagram based on the first dialogue side statement and the second dialogue side statement, the timing sequence number sequence and the timing sequence number corresponding to each dialogue side statement respectively, and obtaining a dialogue text timing diagram;
and the LLM model training module is used for inputting the dialogue text time sequence diagram into a pre-constructed LLM model, and training the pre-constructed LLM model according to the dialogue text time sequence diagram to obtain a trained LLM model.
9. A computer device comprising a memory having stored therein computer readable instructions which when executed by the processor implement the steps of the LLM model training method of any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the LLM model training method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310799459.1A CN116796857A (en) | 2023-06-30 | 2023-06-30 | LLM model training method, device, equipment and storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310799459.1A CN116796857A (en) | 2023-06-30 | 2023-06-30 | LLM model training method, device, equipment and storage medium thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116796857A true CN116796857A (en) | 2023-09-22 |
Family
ID=88036343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310799459.1A Pending CN116796857A (en) | 2023-06-30 | 2023-06-30 | LLM model training method, device, equipment and storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116796857A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117034019A (en) * | 2023-10-09 | 2023-11-10 | 腾讯科技（深圳）有限公司 | Service processing method and device, electronic equipment and storage medium |
| CN117131945A (en) * | 2023-10-26 | 2023-11-28 | 北京睿企信息科技有限公司 | Data training method and storage medium for LLM model |
-
2023
- 2023-06-30 CN CN202310799459.1A patent/CN116796857A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117034019A (en) * | 2023-10-09 | 2023-11-10 | 腾讯科技（深圳）有限公司 | Service processing method and device, electronic equipment and storage medium |
| CN117034019B (en) * | 2023-10-09 | 2024-01-09 | 腾讯科技（深圳）有限公司 | Service processing method and device, electronic equipment and storage medium |
| CN117131945A (en) * | 2023-10-26 | 2023-11-28 | 北京睿企信息科技有限公司 | Data training method and storage medium for LLM model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112131366B (en) | Method, device and storage medium for training text classification model and text classification | |
| CN111444709B (en) | Text classification method, device, storage medium and equipment | |
| Shou et al. | Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis | |
| CN116796857A (en) | LLM model training method, device, equipment and storage medium thereof | |
| JP2015162244A (en) | Methods, programs and computation processing systems for ranking spoken words | |
| Zhang et al. | ScenarioSA: A dyadic conversational database for interactive sentiment analysis | |
| Mittal et al. | Versatile question answering systems: seeing in synthesis | |
| George et al. | Conversational implicatures in English dialogue: Annotated dataset | |
| Lai et al. | Multimodal sentiment analysis with asymmetric window multi-attentions | |
| Grewe et al. | ULearn: understanding and reacting to student frustration using deep learning, mobile vision and NLP | |
| Alarcón et al. | Hulat-ALexS CWI Task-CWI for Language and Learning Disabilities Applied to University Educational Texts. | |
| Kaur et al. | Design and development of a ticket booking system using Smart bot | |
| Pandey et al. | Interview bot with automatic question generation and answer evaluation | |
| CN115714002B (en) | Training method for depression risk detection model, depression symptom early warning method and related equipment | |
| CN111813905B (en) | Corpus generation method, corpus generation device, computer equipment and storage medium | |
| Cyras et al. | Visualization as a tertium comparationis within multilingual communities | |
| US20230260533A1 (en) | Automated segmentation of digital presentation data | |
| US20240160851A1 (en) | Day zero natural language processing model | |
| Singh et al. | Speech Emotion Based Recgognition System | |
| Li et al. | Unsupervised speech representation learning for behavior modeling using triplet enhanced contextualized networks | |
| CN115062136A (en) | Event disambiguation method based on graph neural network and related equipment thereof | |
| Chang et al. | Semantic Template-based Convolutional Neural Network for Text Classification | |
| NV et al. | Conversational AI Chatbot for HealthCare | |
| Shen et al. | Multimodal Knowledge-enhanced Interactive Network with Mixed Contrastive Learning for Emotion Recognition in Conversation | |
| Wang et al. | Multimodal Depression Detection Network Based on Emotional and Behavioral Features in Conversations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |