CN114881046B - Training method and device for task session model, computer equipment and storage medium - Google Patents
Training method and device for task session model, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114881046B CN114881046B CN202210565619.1A CN202210565619A CN114881046B CN 114881046 B CN114881046 B CN 114881046B CN 202210565619 A CN202210565619 A CN 202210565619A CN 114881046 B CN114881046 B CN 114881046B
- Authority
- CN
- China
- Prior art keywords
- slot
- intention
- information
- description
- session
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the technical field of artificial intelligence, and discloses a training method of a task session model, which comprises the following steps: the method comprises the steps of defining an intention list and slot information in advance, distributing slot index identifiers for each slot in the slot information, splicing the slot information into slot description information according to the slot index identifiers, distributing the intention index identifiers for each intention in the intention list, splicing the intention in the intention list into intention state information according to the intention index identifiers, responding to training instructions of a task session model, inputting session samples carried by the training instructions, the slot description information and the intention state information into the task session model for training after being spliced, and adjusting matching relations among the session samples, the slot description information and the intention state information by using a preset loss function in the training process until output values of the network model meet preset convergence conditions, and determining the trained network model as the task session model. The invention can generalize the model to a new prediction task, so that the knowledge learned by the model can be popularized to the intentions of other application scenes.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a training method, a training device, computer equipment and a storage medium for a task session model.
Background
With the continuous development of artificial intelligence technology, task-type session systems are increasingly receiving attention from people due to their strong practicability and wide application scenarios, and gradually enter the aspects of people's life. Generally, the overall architecture of the task-type conversation system mainly relates to text information input, natural language understanding, conversation management and result output, wherein the specific text information can be in a text form obtained by converting a voice signal through a voice recognition module, the natural language understanding can extract key information in the text information, such as pattern recognition, slot extraction, the conversation management comprises state tracking, conversation strategy and the like, and the output result is generated natural language.
Existing task-based conversation systems often require integration of more and more services in the process of training task conversation models to accommodate a wide variety of predictive tasks, such as from a flight reservation service to a hotel reservation service. However, for each new predicted task, since most task session models are trained on a single task-specific ontology schema, the ontology schema is typically represented as a list of possible user intents and slot information. If the ontology mode of the training task is changed, new data needs to be collected and the task session model retrained. Once the ontology mode is strict, the model is difficult to generalize to a new prediction task, so that knowledge learned by the model cannot be popularized to the intentions of other application scenes.
Disclosure of Invention
In view of this, the invention provides a training method, device, computer equipment and storage medium for task session models, which mainly aims to solve the problem that in the prior art, models are difficult to generalize to new prediction tasks, so that knowledge learned by the models is difficult to popularize in intentions of other application scenes.
According to one aspect of the present invention, there is provided a training method of a task session model, the method comprising:
predefining an intention list and slot position information;
distributing a slot index identifier for each slot in the slot information, and splicing the slot information into slot description information according to the slot index identifier;
distributing an intention index identifier for each intention in the intention list, and splicing the intents in the intention list into intention state information according to the intention index identifiers;
responding to a training instruction of a task session model, splicing a session sample carried by the training instruction with the slot description information and the intention state information, and inputting the spliced session sample into a network model for training;
and in the training process, adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function until the output value of the network model accords with a preset convergence condition, and determining the trained network model as a task session model.
According to another aspect of the present invention, there is provided a training apparatus for a task session model, the apparatus comprising:
the definition module is used for predefining an intention list and slot position information;
the first allocation module is used for allocating a slot index identifier for each slot in the slot information and splicing the slot information into slot description information according to the slot index identifiers;
the second distribution module is used for distributing an intention index identifier aiming at each intention in the intention list and splicing the intents in the intention list into intention state information according to the intention index identifiers;
the training module is used for responding to a training instruction of the task session model, splicing the session sample carried by the training instruction with the slot description information and the intention state information, and inputting the spliced session sample into the network model for training;
and the determining module is used for adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function in the training process until the output value of the network model accords with a preset convergence condition, and determining the trained network model as a task session model.
According to yet another aspect of the present invention, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of a training method of a task session model when the computer program is executed by the processor.
According to yet another aspect of the present invention, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a training method for a task session model.
By means of the technical scheme, the training method, the training device, the computer equipment and the storage medium for the task session model are provided, through predefining an intention list and slot information, slot index identifiers are allocated for each slot in the slot information, the slot information is spliced into slot description information according to the slot index identifiers, the intention index identifiers are allocated for each intention in the intention list, the intentions in the intention list are spliced into intention state information according to the intention index identifiers, a training instruction of the task session model is responded, a session sample carried by the training instruction, the slot description information and the intention state information are spliced and then are input into the network model for training, in the training process, a preset loss function is utilized for adjusting the matching relation between the session sample, the slot description information and the intention state information until the output value of the network model accords with a preset convergence condition, and the trained network model is determined to be used as the task session model. Compared with the mode of training the task session model on a single task specific ontology mode in the prior art, the method and the device consider the applicability of the session task to different application scenes, and can effectively convey semantic information by splicing the session sample with the slot description information and the intention state information and outputting all required results at one time, so that the matching relation between the slot or the intention and the session sample is learned in the training process of the model, the semantic understanding of the model in the training process is improved, the model can be generalized to a new prediction task, and the learned knowledge of the model can be popularized to the intention of other application scenes.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a schematic view of an application environment of a training method of a task session model according to an embodiment of the present invention;
FIG. 2 is a flow chart of a training method of a task session model according to an embodiment of the invention;
FIG. 3 is a flowchart illustrating the step S10 in FIG. 2;
FIG. 4 is a flowchart illustrating the step S20 in FIG. 2;
FIG. 5 is a flowchart illustrating the step S30 in FIG. 2;
FIG. 6 is a flowchart illustrating the step S40 in FIG. 2;
FIG. 7 is another flow chart of a training method of a session task model according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating the step S50 in FIG. 2;
FIG. 9 is a schematic diagram of a training apparatus for task session model according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a computer device according to an embodiment of the invention;
FIG. 11 is a schematic diagram of another configuration of a computer device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The training method of the task session model provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server through a network. The server side can define an intention list and slot information in advance, allocate a slot index identifier for each slot in the slot information, splice the slot information into slot description information according to the slot index identifiers, allocate the intention index identifiers for each intention in the intention list, splice the intention in the intention list into intention state information according to the intention index identifiers, respond to a training instruction of a task session model through the client side, splice a session sample carried by the training instruction with the slot description information and the intention state information, input the session sample into a network model to be trained, output session information which is extracted from the session sample and is matched with the slot description information and the intention state information respectively, and feed the session information back to the client side. According to the invention, considering the applicability of the session task to different application scenes, by splicing the session sample with the slot description information and the intention state information and outputting all required results at one time, the semantic information can be effectively transmitted, so that the matching relation between the slot or the intention and the session sample is learned in the training process of the model, the semantic understanding of the model in the training process is improved, the model can be generalized to a new prediction task, and the learned knowledge of the model can be popularized to the intention of other application scenes. The clients may be, but are not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers. The present invention will be described in detail with reference to specific examples.
Referring to fig. 2, fig. 2 is a flow chart of a training method of a task session model according to an embodiment of the present invention, including the following steps:
s10, predefining an intention list and slot position information.
According to the training method of the task session model, the task session model obtained through training can be applied to intelligent question-answering engines such as intelligent customer service or intelligent assistant under various scenes, the intelligent question-answering engines are usually realized through a server, the server can receive session sentences initiated by users in real time and respond correspondingly according to the session sentences, and then interactive session contexts are formed. For example, in the field of ticketing services, users may initiate conversations by telephone or chat on-line, often requiring the help of an intelligent question-and-answer engine to answer some ticketing questions of the customer or to assist the user in completing the ticketing operation in order to enhance the user's ticketing experience. The intelligent question and answer engine needs to perform natural language understanding on the user questions, such as intention recognition, slot extraction and the like, so that user intention and slot are clear, and the user questions are multiplexed back and forth according to the intention and the slot.
For example, the session sentence initiated by the user is "can help me to order a friday to go to the Shanghai with the high-speed railway", the intelligent question-answering engine needs to perform natural language understanding on the user problem after receiving the session sentence, recognizes that the user intends to order a train ticket, the slot is including the friday, and goes to the city to go to the Shanghai, and it can be understood that in the application scene of ordering the train ticket, the user cannot be assisted in completing the ticket ordering operation only by the session sentence, and the user is required to provide other slot information, and can output "of course, you want to start from that station".
The intention in the intention list is a request or an objective of the user, for example, the user says how the weather is today, the intention is "inquiring weather", the user says "i want to order a train ticket", the intention is "order train ticket", and different application scenes correspond to different intents. The slot information corresponds to the intended parameter information, and includes a slot and a slot value, for example, when a user asks about what is the "Beijing weather today", here "today" and "Beijing" are parameter information, the parameter information can be abstracted into different categories, each category is a slot, and further in the "inquiring weather" intention, two slots of "time" and "place" can be abstracted according to "today" and "Beijing". The intent in the intent list may be defined herein in terms of services involved in the application scenario, e.g., the services may be involved for the booking scenario including query tickets, refunds, change notes, etc., the slot information may be defined by extracting well-defined attributes and attribute values for a given entity from a large corpus, e.g., the well-defined attributes for the entities to departure time, return time, departure city, and arrival to city, etc., from a large corpus, also for the booking scenario.
It should be understood that the intent list is an intent set around a predetermined scene, each slot corresponds to a service description in the predetermined scene, and the slot value is a description value corresponding to the service description. As shown in fig. 3, in step S10, that is, the intention list and the slot information are predefined, the method includes the following steps:
s11, acquiring service data of a preset scene from a database aiming at the intention in the preset scene, and abstracting service description of the service data.
S12, traversing each service description, obtaining a description value suitable for being filled into the service description, and associating each service description with the corresponding description value to obtain slot position information.
For steps S11-S12, in the process of predefining the intention list and the slot information, the predetermined scene may be an offline living scene, such as a restaurant, a market, a movie theater, etc., because different intents may generate different service data under the predetermined scene, for example, the order intention of the restaurant scene may generate the service data related to the order, including the number of orders, the order time, the meal information, etc., the shopping intention of the market scene may generate the service data related to the shopping, such as purchasing goods, the color of goods, the performance of goods, etc., where a database is used to record the service data of different scenes stored according to predefined entities and relationships, and further abstract the service description of the service data, there may be multiple service descriptions in the service data, each service description may be used as a slot, each service description has a corresponding description value, for example, the order time is several points, the number of orders is 3-5, etc.
It should be noted that, since the description value corresponding to the service description corresponds to further clarification of the service description, the description value corresponding to each service description may be defined as an enumerated form, for example, the service description is shown as a Beijing airport, the corresponding description value may be defined as an Daxing airport, a Complanation airport, and a capital International airport, and may be defined as an unexploreable form, for example, the service description is shown as a city, and the corresponding description value is not limited. Specifically, whether the description value corresponding to the service description is divided into the enumeration type or not can be judged by traversing each service description, if yes, the description value suitable for being filled into the service description is output, each service description is added to the slot information after being associated with the corresponding description value, and otherwise, the service description is added to the slot information.
For example, the service description includes a beijing railway station, a train type, a seat level, a departure date and a departure time, where the train station, the train type and the seat level may be divided into an enumeratable form, the departure date and the departure time may be divided into a non-enumeratable form, further for the enumeratable form, the output description value filled into the train station includes a beijing south station, a beijing west station, a beijing north station, the description value filled into the train type includes a high-speed rail, a bullet train, a general express train, the filling into the description value as a level has an a-C level, and the service description is added to the slot information after being associated with the description value corresponding thereto, for the non-enumeratable form, the service description is directly added to the slot information without listing the description value corresponding to the service description.
It should be noted that, because some description values corresponding to the service descriptions affect other description values corresponding to the service descriptions, some description values corresponding to the service descriptions need to be adjusted according to the description values corresponding to the affected service descriptions, for example, different train types may affect the selection of the seat class, the description values corresponding to the motor train type and the high-speed rail type include an equal seat and a equal seat, and the description values corresponding to the common express train type include a seat and a no seat.
Further, considering that different expressions may exist in the slot information, the intention list and the slot information may be predefined, and then the preset expressions may be used to perform consistency processing on the intention in the intention list and the service description in the slot information, so that the intention list uses the same expression mode, and the service description in the slot information uses the same expression mode, specifically, the intention list or the intention list may be de-expressed by using descriptive language, each slot or the intention may be accurately described in a human-readable and understandable manner, for example, "train-from", train-to "may be processed as" arrival city of train "," departure city of train ", and the specific intention list may be as shown in table 1 below, the slot information may be as shown in table 2 below, and the intention and the slot corresponding to the intention may be selected by setting the state of the intention in the intention list, considering that the intention is not required to be in an activated state during the training.
Table 1, intention list
| Intent of | Status of |
| Checking train schedule service | |
| Booking train ticket services | Activation of |
| Train ticket change service | |
| Train ticket refund service |
TABLE 2 slot information
| Groove position | Slot position value |
| Train arrival city | Shanghai |
| Departure city of train | Beijing |
| Time of arrival of train | Three afternoon points |
| Departure time of train | |
| Train operation date | Wednesday |
It will be appreciated that tables 1 and 2 above describe the basic facts slots and intents for dialogue state tracking, the intent in the active state being to subscribe to the train ticket service, where the relevant information record for the subscribed train ticket service specifically includes "view train time list", "subscribe to train ticket service", "train ticket change service", "train ticket refund service" in the slots.
S20, distributing a slot index identifier for each slot in the slot information, and splicing the slot information into slot description information according to the slot index identifiers.
Considering that the same type of service description may exist in different slots in the slot information, for example, the departure time and the return time are both time type service descriptions, in order to enable the model to learn the service description from the slot information in the training process, the index marks of the slots correspond to index marks of the service description, for example, S1, S2, S3, etc., and other marks may also be used. In practical application, S0 may be used to represent a service description of the arrival time of the train, S1 may be used to represent a service description of the departure date of the train, S2 may be used to represent a service description of the departure date of the train, S3 may be used to represent a service description of the destination of the train, and S4 may be used to represent a service description of the departure time of the train.
It should be understood that the description value corresponding to each service description in the slot information may be divided into an enumeration type and may also be divided into a non-enumeration type. Specifically, as shown in fig. 4, in step S20, that is, a slot index identifier is allocated to each slot in the slot information, and the slot information is spliced into slot description information according to the slot index identifiers, which includes the following steps:
s21, distributing a slot index identifier for each slot in the slot information, and if the slot value is of an enumerated type, distributing a slot sub-index identifier to the enumerated slot value.
S22, splicing the slot information into slot description information according to the slot index identification and the slot sub-index identification.
For the slot of the enumeration type, since the slot value is usually output, after the slot index identifier is allocated to the slot, the slot sub-index identifier may be allocated to the slot value after enumeration, for example, if the slot value corresponding to the slot "needs to be sleeper" includes "yes" or "no", a) and b) may be used as the slot sub-index identifiers, and specifically, the number of the slot sub-index identifiers may be increased according to the number of the output slot values. For the non-enumeratable slot positions, the slot position sub-index identifiers can not exist, the slot position information is spliced into the slot position description information according to the slot position index identifiers and the slot position sub-index identifiers, and particularly, all the slot position index identifiers and the service description corresponding to the slot positions can be spliced into a slot position index identifier-service description pair mode, so that the slot position splicing information is obtained.
Illustratively, the slot includes: train arrival time, train departure date, train destination and train departure time, wherein the train departure date is divided into enumeration types, the corresponding slot values comprise Monday, tuesday and Tuesday, other slots are divided into non-enumeration types, and the spliced slot description information is as follows: s0 is train arrival time S1 is train departure date a), monday b), tuesday c) wednesday S2 is train departure date S3 is train destination S4 is train departure time.
S30, distributing an intention index identifier for each intention in the intention list, and splicing the intentions in the intention list into intention state information according to the intention index identifiers.
Similarly, similar to the procedure of assigning a slot index identifier to each slot in the slot information in step S20 above, a corresponding intention index identifier may be assigned for an intention in the intention list. To avoid the same as the slot index identification, a different index number than the slot index identification may be used herein.
It should be understood that the intention in the intention list may include a plurality of lower intention with explicit directions, and may also be understood as an intention including a service description, for example, the intention for inquiring a train ticket may include an intention to inquire about a train ticket from Beijing, an intention to inquire about a train ticket from Zhou, and the like. Specifically, as shown in fig. 5, in step S30, that is, an intention index identifier is assigned to each intention in the intention list, and the intention in the intention list is spliced into intention state information according to the intention index identifier, including the steps of:
S31, distributing intention index identifiers to each intention in the intention list, and distributing intention sub-index identifiers to lower intention if the intention also comprises lower intention.
S32, splicing the intents in the intention list into intention state information according to the intention index identifier and the intention sub-index identifier.
The process of splicing the intention state information is similar to the process of splicing the slot description information, i0 and i1 can be used as intention index identifiers, and the intention in the intention list is spliced into the intention state information according to the intention index identifiers.
The intention list comprises a train schedule, train ticket returns, train ticket changes and train ticket reservations, and the spliced intention state information is i1, the train schedule is i2, the train ticket returns i3, the train ticket changes i4 and the train ticket reservations are identified according to the intention index.
S40, responding to a training instruction of a task session model, splicing a session sample carried by the training instruction with the slot description information and the intention state information, and inputting the session sample into a network model for training.
The task session model is used for identifying the intention of a user and corresponding slot information in a session, and can be applied to session assistants of each application suit, and the auxiliary service assistants read the intention of the user and reply to the session content initiated by the user. Here, the session sample corresponds to session content actually generated in the application service scene, and the session content corresponding to both parties of the session is marked with a symbol, for example, [ user ] represents session content of the user party, and [ bot ] represents session content of the session assistant party.
Exemplary, session samples are: [ user ] can help me reserve a friday to go to the Shanghai's high-speed rail ticket? [ bot ] of course ]! What station you want to start from? What do you want to go away? User from Beijing, do me want to know that there is a train that can be reached before 3 PM?
It should be noted that, the splicing order of the three is not limited, and may be a session sample, slot description information, intention state information, slot description information, session sample, or the like. Because the spliced text information is all the results required by one-time output, the most important is that the index mapping slot information and the intention list are adopted, so that the model can learn the relation between the slot description information and the slot index mark and the slot value and the relation between the intention state information and the intention index mark and the intention, and the capability of promoting the learned knowledge of the model to the intention in other application scenes is solved.
Specifically, as shown in fig. 6, in step S40, that is, in response to a training instruction of the task session model, a session sample carried by the training instruction is spliced with slot description information and intention state information and then input into the network model for training, which includes the following steps:
S41, carrying out random collocation on the slot index mark and the slot as well as the intention index mark and the intention to form slot description information and intention state information with unfixed combination.
S42, splicing the session sample carried by the training instruction, the slot description information with unfixed combination and the intention state information, and inputting the spliced session sample, the slot description information and the intention state information into a network model for training.
It will be appreciated that since the model will train whether the slot description information and the intent state information match the session samples for the slot index identity and the intent index identity, respectively, during the training process, in order to prevent the model from remembering the association between the slot index identity-the slot and the intent index-the intent, the slot index identity may be randomly assigned to the slot and the intent index identity may be randomly assigned to the intent during the training process, using dynamic construction such that the model considers the slot description information and the intent state information instead of considering the input as a constant string to make a generalized prediction.
Taking a training process of slot description information as an example for explanation, a session sample comprises five training samples related to a certain train ticket, slot index identifiers are respectively S1-S4, slots are respectively train arrival time, train departure date and train destination, train departure time, slot index identifier S1 is allocated to slot 'train arrival time' in a first sample, slot index identifier S2 is allocated to slot 'train arrival time' in a second sample, slot index identifier S3 is allocated to slot 'train arrival time' in a third sample, the same allocation mode is adopted for other slots, and the input process between the slot index identifiers and the slots is randomly allocated instead of fixed collocation, so that a model learns more contents from the slot description information, the fixed collocation between the slot index identifiers and the slots is not simply memorized, the model still has higher prediction capability in order to be invisible, and the situation that a slot index identifier S1 is associated with the future movie ticket can be seen in the future time, and the slot index identifier S2 represents the reservation time. If the model is a fixed slot index identifier associated with a slot during training, S1 must express the contents of the slot, where the association of the slot index identifier with the slot is disturbed so that the model can actually learn useful prediction information from the slot description information.
And S50, in the training process, adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function until the output value of the network model accords with a preset convergence condition, and determining the trained network model as a task session model.
Further, considering the application of the task session model, the output part of the session task model includes the intention matching result entries of the sample session and the intention state information, and the slot matching result states of the sample session and the slot description information. The output format of the slot matching result entries may be a slot index identifier and a slot value, if the slot value is of an enumerated type, the slot value may use a slot sub-index identifier, for example, the non-enumerated type may output a slot matching result of [ states ] S0: three afternoon, the enumerated type S1 outputs a c option, the slot matching result is of [ states ] S1: S1c, and similarly, the intention matching result entries may be in the form of an intention index identifier of an activated intention, for example, only the intention i4 is activated, the intention matching result [ entries ] i4 is output, if there are a plurality of intents to be activated, for example, [ entries ] i1, i2, i3, …, and if the intention is of an enumerated type, the intention matching result may further include an intention sub-index identifier.
The network model uses a T5 model, and the T5 model can provide a unified model architecture, and can treat various session tasks as tasks in text-to-text form, namely, tasks of inputting text and outputting text, so that the method can be conveniently applied to a series of session tasks of evaluating reading understanding, abstract generation, text classification and the like. Specifically, in the input session sample, unique special symbols < X >, < Y > are used to represent the character string which is randomly masked in the session sample, the target sample which needs to be learned by the model is the masked character string, the special symbols < X >, < Y > at the corresponding positions in the input sample are used as separation, and finally another special symbol < Z > is added to represent the end of the sequence.
As an embodiment, referring to fig. 7, the spliced representation may be: s0: train arrival time S1: train departure date a) monday b) tuesday c) wednesday S2: train departure date S3: train destination S4: train departure time i1: check train schedule i2: train ticket refund i3: train ticket change i4: train ticket reservation [ user ] can help me reserve a high speed railway ticket for going to the sea for one wednesday? [ bot ] of course ]! What station you want to start from? What do you want to go away? User from Beijing, do me want to know that there is a train that can be reached before 3 PM? The spliced text information is further input into a network model for training, and the output result is [ states ] S0, S1cS2, beijing S3 and Shanghai [ intent ] i4 at three afternoon.
Specifically, as shown in fig. 8, in step S50, that is, in the training process, the matching relationship between the session sample, the slot description information and the intention state information is adjusted by using a preset loss function until the output value of the network model meets a preset convergence condition, and the trained network model is determined to be used as a task session model, which includes the following steps:
s51, respectively extracting a plurality of groups of session information matched with the slot description information and the intention state information from the session sample by using the slot description information and the intention description information which are not fixed in combination in the training process.
S52, repeatedly calculating the output value and a loss value formed by session information marked in the session sample in advance by using a preset loss function for a plurality of times.
And S53, if the loss value does not meet the iteration stop condition, adjusting the matching relation between the session sample, the slot description information and the intention state information until the loss value meets the iteration stop condition, determining that the output value of the network model meets the preset convergence condition, and taking the trained network model as a task session model.
In the related art, the limitation of the ontology mode in the task-type session affects the generalization of the model to tasks in other fields, for example, the session model trained in the "flight reservation" scene only knows the intention in the scene, lacks the ability to popularize its knowledge to the intention in other scenes (such as movie ticket reservation scene), even if there is a new ontology overlapping the ontology known to the current task, if the task session model already knows how to reserve an airplane ticket, the ability to increase reservation of a train ticket also requires training on brand new data, so the zero-order learning model training of the task-type session can promote the applicability of the model in different application scenes. However, regarding the application of task-type session in the zero-order learning model training process, the slot values need to be predicted sequentially according to different instruction information, as the number of slots increases, the training efficiency is lower and the slot values are easy to oversample, as most Cao Zhi are in inactive states at any stage of the session period, compared with the method for predicting all states in one-time transmission, the method has higher training efficiency, the traditional process only solves one slot in one-time output process, if 10 slots exist, the model prediction process is 10 times, the model is circulated inside the model, the model outputs corresponding slot information at one time, and the time is longer.
In an embodiment, a training device for a task session model is provided, where the training device for a task session model corresponds to the training method for a task session model in the foregoing embodiment one by one. As shown in fig. 9, the training apparatus of the task session model includes a definition module 101, a first allocation module 102, a second allocation module 103, a training module 104, and a determination module 105. The functional modules are described in detail as follows:
a definition module 101, configured to define an intention list and slot information in advance;
the first allocation module 102 is configured to allocate a slot index identifier for each slot in the slot information, and splice the slot information into slot description information according to the slot index identifier;
a second allocation module 103, configured to allocate an intent index identifier for each intent in the intent list, and splice the intents in the intent list into intent state information according to the intent index identifiers;
the training module 104 is configured to respond to a training instruction of a task session model, splice a session sample carried by the training instruction with the slot description information and the intention state information, and input the spliced session sample into a network model to be trained for training;
And the determining module 105 is configured to adjust a matching relationship between the session sample, the slot description information and the intention state information by using a preset loss function in the training process until an output value of the network model meets a preset convergence condition, and determine the trained network model as a task session model.
In an embodiment, the intent list is an intent set around a predetermined scene, each slot corresponds to a service description in the predetermined scene, the slot value is a description value corresponding to the service description, and the definition module is specifically configured to:
acquiring service data of a preset scene from a database aiming at the intention of the preset scene, and abstracting service description of the service data, wherein the database is recorded with service data of different scenes stored according to defined entities and relations;
traversing each service description, obtaining a description value suitable for filling into the service description, and associating each service description with a corresponding description value to obtain slot position information.
In an embodiment, the definition module is specifically configured to:
traversing each service description, and judging whether a description value corresponding to the service description is divided into an enumeration type or not;
If yes, outputting description values suitable for being filled into the service descriptions, and adding each service description to slot position information after associating the corresponding description value;
otherwise, adding the service description to the slot information.
In an embodiment, the definition module is further configured to:
and after the intention list and the slot information are predefined, carrying out consistency processing on the intention in the intention list and the service description in the slot information respectively by using a preset expression.
In an embodiment, the first allocation module is specifically configured to:
allocating a slot index identifier for each slot in the slot information, and if the slot value is of an enumerated type, allocating a slot sub-index identifier to the enumerated slot value;
and splicing the slot information into slot description information according to the slot index identification and the slot sub-index identification.
In an embodiment, the second allocation module is specifically configured to:
distributing an intention index identifier for each intention in the intention list, and distributing an intention sub-index identifier to a lower intention if the intention also comprises the lower intention;
and splicing the intents in the intention list into intention state information according to the intention index identifier and the intention sub-index identifier.
In an embodiment, the training module is specifically configured to:
the method comprises the steps of respectively carrying out random collocation on a slot index mark and a slot as well as an intention index mark and an intention to form slot description information and intention state information with unfixed combination;
the session sample carried by the training instruction, the slot description information with unfixed combination and the intention state information are spliced and then input into a network model for training;
the determining module is specifically configured to:
respectively extracting a plurality of groups of session information matched with the slot description information and the intention state information from the session sample by using the slot description information and the intention description information which are not fixed in combination in the training process;
repeatedly calculating a loss value formed by the output value and session information marked in a session sample in advance by using a preset loss function for a plurality of times;
and if the loss value does not meet the iteration stop condition, adjusting the matching relation between the session sample, the slot description information and the intention state information until the loss value meets the iteration stop condition, determining that the output value of the network model meets the preset convergence condition, and taking the trained network model as a task session model.
The embodiment provides a training device for a task session model, which considers the applicability of session tasks to different application scenes, and can effectively convey semantic information by splicing session samples with slot description information and intention state information and outputting all required results at one time, so that the matching relation between the slot or intention and the session samples is learned in the training process of the model, the semantic understanding of the model in the training process is improved, the model can be generalized to a new prediction task, and the learned knowledge of the model can be popularized to the intention of other application scenes.
For specific limitations on the training apparatus of the task session model, reference may be made to the above limitation on the training method of the task session model, which is not described herein. The various modules in the training device of the task session model can be implemented in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes non-volatile and/or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is for communicating with an external client via a network connection. The computer program, when executed by a processor, performs the functions or steps of a training method server side of a task session model.
In one embodiment, a computer device is provided, which may be a client, the internal structure of which may be as shown in FIG. 11. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is for communicating with an external server via a network connection. The computer program, when executed by a processor, performs the client-side functions or steps of a training method for a task session model
In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of when executing the computer program:
predefining an intention list and slot position information;
Distributing a slot index identifier for each slot in the slot information, and splicing the slot information into slot description information according to the slot index identifier;
distributing an intention index identifier for each intention in the intention list, and splicing the intents in the intention list into intention state information according to the intention index identifiers;
responding to a training instruction of a task session model, splicing a session sample carried by the training instruction with the slot description information and the intention state information, and inputting the spliced session sample into a network model for training;
and in the training process, adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function until the output value of the network model accords with a preset convergence condition, and determining the trained network model as a task session model.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
predefining an intention list and slot position information;
distributing a slot index identifier for each slot in the slot information, and splicing the slot information into slot description information according to the slot index identifier;
Distributing an intention index identifier for each intention in the intention list, and splicing the intents in the intention list into intention state information according to the intention index identifiers;
responding to a training instruction of a task session model, splicing a session sample carried by the training instruction with the slot description information and the intention state information, and inputting the spliced session sample into a network model for training;
and in the training process, adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function until the output value of the network model accords with a preset convergence condition, and determining the trained network model as a task session model.
It should be noted that, the functions or steps implemented by the computer readable storage medium or the computer device may correspond to the relevant descriptions of the server side and the client side in the foregoing method embodiments, and are not described herein for avoiding repetition.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (8)
1. A method for training a task session model, the method comprising:
the method comprises the steps of defining an intention list and slot information in advance, wherein the intention list is an intention set around a preset scene, each slot is equivalent to one service description in the preset scene, a slot value is a description value corresponding to the service description, service data of the preset scene is obtained from a database specifically aiming at the intention in the preset scene, the service description of the service data is abstracted, and the service data of different scenes stored according to defined entities and relations are recorded in the database; traversing each service description, obtaining a description value suitable for being filled into the service description, and associating each service description with a corresponding description value to obtain slot position information;
Distributing a slot index identifier for each slot in the slot information, and splicing the slot information into slot description information according to the slot index identifier;
distributing an intention index identifier for each intention in the intention list, and splicing the intents in the intention list into intention state information according to the intention index identifiers;
responding to a training instruction of a task session model, splicing a session sample carried by the training instruction with the slot description information and the intention state information, and then inputting the spliced session sample into a network model for training, wherein the slot index identification and the slot, and the intention index identification and the intention are randomly collocated to form slot description information and intention state information with unfixed combination; the session sample carried by the training instruction, the slot description information with unfixed combination and the intention state information are spliced and then input into a network model for training;
in the training process, the matching relation between the session sample, the slot description information and the intention state information is adjusted by using a preset loss function until the output value of the network model accords with a preset convergence condition, the trained network model is determined to be used as a task session model, and particularly in the training process, a plurality of groups of session information matched with the slot description information and the intention state information are respectively extracted from the session sample by using the slot description information and the intention description information which are not fixed in combination to be used as the output value of the network model; repeatedly calculating a loss value formed by the output value and session information marked in a session sample in advance by using a preset loss function for a plurality of times; and if the loss value does not meet the iteration stop condition, adjusting the matching relation between the session sample, the slot description information and the intention state information until the loss value meets the iteration stop condition, determining that the output value of the network model meets the preset convergence condition, and taking the trained network model as a task session model.
2. The method according to claim 1, wherein traversing each service description, obtaining a description value applicable to be filled into the service description, and associating each service description with a corresponding description value, obtaining slot information, specifically includes:
traversing each service description, and judging whether a description value corresponding to the service description is divided into an enumeration type or not;
if yes, outputting description values suitable for being filled into the service descriptions, and adding each service description to slot position information after associating the corresponding description value;
otherwise, adding the service description to the slot information.
3. The method of claim 1, wherein after the predefined intent list and slot information, the method further comprises:
and respectively carrying out consistency processing on the intention in the intention list and the service description in the slot position information by using a preset expression.
4. A method according to any one of claims 1-3, wherein the allocating a slot index identifier for each slot in the slot information and splicing the slot information into slot description information according to the slot index identifier specifically comprises:
Allocating a slot index identifier for each slot in the slot information, and if the slot value is of an enumerated type, allocating a slot sub-index identifier to the enumerated slot value;
and splicing the slot information into slot description information according to the slot index identification and the slot sub-index identification.
5. A method according to any one of claims 1-3, characterized in that said assigning an intention index identity for each intention in said intention list and stitching the intents in said intention list according to said intention index identity into intention state information, in particular comprising:
distributing an intention index identifier for each intention in the intention list, and distributing an intention sub-index identifier to a lower intention if the intention also comprises the lower intention;
and splicing the intents in the intention list into intention state information according to the intention index identifier and the intention sub-index identifier.
6. A training apparatus for a task session model, the apparatus comprising:
the definition module is used for predefining an intention list and slot information, wherein the intention list is an intention set around a preset scene, each slot is equivalent to one service description in the preset scene, the slot value is a description value corresponding to the service description, specifically, service data of the preset scene is obtained from a database aiming at the intention in the preset scene, the service description of the service data is abstracted, and the service data of different scenes stored according to defined entities and relations are recorded in the database; traversing each service description, obtaining a description value suitable for being filled into the service description, and associating each service description with a corresponding description value to obtain slot position information;
The first allocation module is used for allocating a slot index identifier for each slot in the slot information and splicing the slot information into slot description information according to the slot index identifiers;
the second distribution module is used for distributing an intention index identifier aiming at each intention in the intention list and splicing the intents in the intention list into intention state information according to the intention index identifiers;
the training module is used for responding to a training instruction of the task session model, inputting a session sample carried by the training instruction, the slot description information and the intention state information into a network model for training after being spliced, and particularly, carrying out random collocation on a slot index mark and a slot, and an intention index mark and an intention to form slot description information and intention state information with unfixed combination; the session sample carried by the training instruction, the slot description information with unfixed combination and the intention state information are spliced and then input into a network model for training;
the determining module is used for adjusting the matching relation between the session sample, the slot description information and the intention state information by using a preset loss function in the training process until the output value of the network model accords with a preset convergence condition, determining the trained network model as a task session model, and particularly extracting a plurality of groups of session information matched with the slot description information and the intention state information from the session sample by using the slot description information and the intention description information which are not fixed in combination in the training process as the output value of the network model; repeatedly calculating a loss value formed by the output value and session information marked in a session sample in advance by using a preset loss function for a plurality of times; and if the loss value does not meet the iteration stop condition, adjusting the matching relation between the session sample, the slot description information and the intention state information until the loss value meets the iteration stop condition, determining that the output value of the network model meets the preset convergence condition, and taking the trained network model as a task session model.
7. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.
8. A computer storage medium having stored thereon a computer program, which when executed by a processor realizes the steps of the method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210565619.1A CN114881046B (en) | 2022-05-23 | 2022-05-23 | Training method and device for task session model, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210565619.1A CN114881046B (en) | 2022-05-23 | 2022-05-23 | Training method and device for task session model, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114881046A CN114881046A (en) | 2022-08-09 |
| CN114881046B true CN114881046B (en) | 2023-07-25 |
Family
ID=82676740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210565619.1A Active CN114881046B (en) | 2022-05-23 | 2022-05-23 | Training method and device for task session model, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114881046B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209446A (en) * | 2019-04-23 | 2019-09-06 | 华为技术有限公司 | The configuration method and device of slot position are combined in a kind of interactive system |
| CN111309915A (en) * | 2020-03-03 | 2020-06-19 | 爱驰汽车有限公司 | Method, system, device and storage medium for training natural language of joint learning |
| KR20220039339A (en) * | 2020-09-22 | 2022-03-29 | 에스케이플래닛 주식회사 | Method and apparatus for providing chatbot service based on slot filling |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11200506B2 (en) * | 2017-12-15 | 2021-12-14 | Microsoft Technology Licensing, Llc | Chatbot integrating derived user intent |
-
2022
- 2022-05-23 CN CN202210565619.1A patent/CN114881046B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209446A (en) * | 2019-04-23 | 2019-09-06 | 华为技术有限公司 | The configuration method and device of slot position are combined in a kind of interactive system |
| CN111309915A (en) * | 2020-03-03 | 2020-06-19 | 爱驰汽车有限公司 | Method, system, device and storage medium for training natural language of joint learning |
| KR20220039339A (en) * | 2020-09-22 | 2022-03-29 | 에스케이플래닛 주식회사 | Method and apparatus for providing chatbot service based on slot filling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114881046A (en) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11553055B2 (en) | Automated communication-based intelligence engine | |
| US9971766B2 (en) | Conversational agent | |
| JP6793975B2 (en) | Video-based Jobs Job Matching Servers and methods and computer-readable recording media containing programs to perform those methods | |
| US11392775B2 (en) | Semantic recognition method, electronic device, and computer-readable storage medium | |
| US20110231353A1 (en) | Artificial intelligence application in human machine interface for advanced information processing and task managing | |
| US20110161129A1 (en) | Expert locator based on user polling | |
| CN112840628A (en) | Evidence recording of human-computer interaction communication | |
| US11727923B2 (en) | System and method for virtual conversations | |
| CN109977216A (en) | Dialogue recommended method and system based on scene | |
| US20230132894A1 (en) | Chat bot control device, chat bot control method, and chat bot control device system | |
| US20230043260A1 (en) | Persisting an AI-supported conversation across multiple channels | |
| CN111625638A (en) | Question processing method, device and equipment and readable storage medium | |
| CN115203282A (en) | Intelligent enterprise user data processing method and system combined with deep learning | |
| CN114357125A (en) | Natural language identification method, device and equipment in task type dialogue system | |
| CN114881046B (en) | Training method and device for task session model, computer equipment and storage medium | |
| CN109145092A (en) | A kind of database update, intelligent answer management method, device and its equipment | |
| CN115062629A (en) | Session information identification method and device, storage medium and computer equipment | |
| CN113132214B (en) | Dialogue method, dialogue device, dialogue server and dialogue storage medium | |
| CN111309990B (en) | Statement response method and device | |
| CN112559718A (en) | Dialogue processing method and device, electronic equipment and storage medium | |
| Agrawal et al. | WASABI Contextual BOT | |
| US20230259541A1 (en) | Intelligent Assistant System for Conversational Job Search | |
| US11881217B2 (en) | Solution guided response generation for dialog systems | |
| US20230409839A1 (en) | Preserving text and speech semantics using real-time virtual corpora in live interactive chatbot sessions | |
| US20230368773A1 (en) | Methods and systems for generating personal virtual agents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |