CN114077664A

CN114077664A - Text processing method, device and equipment in machine learning platform

Info

Publication number: CN114077664A
Application number: CN202010827214.1A
Authority: CN
Inventors: 陶冶; 陈伟; 周安; 谢佳雨
Original assignee: 4Paradigm Beijing Technology Co Ltd
Current assignee: 4Paradigm Beijing Technology Co Ltd
Priority date: 2020-08-17
Filing date: 2020-08-17
Publication date: 2022-02-22

Abstract

The embodiment of the disclosure discloses a text data processing method, a device and equipment in a machine learning platform, wherein the method comprises the following steps: inputting training text data and a real label corresponding to the training text data into a text processing model training operator to train a text processing model; inputting the text processing model and the predicted text data into a text processing model prediction operator to obtain a prediction label of the predicted text data; inputting the prediction label and a real label corresponding to the predicted text data into a text processing model evaluation operator to obtain an evaluation index value of the text processing model; and correspondingly processing the text processing model according to the evaluation index value.

Description

Text processing method, device and equipment in machine learning platform

Technical Field

The present invention relates to the field of information processing, and more particularly, to a text data processing method for a machine learning platform, a text processing apparatus for a machine learning platform, an apparatus including at least one computing apparatus and at least one storage apparatus, and a computer-readable storage medium.

Background

With the development of artificial intelligence, the value of data is continuously highlighted, and the demand of extracting useful information in text data for utilization is more and more common.

In the prior art, a machine learning model is often used for extracting useful information in text data, however, professionals related to Natural Language Processing (NLP) are needed, and people lacking NLP related experience are difficult to complete, and meanwhile, the existing automatic machine learning tool has too simple functions and cannot cover the whole process of building and applying the machine learning model, that is, the subsequent production and application of the machine learning model cannot be effectively realized, so that the useful information extracted from the text data cannot be quickly applied.

Disclosure of Invention

It is an object of the disclosed embodiments to provide a new technical solution for text data processing in a machine learning platform.

According to a first aspect of the present disclosure, there is provided a text data processing method in a machine learning platform, including:

inputting training text data and a real label corresponding to the training text data into a text processing model training operator to train a text processing model;

inputting the text processing model and the predicted text data into a text processing model prediction operator to obtain a prediction label of the predicted text data;

inputting the prediction label and a real label corresponding to the predicted text data into a text processing model evaluation operator to obtain an evaluation index value of the text processing model; and the number of the first and second groups,

and correspondingly processing the text processing model according to the evaluation index value.

Optionally, the text processing model is a text classification model, the real label and the prediction label are text classification results, the text processing model training operator is a text classification model training operator, the text processing model prediction operator is a text classification model prediction operator, and the text processing model evaluation operator is a text classification model evaluation operator;

alternatively, the first and second electrodes may be,

the text processing model is an entity extraction model, the real label and the prediction label are entity information results, the text processing model training operator is an entity extraction model training operator, the text processing model prediction operator is an entity extraction model prediction operator, and the text processing model evaluation operator is an entity extraction model evaluation operator;

or

The text processing model is a relation extraction model, the real label and the prediction label are entity relation results, the processing model training operator is a relation extraction model training operator, the text processing model prediction operator is a relation extraction model prediction operator, and the text processing model evaluation operator is a relation extraction model evaluation operator.

Optionally, the method further comprises the step of obtaining the training text data and the predictive text data based on the obtained initial text data,

the step of obtaining the training text data and the predictive text data based on the obtained initial text data includes:

inputting the obtained initial text data into a text data splitting operator to split the initial text data into training text data and predicted text data;

and the training text data is used as the input of the training operator of the text processing model, and the predicted text data is used as the input of the predictor of the text processing model.

Optionally, the obtaining the historical text data includes:

providing at least one data import path;

importing the historical text data from the selected data import path; and the number of the first and second groups,

and saving the imported historical text data.

Optionally, the method further comprises:

in response to a configuration operation for the text processing model training operator, providing a first configuration interface for configuring the text processing model training operator; performing feature engineering processing on the training text data and the corresponding real label thereof by the text processing model training operator according to first configuration information input by the first configuration interface to obtain a training text sample; and training a text processing model based on the training text sample according to a preset model training algorithm.

Optionally, the first configuration interface relates to at least one of the following configuration items: the method comprises the following steps of inputting source configuration items, target value field configuration items, model training algorithm configuration items, parameter adjusting algorithms and hyper-parameter configuration items of text processing model training operators and configuration items of text data maximum length.

Optionally, in case the text processing model is the text classification model,

the configuration items related to the first configuration interface further comprise: at least one of a configuration item of the application resource and a configuration item of a text language type corresponding to the text data.

Optionally, in the case that the text processing model is the entity extraction model,

a selection item related to a text data white list in the input source configuration item;

the text data white list is a list defining entity information, and the entity information comprises an entity name and an entity type corresponding to the entity name.

Optionally, the method further comprises:

analyzing the text data white list by utilizing the entity extraction model training operator;

and operating the analyzed text data white list to obtain entity names and entity types corresponding to the entity names defined in the text data white list, so that the instance extraction model predictor predicts the input predicted text data according to the defined entity names and the entity types corresponding to the entity names to obtain the predicted entity types of all the entity names in the predicted text data.

Optionally, the method further comprises:

providing a second configuration interface for configuring the text processing model predictor in response to a triggering operation for the text processing model predictor; and the text processing model predictor provides a prediction label for the predicted text data by utilizing the trained text processing model according to second configuration information input through the second configuration interface.

Optionally, the second configuration interface relates to at least one of the following configuration items: and the input source configuration item and the predicted target value field configuration item of the text processing model predictor.

Optionally, the method further comprises:

providing a third configuration interface for configuring the text processing model evaluation operator in response to a triggering operation for the text processing model evaluation operator; and the evaluation operator of the processing model compares the predicted label with a real label corresponding to the predicted text data according to third configuration information input through the third configuration interface to obtain an evaluation index value of the text processing model.

Optionally, the third configuration interface relates to at least one of the following configuration items: and the text processing model evaluates the input source configuration item of an operator and evaluates the configuration item of an index.

Optionally, the performing, according to the evaluation index value, corresponding processing on the text processing model includes:

and applying the text processing model on line under the condition that the evaluation index value is greater than or equal to an evaluation index threshold value.

Optionally, the performing corresponding processing on the text processing model according to the evaluation index value further includes:

and under the condition that the evaluation index value is smaller than the evaluation index threshold value, continuing to train the text processing model based on the text processing model training operator again.

Optionally, the continuing to train the text processing model based on the text processing model training operator again includes:

and adjusting the hyper-parameters for model training in the text processing model training operator, and continuing to train the text processing model based on the adjusted hyper-parameters.

Optionally, the adjusting the hyper-parameter for model training in the text processing model training operator, and continuing to train the text processing model based on the adjusted hyper-parameter includes:

reselecting a parameter adjusting algorithm used for generating the hyperparameter in the text processing model training operator;

and generating a new hyper-parameter according to the reselected parameter adjusting algorithm, and continuously training the text processing model based on the new hyper-parameter.

Optionally, the resuming the training of the text processing model based on the text processing model training operator further includes:

and adjusting the quantity of the training text data, and inputting the adjusted training text data and the real labels corresponding to the training text data into the training operator of the text processing model to continue training the text processing model.

and adjusting a model training algorithm for training the model in the text processing model training operator to continue training the text processing model according to the adjusted model training algorithm.

According to a second aspect of the present disclosure, there is also provided a text data processing apparatus in a machine learning platform, including:

the model training module is used for inputting training text data and the corresponding real labels into a text processing model training operator so as to train a text processing model;

the model prediction module is used for inputting the text processing model and the predicted text data into a text processing model prediction operator to obtain a prediction label of the predicted text data;

the model evaluation module is used for inputting the prediction label and a real label corresponding to the predicted text data into a text processing model evaluation operator to obtain an evaluation index value of the text processing model; and the number of the first and second groups,

and the model processing module is used for carrying out corresponding processing on the text processing model according to the evaluation index value.

alternatively, the first and second electrodes may be,

or

Optionally, the apparatus further comprises an obtaining module, configured to:

Optionally, the obtaining module is further configured to:

providing at least one data import path;

and saving the imported historical text data.

Optionally, the model training module is further configured to:

Optionally, in case the text processing model is the text classification model,

Optionally, the model training module is further configured to:

operating the analyzed text data white list to obtain entity names and corresponding entity types defined in the text data white list for the purpose of providing

And the instance extraction model predictor predicts the input predicted text data according to the defined entity name and the entity type corresponding to the entity name to obtain a predicted label of the predicted text data.

Optionally, the model prediction module is further configured to:

Optionally, the model estimation module is further configured to:

Optionally, the model processing module is further configured to:

According to a third aspect of the present disclosure, there is also provided an apparatus comprising at least one computing device and at least one storage device, wherein the at least one storage device is configured to store instructions for controlling the at least one computing device to perform the method according to the above first aspect.

According to a fourth aspect of the present disclosure, there is also provided a computer readable storage medium, wherein a computer program is stored thereon, which when executed by a processor, implements the method as described above in the first aspect.

According to the method of the embodiment of the disclosure, a text processing model training operator, a text processing model prediction operator and a text processing model evaluation operator which are independent of each other are provided, training of training text data is completed through the text processing model training operator to train a text processing model, prediction of predicted text data is completed through the text processing model prediction operator and the text processing model to obtain a prediction result, and evaluation of the prediction result is completed through the text processing model evaluation operator to obtain an evaluation result of the text processing model.

Drawings

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Fig. 1 is a block diagram showing an example of a hardware configuration of an electronic device that can be used to implement an embodiment of the present disclosure;

FIG. 2 illustrates a flow diagram of a method of text processing in a machine learning platform of an embodiment of the present disclosure;

3 a-3 c illustrate interface display diagrams of a method of text processing in a machine learning platform according to an embodiment of the present disclosure;

fig. 4 illustrates a functional block diagram of a text processing apparatus in a machine learning platform according to an embodiment of the present disclosure.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

Various embodiments and examples according to embodiments of the present invention are described below with reference to the accompanying drawings.

< hardware configuration >

The method of the embodiments of the present disclosure may be implemented by at least one electronic device, i.e., the apparatus 4000 for implementing the method may be disposed on the at least one electronic device. Fig. 1 shows a hardware structure of an arbitrary electronic device. The electronic device shown in fig. 1 may be a portable computer, a desktop computer, a workstation, a server, or the like, or may be any other device having a computing device such as a processor and a storage device such as a memory, and is not limited herein.

As shown in fig. 1, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. Wherein the processor 1100 is adapted to execute computer programs. The computer program may be written in an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 1200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 1400 is capable of wired or wireless communication, for example, and may specifically include Wifi communication, bluetooth communication, 2G/3G/4G/5G communication, and the like. The display device 1500 is, for example, a liquid crystal display panel, a touch panel, or the like. The input device 1600 may include, for example, a touch screen, a keyboard, a somatosensory input, and the like. The electronic device 1000 may output voice information through the speaker 1700, and may collect voice information through the microphone 1800, and the like.

The electronic device shown in fig. 1 is merely illustrative and is in no way meant to limit the invention, its application, or uses. In an embodiment of the present disclosure, the memory 1200 of the electronic device 1000 is used for storing instructions for controlling the processor 1100 to operate so as to execute the text processing method in the machine learning platform of the embodiment of the present disclosure. The skilled person can design the instructions according to the disclosed solution. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.

In one embodiment, an apparatus is provided that includes at least one computing device and at least one storage device to store instructions to control the at least one computing device to perform a method according to any embodiment of the present disclosure.

The apparatus may include at least one electronic device 1000 as shown in fig. 1 to provide at least one computing device, such as a processor, and at least one storage device, such as a memory, without limitation.

< method examples >

In this embodiment, a text processing method in a machine learning platform is provided, which may be implemented by the electronic device 1000 shown in fig. 1, for example, by the text processing apparatus 4000 in the machine learning platform of the electronic device 1000.

As shown in fig. 2, the text processing method in the machine learning platform of the present embodiment may include the following steps S2100 to S2400:

step S2100 inputs the training text data and the corresponding real labels into a text processing model training operator to train a text processing model.

The model training operator is a tool for performing data preprocessing on input text training data, performing characteristic engineering on the training text data subjected to the data preprocessing, and performing model training according to the result of the characteristic engineering to obtain a text processing model.

The training text data is text data used for performing text processing model training, in this step S2100, the training text data and the following predicted text data may be obtained based on the obtained initial text data, here, the present embodiment may further provide a text data splitting operator to split the initial text data into the training text data and the predicted text data by using the text data splitting operator, and the text data splitting operator may be a "text data splitting operator" in the canvas shown in fig. 3a, 3b, and 3 c. For example, the obtained initial text data may be input to a text data splitting operator to split the initial text data into training text data and predicted text data, the training text data may be used as input of a text processing model training operator in step S2100, and the predicted text data may be used as input of a text processing model predicting operator in step S2200 below. Based on this, the step of acquiring the historical text data in the present embodiment may include the steps of: providing at least one data import path; importing historical text data from the selected data import path; and saving the imported historical text data.

For example, the historical text data may be imported by importing data stored locally into the electronic device 1000, importing data in a database into the electronic device 1000, shallow-copying the data through HDFS, importing data through HDFS, and importing data through Hive. In this embodiment, after the imported initial text data is saved, the initial text data may be dragged to a canvas of the machine learning platform, and then the initial text data may serve as a node in the canvas, where the node may be nlp _ c1 shown in fig. 3a, entry _ cons shown in fig. 3b, and nlp _ re2 shown in fig. 3c, and the node may be "nlp _ c 1" shown in fig. 3a connected to "text data splitting operator", and "entry _ cons" shown in fig. 3b connected to "text data splitting operator", and "nlp _ re 2" shown in fig. 3c connected to "text data splitting operator", so as to obtain corresponding training text data and predicted text data, respectively.

The text processing model includes at least one of a text classification model, an entity extraction model, and a relationship extraction model. The text classification model is used for processing the text data to classify the text data, the entity extraction model is used for processing the text data to extract entity information in the text data, and the relation extraction model is used for processing the text data to extract relations among entities in the text data.

Where the text processing model is a text classification model, the above true labels are true text classification results, where the text processing model training operator is a text classification model training operator, such as the "text classification model training operator" shown in the canvas of FIG. 3 a.

Where the text processing model is an entity extraction model, the above true labels are true entity information results, where the text processing model training operator is an entity extraction model training operator, such as the "entity extraction model training operator" shown in the canvas of FIG. 3 b. The entity information includes entity names and corresponding entity types, and the entity types may be, for example, person names (person), organization names (organization), place names (location), and all other entities identified by names.

Where the text processing model is a relationship extraction model, the above true labels are true entity relationship results, where the text processing model training operator is an entity relationship model training operator, such as the "entity relationship model training operator" shown in the canvas of fig. 3 c.

In this embodiment, the text data splitting operator and the text processing model training operator may be connected to input a training data text obtained by splitting the text data splitting operator and a real label corresponding to the training data text into the text processing model training operator, configure the text processing model training operator, and further perform training of the text processing model according to the configuration information, where the text processing method in the machine learning platform in this embodiment may further include:

providing a first configuration interface for configuring a text processing model training operator in response to a configuration operation for the text processing model training operator; performing feature engineering processing on training text data and real labels corresponding to the training text data by a text processing model training operator according to first configuration information input by a first configuration interface to obtain a training text sample; and training the text processing model based on the training text sample according to a preset model training algorithm.

For example, a click operation may be performed on the text processing model training operator, and the electronic device 1000 provides a configuration interface for configuring the text processing model training operator in response to the click operation. The configuration interface relates to at least one configuration item of the following: the method comprises the following steps of inputting source configuration items, target value field configuration items, model training algorithm configuration items, parameter adjusting algorithms and hyper-parameter configuration items of text processing model training operators and configuration items of text data maximum length.

The above configuration item of the maximum length of the text data is used to limit the length of the text data, for example, the maximum length of the text data processed by the model may be limited, and the maximum length of the text data is typically 300.

The above target value field configuration item is used to represent the field name of the field where the model predicts the target.

The model training algorithm configuration item is used for configuring a model training algorithm of a training model, and the model training algorithm may be a Logistic Regression (LR) algorithm, a Gradient Boost Regression Tree (GBRT) algorithm, a Support Vector Machine (SVM) algorithm, an HE-TreeNet (high-dimensional discrete embedded Tree network) algorithm, a Gradient Boost Decision Tree (GBDT) algorithm, a random forest algorithm, or other Machine learning algorithms for training a Machine learning model, which is not limited herein.

The parameter tuning algorithm is an algorithm for optimizing parameters corresponding to a machine learning algorithm. The parameter adjusting algorithm can be random search, grid search, Bayesian optimization and the like.

The above hyper-parameters may include model hyper-parameters and training hyper-parameters, the model hyper-parameters being hyper-parameters used to define a model, such as but not limited to activation functions (e.g., identity functions, sigmoid functions, truncated ramp functions, etc.), number of hidden layer nodes, number of convolutional layer channels, number of fully-connected layer nodes, and the like. The training hyper-parameters are hyper-parameters used to define the model training process, such as, but not limited to, learning rate, batch size, and number of iterations.

For the canvas shown in fig. 3a, the node nlp _ c1 is connected to the text data splitting operator, the left output point of the text data splitting operator is connected to the text classification model training operator, and after configuration of configuration information is completed, operation is selected, so that the training operator of the text classification model can be used to perform feature engineering processing on training text data and a corresponding real classification result to obtain a training text sample, and the text classification model is trained based on the training text sample according to a configured model training algorithm.

For the canvas shown in fig. 3b, the "entry _ cons" node is connected to the "text data splitting operator", and the left output point of the "text data splitting operator" is connected to the "entity extraction model training operator" to complete the configuration of the configuration information, and then the selection operation is performed, so that the training text data and the corresponding real entity information result are subjected to feature engineering processing by using the text classification model training operator to obtain a training text sample, and the entity extraction model is trained based on the training text sample according to the configured model training algorithm.

For the canvas shown in fig. 3c, the node nlp _ c1 is connected to the text data splitting operator, the left output point of the text data splitting operator is connected to the entity relationship model training operator, and after configuration of configuration information is completed, selection and operation are performed, so that the entity relationship model training operator can be used to perform feature engineering processing on training text data and a corresponding real entity relationship result to obtain a training text sample, and the entity relationship model is trained based on the training text sample according to a configured model training algorithm.

Step S2200 is that the text processing model and the predicted text data are input into a text processing model prediction operator to obtain a prediction label of the predicted text data.

The predicted text data may be obtained based on the obtained initial text data, and how to obtain the predicted text data may refer to step S2100, which is not described herein again in step S2200.

Text processing model predictors are tools for providing predictive services to the prediction data provided.

Where the text processing model is a text classification model, the above predictive label is a predicted text classification result, where the text processing model predictor is a text classification model predictor, such as the "text classification model predictor" shown in the canvas of FIG. 3 a.

Where the text processing model is an entity extraction model, the above true tags are true entity information results, where the text processing model predictor is an entity extraction model predictor, such as the "entity extraction model predictor" shown in the canvas of FIG. 3 b.

Where the text processing model is a relationship extraction model, the above true labels are true entity relationship results, where the text processing model predictor is an entity relationship model predictor, such as the "entity relationship model predictor" shown in the canvas of FIG. 3 c.

In this embodiment, the right output point of the "text data splitting operator" may be connected to the right input point of the "text processing model predictor" to input the predicted text data obtained by splitting into the "text processing model predictor"; and connecting an output point of the "text processing model training operator" with a left input point of the "text processing model predictor" to input the obtained text processing model into the "text processing model predictor", configuring the "text processing model predictor", and predicting predicted text data by using the text processing model according to the configuration information, wherein the text processing method in the machine learning platform in the embodiment further comprises the following steps:

providing a second configuration interface for configuring the text processing model predictor in response to a triggering operation for the text processing model predictor; and providing a prediction label for the text processing model predictor according to second configuration information input through a second configuration interface by using the trained text processing model.

For example, a click operation may be performed on the text processing model predictor, and the electronic device 1000 provides a configuration interface for configuring the text processing model predictor in response to the click operation. The configuration interface relates to at least one configuration item of the following: and the input source configuration item and the predicted target value field configuration item of the text processing model predictor.

For the canvas shown in fig. 3a, the right output point of the "text data splitting operator" is continuously connected to the right input point of the "text classification model predictor", and the output point of the "text classification model training operator" is connected to the right input point of the "text classification model predictor", so that after configuration of configuration information is completed, selection operation is performed, and a prediction classification result can be provided for predicted text data by using the text classification model predictor and the trained text classification model.

As shown in fig. 3b, the canvas continues to connect the right output point of the "text data splitting operator" with the right input point of the "entity extraction model predictor", and connects the output point of the "entity extraction model training operator" with the right input point of the "entity extraction model predictor", and after configuration of configuration information is completed, selection operation is performed, so that a predicted entity information result can be provided for predicted text data by using the entity extraction model predictor and the trained entity extraction model.

As shown in fig. 3c, the canvas continues to connect the right output point of the "text data splitting operator" with the right input point of the "relation extraction model predictor", and connects the output point of the "relation extraction model training operator with the right input point of the" relation extraction model training operator ", and after configuration of configuration information is completed, selection operation is performed, so that a predicted entity relation result can be provided for predicted text data by using the relation extraction model predictor and the trained relation extraction model.

Step S2300, inputting the prediction label and the real label corresponding to the prediction text data into a text processing model evaluation operator to obtain an evaluation index value of the text processing model.

The text processing model evaluation operator is a tool for evaluating the trained text processing model.

In case the text processing model is a text classification model, the text processing model evaluation operator is a text classification model evaluation operator, e.g. the "text classification model evaluation operator" shown in the canvas of fig. 3 a.

Where the text processing model is an entity extraction model, the text processing model evaluation operator is an entity extraction model evaluation operator, such as the "entity extraction model evaluation operator" shown in the canvas of FIG. 3 b.

Where the text processing model is a relationship extraction model, the text processing model evaluation operator is an entity relationship model evaluation operator, such as the "entity relationship model evaluation operator" shown in the canvas of FIG. 3 c.

In this embodiment, the output point of the "text processing model predictor" may be connected to the input point of the "text processing model evaluator", so as to input the real tag and the prediction tag of the predicted text data into the "text processing model evaluator", configure the "text processing model evaluator", and further predict the predicted text data according to the configuration information, where the text processing method in the machine learning platform in this embodiment may further include:

providing a third configuration interface for configuring the text processing model evaluation operator in response to a triggering operation for the text processing model evaluation operator; and enabling the text processing model evaluation operator to compare the predicted label with a real label corresponding to the predicted text data according to third configuration information input through a third configuration interface to obtain an evaluation index value of the text processing model.

For example, a click operation may be performed on the text processing model evaluation operator, and the electronic device 1000 provides a configuration interface for configuring the text processing model evaluation operator in response to the click operation. The configuration interface relates to at least one configuration item of the following: and the text processing model evaluates the input source configuration item of an operator and evaluates the configuration item of an index.

The above evaluation index is an index for measuring the quality of the text processing model. The evaluation index may be at least one of Mean Squared Error (MSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), R2, auc (area Under The cut), Recall, Precision, Accuracy, f1, and log.

For the canvas shown in fig. 3a, the output point of the "text classification model predictor" is continuously connected with the input point of the "text classification model evaluator", and the selection operation is performed after the configuration of the configuration information is completed, so that the evaluation index value of the text classification model can be obtained by using the text classification model evaluator with respect to the real classification result and the predicted classification result of the predicted text data.

As shown in fig. 3b, the canvas continues to connect the output point of the "entity extraction model predictor" with the input point of the "entity extraction model evaluator", and after configuration of the configuration information is completed, the selection operation is performed, so that the evaluation index value of the entity extraction model can be obtained by using the entity extraction model evaluator with respect to the real entity information result and the predicted entity information result of the predicted text data.

For the canvas shown in fig. 3c, the output point of the "relation extraction model predictor" is continuously connected with the input point of the "relation extraction model evaluator", and the selection operation is performed after the configuration of the configuration information is completed, so that the evaluation index value of the relation extraction model can be obtained by using the relation extraction model evaluator aiming at the real entity relation result and the predicted entity relation result of the predicted text data.

And step S2400, performing corresponding processing on the text processing model according to the evaluation index value.

In this embodiment, after obtaining the evaluation index value of the text processing model according to the above step S2300, the text processing model may be processed according to the evaluation index value in the step S2400.

In an example, the performing corresponding processing on the text processing model according to the evaluation index value in step S2400 may include: and under the condition that the evaluation index value is greater than or equal to the evaluation index threshold value, the text processing model is applied on line.

In another example, in step S2400, according to the evaluation index value, performing corresponding processing on the text processing model may further include: and under the condition that the evaluation index value is smaller than the evaluation index threshold value, continuing to train the text processing model based on the text processing model training operator again.

For example, resuming training of the text processing model based on the text processing model training operator may include: and adjusting the hyper-parameters for model training in the text processing model training operator, and continuing to train the text processing model based on the adjusted hyper-parameters. The parameter adjusting algorithm for generating the hyper-parameters in the training operator of the text processing model can be reselected; and generating a new hyper-parameter according to the reselected parameter adjusting algorithm, and continuously training the text processing model based on the new hyper-parameter.

For another example, the number of the training text data is adjusted, and the adjusted training text data and the corresponding real labels thereof are input into a text processing model training operator to continue training the text processing model.

For another example, resuming training of the text processing model based on the text processing model training operator may further include: and adjusting a model training algorithm for training the model in the text processing model training operator to continue training the text processing model according to the adjusted model training algorithm.

In one embodiment, in the case that the text processing model is a text classification model, the configuration items involved in the first configuration interface further include: at least one of a configuration item of the application resource and a configuration item of a text language type corresponding to the text data.

The application resources may include a CPU and a GPU (graphics Processing unit), where a CPU model and a GPU model are built in the electronic device 1000, and the corresponding models are used for training according to selection of a user.

The configuration item of the text language type is the configuration of the language for performing the text classification training, and the text language can be Chinese or English.

According to the embodiment, because the mode of the GPU and the mode of the CPU are simultaneously supported, a user without GPU resources can train, predict, evaluate and go online the text classification model, and the whole process of landing the NLP text classification capability is completed.

Moreover, the method supports text classification of both Chinese and English texts, when a user trains a text classification model, the user only needs to select the Chinese text or the English text to be added for training, and the electronic equipment 1000 automatically trains a model which can best meet business requirements for the user according to different selected languages.

In one embodiment, in the case that the text processing model is an entity extraction model, the input source configuration items in the first configuration interface further relate to selection items of a text data white list. The text data white list is a list defining entity information, and the entity information includes an entity name and an entity type corresponding to the entity name, for example, the defined entity name is "apple", and the corresponding entity type is "company". Because the content of the white list is obtained through certain empirical precipitation, the white list function is integrated during entity extraction model training, and when the model is applied, the content which accords with the setting in the text can be extracted automatically according to the setting of the white list. Here, the text processing method in the machine learning platform of the present disclosure further includes:

training an operator by utilizing an entity extraction model, and analyzing a text data white list; and operating the analyzed text data white list to obtain entity names and corresponding entity types defined in the text data white list, so that an entity extraction model predictor predicts the input predicted text data according to the defined entity names and the corresponding entity types to obtain the predicted entity types of all the entity names in the predicted text data.

In this embodiment, the text data white list may be stored in an Xml format or a Json format, where an entity extraction model training operator needs to be used to analyze the text data white list, and then the text data white list after analysis is run to obtain entity names and entity types corresponding to the entity names defined in the text data white list, for example, an entity type corresponding to an entity name "apple" in the white list obtained by the entity extraction model training is "company", and in the following entity extraction model prediction, as long as an entity name "apple" appears in the text data, the entity name is extracted as a "company" type.

< apparatus embodiment >

In this embodiment, a text data processing apparatus 4000 in a machine learning platform is provided, as shown in fig. 4, including a model training module 4100, a model prediction module 4200, a model evaluation module 4300, and a model processing module 4400.

The model training module 4100 is configured to input training text data and corresponding real tags into a text processing model training operator to train a text processing model.

The model prediction module 4200 is configured to input the text processing model and the predicted text data into a text processing model predictor, and obtain a prediction tag of the predicted text data.

The model evaluation module 4300 is configured to input the prediction tag and the real tag corresponding to the predicted text data into a text processing model evaluation operator to obtain an evaluation index value of the text processing model.

The model processing module 4400 is configured to perform corresponding processing on the text processing model according to the evaluation index value.

In one embodiment, the text processing model is a text classification model, the true labels and the predicted labels are text classification results, the text processing model training operator is a text classification model training operator, the text processing model prediction operator is a text classification model prediction operator, and the text processing model evaluation operator is a text classification model evaluation operator;

alternatively, the first and second electrodes may be,

or

In one embodiment, the apparatus 4000 further comprises an acquisition module (not shown in the figures) configured to: inputting the obtained initial text data into a text data splitting operator to split the initial text data into training text data and predicted text data;

In one embodiment, the obtaining module is further configured to: providing at least one data import path; importing the historical text data from the selected data import path; and saving the imported historical text data.

In one embodiment, the model training module 4100 is further configured to: in response to a configuration operation for the text processing model training operator, providing a first configuration interface for configuring the text processing model training operator; performing feature engineering processing on the training text data and the corresponding real label thereof by the text processing model training operator according to first configuration information input by the first configuration interface to obtain a training text sample; and training a text processing model based on the training text sample according to a preset model training algorithm.

In one embodiment, the first configuration interface relates to at least one of the following configuration items: the method comprises the following steps of inputting source configuration items, target value field configuration items, model training algorithm configuration items, parameter adjusting algorithms and hyper-parameter configuration items of text processing model training operators and configuration items of text data maximum length.

In one embodiment, where the text processing model is the text classification model,

In one embodiment, where the text processing model is the entity extraction model,

In one embodiment, the model training module 4100 is further configured to: analyzing the text data white list by utilizing the entity extraction model training operator; and operating the analyzed text data white list to obtain entity names and entity types corresponding to the entity names defined in the text data white list, so that the instance extraction model predictor predicts the input predicted text data according to the defined entity names and the entity types corresponding to the entity names to obtain the predicted entity types of all the entity names in the predicted text data.

In one embodiment, the model prediction module 4200 is further configured to: providing a second configuration interface for configuring the text processing model predictor in response to a triggering operation for the text processing model predictor; and the text processing model predictor provides a prediction label for the predicted text data by utilizing the trained text processing model according to second configuration information input through the second configuration interface.

In one embodiment, the second configuration interface relates to at least one of the following configuration items: and the input source configuration item and the predicted target value field configuration item of the text processing model predictor.

In one embodiment, the model predictor module 4300 is further configured to: providing a third configuration interface for configuring the text processing model evaluation operator in response to a triggering operation for the text processing model evaluation operator; and the evaluation operator of the processing model compares the predicted label with a real label corresponding to the predicted text data according to third configuration information input through the third configuration interface to obtain an evaluation index value of the text processing model.

In one embodiment, the third configuration interface relates to at least one of the following configuration items: and the text processing model evaluates the input source configuration item of an operator and evaluates the configuration item of an index.

In one embodiment, the model processing module 4400 is further configured to: and applying the text processing model on line under the condition that the evaluation index value is greater than or equal to an evaluation index threshold value.

In one embodiment, the model processing module 4400 is further configured to: and under the condition that the evaluation index value is smaller than the evaluation index threshold value, continuing to train the text processing model based on the text processing model training operator again.

In one embodiment, the model processing module 4400 is further configured to: and adjusting the hyper-parameters for model training in the text processing model training operator, and continuing to train the text processing model based on the adjusted hyper-parameters.

In one embodiment, the model processing module 4400 is further configured to: reselecting a parameter adjusting algorithm used for generating the hyperparameter in the text processing model training operator; and generating a new hyper-parameter according to the reselected parameter adjusting algorithm, and continuously training the text processing model based on the new hyper-parameter.

In one embodiment, the model processing module 4400 is further configured to: and adjusting the quantity of the training text data, and inputting the adjusted training text data and the real labels corresponding to the training text data into the training operator of the text processing model to continue training the text processing model.

In one embodiment, the model processing module 4400 is further configured to: and adjusting a model training algorithm for training the model in the text processing model training operator to continue training the text processing model according to the adjusted model training algorithm.

< storage Medium embodiment >

The present embodiment provides a computer-readable storage medium, wherein a computer program is stored thereon, which computer program, when being executed by a processor, realizes the method according to any one of the above-mentioned method embodiments.

The present invention may be an apparatus, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A text data processing method in a machine learning platform comprises the following steps:

2. The method of claim 1, wherein,

the text processing model is a text classification model, the real label and the prediction label are text classification results, the text processing model training operator is a text classification model training operator, the text processing model prediction operator is a text classification model prediction operator, and the text processing model evaluation operator is a text classification model evaluation operator;

alternatively, the first and second electrodes may be,

or

3. The method of claim 1, wherein the method further comprises the step of obtaining the training text data and the predictive text data based on the obtained initial text data,

4. The method of claim 3, wherein obtaining the historical text data comprises:

providing at least one data import path;

and saving the imported historical text data.

5. The method of claim 2, wherein the method further comprises:

6. The method of claim 5, wherein the first configuration interface relates to at least one of the following configuration items: the method comprises the following steps of inputting source configuration items, target value field configuration items, model training algorithm configuration items, parameter adjusting algorithms and hyper-parameter configuration items of text processing model training operators and configuration items of text data maximum length.

7. The method of claim 6, wherein, in the case that the text processing model is the text classification model,

8. A text data processing apparatus in a machine learning platform, comprising:

9. An apparatus comprising at least one computing device and at least one storage device, wherein the at least one storage device is to store instructions for controlling the at least one computing device to perform the method of any of claims 1 to 7; alternatively, the apparatus implements the apparatus of claim 8 through the computing device and the storage device.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.