CN111191795B - Method, device and system for training machine learning model - Google Patents
Method, device and system for training machine learning model Download PDFInfo
- Publication number
- CN111191795B CN111191795B CN201911412450.0A CN201911412450A CN111191795B CN 111191795 B CN111191795 B CN 111191795B CN 201911412450 A CN201911412450 A CN 201911412450A CN 111191795 B CN111191795 B CN 111191795B
- Authority
- CN
- China
- Prior art keywords
- parameter
- setting
- index
- machine learning
- learning model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The invention provides a method, a device and a system for training a machine learning model, wherein the method comprises the following steps: acquiring preset parameters of a machine learning model, parameter values of the preset parameters and setting indexes for evaluating the effect of the machine learning model; acquiring an index value of a set index obtained by training a machine learning model according to a parameter value of a set parameter; recording the parameter value of the set parameter and the index value of the set index in at least one training process of the machine learning model, and generating a chart reflecting the corresponding relation between the parameter value of the set parameter and the index value of the set index.
Description
Technical Field
The present invention relates to the field of machine learning technology, and more particularly, to a method of training a machine learning model, a device for training a machine learning model, a system comprising at least one computing device and at least one storage device, and a readable storage medium.
Background
Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence.
In the training process of the traditional machine learning model, the machine learning model is usually trained for multiple times by adjusting the parameter value of at least one parameter of the machine learning model, and information such as the parameter values of all parameters, index values of all indexes, training samples and the like in each training process is recorded in a corresponding training log.
If the user needs to adjust the parameter values according to the historical parameter values and the corresponding index values in the training process, the training logs obtained by training the machine learning model each time need to be searched, and the needed parameter values and index values are screened out from a large amount of information recorded in the training logs, so that the searching process is very tedious and time-consuming.
Disclosure of Invention
It is an object of the present invention to provide a new solution for training a machine learning model.
According to a first aspect of the present invention, there is provided a method of training a machine learning model, comprising:
acquiring preset parameters of a machine learning model, parameter values of the preset parameters and setting indexes for evaluating the effect of the machine learning model;
acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter;
Recording parameter values of the set parameters and index values of the set indexes in at least one training process of the machine learning model, and generating a chart reflecting the corresponding relation between the parameter values of the set parameters and the index values of the set indexes.
Optionally, the obtaining the preset parameters of the machine learning model, the parameter values of the preset parameters, and the setting indexes for evaluating the effect of the machine learning model includes:
providing a first interface for acquiring the setting parameters, parameter values of the setting parameters and the setting indexes;
and acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes through the first interface.
Optionally, the obtaining the setting parameter, the parameter value of the setting parameter, and the setting index through the first interface includes:
acquiring an algorithm code of the machine learning model through the first interface;
identifying all parameters involved in the algorithm code or preset reference parameters as the setting parameters; identifying all indexes or preset reference indexes involved in the algorithm code as the set indexes;
And acquiring parameter values of the set parameters in the algorithm codes.
Optionally, the providing the first interface for obtaining the setting parameter, the parameter value of the setting parameter, and the setting index includes:
providing a graphical interface of a data processing flow chart of the machine learning model, wherein the graphical interface comprises the first interface.
Optionally, the graphical interface further includes a second interface for obtaining a logic code, where the logic code is configured to obtain the setting index according to the parameter value of the setting parameter;
the method further comprises the steps of:
acquiring the logic code through the second interface;
and obtaining the index value of the setting index according to the logic code and the parameter value of the setting parameter.
Optionally, the providing the first interface for obtaining the setting parameter, the parameter value of the setting parameter, and the setting index includes:
the obtaining the setting parameter, the parameter value of the setting parameter and the setting index through the first interface includes:
acquiring a designated command line input through the first interface; the specified command line is used for calling a recording service, and comprises the set parameters, parameter values of the set parameters and the set indexes;
And acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes according to the specified command line.
Optionally, the obtaining the index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter includes:
responding to a request for reading the parameter value of the set parameter in the training process of the machine learning model, providing the parameter value of the set parameter, and training the machine learning model;
and acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
Optionally, the method further comprises:
time information is recorded for each training of the machine learning model.
Optionally, the recording the parameter value of the setting parameter and the index value of the setting index in at least one training process of the machine learning model includes:
and converting the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information acquired in each training process of the machine learning model into json files according to a preset format, and storing the json files in a database.
Optionally, the setting parameters are multiple, the setting indexes are multiple,
the method further comprises the steps of:
providing a third interface for selecting setting parameters and a fourth interface for selecting setting indexes;
acquiring target setting parameters selected by a user through the third interface and target setting indexes selected through the fourth interface;
the generating a graph reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index includes:
and generating a relation diagram reflecting the corresponding relation between the parameter value of the target setting parameter and the index value of the target setting index.
Optionally, the method further comprises:
responding to the operation of adjusting the parameter value of the target setting parameter by a user, and acquiring the adjusted parameter value;
acquiring a predicted value of the target setting index obtained according to the adjusted parameter value;
and displaying the predicted value of the target setting index corresponding to the parameter value of the target setting parameter after adjustment in the relation diagram.
Optionally, the method further comprises:
acquiring an actual value of the target setting index obtained by retraining the machine learning model according to the parameter value adjusted by the target setting parameter in response to the operation of training the machine learning model according to the adjusted parameter value;
And displaying the actual value of the target setting index corresponding to the parameter value after the target setting parameter is adjusted in a relation chart.
Optionally, the target setting parameter is one, and the relation graph is a scatter graph; alternatively, the target setting parameters are two, and the relationship diagram is a thermodynamic diagram.
According to a second aspect of the present invention, there is provided an apparatus for training a machine learning model, comprising:
the first acquisition module is used for acquiring preset parameters of a preset machine learning model, parameter values of the preset parameters and setting indexes for evaluating the effect of the machine learning model;
the second acquisition module is used for acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter;
the chart generation module is used for recording the parameter value of the set parameter and the index value of the set index in at least one training process of the machine learning model, and generating a chart reflecting the corresponding relation between the parameter value of the set parameter and the index value of the set index.
Optionally, the first obtaining module is specifically configured to:
providing a first interface for acquiring the setting parameters, parameter values of the setting parameters and the setting indexes;
And acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes through the first interface.
Optionally, the obtaining the setting parameter, the parameter value of the setting parameter, and the setting index through the first interface includes:
acquiring an algorithm code of the machine learning model through the first interface;
identifying all parameters involved in the algorithm code or preset reference parameters as the setting parameters; identifying all indexes or preset reference indexes involved in the algorithm code as the set indexes;
and acquiring parameter values of the set parameters in the algorithm codes.
Optionally, the providing the first interface for obtaining the setting parameter, the parameter value of the setting parameter, and the setting index includes:
providing a graphical interface of a data processing flow chart of the machine learning model, wherein the graphical interface comprises the first interface.
Optionally, the graphical interface further includes a second interface for obtaining a logic code, where the logic code is configured to obtain the setting index according to the parameter value of the setting parameter;
The apparatus further comprises:
means for obtaining the logical code through the second interface;
and a module for obtaining the index value of the setting index according to the logic code and the parameter value of the setting parameter.
Optionally, the providing the first interface for obtaining the setting parameter, the parameter value of the setting parameter, and the setting index includes:
the obtaining the setting parameter, the parameter value of the setting parameter and the setting index through the first interface includes:
acquiring a designated command line input through the first interface; the specified command line is used for calling a recording service, and comprises the set parameters, parameter values of the set parameters and the set indexes;
and acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes according to the specified command line.
Optionally, the second obtaining module is configured to:
responding to a request for reading the parameter value of the set parameter in the training process of the machine learning model, providing the parameter value of the set parameter, and training the machine learning model;
and acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
Optionally, the method further comprises:
and means for recording time information for each training of the machine learning model.
Optionally, the recording the parameter value of the setting parameter and the index value of the setting index in at least one training process of the machine learning model includes:
and converting the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information acquired in each training process of the machine learning model into json files according to a preset format, and storing the json files in a database.
Optionally, the setting parameters are multiple, the setting indexes are multiple,
the apparatus further comprises:
a module for providing a third interface for selecting setting parameters and a fourth interface for selecting setting indicators;
a module for acquiring the target setting parameters selected by the user through the third interface and the target setting indexes selected by the user through the fourth interface;
the generating a graph reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index includes:
and generating a relation diagram reflecting the corresponding relation between the parameter value of the target setting parameter and the index value of the target setting index.
Optionally, the method further comprises:
a module for acquiring an adjusted parameter value in response to an operation of adjusting the parameter value of the target setting parameter by a user;
a module for obtaining a predicted value of the target setting index obtained according to the adjusted parameter value;
and a module for displaying the predicted value of the target setting index corresponding to the parameter value after the target setting parameter adjustment in the relation chart.
Optionally, the method further comprises:
means for obtaining an actual value of the target setting index obtained by retraining the machine learning model from the adjusted parameter value according to the target setting parameter in response to an operation of training the machine learning model from the adjusted parameter value;
and a module for displaying the actual value of the target setting index corresponding to the parameter value after the target setting parameter adjustment in a relation chart.
Optionally, the target setting parameter is one, and the relation graph is a scatter graph; alternatively, the target setting parameters are two, and the relationship diagram is a thermodynamic diagram.
According to a third aspect of the present invention there is provided a system comprising at least one computing device and at least one storage device, wherein the at least one storage device is adapted to store instructions for controlling the at least one computing device to perform a method according to the first aspect of the present invention.
According to a fourth aspect of the present invention there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method according to the first aspect of the present invention.
The invention has the beneficial effects that the corresponding relation between the parameter value of the set parameter and the index value of the set index in each training process is displayed through the visual chart, so that a user can conveniently check the training result of the machine learning model, and the parameter value of the set parameter can be adjusted according to the training result.
Other features of the present invention and its advantages will become apparent from the following detailed description of exemplary embodiments of the invention, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a block diagram of one example of a hardware configuration of an electronic device that may be used to implement an embodiment of the invention.
FIG. 2 is a flow diagram of a method of training a machine learning model according to an embodiment of the present invention;
FIGS. 3 and 4 are related codes of a logistic regression algorithm according to an embodiment of the present invention;
FIG. 5 is a schematic illustration of a graphical interface according to an embodiment of the invention;
FIG. 6 is a schematic diagram of a second interface according to an embodiment of the invention;
FIG. 7 is a table schematic diagram according to an embodiment of the invention;
FIG. 8 is a schematic diagram of a scatter plot according to an embodiment of the invention;
FIG. 9 is a schematic diagram of a thermodynamic diagram in accordance with an embodiment of the present invention;
FIG. 10 is a block schematic diagram of an apparatus for training a machine learning model according to an embodiment of the present invention;
fig. 11 is a block schematic diagram of a system according to an embodiment of the invention.
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.
The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
Various embodiments and examples according to embodiments of the present invention are described below with reference to the accompanying drawings.
< hardware configuration >
Fig. 1 is a block diagram showing a hardware configuration of an electronic device 1000 in which an embodiment of the present invention can be implemented.
The electronic device 1000 may be a laptop, desktop, cell phone, tablet, etc. As shown in fig. 1, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a central processing unit CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, ROM (read only memory), RAM (random access memory), nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 1400 can be capable of wired or wireless communication, and specifically can include Wifi communication, bluetooth communication, 2G/3G/4G/5G communication, and the like. The display device 1500 is, for example, a liquid crystal display, a touch display, or the like. The input device 1600 may include, for example, a touch screen, keyboard, somatosensory input, and the like. A user may input/output voice information through the speaker 1700 and microphone 1800.
The electronic device shown in fig. 1 is merely illustrative and is in no way meant to limit the invention, its application or uses. In an embodiment of the present invention, the memory 1200 of the electronic device 1000 is configured to store instructions for controlling the processor 1100 to operate to perform any one of the methods for training a machine learning model provided by the embodiment of the present invention. It will be appreciated by those skilled in the art that although a plurality of devices are shown for the electronic apparatus 1000 in fig. 1, the present invention may relate to only some of the devices thereof, for example, the electronic apparatus 1000 relates to only the processor 1100 and the storage device 1200. The skilled person can design instructions according to the disclosed solution. How the instructions control the processor to operate is well known in the art and will not be described in detail here.
< method example >
In this embodiment, a method of training a machine learning model is provided. The method of training a machine learning model may be implemented by an electronic device. The electronic device may be an electronic device 1000 as shown in fig. 1.
As shown in fig. 2, the method for training a machine learning model of the present embodiment may include the following steps S2100 to S2300:
In step S2100, a preset parameter of the machine learning model, a parameter value of the preset parameter, and a setting index for evaluating the effect of the machine learning model are obtained.
The set parameters may be parameters for training the machine learning model, and may be, for example, super parameters of the machine learning model, the number of training rounds, the step size, and the like.
The set index may be an index for evaluating the effect of the machine learning model. Specifically, the setting index may be an Area (AUC) surrounded by the coordinate axis Under the receiver operation characteristic Curve (ROC Curve), an accuracy rate, a recall rate, and the like.
In one embodiment of the present invention, the step of obtaining the preset parameters of the machine learning model, the parameter values of the set parameters, and the set indexes for evaluating the effects of the machine learning model may include steps S2110 to S2120 as follows:
step S2110 provides a first interface for acquiring the setting parameters, parameter values of the setting parameters, and setting indexes.
In step S2120, the setting parameters, parameter values of the setting parameters, and setting indexes are acquired through the first interface.
In a first embodiment of the present invention, acquiring the setting parameters, parameter values of the setting parameters, and setting indexes through the first interface may include:
Acquiring a designated command line input through a first interface; the command line is designated for calling the recording service, wherein the command line comprises a setting parameter, a parameter value of the setting parameter and a setting index;
and acquiring the setting parameters, parameter values of the setting parameters and setting indexes according to the designated command line.
Correspondingly, in this embodiment, the first interface may be a window for inputting a specified command line, and the window may be provided in response to a specified operation by the user. For example, the first interface may be a shell under the linux operating system. Shell is the user interface of the system, providing an interface for the user to interact with the kernel.
In this embodiment, a command line for calling a service for recording a setting parameter and a parameter value of the setting parameter, and a command line for calling an index value for recording a setting index and a setting index may be input through the first interface, respectively.
Taking Python code as an example, there is a lib named "page", and the service for calling the parameter value of the setting parameter is log_parameter, then the designated command line for calling the service for calling the parameter value of the setting parameter may be:
sage.log_parameter(key＝“metrics_1”,value＝metrics_1)
The parameters of the service log_parameter include a name (key) of a specified parameter and a source (value) of a parameter value of the specified parameter.
Taking Python code as an example, for example, there is a lib named as page, and the service for calling the index value recording the setting index and setting index is log_metric, then the designated command line for calling the service for recording the index value recording the setting index and setting index may be:
sage.log_metric(key＝“AUC”,value＝auc)
the parameters of the service log_metric include a name (key) of the specified index and a source (value) of the index value of the specified index.
In the second embodiment of the present invention, the acquiring the setting parameters, the parameter values of the setting parameters, and the setting index through the first interface may include steps S2121 to S2123 as follows:
in step S2121, an algorithm code of the machine learning model is acquired through the first interface.
Correspondingly, the first interface may be a region in the interface into which a code file can be dragged or an algorithm code can be input, and the user may be to put a file of the algorithm code of the machine learning model into the first interface or input the algorithm code of the machine learning model into the first interface.
Step S2122, identifying all parameters involved in the algorithm code or preset reference parameters as setting parameters; all indexes or preset reference indexes involved in the algorithm codes are identified and used as setting indexes.
In one embodiment of the present invention, the preset reference parameters and preset reference indexes may be preset according to an application scenario or specific requirements, and may not be set only for the algorithm codes of the machine learning model acquired through the first interface. Thus, the reference parameters may include parameters not related to the algorithm code, and the reference indexes may also include indexes not related to the algorithm code.
In one embodiment of the present invention, for some commonly used machine learning packages, such as sklearn, xgboost, keras, the recording mode of the setting parameters and the setting indexes in the API interface of these machine learning packages may be fixed.
Fig. 3 and 4 are examples of sklearn logistic regression algorithms. In the example shown in fig. 3, the part in the box is the relevant code defining the parameter value of the setting parameter. In the example shown in fig. 4, the relevant code for calculating the setting index of the accuracy_score is divided into the blocks.
For the algorithm code of this type of tool package, the setting parameters and the setting indexes can be directly identified from the algorithm code.
For example, the auto_log_parameter and auto_log_metric methods may be added, and all parameters and metrics may be automatically recorded according to the related codes of the set parameters and set metrics in the algorithm codes of this type of tool package.
In step S2123, a parameter value of the set parameter in the algorithm code is acquired.
In the present embodiment, the parameter values of the setting parameters may be defined in advance in the algorithm code. Thus, the parameter value of the setting parameter can be obtained from the algorithm code.
In a third embodiment of the present invention, providing a first interface for acquiring a setting parameter, a parameter value of the setting parameter, and a setting index may include:
a graphical interface of a data processing flow chart of the machine learning model is provided, wherein the graphical interface comprises a first interface.
The data processing flow diagrams may be used to represent the flow of performing data processing by various nodes in the diagrams. The nodes may be operators or models for performing data processing, may be data for processing, and may be samples for training operators or models.
In one embodiment of the invention, the data processing flow diagram is a directed acyclic graph. The directed acyclic graph is called DAG graph for short, and refers to a loop-free directed graph.
The window including the first interface in the graphical interface may be as shown in fig. 5, and the user may input the setting parameters of the machine learning model by clicking a button for adding the setting parameters. It is also possible to provide all the parameters involved in the machine learning model for the user to select the setting parameters from in response to the user clicking the input box for setting parameters.
Correspondingly, the user may input the setting index for evaluating the machine learning model by clicking a button for adding the setting index. It is also possible to provide all the indexes related to the machine learning model for the user to select the set indexes in response to the operation of clicking the input box of the set indexes by the user.
In one embodiment of the present invention, the setting parameters, parameter values of the setting parameters, and setting indexes may be obtained directly from the first interface.
In one embodiment of the present invention, the set parameter and the set index may be obtained through the first interface, and then the parameter value of the set parameter in the algorithm code may be obtained.
Step S2200, obtaining the index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
In another embodiment of the present invention, the graphical interface may further include a second interface for acquiring a logic code, where the logic code is configured to acquire the setting index according to the parameter value of the setting parameter; then, the manner of acquiring the index value of the specified index may further include:
obtaining a logic code through a second interface;
and obtaining the index value of the setting index according to the logic code and the parameter value of the setting parameter.
As partially shown in the block of fig. 6, the second interface may be a logic code input window through which a user can input a logic code for configuring a manner of acquiring a setting index according to a parameter value of a setting parameter.
Further, the index value of the setting index may be obtained from a logic code and a machine learning model trained on the parameter values of the setting parameters.
In the foregoing first and third embodiments, the parameter values of the setting parameters may not be set in the algorithm code in advance, but set by the user through the command line or the graphical interface. Then, the method of acquiring the index value of the setting index may include steps S2210 to S2220 as follows:
in step S2210, the parameter values of the set parameters are provided to perform training of the machine learning model in response to a request for reading the parameter values of the set parameters during the training of the machine learning model.
In one embodiment of the present invention, the user may set the parameter value of the set parameter in a preset recording function module, and correspondingly, the method of the embodiment of the present invention may be executed by the recording function module.
Specifically, the recording function module may provide the parameter values of the set parameters to the machine learning model for training by responding to a request of the machine learning model to read the parameter values of the set parameters in a training process.
Step S2220 is to obtain the index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
Specifically, after the machine learning model trains the obtained index value of the set index according to the parameter value of the set parameter, the recording function module obtains the index value of the set index.
Step S2300, recording the parameter value of the set parameter and the index value of the set index in at least one training process of the machine learning model, and generating a graph reflecting the correspondence between the parameter value of the set parameter and the index value of the set index.
In the embodiment of the invention, the corresponding relation between the parameter value of the set parameter and the index value of the set index in each training process is displayed through the visual chart, so that a user can conveniently check the training result of the machine learning model, and the parameter value of the set parameter can be adjusted according to the training result.
In one embodiment of the present invention, the method may further comprise:
And recording time information of each training machine learning model, and displaying the time information of each training machine learning model in the chart. For example, as shown in fig. 7.
In one embodiment of the present invention, the method for recording the parameter value of the set parameter and the index value of the set index in at least one training process of the machine learning model may include:
the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information of the training machine learning model are converted into json files according to a preset format and stored in a database.
Thus, a map reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index can be generated from the json file obtained by training the machine learning model each time.
In one embodiment of the present invention, the chart may be a table, which may be a parameter value representing a set parameter and an index value of a set index of the machine learning model during each training as shown in fig. 7. In the table shown in fig. 7, it is possible that the parameter value and the index value of each row correspond to the same training process.
In one embodiment of the present invention, in the case where the number of setting indices is 1 and the number of setting parameters is 1 or 2, the map may be a relationship map reflecting the correspondence between the parameter values of the setting parameters and the index values of the setting indices.
Specifically, when the number of setting indices is 1 and the number of setting parameters is 1, the relationship map may be a scatter diagram, for example, as shown in fig. 8. When the number of setting indices is 1 and the number of setting parameters is 2, the map may be a thermodynamic diagram, for example, as shown in fig. 9.
In one embodiment of the present invention, the setting parameters are plural, and the setting indexes are plural, then the method may further include:
providing a third interface for selecting setting parameters and a fourth interface for selecting setting indexes; and acquiring the target setting parameters selected by the user through the third interface and the target setting indexes selected through the fourth interface.
On this basis, generating a graph reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index may include:
a relationship map is generated that reflects the correspondence between the parameter values of the target setting parameters and the index values of the target setting indices.
In one embodiment of the present invention, in response to a user clicking on the third interface, a list of setting parameters is provided, and all setting parameters may be included in the list for the user to select at most two target setting parameters from the list. Correspondingly, in response to the operation of clicking the fourth interface by the user, a list of setting indexes is provided, wherein the list can comprise all setting indexes so that the user can select at most one target setting index from the list.
In the case where the target setting parameter is one, the relationship map may be a scatter diagram; when the target setting parameters are two, the map is a thermodynamic diagram.
In another embodiment of the present invention, the third interface may be an input box for inputting a target setting parameter, and the fourth interface may be an input box for inputting a target setting index. For example, as shown in fig. 8 and 9.
In the example shown in fig. 8, the third interface may be an input box for selecting the abscissa and the fourth interface may be an input box for selecting the ordinate. In response to a user clicking on a button for generating a scatter diagram, a corresponding relation scatter diagram is generated between a parameter value reflecting a target setting parameter and an index value of a target setting index.
In the example shown in fig. 9, the third interface may be an input box for selecting an abscissa and an input box for selecting an ordinate, and the fourth interface may be an input box for selecting a thermodynamic item. In response to a user clicking a button for generating a thermodynamic diagram, a thermodynamic diagram reflecting a correspondence relationship between a parameter value of a target setting parameter and an index value of a target setting index is generated.
In one embodiment of the present invention, the method may further comprise:
responding to the operation of adjusting the parameter value of the target setting parameter by a user, and acquiring the adjusted parameter value; acquiring a predicted value of a set index obtained according to the adjusted parameter value; the predicted value of the setting index corresponding to the parameter value after the target setting parameter adjustment is shown in the relation chart.
Specifically, the user may reenter the parameter value of the target setting parameter through the first interface, so as to achieve the effect of adjusting the parameter value of the target setting parameter.
In one embodiment of the present invention, a button for providing a predicted index expression in the graphical interface may be provided, as shown in fig. 8, and the user triggers to obtain a predicted value of the set index obtained according to the adjusted parameter value by clicking the button for providing the predicted index expression; the operation of the predicted value of the setting index corresponding to the parameter value after the target setting parameter adjustment is shown in the relationship diagram.
The predicted value of the target setting index is not obtained by training the machine learning model according to the parameter value after the adjustment of the target setting parameter, and may be the same as or different from the actual index value of the target setting index obtained by training the machine learning model according to the parameter value after the adjustment of the target setting parameter.
In one embodiment of the present invention, the predicted value of the target setting index corresponding to the parameter value after the adjustment of the target setting parameter may be obtained according to the historical relative relationship between the target setting parameter and the target setting index.
In one embodiment of the present invention, the method may further comprise:
responding to the operation of training the machine learning model according to the adjusted parameter values, and acquiring actual values of target setting indexes obtained by retraining the machine learning model according to the parameter values adjusted by the target setting parameters; the actual values of the target setting indicators corresponding to the parameter values after the target setting parameters are adjusted are shown in the relation diagram.
In one embodiment of the present invention, a button executed by using a new parameter may be provided in the graphical interface, as shown in fig. 8, and the user may trigger an operation of training the machine learning model according to the adjusted parameter value by clicking the button represented by the prediction index, and display an actual value of the target setting index corresponding to the adjusted parameter value in the relationship graph, or replace the prediction value of the target setting index corresponding to the adjusted parameter value in the relationship graph with the actual index value.
In one embodiment of the present invention, in the relationship diagram, the predicted value of the target setting index corresponding to the parameter value after the adjustment of the target setting parameter and the actual value of the target setting index obtained by training the model may be different from each other, so as to distinguish the predicted value and the actual value of the target setting index.
< device example >
In this embodiment, an apparatus 5000 for training a machine learning model is provided, as shown in fig. 10, including a first obtaining module 5100, a second obtaining module 5200, and a chart generating module 5300. The first obtaining module 5100 is configured to obtain a preset parameter of the machine learning model, a parameter value of the preset parameter, and a setting index for evaluating an effect of the machine learning model; the second obtaining module 5200 is configured to obtain an index value of a set index obtained by training a machine learning model according to a parameter value of a set parameter; the chart generation module 5300 is configured to record a parameter value of a set parameter and an index value of a set index of the machine learning model in at least one training process, and generate a chart reflecting a correspondence between the parameter value of the set parameter and the index value of the set index.
In one embodiment of the present invention, the first acquiring module 5100 may be specifically configured to:
providing a first interface for acquiring a setting parameter, a parameter value of the setting parameter and a setting index;
the setting parameters, parameter values of the setting parameters and setting indexes are obtained through the first interface.
In one embodiment of the present invention, obtaining the setting parameters, parameter values of the setting parameters, and setting indexes through the first interface includes:
acquiring an algorithm code of a machine learning model through a first interface;
identifying all parameters involved in the algorithm code or preset reference parameters as setting parameters; identifying all indexes or preset reference indexes involved in algorithm codes as set indexes;
and acquiring parameter values of set parameters in the algorithm codes.
In one embodiment of the present invention, providing a first interface for acquiring a setting parameter, a parameter value of the setting parameter, and a setting index includes:
a graphical interface of a data processing flow diagram of a machine learning model is provided, wherein the graphical interface includes a first interface.
In one embodiment of the present invention, the graphical interface further includes a second interface for acquiring a logic code, where the logic code is configured to acquire a setting index according to a parameter value of the setting parameter;
The apparatus 5000 for training a machine learning model may further include:
a module for obtaining a logic code through a second interface;
and a module for obtaining the index value of the setting index according to the logic code and the parameter value of the setting parameter.
In one embodiment of the present invention, providing a first interface for acquiring a setting parameter, a parameter value of the setting parameter, and a setting index includes:
the step of obtaining the setting parameters, the parameter values of the setting parameters and the setting indexes through the first interface comprises the following steps:
acquiring a designated command line input through a first interface; the command line is designated for calling the recording service, wherein the command line comprises a setting parameter, a parameter value of the setting parameter and a setting index;
and acquiring the setting parameters, parameter values of the setting parameters and setting indexes according to the designated command line.
In one embodiment of the invention, the second acquisition module 5200 may also be configured to:
responding to a request for reading parameter values of set parameters in the training process of the machine learning model, providing the parameter values of the set parameters, and training the machine learning model;
and acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
In one embodiment of the present invention, the apparatus 5000 for training a machine learning model may further include:
a module for recording time information for each training of the machine learning model.
In one embodiment of the present invention, recording the parameter values of the set parameters and the index values of the set indices of the machine learning model during at least one training process includes:
the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information acquired in each training process of the machine learning model are converted into json files according to a preset format and are stored in a database.
In one embodiment of the present invention, the setting parameters are plural, the setting indexes are plural, and the apparatus 5000 for training a machine learning model may further include:
a module for providing a third interface for selecting setting parameters and a fourth interface for selecting setting indicators;
a module for acquiring the target setting parameters selected by the user through the third interface and the target setting indexes selected by the user through the fourth interface;
generating a graph reflecting a correspondence between a parameter value of the setting parameter and an index value of the setting index includes:
A relationship map is generated that reflects the correspondence between the parameter values of the target setting parameters and the index values of the target setting indices.
In one embodiment of the present invention, the apparatus 5000 for training a machine learning model may further include:
a module for acquiring an adjusted parameter value in response to an operation of adjusting a parameter value of a target setting parameter by a user;
a module for obtaining a predicted value of the target setting index obtained according to the adjusted parameter value;
and a module for displaying the predicted value of the target setting index corresponding to the parameter value after the adjustment of the target setting parameter in the relation diagram.
In one embodiment of the present invention, the apparatus 5000 for training a machine learning model may further include:
a module for obtaining an actual value of a target setting index obtained by retraining the machine learning model according to the parameter value adjusted by the target setting parameter in response to an operation of training the machine learning model according to the adjusted parameter value;
and a module for displaying the actual value of the target setting index corresponding to the parameter value after the adjustment of the target setting parameter in the relation diagram.
In one embodiment of the present invention, the target setting parameter is one, and the relationship diagram is a scatter diagram; alternatively, the target setting parameters are two, and the relationship diagram is a thermodynamic diagram.
It will be appreciated by those skilled in the art that the means 5000 for training the machine learning model may be implemented in various ways. For example, the means 5000 for training the machine learning model may be implemented by an instruction configuration processor. For example, instructions may be stored in ROM and when the device is booted, the instructions are read from ROM into a programmable device to implement the apparatus 5000 that trains the machine learning model. For example, the machine learning model training apparatus 5000 may be solidified into a dedicated device (e.g., ASIC). The means 5000 for training the machine learning model may be divided into separate units or they may be combined together. The means 5000 for training the machine learning model may be implemented by one of the above-described various implementations, or may be implemented by a combination of two or more of the above-described various implementations.
In this embodiment, the apparatus 5000 for training a machine learning model may have various implementation forms, for example, the apparatus 5000 for training a machine learning model may be any functional module running in a software product or an application program that provides a model training service, or a peripheral insert, a plug-in, a patch, etc. of the software product or the application program, or may be the software product or the application program itself.
< System example >
In this embodiment, as shown in fig. 11, a system 6000 of at least one computing device 6100 and at least one storage device 6200 is also provided. The at least one storage 6200 is configured to store executable instructions; the instructions are for controlling at least one computing device 6100 to perform a method of training a machine learning model according to any embodiment of the present invention.
In this embodiment, the system 6000 may be a mobile phone, a tablet computer, a palm computer, a desktop computer, a notebook computer, a workstation, a game console, or a distributed system formed by a plurality of devices.
< computer-readable storage Medium >
In this embodiment, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of training a machine learning model as in any of the embodiments of the invention.
The present invention may be an apparatus, method and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present invention.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.
The foregoing description of embodiments of the invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.
Claims (22)
1. A method of exposing machine learning model process data, comprising:
providing a graphical interface of a data processing flow chart of the machine learning model, wherein the graphical interface comprises a first interface for acquiring setting parameters, parameter values of the setting parameters and setting indexes; the graphical interface further comprises a second interface for obtaining a logic code, wherein the logic code is used for configuring a mode for obtaining the setting index according to the parameter value of the setting parameter;
Acquiring preset parameters of a machine learning model, parameter values of the preset parameters and setting indexes for evaluating the effect of the machine learning model through the first interface;
acquiring the logic code through the second interface, and acquiring an index value of the set index obtained by training the machine learning model according to the logic code and the parameter value of the set parameter;
recording parameter values of the set parameters and index values of the set indexes in at least one training process of the machine learning model, and generating a chart reflecting the corresponding relation between the parameter values of the set parameters and the index values of the set indexes;
the chart is shown.
2. The method of claim 1, the obtaining the setting parameter, the parameter value of the setting parameter, and the setting index through the first interface comprising:
acquiring an algorithm code of the machine learning model through the first interface;
identifying all parameters involved in the algorithm code or preset reference parameters as the setting parameters; identifying all indexes or preset reference indexes involved in the algorithm code as the set indexes;
And acquiring parameter values of the set parameters in the algorithm codes.
3. The method of claim 1, the providing a first interface for obtaining the setting parameter, a parameter value of the setting parameter, and the setting index comprising:
the obtaining the setting parameter, the parameter value of the setting parameter and the setting index through the first interface includes:
acquiring a designated command line input through the first interface; the specified command line is used for calling a recording service, and comprises the set parameters, parameter values of the set parameters and the set indexes;
and acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes according to the specified command line.
4. The method of claim 1, wherein the obtaining the index value of the set index trained by the machine learning model according to the parameter value of the set parameter comprises:
responding to a request for reading the parameter value of the set parameter in the training process of the machine learning model, providing the parameter value of the set parameter, and training the machine learning model;
and acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
5. The method of claim 1, further comprising:
time information is recorded for each training of the machine learning model.
6. The method of claim 5, the recording parameter values for the set parameters and index values for set indices of the machine learning model during at least one training process comprising:
and converting the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information acquired in each training process of the machine learning model into json files according to a preset format, and storing the json files in a database.
7. The method of claim 1, wherein the set parameters are plural, the set index is plural,
the method further comprises the steps of:
providing a third interface for selecting setting parameters and a fourth interface for selecting setting indexes;
acquiring target setting parameters selected by a user through the third interface and target setting indexes selected through the fourth interface;
the generating a graph reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index includes:
And generating a relation diagram reflecting the corresponding relation between the parameter value of the target setting parameter and the index value of the target setting index.
8. The method of claim 7, further comprising:
responding to the operation of adjusting the parameter value of the target setting parameter by a user, and acquiring the adjusted parameter value;
acquiring a predicted value of the target setting index obtained according to the adjusted parameter value;
and displaying the predicted value of the target setting index corresponding to the parameter value of the target setting parameter after adjustment in the relation diagram.
9. The method of claim 8, further comprising:
acquiring an actual value of the target setting index obtained by retraining the machine learning model according to the parameter value adjusted by the target setting parameter in response to the operation of training the machine learning model according to the adjusted parameter value;
and displaying the actual value of the target setting index corresponding to the parameter value after the target setting parameter is adjusted in a relation chart.
10. The method of claim 7, wherein the target setting parameter is one, and the relationship graph is a scatter graph; alternatively, the target setting parameters are two, and the relationship diagram is a thermodynamic diagram.
11. An apparatus for exposing machine learning model process data, comprising:
the first acquisition module is used for providing a graphical interface of a data processing flow chart of the machine learning model, wherein the graphical interface comprises a first interface used for acquiring setting parameters, parameter values of the setting parameters and setting indexes; the graphical interface further comprises a second interface for obtaining a logic code, wherein the logic code is used for configuring a mode for obtaining the setting index according to the parameter value of the setting parameter;
acquiring preset parameters of a machine learning model, parameter values of the preset parameters and setting indexes for evaluating the effect of the machine learning model through the first interface;
the second acquisition module is used for acquiring the logic code through the second interface, and acquiring an index value of the set index obtained by training the machine learning model according to the logic code and the parameter value of the set parameter;
the chart generation module is used for recording the parameter value of the set parameter and the index value of the set index in at least one training process of the machine learning model and generating a chart reflecting the corresponding relation between the parameter value of the set parameter and the index value of the set index;
And the chart display module is used for displaying the chart.
12. The apparatus of claim 11, the obtaining the setting parameter, a parameter value of the setting parameter, and the setting index through the first interface comprising:
acquiring an algorithm code of the machine learning model through the first interface;
identifying all parameters involved in the algorithm code or preset reference parameters as the setting parameters; identifying all indexes or preset reference indexes involved in the algorithm code as the set indexes;
and acquiring parameter values of the set parameters in the algorithm codes.
13. The apparatus of claim 11, the providing a first interface for obtaining the setting parameter, a parameter value of the setting parameter, and the setting indicator comprising:
the obtaining the setting parameter, the parameter value of the setting parameter and the setting index through the first interface includes:
acquiring a designated command line input through the first interface; the specified command line is used for calling a recording service, and comprises the set parameters, parameter values of the set parameters and the set indexes;
And acquiring the setting parameters, the parameter values of the setting parameters and the setting indexes according to the specified command line.
14. The apparatus of claim 11, the second acquisition module to:
responding to a request for reading the parameter value of the set parameter in the training process of the machine learning model, providing the parameter value of the set parameter, and training the machine learning model;
and acquiring an index value of the set index obtained by training the machine learning model according to the parameter value of the set parameter.
15. The apparatus of claim 11, further comprising:
and means for recording time information for each training of the machine learning model.
16. The apparatus of claim 15, the recording parameter values for the set parameters and index values for set indices of the machine learning model during at least one training process comprising:
and converting the set parameters, the current parameter values of the set parameters, the set indexes, the current index values of the set indexes and the time information acquired in each training process of the machine learning model into json files according to a preset format, and storing the json files in a database.
17. The apparatus according to claim 11, wherein the setting parameters are plural, the setting index is plural,
the apparatus further comprises:
a module for providing a third interface for selecting setting parameters and a fourth interface for selecting setting indicators;
a module for acquiring the target setting parameters selected by the user through the third interface and the target setting indexes selected by the user through the fourth interface;
the generating a graph reflecting the correspondence between the parameter value of the setting parameter and the index value of the setting index includes:
and generating a relation diagram reflecting the corresponding relation between the parameter value of the target setting parameter and the index value of the target setting index.
18. The apparatus of claim 17, further comprising:
a module for acquiring an adjusted parameter value in response to an operation of adjusting the parameter value of the target setting parameter by a user;
a module for obtaining a predicted value of the target setting index obtained according to the adjusted parameter value;
and a module for displaying the predicted value of the target setting index corresponding to the parameter value after the target setting parameter adjustment in the relation chart.
19. The apparatus of claim 18, further comprising:
means for obtaining an actual value of the target setting index obtained by retraining the machine learning model from the adjusted parameter value according to the target setting parameter in response to an operation of training the machine learning model from the adjusted parameter value;
and a module for displaying the actual value of the target setting index corresponding to the parameter value after the target setting parameter adjustment in a relation chart.
20. The apparatus of claim 17, wherein the target setting parameter is one, and the relationship graph is a scatter graph; alternatively, the target setting parameters are two, and the relationship diagram is a thermodynamic diagram.
21. A system comprising at least one computing device and at least one storage device, wherein the at least one storage device is to store instructions to control the at least one computing device to perform the method of any one of claims 1 to 10.
22. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911412450.0A CN111191795B (en) | 2019-12-31 | 2019-12-31 | Method, device and system for training machine learning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911412450.0A CN111191795B (en) | 2019-12-31 | 2019-12-31 | Method, device and system for training machine learning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111191795A CN111191795A (en) | 2020-05-22 |
| CN111191795B true CN111191795B (en) | 2023-10-20 |
Family
ID=70707949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911412450.0A Active CN111191795B (en) | 2019-12-31 | 2019-12-31 | Method, device and system for training machine learning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111191795B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112256537B (en) * | 2020-11-12 | 2024-03-29 | 腾讯科技（深圳）有限公司 | Model running state display method and device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6920608B1 (en) * | 1999-05-21 | 2005-07-19 | E Numerate Solutions, Inc. | Chart view for reusable data markup language |
| CN107844837A (en) * | 2017-10-31 | 2018-03-27 | 第四范式（北京）技术有限公司 | The method and system of algorithm parameter tuning are carried out for machine learning algorithm |
| CN109711548A (en) * | 2018-12-26 | 2019-05-03 | 歌尔股份有限公司 | Choosing method, application method, device and the electronic equipment of hyper parameter |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110088011A1 (en) * | 2009-10-14 | 2011-04-14 | Vermeg Sarl | Automated Enterprise Software Development |
| US10817757B2 (en) * | 2017-07-31 | 2020-10-27 | Splunk Inc. | Automated data preprocessing for machine learning |
| US20190102693A1 (en) * | 2017-09-29 | 2019-04-04 | Facebook, Inc. | Optimizing parameters for machine learning models |
-
2019
- 2019-12-31 CN CN201911412450.0A patent/CN111191795B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6920608B1 (en) * | 1999-05-21 | 2005-07-19 | E Numerate Solutions, Inc. | Chart view for reusable data markup language |
| CN107844837A (en) * | 2017-10-31 | 2018-03-27 | 第四范式（北京）技术有限公司 | The method and system of algorithm parameter tuning are carried out for machine learning algorithm |
| CN109711548A (en) * | 2018-12-26 | 2019-05-03 | 歌尔股份有限公司 | Choosing method, application method, device and the electronic equipment of hyper parameter |
Non-Patent Citations (1)
| Title |
|---|
| 面向自然语言处理的深度学习研究;奚雪峰;周国栋;;自动化学报(10);第1445-1465页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111191795A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11036523B2 (en) | Systems and methods for adaptive user interfaces | |
| US20190087490A1 (en) | Text classification method and apparatus | |
| US9460083B2 (en) | Interactive dashboard based on real-time sentiment analysis for synchronous communication | |
| US10503827B2 (en) | Supervised training for word embedding | |
| CN111190718A (en) | Method, device and system for realizing task scheduling | |
| JP2017513138A (en) | Predictive analysis for scalable business process intelligence and distributed architecture | |
| US10976904B2 (en) | Graphical chatbot interface facilitating user-chatbot interaction | |
| CN114787814A (en) | Reference resolution | |
| TW201933830A (en) | Traffic switching method and device and computer equipment | |
| US10592304B2 (en) | Suggesting application programming interfaces based on feature and context analysis | |
| CN112384888A (en) | User interface format adaptation based on context state | |
| CN109062963B (en) | Self-media recommendation method and device and electronic equipment | |
| CN108932320A (en) | Article search method, apparatus and electronic equipment | |
| US20180189403A1 (en) | Website domain specific search | |
| US11869050B2 (en) | Facilitating responding to multiple product or service reviews associated with multiple sources | |
| KR20230006601A (en) | Alignment methods, training methods for alignment models, devices, electronic devices and media | |
| CN111191795B (en) | Method, device and system for training machine learning model | |
| US9760933B1 (en) | Interactive shopping advisor for refinancing product queries | |
| CN114116441A (en) | UI (user interface) testing method and device, electronic equipment and storage medium | |
| US20210117489A1 (en) | Recommendation system based on adjustable virtual indicium | |
| KR20210043894A (en) | Electronic apparatus and method of providing sentence thereof | |
| CN115994807A (en) | Material recommendation method, device and system | |
| US11501172B2 (en) | Accurately identifying members of training data in variational autoencoders by reconstruction error | |
| US11138273B2 (en) | Onboarding services | |
| US20190050474A1 (en) | User interaction during ground truth curation in a cognitive system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |