WO2020239033A1 - 机器学习自动建模过程的展示方法及系统 - Google Patents

机器学习自动建模过程的展示方法及系统 Download PDF

Info

Publication number
WO2020239033A1
WO2020239033A1 PCT/CN2020/092944 CN2020092944W WO2020239033A1 WO 2020239033 A1 WO2020239033 A1 WO 2020239033A1 CN 2020092944 W CN2020092944 W CN 2020092944W WO 2020239033 A1 WO2020239033 A1 WO 2020239033A1
Authority
WO
WIPO (PCT)
Prior art keywords
stage
running
machine learning
card panel
phase
Prior art date
Application number
PCT/CN2020/092944
Other languages
English (en)
French (fr)
Inventor
娄辰
徐昀
Original Assignee
第四范式（北京）技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 第四范式（北京）技术有限公司 filed Critical 第四范式（北京）技术有限公司
Priority to EP20813203.5A priority Critical patent/EP3979148A4/en
Publication of WO2020239033A1 publication Critical patent/WO2020239033A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G06N3/105—Shells for specifying net layout

Definitions

  • the present disclosure generally relates to the field of machine learning automatic modeling, and more specifically, to a display method and system for optimizing the machine learning automatic modeling process.
  • the existing RapidMiner Auto Model interface can be used to browse the results of automatic modeling in the form of data and charts by switching the results directory tree on the left.
  • the existing H2O Driveless AI You can browse the running status of the automatic modeling and general running results and resource occupancy through the interface.
  • the exemplary embodiments of the present disclosure aim to overcome the above-mentioned defect that the existing machine learning automatic modeling display method cannot intuitively and effectively browse the progress of each stage.
  • a method for displaying a machine learning automatic modeling process including: displaying a process card panel corresponding to each stage of the machine learning automatic modeling process, wherein the machine learning automatic modeling process Divided into multiple stages; displaying flowcharts corresponding to the multiple stages, wherein the steps of displaying the progress card panel corresponding to each stage of the machine learning automatic modeling process include: the current process of automatic modeling according to machine learning During the running phase, expand the process card panel corresponding to the currently running phase, while keeping the process card panel corresponding to the phases that have been run and the phase that is not running in a folded state; according to the running process of the currently running phase, The expanded process card panel displays in real time a dynamic effect reflecting at least one of the running process and the running result of the currently running stage, wherein the step of displaying the flowchart corresponding to the multiple stages includes: The running process of the running stage is displayed in the flow chart in real time to reflect the dynamic effect of the running process of the currently running stage.
  • a display system for a machine learning automatic modeling process including: a display; a controller, configured to perform the following operations: controlling the display of the display and each of the machine learning automatic modeling process The process card panel corresponding to each stage, in which the machine learning automatic modeling process is divided into multiple stages, and the display is controlled to display flowcharts corresponding to the multiple stages, where the controller is currently running according to the machine learning automatic modeling Control the display to expand the process card panel corresponding to the currently running stage, while keeping the process card panel corresponding to the stage that has been run and the stage that is not running in a folded state, and according to the running process of the currently running stage ,
  • the control display displays in real time the dynamic effect reflecting at least one of the running process and the running result of the currently running stage in the expanded process card panel, wherein the controller controls the display according to the running process of the currently running stage
  • the dynamic effect of the running process of the current running stage is displayed in the flow chart in real time.
  • a system including at least one computing device and at least one storage device storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one A computing device executes the display method of the machine learning automatic modeling process according to the present disclosure.
  • a computer-readable storage medium storing instructions, wherein when the instructions are executed by at least one computing device, the at least one computing device is caused to execute the machine according to the present disclosure. Learn how to display the automatic modeling process.
  • the progress of the machine learning automatic modeling process can be clearly presented to the user in the form of an expandable and foldable process card, so as to help the user understand the machine learning automatic modeling process.
  • What stage the model is in, and the flow chart corresponding to each stage can be presented to the user to help the user understand the process of machine learning automatic modeling.
  • the resource occupation information in the machine learning automatic modeling process can be presented to the user, which is convenient for the user to monitor the resource situation in the modeling process to reasonably allocate system resources , To overcome the inability of users to monitor resources in real time due to unshown resource occupation.
  • the log information of the machine learning automatic modeling process can be presented to the user, so that the user can perceive that the modeling process is continuing, and can satisfy the user's error Information is consulted at any time and the request of debug.
  • the dynamic progress information of the machine learning automatic modeling process can be presented to the user to help the user understand real-time data related to the machine learning automatic modeling process.
  • buttons for local adjustment of the modeling scheme for example, feature editing or feature deletion, etc.
  • the automatic modeling process can also adjust the modeling plan according to the needs, which overcomes the shortcomings of model overfitting that may occur in the existing automatic modeling process and lead to poor model effects.
  • Figures 1 and 2 show an example of the interface of the machine learning automatic modeling process in the prior art
  • FIG. 3 shows a flowchart of a method for displaying a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • FIG. 4 shows a schematic diagram of an interface of a data table splicing stage in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • Fig. 5 shows a schematic diagram of an interface in a feature extraction stage in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • 6A and 6B show schematic diagrams of a parameter setting page in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • FIG. 7 shows another schematic diagram showing the interface of the feature extraction stage in the machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • FIG. 8 shows a schematic diagram of an interface in a model training phase in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure
  • FIG. 9 shows a block diagram of a display system of a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • the present disclosure proposes an optimized display method and system of the machine learning automatic modeling process, which can clearly present the progress stage of the machine learning automatic modeling process to the user in the form of an expandable and foldable process card, and can present the user Flow chart corresponding to each stage.
  • users can be presented with resource occupancy information and log information in the automatic modeling process of machine learning.
  • users can be presented with dynamic progress information of the machine learning automatic modeling process.
  • a button for local adjustment (for example, feature editing or feature deletion, etc.) of the modeling scheme can be provided to the user, so that the user can make corresponding local adjustments.
  • Fig. 3 shows a flowchart of a method for displaying a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • step S301 the machine learning automatic modeling process can be divided into multiple stages.
  • the various stages of the machine learning automatic modeling process can be pre-divided according to human settings, and then the various stages can be displayed in the display method of the machine learning automatic modeling process according to the exemplary embodiment of the present invention.
  • the machine learning automatic modeling process can be clearly divided into a data table splicing phase, a feature extraction phase, and a model training phase.
  • the data table splicing stage refers to a stage of generating a splicing table based on at least one data table containing attribute data and at least one data table containing target result data.
  • the attribute data may be behavior data
  • the target result data may be feedback result data corresponding to the behavior data, which may be a direct final result or related data that can deduce the final result.
  • the feature extraction stage refers to the stage of generating features based on attribute data.
  • the model training phase includes at least one algorithm sub-phase for model training using at least one algorithm.
  • the algorithm may include at least one of LR (Logistic Regression) algorithm, GBDT (Gradient Boosting Decision Tree) algorithm, and NN (Neural Network) algorithm, but it is not limited to this, and the algorithm may also include other different types that can be used for model training. algorithm.
  • the machine learning automatic modeling process may not be limited to being divided into the above three stages, it may be divided into at least one of the above three stages, or may be divided into other multiple stages according to user needs and at least one of different standards. Stages.
  • a progress card panel corresponding to each of the multiple stages may be displayed.
  • the process card panel corresponding to the currently running phase can be expanded, while the process card panel corresponding to the phases that have been run and the phases that are not running remain folded.
  • a dynamic effect reflecting at least one of the running process and the running result of the currently running stage can be displayed in the expanded process card panel in real time.
  • a task tag indicating the stage of the task and the process position of the task in this stage can be generated for the current task in the background, and the background information can be queried in the foreground.
  • Task tag, and the running process of the currently running stage can be determined in the foreground according to the task tag queried.
  • the current running stage in automatic machine learning modeling can be determined according to the stage of the current task indicated in the task label, and the current running stage of the current task in the current running stage can be determined according to the current task indicated in the task label. The running process of the running stage.
  • the running running state can be identified in the expanded process card panel, and output progress information corresponding to the running process of the currently running stage can be displayed in real time.
  • the completed running state can be identified in the process card panel corresponding to the completed stage in the folded state, and part of the output result information of the stage can be displayed.
  • the running state that is not running is identified in the corresponding process card panel of the running stage.
  • the process card panel when a user input for a process card panel corresponding to a stage that has been completed in a collapsed state is received, the process card panel may be expanded and displayed in the expanded process card panel Information on the output results of this stage.
  • step S303 a flowchart corresponding to the multiple stages may be displayed. Specifically, according to the running process of the currently running stage, the dynamic effect reflecting the running process of the currently running stage can be displayed in the flowchart in real time.
  • steps S302 and S303 may be performed simultaneously. That is, in the display interface of the machine learning automatic modeling, the process card panel corresponding to each of the multiple stages and the flowchart corresponding to the multiple stages can be displayed at the same time. For example, the entry card panel and flow chart can be displayed in different areas on the display screen.
  • the method for displaying the machine learning automatic modeling process may further include: as the machine learning automatic modeling progresses, collecting and displaying resource occupation information in the machine learning automatic modeling process in real time, for example, At least one of the CPU occupancy and the memory occupancy.
  • the method for displaying the machine learning automatic modeling process may further include: according to the progress of the machine learning automatic modeling, generating and displaying log information in the machine learning automatic modeling process in real time, for example, about the machine learning automatic modeling process. Learn the tips of at least one of information, warnings and exceptions in the automatic modeling process.
  • the method for displaying the machine learning automatic modeling process may further include: displaying dynamic progress information about the machine learning automatic modeling process.
  • the dynamic progress information can be used to indicate dynamic changes in the progress of the current stage or the overall plan.
  • the dynamic progress information about the machine learning automatic modeling process may include at least one of the following items: information about the number of exploration rounds of at least one of the automatic feature engineering and the model scheme, and information about the time spent modeling , Information about the remaining modeling time, information about the highest real-time AUC value, where the AUC value is a value used to evaluate the effect of the model.
  • the method for displaying the machine learning automatic modeling process may further include: a button for suspending the machine learning automatic modeling process may be displayed, and when a user input for the button is received, pause Machine learning to automate the modeling process.
  • FIGS. 4 to 7 Display method shows a schematic diagram of an interface in a data table splicing phase in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • Fig. 5 shows a schematic diagram of an interface in a feature extraction stage in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • Fig. 6 shows another schematic diagram showing an interface in a feature extraction stage in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • Fig. 7 shows a schematic diagram of an interface of a model training stage in a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • the process card panel can be displayed on the left part of the display interface of the machine learning automatic modeling process, and the flowchart can be displayed in the machine learning automatic modeling process.
  • the resource occupation information during the machine learning automatic modeling process can be displayed in the upper right part of the display interface of the machine learning automatic modeling process
  • the log information during the machine learning automatic modeling process can be displayed in the machine learning automatic modeling process.
  • the lower right part of the display interface, and the dynamic progress information about the machine learning automatic modeling process can be displayed in the upper middle part of the display interface of the machine learning automatic modeling process.
  • a button for pausing the automatic modeling process of machine learning can be displayed below the area where the flowchart is displayed.
  • the display interface of the machine learning automatic modeling process is not limited to this, and the display interface of the machine learning automatic modeling process can also be arranged in any different layout.
  • the progress card panel corresponding to the data table splicing phase is expanded, and due to the feature extraction phase and The model training phase is not yet running, so the process card panel corresponding to the feature extraction phase and the process card panel corresponding to the model training phase remain folded.
  • the output progress information can be displayed in the form of at least one of text and graphs, for example, in at least one of the following ways: display by text
  • the number of spliced data is 234, and the number of spliced data is displayed by a graph with time as the horizontal axis and the number of data as the vertical axis.
  • the running state that is not running is identified.
  • the flowchart may include nodes indicating the data table splicing stage.
  • the data table splicing stage in the machine learning automatic modeling process is currently running, according to the running process generated by the splicing table, the data table splicing stage is indicated in the flowchart.
  • the dynamic effect of the running process generated by the splicing table is displayed in the node in real time.
  • the dynamic progress information area (for example, the upper middle part of the display interface) can be used to display the dynamics of the machine learning automatic modeling process
  • Progress information for example, can dynamically display the time spent modeling and remaining modeling time in real time.
  • the area used to display resource occupation information (for example, the upper right part of the display interface) can dynamically display the current data table splicing phase in real time.
  • Resource occupancy and can dynamically display the log information of the current execution data table splicing stage in the area used to display the log information (for example, the lower right part of the display interface).
  • the data table splicing phase in the machine learning automatic modeling process when the data table splicing phase in the machine learning automatic modeling process has been completed, and the feature extraction phase in the machine learning automatic modeling process is currently running, it is related to the feature extraction
  • the corresponding process card panel of the stage is expanded, and since the data table splicing phase has been completed and the model training phase has not yet run, the process card panel corresponding to the data table splicing phase and the process card panel corresponding to the model training phase remain collapsed.
  • the output progress information can be displayed in the form of at least one of text, graphs, and charts, for example, displaying the current feature dimension through text
  • the number and importance of the generated features are displayed through a graph with the number of features as the horizontal axis and the importance as the vertical axis, and the generated features are displayed through the feature name, importance histogram and importance value graph The name and importance of the feature.
  • the running status that has been completed (for example, "completed") is identified, and the final processed data quantity is displayed, for example, 26537.
  • the corresponding data table splicing stage can be expanded again Process card panel, and display the output result information about the final processed data quantity in the expanded process card panel corresponding to the data table splicing stage, for example, display text and graphs.
  • the collapsed progress card panel corresponding to the completed data table splicing stage when expanded again through user input (for example, user click), the currently expanded one corresponding to the feature extraction stage can be automatically folded The progress card panel.
  • the automatically folded process card panel corresponding to the feature extraction stage can be expanded again through user input (for example, user click).
  • the flowchart may include nodes indicating the feature extraction stage.
  • the nodes indicating the feature extraction stage in the flowchart can be used in real time. Shows the dynamic effect reflecting the running process of the feature extraction stage.
  • the dynamic progress of the machine learning automatic modeling process can be displayed in the area for displaying dynamic progress information (for example, the upper middle part of the display interface) Information, for example, can dynamically display the time spent modeling and remaining modeling time in real time.
  • the number of feature exploration rounds can also be dynamically displayed in real time, that is, the number of automatic feature engineering exploration rounds.
  • the area used to display resource occupation information (for example, the upper right part of the display interface) can dynamically display the resource occupation of the current feature extraction phase in real time It is possible to dynamically display the log information of the current execution feature extraction stage in the area for displaying the log information (for example, the lower right part of the display interface) in real time.
  • each round of feature exploration of multiple rounds of feature exploration can be automatically completed.
  • the user can choose to enable automatic feature exploration in the parameter setting page.
  • the user may choose to turn off automatic feature exploration in the parameter setting page.
  • turning off automatic feature exploration in the feature extraction stage refer to Figure 7.
  • the feature extraction stage is currently running, whenever a round of feature exploration is completed, feature exploration is paused, and the result information of this round of feature exploration is displayed for editing features Button, button to continue feature exploration, and button to enter the model training phase.
  • the feature exploration can be paused, and information about the first round of feature exploration and related buttons can be displayed in the area originally used to display the flowchart (for example, the lower part of the display interface), such as , Displays the information indicating the completion of the first round of feature exploration, the first round of feature exploration produced 234 new features, the current total of 234 new features have been explored, the button for editing features, and the button for continuing feature exploration.
  • a button used to enter the model training phase is displayed.
  • the model training phase in the machine learning automatic modeling process when the feature extraction phase in the machine learning automatic modeling process has been run, and the model training phase in the machine learning automatic modeling process is currently running (here, the model training phase It may include at least one algorithm sub-stage for model training using at least one algorithm), the process card panel corresponding to the model training stage is expanded, and since the data table splicing stage and feature extraction stage have been completed, they are spliced with the data table The process card panel corresponding to the phase and the process card panel corresponding to the feature extraction phase remain folded.
  • the expanded process card panel corresponding to the model training phase identify the running status (for example, "in progress"), and display the output progress information corresponding to the running process of the model training phase in real time, for example, each The AUC value obtained by the algorithm with the number of algorithm exploration rounds for evaluating the effect of the model and the output progress information of the real-time highest AUC value of each algorithm.
  • the output progress information can be displayed in the form of at least one of text and graph
  • the current number of exploration rounds of each algorithm and the highest AUC value of all rounds explored by each algorithm are displayed through text (for example, the current number of exploration rounds of the LR algorithm is 544, and the first 543 rounds explored
  • the highest AUC value is 0.8233
  • the current exploration round number of the GBDT algorithm is 544
  • the highest AUC value in the first 543 rounds explored is 0.8124
  • the current number of exploration rounds of the NN algorithm is 544
  • the highest AUC value in the first 543 rounds explored 0.8092
  • the algorithm exploration information is displayed through a graph with the number of algorithm exploration rounds as the horizontal axis and the AUC value as the vertical axis.
  • the process card panel corresponding to the splicing stage of the completed data table and the process card panel corresponding to the feature extraction stage of the run identify the running state that has been completed (for example, "completed"), and in the folded
  • the process card panel corresponding to the splicing stage of the completed data table displays the final processed data quantity, for example, 26537, and the current feature dimension is displayed in the collapsed process card panel corresponding to the feature extraction stage of the run, for example, 23423 .
  • the process card panel corresponding to the data table splicing phase and the process card panel corresponding to the feature extraction phase in the collapsed state for example, clicking the process corresponding to the data table splicing phase
  • the corresponding progress card panel of the splicing stage displays the output result information about the final processed data quantity (for example, displaying text and graphs)
  • the expanded progress card panel corresponding to the feature extraction stage displays the final generated data Output result information of at least one of the number, name, and importance of features (for example, display text, graphs, and charts).
  • the process card panel corresponding to the running data table splicing stage and the process card panel corresponding to the running feature extraction stage when at least one of the process card panel corresponding to the running data table splicing stage and the process card panel corresponding to the running feature extraction stage is folded through user input (for example, user When clicking) is expanded again, the currently expanded process card panel corresponding to the model training phase can be automatically collapsed.
  • the automatically collapsed progress card panel corresponding to the model training phase can be expanded again through user input (for example, user click).
  • the flowchart may include at least one node indicating the at least one algorithm sub-phase, when the model training phase in the machine learning automatic modeling process is currently running, according to each algorithm in the at least one algorithm sub-phase
  • the running process of the sub-phases is displayed in real time in at least one node in the flowchart indicating the at least one algorithm sub-phase, reflecting the dynamic effect of the running process of each algorithm sub-phase in the at least one algorithm sub-phase.
  • the dynamic progress of the machine learning automatic modeling process can be displayed in the area used to display dynamic progress information (for example, the upper middle part of the display interface)
  • Information can dynamically display the real-time highest AUC value, the number of exploration rounds of the model scheme, the time spent modeling, and the remaining modeling time.
  • the real-time highest AUC value may refer to the highest AUC value among all current exploration rounds in all algorithms.
  • the real-time highest AUC value may refer to the highest AUC value of 0.8233 among the LR algorithm, GBDT algorithm, and NN algorithm in the first 543 rounds that have been explored.
  • the area used to display resource occupation information (for example, the upper right part of the display interface) can dynamically display the resource occupation of the current model training phase in real time It is possible to dynamically display the log information of the current execution model training phase in the area used to display the log information (for example, the lower right part of the display interface) in real time.
  • the expanded process card panel corresponding to the model training phase can display the output including the AUC value obtained by each algorithm with the number of algorithm exploration rounds and the final highest AUC value of each algorithm. Result information.
  • FIG. 9 shows a block diagram of a display system of a machine learning automatic modeling process according to an exemplary embodiment of the present disclosure.
  • a display system 900 of a machine learning automatic modeling process may include a display 901 and a controller 902.
  • controller 902 is configured to perform the following operations: control the display 901 to display a process card panel corresponding to each stage of the machine learning automatic modeling process, and control the display 901 to display a flowchart corresponding to each stage, where the machine learning The automatic modeling process is divided into multiple stages.
  • the controller 902 controls the display 901 to expand the process card panel corresponding to the currently running stage, and at the same time makes the process corresponding to the stage that has been completed and the stage that is not running The card panel remains folded, and according to the running process of the currently running stage, the display 901 is controlled to display the dynamic effect of at least one of the running process and the running result of the currently running stage in the expanded process card panel in real time. .
  • the controller 902 controls the display 901 to display the dynamic effect reflecting the running progress of the current running stage in real time in the flowchart according to the running progress of the currently running stage.
  • the controller 902 may determine the running process of the currently running stage in the following way: when performing machine learning automatic modeling, generate an indication of the stage of the task and the progress of the task in this stage in the background for the current task The task label of the location; the task label of the background is queried in the foreground; the running process of the currently running stage is determined in the foreground according to the queried task label.
  • the controller 902 may control the display 901 to display a dynamic effect reflecting at least one of the running process and the running result of the currently running stage in the expanded progress card panel in the following manner: in the expanded progress card panel Mark the running status in the, and display the output progress information corresponding to the running process of the current running stage in real time.
  • the controller 902 can control the display 901 to keep the process card panels corresponding to the phases that have been run and the phases that are not running in the folded state: in the folded state, the process card panel that corresponds to the phase that has been run is in the folded state. It identifies the running state that has been completed and displays part of the output result information of the stage. In the collapsed state, the non-running state is identified in the process card panel corresponding to the stage that is not running.
  • the controller 902 controls the display 901 to expand the process card panel, and displays the process card panel in the expanded process card panel.
  • the output result information of the stage when receiving a user input for a process card panel corresponding to a stage that is in a collapsed state, the controller 902 controls the display 901 to expand the process card panel, and displays the process card panel in the expanded process card panel. The output result information of the stage.
  • the controller 902 collects the resource occupation information in the machine learning automatic modeling process in real time, and controls the display 901 to display the resource occupation information.
  • the controller 902 generates log information in the machine learning automatic modeling process in real time, and controls the display 901 to display the log information.
  • the controller 902 controls the display 901 to display dynamic progress information about the machine learning automatic modeling process.
  • the dynamic progress information about the machine learning automatic modeling process includes at least one of the following items: information about the number of exploration rounds of at least one of the automatic feature engineering and the model scheme, and information about the time spent modeling , Information about the remaining modeling time, information about the highest real-time AUC value, where the AUC value is a value used to evaluate the effect of the model.
  • the controller 902 controls the display 901 to display a button for suspending the machine learning automatic modeling process, and when receiving a user input for the button, the controller 902 suspends the machine learning automatic modeling process.
  • the multiple stages include at least one of a data table splicing stage, a feature extraction stage, and a model training stage.
  • the data table splicing stage refers to the stage of generating a splicing table based on at least one data table containing attribute data and one data table including target result data, wherein the controller 902 controls the display 901 to display and the machine learning automatic modeling process
  • the operation of the corresponding progress card panel at each stage of the data table includes at least one of the following operations: when the data table splicing stage is running, the expanded progress card panel corresponding to the data table splicing stage displays in real time about the spliced time The output progress information of the amount of data; when the data sheet splicing phase is completed, the final processed data amount is displayed in the process card panel corresponding to the data sheet splicing phase in the folded state.
  • the flowcharts corresponding to the multiple stages include nodes indicating the data table splicing stage, and the controller 902 controls the display 901 to display in real time in the flowchart reflecting the current running progress according to the running progress of the currently running stage.
  • the operation of the dynamic effect of the running process of the running stage includes: when the data table splicing phase is running, according to the running process generated by the splicing table, the node indicating the data table splicing stage in the flowchart indicates the data table splicing stage in real time. The dynamic effect of the running process.
  • the feature extraction stage refers to the stage of generating features based on attribute data, where the controller 902 controls the display 901 to display the progress card panel corresponding to each stage of the machine learning automatic modeling process.
  • the operation includes at least one of the following operations: : When the feature extraction stage is running, the output progress information including at least one of the number, name, and importance of the generated features is displayed in real time in the expanded process card panel corresponding to the feature extraction stage; When the extraction phase is completed, the final number of features generated will be displayed in the process card panel corresponding to the feature extraction phase in the collapsed state.
  • the flowcharts corresponding to the multiple stages include nodes indicating the feature extraction stage, and the controller 902 controls the display 901 to display in the flowchart in real time the current running progress according to the running progress of the currently running stage
  • the steps of the dynamic effect of the running process of the phase include: when the feature extraction phase is running, according to the running process of the feature extraction phase, real-time display reflecting the running process of the feature extraction phase in the node indicating the feature extraction phase in the flowchart dynamic effect.
  • the model training phase includes at least one algorithm sub-phase for model training using at least one algorithm, where the controller 902 controls the display 901 to display the progress card panel corresponding to each phase of the machine learning automatic modeling process.
  • the operation includes At least one of the following operations: When the model training phase is running, display in real time the AUC value used to evaluate the model effect obtained by each algorithm with the number of algorithm exploration rounds in the expanded process card panel corresponding to the model training phase And the output progress information of the real-time highest AUC value of each algorithm; when the model training phase is completed, the progress card panel corresponding to the model training phase will be displayed including the AUC value obtained by each algorithm with the number of algorithm exploration rounds. And the output result information of the final highest AUC value of each algorithm.
  • the flowcharts corresponding to the multiple stages include at least one node indicating the at least one algorithm sub-stage respectively, and the controller 902 controls the display in the flowchart in real time according to the running progress of the currently running stage.
  • the operation of displaying the dynamic effect of the running process reflecting the current running stage includes: when the model training phase is running, according to the running process of each algorithm sub-stage of the at least one algorithm sub-stage, the difference in the flow chart is Instructing the at least one node of the at least one algorithm sub-phase to display in real time a dynamic effect reflecting the running process of each algorithm sub-phase in the at least one algorithm sub-phase.
  • controller 902 performs the following operations: when the automatic feature exploration is turned on in the feature extraction stage, each round of feature exploration of multiple rounds of feature exploration can be automatically completed.
  • the controller 902 performs the following operations: in the case of turning off automatic feature exploration in the feature extraction phase, when the feature extraction phase is running, each time a round of feature exploration is completed, the feature exploration is suspended, and the display 901 is controlled to display the current round of features Exploration result information, buttons for editing features, buttons for continuing feature exploration, and buttons for entering the model training stage; when receiving user input for the buttons for editing features, the user is allowed to control the current round At least one of the explored features in the previous round is edited; when a user input for a button for continuing feature exploration is received, the next round of feature exploration is performed; when a user with a button for entering the model training phase is received When inputting, end the feature exploration and start the model training phase.
  • the progress of the machine learning automatic modeling process can be clearly presented to the user in the form of an expandable and foldable process card, so as to help the user understand the machine learning automatic modeling process.
  • What stage the model is in, and the flow chart corresponding to each stage can be presented to the user to help the user understand the process of machine learning automatic modeling.
  • the resource occupation information in the machine learning automatic modeling process can be presented to the user, which is convenient for the user to monitor the resource situation in the modeling process to reasonably allocate system resources .
  • the log information of the machine learning automatic modeling process can be presented to the user, so that the user can perceive that the modeling process is continuing, and can satisfy the user's error Information is consulted at any time and the request of debug.
  • the dynamic progress information of the machine learning automatic modeling process can be presented to the user to help the user understand real-time data related to the machine learning automatic modeling process.
  • buttons for local adjustment of the modeling scheme for example, feature editing or feature deletion, etc.
  • the automatic modeling process can also adjust the modeling plan as needed.
  • the systems, devices, and units shown in FIG. 8 can be respectively configured as software, hardware, firmware, or any combination of the foregoing to perform specific functions.
  • these systems, devices, or units can correspond to dedicated integrated circuits, can also correspond to pure software codes, or can correspond to modules that combine software and hardware.
  • one or more functions implemented by these systems, devices, or units may also be uniformly executed by components in physical physical devices (for example, processors, clients, or servers).
  • the method described with reference to FIG. 3 can be implemented by a program (or instruction) recorded on a computer-readable storage medium.
  • a computer-readable storage medium for displaying the automatic modeling process of machine learning may be provided, wherein the computer-readable storage medium is recorded on the computer-readable storage medium for executing A computer program (or instruction) showing the steps of the machine learning automatic modeling process described.
  • the computer program can be used to perform the following method steps: display a process card panel corresponding to each stage of the machine learning automatic modeling process, where the machine learning automatic modeling process is divided into multiple stages; display The flowchart corresponding to the multiple stages, wherein the step of displaying the progress card panel corresponding to each stage of the machine learning automatic modeling process includes: according to the current running stage of the machine learning automatic modeling, expand and current The corresponding process card panel of the running stage, while keeping the process card panel corresponding to the stage that has been run and the stage that is not running in a folded state; according to the running process of the currently running stage, it will be displayed in the expanded process card panel in real time
  • the ground display reflects the dynamic effect of at least one of the running process and the running result of the currently running stage, wherein the step of displaying the flowcharts corresponding to the multiple stages includes: according to the running process of the currently running stage, The dynamic effect of the running process of the current running stage is displayed in the flow chart in real time.
  • the computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in computer equipment such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps. Or, more specific processing is performed when the above steps are performed. These additional steps and further processing content have been mentioned in the description of the related method with reference to FIG. 3, so in order to avoid repetition, details will not be repeated here.
  • the display system of the machine learning automatic modeling process can completely rely on the operation of the computer program to realize the corresponding function, that is, each unit corresponds to each step in the functional architecture of the computer program, so that The whole system is called through a special software package (for example, lib library) to realize corresponding functions.
  • lib library for example, lib library
  • each device shown in FIG. 8 can also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof.
  • the program code or code segment used to perform the corresponding operation can be stored in a computer-readable storage medium such as a storage medium, so that the processor can read and run the corresponding Program code or code segment to perform the corresponding operation.
  • the exemplary embodiments of the present disclosure may also be implemented as a computing device, which includes a storage component and a processor.
  • the storage component stores a set of computer-executable instructions. When the set of computer-executable instructions is used by the processor When executed, the method for displaying the machine learning automatic modeling process according to the exemplary embodiment of the present disclosure is executed.
  • the computing device can be deployed in a server or a client, and can also be deployed on a node device in a distributed network environment.
  • the computing device may be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application, or other devices capable of executing the above set of instructions.
  • the computing device does not have to be a single computing device, and may also be any device or a collection of circuits that can execute the above-mentioned instructions (or instruction sets) individually or jointly.
  • the computing device may also be a part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.
  • the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor.
  • the processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
  • Some operations described in the method for displaying the automatic modeling process of machine learning according to exemplary embodiments of the present disclosure can be implemented in software, and some operations can be implemented in hardware. In addition, it can also be implemented through a combination of software and hardware. Way to achieve these operations.
  • the processor can run instructions or codes stored in one of the storage components, where the storage component can also store data. Instructions and data can also be sent and received via a network via a network interface device, wherein the network interface device can use any known transmission protocol.
  • the storage component can be integrated with the processor, for example, RAM or flash memory is arranged in an integrated circuit microprocessor or the like.
  • the storage component may include an independent device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system.
  • the storage component and the processor may be operatively coupled, or may communicate with each other, for example, through an I/O port, a network connection, or the like, so that the processor can read files stored in the storage component.
  • the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be connected to each other via at least one of a bus and a network.
  • a video display such as a liquid crystal display
  • a user interaction interface such as a keyboard, mouse, touch input device, etc.
  • the method described with reference to FIG. 3 may be implemented by a system including at least one computing device and at least one storage device storing instructions.
  • the at least one computing device is a computing device that executes the display method of the machine learning automatic modeling process according to the exemplary embodiment of the present disclosure, and a set of computer-executable instructions is stored in the storage device.
  • the computer executable instruction set is executed by the at least one computing device, the method steps described with reference to FIG. 3 are executed.
  • the following method steps may be executed: display a progress card panel corresponding to each stage of the machine learning automatic modeling process, wherein the machine learning automatic modeling process
  • the modeling process is divided into multiple stages; the flowchart corresponding to the multiple stages is displayed, wherein the steps of displaying the process card panel corresponding to each stage of the machine learning automatic modeling process include: automatic modeling according to machine learning At the current running stage, expand the process card panel corresponding to the currently running stage, and at the same time keep the process card panel corresponding to the stage that has been run and the stage that is not running in a folded state; according to the running process of the currently running stage , Displaying the dynamic effect of at least one of the running process and the running result of the currently running stage in the expanded process card panel in real time, wherein the step of displaying the flowchart corresponding to the multiple stages includes: The running process of the currently running stage is displayed in the flow chart in real time to reflect the dynamic effects of the running process of the currently running stage
  • the display method, system, and computer-readable storage medium of the machine learning automatic modeling process provided in the present disclosure can clearly present the progress stage of the machine learning automatic modeling process to the user in the form of an expandable and foldable process card to help The user understands what stage the machine learning automatic modeling is in, and can present the user with a flowchart corresponding to each stage to help the user understand the process of machine learning automatic modeling.
  • the display method, system, and computer-readable storage medium of the machine learning automatic modeling process provided by the present disclosure can present the resource occupation information in the machine learning automatic modeling process to the user to facilitate the user to monitor the resource situation in the modeling process , In order to allocate system resources reasonably, and overcome the inability of users to monitor resources in real time due to unshown resource occupation.
  • the display method, system, and computer-readable storage medium of the machine learning automatic modeling process provided by the present disclosure can present the log information of the machine learning automatic modeling process to the user, so that the user can perceive that the modeling process is continuing, And it can meet the user's request for error information and debug at any time.
  • the display method, system, and computer-readable storage medium of the machine learning automatic modeling process provided by the present disclosure can present the dynamic progress information of the machine learning automatic modeling process to the user to help the user understand the machine learning automatic modeling process Relevant real-time data.
  • the display method, system, and computer-readable storage medium of the machine learning automatic modeling process provided by the present disclosure can provide users with buttons for local adjustment of the modeling scheme (for example, feature editing or feature deletion, etc.), which is convenient for Experienced users can also adjust the modeling plan according to their needs in the machine learning automatic modeling process, which overcomes the shortcomings of model over-fitting that may occur in the existing automatic modeling process, which leads to poor model effects.
  • the modeling scheme for example, feature editing or feature deletion, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

提供一种机器学习自动建模过程的展示方法及系统。所述方法包括：显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；显示与所述多个阶段相应的流程图，其中，根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，其中，根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。

Description

机器学习自动建模过程的展示方法及系统
本申请要求申请号为201910451079.2，申请日为2019年5月28日，名称为“机器学习自动建模过程的展示方法及系统”的中国专利申请的优先权，其中，上述申请公开的内容通过引用结合在本申请中。
技术领域
本公开总体说来涉及机器学习自动建模领域，更具体地说，涉及一种机器学习自动建模过程的优化的展示方法及系统。
背景技术
目前，在机器学习自动建模过程中，用户可通过界面浏览自动建模的运行状态。例如，如图1和图2所示，现有的RapidMiner Auto Model的界面可通过切换左侧的结果目录树，浏览以数据和图表的形式展示的自动建模运行结果，现有的H2O Driveless AI可通过界面浏览自动建模的运行的状态以及笼统的运行结果和资源占用情况。
然而，建模过程涉及的内容和阶段很多，现有的建模过程的展示方法(例如，RapidMiner和H2O)，都没有清晰地将机器学习自动建模的过程划分为多个阶段，也无法在有限的屏幕内有效地向用户展示机器学习自动建模的不同阶段的进展情况。这使用户，特别是一些对机器学习自动建模了解有限的新手用户，不能很好地理解这一过程，增加了他们的认知和操作门槛。
另外，在现有的机器学习自动建模过程中，由于缺少对机器学习自动建模的清晰的阶段划分，因此用户也不能从界面上直观感受到当前自动建模进展如何，已经经历了什么阶段，正在进行什么阶段，即将运行什么阶段。这会增加用户等待的焦虑，也给用户一种不可控制的负向感受。
也就是说，在现有的机器学习自动建模过程中，用户无法有效了解到自动建模的具体进程，特别是对实际进度的变化缺少有效感知。
发明内容
本公开的示例性实施例旨在克服上述现有的机器学习自动建模的展示方法无法直观有效地浏览各阶段进展的缺陷。
根据本公开的示例性实施例，提供一种机器学习自动建模过程的展示方法，包括：显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；显示与所述多个阶段相应的流程图，其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括：根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之 中的至少一个的动态效果，其中，显示与所述多个阶段相应的流程图的步骤包括：根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
根据本公开的另一示例性实施例，提供一种机器学习自动建模过程的展示系统，包括：显示器；控制器，被配置为执行以下操作：控制显示器显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段，控制显示器显示与所述多个阶段相应的流程图，其中，控制器根据机器学习自动建模的当前正在运行的阶段，控制显示器展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态，并根据当前正在运行的阶段的运行进程，控制显示器在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，其中，控制器根据当前正在运行的阶段的运行进程，控制显示器在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
根据本公开的另一示例性实施例，提供一种包括至少一个计算装置和至少一个存储指令的存储装置的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置执行根据本公开的机器学习自动建模过程的展示方法。
根据本公开的另一示例性实施例，提供一种存储指令的计算机可读存储介质，其中，当所述指令被至少一个计算装置运行时，促使所述至少一个计算装置执行根据本公开的机器学习自动建模过程的展示方法。
根据本公开的机器学习自动建模过程的展示方法和系统，可通过可展开可折叠的进程卡片的形式向用户清晰地呈现机器学习自动建模过程的进展阶段，以帮助用户了解机器学习自动建模进行到了什么阶段，并可向用户呈现与各个阶段相应的流程图，以帮助用户理解机器学习自动建模的过程。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学习自动建模过程中的资源占用信息，方便用户监控建模过程中的资源情况，以合理分配系统资源，克服因未展示资源占用情况导致用户无法对资源进行实时监控。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学习自动建模过程中的日志信息，可使用户感知到建模进程正在持续，并可满足用户对错误信息随时查阅和debug的诉求。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学习自动建模过程的动态进度信息，以帮助用户了解与机器学习自动建模过程相关的实时数据。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户提供对建模方案进行局部调整(例如，特征编辑或特征删除等)的按钮，方便有经验的用户在机器学习自动建模过程也能够根据需要调整建模方案，克服了现有自动建模过程中有可能出现的模型过拟合而导致模型效果不佳的缺陷。
附图说明
从下面结合附图对本公开实施例的详细描述中，本公开的这些和其他方面和优点将变得更加清楚并更容易理解，其中：
图1和图2示出现有技术的机器学习自动建模过程的界面的示例；
图3示出根据本公开的示例性实施例的机器学习自动建模过程的展示方法的流程图；
图4示出根据本公开的示例性实施例的机器学习自动建模过程中的数据表拼接阶段的界面的示意图；
图5示出根据本公开的示例性实施例的机器学习自动建模过程中的特征抽取阶段的界面的示意图；
图6A和图6B示出根据本公开的示例性实施例的机器学习自动建模过程中的参数设置页面的示意图；
图7示出示出根据本公开的示例性实施例的机器学习自动建模过程中的特征抽取阶段的界面的另一示意图；
图8示出根据本公开的示例性实施例的机器学习自动建模过程中的模型训练阶段的界面的示意图；
图9示出根据本公开的示例性实施例的机器学习自动建模过程的展示系统的框图。
具体实施方式
为了使本领域技术人员更好地理解本公开，下面结合附图和具体实施方式对本公开的示例性实施例作进一步详细说明。
现将详细参照本公开的实施例，所述实施例的示例在附图中示出，其中，相同的标号始终指的是相同的部件。以下将通过参照附图来说明所述实施例，以便解释本公开。在此需要说明的是，在本公开中出现的“若干项之中的至少一项”均表示包含“该若干项中的任意一项”、“该若干项中的任意多项的组合”、“该若干项的全体”这三类并列的情况。例如“包括A和B之中的至少一个”即包括如下三种并列的情况：(1)包括A；(2)包括B；(3)包括A和B。又例如“执行步骤一和步骤二之中的至少一个”，即表示如下三种并列的情况：(1)执行步骤一；(2)执行步骤二；(3)执行步骤一和步骤二。
本公开提出一种机器学习自动建模过程的优化的展示方法和系统，可通过可展开可折叠的进程卡片的形式向用户清晰地呈现机器学习自动建模过程的进展阶段，并可向用户呈现与各个阶段相应的流程图。此外，可向用户呈现机器学习自动建模过程中的资源占用信息和日志信息。此外，可向用户呈现机器学习自动建模过程的动态进度信息。此外，根据本公开的机器学习自动建模过程的展示方法，可向用户提供对建模方案进行局部调整(例如，特征编辑或特征删除等)的按钮，使得用户可进行相应的局部调整。下面，将参照图3至图8具体描述本公开示例性实施例的机器学习自动建模过程的展示方法和系统。另外，为了清楚和简洁，将省略对公知的功能和结构的描述。
图3示出根据本公开的示例性实施例的机器学习自动建模过程的展示方法的流程图。
在步骤S301，可将机器学习自动建模过程划分为多个阶段。
应理解，这里，机器学习自动建模过程的各个阶段可预先根据人为设定而进行划分，进而在执行根据本发明示例性实施例的机器学习自动建模过程的展示方法中可显示与各个阶段相应的进程卡片面板和流程图。也就是说，上述步骤S301并不必然包括在根据本发明示例性实施例的机器学习自动建模过程的展示方法中。
根据本公开的示例性实施例，可将机器学习自动建模过程清晰地划分为数据表拼接阶段、特征抽取阶段和模型训练阶段。这里，数据表拼接阶段是指基于包含属性数据的至少 一个数据表和包含目标结果数据的至少一个数据表来生成拼接表的阶段。这里，作为示例，属性数据可以是行为数据，目标结果数据可以是行为数据对应的反馈结果数据，其既可以是直接的最终结果，也可以是能够推导出最终结果的相关数据。特征抽取阶段是指基于属性数据生成特征的阶段。模型训练阶段包括使用至少一种算法进行模型训练的至少一个算法子阶段。这里，算法可包括LR(逻辑回归)算法、GBDT(梯度提升决策树)算法和NN(神经网络)算法中的至少一个，但不限于此，算法也可包括其它可用于模型训练的不同种类的算法。此外，机器学习自动建模过程可不限于被划分成以上三个阶段，可被划分成以上三个阶段中的至少一个阶段，或者可根据用户需求和不同标准之中的至少一个被划分成其它多个阶段。
在步骤S302，可显示与所述多个阶段的每个阶段相应的进程卡片面板。具体地，可根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态，并可根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果。
根据本公开的示例性实施例，当机器学习自动建模时，可在后台针对当前任务产生指示该任务所在的阶段和该任务在该阶段中的进程位置的任务标签，可在前台查询后台的任务标签，并可在前台根据查询到的任务标签确定当前正在运行的阶段的运行进程。具体地说，可根据任务标签中指示的当前任务所在的阶段确定机器学习自动建模中当前正在运行的阶段，并可根据任务标签中指示的当前任务在当前正在运行阶段中的进程位置确定当前正在运行的阶段的运行进程。
根据本公开的示例性实施例，可在展开的进程卡片面板中标识正在运行的运行状态，并实时地显示与当前正在运行的阶段的运行进程相应的产出进展信息。此外，可在处于折叠状态的与已运行完毕的阶段相应的进程卡片面板中标识已运行完毕的运行状态，并显示该阶段的产出结果信息的一部分信息，并可在处于折叠状态的与未运行的阶段相应的进程卡片面板中标识未运行的运行状态。
根据本公开的示例性实施例，当接收到针对处于折叠状态的与已运行完毕的阶段相应的进程卡片面板的用户输入时，可展开该进程卡片面板，并在展开的该进程卡片面板中显示该阶段的产出结果信息。
在步骤S303，可显示与所述多个阶段相应的流程图。具体地，可根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
根据本公开的示例性实施例，可同时执行步骤S302和S303。也就是说，可在机器学习自动建模的展示界面中，同时显示与所述多个阶段的每个阶段相应的进程卡片面板和与所述多个阶段相应的流程图。例如，可将进城卡片面板和流程图显示在显示屏幕上的不同区域。
根据本公开的示例性实施例，机器学习自动建模过程的展示方法还可包括：随着机器学习自动建模的进行，实时收集并显示机器学习自动建模过程中的资源占用信息，例如，占用CPU的情况和占用内存的情况之中的至少一个。
根据本公开的示例性实施例，机器学习自动建模过程的展示方法还可包括：根据机器学习自动建模的进行，实时产生并显示机器学习自动建模过程中的日志信息，例如，关于 机器学习自动建模过程中的信息、警告和异常之中的至少一个的提示。
根据本公开的示例性实施例，机器学习自动建模过程的展示方法还可包括：显示关于机器学习自动建模过程的动态进度信息。所述动态进度信息可用于指示当前阶段或整体方案的进度方面的动态变化。
这里，关于机器学习自动建模过程的动态进度信息可包括以下项之中的至少一个：关于自动特征工程和模型方案之中的至少一个的探索轮数的信息、关于建模已花费时长的信息、关于剩余建模时间的信息、关于实时最高AUC值的信息，其中，AUC值是用于评估模型效果的值。
根据本公开的示例性实施例，机器学习自动建模过程的展示方法还可包括：可显示用于暂停机器学习自动建模过程的按钮，并当接收到针对所述按钮的用户输入，可暂停机器学习自动建模过程。
下面，以将机器学习自动建模过程划分为数据表拼接阶段、特征抽取阶段和模型训练阶段为例，参照图4至图7详细描述根据本公开的示例性实施例的机器学习自动建模过程的展示方法。图4示出根据本公开的示例性实施例的机器学习自动建模过程中的数据表拼接阶段的界面的示意图。图5示出根据本公开的示例性实施例的机器学习自动建模过程中的特征抽取阶段的界面的示意图。图6示出示出根据本公开的示例性实施例的机器学习自动建模过程中的特征抽取阶段的界面的另一示意图。图7示出根据本公开的示例性实施例的机器学习自动建模过程中的模型训练阶段的界面的示意图。
根据本公开的示例性实施例，参照图4、图5和图7，可将进程卡片面板显示在机器学习自动建模过程的展示界面的左侧部分，并将流程图显示在机器学习自动建模过程的展示界面的中下部分。此外，可将机器学习自动建模过程中的资源占用信息显示在机器学习自动建模过程的展示界面的右上部分，将机器学习自动建模过程中的日志信息显示在机器学习自动建模过程的展示界面的右下部分，并可将关于机器学习自动建模过程的动态进度信息显示在机器学习自动建模过程的展示界面的中上部分。可将用于暂停机器学习自动建模过程的按钮显示在用于显示流程图的区域的下方。当然，机器学习自动建模过程的展示界面不限于此，还可以以任何不同的布局来布置机器学习自动建模过程的展示界面。
根据本公开的示例性实施例，参照图4，当当前正在运行机器学习自动建模过程中的数据表拼接阶段时，与数据表拼接阶段相应的进程卡面板被展开，并且由于特征抽取阶段和模型训练阶段尚未运行，因此，与特征抽取阶段相应的进程卡面板和与模型训练阶段相应的进程卡面板保持折叠状态。在展开的与数据表拼接阶段相应的进程卡面板中标识正在运行的运行状态(例如，“进行中”)，并实时地显示与数据表拼接阶段的运行进程相应的产出进展信息，例如，关于随时间已拼接的数据数量的产出进展信息，该产出进展信息可通过文本和曲线图之中的至少一个的形式显示，例如，通过以下方式之中的至少一个进行显示：通过文本显示已拼接的数据数量为234、通过以时间为横轴以数据数量为纵轴的曲线图来显示已拼接的数据数量。在折叠的与特征抽取阶段相应的进程卡面板和与模型训练阶段相应的进程卡面板中标识未运行的运行状态(例如，“未开始”)。
此外，流程图可包括指示数据表拼接阶段的节点，当当前正在运行机器学习自动建模过程中的数据表拼接阶段时，根据拼接表生成的运行进程，在流程图中指示数据表拼接阶段的节点中实时地显示反映拼接表生成的运行进程的动态效果。
此外，当当前正在运行机器学习自动建模过程中的数据表拼接阶段时，可在用于显示 动态进度信息的区域(例如，展示界面的中上部分)显示关于机器学习自动建模过程的动态进度信息，例如，可实时动态地显示建模已花费时长和剩余建模时间。
此外，当当前正在运行机器学习自动建模过程中的数据表拼接阶段时，可在用于显示资源占用信息的区域(例如，展示界面的右上部分)实时动态地显示当前执行数据表拼接阶段的资源占用情况，并可在用于显示日志信息的区域(例如，展示界面的右下部分)实时动态地显示当前执行数据表拼接阶段的日志信息。
根据本公开的示例性实施例，参照图5，当机器学习自动建模过程中的数据表拼接阶段已运行完毕，并且当前正在运行机器学习自动建模过程中的特征抽取阶段时，与特征抽取阶段相应的进程卡片面板被展开，并且由于数据表拼接阶段已运行完毕，模型训练阶段尚未运行，因此与数据表拼接阶段相应的进程卡片面板和与模型训练阶段相应的进程卡片面板保持折叠状态。在展开的与特征抽取阶段相应的进程卡面板中标识正在运行的运行状态(例如，“进行中”)，并实时地显示与特征抽取阶段的运行进程相应的产出进展信息，例如，已生成的特征的数量、名称以及重要性之中的至少一个的产出进展信息，该产出进展信息可通过文本、曲线图以及图表之中的至少一个的形式显示，例如，通过文本显示当前特征维度为23423，通过以特征数量为横轴以重要性为纵轴的曲线图来显示已生成的特征的数量和重要性，通过特征名称、重要性柱状图和重要性数值的图表来显示已生成的特征的名称和重要性。
在折叠的与运行完毕的数据表拼接阶段相应的进程卡面板中标识已运行完毕的运行状态(例如，“已完成”)，并显示最终已处理的数据数量，例如，26537。另外，当接收到针对处于折叠状态的与数据表拼接阶段相应的进程卡片面板的用户输入(例如，点击与数据表拼接阶段相应的进程卡片面板)时，可再次展开与数据表拼接阶段相应的进程卡片面板，并在展开的与数据表拼接阶段相应的进程卡片面板中显示关于最终已处理的数据数量的产出结果信息，例如，显示文本和曲线图。根据本公开的另一实施例，当折叠的与运行完毕的数据表拼接阶段相应的进程卡面板通过用户输入(例如，用户点击)被再次展开时，可自动折叠当前展开的与特征抽取阶段相应的进程卡片面板。自动折叠的与特征抽取阶段相应的进程卡片面板可通过用户输入(例如，用户点击)被再次展开。
在折叠的与尚未运行的模型训练阶段相应的进程卡面板中标识未运行的运行状态(例如，“未开始”)。
此外，流程图可包括指示特征抽取阶段的节点，当当前正在运行机器学习自动建模过程中的特征抽取阶段时，根据特征抽取的运行进程，在流程图中指示特征抽取阶段的节点中实时地显示反映特征抽取阶段的运行进程的动态效果。
此外，当当前正在运行机器学习自动建模过程中的特征抽取阶段时，可在用于显示动态进度信息的区域(例如，展示界面的中上部分)显示关于机器学习自动建模过程的动态进度信息，例如，可实时动态地显示建模已花费时长和剩余建模时间。此外，当特征抽取阶段涉及多轮特征探索时，还可实时动态地显示特征已探索轮数，即，自动特征工程探索轮数。
此外，当当前正在运行机器学习自动建模过程中的特征抽取阶段时，可在用于显示资源占用信息的区域(例如，展示界面的右上部分)实时动态地显示当前执行特征抽取阶段的资源占用情况，并可在用于显示日志信息的区域(例如，展示界面的右下部分)实时动态地显示当前执行特征抽取阶段的日志信息。
根据本公开的示例性实施例，在特征抽取阶段开启自动特征探索的情况下，可自动完成多轮特征探索的每一轮特征探索。例如，参照图6A，用户可在参数设置页面中选择开启自动特征探索。
根据本公开的另一示例性实施例，参照图6B，用户可在参数设置页面中选择关闭自动特征探索。在特征抽取阶段关闭自动特征探索的情况下，参照图7，当当前正在运行特征抽取阶段时，每当完成一轮特征探索，暂停特征探索，并显示本轮特征探索结果信息、用于编辑特征的按钮、用于继续特征探索的按钮以及用于进入模型训练阶段的按钮。例如，当第一轮特征探索完成时，可暂停特征探索，并在原本用于显示流程图的区域(例如，展示界面的中下部分)显示关于第一轮特征探索的信息以及相关按钮，例如，显示表示第一轮特征探索完成的信息、第一轮特征探索产出234个新特征、当前总共探索出234个新特征、用于编辑特征的按钮和用于继续特征探索的按钮，并在原本用于显示暂停机器学习自动建模过程的按钮处显示用于进入模型训练阶段的按钮。当接收到针对用于编辑特征的按钮的用户输入时，允许用户对本轮和先前轮之中的至少一个探索的特征进行编辑。当接收到针对用于继续特征探索的按钮的用户输入时，进行下一轮特征探索。当接收到针对用于进入模型训练阶段的按钮的用户输入时，结束特征探索并开始运行模型训练阶段。
根据本公开的示例性实施例，参照图8，当机器学习自动建模过程中的特征抽取阶段已运行完毕，并且当前正在运行机器学习自动建模过程中的模型训练阶段(这里，模型训练阶段可包括使用至少一种算法进行模型训练的至少一个算法子阶段)时，与模型训练阶段相应的进程卡片面板被展开，并且由于数据表拼接阶段和特征抽取阶段已运行完毕，因此与数据表拼接阶段相应的进程卡片面板和与特征抽取阶段相应的进程卡片面板保持折叠状态。在展开的与模型训练阶段相应的进程卡面板中标识正在运行的运行状态(例如，“进行中”)，并实时地显示与模型训练阶段的运行进程相应的产出进展信息，例如，每个算法随算法探索轮数获得的用于评估模型效果的AUC值和每个算法的实时最高AUC值的产出进展信息，该产出进展信息可通过文本和曲线图之中的至少一个的形式显示，例如，通过文本显示每个算法的当前探索轮数和每个算法的已探索的所有轮数中的最高AUC值(例如，LR算法的当前探索轮数为544，已探索的前543轮中最高AUC值为0.8233，GBDT算法的当前探索轮数为544，已探索的前543轮中最高AUC值为0.8124，NN算法的当前探索轮数为544，已探索的前543轮中最高AUC值为0.8092)，通过以算法探索轮数为横轴以AUC值为纵轴的曲线图来显示算法探索信息。
在折叠的与运行完毕的数据表拼接阶段相应的进程卡面板和与运行完毕的特征抽取阶段相应的进程卡面板中标识已运行完毕的运行状态(例如，“已完成”)，并在折叠的与运行完毕的数据表拼接阶段相应的进程卡面板中显示最终已处理的数据数量，例如，26537，在折叠的与运行完毕的特征抽取阶段相应的进程卡面板中显示当前特征维度，例如，23423。另外，当接收到针对处于折叠状态的与数据表拼接阶段相应的进程卡片面板和与特征抽取阶段相应的进程卡面板之中的至少一个的用户输入(例如，点击与数据表拼接阶段相应的进程卡片面板)时，可再次展开与数据表拼接阶段相应的进程卡片面板和与特征抽取阶段相应的进程卡面板之中的至少一个，并执行以下操作之中的至少一个：在展开的与数据表拼接阶段相应的进程卡片面板中显示关于最终已处理的数据数量的产出结果信息(例如，显示文本和曲线图)、在展开的与特征抽取阶段相应的进程卡面板中显示包括最终已生成的特征的数量、名称以及重要性之中的至少一个的产出结果信息(例如， 显示文本、曲线图和图表)。根据本公开的另一实施例，当折叠的与运行完毕的数据表拼接阶段相应的进程卡面板和与运行完毕的特征抽取阶段相应的进程卡面板之中的至少一个通过用户输入(例如，用户点击)被再次展开时，可自动折叠当前展开的与模型训练阶段相应的进程卡片面板。自动折叠的与模型训练阶段相应的进程卡片面板可通过用户输入(例如，用户点击)被再次展开。
此外，流程图可包括分别指示所述至少一个算法子阶段的至少一个节点，当当前正在运行机器学习自动建模过程中的模型训练阶段时，根据所述至少一个算法子阶段中的每个算法子阶段的运行进程，在流程图中的分别指示所述至少一个算法子阶段的至少一个节点中实时地显示反映所述至少一个算法子阶段中的每个算法子阶段的运行进程的动态效果。
此外，当当前正在运行机器学习自动建模过程中的模型训练阶段时，可在用于显示动态进度信息的区域(例如，展示界面的中上部分)显示关于机器学习自动建模过程的动态进度信息，例如，可实时动态地显示实时最高AUC值、模型方案探索轮数、建模已花费时长、剩余建模时间。这里，实时最高AUC值可指所有算法中当前所有探索轮数中的最高AUC值。例如，如图7所示，实时最高AUC值可指在已探索的前543轮中，LR算法、GBDT算法、NN算法之中的最高AUC值0.8233。
此外，当当前正在运行机器学习自动建模过程中的模型训练阶段时，可在用于显示资源占用信息的区域(例如，展示界面的右上部分)实时动态地显示当前执行模型训练阶段的资源占用情况，并可在用于显示日志信息的区域(例如，展示界面的右下部分)实时动态地显示当前执行模型训练阶段的日志信息。
此外，当模型训练阶段运行完毕时，可在展开的与模型训练阶段相应的进程卡片面板中显示包括每个算法随算法探索轮数获得的AUC值以及每个算法的最终最高AUC值的产出结果信息。
图9示出根据本公开的示例性实施例的机器学习自动建模过程的展示系统的框图。
参照图9，根据本公开的示例性实施例的机器学习自动建模过程的展示系统900可包括显示器901和控制器902。
其中，控制器902被配置为执行以下操作：控制显示器901显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，控制显示器901显示与每个阶段相应的流程图，其中，机器学习自动建模过程划分为多个阶段。
其中，控制器902根据机器学习自动建模的当前正在运行的阶段，控制显示器901展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态，并根据当前正在运行的阶段的运行进程，控制显示器901在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果。并且，控制器902根据当前正在运行的阶段的运行进程，控制显示器901在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
作为示例，控制器902可通过以下方式确定当前正在运行的阶段的运行进程：当进行机器学习自动建模时，在后台针对当前任务产生指示该任务所在的阶段和该任务在该阶段中的进程位置的任务标签；在前台查询后台的任务标签；在前台根据查询到的任务标签确定当前正在运行的阶段的运行进程。
作为示例，控制器902可通过以下方式控制显示器901在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果：在展 开的进程卡片面板中标识正在运行的运行状态，并实时地显示与当前正在运行的阶段的运行进程相应的产出进展信息。
作为示例，控制器902可通过以下方式控制显示器901使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态：在处于折叠状态的与已运行完毕的阶段相应的进程卡片面板中标识已运行完毕的运行状态，并显示该阶段的产出结果信息的一部分信息，在处于折叠状态的与未运行的阶段相应的进程卡片面板中标识未运行的运行状态。
作为示例，当接收到针对处于折叠状态的与已运行完毕的阶段相应的进程卡片面板的用户输入时，控制器902控制显示器901展开该进程卡片面板，并在展开的该进程卡片面板中显示该阶段的产出结果信息。
作为示例，随着机器学习自动建模的进行，控制器902实时收集机器学习自动建模过程中的资源占用信息，并控制显示器901显示所述资源占用信息。
作为示例，根据机器学习自动建模的进行，控制器902实时产生机器学习自动建模过程中的日志信息，并控制显示器901显示所述日志信息。
作为示例，控制器902控制显示器901显示关于机器学习自动建模过程的动态进度信息。
作为示例，关于机器学习自动建模过程的动态进度信息包括以下项之中的至少一个：关于自动特征工程和模型方案之中的至少一个的探索轮数的信息、关于建模已花费时长的信息、关于剩余建模时间的信息、关于实时最高AUC值的信息，其中，AUC值是用于评估模型效果的值。
其中，控制器902控制显示器901显示用于暂停机器学习自动建模过程的按钮，当接收到针对所述按钮的用户输入，控制器902暂停机器学习自动建模过程。
作为示例，所述多个阶段包括数据表拼接阶段、特征抽取阶段和模型训练阶段之中的至少一个阶段。
作为示例，数据表拼接阶段是指基于包含属性数据的至少一个数据表和包括目标结果数据的一个数据表来生成拼接表的阶段，其中，控制器902控制显示器901显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的操作包括以下操作中的至少一个：当正在运行数据表拼接阶段时，在展开的与数据表拼接阶段相应的进程卡片面板中实时地显示关于随时间已拼接的数据数量的产出进展信息；当数据表拼接阶段运行完毕时，在处于折叠状态的与数据表拼接阶段相应的进程卡片面板中显示最终已处理的数据数量，当接收到针对处于折叠状态的与数据表拼接阶段相应的进程卡片面板的用户输入时，展开与数据表拼接阶段相应的进程卡片面板，并在展开的与数据表拼接阶段相应的进程卡片面板中显示关于最终已处理的数据数量的产出结果信息。
其中，与所述多个阶段相应的流程图包括指示数据表拼接阶段的节点，并且，控制器902根据当前正在运行的阶段的运行进程，控制显示器901在流程图中的实时地显示反映当前正在运行的阶段的运行进程的动态效果的操作包括：当正在运行数据表拼接阶段时，根据拼接表生成的运行进程，在流程图中指示数据表拼接阶段的节点中实时地显示反映数据表拼接阶段的运行进程的动态效果。
作为示例，特征抽取阶段是指基于属性数据生成特征的阶段，其中，控制器902控制显示器901显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的操作包括以下操作中的至少一个：当正在运行特征抽取阶段时，在展开的与特征抽取阶段相应的进程 卡片面板中实时地显示包括已生成的特征的数量、名称以及重要性之中的至少一个的产出进展信息；当特征抽取阶段运行完毕时，在处于折叠状态的与特征抽取阶段相应的进程卡片面板中显示最终已生成的特征数量，当接收到针对处于折叠状态的与特征抽取阶段相应的进程卡片面板的用户输入时，展开与特征抽取阶段相应的进程卡片面板，并在展开的与特征抽取阶段相应的进程卡片面板中显示包括最终已生成的特征的数量、名称以及重要性之中的至少一个的产出结果信息。
其中，与所述多个阶段相应的流程图包括指示特征抽取阶段的节点，并且，控制器902根据当前正在运行的阶段的运行进程，控制显示器901在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：当正在运行特征抽取阶段时，根据特征抽取阶段的运行进程，在流程图中的指示特征抽取阶段的节点中实时地显示反映特征抽取阶段的运行进程的动态效果。
作为示例，模型训练阶段包括使用至少一种算法进行模型训练的至少一个算法子阶段，其中，控制器902控制显示器901显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的操作包括以下操作中的至少一个：当正在运行模型训练阶段时，在展开的与模型训练阶段相应的进程卡片面板中实时地显示包括每个算法随算法探索轮数获得的用于评估模型效果的AUC值和每个算法的实时最高AUC值的产出进展信息；当模型训练阶段运行完毕时，在展开的与模型训练阶段相应的进程卡片面板中显示包括每个算法随算法探索轮数获得的AUC值以及每个算法的最终最高AUC值的产出结果信息。
其中，与所述多个阶段相应的流程图包括分别指示所述至少一个算法子阶段的至少一个节点，并且，控制器902根据当前正在运行的阶段的运行进程，控制显示器在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的操作包括：当正在运行模型训练阶段时，根据所述至少一个算法子阶段中的每个算法子阶段的运行进程，在流程图中的分别指示所述至少一个算法子阶段的至少一个节点中实时地显示反映所述至少一个算法子阶段中的每个算法子阶段的运行进程的动态效果。
作为示例，控制器902执行以下操作：在特征抽取阶段开启自动特征探索的情况下，可自动完成多轮特征探索的每一轮特征探索。
作为示例，控制器902执行以下操作：在特征抽取阶段关闭自动特征探索的情况下，当正在运行特征抽取阶段时，每当完成一轮特征探索，暂停特征探索，并控制显示器901显示本轮特征探索结果信息、用于编辑特征的按钮、用于继续特征探索的按钮以及用于进入模型训练阶段的按钮；当接收到针对用于编辑特征的按钮的用户输入时，允许用户对本轮和先前轮之中的至少一个探索的特征进行编辑；当接收到针对用于继续特征探索的按钮的用户输入时，进行下一轮特征探索；当接收到针对用于进入模型训练阶段的按钮的用户输入时，结束特征探索并开始运行模型训练阶段。
以上已经参照图3到图7对上述操作的具体细节进行了示例性描述，这里将不再赘述由图8所示的展示系统执行上述操作时的具体细节，以避免重复。
根据本公开的机器学习自动建模过程的展示方法和系统，可通过可展开可折叠的进程卡片的形式向用户清晰地呈现机器学习自动建模过程的进展阶段，以帮助用户了解机器学习自动建模进行到了什么阶段，并可向用户呈现与各个阶段相应的流程图，以帮助用户理解机器学习自动建模的过程。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学 习自动建模过程中的资源占用信息，方便用户监控建模过程中的资源情况，以合理分配系统资源。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学习自动建模过程中的日志信息，可使用户感知到建模进程正在持续，并可满足用户对错误信息随时查阅和debug的诉求。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户呈现机器学习自动建模过程的动态进度信息，以帮助用户了解与机器学习自动建模过程相关的实时数据。
此外，根据本公开的机器学习自动建模过程的展示方法和系统，可向用户提供对建模方案进行局部调整(例如，特征编辑或特征删除等)的按钮，方便有经验的用户在机器学习自动建模过程也能够根据需要调整建模方案。
以上已参照图3至图8描述了根据本公开示例性实施例的机器学习自动建模过程的展示方法和系统。
图8所示出的系统、装置及单元可被分别配置为执行特定功能的软件、硬件、固件或上述项的任意组合。例如，这些系统、装置或单元可对应于专用的集成电路，也可对应于纯粹的软件代码，还可对应于软件与硬件相结合的模块。此外，这些系统、装置或单元所实现的一个或多个功能也可由物理实体设备(例如，处理器、客户端或服务器等)中的组件来统一执行。
此外，参照图3所描述的方法可通过记录在计算机可读存储介质上的程序(或指令)来实现。例如，根据本公开的示例性实施例，可提供一种用于机器学习自动建模过程展示的计算机可读存储介质，其中，在所述计算机可读存储介质上记录有用于执行参照图3所描述的机器学习自动建模过程的展示方法步骤的计算机程序(或指令)。例如，所述计算机程序(或指令)可用于执行以下方法步骤：显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；显示与所述多个阶段相应的流程图，其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括：根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，其中，显示与所述多个阶段相应的流程图的步骤包括：根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
上述计算机可读存储介质中的计算机程序可在诸如客户端、主机、代理装置、服务器等计算机设备中部署的环境中运行，应注意，所述计算机程序还可用于执行除了上述步骤以外的附加步骤或者在执行上述步骤时执行更为具体的处理，这些附加步骤和进一步处理的内容已经在参照图3进行相关方法的描述过程中提及，因此这里为了避免重复将不再进行赘述。
应注意，根据本公开示例性实施例的机器学习自动建模过程的展示系统可完全依赖计算机程序的运行来实现相应的功能，即，各个单元在计算机程序的功能架构中与各步骤相应，使得整个系统通过专门的软件包(例如，lib库)而被调用，以实现相应的功能。
另一方面，图8所示的各个装置也可以通过硬件、软件、固件、中间件、微代码或其 任意组合来实现。当以软件、固件、中间件或微代码实现时，用于执行相应操作的程序代码或者代码段可以存储在诸如存储介质的计算机可读存储介质中，使得处理器可通过读取并运行相应的程序代码或者代码段来执行相应的操作。
例如，本公开的示例性实施例还可以实现为计算装置，该计算装置包括存储部件和处理器，存储部件中存储有计算机可执行指令集合，当所述计算机可执行指令集合被所述处理器执行时，执行根据本公开的示例性实施例的机器学习自动建模过程的展示方法。
具体说来，所述计算装置可以部署在服务器或客户端中，也可以部署在分布式网络环境中的节点装置上。此外，所述计算装置可以是PC计算机、平板装置、个人数字助理、智能手机、web应用或其他能够执行上述指令集合的装置。
这里，所述计算装置并非必须是单个的计算装置，还可以是任何能够单独或联合执行上述指令(或指令集)的装置或电路的集合体。计算装置还可以是集成控制系统或系统管理器的一部分，或者可被配置为与本地或远程(例如，经由无线传输)以接口互联的便携式电子装置。
在所述计算装置中，处理器可包括中央处理器(CPU)、图形处理器(GPU)、可编程逻辑装置、专用处理器系统、微控制器或微处理器。作为示例而非限制，处理器还可包括模拟处理器、数字处理器、微处理器、多核处理器、处理器阵列、网络处理器等。
根据本公开示例性实施例的机器学习自动建模过程的展示方法中所描述的某些操作可通过软件方式来实现，某些操作可通过硬件方式来实现，此外，还可通过软硬件结合的方式来实现这些操作。
处理器可运行存储在存储部件之一中的指令或代码，其中，所述存储部件还可以存储数据。指令和数据还可经由网络接口装置而通过网络被发送和接收，其中，所述网络接口装置可采用任何已知的传输协议。
存储部件可与处理器集成为一体，例如，将RAM或闪存布置在集成电路微处理器等之内。此外，存储部件可包括独立的装置，诸如，外部盘驱动、存储阵列或任何数据库系统可使用的其他存储装置。存储部件和处理器可在操作上进行耦合，或者可例如通过I/O端口、网络连接等互相通信，使得处理器能够读取存储在存储部件中的文件。
此外，所述计算装置还可包括视频显示器(诸如，液晶显示器)和用户交互接口(诸如，键盘、鼠标、触摸输入装置等)。计算装置的所有组件可经由总线和网络之中的至少一个而彼此连接。
根据本公开示例性实施例的机器学习自动建模过程的展示方法所涉及的操作可被描述为各种互联或耦合的功能块或功能示图。然而，这些功能块或功能示图可被均等地集成为单个的逻辑装置或按照非确切的边界进行操作。
因此，参照图3所描述的方法可通过包括至少一个计算装置和至少一个存储指令的存储装置的系统来实现。
根据本公开的示例性实施例，所述至少一个计算装置是执行根据本公开示例性实施例的机器学习自动建模过程的展示方法的计算装置，存储装置中存储有计算机可执行指令集合，当所述计算机可执行指令集合被所述至少一个计算装置执行时，执行参照图3所描述的方法步骤。例如，当所述计算机可执行指令集合被所述至少一个计算装置执行时，可执行以下方法步骤：显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；显示与所述多个阶段相应的流程图，其中，显示 与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括：根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，其中，显示与所述多个阶段相应的流程图的步骤包括：根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
以上描述了本公开的各示例性实施例，应理解，上述描述仅是示例性的，并非穷尽性的，本公开不限于所披露的各示例性实施例。在不偏离本公开的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。因此，本公开的保护范围应该以权利要求的范围为准。
工业实用性
本公开提供的机器学习自动建模过程的展示方法、系统、计算机可读存储介质，可通过可展开可折叠的进程卡片的形式向用户清晰地呈现机器学习自动建模过程的进展阶段，以帮助用户了解机器学习自动建模进行到了什么阶段，并可向用户呈现与各个阶段相应的流程图，以帮助用户理解机器学习自动建模的过程。
此外，本公开提供的机器学习自动建模过程的展示方法、系统、计算机可读存储介质，可向用户呈现机器学习自动建模过程中的资源占用信息，方便用户监控建模过程中的资源情况，以合理分配系统资源，克服因未展示资源占用情况导致用户无法对资源进行实时监控。
此外，本公开提供的机器学习自动建模过程的展示方法、系统、计算机可读存储介质，可向用户呈现机器学习自动建模过程中的日志信息，可使用户感知到建模进程正在持续，并可满足用户对错误信息随时查阅和debug的诉求。
此外，本公开提供的机器学习自动建模过程的展示方法、系统、计算机可读存储介质，可向用户呈现机器学习自动建模过程的动态进度信息，以帮助用户了解与机器学习自动建模过程相关的实时数据。
此外，本公开提供的机器学习自动建模过程的展示方法、系统、计算机可读存储介质，可向用户提供对建模方案进行局部调整(例如，特征编辑或特征删除等)的按钮，方便有经验的用户在机器学习自动建模过程也能够根据需要调整建模方案，克服了现有自动建模过程中有可能出现的模型过拟合而导致模型效果不佳的缺陷。

Claims (32)

  1. 一种包括至少一个计算装置和至少一个存储指令的存储装置的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置执行用于机器学习自动建模过程的展示的以下步骤，包括：
    显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；
    显示与所述多个阶段相应的流程图，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括：
    根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；
    根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，
    其中，显示与所述多个阶段相应的流程图的步骤包括：
    根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
  2. 如权利要求1所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：通过以下方式确定当前正在运行的阶段的运行进程：
    当进行机器学习自动建模时，在后台针对当前任务产生指示该任务所在的阶段和该任务在该阶段中的进程位置的任务标签；
    在前台查询后台的任务标签；
    在前台根据查询到的任务标签确定当前正在运行的阶段的运行进程。
  3. 如权利要求1所述的系统，其中，通过以下方式在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果：
    在展开的进程卡片面板中标识正在运行的运行状态，并实时地显示与当前正在运行的阶段的运行进程相应的产出进展信息。
  4. 如权利要求1所述的系统，其中，通过以下方式使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态：
    在处于折叠状态的与已运行完毕的阶段相应的进程卡片面板中标识已运行完毕的运行状态，并显示该阶段的产出结果信息的一部分信息，
    在处于折叠状态的与未运行的阶段相应的进程卡片面板中标识未运行的运行状态。
  5. 如权利要求1所述的系统，其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤还包括：
    当接收到针对处于折叠状态的与已运行完毕的阶段相应的进程卡片面板的用户输入时，展开该进程卡片面板，并在展开的该进程卡片面板中显示该阶段的产出结果信息。
  6. 如权利要求1所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：
    随着机器学习自动建模的进行，实时收集并显示机器学习自动建模过程中的资源占用信息。
  7. 如权利要求1所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：
    根据机器学习自动建模的进行，实时产生并显示机器学习自动建模过程中的日志信息。
  8. 如权利要求1所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：
    显示关于机器学习自动建模过程的动态进度信息。
  9. 如权利要求8所述的系统，其中，关于机器学习自动建模过程的动态进度信息包括以下项之中的至少一个：关于自动特征工程和模型方案之中的至少一个的探索轮数的信息、关于建模已花费时长的信息、关于剩余建模时间的信息、关于实时最高AUC值的信息，其中，AUC值是用于评估模型效果的值。
  10. 如权利要求1所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：
    显示用于暂停机器学习自动建模过程的按钮；
    当接收到针对所述按钮的用户输入，暂停机器学习自动建模过程。
  11. 如权利要求1至10中的任意一个权利要求所述的系统，其中，所述多个阶段包括数据表拼接阶段、特征抽取阶段和模型训练阶段之中的至少一个阶段。
  12. 如权利要求11所述的系统，其中，数据表拼接阶段是指基于包含属性数据的至少一个数据表和包含目标结果数据的至少一个数据表来生成拼接表的阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行数据表拼接阶段时，在展开的与数据表拼接阶段相应的进程卡片面板中实时地显示关于随时间已拼接的数据数量的产出进展信息；
    当数据表拼接阶段运行完毕时，在处于折叠状态的与数据表拼接阶段相应的进程卡片面板中显示最终已处理的数据数量，当接收到针对处于折叠状态的与数据表拼接阶段相应的进程卡片面板的用户输入时，展开与数据表拼接阶段相应的进程卡片面板，并在展开的与数据表拼接阶段相应的进程卡片面板中显示关于最终已处理的数据数量的产出结果信息，
    其中，与所述多个阶段相应的流程图包括指示数据表拼接阶段的节点，并且，根据当前正在运行的阶段的运行进程，在流程图中的实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行数据表拼接阶段时，根据拼接表生成的运行进程，在流程图中指示数据表拼接阶段的节点中实时地显示反映数据表拼接阶段的运行进程的动态效果。
  13. 如权利要求11所述的系统，其中，特征抽取阶段是指基于属性数据生成特征的阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行特征抽取阶段时，在展开的与特征抽取阶段相应的进程卡片面板中实时地显示包括已生成的特征的数量、名称以及重要性之中的至少一个的产出进展信息；
    当特征抽取阶段运行完毕时，在处于折叠状态的与特征抽取阶段相应的进程卡片面板中显示最终已生成的特征数量，当接收到针对处于折叠状态的与特征抽取阶段相应的进程卡片面板的用户输入时，展开与特征抽取阶段相应的进程卡片面板，并在展开的与特征抽取阶段相应的进程卡片面板中显示包括最终已生成的特征的数量、名称以及重要性之中的至少一个的产出结果信息，
    其中，与所述多个阶段相应的流程图包括指示特征抽取阶段的节点，并且，根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行特征抽取阶段时，根据特征抽取阶段的运行进程，在流程图中的指示特征抽取阶段的节点中实时地显示反映特征抽取阶段的运行进程的动态效果。
  14. 如权利要求11所述的系统，其中，模型训练阶段包括使用至少一种算法进行模型训练的至少一个算法子阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行模型训练阶段时，在展开的与模型训练阶段相应的进程卡片面板中实时地显示包括每个算法随算法探索轮数获得的用于评估模型效果的AUC值和每个算法的实时最高AUC值的产出进展信息；
    当模型训练阶段运行完毕时，在展开的与模型训练阶段相应的进程卡片面板中显示包括每个算法随算法探索轮数获得的AUC值以及每个算法的最终最高AUC值的产出结果信息，
    其中，与所述多个阶段相应的流程图包括分别指示所述至少一个算法子阶段的至少一个节点，并且，根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行模型训练阶段时，根据所述至少一个算法子阶段中的每个算法子阶段的运行进程，在流程图中的分别指示所述至少一个算法子阶段的至少一个节点中实时地显示反映所述至少一个算法子阶段中的每个算法子阶段的运行进程的动态效果。
  15. 如权利要求11所述的系统，其中，所述指令在被所述至少一个计算装置运行时，促使所述至少一个计算装置还执行以下步骤：
    在特征抽取阶段关闭自动特征探索的情况下，当正在运行特征抽取阶段时，每当完成一轮特征探索，暂停特征探索，并显示本轮特征探索结果信息、用于编辑特征的按钮、用于继续特征探索的按钮以及用于进入模型训练阶段的按钮；
    当接收到针对用于编辑特征的按钮的用户输入时，允许用户对本轮和先前轮之中的至少一个探索的特征进行编辑；
    当接收到针对用于继续特征探索的按钮的用户输入时，进行下一轮特征探索；
    当接收到针对用于进入模型训练阶段的按钮的用户输入时，结束特征探索并开始运行模型训练阶段。
  16. 一种机器学习自动建模过程的展示系统，包括：
    显示器；
    控制器，被配置为执行以下操作：
    控制显示器显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机 器学习自动建模过程划分为多个阶段，
    控制显示器显示与所述多个阶段相应的流程图，
    其中，控制器根据机器学习自动建模的当前正在运行的阶段，控制显示器展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态，并根据当前正在运行的阶段的运行进程，控制显示器在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，
    其中，控制器根据当前正在运行的阶段的运行进程，控制显示器在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
  17. 一种机器学习自动建模过程的展示方法，包括：
    显示与机器学习自动建模过程的每个阶段相应的进程卡片面板，其中，机器学习自动建模过程划分为多个阶段；
    显示与所述多个阶段相应的流程图，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括：
    根据机器学习自动建模的当前正在运行的阶段，展开与当前正在运行的阶段相应的进程卡片面板，同时使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态；
    根据当前正在运行的阶段的运行进程，在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果，
    其中，显示与所述多个阶段相应的流程图的步骤包括：
    根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果。
  18. 如权利要求17所述的展示方法，还包括：通过以下方式确定当前正在运行的阶段的运行进程：
    当进行机器学习自动建模时，在后台针对当前任务产生指示该任务所在的阶段和该任务在该阶段中的进程位置的任务标签；
    在前台查询后台的任务标签；
    在前台根据查询到的任务标签确定当前正在运行的阶段的运行进程。
  19. 如权利要求17所述的展示方法，其中，通过以下方式在展开的进程卡片面板中实时地显示反映当前正在运行的阶段的运行进程和运行结果之中的至少一个的动态效果：
    在展开的进程卡片面板中标识正在运行的运行状态，并实时地显示与当前正在运行的阶段的运行进程相应的产出进展信息。
  20. 如权利要求17所述的展示方法，其中，通过以下方式使与已运行完毕的阶段和未运行的阶段相应的进程卡片面板保持折叠状态：
    在处于折叠状态的与已运行完毕的阶段相应的进程卡片面板中标识已运行完毕的运行状态，并显示该阶段的产出结果信息的一部分信息，
    在处于折叠状态的与未运行的阶段相应的进程卡片面板中标识未运行的运行状态。
  21. 如权利要求17所述的展示方法，其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤还包括：
    当接收到针对处于折叠状态的与已运行完毕的阶段相应的进程卡片面板的用户输入 时，展开该进程卡片面板，并在展开的该进程卡片面板中显示该阶段的产出结果信息。
  22. 如权利要求17所述的展示方法，还包括：
    随着机器学习自动建模的进行，实时收集并显示机器学习自动建模过程中的资源占用信息。
  23. 如权利要求17所述的展示方法，还包括：
    根据机器学习自动建模的进行，实时产生并显示机器学习自动建模过程中的日志信息。
  24. 如权利要求17所述的展示方法，还包括：
    显示关于机器学习自动建模过程的动态进度信息。
  25. 如权利要求24所述的展示方法，其中，关于机器学习自动建模过程的动态进度信息包括以下项之中的至少一个：关于自动特征工程和模型方案之中的至少一个的探索轮数的信息、关于建模已花费时长的信息、关于剩余建模时间的信息、关于实时最高AUC值的信息，其中，AUC值是用于评估模型效果的值。
  26. 如权利要求17所述的展示方法，还包括：
    显示用于暂停机器学习自动建模过程的按钮；
    当接收到针对所述按钮的用户输入，暂停机器学习自动建模过程。
  27. 如权利要求17至26中的任意一个权利要求所述的展示方法，其中，所述多个阶段包括数据表拼接阶段、特征抽取阶段和模型训练阶段之中的至少一个阶段。
  28. 如权利要求27所述的展示方法，其中，数据表拼接阶段是指基于包含属性数据的至少一个数据表和包含目标结果数据的至少一个数据表来生成拼接表的阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行数据表拼接阶段时，在展开的与数据表拼接阶段相应的进程卡片面板中实时地显示关于随时间已拼接的数据数量的产出进展信息；
    当数据表拼接阶段运行完毕时，在处于折叠状态的与数据表拼接阶段相应的进程卡片面板中显示最终已处理的数据数量，当接收到针对处于折叠状态的与数据表拼接阶段相应的进程卡片面板的用户输入时，展开与数据表拼接阶段相应的进程卡片面板，并在展开的与数据表拼接阶段相应的进程卡片面板中显示关于最终已处理的数据数量的产出结果信息，
    其中，与所述多个阶段相应的流程图包括指示数据表拼接阶段的节点，并且，根据当前正在运行的阶段的运行进程，在流程图中的实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行数据表拼接阶段时，根据拼接表生成的运行进程，在流程图中指示数据表拼接阶段的节点中实时地显示反映数据表拼接阶段的运行进程的动态效果。
  29. 如权利要求27所述的展示方法，其中，特征抽取阶段是指基于属性数据生成特征的阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行特征抽取阶段时，在展开的与特征抽取阶段相应的进程卡片面板中实时地显示包括已生成的特征的数量、名称以及重要性之中的至少一个的产出进展信息；
    当特征抽取阶段运行完毕时，在处于折叠状态的与特征抽取阶段相应的进程卡片面板 中显示最终已生成的特征数量，当接收到针对处于折叠状态的与特征抽取阶段相应的进程卡片面板的用户输入时，展开与特征抽取阶段相应的进程卡片面板，并在展开的与特征抽取阶段相应的进程卡片面板中显示包括最终已生成的特征的数量、名称以及重要性之中的至少一个的产出结果信息，
    其中，与所述多个阶段相应的流程图包括指示特征抽取阶段的节点，并且，根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行特征抽取阶段时，根据特征抽取阶段的运行进程，在流程图中的指示特征抽取阶段的节点中实时地显示反映特征抽取阶段的运行进程的动态效果。
  30. 如权利要求27所述的展示方法，其中，模型训练阶段包括使用至少一种算法进行模型训练的至少一个算法子阶段，
    其中，显示与机器学习自动建模过程的每个阶段相应的进程卡片面板的步骤包括以下步骤中的至少一个：
    当正在运行模型训练阶段时，在展开的与模型训练阶段相应的进程卡片面板中实时地显示包括每个算法随算法探索轮数获得的用于评估模型效果的AUC值和每个算法的实时最高AUC值的产出进展信息；
    当模型训练阶段运行完毕时，在展开的与模型训练阶段相应的进程卡片面板中显示包括每个算法随算法探索轮数获得的AUC值以及每个算法的最终最高AUC值的产出结果信息，
    其中，与所述多个阶段相应的流程图包括分别指示所述至少一个算法子阶段的至少一个节点，并且，根据当前正在运行的阶段的运行进程，在流程图中实时地显示反映当前正在运行的阶段的运行进程的动态效果的步骤包括：
    当正在运行模型训练阶段时，根据所述至少一个算法子阶段中的每个算法子阶段的运行进程，在流程图中的分别指示所述至少一个算法子阶段的至少一个节点中实时地显示反映所述至少一个算法子阶段中的每个算法子阶段的运行进程的动态效果。
  31. 如权利要求27所述的展示方法，还包括：
    在特征抽取阶段关闭自动特征探索的情况下，当正在运行特征抽取阶段时，每当完成一轮特征探索，暂停特征探索，并显示本轮特征探索结果信息、用于编辑特征的按钮、用于继续特征探索的按钮以及用于进入模型训练阶段的按钮；
    当接收到针对用于编辑特征的按钮的用户输入时，允许用户对本轮和先前轮之中的至少一个探索的特征进行编辑；
    当接收到针对用于继续特征探索的按钮的用户输入时，进行下一轮特征探索；
    当接收到针对用于进入模型训练阶段的按钮的用户输入时，结束特征探索并开始运行模型训练阶段。
  32. 一种存储指令的计算机可读存储介质，其中，当所述指令被至少一个计算装置运行时，促使所述至少一个计算装置执行如权利要求17到31中的任一权利要求所述的方法。
PCT/CN2020/092944 2019-05-28 2020-05-28 机器学习自动建模过程的展示方法及系统 WO2020239033A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP20813203.5A EP3979148A4 (en) 2019-05-28 2020-05-28 METHOD AND SYSTEM FOR DISPLAYING AN AUTOMATIC MACHINE LEARNING MODELING PROCESS

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910451079.2A CN112016692A (zh) 2019-05-28 2019-05-28 机器学习自动建模过程的展示方法及系统
CN201910451079.2 2019-05-28

Publications (1)

Publication Number Publication Date
WO2020239033A1 true WO2020239033A1 (zh) 2020-12-03

Family

ID=73500581

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/092944 WO2020239033A1 (zh) 2019-05-28 2020-05-28 机器学习自动建模过程的展示方法及系统

Country Status (3)

Country Link
EP (1) EP3979148A4 (zh)
CN (1) CN112016692A (zh)
WO (1) WO2020239033A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114418456A (zh) * 2022-03-11 2022-04-29 希望知舟技术(深圳)有限公司 一种基于工况的机器学习进度管控方法及相关装置
US11620582B2 (en) 2020-07-29 2023-04-04 International Business Machines Corporation Automated machine learning pipeline generation
US11688111B2 (en) * 2020-07-29 2023-06-27 International Business Machines Corporation Visualization of a model selection process in an automated model selection system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160232457A1 (en) * 2015-02-11 2016-08-11 Skytree, Inc. User Interface for Unified Data Science Platform Including Management of Models, Experiments, Data Sets, Projects, Actions and Features
CN107169575A (zh) * 2017-06-27 2017-09-15 北京天机数测数据科技有限公司 一种可视化机器学习训练模型的建模系统和方法
CN108710949A (zh) * 2018-04-26 2018-10-26 第四范式（北京）技术有限公司 用于创建机器学习建模模板的方法及系统
CN108898229A (zh) * 2018-06-26 2018-11-27 第四范式（北京）技术有限公司 用于构建机器学习建模过程的方法及系统
CN108960433A (zh) * 2018-06-26 2018-12-07 第四范式（北京）技术有限公司 用于运行机器学习建模过程的方法及系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830383B (zh) * 2018-05-30 2021-06-08 第四范式（北京）技术有限公司 用于展示机器学习建模过程的方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160232457A1 (en) * 2015-02-11 2016-08-11 Skytree, Inc. User Interface for Unified Data Science Platform Including Management of Models, Experiments, Data Sets, Projects, Actions and Features
CN107169575A (zh) * 2017-06-27 2017-09-15 北京天机数测数据科技有限公司 一种可视化机器学习训练模型的建模系统和方法
CN108710949A (zh) * 2018-04-26 2018-10-26 第四范式（北京）技术有限公司 用于创建机器学习建模模板的方法及系统
CN108898229A (zh) * 2018-06-26 2018-11-27 第四范式（北京）技术有限公司 用于构建机器学习建模过程的方法及系统
CN108960433A (zh) * 2018-06-26 2018-12-07 第四范式（北京）技术有限公司 用于运行机器学习建模过程的方法及系统

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of EP3979148A4
XIANG JIE KA FEI: "A Visual Introduction to Machine Learning Models", 11 November 2018 (2018-11-11), CN, pages 1 - 4, XP009532575, Retrieved from the Internet <URL:https://zhuanlan.zhihu.com/p/37415489> *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11620582B2 (en) 2020-07-29 2023-04-04 International Business Machines Corporation Automated machine learning pipeline generation
US11688111B2 (en) * 2020-07-29 2023-06-27 International Business Machines Corporation Visualization of a model selection process in an automated model selection system
CN114418456A (zh) * 2022-03-11 2022-04-29 希望知舟技术(深圳)有限公司 一种基于工况的机器学习进度管控方法及相关装置

Also Published As

Publication number Publication date
EP3979148A4 (en) 2022-08-31
EP3979148A1 (en) 2022-04-06
CN112016692A (zh) 2020-12-01

Similar Documents

Publication Publication Date Title
WO2020239033A1 (zh) 机器学习自动建模过程的展示方法及系统
CN111310934B (zh) 一种模型生成方法、装置、电子设备和存储介质
CN105843873B (zh) 用于管理数据建模的系统及其方法
US10419585B2 (en) Dynamically surfacing UI controls
US10617952B2 (en) Information processing apparatus, information processing method, non-transitory computer-readable storage medium
CN108830383B (zh) 用于展示机器学习建模过程的方法及系统
US9898255B2 (en) Grid designer for multiple contexts
US10300387B2 (en) Game development system
CN108898229B (zh) 用于构建机器学习建模过程的方法及系统
WO2023284312A1 (zh) 工作流程的构建方法及装置、设备、计算机存储介质及计算机程序产品
JP2018106440A (ja) プログラマブルコントローラ、管理装置および制御システム
US11328236B2 (en) Information processing apparatus and non-transitory computer readable medium storing information processing program
JP4941674B2 (ja) シミュレーション・システム
JP6173587B2 (ja) 情報処理装置および情報処理方法
CN108960433B (zh) 用于运行机器学习建模过程的方法及系统
US10108761B2 (en) Predictive simulation
JP7506339B2 (ja) 情報処理システム、その制御方法及びプログラム
JP2011081713A (ja) 情報処理装置、及び制御方法、プログラム、記録媒体。
US20220229622A1 (en) Information processing apparatus, control method of information processing apparatus, and storage medium
JP6588961B2 (ja) 計算機システムおよび推奨パラメータ値抽出方法
JP7323755B2 (ja) 情報処理システム、その制御方法及びプログラム
US10585974B2 (en) UI producing apparatus, UI producing method, and storage medium
CN109343895A (zh) 一种操作指令的处理方法、装置及计算机可读存储介质
JP2019082900A (ja) 情報処理装置、情報処理方法、プログラム
JP2014222450A (ja) 情報処理装置とその処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20813203

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020813203

Country of ref document: EP

Effective date: 20220103