KR20210141917A - Explainability-Based Tuning of Machine Learning Models - Google Patents

Abstract

An apparatus for explainability-based tuning of a machine learning model, comprising: a first prediction module, configured to receive a first set of predictions from a primary machine learning model based on a data set, a prediction from an auxiliary machine learning model based on the data set trigger one or more actions related to the underlying machine learning model in response to a second prediction module configured to receive a second set of and an action module configured to do so.

Description

Explainability-Based Tuning of Machine Learning Models

[Cross Reference to Related Applications]

This application is entitled to the benefit of, and priority to, U.S. Patent Application Serial No. 16/221,039 (Attorney Docket No. DRB-103) filed on December 14, 2018, entitled "Interpretability-Based Machine Learning Adjustment During Production." claims, the contents of which are incorporated herein by reference to the maximum extent permitted by applicable law. Moreover, subject matter of this disclosure is disclosed in U.S. Patent Application Serial No. 15/790,756 (Attorney Docket No. DRB-002BCP), U.S. Patent Application Serial No. 16/049,647, entitled “Determining Validity of Machine Learning Algorithms for Datasets,” filed July 30, 2018 (Attorney Docket No. DRB-102), and/or To be related to the subject matter disclosed in International Patent Application No. PCT/US2019/044250 (Attorney Docket No. DRB-102WO), entitled "Determining Suitability of Machine Learning Models for Datasets" filed on July 30, 2019 may, the contents of each of which are incorporated herein by reference to the maximum extent permitted by applicable law.

[Field]

BACKGROUND This disclosure relates to machine learning, and more particularly, to adapting a machine learning model to meet explainability criteria.

“Machine learning” generally refers to the application of certain techniques (eg, pattern recognition and/or statistical reasoning techniques) by a computer system to perform a particular task. A machine learning system may build a predictive model based on sample data (eg, “training data”) and use validation data (eg, “test data”) to validate the model. You can also check Sample and validation data may be organized into a set of records (eg, “observations”), each record containing a different data field (eg, “independent variable”, “input” or “ It represents the value of a specified data field (eg, “dependent variable”, “output”, or “target”) based on the value of the “feature”). When provided with other data similar to or related to sample data (e.g., "inference data"), the machine learning system may use such predictive models to accurately predict unknown values of targets in the inference data set. have.

After a prediction problem is identified, the process of using machine learning to build a predictive model that accurately solves the prediction problem is, in general, of data collection, data cleansing, feature engineering, model creation, and model deployment. includes steps. “Automated machine learning” techniques may be used to automate steps or portions of a machine learning process.

Machine learning is being incorporated into a wide range of use cases and industries. Unlike many other types of applications, machine learning applications (including those involving deep learning and advanced analytics) are a number of stand-alone components that must work cohesively to provide accurate and relevant results. is generally provided. Moreover, slight changes to the input data can cause non-linear changes in the results. This complexity can make it difficult to manage or monitor all interdependent aspects of a machine learning system.

Machine learning models may also be used to generate predictions to make important decisions. However, machine learning models, and predictions made by machine learning models, may contain various biases, which may negatively affect those who are directly affected by decisions made based on the predictions. Accordingly, industry and governments may enact regulations stipulating that predictions can be explained if they are made by machine learning models used to make decisions. More generally, it is often important for a machine learning model to be able to explain how a particular prediction was made.

For simple machine learning models, it can be straightforward to identify factors contributing to the predictions made by the machine learning model. However, as machine learning models become more complex, concise identification of these contributing factors can become increasingly difficult, especially when the models are deployed. Preventing the launch of proprietary model-building techniques can further exacerbate the challenge of making models easy to understand.

In some embodiments, a secondary machine learning model may be used to approximate the primary machine learning model and to describe the primary machine learning model or predictions thereof. Specifically, the auxiliary machine learning model may provide one or more factors contributing to the prediction made by the primary machine learning model, and/or a human comprehensible explanation of how the one or more factors influenced the prediction. In other words, an auxiliary machine learning model can provide "explainability" for a primary machine learning model.

In practice, however, predictions from the primary and secondary machine learning models may diverge over time, and as such divergence occurs, the secondary model is an accurate description of the primary model's predictions. is unlikely to provide Thus, if a machine learning system can monitor primary and secondary machine learning models, it would be helpful to determine whether a particular secondary model can provide an accurate description of the prediction of a particular primary model.

In accordance with an aspect of the present disclosure, determining whether the secondary machine learning model is an appropriate approximation of the primary machine learning model, and thus whether the secondary machine learning model can adequately describe the primary machine learning model and/or its prediction "Explainability criteria" may be used to For example, one explainability criterion may be that the divergence of predictions between the primary machine learning model and the secondary machine learning model is less than a threshold divergence. A violation of the explainability criterion may indicate that the auxiliary machine learning model is not an adequate approximation of the underlying machine learning model, and thus the auxiliary machine learning model does not adequately describe the underlying machine learning model and/or its predictions. Basic machine learning models may be tuned in development and/or deployment to prevent or respond to violations of explainability criteria.

In general, one innovative aspect of the subject matter described herein may be embodied in an apparatus comprising a first prediction module, a second prediction module, and an action module. The first prediction module is configured to obtain a first set of predictions based on the inference data set from the deployed basic machine learning model. The second prediction module is configured to obtain a second set of predictions based on the inference data set from the deployed auxiliary machine learning model. The secondary machine learning model is different from the primary machine learning model and is configured to mimic the predictions of the primary machine learning model. The action module is configured to trigger one or more actions related to the primary machine learning model in response to determining that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. This determination is based on a comparison of the first and second sets of predictions.

Another embodiment of this aspect includes a corresponding computer system, a computer implemented method, and a computer program recorded on one or more computer storage devices, each configured to perform an action of the apparatus. A system of one or more computers is by virtue of having software, firmware, hardware, or a combination thereof (eg, instructions stored on one or more storage devices) installed on the system that, during operation, cause the system to perform an action. It may be configured to perform a specific action. One or more computer programs may be configured to perform a particular action by virtue of including instructions that, when executed by the data processing apparatus, cause the apparatus to perform the action.

Each of the foregoing and other embodiments, alone or in combination, may optionally include one or more of the following features. In some embodiments, the apparatus can further include a comparison module configured to determine that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. To determine that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, the comparison module determines the difference between the predictions in the first set of predictions and the predictions in the second set of predictions. Based on it, one or more values of a divergence metric for the first and second sets of predictions may be determined. The comparison module may then determine that at least one of the one or more values of the divergence metric for the first and second sets of predictions is greater than a divergence threshold. Finally, the comparison module is configured to: based on a determination that at least one value of a divergence metric for the first and second sets of predictions is greater than a divergence threshold, the primary and secondary machine learning models are configured to compare the one or more values for the primary machine learning model with respect to the primary machine learning model. It can be determined that the explainability criterion is not met.

In certain embodiments, to determine one or more values of the divergence metric, the comparison module determines, on a per prediction basis, a respective value of the divergence metric for each prediction of the first and second sets of predictions. In an alternative embodiment, to determine one or more values of the divergence metric, the comparison module determines respective values of the divergence metric for one or more subsets of the first and second sets of predictions.

In some embodiments, the one or more values of the divergence metric are one or more deployment values of the divergence metric. In such an embodiment, the comparison module comprises: (i) a third set of predictions generated by the base machine learning model based on the training data set during development of the base machine learning model and (ii) training during development of the base machine learning model One or more development values of a divergence metric representing differences between the fourth set of predictions generated by the underlying machine learning model based on the data set may be obtained. The comparison module may then compare one or more distribution values of the divergence metric to one or more developed values of the divergence metric to detect a deviation between the inference data set and the training data set.

In certain embodiments, the one or more actions triggered by the action module include sending an alert regarding a violation of one or more explainability criteria for the underlying machine learning model, wherein the alert includes one or more recommendations for responding to the violation ( recommendations) are included. In a further embodiment, the one or more actions triggered by the action module include exchanging the underlying machine learning model for a different machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises retraining the underlying machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises switching the primary machine learning model to the secondary machine learning model. In a further embodiment, the one or more actions triggered by the action module compare the values of the divergence metrics for the third and fourth sets of predictions generated by the primary and secondary machine learning models during development based on the training data set. identifying deviations between the training data set and the inference data set by doing, wherein the values of the divergence metrics for the first and second sets of predictions are set to the primary and secondary machine learning models during deployment based on the inference data set. is created by In further embodiments, the one or more actions triggered by the action module are triggered without receiving confirmation from the user to perform the one or more actions.

In general, another innovative aspect of the subject matter described herein is to obtain a first set of predictions based on an inference data set from a deployed primary machine learning model, based on an inference data set from a deployed secondary machine learning model. to obtain a second set of predictions, and triggering one or more actions related to the primary machine learning model in response to determining that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. It may be embodied in a method comprising: The secondary machine learning model is different from the primary machine learning model and is configured to mimic the predictions of the primary machine learning model. A determination that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model is based on a comparison of the first and second sets of predictions.

Another embodiment of this aspect includes a corresponding computer system, apparatus, and computer program recorded on one or more computer storage devices, each configured to perform an action of the method. A system of one or more computers is by virtue of having software, firmware, hardware, or a combination thereof (eg, instructions stored on one or more storage devices) installed on the system that, during operation, cause the system to perform an action. It may be configured to perform a specific action. One or more computer programs may be configured to perform a particular action by virtue of including instructions that, when executed by the data processing apparatus, cause the apparatus to perform the action.

Each of the foregoing and other embodiments, alone or in combination, may optionally include one or more of the following features. In some embodiments, the actions of the method may further include determining that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. To determine that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, the action of the method includes: a comparison module: a prediction in a first set of predictions and a prediction in a second set of predictions determining, based on the difference between the predictions, one or more values of a divergence metric for the first and second sets of predictions. Then, the actions of the method may include determining that at least one of the one or more values of the divergence metric for the first and second sets of predictions is greater than a divergence threshold. Finally, the action of the method is that, based on determining that at least one value of a divergence metric for the first and second sets of predictions is greater than a divergence threshold, the primary and secondary machine learning models are one for the primary machine learning model. determining that the above explainability criteria are not met.

In certain embodiments, to determine one or more values of the divergence metric, the actions of the method include, on a per-prediction basis, determining a respective value of the divergence metric for each prediction of the first and second sets of predictions. may include In an alternative embodiment, to determine one or more values of the divergence metric, the actions of the method may include determining respective values of the divergence metric for one or more subsets of the first and second sets of predictions. have.

In some embodiments, the one or more values of the divergence metric are one or more deployment values of the divergence metric. In such an embodiment, the actions of the method include: (i) a third set of predictions generated by the base machine learning model based on the training data set during development of the base machine learning model and (ii) during development of the base machine learning model. obtaining one or more development values of a divergence metric representing a difference between the fourth set of predictions generated by the underlying machine learning model based on the training data set. The actions of the method may then include comparing one or more distribution values of the divergence metric to one or more developed values of the divergence metric to detect a deviation between the inference data set and the training data set.

In certain embodiments, the one or more actions triggered by the action module include sending an alert regarding a violation of one or more explainability criteria for the underlying machine learning model, wherein the alert includes one or more recommendations for responding to the violation. include more In a further embodiment, the one or more actions triggered by the action module include exchanging the underlying machine learning model for a different machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises retraining the underlying machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises switching the primary machine learning model to the secondary machine learning model. In a further embodiment, the one or more actions triggered by the action module compare the values of the divergence metrics for the third and fourth sets of predictions generated by the primary and secondary machine learning models during development based on the training data set. identifying deviations between the training data set and the inference data set by doing, wherein the values of the divergence metrics for the first and second sets of predictions are set to the primary and secondary machine learning models during deployment based on the inference data set. is created by In further embodiments, the one or more actions triggered by the action module are triggered without receiving confirmation from the user to perform the one or more actions.

In general, other innovative aspects of the subject matter described herein include means for obtaining a first set of predictions based on an inference data set from a deployed primary machine learning model, to an inference data set from a deployed secondary machine learning model. means for obtaining a second set of predictions based on, and one or more actions related to the primary machine learning model in response to determining that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. may be embodied in an apparatus comprising means for triggering The secondary machine learning model is different from the primary machine learning model and is configured to mimic the predictions of the primary machine learning model. A determination that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model is based on a comparison of the first and second sets of predictions.

Another embodiment of this aspect includes a corresponding computer system, a computer implemented method, and a computer program recorded on one or more computer storage devices, each configured to perform an action of the apparatus. A system of one or more computers is by virtue of having software, firmware, hardware, or a combination thereof (eg, instructions stored on one or more storage devices) installed on the system that, during operation, cause the system to perform an action. It may be configured to perform a specific action. One or more computer programs may be configured to perform a particular action by virtue of including instructions that, when executed by the data processing apparatus, cause the apparatus to perform the action.

Each of the foregoing and other embodiments, alone or in combination, may optionally include one or more of the following features. In some embodiments, the apparatus can further include means for determining that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model. To determine that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, the apparatus is further configured to: perform a comparison between the predictions in the first set of predictions and the predictions in the second set of predictions. means for determining one or more values of a divergence metric for the first and second sets of predictions based on the difference in . The apparatus may then include means for determining that at least one of the one or more values of the divergence metric for the first and second sets of predictions is greater than a divergence threshold. Finally, the apparatus determines that, based on a determination that at least one value of a divergence metric for the first and second sets of predictions is greater than a divergence threshold, the primary and secondary machine learning models are configured as one or more descriptions of the primary machine learning model. It may include means for determining that the likelihood criterion is not met.

In certain embodiments, for determining one or more values of the divergence metric, the apparatus includes means for determining, on a per prediction basis, each value of the divergence metric for each prediction of the first and second sets of predictions. may include. In an alternative embodiment, for determining one or more values of the divergence metric, the apparatus may include means for determining each value of the divergence metric for one or more subsets of the first and second sets of predictions. have.

In some embodiments, the one or more values of the divergence metric are one or more deployment values of the divergence metric. In such an embodiment, the apparatus comprises (i) a third set of predictions generated by the base machine learning model based on the training data set during development of the base machine learning model and (ii) training data during development of the base machine learning model. means for obtaining one or more development values of a divergence metric representing a difference between the fourth set of predictions generated by the underlying machine learning model based on the set. The apparatus may then include means for comparing one or more distribution values of the divergence metric to one or more developed values of the divergence metric to detect a deviation between the inference data set and the training data set.

In certain embodiments, the one or more actions triggered by the action module include sending an alert regarding a violation of one or more explainability criteria for the underlying machine learning model, wherein the alert includes one or more recommendations for responding to the violation. include more In a further embodiment, the one or more actions triggered by the action module include exchanging the underlying machine learning model for a different machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises retraining the underlying machine learning model. In a further embodiment, the one or more actions triggered by the action module comprises switching the primary machine learning model to the secondary machine learning model. In a further embodiment, the one or more actions triggered by the action module compare the values of the divergence metrics for the third and fourth sets of predictions generated by the primary and secondary machine learning models during development based on the training data set. identifying deviations between the training data set and the inference data set by doing, wherein the values of the divergence metrics for the first and second sets of predictions are set to the primary and secondary machine learning models during deployment based on the inference data set. is created by In further embodiments, the one or more actions triggered by the action module are triggered without receiving confirmation from the user to perform the one or more actions.

Certain embodiments of the subject matter described herein may be implemented to realize one or more of the following advantages. Descriptions of predictions made by machine learning models can be used to justify and support important decisions made based on these predictions. Moreover, by triggering an action in response to a determination that the machine learning model does not meet the explainability criteria, the accuracy of the prediction description can be monitored and improved in real time, so that the accurate description of the prediction is consistently maintained. do. Further, by detecting a change in the value of the divergence metric for predictions generated by the primary and secondary machine learning models, data drift may be detected and appropriate action may be taken to account for such data drift.

The foregoing subject matter, including description of several embodiments, motivations therefor, and/or advantages thereof, is intended to assist the reader in understanding the present disclosure and limits the scope of any of the claims in any way. don't

In order that the advantages of the present invention may be readily understood, a more specific description of some embodiments will be provided with reference to the specific embodiments illustrated in the accompanying drawings. With the understanding that these drawings depict only several embodiments and therefore are not to be considered limiting of their scope, several embodiments will be described and explained in additional specific and detail through the use of the accompanying drawings.
1 is a schematic block diagram illustrating a system for explainability-based tuning of a machine learning model, according to one embodiment.
2A is a schematic block diagram illustrating a machine learning system for explainability-based tuning of a machine learning model, according to an embodiment.
2B is a schematic block diagram illustrating a machine learning system for explainability-based tuning of a machine learning model, according to another embodiment.
2C is a schematic block diagram illustrating a machine learning system for explainability-based tuning of a machine learning model, according to another embodiment.
3 is a schematic block diagram illustrating an apparatus for explainability-based tuning of a machine learning model, according to an embodiment.
4 is a schematic flowchart diagram illustrating a method for explainability-based tuning of a machine learning model, according to an embodiment.
5 is a schematic flowchart diagram illustrating a method for explainability-based tuning of a machine learning model, according to another embodiment.
6A is a schematic flowchart diagram illustrating a method for generating an assisted machine learning model, according to one embodiment.
6B is a schematic flowchart diagram illustrating a method for performing a second-order modeling procedure, according to one embodiment.

References throughout this specification to “one embodiment,” “an embodiment,” “some embodiments,” or similar language refers to a particular feature, structure, or characteristic being described in connection with the embodiment in at least one is meant to be included in the embodiment. Thus, appearances of "in one embodiment," "in an embodiment," "in some embodiments," and similar language throughout this specification are necessarily all referring to the same embodiment, unless expressly stated otherwise. Rather than referring to it, it may also mean “one or more but not all embodiments.”

Moreover, the described features, advantages, and characteristics of the embodiments may be combined in any suitable manner. Those skilled in the art will recognize that the embodiments may be practiced without one or more of the specific features or advantages of the specific embodiments. Additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments in other instances.

These features and advantages of the embodiments will become more fully apparent from the following description and appended claims, or may be learned by practice of the embodiments as set forth below. As will be appreciated by those skilled in the art, aspects of the subject matter described herein may be embodied as systems, methods, and/or computer program products. Thus, aspects of some embodiments are generally hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or "circuits", "modules", or "systems" herein generally. It may take the form of an embodiment combining software and hardware aspects, which may be referred to as Moreover, aspects of some embodiments may take the form of a computer program product embodied in one or more computer readable medium(s) embodied in program code.

Many of the functional units described in this specification have been labeled as modules, to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit including custom VLSI circuitry or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in a programmable hardware device, such as a field programmable gate array, programmable array logic, programmable logic device, or the like.

Modules may also be implemented in software for execution by various types of processors. For example, an identified module of program code may include one or more physical or logical blocks of computer instructions, which may be organized as, for example, objects, procedures, or functions. Nevertheless, the executables of the identified modules do not need to be physically co-located, but rather of a completely different kind that, when logically joined together, contain the modules and are stored in different locations that serve the stated purpose for the modules. may contain commands.

Indeed, a module of program code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may herein be identified and illustrated within modules, embodied in any suitable form, and organized within any suitable type of data structure. Operational data may be collected as a single data set, or may be distributed across different locations, including distributed across different storage devices, and may exist, at least in part, only as electronic signals on a system or network. If a module or a portion of a module is implemented in software, the program code may be stored on and/or propagated on one or more computer-readable medium(s).

A computer program product may include a computer readable storage medium (or media) having computer readable program instructions for causing a processor to execute aspects of some embodiments.

A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. . A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (“RAM”), read-only memory;” ROM"), erasable programmable read-only memory ("EPROM" or flash memory), static random access memory ("SRAM"), portable compact disk disc read-only memory (“CD-ROM”), digital versatile disk (“DVD”), memory sticks, floppy disks, structures raised in the grooves on which instructions are written, or mechanical devices such as punch cards. devices encoded with , and any suitable combination of the foregoing. A “non-transitory” computer-readable storage medium, as used herein, refers to a transitory signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., , pulses of light flowing through a fiber optic cable), or electrical signals transmitted over wires.

The computer readable program instructions described herein can be transmitted from a computer readable storage medium to each computing/processing device, or via a network such as the Internet, local area network, wide area network and/or wireless network to an external computer or an external network. It can be downloaded to a storage device. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within each computing/processing device. do.

Computer readable program instructions for executing the operations of some embodiments may include assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. It may be either a code or an object code. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a standalone software package, partly on the user's computer and partly on a remote computer or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be (e.g., For example, via the Internet using an Internet service provider) to an external computer. In some embodiments, including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs) or programmable logic arrays (PLAs) to perform aspects of some embodiments. The electronic circuitry may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry.

Aspects of some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products in accordance with some embodiments. It will be understood that each block in the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to create a machine, such that the instructions executed by the processor of the other programmable data processing device or computer may include: , flowcharts and/or block diagrams create means for implementing the function/act specified in the block or blocks. These computer readable program instructions capable of instructing a computer, programmable data processing apparatus, and/or other device to function in a particular manner may also be stored in a computer readable storage medium, resulting in the computer readable instructions stored thereon. Capable storage media includes an article of manufacture comprising instructions implementing aspects of a function/act specified in a flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to create a computer implemented process, As a result, instructions executed on a computer, other programmable apparatus, or other device implement the function/act specified in the flowchart and/or block diagram block or blocks.

Schematic flowchart diagrams and/or schematic block diagrams in the drawings illustrate the architecture, functionality and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments. In this regard, each block in the schematic flowchart diagrams and/or schematic block diagrams represents a module, segment or block of code comprising one or more executable instructions of program code for implementing the specified logical function(s). Some may indicate

It should also be noted that, in some alternative implementations, the functions recited in the blocks may occur out of the order recited in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or blocks may sometimes be executed in reverse order, depending on the functionality involved. Other steps and methods may be devised that are equivalent in function, logic or effect to one or more blocks of the illustrated figures, or portions thereof.

Although various arrow types and line types may be utilized in flowcharts and/or block diagrams, it is understood that they do not limit the scope of the corresponding embodiments. Indeed, some arrows or other connectors may be used to indicate only the logical flow of the depicted embodiment. For example, an arrow may indicate a waiting or monitoring period of unspecified duration between enumerated steps of the depicted embodiment. Some arrows or other connectors may be used to indicate the flow of data. Each block in the block diagrams and/or flowchart diagrams, and combinations of blocks in the block diagrams and/or flowchart diagrams, is a special-purpose hardware-based system, or special-purpose hardware and It will also be noted that it may be implemented by a combination of program code.

Apparatus, systems, program products, and methods are disclosed for explainability-based tuning of machine learning models during development and/or deployment.

As referred to herein, the term “machine learning model” may refer to any suitable model artifact generated by the process of training a machine learning algorithm on a particular training data set. Machine learning models may be used to generate predictions. Those skilled in the art will understand that the phrase "fitness of a machine learning model" may refer to the suitability of a model artifact and/or the suitability of an algorithm used by a model artifact to make a prediction based on an inference data set. will be.

As referred to herein, the term “machine learning system” may refer to any environment in which a machine learning model is operated. As discussed in detail below, a machine learning system may include various components, pipelines, data sets, other infrastructure, and the like.

As referred to herein, the term “development” with respect to a machine learning model may refer to the construction of a machine learning model. A machine learning model is constructed by a computer using a training data set. Thus, the development of a machine learning model may refer to the training of a machine learning model using a training data set. In some embodiments, the training data set used to train the machine learning model may include known results (eg, labels). In an alternative embodiment, the training data set does not include known results.

In contrast to the development of a machine learning model, as referred to herein, the term "distribution" in the context of a machine learning model may refer to the use of the developed machine learning model to generate real-world predictions. A deployed machine learning model may have completed development (eg, training). The model may be deployed on any system, including the system on which the model was developed and/or third-party systems. Deployed machine learning models can make real-world predictions based on inference data sets. Unlike certain embodiments of training data sets, inference data sets generally do not contain known results. Rather, the deployed machine learning model is used to generate predictions of outcomes based on the inference data set.

Machine learning models may be used to generate predictions for making important decisions, such as, for example, a person's likelihood of committing a future crime, confidence in loan approval, and medical diagnosis. Machine learning models, and predictions generated by machine learning models, may include various biases based, for example, on gender, geographic location, race, and the like, which may be directly determined by decisions made based on the predictions. It can have a negative impact on the person affected. Accordingly, industry and governments may enact regulations stipulating that predictions can be explained if they are made by machine learning models used to make decisions.

Thus, as used herein, the term “explainability” in reference to a machine learning model or prediction by a model means (i) the extent to which one or more factors contributing to the generation of a prediction by the machine learning model can be accurately identified; and/or (ii) the manner in which the identified factor(s) influenced the prediction may be accurately represented. For simple machine learning models, it can be straightforward to identify factors contributing to the predictions made by the machine learning model. However, as machine learning models become more complex, concise identification of these contributing factors can become increasingly difficult, especially when the models are deployed. Preventing the launch of proprietary model-building techniques can further exacerbate the challenge of making models easy to understand.

In some embodiments, the auxiliary machine learning model may be used to approximate the base machine learning model and to describe the base machine learning model or its prediction. A secondary machine learning model may be relatively simpler than a primary machine learning model, but may be configured to mimic the predictions made by the primary machine learning model. Embodiments of an assisted machine learning model are discussed in detail below with respect to FIGS. 6A and 6B .

Due to the relative simplicity of the auxiliary machine learning model, as well as its approximation of the underlying machine learning model, the auxiliary machine learning model may contain one or more factors and/or identified factor(s) that contribute to the prediction made by the underlying machine learning model. It could also provide a human-comprehensible explanation of the way this prediction was influenced. In other words, the auxiliary machine learning model may provide "explainability" for the primary machine learning model. In some cases, such auxiliary machine learning models may be referred to herein as “canary” models.

However, in practice, the inference data processed by the primary machine learning model during deployment often deviates from the training data processed by the primary machine learning model during development, which causes predictions from the primary and secondary machine learning models to diverge. can do. Additionally, anomalies may exist in the primary machine learning model, which may also cause predictions from the primary and secondary machine learning models to diverge. Such divergence in prediction may make the primary machine learning model unexplainable by the secondary machine learning model, thereby violating the explainability criteria for the primary machine learning model.

As referred to herein, the term “explainability criterion” may refer to a criterion indicating whether an auxiliary machine learning model can explain the prediction of a primary machine learning model. For example, one explainability criterion may be that the divergence of predictions between the primary machine learning model and the secondary machine learning model is less than a threshold divergence. To determine whether the secondary machine learning model is an appropriate approximation of the primary machine learning model, and thus whether the secondary machine learning model can adequately explain the primary machine learning model and/or its predictions, an explainability criterion this can be used

Moreover, as referred to herein, the term “violation of an explainability criterion” may refer to a failure to fulfill (eg, meet) an explainability criterion. More specifically, a violation of the explainability criterion may indicate a failure of a divergence metric determined for predictions from primary and secondary machine learning models that fall below a threshold divergence. A violation of the explainability criterion may indicate that the auxiliary machine learning model is not an adequate approximation of the underlying machine learning model, and thus the auxiliary machine learning model does not adequately describe the underlying machine learning model and/or its predictions.

To prevent violations of the explainability criterion, the underlying machine learning model may be tuned in development and/or deployment by performing one or more actions related to the underlying machine learning model. Thus, as referred to herein, "explainability-based tuning" of a machine learning model is an adjustment of an underlying machine learning model to meet an explainability criterion (e.g., an adjustment for the purpose of meeting an explainability criterion; and/or adjustments having the effect of meeting explainability criteria). Explainability-based tuning of machine learning models is discussed in detail throughout the remainder of this disclosure.

1 is a schematic block diagram illustrating a system 100 for explainability-based tuning of a machine learning model, in accordance with one embodiment. In one embodiment, system 100 includes one or more information handling devices 102 , one or more machine learning (ML) management devices 104 , one or more data networks 106 , and one or more servers 108 . ) is included. In certain embodiments, although a particular number of information handling devices 102 , ML management apparatus 104 , data network 106 , and server 108 are depicted in FIG. 1 , those skilled in the art will In view of the disclosure, it will be appreciated that any number of information handling devices 102 , ML management apparatus 104 , data network 106 , and server 108 may be included in system 100 .

In one embodiment, system 100 includes one or more information handling devices 102 . The information handling device 102 is a desktop computer, laptop computer, tablet computer, smartphone, smart speaker (eg, Amazon Echo®, Google Home®, Apple HomePod®) ), security systems, set-top boxes, gaming consoles, smart TVs, smart watches, fitness bands or other wearable activity tracking devices, optical head-mounted displays (e.g., virtual reality headsets, smart glasses, or the like), high-definition multimedia interfaces ( High-Definition Multimedia Interface (“HDMI”) or other electronic display dongle, personal digital assistant, digital camera, video camera, or processor (eg, central processing unit (“CPU”) ), processor core, field programmable gate array (“FPGA”) or other programmable logic, application specific integrated circuit (“ASIC”), controller, microcontroller, and/or other semiconductor integrated circuit device), volatile memory, and/or other computing devices including non-volatile storage media.

In certain embodiments, the information handling device 102 is communicatively coupled to one or more servers 108 and/or to one or more other information handling devices 102 via a data network 106 described below. do. The information handling device 102 may, in additional embodiments, include processors, processor cores, and/or the like configured to execute various programs, program code, applications, instructions, functions, and/or the like. The information handling device 102 may include executable code, functions, instructions, operating systems, and/or the like for performing various machine learning operations, as described in more detail below.

In some embodiments, the ML management device 104 is configured to determine whether the auxiliary machine learning model can adequately describe the base machine learning model. In other words, the ML management apparatus 104 may be configured to determine whether the auxiliary machine learning model can accurately identify one or more factors that contributed to the identification of the prediction by the base machine learning model.

To determine whether the secondary machine learning model can adequately describe the primary machine learning model, the ML management unit 104 monitors (e.g., the primary machine learning model and the secondary machine learning model) during development and/or deployment. continuous or incremental monitoring). Specifically, the ML management device 104 is configured to assist based on a set of predictions made by a basic machine learning model based on a data set (eg, a training data set during development or an inference data set during deployment) and the same data set. may receive a set of predictions made by the machine learning model, for example, a root-mean-square error (“RMSE”) analysis, other statistical analysis, and/or any other suitable technique. can be used to compare different sets of predictions. Based on this comparison of the primary and secondary sets of predictions, the ML management apparatus 104 may determine a value of a divergence metric indicating a degree of divergence between the sets of predictions. The determined divergence metric may then be evaluated according to the explainability criterion for the underlying machine learning model to determine whether the explainability criterion has been violated. For example, the determined divergence metric may be compared to a threshold divergence to determine whether an explainability criterion for the underlying machine learning model has been violated.

A non-violation of the explainability criterion may indicate that the secondary machine learning model is a good approximation of the primary machine learning model, moreover, that the secondary machine learning model can adequately describe the primary machine learning model. . On the other hand, a violation of the explainability criterion may indicate that the secondary machine learning model is not a good approximation of the primary machine learning model, moreover, that the secondary machine learning model cannot adequately describe the primary machine learning model. In some embodiments, a determination made during development of the primary machine learning model (eg, a determination that the secondary machine learning model is a good approximation of the primary machine learning model) may no longer be true following deployment of the primary machine learning model. may be This may be the result of a deviation between the training dataset used during development and the inference dataset used during deployment of primary and secondary machine learning models.

When the explainability criterion for the basic machine learning model is violated, the ML management device 104 may take an action related to the basic machine learning model. For example, the ML management device 104 may investigate why the predictions from the two models do not meet the explainability criteria, and generate an alert indicating that the predictions for the two models do not meet the explainability criteria. and/or adjust the underlying machine learning model to meet explainability criteria. In this way, the underlying machine learning model may be monitored and, if necessary, modified (eg, continuously or incrementally) during development and/or deployment.

The ML management apparatus 104 , including any sub-modules, is located on one or more of the information handling devices 102 , one or more of the servers 108 , one or more network devices, and/or other components of the system 100 . can be In various embodiments, the ML management apparatus 104 (or any module thereof) is configured on one or more of the information handling devices 102 , on one or more of the servers 108 , or elsewhere on the data network 106 . It may also be embodied as a hardware appliance that can be installed or deployed. The hardware appliance of the ML management device 104 is configured to perform the functions described herein with respect to a power interface, a wired and/or wireless network interface, a graphical interface attached to a display, and/or the ML management device 104 . semiconductor integrated circuit devices that may be In certain embodiments, the ML management apparatus 104 may include a hardware device such as a secure hardware dongle, set-top box, network appliance, or other hardware device. In some cases, the hardware device may be connected via a wired connection (eg, a universal serial bus (“USB”) connection) or a wireless connection (eg, Bluetooth® (Bluetooth), Wi-Fi (Wi-Fi), and/or to a device such as a laptop computer, one or more of the servers 108, a tablet computer, a smartphone, a security system, or the like, by any one of and/or near-field communication (“NFC”). can In alternative embodiments, the hardware device is an electronic display device (eg, a television or monitor using an HDMI port, a DisplayPort port, a Mini DisplayPort port, a VGA port, and/or a DVI port). ) can be attached to

As noted above, in some embodiments, the ML management apparatus 104 is a semiconductor integrated circuit device (eg, one or more chips, dies, or other discrete logic hardware), such as a field programmable gate array (“FPGA”). ") or other programmable logic, firmware for FPGA or other programmable logic, microcode for execution on a microcontroller, application specific integrated circuits ("ASICs"), processors, processor cores, or the like. In one embodiment, the ML management device 104 provides one or more electrical lines or connections (eg, to volatile memory, non-volatile storage media, network interfaces, peripheral devices, and/or graphics/display interfaces). It can be mounted on a printed circuit board with The ML management device 104 may include one or more pins, pads, other electrical connections configured to transmit and receive data (eg, in communication with one or more electrical lines of a printed circuit board), one or more hardware circuits, and/or Alternatively, it may include other electrical circuitry configured to perform various functions of the ML management device 104 .

The semiconductor integrated circuit device or other hardware appliance of the ML management apparatus 104 may include, but is not limited to, random access memory (“RAM”), dynamic RAM (“DRAM”), or cache. may include and/or be communicatively coupled to one or more volatile memory media. In one embodiment, the semiconductor integrated circuit device or other hardware appliance of the ML management apparatus 104 is a NAND flash memory, a NOR flash memory, a nano random access memory (nano RAM or NRAM), a nanocrystal wire based memory, silicon oxide based sub-10 nanometer process memory, graphene memory, silicon-oxide-nitride-oxide-silicon (“SONOS”), resistive RAM (RAM); "RRAM"), programmable metallization cell ("PMC"), conductive-bridging RAM ("CBRAM"), magneto-resistive RAM ("MRAM"), dynamic RAM ( “DRAM”), phase change RAM (“PRAM” or “PCM”), magnetic storage media (eg, hard disk, tape), and/or optical storage media, but these It may include and/or be communicatively coupled to one or more non-volatile memory media, including but not limited to.

The data network 106 of the system may be a digital communications network that transmits digital communications. The data network 106 may include a wireless network, such as a wireless cellular network, a local wireless network, such as a Wi-Fi network, a Bluetooth® network, a near field communication (“NFC”) network, and/or an ad hoc network. The data network 106 may include a wide area network (“WAN”), a storage area network (“SAN”), a local area network (LAN), a fiber optic network, the Internet, and/or other digital communication networks. have. The data network 106 may include two or more networks. Data network 106 may include one or more servers, routers, switches, and/or other networking devices. Data network 106 may also include one or more computer-readable storage media, such as hard disk drives, optical drives, nonvolatile memory, and/or RAM.

The wireless connection may be a mobile phone network. The wireless connection may also utilize a Wi-Fi network based on any one of the American Institute of Electrical and Electronics Engineers (“IEEE”) 802.6 standard. Alternatively, the wireless connection may be a Bluetooth® connection. In addition, wireless connectivity is supported by the International Organization for Standardization (“ISO”), the International Electrotechnical Commission (“IEC”), the American Society for Testing and Materials® (ASTM®), Radio Frequency Identification (“RFID”) communications may be utilized, including the RFID standard established by the DASH7™ Alliance and EPCGlobal™. Alternatively, the wireless connection may utilize a ZigBee® connection based on the IEEE 802 standard. In one embodiment, the wireless connection utilizes a Z-Wave® connection as designed by Sigma Designs®. Alternatively, the wireless connection may utilize an ANT® and/or ANT+® connection as defined by Dynastream® Innovations Inc. of Cochrane, Canada.

A wireless connection is at least an infrared connection, including a connection that complies with the Infrared Physical Layer Specification (“IrPHY”) as defined by the Infrared Data Association® (“IrDA®”). may be Alternatively, the wireless connection may be a cellular telephone network communication. Each standard and/or connection type may include the latest version and revision of the standard and/or connection type as of the filing date of this disclosure.

One or more servers 108 may be embodied as blade servers, mainframe servers, tower servers, rack servers, and/or the like. One or more servers 108 may be configured as mail servers, web servers, application servers, FTP servers, media servers, data servers, web servers, file servers, virtual servers, and/or the like. One or more servers 108 may be communicatively coupled (eg, networked) to one or more of the information handling devices 102 via a data network 106 . The one or more servers 108 may store data related to one or more of the information handling devices 102 , such as machine learning data, models, training models, and/or the like.

2A is a schematic block diagram illustrating a machine learning system 200 for explainability-based tuning of a machine learning model, according to one embodiment. In the embodiment shown in FIG. 2A , the machine learning system 200 may include, for example, a logical machine learning layer. In some embodiments, the logical machine learning layer may be an intelligence overlay network (“ION”). A machine learning system may include various components, pipelines, data sets, and infrastructure. The machine learning system 200 includes one or more policy pipelines 202 , one or more development pipelines 204 , one or more deployment pipelines 206a - c , one or more optional databases 208 , and a data set. 210 , and/or one or more of the ML management devices 104 discussed above.

As used herein, pipelines 202 , 204 , 206a - c are pipelines to perform various machine learning operations, such as model development, model deployment, feature engineering, validation, scoring, and/or the like. It may include various machine learning features, engines, models, components, objects, and/or modules that may be used. For example, pipelines 202 , 204 , 206a - c may analyze or process data from data set 210 . This data processing occurs live - in this case, the data is processed in real time - , in batch processing - in this case the data is processed in sets from a static source - , via streaming - in this case the data is incremented in time Processed in units - or via micro-batch processing - in which case the data is processed according to a combination of batch processing and streaming - may be performed.

Although a particular number of pipelines 202 , 204 , 206a - c are depicted in FIG. 2A , any suitable number of pipelines 202 , 204 , 206a - c may be present in the machine learning system 200 . . Moreover, as depicted in FIG. 2A , the various pipelines 202 , 204 , 206a - c may be connected to the information handling device 102 described above, a virtual machine, and/or a device such as a cloud or other remote device ( 203, 205, 207a-c) may be located on different nodes.

In one embodiment, each pipeline 202 , 204 , 206a-c runs on the particular type of analysis engine on which the pipeline 202 , 204 , 206a-c is configured. As used herein, an analysis engine includes instructions, code, functions, and/or libraries for performing machine learning numerical calculations and analysis. Some non-limiting examples of analytics engines include Spark, Flink, TensorFlow, Caffe, Theano, and PyTorch. The pipelines developed for these engines (202, 204, 206a-c) use components provided in libraries for specific analytics engines (eg, Spark-ML/MLlib for Spark and Flink-ML for Flink). may include A custom program developed for each analytics engine using an application programming interface to the analytics engine (eg, Data set/DataStream for Flink) may also be included. Moreover, each pipeline 202 , 204 , 206a - c may be implemented using several different platforms, libraries, and/or programming languages. For example, one deployment pipeline 206a may be implemented using Python (Python), while a different deployment pipeline 206b may be implemented using Java (Java).

In one embodiment, the machine learning system 200 includes a physical and/or logical grouping of pipelines 202 , 204 , 206a - c based on an action to be performed. For example, the ML management device 104 is configured to make predictions based on an inference data set, as well as a development pipeline 204 for developing a machine learning model to perform a specific prediction based on a training data set. It is also possible to select one or more distribution pipelines 206a - c to be used. The distribution pipelines 206a - c may process the inference data set using the analysis engine on which the selected distribution pipelines 206a - c are configured. Thus, a pipeline grouping may include multiple analysis engines. Additionally, the analysis engine may be part of multiple pipeline groupings.

Grouping includes processing data from data set 210, managing the behavior of other groupings, monitoring predictions of other groupings, and experimenting with different machine learning models in a controlled environment (e.g. for example, sandboxing), and/or any other action. For example, the logical grouping of pipelines 202 , 204 , 206a-c may provide feedback, predictions, messages, and/or other indicators from the monitored logical grouping of pipelines 202 , 204 , 206a-c. pipelines 202, 204, by processing and/or by providing input to monitored logical groupings of pipelines 202, 204, 206a-c to detect anomalies, errors, and/or other information. 206a-c) may be configured to analyze prediction, performance, operation, health, and/or other features of the different logical groupings.

In some embodiments, multiple different pipelines 202 , 204 , 206a - c run on the same device. In an alternative embodiment, each pipeline 202 , 204 , 206a - c runs on a separate device. Devices 203 , 205 , 207a - c may be located in a single location, connected to the same network, located in the cloud or other remote location, and/or located in any combination of the foregoing. have.

Because pipelines 202, 204, 206a-c may be located on different devices 203, 205, 207a-c, or on the same device 203, 205, 207a-c, ML management apparatus 104 may logically group pipelines 202 , 204 , 206a - c that are best configured to perform a given action. As will be described in more detail below, the logical grouping of pipelines 202, 204, 206a-c is such that the logical grouping of pipelines 202, 204, 206a-c is specifically configured for the particular action to be performed. It may be predefined.

In certain embodiments, the ML management device 104 is configured to, when a request for an action is determined or received based on the characteristics and/or settings of the pipelines 202 , 204 , 206a - c, Dynamically select pipelines 202, 204, 206a-c. In certain embodiments, multiple different logical groupings of pipelines 202 , 204 , 206a - c may share the same physical infrastructure, platform, device, and/or virtual machine. Moreover, different logical groupings of pipelines 202 , 204 , 206a - c may be merged based on a given action to be performed.

The policy pipeline 202 may be configured to manage the operation of the machine learning system 200 . In certain embodiments, the policy pipeline 202 receives the trained machine learning model from the development pipeline 204 and distributes the trained machine learning model to the deployment pipeline 206a to make predictions based on the inference data set. -c) to push. In further embodiments, the policy pipeline 202 may receive user input, receive feedback information from other pipelines 204 , 206a - c, validate the machine learning model, and It may facilitate data transmission between lines 202 , 204 , 206a - c and/or perform additional actions.

In one embodiment, policy pipeline 202 includes one or more policies that define how pipelines 204 , 206a - c interact with each other. For example, the development pipeline 204 may output a machine learning model after the training cycle is complete. Several possible policies can define how machine learning models are handled. For example, a policy may specify that the machine learning model should be automatically pushed to the deployment pipeline 206a-c, while another policy may specify that the policy pipeline 202 pushes the machine learning model to the deployment pipeline ( You can also specify that user input is required to approve the machine learning model before pushing to 206a-c).

Policies may further define how machine learning models are updated. For example, a policy may indicate that a machine learning model should be automatically updated based on feedback (e.g., based on machine learning predictions received from development pipeline 204 and/or deployment pipeline 206a-c). You can also specify that As another example, a policy may specify scheduling information within the machine learning system 200 , such as how often (eg, intermittently or continuously) the machine learning model is updated.

Policies may define how different logical groupings of pipelines 202 , 204 , 206a - c interact. For example, the policy states that predictions produced by one grouping of pipelines 202, 204, 206a-c should be used as input to a different grouping of pipelines 202, 204, 206a-c. may be specified. For example, one grouping of pipelines 202 , 204 , 206a-c may receive a training data set to train a machine learning model, while pipelines 202 , 204 , 206a-c may receive a training data set. Another grouping may receive an inference data set for input into a deployed machine learning model.

In one embodiment, policy pipeline 202 maintains a mapping of pipelines 204 and 206a-c, where the mapping includes logical groupings of pipelines 204 and 206a-c. The policy pipeline 202 may further adjust various settings of the pipelines 204 , 206a - c in response to user input, feedback, and/or events. For example, if the distribution pipeline 206a produces an incorrect machine learning prediction, the policy pipeline 202 may receive a message from the distribution pipeline 202 indicating that the prediction is incorrect, the distribution pipe It may instruct the development pipeline 204 to develop a new machine learning model for line 206a.

The development pipeline 204 may be configured to develop a machine learning model for performing a particular prediction, based on a training data set associated with the prediction. More specifically, the development pipeline 204 may be configured to develop a machine learning model for performing a particular prediction by running the training model on a training data set associated with the prediction. A machine learning model may be an artifact produced by a training process that captures patterns in a training data set that map the training data set to a desired prediction. The training data set may be a static data set, data accessible from an online source, a streaming data set, and/or any other type of data set.

In some embodiments, new machine learning models are developed periodically through batch processing in a development pipeline 204 that may be run on large clustered analytics engines within a data center. As the development pipeline 204 creates new machine learning models, in certain embodiments, the administrator may be notified via the policy pipeline 202 . The administrator may review the generated machine learning model, and if the administrator approves, the machine learning model may be pushed to the deployment pipeline 206a - c.

The deployment pipelines 206a - c may use machine learning models and corresponding analytics engines developed to generate machine learning predictions based on inference data sets. The inference data set may include data relating to predictions to be made, but not part of the training data set used to train the machine learning model. Moreover, unlike training data sets, inference data sets generally do not contain known results. For example, if a user wants to know whether an e-mail is spam or not, the development pipeline 204 can generate a training data set containing both e-mail known to be spam and e-mail known not to be spam. It can also be used to develop machine learning models. After the machine learning model is developed, the policy pipeline 202 may push the machine learning model to the deployment pipelines 206a-c, where the machine learning model determines that one or more emails provided as inference data are spam. It can be used to predict cognition.

2B is a schematic block diagram illustrating a machine learning system 225 for explainability-based tuning of a machine learning model, according to another embodiment. In the embodiment shown in FIG. 2B , the machine learning system 225 may include, for example, a logical machine learning layer. In one embodiment, the machine learning system 225 of FIG. 2B is substantially similar to the machine learning system 200 depicted in FIG. 2A . In addition to the elements of machine learning system 200 depicted in FIG. 2A , machine learning system 225 of FIG. 2B includes a plurality of development pipelines 204a - b executing on development devices 205a - b do.

In the depicted embodiment, the development pipelines 204a - b develop machine learning models for making predictions, based on one or more sets of training data for the predictions. The training data set may be different for each of the development pipelines 204a - b. For example, the training data set for the first development pipeline 204a may include historical data for a first predefined time period, while the training data set for the second development pipeline 204b may include historical data for a different second predefined time period. Variations in the training data set may include different types of data, data collected at different time periods, different amounts of data, and/or other types of variations in data.

In another embodiment, the development pipelines 204a - b may run different machine learning models on different training data sets or on the same training data set. For example, the first development pipeline 204a may implement a machine learning model TensorFlow using Python, while the second development pipeline 204b may implement a different machine learning model in Spark using Java. can be implemented.

In one embodiment, the machine learning system 225 receives the machine learning models that the development pipelines 204a - b develop, and determines which of the machine learning models is the best fit for making a particular prediction. and a model selection module 212 configured to: The most suitable machine learning model is the machine learning model (e.g., the most accurate machine learning model) that produced a prediction that most closely resembles a reference prediction (e.g., a "label" or "target") on the training data. ), the machine learning model that runs the fastest, the machine learning model that requires the least amount of configuration, and/or the machine learning model that demonstrates good performance according to any other suitable metric.

In one embodiment, the model selection module 212 is configured to perform a hyper-parameter search to develop a machine learning model and to determine which of the developed machine learning models is best suited for a given prediction problem. ) is performed. As used herein, hyperparameter search (or hyperparameter “optimization” or “tuning”) is the problem of selecting a set of hyperparameters (eg, an optimal set of hyperparameters) for a machine learning algorithm. In certain embodiments, the same kind of machine learning model may use different constraints, weights, and/or learning rates to generalize different data patterns. These measurements may be referred to as hyperparameters, and the model may be tuned to solve (eg, optimally solve) a machine learning problem. The goal of hyperparameter optimization is to create a machine learning model (e.g., optimal It may be finding a set of hyperparameters that yield a machine learning model). In certain embodiments, the model selection module 212 combines different features of different machine learning models to create a single combined model. In one embodiment, the model selection module 212 pushes the selected machine learning model to the policy pipeline 202 for propagation to the deployment pipelines 206a - c. In various embodiments, the model selection module 212 is part of, communicatively coupled to, and/or operatively coupled to the ML management device 104 .

2C is a schematic block diagram illustrating a machine learning system 250 for explainability-based tuning of a machine learning model, according to another embodiment. In the embodiment shown in FIG. 2C , machine learning system 250 may include, for example, a logical machine learning layer. The machine learning system 250 of FIG. 2C may be substantially similar to the machine learning systems 200 and 225 depicted in FIGS. 2A and 2B , respectively. However, FIG. 2C further illustrates a federated learning embodiment of the machine learning system 250 .

In a federated machine learning system, development pipelines 204a - c may be located on the same device as corresponding deployment pipelines 206a - c. In such an embodiment, the development pipelines 204a - c may develop different machine learning models and send the machine learning models to the model selection module 212 , which determines which machine learning model for the machine learning system 250 . You may decide which one is the best fit or you may combine different machine learning models as described above. The selected machine learning model may be pushed to the policy pipeline 202 for validation. The policy pipeline 202 may then push the model back to the deployment pipelines 206a-c.

3 is a schematic block diagram illustrating an apparatus 300 for explainability-based tuning of a machine learning model, according to an embodiment. In the embodiment depicted in FIG. 3 , the apparatus 300 comprises an embodiment of the ML management apparatus 104 . The ML management device 104 includes a first prediction module 302 , a second prediction module 304 , a comparison module 306 , and an action module 308 , each of which is described in detail below.

The first prediction module 302 may be configured to obtain (eg, receive) a first set of predictions made by the underlying machine learning model. The basic machine learning model may be a complex machine learning model, such as a first-order machine learning model. For example, the base machine model may include an artificial neural network, such as a multilayer perceptron (“MLP”), comprising multiple different layers of nodes or pipelines. Other non-limiting examples of complex machine learning models include deep learning models, ensemble (or "blended") models (eg, models comprising a combination of a number of different machine learning models), boosting models. model), a bagging model, and/or a support vector machine. A basic machine learning model may be trained using a training data set similar to the inference data set that the model will process during deployment.

In some embodiments, the first set of predictions received by the first prediction module 302 is made by the underlying machine learning model during deployment, based on the inference data set. In some additional embodiments, the first set of predictions is made by the underlying machine learning model in a live deployment (eg, in real time), based on the inference data set. As an example, the underlying machine learning model could be an MLP model deployed to analyze e-mail messages in real time (e.g. when they are received), to flag e-mail messages as spam or junk mail. have. In an alternative embodiment, the first set of predictions received by the first predictive model 302 are made by the underlying machine learning model during development, based on the training data set.

The first set of predictions made by the underlying machine learning model may include a list, table, database, and/or any other data structure that stores predictions that the underlying machine learning model makes. The first prediction module 302 may store the first set of predictions locally or on a remote device (eg, a cloud device). The first prediction module 302 may query the first set of predictions, poll for the first set of predictions, and/or receive the first set of predictions pushed to it. The first prediction module 302 generates a first set of predictions live (eg, when predictions are generated), in a batch (eg, every 100 predictions), and via streaming. may be received incrementally (eg, every 30 seconds, every minute, or the like), or according to any other suitable schedule.

Bayes(나이브 베이즈) 모델, k 개의 최근접 이웃 모델(k-nearest neighbors model), 및/또는 등등을 포함할 수도 있다. 보조 머신 러닝 모델의 몇몇 실시형태는 도 6a 및 도 6b를 참조하여 하기에서 상세하게 논의된다. 그들의 상대적인 단순성에 기인하여, 보조 머신 러닝 모델은 설명 가능한 경향이 있을 수도 있다. 다시 말하면, 상기에서 설명되는 바와 같이, 소정의 예측을 행하는 보조 머신 러닝 모델에 기여하는 요인이 결정될 수도 있다.The second prediction module 304 may be configured to obtain (eg, receive) a second set of predictions made by the auxiliary machine learning model. In contrast to the primary machine learning model, the secondary machine learning model may be a relatively simple machine learning model. Some non-limiting examples of assisted machine learning models include decision tree models, logistic regression models, linear models, RuleFit models,

a Bayes model, a k-nearest neighbors model, and/or the like. Some embodiments of an assisted machine learning model are discussed in detail below with reference to FIGS. 6A and 6B . Due to their relative simplicity, assisted machine learning models may tend to be accountable. In other words, as described above, factors contributing to an assistive machine learning model making a given prediction may be determined.

Notwithstanding these differences between the primary machine learning model and the secondary machine learning model, the secondary machine learning model may be configured to approximate the primary machine learning model and thus mimic the predictions made by the primary machine learning model. Specifically, the secondary machine learning model may be trained using the same, or substantially similar training data as the primary machine learning model. Moreover, since the first and auxiliary machine learning models are trained using the same training data set, the auxiliary machine learning model may be used to process the same inference data set as the primary machine learning model. Consequently, the secondary machine learning model may mimic the predictions of the primary machine learning model, and any variability in the predictions between the two models is not due to differences in the training or inference data sets. Thus, because an auxiliary machine learning model is not only an explanatory machine learning model, it is also constructed to mimic the predictions of the underlying machine learning model, so to account for the predictions made by the relatively more complex underlying machine learning model, the auxiliary machine learning model is can be used Moreover, in some embodiments, predictions made by primary and secondary machine learning models during development and during deployment are used to determine whether characteristics of a data set processed during deployment differ significantly from characteristics of a data set processed during development. may be compared.

In certain embodiments, the second prediction module 304 selects an auxiliary machine learning model used to approximate the base machine learning model from a plurality of available machine learning models that can be described. For example, the second prediction module 304 may generate a prediction, based on the same training data set, that any of the models is closest to a prediction that the underlying machine learning model makes, or that is within a threshold measure of similarity to that prediction. Each of a plurality of available machine learning models may be trained to determine whether In some embodiments, the secondary machine learning model selected to approximate the base machine learning model may be trained on the same training data set as the base machine learning model, where labels in the training data set indicate that the base machine learning model is the training data. has the modification that it is replaced by the prediction it generates for

In some embodiments, the comparison module 306 is configured to generate predictions (eg, first and second sets of predictions obtained by the first and second prediction modules, or to compare some of them). and determine a value of the divergence metric indicating the degree of any difference between predictions. Specifically, the comparison module 306 may calculate a difference between predictions made by the primary and secondary machine learning models to determine a value of a divergence metric for the primary and secondary machine learning models. The comparison module 306 may utilize any statistical method for comparing predictions to calculate a value of a divergence metric. For example, the comparison module 306 may calculate a root mean square error (“RMSE”) or mean absolute error (“MAE”) for the prediction to determine a value of the divergence metric.

In some embodiments, the comparison module 306 may compute values of the divergence metrics for primary and secondary machine learning models on a per-prediction basis (eg, on a per-prediction basis). For example, the comparison module 306 may compute a value of a divergence metric for each prediction made by the primary and secondary machine learning models when the prediction is generated. In an alternative embodiment, the comparison module 306 may compute, for a set of predictions, the values of the divergence metrics for the primary and secondary machine learning models in batches. For example, the comparison module 306 may compute a value of the divergence metric for a set of N most recent predictions made by the primary and secondary machine learning models, where N is any suitable integer (eg, 100).

As discussed above, the value of the divergence metric may violate the explainability criterion. For example, the explainability criteria may include a threshold divergence of predictions of a primary machine learning model and a secondary machine learning model. Violation of the explainability criterion may indicate a failure of the determined value of the divergence metric to fall below a threshold divergence value. For example, if the value of the determined divergence metric is greater than a predetermined divergence threshold (eg, 0.5, 1, 5, or the like), the explainability criterion may be violated.

In an embodiment in which the comparison module 306 calculates the value of one or more divergent metrics on a per-prediction basis, if the value of the per-prediction divergence metric is greater than a predetermined divergence threshold, then the comparison module 306 calculates a corresponding explanation It may also determine that the likelihood criterion is violated. Similarly, in an embodiment in which the comparison module computes, for a set of predictions, the values of one or more divergence metrics in batches, if the value of the divergence metric for the set of predictions is greater than a predetermined divergence threshold, then: The comparison module 306 may determine that a corresponding explainability criterion is violated. A violation of one or more explainability criteria may indicate that the secondary machine learning model is no longer an adequate approximation of the primary machine learning model, and thus the secondary machine learning model does not provide adequate explainability of the primary machine learning model. .

In some embodiments, to determine whether a violation of one or more explainability criteria indicates that the secondary machine learning model is no longer an appropriate approximation of the primary machine learning model, the comparison module 306 is configured to: Determine whether a predetermined amount or ratio of divergence metric values for the learning model violates an explainability criterion. For example, if only one divergence metric value in the set of 500 divergence metric values for the primary and secondary machine learning models violates one or more explainability criteria, the comparison module 306 is configured to: It may be determined that one divergence metric value that violates is a criterion and does not indicate that the secondary machine learning model is an imprecise approximation of the primary machine learning model. On the other hand, if 450 divergent metric values in the set of 500 divergent metric values for the primary and secondary machine learning models violate one or more explainability criteria, the comparison module 306 is configured to: It may be determined that these violations of the explainability criterion indicate that it is an incorrect approximation of the learning model. In some embodiments, after it is determined that a violation of one or more explainability criteria indicates that the secondary machine learning model is not an exact approximation of the primary machine learning model, adjusting the primary machine learning model to meet one or more explainability criteria Corrective action may be taken to

In some embodiments, the foregoing evaluation of the meaning of the set of explainability violations may be performed with respect to a set of predictions obtained within a predetermined period of time. In some embodiments, the foregoing evaluation of the meaning of the set of explainability violations may be performed with respect to a set of N most recently obtained predictions, where N is any suitable integer.

The action module 308 may be configured to trigger one or more actions related to the primary machine learning model in response to a determination by the comparison module 306 that the primary and secondary machine learning models violate the explainability criterion. For example, the action module 308 may send an alert indicating that one or more explainability criteria have been violated. The alert may further include one or more recommendations for responding to a violation of the explainability criterion. These recommendations may include a recommendation for retraining a primary machine learning model using a different training data set, a recommendation for selecting a different primary machine learning model, a recommendation for deploying a secondary machine learning model in place of the primary machine learning, and/or Any other suitable recommendations may be included.

In one embodiment, the action module 308 automatically performs the action in response to a determination by the comparison module 306 that the primary and secondary machine learning models violate the explainability criteria. For example, the action module 308 may automatically exchange a base machine learning model under development for a different base machine learning model that may meet explainability criteria. As another example, the action module 308 may use a different training data set (e.g., using the same machine learning algorithm currently used to train the underlying machine learning model) until the explainability criterion is met. or by using a different machine learning algorithm) to automatically start training the underlying machine learning model. More specifically, the action module 308 may iterate over different training data sets to retrain the underlying machine learning model until the explainability criteria are met. In some embodiments, the training data set may include training data similar to inference data recently processed by the underlying machine learning model. As another example, the action module 308 may automatically exchange a deployed primary machine learning model with a secondary machine learning model. In such embodiments, the secondary machine learning model may permanently or temporarily replace the primary machine learning model, while the primary machine learning model undergoes modifications to meet the explainability criteria.

In an alternative embodiment, the action module 308 performs the action in response to a determination by the comparison module 306 that the primary and auxiliary machine learning models violate the explainability criterion, rather than automatically performing the action. The user may be prompted for confirmation to do so. For example, the action module 308 may be configured to exchange different training data sets for exchanging a primary machine learning model under development with a different primary machine learning model, and for exchanging a deployed primary machine learning model with a secondary machine learning model. can be used to prompt the user for confirmation to train the underlying machine learning model, and/or to perform any other corrective action. In certain embodiments, in response to a prompt from the action module 308 , the user may provide a simple yes/no answer. In an alternative embodiment, in response to a prompt from the action module 308 , the user may provide a more detailed response, such as selecting a different base machine learning model or a different training data set.

In some other embodiments, the action module 308 is also configured to determine whether characteristics of the inference data set seen in deployment deviate from characteristics of the training data set seen during development (eg, the training data set). (based on) the values of the divergence metrics for the predictions generated by the primary and secondary machine learning models during development (e.g., based on the inference data set); can be compared with the value of the divergence metric for As discussed above, this deviation (sometimes referred to as "data drift" or "drift") between the training data set and the inference data set can contribute to the divergence in the predictions made by the primary and secondary machine learning models. have. Accordingly, in some embodiments, the action module 308 may take the various actions described above to correct any determined deviation. For example, the action module 308 may be configured to perform one or more of the actions described above when a difference between a value of a divergence metric for a prediction generated during deployment and a prediction generated during development exceeds a threshold difference. may be In this context, the “difference” between the values of the divergence metric may be an absolute difference, a percentage difference, or any other suitable type of difference.

4 is a schematic flowchart diagram illustrating a method 400 for explainability-based tuning of a machine learning model, according to an embodiment. In one embodiment, the method 400 begins and the first prediction module 302 generates a first set of predictions from the underlying machine learning model during development based on a training data set, or during deployment based on an inference data set. Receive (402). In embodiments in which a first set of predictions is received from an underlying machine learning model during deployment based on an inference data set, the first set of predictions is based on live (eg, real-time) processing of the inference data set for basic machine learning. This can be done by the model.

In one embodiment, the second prediction module 304 is configured to generate a second set of predictions from the auxiliary machine learning model based on a data set that was used by the underlying machine learning model (eg, a training data set or an inference data set). Receive (404). The secondary machine learning model may be different from the primary machine learning model and may be configured to mimic the predictions of the primary machine learning model.

The comparison module 306 is configured to calculate a value of a divergence metric for predictions generated by the primary and secondary machine learning models, and to determine whether the divergence metric violates one or more descriptive criteria, first and second The two predictions may be compared (406). If the determined divergence metric does not violate the explainability criterion, the method 400 may end. If the determined divergence metric violates the explainability criterion, the action module 308 may trigger 408 one or more actions related to the underlying machine learning model, and the method 400 may then end.

5 is a schematic flowchart diagram illustrating a method 500 for explainability-based tuning of a machine learning model, according to another embodiment. In one embodiment, the method 500 begins and the first prediction module 302 generates a first set of predictions from the underlying machine learning model during development based on a training data set, or during deployment based on an inference data set. Receive (502). In embodiments in which a first set of predictions is received from an underlying machine learning model during deployment based on an inference data set, the first set of predictions is based on live (eg, real-time) processing of the inference data set for basic machine learning. It can also be done by model. A first set of predictions from the basic machine learning model includes an ML management unit 104, a first prediction module 302, a device driver, a controller running on a host computing device, a processor, an FPGA, an ASIC, other logic hardware, and and/or from one or more of other executable code stored on a computer-readable storage medium.

In one embodiment, the second prediction module 304 generates predictions from the secondary machine learning model during training based on a data set that was used by the primary machine learning model (eg, a training data set or an inference data set). Two sets are received (504). The secondary machine learning model may be different from the primary machine learning model and may be configured to mimic the predictions of the primary machine learning model. The second set of predictions from the auxiliary machine learning model includes the ML management unit 104, the second prediction module 304, a device driver, a controller running on a host computing device, a processor, an FPGA, an ASIC, other logic hardware, and and/or from one or more of other executable code stored on a computer-readable storage medium.

In some embodiments, the comparison module 306 compares ( 506 ) the first and second sets of predictions to compute a value of a divergent metric for the primary and secondary machine learning models, wherein the value of the divergent metric is one or more Determines whether the explainability criterion is violated. If the determined value of the divergence metric does not violate the explainability criterion, the method 500 may end. If the determined value of the divergence metric violates one or more explainability criteria, the action module 308 triggers one or more actions related to the underlying machine learning model.

For example, the action module 308 may exchange 508 the base machine learning model under development, retrain the base machine learning model under development 510, and the base machine learning model under deployment. can switch 512 to the secondary machine learning model, and/or change the divergence metric values for predictions generated by the primary and secondary machine learning models based on the training data set to the primary and secondary based on the inference data set. A deviation between the training data set and the inference data set may be detected ( 514 ) by comparing it to a value of a divergence metric for predictions generated by the machine learning model. When the primary machine learning model is exchanged 508 or retrained 510, the secondary machine learning model may also be exchanged or retrained. One or more actions related to the primary machine learning model to be taken in response to a value of a divergence metric of the primary and secondary machine learning models not meeting the explainability criterion include: the ML management apparatus 104 , the action module 308 , the device may be performed or controlled by one or more of a driver, a controller running on a host computing device, a processor, an FPGA, an ASIC, other logic hardware, and/or other executable code stored on a computer-readable storage medium. . The method 500 then ends.

6A and 6B are schematic flowchart diagrams illustrating methods 600 and 620 for generating an assisted machine learning model, in accordance with some embodiments. In general, as discussed throughout this disclosure, it may be desirable to create a relatively simple auxiliary machine learning model to provide the descriptive potential of a relatively complex basic machine learning model. In this way, the secondary machine learning model can provide insight into the primary machine learning model that would otherwise be a "black box."

To create a relatively simple auxiliary machine learning model that mimics the predictions of the primary machine learning model without significantly reducing the model's accuracy, the secondary machine learning model is a secondary machine learning model of the primary ("primary") machine learning model. It could be a model. The secondary secondary machine learning model may mimic the predictions of the primary machine learning model based on the same input (or a subset thereof) and/or one or more inputs not used by the primary machine learning model.

In some embodiments, the secondary assisted machine learning model is one or more modeling techniques (e.g., RuleFit, generalized additive model (e.g., RuleFit, generalized additive model) Generalized Additive Models), etc.). Such techniques are referred to herein as “second-order modeling techniques”. After the primary machine learning model is generated, a secondary secondary machine learning model of the primary machine learning model may be generated using secondary modeling techniques.

For each input to the basic machine learning model, there is a corresponding set of input values from, or derived from, the original training data set. The quadratic modeling technique may use the same inputs as the underlying machine learning model, and thus may use the original values of those inputs, or a subset thereof, for the training and validation data sets of the quadratic modeling technique. In some embodiments, instead of using the actual values of the output from the original training data set, the quadratic modeling technique uses the predicted values of the output from the underlying machine learning model.

In some cases, secondary modeling techniques may use alternative or supplemental training and/or validation data. Such alternatives may include other real-world data from either the same or a different data source, (e.g., via interpolation and/or extrapolation) (e.g., than that present in a real-world sample). real-world data combined with machine-generated data (for purposes of covering a wider range of possibilities), or data generated entirely by machine-based stochastic models.

In many cases, the secondary assisted machine learning model is exactly as accurate as the corresponding primary ML model, or even more accurate than the corresponding primary ML model, and the software implementing the secondary assisted ML model is the corresponding primary ML model. It is much more efficient (e.g., uses less computational resources) than the software implementing it. As discussed throughout this disclosure, secondary assisted machine learning models may be useful for understanding primary machine learning models, and/or may simplify the task of creating software that implements accurate machine learning models. have.

However, despite these advantages of secondary assisted machine learning models, concerns remain that secondary assisted machine learning models may be systematically less accurate for prediction than the primary machine learning models from which they are derived. to reduce any loss of accuracy associated with moving from a primary machine learning model to a secondary secondary machine learning model (and, in some cases, secondary secondary machine learning models having greater accuracy than their corresponding primary model); Some embodiments of a technique for generating

One concern is that when building the secondary assisted machine learning model, any errors in the primary machine learning model may deepen or magnify, thereby systematically reducing the accuracy of the secondary assisted machine learning model. . As discussed below, the empirical results indicate that concerns about the accuracy of secondary assisted machine learning models are largely misguided. Even when these concerns are reasonable, using a more accurate primary machine learning model has the potential to reduce the loss of accuracy in secondary secondary models. For example, because mixed models are sometimes more accurate than any single model, fitting a secondary auxiliary machine learning model to a mix of primary machine learning models will reduce any loss of accuracy associated with quadratic modeling. can

Testing was performed on 381 data sets resulting in 695 classifications and 1849 regression basic machine learning models. For the tested classification models, 43% of the secondary secondary machine learning models were less accurate than the corresponding primary machine learning model, but worse by 10% or less according to the log-loss scale of accuracy. Thirty percent of secondary secondary machine learning models were actually more accurate than primary machine learning models. In only 27% of cases, the secondary assisted machine learning model was more than 10% less accurate than the primary machine learning model. Approximately one third of these cases (approximately 9% of the total population) occurred when the data set was very small. Another third of these cases (again, approximately 9% of the total population) occurred when the primary machine learning model was very accurate at less than 0.1 log loss and the secondary model was still very accurate also at less than 0.1 log loss . Thus, in more than 90% of cases with sufficiently large data sets, the secondary assisted machine learning model was within 10% of the primary machine learning model, or the secondary assisted ML model was very accurate according to absolute criteria. In 41% of cases, the best secondary assisted machine learning model was derived from a mix of primary machine learning models.

For the tested regression model, 39% of the secondary auxiliary machine learning models were less accurate than the corresponding primary machine learning model, however, according to the residual mean squared error measure of accuracy, no more than 10% were more accurate. it got worse 47% of secondary assisted machine learning models were actually more accurate than primary ML models. In only 14% of cases, the secondary assisted machine learning model was more than 10% less accurate than the primary machine learning model. Approximately 10% of these cases (approximately 1.5% of the total population) occurred when the data set was very small. In 35% of all cases, the best secondary assisted machine learning model was derived from a mix of primary machine learning models.

Based on this empirical data, the inventors have recognized and recognized that if the original data set is large enough, the quadratic model is generally relatively accurate or absolutely accurate. In many cases, secondary assisted machine learning models are actually more accurate than primary machine learning models. Finally, in over a third of all classification and regression problems tested, the most accurate secondary secondary machine learning model was derived from a mix of primary machine learning models.

Secondary auxiliary machine learning models are used to (a) understand and/or describe complex underlying machine learning models, (b) simplify the task of generating predictive code for machine learning models, and (c) proprietary It can be useful to protect your model building skills. A blend of basic machine learning models enhances these benefits in many cases. Although blended models produce the best predictive results most of the time, they are also generally more complex because they combine the complexity of all components of all models involved in blending. Also, generating predictive code for a blended model generally combines the challenges of generating predictive code for all component models involved in blending. In addition to these component challenges, blended models are, in general, slower to generate predictions (and/or more requires computational resources). Secondary-assisted machine learning models typically reduce this time (and/or use of computational resources) to the time to compute a single model. In addition, the blended model contains the exclusive input of each component model. Thus, the secondary assisted machine learning model and the mixed machine learning model operate in a highly complementary fashion. Furthermore, since the data set can be easily split into training, validation, and holdout partitions, it is easy to determine whether any particular secondary ML model performs properly compared to the primary ML model. can be decided.

6A is a schematic flowchart diagram illustrating a method 600 for generating a secondary assisted machine learning model, according to one embodiment. In the embodiment depicted in FIG. 6A , method 600 includes steps 610 and 620 . In step 610, a basic machine learning model is obtained. The basic machine learning model is configured to predict values of one or more output variables of the prediction problem based on values of the one or more first input variables. The underlying machine learning model can be any (e.g., executing machine executable modules implementing machine learning modeling procedures, performing embodiments of machine learning modeling methods, mixing two or more machine learning models, etc.) may be obtained using any suitable technique of , and may be any suitable type of machine learning model (eg, a time series model, etc.). In step 620, a quadratic modeling procedure is performed on the underlying machine learning model. Any type of machine learning model (e.g., a RuleFit model, a generalized additive model, any model organized as a set of conditional rules, a mixture of two or more of the above types of models, etc.) is suitable for use as a secondary auxiliary model. may be particularly well suited for

6B is a schematic flowchart diagram illustrating a method 620 for performing a quadratic modeling procedure, according to one embodiment. In the embodiment depicted in FIG. 6B , method 620 includes steps 622 , 624 , 626 , and 628 . In step 622, secondary input data is generated. The secondary input data includes a plurality of secondary observation results. Each secondary observation includes an observed value of one or more second input variables and a value of an output variable predicted by the underlying machine learning model based on a value of the first input variable corresponding to the value of the second input variable. do. Generating the secondary input data may, for each secondary observation result, include: obtaining an observed value of the second input variable and a corresponding observed value of the first input variable; and applying the underlying machine learning model to corresponding observed values of the first input variable to produce predicted values of the output variable.

A primary machine learning model and a secondary secondary machine learning model can use the same set of input variables. Alternatively, the secondary auxiliary machine learning model may use one or more (eg, all) of the input variables of the primary machine learning model, and may also use one or more other input variables. Alternatively, the secondary auxiliary machine learning model may use none of the input variables of the primary machine learning model and may use one or more other input variables.

At step 624 , secondary training data and secondary validation data are generated from the secondary input data (eg, by partitioning the secondary input data).

In step 626, a secondary assisted machine learning model of the base machine learning model is generated by fitting the secondary assisted machine learning model to secondary training data.

At step 628 , the secondary secondary machine learning model of the primary machine learning model is validated against secondary validation data. To fit and/or validate secondary secondary models, cross-validation (including but not limited to nested cross-validation) and holdout techniques may be used.

In some embodiments, primary and secondary machine learning models may be compared. For example, an accuracy score may be determined for each of the models. The accuracy score of each model may indicate the accuracy with which the model predicts the outcome of one or more prediction problems. (In some embodiments, the accuracy score of the secondary assisted machine learning model indicates the accuracy with which the model predicts observations of the output variable, rather than the accuracy with which the model predicts the output variable value predicted by the primary machine learning model. In other embodiments, the accuracy score of the secondary assisted machine learning model represents the accuracy with which the model predicts the output variable value predicted by the primary machine learning model.) In some cases, the accuracy score of the secondary assisted machine learning model exceeds the accuracy score of the underlying machine learning model. As another example, the amount of computational resources used by each of the machine learning models to predict the outcome of one or more prediction problems may be compared. In some cases, the amount of computational resources used by the secondary secondary machine learning model is less than the amount of computational resources used by the primary machine learning model.

In some embodiments, multiple secondary modeling procedures are performed, and multiple secondary secondary machine learning models of the primary machine learning model are generated. In some embodiments, accuracy scores and/or computational resource usage of secondary assisted machine learning models are determined and compared. One or more of the secondary secondary machine learning models may be selected for deployment based at least in part on the accuracy score and/or computational resource usage of the model.

Several embodiments of techniques for tuning a machine learning model to meet explainability criteria have been described. Those skilled in the art will recognize that “explainability” and “interpretability” are closely related terms in the fields of artificial intelligence and machine learning. To the extent that the scope of “interpretability” is understood to depart from the scope of “explainability” as used herein, some embodiments of the techniques described herein are useful for adjusting a machine learning model to meet interpretability criteria. also suitable for

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the present invention. The described embodiments are to be considered in all respects only as illustrative and not restrictive. Accordingly, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All modifications that come within the meaning and scope of equivalence of the claims should be embraced within their scope.

vernacular

The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

As used herein and in the claims, the term “approximately”, the phrase “approximately equal” and other similar phrases (eg, “X has a value of approximately Y” or “X approximately equals Y”) be") should be understood to mean that one value (X) is within a predetermined range of the other value (Y). The predetermined range may be less than plus or minus 20%, 10%, 5%, 1%, 0.1%, or 0.1%, unless otherwise indicated.

The indefinite articles “a” and “an”, as used in the specification and claims, should be understood to mean “at least one” unless explicitly indicated to the contrary. The phrase “and/or”, as used herein and in the claims, means “of an element so combined, that is, an element that is present conjunctively in some cases and disjunctively in other cases. either or both" should be understood. Multiple elements listed with “and/or” should be construed in the same way, ie, “one or more” of the elements so combined. In addition to the elements specifically identified by the "and/or" clause, other elements may optionally be present, whether related or unrelated to those elements specifically identified. Thus, by way of non-limiting example, reference to “A and/or B” when used in connection with an open-ended language such as “comprising,” in one embodiment, A only (optionally including elements other than B); In other embodiments, only B (optionally including elements other than A); In still other embodiments, both A and B (optionally including other elements); can refer to, etc.

As used herein and in the claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating items in a list, "or" or "and/or" is inclusive, i.e., at least one of a number of elements or lists of elements, as well as one or more of them, and optionally , should be construed as the inclusion of additional unlisted items. "only one of" or "exactly one of", or, as used in the claims, such as "consisting of" Only the term shall refer to the inclusion of exactly one element of a plurality of elements or lists of elements. Generally, the term "or," as used, is followed by a term of exclusivity, such as "either", "one of", "only one of", or "exactly one of"; It should only be construed as indicating an exclusive alternative (ie, "one or the other but not both"). “Consisting essentially of”, when used in the claims, should have its general meaning as used in the field of patent law.

As used herein and in the claims, the phrase “at least one” refers to, with respect to a list of one or more elements, at least one selected from any one or more of the elements in that list of elements. It is to be understood as meaning an element, and does not necessarily include at least one of each and every element specifically recited in that list of elements, nor does it exclude any combination of elements in that list of elements. This definition may also optionally exist for elements other than those explicitly identified within the list of elements to which the phrase "at least one" refers, whether or not related to those elements explicitly identified. allow it to be Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently, “at least one of, A and/or B”) is: In one embodiment, B is absent (and optionally includes elements other than B), at least one A, optionally including more than one; In other embodiments, A is absent (and optionally includes elements other than A), at least one B, optionally including more than one; in still other embodiments, at least one A, optionally including more than one, and at least one B, optionally including more than one, (and optionally including other elements); can refer to, etc.

The terms "including," "comprising," "having," "containing," "involving," and variations thereof means "including but not limited to" unless otherwise specified. An enumerated listing of items does not imply that any or all of the items are mutually exclusive and/or mutually inclusive, unless expressly stated otherwise.

The use of ordinal terms such as "first", "second", "third", etc. in a claim to modify a claim element is, by itself, used in any way of one claim element over another. It does not imply priority, precedence, or the temporal order in which acts of the order or method are performed. The ordinal term is merely used as a label to distinguish one claim element with a given name from other elements with the same name (except for the use of the ordinal term) in order to distinguish claim elements. .

Claims (18)

As a device,
a first prediction module, configured to obtain a first set of predictions based on the inference data set from a deployed primary machine learning model;
of a prediction based on the set of inference data from a deployed secondary machine learning model, the secondary machine learning model being different from the primary machine learning model and configured to mimic the predictions of the primary machine learning model. a second prediction module, configured to obtain a second set; and
responsive to a determination that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, wherein the determination is based on a comparison of the first and second sets of predictions. an action module, configured to trigger one or more actions related to the basic machine learning model.
A device comprising a. According to claim 1,
that the primary and secondary machine learning models do not meet the one or more explainability criteria for the primary machine learning model;
determining one or more values of a divergence metric for the first and second sets of predictions based on a difference between the predictions in the first set of predictions and the predictions in the second set of predictions ;
determining that at least one of the one or more values of the divergence metric for the first and second sets of predictions is greater than a divergence threshold; and
Based on a determination that at least one value of the divergence metric for the first and second sets of predictions is greater than the divergence threshold, the primary and secondary machine learning models are the one or more descriptions of the primary machine learning model. Determining that the likelihood criterion is not met
and a comparison module configured to determine by 3. The method of claim 2,
and determining the one or more values of the divergence metric comprises determining, on a prediction unit basis, a respective value of the divergence metric for each prediction of the first and second sets of predictions. 3. The method of claim 2,
and determining the one or more values of the divergence metric comprises determining a respective value of the divergence metric for one or more subsets of the first and second sets of predictions. 3. The method of claim 2,
wherein the one or more values of the divergence metric are one or more deployment values of the divergence metric;
The comparison module is further configured to: (i) a third set of predictions generated by the base machine learning model based on a training data set during development of the base machine learning model; and (ii) the base machine learning model during development of the auxiliary machine learning model. configured to obtain one or more development values of the divergence metric indicative of a difference between a fourth set of predictions generated by the base machine learning model based on a set of training data, and
wherein the comparison module is further configured to compare the one or more distribution values of the divergence metric with the one or more developed values of the divergence metric to detect a deviation between the inference data set and the training data set. Device. According to claim 1,
wherein the one or more actions include sending an alert regarding a violation of the one or more explainability criteria for the underlying machine learning model, the alert further comprising one or more recommendations for responding to the violation. . According to claim 1,
and the one or more actions include swapping the base machine learning model for a different machine learning model. According to claim 1,
and the one or more actions include retraining the base machine learning model. According to claim 1,
and the one or more actions include switching the primary machine learning model to the secondary machine learning model. According to claim 1,
The one or more actions may be performed with a training data set by comparing values of the divergence metrics for third and fourth sets of predictions generated by the primary and secondary machine learning models during development based on the training data set. identifying deviations between the inference data sets, wherein the values of the divergence metrics for the first and second sets of predictions are generated by the primary and secondary machine learning models during deployment based on the inference data sets. The device that becomes. According to claim 1,
and the one or more actions are triggered without receiving confirmation from a user to perform the one or more actions. As a method,
obtaining a first set of predictions based on the set of inference data from the deployed basic machine learning model;
obtaining a second set of predictions based on the set of inference data from a deployed auxiliary machine learning model, the auxiliary machine learning model being different from the underlying machine learning model and configured to mimic predictions of the underlying machine learning model step; and
the primary machine in response to a determination that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, wherein the determination is based on a comparison of the first and second sets of predictions. triggering one or more actions related to the learning model.
A method comprising 13. The method of claim 12,
Determining that the primary and secondary machine learning models do not meet the one or more explainability criteria for the primary machine learning model comprises:
determining one or more values of a divergence metric for the first and second sets of predictions based on a difference between the predictions in the first set of predictions and the predictions in the second set of predictions;
determining that at least one of the one or more values of the divergence metric for the first and second sets of predictions is greater than a divergence threshold; and
Based on a determination that at least one value of the divergence metric for the first and second sets of predictions is greater than the divergence threshold, the primary and secondary machine learning models are one or more explanatory probabilities for the primary machine learning model. Determining that criteria are not met
A method comprising: 14. The method of claim 13,
the one or more values of the divergence metric are one or more distribution values of the divergence metric;
The method comprises (i) a third set of predictions generated by the base machine learning model based on a set of training data during development of the base machine learning model and (ii) the training data during development of the base machine learning model. further comprising obtaining one or more development values of the divergence metric indicative of a difference between a fourth set of predictions generated by the base machine learning model based on the set, and
The method further comprises comparing the one or more distribution values of the divergence metric to the one or more developed values of the divergence metric to detect a deviation between the inference data set and the training data set. , Way. 13. The method of claim 12,
and the one or more actions include exchanging the base machine learning model for a different machine learning model. 13. The method of claim 12,
and the one or more actions include switching the primary machine learning model to the secondary machine learning model. 13. The method of claim 12,
The one or more actions may be performed with a training data set by comparing values of the divergence metrics for third and fourth sets of predictions generated by the primary and secondary machine learning models during development based on the training data set. identifying deviations between the inference data sets, wherein the values of the divergence metrics for the first and second sets of predictions are generated by the primary and secondary machine learning models during deployment based on the inference data sets. How to be. As a device,
means for obtaining a first set of predictions based on the set of inference data from the deployed basic machine learning model;
obtaining a second set of predictions based on the set of inference data from a deployed auxiliary machine learning model, the auxiliary machine learning model being different from the underlying machine learning model and configured to mimic predictions of the underlying machine learning model means for; and
the primary machine in response to a determination that the primary and secondary machine learning models do not meet one or more explainability criteria for the primary machine learning model, wherein the determination is based on a comparison of the first and second sets of predictions. means for triggering one or more actions related to the learning model
A device comprising a.

Images