WO2023024578A1

WO2023024578A1 - Method and apparatus for configuring decision apparatus, and related device

Info

Publication number: WO2023024578A1
Application number: PCT/CN2022/091969
Authority: WO
Inventors: 李治军; 张庭豪; 李涛; 谢达奇
Original assignee: 华为云计算技术有限公司
Priority date: 2021-08-25
Filing date: 2022-05-10
Publication date: 2023-03-02
Also published as: CN115730205A

Abstract

The present application provides a method for configuring a decision apparatus. A first inference result corresponding to a first type of samples and a second inference result corresponding to a second type of samples are first acquired, and the precision of inferring the first type of samples by using a first inference model is lower than the precision of inferring the second type of samples by using the first inference model; decision parameters in a decision apparatus are configured according to the first inference result and the second inference result. In this way, on the basis of the configured decision parameters, the decision apparatus generally can accurately recognize the first type of samples that is difficult to be accurately inferred by using the first inference model having a smaller specification, and transmit the first type of samples to a second device set, so as to perform inference by using the second inference model having a larger specification, so that the precision of inferring a model input sample can be maintained at a high level. In addition, the present application further provides a corresponding apparatus and a related device.

Description

A method, device and related equipment for configuring a decision-making device

This application claims the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on August 25, 2021, the application number is 202110981922.5, and the invention title is "a method, device and related equipment for configuring a decision-making device", the entire content of which Incorporated in this application by reference.

technical field

The present application relates to the technical field of artificial intelligence, and in particular to a method, device and related equipment for configuring a decision-making device.

Background technique

In the field of artificial intelligence (AI), machine learning technology is an important method and means in the field of AI. Sample data for inference.

At present, a two-level inference mechanism can be set according to the amount of resources in the environment where the inference model is deployed. For example, in an edge-cloud collaborative inference scenario, inference models of different specifications can be set on the edge side and on the cloud, and since the computing resources on the edge side are usually less than those on the cloud, the reasoning models deployed on the edge side The model may be a model with a smaller size obtained by compressing the inference model in the cloud. Correspondingly, for the same model input sample, the inference effect (such as inference accuracy, efficiency, etc.) of the inference model on the cloud for the input sample of the model is usually better than that of the inference model on the edge side for the input sample of the model. At the same time, a decision-making device is deployed on the edge side, and the decision-making device can determine to send the model input sample to the cloud when the reasoning model on the edge side is difficult to infer the validity of the model input sample (such as the confidence of the reasoning result is low, etc.). , in order to use the inference model with a larger specification in the cloud to infer the input samples of the model, so as to improve the accuracy of the final inference result.

However, in practical applications, it may be difficult to maintain the accuracy of the inference system at a high level. For example, in a certain period of time, the accuracy of the inference results determined by the inference system for the model input samples is low. Therefore, there is an urgent need for an inference scheme so that the accuracy of the inference system inference model input samples can be kept at a high level.

Contents of the invention

The present application provides a method for configuring a decision-making device, which is used to keep the reasoning accuracy of the reasoning system at a high level for model input samples. In addition, the present application also provides a device for configuring a decision-making device, a computer device, a computer-readable storage medium, and a computer program product.

In a first aspect, the present application provides a method for configuring a decision-making device, the method is applied to an inference system including a first device set, a second device set, and a decision-making device, wherein the first device set and the second device set both include At least one computing device, and the size of the first inference model in the first set of devices is smaller than the size of the second inference model in the second set of devices. When executing the method, the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample are obtained first, and the accuracy of the first inference model inference of the first type of sample is lower than that of the first inference model inference of the first inference result. The accuracy of the second type of samples, in actual application, the first type of samples can be called difficult samples, and the second type of samples can be called simple samples; and according to the first reasoning result and the second reasoning result, configure the decision-making device A decision parameter for identifying a model input sample inferred by the first inference model as a model input sample transmitted to the second set of devices.

Since the decision parameters of the decision-making device are configured through the inference results of the first type of samples that the first inference model can infer accurately and the inference results of the second type of samples that the first inference model is difficult to infer accurately, the decision-making device is based on the decision The parameters are usually able to accurately identify the second class of samples that the first inference model has difficulty inferring accurately. In this way, the inference system can use the second inference model with a larger specification to infer this type of sample, so that the accuracy of the inference model input samples of the inference system can be maintained at a relatively high level.

In a possible implementation manner, when obtaining the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, it may specifically instruct the first device set to use the first inference model to respectively The samples of one type and the samples of the second type are inferred to obtain a first inference result corresponding to the sample of the first type and a second inference result corresponding to the sample of the second type. In this way, the first inference model can be used to infer the two types of samples respectively to obtain the inference results corresponding to the two types of samples, so that the decision parameters in the decision-making device can be configured subsequently based on the inference results.

In a possible implementation manner, when obtaining the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, it may specifically instruct the first device set to use the first inference model to Perform inference on one type of sample to obtain the first inference result corresponding to the first type of sample; at the same time, instruct the second device set to use the second inference model to perform inference on the second type of sample to obtain the second inference result corresponding to the second type of sample . In this way, the inference results corresponding to the two types of samples can be obtained through the first inference model and the second inference model, so as to subsequently configure the decision parameters in the decision-making device based on the inference results.

In a possible implementation manner, before obtaining the inference results, multiple samples may be obtained first, and the inference results corresponding to the multiple samples may be further obtained, and the inference results corresponding to the multiple samples are respectively analyzed by the first inference model Multiple samples are obtained by reasoning; then, according to the reasoning results corresponding to the multiple samples, the samples of the first type and the samples of the second type among the multiple samples can be determined. In this way, according to the inference results of the first inference model on the multiple samples, the first type of samples that can be more accurately identified by the first inference model and the second type of samples that are difficult to accurately identify by the first inference model can be determined from the multiple samples.

In a possible implementation manner, when determining the samples of the first type and the samples of the second type among the multiple samples according to the inference results corresponding to the multiple samples, the labeling interface may be presented first, and the labeling interface includes multiple samples The corresponding inference results, so that the first type of samples and the second type of samples among the multiple samples can be determined according to the labeling operation of the labeling personnel on the inference results corresponding to the multiple samples. In this way, the first type of samples and the second type of samples can be determined from multiple samples through the manual annotation results of the annotators, so as to improve the accuracy of determining the first type of samples and the second type of samples.

In a possible implementation manner, before obtaining the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, the first inference model may be configured for the first device set, and the first inference model may be configured for the second type of sample. The second device set configures the second reasoning model, so that the configured first reasoning model and/or the second reasoning model can be used to determine decision parameters later, wherein the first reasoning model is obtained by performing model compression on the second reasoning model. For example, a structured search of the second reasoning model can be performed first through a reinforcement learning algorithm to determine the network structure of the first reasoning model; and then the network parameters of the first reasoning model can be determined by performing knowledge distillation on the second reasoning model, In this way, the first reasoning model is obtained.

In a possible implementation, the inference system can be deployed in a device-edge collaborative manner, that is, the first device set is deployed on the local network, and the second device set is deployed on the edge network; or, the inference system can be deployed in an edge-cloud collaborative manner. Deployment in a different way, that is, the first set of devices is deployed on the edge network, and the second set of devices is deployed on the cloud.

In a second aspect, the present application provides a method for configuring a decision-making device. The method is applied to an inference system including a first device set, a second device set, and a decision-making device. The first device set and the second device set each include at least one Computing devices, and the specification of the first inference model in the first set of devices is smaller than the specification of the second inference model in the second set of devices. When executing the method, the first type of samples and the second type of samples can be obtained first, the accuracy of the first inference model inferring the first type of samples is lower than the accuracy of the first inference model inference of the second type of samples; then, according to the first type The sample and the second type of sample configure the decision parameter in the decision device, and the decision parameter is used to identify the model input sample inferred by the first inference model as the model input sample transmitted to the second device set.

Since the decision parameters of the decision-making device are configured through the first type of samples that the first reasoning model can reason about accurately and the second type of samples that the first reasoning model is difficult to reason about accurately, the decision-making device can usually accurately identify The first type of samples that the first inference model is difficult to infer accurately. In this way, the inference system can use the second inference model with a larger specification to infer this type of sample, so that the accuracy of the inference model input samples of the inference system can be maintained at a relatively high level.

In a third aspect, the present application provides a configuration device, and the configuration device includes various modules for implementing the method of the configuration decision device in the first aspect.

In a fourth aspect, the present application provides a configuration device. The configuration device is applied to an inference system. The reasoning system includes a first device set, a second device set, and a decision-making device. The first device set and the second device set The device sets each include at least one computing device, the specification of the first reasoning model in the first device set is smaller than the specification of the second reasoning model in the second device set, and the configuration device includes: a sample acquisition module, configured to acquire The first type of sample and the second type of sample, the accuracy of the first inference model inferring the first type of sample is lower than the accuracy of the first inference model inference of the second type of sample; the configuration module is used to according to the The first type of samples and the second type of samples are configured, and the decision parameters in the decision-making device are configured, and the decision parameters are used to identify the model input samples inferred by the first inference model as being transmitted to the second device A collection of model input samples.

In a fifth aspect, the present application provides a computer device, the computer device includes a processor and a memory; the memory is used to store instructions, and when the computer device is running, the processor executes the instructions stored in the memory, so that the The computer device executes the first aspect above or the method for configuring the decision-making apparatus in any possible implementation manner of the first aspect. It should be noted that the memory may be integrated in the processor, or independent of the processor. A computer device may also include a bus. Wherein, the processor is connected to the memory through the bus. Wherein, the memory may include a readable memory and a random access memory.

In a sixth aspect, the present application provides a computer device, the computer device includes a processor and a memory; the memory is used to store instructions, and when the computer device is running, the processor executes the instructions stored in the memory, so that the The computer equipment executes the method of configuring the decision-making device in the second aspect above. It should be noted that the memory may be integrated in the processor, or independent of the processor. A computer device may also include a bus. Wherein, the processor is connected to the memory through the bus. Wherein, the memory may include a readable memory and a random access memory.

In a seventh aspect, the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer device, the computer device executes the above-mentioned first aspect or any of the first aspects. A method described in an implementation.

In an eighth aspect, the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer device, the computer device executes the method described in the second aspect above.

In a ninth aspect, the present application provides a computer program product containing instructions, which, when run on a computer device, causes the computer device to execute the method described in the first aspect or any implementation manner of the first aspect.

In a tenth aspect, the present application provides a computer program product containing instructions, which, when run on a computer device, causes the computer device to execute the method described in the second aspect above.

On the basis of the implementation manners provided in the foregoing aspects, the present application may further be combined to provide more implementation manners.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some implementations recorded in the application. For example, those skilled in the art can also obtain other drawings based on these drawings.

FIG. 1 is a schematic diagram of the architecture of an inference system provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of another reasoning system provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of a method for configuring a decision-making device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an exemplary labeling interface provided by the embodiment of the present application;

FIG. 5 is a schematic structural diagram of a computer device 500 provided in an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a computer device 600 provided by an embodiment of the present application.

Detailed ways

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is merely a description of the manner in which objects with the same attribute are described in the embodiments of the present application.

Referring to FIG. 1 , it is a schematic diagram of an architecture of an inference system. As shown in FIG. 1 , the reasoning system 100 includes a first device set 101 , a second device set 102 and a decision-making device 103 . Wherein, both the first device set 101 and the second device set 102 include at least one computing device. In FIG. 1 , it is taken that the first device set 101 and the second device set 102 respectively include multiple servers as an example. In practical applications, the computing devices constituting the first device set 101 and the second device set 102 may also be other devices with computing capabilities, and are not limited to the servers shown in FIG. 1 . The first device set 101 and the second device set 102 may be deployed in different environments. Exemplarily, as shown in FIG. 1 , the first set of devices 101 can be deployed on the edge network to execute corresponding calculation processes on the edge side, such as the following inference process based on the first inference model; the second set of devices 102 can be It is deployed on the cloud, and is used to execute a corresponding calculation process on the cloud, such as the following reasoning process based on the second reasoning model. In other examples, the first set of devices 101 may be deployed on a local network on the user side, such as a local terminal or server; the second set of devices 102 may be deployed on an edge network. In this embodiment, specific deployment manners of the first device set 101 and the second device set 102 are not limited.

The decision-making device 103 can be deployed in the same environment as the first device set 101. For example, the decision-making device 103 can be deployed with the first device set 101 on the edge side network as shown in the figure, or can also be deployed with the first device set 101 Deployed on a local network, etc. Wherein, the decision-making device 103 may be implemented by software or hardware. When implemented by software, the decision-making device 103 may be an application program applied to a computing device, and the computing device is deployed in the same environment as the first device set 101 . When implemented by hardware, the decision-making device 103 may be a computing device located in the same environment as the first device set 101, such as a server; or, the decision-making device 103 may be realized by using an application-specific integrated circuit (ASIC), or Devices implemented by programmable logic devices (programmable logic device, PLD), etc. Wherein, the above-mentioned PLD can be implemented by complex programmable logic device (complex programmable logical device, CPLD), field-programmable gate array (field-programmable gate array, FPGA), general array logic (generic array logic, GAL) or any combination thereof.

Wherein, inference models can be configured in both the first device set 101 and the second device set 102. For the convenience of description, the reasoning model configured in the first device set 101 will be referred to as the first reasoning model below, and the configuration in The reasoning models in the second device set 102 are called the second reasoning models. In an actual application scenario, based on the difference in physical resources in the deployment environments of the first device set 101 and the second device set 102, the specification of the first inference model is different from that of the second inference model. For example, when the first set of devices is deployed in When the edge network and the second set of devices are deployed on the cloud, the size of the first inference model is 400KB (kilobytes), and the size of the second reasoning model is 40000KB. In this application, the specification of the first reasoning model is smaller than the specification of the second reasoning model as an example for illustration.

When the inference system 100 infers model input samples, as shown in FIG. 1 , the first set of devices 101 can receive the model input samples sent by the terminal device 104 on the user side, and the model input samples can be taken by the terminal device 104 (or through other equipment shooting) obtained images, etc. Then, the first set of devices 101 may use the preconfigured first inference model to perform inference on the acquired model input samples, and obtain an inference result. Then, the decision-making device 103 can determine whether the model input sample is a first-type sample based on the pre-configured decision parameters and the inference result output by the first inference model, that is, determine whether the inference performed by the first inference model on the model input sample is accurate. . When it is determined that the model input sample is the first type of sample, the characterization decision-making unit 103 determines that the reasoning of the model input sample by the first reasoning model is inaccurate, and the decision-making unit 103 may instruct the first device set 101 to send the model input sample to The second set of devices 102, so that the second set of devices 102 uses a second reasoning model with a larger specification to perform accurate reasoning on the model input samples and give feedback. And when it is determined that the model input sample is a sample of the second type, the characterization decision-making means 103 determines that the first inference model can accurately infer the model input sample, then the decision-making means 103 can instruct the first device set 101 to directly feed back the model to the terminal device 104 The inference result corresponding to the input sample. In this way, the inference accuracy of the inference system 100 for the model input samples can reach a higher level.

However, in actual application scenarios, the decision parameters in the decision-making device 103 are usually manually set by technicians based on experience, which makes the decision-making device 103 determine whether the model input samples are the first-type samples based on the manually-set decision parameters. , the judgment accuracy is low. In this way, the model input samples that actually belong to the first type of samples are misidentified as the second type of samples due to the erroneous determination of the decision-making device 103, so that the model input samples are not transmitted to the second device set 102 for reasoning. At the same time, the inference accuracy of the first inference model for the first type of samples is relatively low. In this way, the inference accuracy of the inference system 100 for model input samples is reduced. However, if all the model input samples are transmitted to the second device set 102, and the second inference model in the second device set 102 is used for inference, the space between the first device set 101 and the second device set 102 will be occupied. Mass transfer bandwidth.

Based on this, the embodiment of the present application provides a method for configuring a decision-making device to improve the accuracy of the decision-making device 103 in determining the first type of samples, thereby improving the reasoning accuracy of the reasoning system 100 for model input samples. The method for configuring a decision-making device can be applied to the reasoning system 200 shown in FIG. 2 . On the basis of the reasoning system 100 shown in FIG. 1 , a configuration device 105 is added to the reasoning system 200 shown in FIG. 2 , and the configuration device 105 can be used to configure decision parameters in the decision device 103 . During specific implementation, the configuration device 105 first obtains the first inference result corresponding to the first type of sample that the first inference model is difficult to accurately identify, and the second inference result corresponding to the second type of sample that the first inference model can accurately identify, that is, the first The accuracy of the inference model for inferring the first type of samples is lower than the accuracy of the first inference model for inferring the second type of samples. Then, the configuration means 105 configures the decision parameters in the decision means 103 according to the first reasoning result and the second reasoning result. Since the decision parameters of the decision-making device 103 are configured through the inference results corresponding to the first-type samples and the inference results corresponding to the second-type samples respectively corresponding to the first inference model, the decision-making device 103 can usually accurately identify The first type of samples that the first inference model is difficult to infer accurately. In this way, the inference system 100 can use the second inference model with a larger specification to perform inference on the first type of samples, thereby improving the accuracy of the inference system 100 inference model input samples.

Wherein, the configuration apparatus 105 may be deployed in the same environment as the first device set 101 , or may be deployed in the same environment as the second device set 102 . Moreover, the configuring device 105 may be realized by means of software or hardware. When implemented by software, the configuration means 105 may be an application program applied to a computing device in the reasoning system 200, such as a program applied to any computing device in the first device set 101, or applied to the second device set A program on any computing device in 102, or a program applied to a computing device deployed separately in the reasoning system 200, etc. When implemented by hardware, the configuration device 105 may be a computing device deployed separately in the inference system 200, such as a server with a configuration function.

It should be noted that the reasoning system shown in FIG. 2 is only used as an exemplary description, and is not used to limit the specific implementation of the reasoning system. For example, in other possible implementations, the inference system 200 may include more functional modules to support the inference system to have more other functions; or, when the first device set 101 in the inference system 200 is deployed on the local In network, the devices in the first device set 101 may specifically be terminal devices 104 and the like.

For ease of understanding, embodiments of the configuration decision-making device provided by the present application are described below with reference to the accompanying drawings.

Referring to FIG. 3 , FIG. 3 is a schematic flowchart of a method for configuring a decision-making device according to an embodiment of the present application. Wherein, the reasoning method shown in FIG. 3 can be applied to the reasoning system 200 shown in FIG. 2 , or to other applicable reasoning systems. For ease of description, in this embodiment, it is applied to the inference system 200 shown in FIG. 2 and executed by the configuration device 105 in the inference system 200 as an example for illustration.

Based on the reasoning system 200 shown in FIG. 2, the method for configuring the decision-making device shown in FIG. 3 may specifically include:

S301: The configuration module 105 configures a first inference model and a second inference model in the first device set 101 and the second device set 102 respectively, wherein a specification of the first inference model is smaller than a specification of the second inference model.

In this embodiment, the inference system 200 may use a two-stage inference mechanism to infer model input samples. During specific implementation, the inference system 200 may preferentially use the first inference model deployed in the first device set of the edge network to perform inference on model input samples. If the reasoning result of the first reasoning model is relatively accurate, the reasoning system 100 may use the reasoning result output by the first reasoning model as the reasoning result fed back to the terminal device 104 . And if the inference result of the first inference model is inaccurate, the first set of devices 101 can transmit the model input samples to the second set of devices 102 deployed in the cloud, so that the second model with a larger specification in the cloud can be used to reason about the model. The input samples are used for precise reasoning, so the reasoning result fed back by the reasoning system 200 to the terminal device 104 is the reasoning result output by the second reasoning model. Wherein, whether to transmit the model input samples to the second device set 102 may be determined by the decision-making unit 103 .

In a possible implementation manner, the second inference model in the second device set 102 may complete the model building and training process in advance with the intervention of technicians, so that the second inference model can achieve higher inference accuracy. Moreover, after the second reasoning model is generated, the configuration module 1052 in the configuring device 105 can configure it in the second device set 102 . When generating the first inference model, the configuration module 1052 can generate the first inference model with a smaller size based on the second inference model by means of model compression. For example, the configuration module 1052 can perform a structural search on the second reasoning model based on a reinforcement learning (RL) algorithm to determine the network structure of the first reasoning model; then, the configuration module 1052 can perform knowledge distillation on the existing The first reasoning model that determines the network structure is trained to determine network parameters in the first reasoning model. At this time, the first reasoning model can be called a student model, and the second reasoning model can be called a teacher model. Since the specific implementation process of generating the student model based on the teacher model has already been applied in related technologies, details will not be described here. Alternatively, the configuration module 1052 may also construct and train the first inference model in a manner similar to generating the second inference model. Then, the configuration module 1052 can deploy the generated first reasoning model in the first device set 101 .

S302: The configuration device 105 obtains the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, wherein the accuracy of the first inference model inferring the first type of sample is lower than that of the first inference model inferring the second The precision of class samples.

Wherein, the first type of samples refers to samples that are difficult to be accurately inferred by the first inference model. In practical applications, such samples may also be referred to as difficult samples corresponding to the first inference model. For example, when the confidence of the inference result obtained by the first inference model inference on the sample is less than the first preset value, the sample may be determined as the first type of sample (ie, a difficult sample). Correspondingly, the second type of samples refers to samples that can be inferred relatively accurately by the first reasoning model. In practical applications, such samples can also be called simple samples corresponding to the first reasoning model. For example, a sample whose confidence degree of the inference result is greater than a second preset value may be determined as the second type of sample (that is, a simple example sample), etc., and the second preset value is greater than the aforementioned first preset value. In this embodiment, the configuration device 105 can configure the decision-making device 103 by acquiring the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, so that the configured decision-making device 103 can identify The first type of samples that the first reasoning model is difficult to accurately reason about and the second type of samples that the first reasoning model can accurately identify.

In a possible implementation manner, as shown in FIG. 2 , the configuration device 105 may include an inference result acquisition module 1051 and a configuration module 1052 . Wherein, the inference result obtaining module 1051 may obtain multiple samples, and the multiple samples may be provided by technicians, for example. Then, the inference result acquisition module 1051 may send the multiple samples to the first device set 101 . At the same time, the configuration module 1052 may instruct the first set of devices 101 to use the first inference model to perform inference on each sample in the plurality of samples, and obtain an inference result corresponding to each sample. In this way, the inference result acquisition module 1051 can further determine the first type of samples and the second type of samples in the multiple samples relative to the first inference model according to the inference results corresponding to the multiple samples, for example, the inference results in the multiple samples can be The samples whose confidence degree is less than the first preset value are determined as the first type of samples, and the samples whose inference result confidence degree is greater than the second preset value among the plurality of samples are determined as the second type of samples, and the second preset value not less than the first preset value. In this way, the inference result acquisition module 1051 can determine the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample from the inference results corresponding to the plurality of samples.

As an implementation example of determining the samples of the first type, the inference result obtaining module 1051 may specifically determine the samples of the first type according to the manually marked results. Specifically, the inference result acquisition module 1051 may present a labeling interface to the labeler, for example, the labeling interface shown in FIG. 4 may be presented, and the labeling interface may include inference results corresponding to multiple samples of the first inference model. In this way, the annotator can mark the reasoning results corresponding to each sample on the labeling interface for correct reasoning and wrong reasoning, so that the inference result acquisition module 1051 can determine the first type of samples from multiple samples according to the labeling operation of the labeler As well as the second type of samples, for example, the inference result acquisition module 1051 may determine the samples corresponding to the inference results labeled "correct inference" as the second type of samples, and determine the samples corresponding to the inference results labeled "inference error" as the second type of samples. A class of samples, etc.

It is worth noting that the number of samples of the second type that can be accurately identified by the first inference model and the number of samples of the first type that are difficult to accurately identify may vary greatly. For example, in actual application scenarios, the number of samples of the first type may be much smaller than that of the first type. The number of samples of the second class. Therefore, in some possible implementations, the inference result acquisition module 1051 can obtain a corresponding number of first-class samples by means of query-by-committee (QBC) based on the determined first-type samples or second-type samples. class samples and the second class samples. Alternatively, the inference result acquisition module 1051 may also obtain a comparable number of first-type samples and second-type samples from the determined first-type samples or second-type samples by means of sampling with replacement. In this embodiment, there is no limitation on the specific implementation manner of obtaining a sufficient number of samples of the first type and samples of the second type by the inference result obtaining module 1051 .

S303: The configuration device 105 configures the decision parameters in the decision device 103 according to the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, and the decision parameter is used to infer the first reasoning model to the model The input samples are identified as model input samples transmitted to the second set of devices 102 .

As an example, the decision parameter in the decision-making device 103 may be, for example, a confidence threshold, so that the decision-making device 103 can determine whether the model input sample corresponding to the reasoning result is the first by comparing the confidence threshold with the confidence of the reasoning result. A first type of sample that is difficult for the inference model to infer accurately. Specifically, when the confidence of the inference result output by the first inference model is less than the confidence threshold, the decision-making device 103 may determine that the model input sample corresponding to the inference result is a first-type sample, and may further instruct the first device set 101 to The model input samples are transmitted to the second set of devices 102 . And when the confidence of the inference result output by the first inference model is greater than the confidence threshold, the decision-making device 103 may determine that the model input sample corresponding to the inference result is the second type of sample that the first inference model can accurately identify, so that the decision-making device 103 may instruct the first set of devices 101 to feed back the inference result of the first inference model for the model input sample to the terminal device 104 .

In another example, the decision parameters in the decision device 103 may be, for example, network parameters in a neural network model. During specific implementation, the decision-making device 103 may include a neural network model, the input of the neural network model is the model input sample or the inference result output by the first reasoning model for the model input sample, and the output of the neural network model is the model input sample The decision parameters determined by the configuration device 105 are the network parameters in the neural network model for the determination result of the first type of sample or the second type of sample.

Correspondingly, when the configuring device 105 determines the decision-making parameters in the decision-making device 103 , it may specifically use the obtained samples of the first type and samples of the second type to determine.

In a possible implementation manner, the configuration module 1052 may instruct the first set of devices 101 to use the first inference model to perform inference on the first type of samples and the second type of samples, and obtain the first inference results corresponding to the first type of samples and The second type of sample corresponds to the second inference result, so the configuration module 1052 can determine the decision parameters in the decision-making device 103 according to the first inference result and the second inference result.

For example, when the decision parameter is specifically a confidence threshold, the first inference result corresponding to the first type of sample may include a first confidence level, and the first confidence level may be, for example, the inference result corresponding to a plurality of first type samples respectively. The average value of the confidence degree, etc., the second inference result corresponding to the second type of sample may include the second confidence degree, for example, the second confidence degree may be the average value of the confidence degree of the inference result corresponding to the plurality of second type samples respectively etc., so that the configuration module 1052 can determine the value of the confidence threshold of the decision parameter according to the values of the first confidence degree and the second confidence degree, and configure the decision parameter for the decision-making device 103 based on the determined value.

For another example, when the decision parameters are specifically network parameters in the neural network model, the configuration module 1052 may use the first inference results corresponding to some samples of the first type and the second inference results corresponding to some samples of the second type as the decision-making device 103 The input of the neural network model in , the labeling result of whether the sample belongs to the first type of sample or the second type of sample is used as the output of the neural network model, so as to train the network parameters in the neural network model. Then, the configuration module 1052 can use the first inference results corresponding to the remaining samples of the first type and the second inference results corresponding to the remaining samples of the second type as a test set to test the trained neural network model to test The classification accuracy of the neural network model for the first type of samples and the second type of samples, for example, the neural network model may be a binary classification model. When the neural network model passes the test, the network parameters in the neural network model are the decision parameters finally determined by the configuration module 1052 .

In the above embodiment, the first reasoning result and the second reasoning result are the reasoning results output by the first reasoning model, and in another possible embodiment, the first reasoning result and the second reasoning result are output by different reasoning models output inference results. Specifically, the configuration module 1052 may instruct the first set of devices 101 to use the first inference model to perform inference on samples of the second type to obtain a second inference result corresponding to the samples of the second type. At the same time, the configuration module 1052 can also instruct the second set of devices 102 to use the second inference model to perform inference on the first type of sample to obtain the first inference result corresponding to the first type of sample, so that the configuration module 1052 can use the first inference result and The second reasoning result is to determine the decision parameters in the decision device 103 . Wherein, for the specific process of the configuration module 1052 determining the decision parameters according to the first reasoning result and the second reasoning result, please refer to the description of the specific process of determining the decision parameters according to the first reasoning result and the second reasoning result in the foregoing embodiment, which will not be described here. repeat.

It is worth noting that the specific implementation methods for determining the decision-making parameters above are only used as some exemplary illustrations. In practical applications, the decision-making parameters can also be other types of parameters, so that the configuration module 1052 can determine the decision-making parameters through other possible implementation methods. The embodiment does not limit this.

In this way, after the decision parameter is determined, the decision-making device 103 can analyze whether each model input sample subsequently inferred by the first inference model belongs to the first type of sample according to the decision parameter, so that when it is determined that it belongs to the first type of sample, indicate the first A set of devices 101 sends the first-type samples to the second set of devices 102 for inference, so as to improve the inference accuracy for the first-type samples, so that the inference accuracy of the inference system 200 for model input samples can be maintained at a high level level. At the same time, for the second type of samples, accurate inference can be completed by using the first inference model in the first device set 101, so that it is not necessary to transmit them to the second device set 102, thereby reducing the number of connections between the first device set 101 and the second set of devices. Resource consumption of the transmission bandwidth between the two device sets 102 .

It should be noted that in this embodiment, the configuration device 105 determines the decision parameters according to the inference results corresponding to the two types of samples as an example for illustration. In other possible embodiments, the configuration device 105 may also directly Class samples determine the decision parameters. For example, in a possible implementation, when the decision-making parameters are specifically network parameters in the neural network model, the configuration module 1052 in the configuration device 105 can use some samples of the first type and some samples of the second type as the decision-making device The input of the neural network model in 103 is to use the labeling result of whether the sample belongs to the first type of sample or the second type of sample as the output of the neural network model, so as to train the network parameters in the neural network model. Then, the configuration module 1052 can use the remaining part of the first-type samples and the remaining part of the second-type samples as a test set to test the trained neural network model, so as to test that the neural network model recognizes the first type and the second type accuracy. For example, the neural network model can perform difference analysis with the sample features (such as image features, etc.) in the first type of samples and the second type of samples according to the characteristics of the samples in the test set to determine that the samples in the test set belong to the first type of samples Or the second type of samples. In this way, when the neural network model passes the test, the network parameters in the neural network model are the decision parameters finally determined by the configuration module 1052 .

In this embodiment, by testing on public data sets such as MNIST, SVHN, and CIFAR-10, it can be determined that after the decision-making device 103 uses the decision parameters to judge the first type of sample or the second type of sample on the model input sample, it can be Make the reasoning accuracy of the reasoning system 200 reach a higher level (approximate to the accuracy of reasoning using the second reasoning model), as shown in Table 1 below:

Table 1

Specifically, after the decision parameters are determined in this embodiment, the following steps of reasoning the model input samples may also be included:

S304: The first device set 101 receives a model input sample.

For example, the terminal device 104 on the user side may send a model input sample to the first set of devices 101. The model input sample may be, for example, a captured image, such as a captured image of a construction site in a helmet scene, or other A sample as input to the first inference model.

S305: The first set of devices 101 uses the pre-configured first inference model with a smaller specification to perform inference on the model input samples to obtain an inference result.

S306: The decision-making device 103 determines whether the model input sample is a first-type sample according to the determined decision parameters and the inference result of the first inference model for the model input sample.

For example, assuming that the decision-making parameter is specifically a confidence threshold, if the confidence of the first inference model’s inference result for the input sample of the model is greater than the confidence threshold, it means that the inference result output by the first inference model is correct and credible A higher degree means that it can be regarded as a higher accuracy of the inference result. At this time, the decision-making device 103 may determine that the model input sample is a second-type sample, and may further instruct the first set of devices 101 to feed back the inference result output by the first inference model to the terminal device 104 .

Conversely, if the confidence of the inference result of the first inference model for the model input sample is less than the confidence threshold, it may be considered that the inference result output by the first inference model is inaccurate. At this time, the decision-making unit 103 can determine that the model input sample is a first-type sample, and can further instruct the first device set 101 to send the model input sample to the second device set 102, so as to utilize the specification on the second device set 102 The larger second inference model makes more accurate inferences on the model input samples.

It should be noted that when the decision-making parameters in the decision-making device 103 are determined according to the samples of the first type and the samples of the second type, in other possible embodiments, the decision-making device 103 determines the current The inferred model input sample is used to determine whether the model input sample is a first-type sample.

S307: The decision-making unit 103 instructs the first set of devices 101 to upload the model input sample to the second set of devices 102 when determining that the model input sample is a first-type sample.

S308: The first set of devices 101 sends the model input samples to the second set of devices 102 .

S309: The second set of devices 102 performs inference on the received model input samples by using a pre-configured second inference model with a larger specification to obtain an inference result.

S310: The second set of devices 102 sends the inference result to the terminal device 104.

It should be noted that this embodiment uses the first set of devices 101 deployed on the edge network and the second set of devices 102 deployed on the cloud as an example for illustration. In other implementations, the first set of devices 101 may also be deployed on The local network, while the second set of devices 102 is deployed on the edge network. At this time, the inference process of the inference system 200 on the input samples of the model and the process of updating the confidence threshold and the model are similar to the above-mentioned process. For details, please refer to the related described here, and will not be repeated here.

In the above-mentioned embodiments, the configuration device 105 involved in the process of configuring the decision-making device may be implemented as a separate hardware device, and in other possible implementation manners, it may also be software configured on a computer device, and, By running the software on the computer equipment, the computer equipment can respectively realize the functions of the configuration device 105 described above. In the following, based on the perspective of hardware device implementation, the configuring device 105 involved in the process of configuring the decision-making device will be introduced in detail respectively.

Figure 5 shows a computer device. The computer device 500 shown in FIG. 5 can be specifically used to implement the functions of the configuration apparatus 105 in the above-mentioned embodiment shown in FIG. 3 .

The computer device 500 includes a bus 501 , a processor 502 , a communication interface 503 and a memory 504 . The processor 502 , the memory 504 and the communication interface 503 communicate through the bus 501 . The bus 501 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 5 , but it does not mean that there is only one bus or one type of bus. The communication interface 503 is used for communicating with the outside, for example, instructing the first set of devices to use the first reasoning model to perform reasoning and the like.

Wherein, the processor 502 may be a central processing unit (central processing unit, CPU). The memory 504 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM). The memory 504 may also include a non-volatile memory (non-volatile memory), such as a read-only memory (read-only memory, ROM), flash memory, HDD or SSD.

Executable codes are stored in the memory 504 , and the processor 502 executes the executable codes to execute the method executed by the aforementioned configuration device 105 .

Specifically, in the case of implementing the embodiment shown in FIG. 3, and the configuration device 105 described in the embodiment shown in FIG. Software or program codes are stored in the memory 504 , the interaction between the configuration device 105 and other devices is realized through the communication interface 503 , and the processor is used to execute the instructions in the memory 504 to realize the method executed by the configuration device 105 .

FIG. 6 shows another computing device. The computer device 600 shown in FIG. 6 includes a bus 601 , a processor 602 , a communication interface 603 and a memory 604 . The processor 602 , the memory 604 and the communication interface 603 communicate through the bus 601 . The bus 601 can be a PCI bus or an EISA bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 6 , but it does not mean that there is only one bus or one type of bus. The communication interface 603 is used for communicating with the outside, for example, instructing the first set of devices to use the first reasoning model to perform reasoning and the like.

Wherein, the processor 602 may be a CPU. Memory 604 may include volatile memory, such as RAM. The memory 604 may also include non-volatile memory (non-volatile memory), such as ROM, flash memory, HDD or SSD.

Executable codes are stored in the memory 604, and the processor 602 executes the executable codes to perform the following steps:

Acquiring samples of the first type and samples of the second type, the accuracy of inferring the samples of the first type by the first inference model is lower than the accuracy of inferring the samples of the second type by the first inference model;

According to the first type of samples and the second type of samples, configure the decision parameters in the decision-making device, the decision parameters are used to identify the model input samples inferred by the first inference model as being transmitted to the second inference model Sample model input for the two-device collection.

In addition, the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores instructions, and when it is run on the computer equipment, the computer equipment executes the configuration device 105 of the above-mentioned embodiment. method.

In addition, an embodiment of the present application further provides a computer program product, and when the computer program product is executed by a computer, the computer executes any one of the aforementioned methods for configuring a decision-making device. The computer program product may be a software installation package, which may be downloaded and executed on a computer if any of the above-mentioned methods for configuring the decision-making device needs to be used.

In addition, it should be noted that the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course it can also be realized by special hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, Special components, etc. to achieve. In general, all functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure used to realize the same function can also be varied, such as analog circuits, digital circuits or special-purpose circuit etc. However, for this application, software program implementation is a better implementation mode in most cases. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, training device, or network device, etc.) execute the instructions described in various embodiments of the present application method.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transferred from a website, computer, training device, or data The center transmits to another website site, computer, training device or data center via wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a training device or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc.

Claims

A method for configuring a decision-making device, characterized in that the method is applied to a reasoning system, the reasoning system includes a first device set, a second device set, and a decision-making device, the first device set and the second device The sets each include at least one computing device, the size of the first inference model in the first set of devices is smaller than the size of the second inference model in the second set of devices, the method comprising:

Acquiring a first inference result corresponding to the first type of sample and a second inference result corresponding to the second type of sample, the accuracy of the first inference model inferring the first type of sample is lower than that of the first inference model inferring the first inference result The precision of the second class sample;

According to the first inference result and the second inference result, configure the decision parameters in the decision-making device, the decision parameters are used to identify the model input samples inferred by the first inference model as being transmitted to the second inference model Sample model input for the two-device collection.
The method according to claim 1, wherein said acquiring the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample comprises:

Instructing the first set of devices to use the first inference model to perform inference on the first type of samples and the second type of samples respectively, to obtain a first inference result corresponding to the first type of samples and the second The second inference result corresponding to the class sample.
The method according to claim 1, wherein said acquiring the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample comprises:

Instructing the first set of devices to use the first inference model to perform inference on the first type of samples, and obtain a first inference result corresponding to the first type of samples;

Instructing the second set of devices to use the second inference model to perform inference on the second type of samples to obtain a second inference result corresponding to the second type of samples.
The method according to any one of claims 1 to 3, wherein the method further comprises:

Get multiple samples;

Acquiring inference results corresponding to the multiple samples, where the inference results corresponding to the multiple samples are respectively obtained by inferring the multiple samples through the first inference model;

According to the inference results corresponding to the multiple samples, the first type of samples and the second type of samples among the multiple samples are determined.
The method according to claim 4, wherein said determining the first type of samples and the second type of samples among the plurality of samples according to the inference results corresponding to the plurality of samples includes:

Presenting an annotation interface, the annotation interface including inference results corresponding to the plurality of samples;

According to the labeling operation on the inference results corresponding to the multiple samples, the first type of samples and the second type of samples among the multiple samples are determined.
The method according to any one of claims 1 to 5, wherein, before acquiring the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, the method further includes:

configuring the first inference model for the first set of devices, and configuring the second inference model for the second set of devices, the first inference model being obtained by performing model compression on the second inference model .
The method according to any one of claims 1 to 6, wherein the first set of devices is deployed on a local network, and the second set of devices is deployed on an edge network;

Or, the first set of devices is deployed on the edge network, and the second set of devices is deployed on the cloud.
A method for configuring a decision-making device, characterized in that the method is applied to a reasoning system, the reasoning system includes a first device set, a second device set, and a decision-making device, the first device set and the second device The sets each include at least one computing device, the size of the first inference model in the first set of devices is smaller than the size of the second inference model in the second set of devices, the method comprising:

Acquiring samples of the first type and samples of the second type, the accuracy of inferring the samples of the first type by the first inference model is lower than the accuracy of inferring the samples of the second type by the first inference model;

According to the first type of samples and the second type of samples, configure the decision parameters in the decision-making device, the decision parameters are used to identify the model input samples inferred by the first inference model as being transmitted to the second inference model Sample model input for the two-device collection.
A configuration device, characterized in that the configuration device is applied to a reasoning system, and the reasoning system includes a first device set, a second device set, and a decision-making device, and the first device set and the second device set are both Including at least one computing device, the specification of the first inference model in the first set of devices is smaller than the specification of the second inference model in the second set of devices, the configuration means includes:

An inference result acquisition module, configured to acquire a first inference result corresponding to a first type of sample and a second inference result corresponding to a second type of sample, the accuracy of the first inference model inferring the first type of sample is lower than that of the first type an inference model infers the accuracy of the second type of samples;

A configuration module, configured to configure a decision parameter in the decision device according to the first reasoning result and the second reasoning result, the decision parameter is used to identify a model input sample reasoned by the first reasoning model as Model input samples transmitted to the second set of devices.
The configuration device according to claim 9, wherein the inference result acquisition module is specifically used for:

Instructing the first set of devices to use the first inference model to perform inference on the first type of samples and the second type of samples respectively, to obtain a first inference result corresponding to the first type of samples and the second The second inference result corresponding to the class sample.
The configuration device according to claim 9, wherein the inference result acquisition module is specifically used for:

Instructing the first set of devices to use the first inference model to perform inference on the first type of samples, and obtain a first inference result corresponding to the first type of samples;

Instructing the second set of devices to use the second inference model to perform inference on the second type of samples to obtain a second inference result corresponding to the second type of samples.
The configuration device according to any one of claims 9 to 11, wherein the inference result acquisition module is also used for:

Get multiple samples;

Acquiring inference results corresponding to the multiple samples, where the inference results corresponding to the multiple samples are respectively obtained by inferring the multiple samples through the first inference model;

According to the inference results corresponding to the multiple samples, the first type of samples and the second type of samples among the multiple samples are determined.
The configuration device according to claim 12, wherein the inference result acquisition module is specifically used for:

Presenting an annotation interface, the annotation interface including inference results corresponding to the plurality of samples;

According to the labeling operation on the inference results corresponding to the multiple samples, the first type of samples and the second type of samples among the multiple samples are determined.
The configuration device according to any one of claims 9 to 13, characterized in that, before obtaining the first inference result corresponding to the first type of sample and the second inference result corresponding to the second type of sample, the configuration device also uses At:

configuring the first inference model for the first set of devices, and configuring the second inference model for the second set of devices, the first inference model being obtained by performing model compression on the second inference model .
The configuration device according to any one of claims 9 to 14, wherein the first set of devices is deployed on a local network, and the second set of devices is deployed on an edge network;

Or, the first set of devices is deployed on the edge network, and the second set of devices is deployed on the cloud.
A configuration device, characterized in that the configuration device is applied to a reasoning system, and the reasoning system includes a first device set, a second device set, and a decision-making device, and the first device set and the second device set are both Including at least one computing device, the specification of the first inference model in the first set of devices is smaller than the specification of the second inference model in the second set of devices, the configuration means includes:

A sample acquisition module, configured to acquire a first type of sample and a second type of sample, the accuracy of the first inference model inferring the first type of sample is lower than the accuracy of the first inference model inference of the second type of sample;

A configuration module, configured to configure decision parameters in the decision-making device according to the first type of samples and the second type of samples, and the decision parameters are used to identify model input samples inferred by the first inference model as Model input samples transmitted to the second set of devices.
A computer device, characterized in that the computer device includes a processor and a memory;

The processor is configured to execute instructions stored in the memory, so that the computer device performs the method of any one of claims 1-7.
A computer device, characterized in that the computer device includes a processor and a memory;

The processor is configured to execute instructions stored in the memory to cause the computer device to perform the method of claim 8 .
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computing device, the computing device executes the described method.
A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computing device, the computing device executes the method as claimed in claim 8 .