WO2023160508A1

WO2023160508A1 - Model management method and communication apparatus

Info

Publication number: WO2023160508A1
Application number: PCT/CN2023/077287
Authority: WO
Inventors: 黄谢田; 曹龙雨; 于益俊
Original assignee: 华为技术有限公司
Priority date: 2022-02-25
Filing date: 2023-02-21
Publication date: 2023-08-31
Also published as: CN116703422A

Abstract

The present application provides a model management method and a communication apparatus. The method comprises: a model authentication entity receives first provider information from a model management entity, the first provider information being used for indicating that an inference model is provided by a first model provider; the model authentication entity compares the first model provider with a device provider of a model inference entity, so as to generate a comparison result, wherein the model inference entity is an entity for operating the inference model; and on the basis of the comparison result, the model authentication entity sends to the model management entity an authentication result of the inference model, the authentication result indicating whether the inference model can operate in the model inference entity. Thus, the operation reliability of the inference model can be improved.

Description

Method and communication device for model management

This application claims the priority of the Chinese patent application with application number 202210178362.4 and application title "A Method and Communication Device for Model Management" filed with the State Intellectual Property Office of China on February 25, 2022, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the field of communication technologies, and more specifically, relates to a model management method and a communication device.

Background technique

In order to improve the intelligence and automation level of the network, reasoning models, such as artificial intelligence (AI) models and machine learning (ML) models, are used in more and more technical fields. Currently, inference models provided by model providers (such as manufacturers or operators) can be released to the model market. The model management entity can obtain the reasoning model from the model market, and deploy the reasoning model in the device provider, and then the device provider can run the deployed reasoning model.

However, due to differences in private data and expert experience of different providers, when a certain equipment provider runs an inference model from another provider, it may not work properly.

Contents of the invention

The present application provides a model management method and a communication device, so as to improve the reliability of reasoning model operation.

In a first aspect, a method for model management is provided, the method may be executed by a model certification entity or a chip in the model certification entity, and the method includes: the model certification entity receives first provider information from a model management entity, and the first provider The vendor information indicates that the inference model is provided by the first model provider; the model authentication entity compares the first model provider with the device provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the inference model The model authentication entity sends the authentication result of the reasoning model to the model management entity based on the comparison result, and the authentication result indicates whether the reasoning model can run in the model reasoning entity.

Therefore, in this application, the model management entity can send the first provider information indicating the first model provider that provides the reasoning model to the model certification entity, and the model certification entity can compare the first model provider with the device provider , and the authentication result can be generated according to the comparison result to notify the model management entity, and then the model management entity can adopt different processing based on different authentication results, which can improve the reliability of the reasoning model operation.

In combination with the first aspect, in some implementations of the first aspect, the comparison result is that the first model provider and the device provider are inconsistent, the accuracy of the reasoning model is less than the first threshold, and the authentication result indicates that the reasoning model cannot be used for model reasoning. or, the comparison result shows that the first model provider and the device provider are inconsistent, the accuracy of the reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the reasoning model can run in the model reasoning entity.

Therefore, in this application, if the model authentication network element judges that the first model provider of the reasoning model is inconsistent with the equipment provider, then the accuracy of the reasoning model can be further determined, and whether it can be determined according to the accuracy of the reasoning model The decision reasoning model has passed the certification, which can improve the flexibility of the system.

With reference to the first aspect, in some implementations of the first aspect, before the model authentication entity compares the first model provider and the equipment provider, the method further includes: the model authentication entity receives the reasoning model from the model management entity; the model authentication The entity determines that the first model provider is the provider of the inference model.

Therefore, in this application, before judging whether the first model provider and the equipment provider are consistent, the model authentication network element can first judge whether the reasoning model provided by the first model provider, that is, judge whether the information of the first provider Authenticity can improve the reliability of model operation.

With reference to the first aspect, in some implementation manners of the first aspect, the model certification entity determining that the first model provider is the provider of the inference model includes: the model certification entity determining that the first format is consistent with the second format; or , the model certification entity determines that the second model provider is consistent with the first model provider; or, the model certification entity determines that the first format is consistent with the second format, and that the second model provider is consistent with the first model provider; wherein, the first The first format is the format of the reasoning model, the second format is determined according to the information of the first provider, and the second model provider is determined according to the reasoning model.

With reference to the first aspect, in some implementation manners of the first aspect, the method further includes: the model authentication entity sends cause information to the model management entity, where the cause information indicates the reason for obtaining the authentication result.

Therefore, in this application, the model certification entity can inform the model management entity of the reason why the inference model certification fails, and then the model management entity can perform different processing based on different reasons, which can improve the reliability of model operation.

With reference to the first aspect, in some implementation manners of the first aspect, the comparison result is that the first model provider and the device provider are consistent, and the authentication result indicates that the reasoning model can run in the model reasoning entity.

Therefore, in this application, the model authentication network element can judge whether the first model provider of the inference model is consistent with the equipment provider of the model inference entity, and if they are consistent, it can be determined that the inference model is authenticated, and then the model management entity can pass the The certified inference model is deployed in the model inference entity, which can improve the reliability of the model operation.

With reference to the first aspect, in some implementations of the first aspect, before the model certification entity compares the first model provider and the equipment provider, the method further includes: the model certification entity receives the second provider information from the model management entity , the second provider information indicates the device provider.

In a second aspect, a method for model management is provided. The method may be executed by a model management entity or a chip in the model management entity. The method includes: the model management entity sends the first provider information to the model certification entity, and the first provider The supplier information indicates that the first inference model is provided by the first model provider; the model management entity receives the authentication result of the first inference model from the model authentication entity, and the authentication result indicates whether the first inference model can run in the model inference entity.

In conjunction with the second aspect, in some implementations of the second aspect, the first model provider and the device provider of the model inference entity are inconsistent, the accuracy of the first inference model is less than the first threshold, and the authentication result indicates that the first inference model Cannot run in the model inference entity; or, the first model provider and the device provider are inconsistent, the accuracy of the first inference model is greater than or equal to the first threshold, and the authentication result indicates that the first inference model can run in the model inference entity .

With reference to the second aspect, in some implementation manners of the second aspect, the method further includes: the model management entity sends the first reasoning model to the model certification entity.

With reference to the second aspect, in some implementations of the second aspect, the first model provider is an inference model provider, including: the first format is consistent with the second format; or, the second model provider and the first model The provider is consistent; or, the first format is consistent with the second format, and the second model provider is consistent with the first model provider; wherein, the first format is the format of the inference model, and the second format is based on the information of the first provider Yes, the second model provider is determined according to the reasoning model.

With reference to the second aspect, in some implementations of the second aspect, the method further includes: the model management entity receives cause information from the model certification entity, where the cause information indicates a reason for obtaining the certification result.

With reference to the second aspect, in some implementations of the second aspect, the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further includes: the model management entity sends at least one of the following information to the model training entity : authentication result, reason information and first adjustment information, wherein, the reason information indicates the reason for obtaining the authentication result, and the first adjustment information indicates that the model training entity adjusts the first reasoning model; the model management entity receives the second reasoning model from the model training entity An inference model, the second inference model is adjusted based on the first inference model.

Therefore, in this application, when the inference model authentication fails, the model training entity can improve the flexibility of model management by reacquiring or retraining the inference model.

With reference to the second aspect, in some implementations of the second aspect, the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further includes: the model management entity sends the first inference model and the second inference model to the model inference entity Two adjustment information, the second adjustment information is used to instruct the model training entity to adjust the first reasoning model.

Therefore, in this application, when the inference model authentication fails, the model inference entity can retrain the inference model before deploying the inference model, which improves the flexibility of model management.

With reference to the second aspect, in some implementation manners of the second aspect, the first model provider is the same as the device provider of the model inference entity, and the authentication result indicates that the first inference model can run in the model inference entity.

With reference to the second aspect, in some implementations of the second aspect, before receiving the authentication result of the first reasoning model from the model authentication entity, the method further includes: the model management entity sends the second provider information to the model authentication entity, the first The second provider information indicates the device provider.

In a third aspect, a communication device for model management is provided, the device includes a transceiver module and a processing module, the transceiver module is used to receive first provider information from a model management entity, and the first provider information indicates that the reasoning model is provided by the first Provided by the model provider; the processing module is used to compare the first model provider and the equipment provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the inference model; the transceiver module is also used Based on sending the authentication result of the inference model to the model management entity based on the comparison result, the authentication result indicates whether the inference model can run in the model inference entity.

In combination with the third aspect, in some implementations of the third aspect, the comparison result is that the first model provider and the device provider are inconsistent, the accuracy of the inference model is less than the first threshold, and the authentication result indicates that the inference model cannot be used for model reasoning. or, the comparison result shows that the first model provider and the device provider are inconsistent, the accuracy of the reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the reasoning model can run in the model reasoning entity.

In combination with the third aspect, in some implementations of the third aspect, the transceiver module is also used to: receive from the model management entity Receive an inference model; the processing module is also used to: determine that the first model provider is the provider of the inference model.

With reference to the third aspect, in some implementation manners of the third aspect, the processing module is specifically configured to: determine that the first format is consistent with the second format; or determine that the second model provider is consistent with the first model provider; or, Determine that the first format is consistent with the second format, and that the second model provider is consistent with the first model provider; wherein, the first format is the format of the reasoning model, the second format is determined according to the information of the first provider, and the second The model provider is determined based on the inference model.

With reference to the third aspect, in some implementation manners of the third aspect, the transceiver module is further configured to: send cause information to the model management entity, where the cause information indicates the reason for obtaining the authentication result.

With reference to the third aspect, in some implementation manners of the third aspect, the comparison result indicates that the first model provider is consistent with the device provider, and the authentication result indicates that the inference model can run in the model inference entity.

With reference to the third aspect, in some implementation manners of the third aspect, the transceiving module is further configured to: receive second provider information from the model management entity, where the second provider information is used to indicate the device provider.

In a fourth aspect, a communication device for model management is provided, the device includes a transceiver module and a processing module, the processing module is used to generate first provider information, and the first provider information is used to indicate that the first reasoning model is provided by the first Provided by the model provider; the transceiver module is used to send the first provider information to the model authentication entity; the transceiver module is also used to receive the authentication result of the first inference model from the model authentication entity, and the authentication result indicates whether the first inference model can be used in Model inference runs in the entity.

Therefore, in this application, the model management entity can send the first provider information indicating the first model provider that provides the reasoning model to the model certification entity, and the model certification entity can compare the first model provider with the device provider , and the authentication result can be generated according to the comparison result to inform the processing module, and then the model management entity can take different processing based on different authentication results, which can improve the reliability of the reasoning model operation.

With reference to the fourth aspect, in some implementations of the fourth aspect, the first model provider and the device provider are inconsistent, the accuracy of the first inference model is less than the first threshold, and the authentication result indicates that the first inference model cannot be used in the model or, the first model provider is inconsistent with the device provider, the accuracy of the first reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the first reasoning model can run in the model reasoning entity.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the transceiver module is further configured to: send the first reasoning model to the model verification entity.

With reference to the fourth aspect, in some implementations of the fourth aspect, the first model provider is a provider of the first reasoning model, including: the first format is consistent with the second format; or, the second model provider and the first inference model A model provider is consistent; or, the first format is consistent with the second format, and the second model provider is consistent with the first model provider; wherein, the first format is the format of the inference model, and the second format is provided according to the first The provider information is determined, and the second model provider is determined according to the reasoning model.

With reference to the fourth aspect, in some implementations of the fourth aspect, the transceiver module is further configured to: receive cause information from the model authentication entity, where the cause information indicates the reason for obtaining the authentication result.

With reference to the fourth aspect, in some implementations of the fourth aspect, the authentication result indicates that the first inference model cannot run in the model inference entity, and the transceiver module is further used to: send at least one of the following information to the model training entity: Authentication result, reason information and first adjustment information, wherein the reason information indicates the reason for obtaining the authentication result, and the first adjustment information indicates that the model training entity adjusts the first inference model; receiving the second inference model from the model training entity, the first The second reasoning model is adjusted based on the first reasoning model.

With reference to the fourth aspect, in some implementations of the fourth aspect, the authentication result indicates that the reasoning model cannot run in the model reasoning entity, and the sending and receiving module is also used to: send the first reasoning model and the second adjustment information to the model reasoning entity , the second adjustment information instructs the model inference entity to adjust the first inference model.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the first model provider is consistent with the device provider, and the authentication result indicates that the first inference model can run in the model inference entity.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the transceiving module is further configured to: send second provider information to the model verification entity, where the second provider information indicates the device provider of the model reasoning entity.

In a fifth aspect, a communication device is provided, and the device may include a processing module, a sending unit, and a receiving unit. Optionally, the sending unit and the receiving unit may also be transceiver modules.

When the device is a model certification entity, the processing module can be a processor, and the sending unit and the receiving unit can be transceivers; the device can also include a storage unit, which can be a memory; the storage unit is used to store instructions , the processing module executes the instructions stored in the storage unit, so that the model certification entity executes any method in the first aspect. When the device is a chip in a model certification entity, the processing module may be a processor, and the sending unit and receiving unit may be input/output interfaces, pins or circuits, etc.; the processing module executes the instructions stored in the storage unit, to make the chip execute any one of the methods in the first aspect. The storage unit is used to store instructions, and the storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit outside the chip in the model certification entity (for example, a read-only memory , random access memory, etc.).

When the device is a model management entity, the processing module may be a processor, and the sending unit and the receiving unit may be transceivers; the device may also include a storage unit, which may be a memory; the storage unit is used to store instructions , the processing module executes the instruction stored in the storage unit, so that the model management entity executes any method in the second aspect. When the device is a chip in the model management entity, the processing module may be a processor, and the sending unit and the receiving unit may be input/output interfaces, pins or circuits, etc.; the processing module executes the instructions stored in the storage unit, to make the chip execute any one of the methods in the second aspect. The storage unit is used to store instructions, and the storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit outside the chip in the model management entity (for example, a read-only memory , random access memory, etc.).

In a sixth aspect, a communication device is provided, including a processor and an interface circuit, and the interface circuit is used to receive signals from other communication devices other than the communication device and transmit them to the processor or send signals from the processor For other communication devices other than the communication device, the processor is used to implement any method in the aforementioned first aspect or second aspect through a logic circuit or executing code instructions.

In a seventh aspect, a computer-readable storage medium is provided, and a computer program or instruction is stored in the computer-readable storage medium. When the computer program or instruction is executed, any of the aforementioned first or second aspects can be realized. One method.

In an eighth aspect, a computer program product containing instructions is provided, and when the instructions are executed, any one of the methods in the aforementioned first aspect or second aspect is implemented.

In a ninth aspect, a computer program is provided, the computer program includes codes or instructions, and when the codes or instructions are executed, any method in the aforementioned first aspect or second aspect is implemented.

In a tenth aspect, a chip system is provided, and the chip system includes a processor and may further include a memory, configured to implement any method in the aforementioned first aspect or second aspect. The system-on-a-chip may consist of chips, or may include chips and other discrete devices.

In an eleventh aspect, a communication system is provided, and the system includes the device described in any one of the third aspect and the fourth aspect.

Description of drawings

FIG. 1 is a schematic diagram of a system for model management provided by an embodiment of the present application.

Fig. 2 is a schematic flowchart of an example of a model management method provided by an embodiment of the present application.

Fig. 3 is a schematic flow chart of another example of an Italian model management method provided by an embodiment of the present application.

Fig. 4 is a schematic flowchart of another example of the model management method provided by the embodiment of the present application.

FIG. 5 and FIG. 6 are schematic structural diagrams of a possible communication device provided by an embodiment of the present application.

Detailed ways

The technical solution in this application will be described below with reference to the accompanying drawings.

The method in the embodiment of the present application can be applied to a long term evolution technology (long term evolution, LTE) system, a long term evolution advanced technology (long term evolution-advanced, LTE-A) system, an enhanced long term evolution technology (enhanced long term evolution-advanced , eLTE), the fifth generation (the ^5th Generation, 5G) mobile communication system New Radio (NR) system, can also be extended to similar wireless communication systems, such as wireless-fidelity (wireless-fidelity, WiFi) , worldwide interoperability for microwave access (WIMAX), and cellular systems related to the ^3rd generation partnership project (3gpp).

For clarity, some terms in the embodiments of the present application are explained below.

Inference model (may also be referred to simply as model): A function learned from data that can implement a specific function/map. The model can be obtained based on artificial intelligence (AI) or machine learning (ML) technology, so it can also be called artificial intelligence/AI model, machine learning/ML model, etc. Commonly used algorithms for generating AI/ML models include: supervised learning, unsupervised learning, and enhanced learning. The corresponding models can be called supervised learning models, unsupervised learning models, and enhanced learning models. Exemplarily, the supervised learning model may be a classification model, a prediction model, a regression model, etc., and the unsupervised learning model may be a clustering model. In addition, the model can also be obtained based on neural network (NN) technology, and this model can also be called a neural network model, a deep learning model, and the like.

Model training: Use the training data to train the available models.

Model reasoning: perform reasoning or prediction based on the model, and generate reasoning results. Additionally, model inference entities can be used for model inference.

Model Deployment: Deploy the model in the model inference entity.

Model activation: Activate the model deployed in the model inference entity to start running.

Model evaluation: Evaluate whether the performance of the model running in the model inference entity meets the requirements.

Model certification: Determine whether the entity for model training is consistent with the entity deployed by the model, and when the entity trained by the model is inconsistent with the entity deployed by the model, determine whether the operating performance of the model after deployment can meet expectations.

Model management: Manage models during their life cycle. For example, manage model deployment, model activation, model evaluation, model training, etc.

To facilitate understanding of the embodiment of the present application, an application scenario of the embodiment of the present application is described in detail first with reference to FIG. 1 .

Fig. 1 is a schematic structural diagram of a communication system to which the embodiment of the present application is applicable. Firstly, the devices that may be involved in the communication system will be described.

1. Model management entity 110: used to manage the model within the life cycle. For example, the model management entity 110 may be a network management system (network management system, NMS).

In this embodiment of the application, the model management entity 110 may be deployed in the operator's equipment.

2. Model training entity 120: used to obtain available models through training. For example, the model training entity may be an operator's platform or a manufacturer's training platform, or other entities deploying a model training function.

In this embodiment of the application, the model training entity 120 can publish the trained model to the model market, and the model management entity 110 can obtain the model from the model market, and deploy the model to the model reasoning entity 130 . Wherein, the model market can be deployed in the model management entity 110 , can also be deployed in the model training entity 120 , or can be deployed independently, which is not particularly limited in this application.

In addition, in this embodiment of the application, the provider of the model training entity 120 may be referred to as a model provider.

3. Model reasoning entity 130: used to perform reasoning or calculation based on the model, and generate reasoning results. For example, the model reasoning entity 130 may be an element management system (element management system, EMS) or a management data analysis function (management data analytics function, MDAF), a radio access network (radio access network, RAN), or a 5G system A network element (for example, a network data analysis function (network data analytics function, NWDAF) network element).

In the embodiment of the present application, the model reasoning entity 120 may be deployed in a manufacturer's equipment, and the provider of the model reasoning entity 120 may be referred to as a device provider.

In the embodiment of this application, the communication system may also include a model certification entity 140:

4. Model certification entity 140: the model certification entity 140 can be used to certify the model, for example, can be used to determine whether the operating performance of the model can meet expectations.

It should be noted that the solution in this application can be applied to other systems including corresponding entities, which is not limited in this application. It can be understood that the above entity or function may be a network element in a hardware device, a software function running on dedicated hardware, or a virtualization function instantiated on a platform (eg, a cloud platform). Optionally, the above entity or function may be implemented by one device, or jointly implemented by multiple devices, or may be a functional module in one device, which is not specifically limited in this embodiment of the present application. For example, in this embodiment of the application, the model management entity 110 and the model training entity 120 may be different functional models in one device, and the model inference entity 130 and the model certification entity 140 may be different functional modules in one device.

When the model inference entity runs the models provided by different providers, it may not be able to run the model normally. Therefore, the embodiment of the present application provides a model management method and communication device, which can improve the reliability of the model operation. The following first introduces the model The method of management is explained. Wherein, the method embodiments shown in FIG. 2 to FIG. 4 can be combined with each other, and the steps in the method embodiments shown in FIG. 2 to FIG. 4 can be referred to each other. For example, in the embodiment of the present application, the method embodiments shown in FIG. 3 and FIG. 4 may respectively be an implementation manner for realizing the functions of the method embodiment shown in FIG. 2 .

FIG. 2 is a schematic flowchart of a method 200 for model management provided by an embodiment of the present application.

S201, the model management entity sends the first provider information to the model certification entity, and correspondingly, the model certification entity receives the first provider information from the model management entity.

The first provider information indicates that the reasoning model (ie, the first reasoning model) is provided by the first model provider. In other words, the first provider information indicates that the first inference model is trained and generated by the first model provider.

The first model provider may be a first manufacturer or a first supplier. The first inference model is provided by the first model provider may mean that: the device providing the first inference model belongs to the first model provider, or in other words, the first inference model is trained The device of the model belongs to the first model provider, or in other words, the device generating the first inference model belongs to the first model provider.

It should be understood that in the embodiment of the present application, the first model provider may train the first reasoning model based on the data of the first model provider, or obtain and train the first reasoning model based on data from other providers, This application does not specifically limit it.

The first provider information may be the name or identification information of the first model provider, for example, manufacturers such as Huawei, ZTE, and Ericsson, or operators such as China Mobile and China Telecom.

S202. The model verification entity compares the first model provider with the device provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the first inference model.

For example, the model authentication entity determines whether the first model provider is consistent with the equipment provider for providing the model inference entity.

The equipment provider may be a second manufacturer or a second operator.

The model reasoning entity being an entity that will run the first reasoning model may refer to: the device to be received by the first reasoning model (that is, the model reasoning entity) is produced or provided by the device provider, or the first reasoning model to be run An inference model device is produced or provided by the device provider.

For example, when the first model provider and the equipment provider are the same manufacturer, or the first model provider and the equipment provider are the same operator, the model certification entity may determine that the first model provider and the equipment provider are consistent; When the first model provider and the device provider are different manufacturers, or different operators, or are manufacturers and operators respectively, it is determined that the first model provider and the device provider are inconsistent.

Therefore, in this embodiment of the application, before the first inference model is run, the model certification entity can first determine whether the provider who trained the first inference model is consistent with the provider who runs the model, so as to be able to implement different results. Different handling to improve the reliability of running models.

Optionally, when the model authentication entity and the model inference entity are deployed in the same device, or in other words, when the model authentication entity and the model inference entity are provided by the same provider, the information of the device provider can be recorded in the In the model authentication entity.

Or, optionally, when the model authentication entity is not provided by the equipment provider, the method 200 may acquire the information of the equipment provider by performing step S203.

Optionally, in S203, the model management entity sends the second provider information to the model certification entity, and correspondingly, the model certification entity receives the second provider information from the model management entity.

The second provider information indicates the device provider.

Wherein, the second provider information may be the name or identification information of the device provider and/or the name or identification information of the model reasoning entity. Before sending the second provider information, the model management entity may determine that the first inference model needs to be deployed in the device provider's model inference entity, and then send the second provider information to the model authentication entity.

S204, the model authentication entity sends an authentication result to the model management entity based on the comparison result, and correspondingly, the model management entity receives the authentication result from the model authentication entity.

The authentication result indicates whether the first inference model can run in the model inference entity.

For example, the model verification entity determines whether the first reasoning model can run in the model reasoning entity according to the comparison result, so as to generate a verification result. If the model authentication entity determines that the first inference model can run in the model inference entity, then the first inference model is certified. Conversely, if the model certification entity determines that the first inference model cannot type reasoning entity, then the authentication of the first reasoning model fails.

Optionally, when the comparison result shows that the first model provider and the device provider are consistent, the authentication result indicates that the first reasoning model can run in the model reasoning entity. Alternatively, the authentication result indicates that the first model provider and the device provider are identical. Alternatively, the authentication result instructs the model management entity to deploy the first reasoning model. Alternatively, the authentication result indicates that the authentication status of the first reasoning model is passed.

Optionally, the model authentication entity may also send the authentication result to the model inference entity, and correspondingly, the model inference entity receives the authentication result from the model authentication entity.

The authentication result may also indicate an authenticated ID or an authenticated password, and the authenticated ID or the authenticated password may be used by the model inference entity to determine that the first inference model is authenticated. For example, the authentication result includes status information indicating that the authentication is passed, and an authentication-passed identifier or a pass-authentication password.

For example, the model authentication entity may generate an authentication pass ID or an authentication pass password according to the version of the first inference model or the identifier of the first inference model, and send the authentication results to the model management entity and the model inference entity respectively. The model management entity carries the authentication result in the model deployment message used to deploy the first inference model, and the model inference entity determines the identity or password carried in the model deployment message and the identity or password indicated by the authentication result received from the model authentication entity. Whether the passwords are consistent or not determines whether the first reasoning model is authenticated.

In addition, the model deployment message may also carry at least one of the following information: identification information of the first inference model, version information of the first inference model, first inference model, usage information of the first inference model.

Wherein, the usage information of the first reasoning model may be used to indicate the operating conditions of the first reasoning model, for example, the usage information may include at least one of the following information: the usage format of the first reasoning model, the usage time of the first reasoning model and information such as the use area of the first inference model.

Optionally, when the result of the comparison is that the first model provider and the device provider are inconsistent, the method 200 may also perform step S205.

Optionally, in S205, the model verification entity determines the verification result according to the comparison result and performance information.

For example, the model certification entity determines whether the first reasoning model is certified according to the performance information, where the performance information is used to indicate the accuracy of the first reasoning model.

The comparison result shows that the first model provider and the device provider are inconsistent, the accuracy of the first inference model is less than the first threshold, and the authentication result indicates that the first inference model fails to pass the authentication; or, the comparison result is that the first model provider and the device provide If the quotients are inconsistent, the accuracy of the first inference model is greater than or equal to the first threshold, and the authentication result indicates that the first inference model has passed the authentication.

Wherein, determining that the first inference model is authenticated according to the performance information may refer to: the first inference model is deployed in the model inference entity to achieve the expected performance; determining that the first inference model is not certified according to the performance information may refer to: the first The inference model deployed on the model inference entity cannot achieve the expected performance.

Before the model certification entity determines whether the first inference model is certified, the model certification entity may acquire the performance information, for example, the model certification entity may obtain the performance information by testing the first inference model.

As an example but not a limitation, the first inference model is used to predict the second data in the second time period using the first data in the first time period, the first data and the second data are historical data saved by the model inference entity, In other words, the first data and the second data are real data in two time periods. The model verification entity takes the first data as input data, and runs the first reasoning model to obtain forecast data within the second forecast period. Furthermore, the model certification entity may determine the accuracy of the first reasoning model by determining the matching degree of the second data and the predicted data, so as to generate performance information.

Optionally, the model authentication entity requests the model inference entity to acquire the first data and the second data according to the usage information of the first inference model.

Wherein, the first threshold may be preset, or may be indicated by the model performance information sent by the model management entity to the model authentication entity. achievable performance.

Therefore, even if the first model provider is inconsistent with the equipment provider, if the performance of the first inference model can meet expectations, then it can be determined that the first inference model has passed the certification, and the model management entity can deploy the first inference model in the model inference entity.

Optionally, the authentication result indicates that the first reasoning model cannot run in the model reasoning entity (that is, when the first model provider is inconsistent with the device provider, and the accuracy of the first reasoning model is less than the first threshold). Alternatively, the authentication result indicates that the authentication of the first reasoning model fails. Or, the authentication result indicates that the first reasoning model deployed in the model reasoning entity cannot run normally. Or, the authentication result indicates that the first reasoning model deployed on the model reasoning entity cannot achieve expected performance.

Optionally, in S206, the model certification entity sends reason information to the model management entity, and correspondingly, the model management entity receives the reason information from the model certification entity.

The cause information indicates the reason for the authentication result.

Optionally, the model authentication entity may send cause information to the model management entity only when the authentication result indicates that the authentication of the first reasoning model fails.

For example, the reason information indicates the reason why the first reasoning model fails the authentication. For example, the cause information may indicate that the performance information of the first reasoning model cannot meet expectations, and may also indicate that the first model provider and the device provider are inconsistent.

The cause information may include usage information, accuracy of the first reasoning model, and information indicating that the first model provider and the device provider are inconsistent. Wherein, the usage information and the accuracy may refer to: the accuracy that the first inference model is expected to achieve by running the test in a manner indicated by the usage information.

After the model management entity receives the authentication result indicating that the first inference model cannot be run in the model inference entity, it may process it in two ways, which will be introduced respectively below.

Method 1:

In manner 1, the method 200 may execute steps S207 to S208.

Optionally, in S207, the model management entity sends at least one of the following information to the model training entity: authentication result, cause information, and first adjustment information.

Wherein, the first adjustment information is used to indicate to adjust the first reasoning model. to generate an adjusted inference model (in the embodiment of the present application, the adjusted first inference model is represented by a second inference model).

The authentication result and cause information may correspond to the identification information of the first inference model, and then the model training entity may regenerate the second inference model according to the authentication result and/or cause information, or when receiving the adjustment information, may update the first inference model The model is tuned to generate a second inference model

Wherein, adjusting the first reasoning model may refer to: retraining the first reasoning model.

It should be understood that when the model management entity only sends the reason information to the model training entity, the reason information may implicitly indicate that the first reasoning model fails the authentication, and the reason information may also implicitly indicate that the first reasoning model The model is adjusted.

The model management entity may also send information indicating the device provider, model identification information, model performance information or model version information to the model training entity to assist the model training entity in determining the second inference model.

Optionally, in S208, the model training entity determines a second inference model.

The model training entity may determine that the first inference model fails the authentication according to the authentication result, the cause information, or the first adjustment information, and then generate a second inference model, or adjust the first inference model to generate a second inference model.

For example, the model training entity may acquire the second inference model trained and generated by the device provider.

Or, for another example, the model training entity may extract the data set of the device provider and retrain the first inference model to generate the second inference model.

Optionally, the model training entity sends the second inference model to the model management entity, and correspondingly, the model management entity receives the second inference model from the model management entity.

Furthermore, the model management entity may perform an authentication process similar to steps S201 to S206 on the second reasoning model.

Therefore, in mode 1, when the first inference model fails to pass the authentication, the model training entity can generate the second inference model by reacquiring or retraining the first inference model.

Method 2:

Optionally, the model management entity sends the first inference model and the second adjustment information to the model inference entity, and correspondingly, the model inference entity receives the first inference model and the second adjustment information from the model management entity.

The second adjustment information is used to indicate to adjust the first reasoning model.

For example, the model management entity can send a model deployment request message to the model inference entity, and the model deployment request message carries the first inference model and the second adjustment information, and then, the model inference entity can know the first inference model according to the second adjustment information If the model authentication fails, or the model inference entity knows that running the first inference model cannot achieve the expected performance according to the second adjustment information, the model inference entity can retrain the first inference model before running the first inference model.

It should be noted that when the model authentication entity is deployed separately from the model inference entity, the model management entity can also send the identification information, version information, model performance information or usage information of the first inference model to the model inference entity , used to assist the model reasoning entity to retrain the first reasoning model, so that the retrained first reasoning model can achieve the performance indicated by the model performance information.

It should be understood that, in a possible implementation manner, when the model authentication entity and the model inference entity are deployed in one device, the model authentication entity and the model inference entity receive the first provider information, and the first provider information is carried in the model In the deployment request message, that is, the model authentication entity has acquired the first inference model before determining whether the first inference model is authenticated. When the model authentication entity determines that the first inference model is not certified, it can notify the model inference entity through the internal interface If the authentication of the first reasoning model fails, the model reasoning entity may adjust the first reasoning model before running it. In this case, the model management entity may not need to send the adjustment information to the model inference entity.

Therefore, in mode 2, when the first inference model fails to pass the authentication, the model inference entity may retrain the first inference model before deploying the first inference model.

In the above description, before running the first inference model, the model certification entity may first determine whether the first model provider that provides the first inference model and the device provider to which the first inference model is to be deployed, indicated by the first provider information, are Consistent, when they are consistent, the model certification entity may consider that the first reasoning model has passed the certification; when they are inconsistent, the model certification entity may further determine whether the first reasoning model has passed the certification based on the performance information of the first reasoning model. However, mod The first provider information sent by the type management entity may be unauthentic. Therefore, before step S202, the method 200 may also execute steps S209 to S210 to determine whether the first provider information is authentic, which will be introduced below.

Optionally, in S209, the model management entity sends the first inference model to the model certification entity, and correspondingly, the model certification entity receives the first inference model from the model management entity.

The first reasoning model may be carried in the form of a model file or a model file address. Wherein, the model file refers to information describing the first reasoning model, which is recorded in a file format, and the model file address refers to address information for indexing to the model file. In addition, it should be understood that the model file may consist of multiple sub-files.

The information describing the first inference model may include at least one of the following information: a name of the first inference model, information of a second model provider providing the first inference model, and an identifier of the first inference model.

Optionally, in S210, the first inference model of the model certification entity determines that the first model provider is the provider of the first inference model.

For example, the model certification entity determines whether the first inference model is provided or trained by the first model provider according to the first inference model and the first provider information.

It should be understood that the first provider information is recorded outside the model file, and the difference from the second model provider information in the first inference model is that the second model provider information in the first inference model is originally recorded in The real information in the model file, the first provider information outside the model file is the information specified by the model management entity and may have errors.

The model certification entity may use three methods to determine whether the first reasoning model is trained by the first model provider, and the three methods will be described below.

Method A:

When the second model provider is consistent with the first model provider, the model certification entity determines that the first inference model is provided by the first model provider.

When the second model provider is inconsistent with the first model provider, the model certification entity determines that the first inference model is not provided by the first model provider.

Wherein, the second model provider is the provider indicated by the first inference model that provides the first inference model, or in other words, the second model provider is the provider recorded in the model file of the first inference model.

Therefore, in mode A, the model certification entity can determine whether the first provider information is authentic according to whether the first provider information is consistent with the second model provider recorded in the first reasoning model.

Method B:

When the first format is consistent with the second format, the model certification entity determines that the first reasoning model is provided by the first model provider;

When the first format is inconsistent with the second format, the model certification entity determines that the first inference model is not provided by the first model provider;

Wherein, the first format is the format of the first reasoning model, and the second format is determined according to the first provider information.

The format of the first reasoning model may refer to: the format of the model file of the first reasoning model, such as file format, syntax, and the like.

The fact that the second format is determined according to the first provider information may mean that the model certification entity may determine the second format of the model trained by the first model provider according to the first model provider indicated by the first provider information. For example, the model certification entity may record the formats of the training models of multiple suppliers, and the model certification entity may determine the first format corresponding to the first model provider after receiving the information of the first provider.

Therefore, in mode B, the model certification entity can determine whether the information of the first provider is authentic according to whether the format of the first model provider is consistent with the format of the first reasoning model.

Method C:

When the second model provider is consistent with the first model provider, and the first format is consistent with the second format, the model certification entity determines that the first reasoning model is provided by the first model provider;

When the second model provider is inconsistent with the first model provider, and/or, the first format is inconsistent with the second format, the model certification entity determines that the first inference model is not provided by the first model provider;

Wherein, the second model provider is the supplier indicated by the first inference model to provide the first inference model, the first format is the format of the first inference model, and the second format is determined according to the information of the first provider.

Therefore, in method C, the model certification entity can simultaneously determine whether the first provider information is consistent with the second model provider recorded in the first inference model, and whether the format of the first model provider and the format of the first inference model Consistent, when both conditions are consistent, it is determined that the information of the first provider is true.

In the above three manners, when the model certification entity determines that the first reasoning model is provided by the first model provider, the method 200 may execute steps S202 to S209 to determine whether the first model provider and the equipment provider are consistent. When the model certification entity determines that the first reasoning model is not trained by the first model provider, the model certification entity may consider that the first reasoning model certification is unsuccessful, and the method may perform steps S206 to S209 after the first reasoning model certification is unsuccessful Actions. In this case, the cause information in step S207 may also be used to indicate that the first inference model is not trained by the first model provider, or indicate that the information of the first provider is wrong.

Therefore, in this application, the model management entity can send the first provider information indicating the first model provider that provides the first reasoning model to the model certification entity, and the model certification entity can verify the first model provider and the equipment provider The comparison is performed, and an authentication result can be generated according to the comparison result to notify the model management entity, and then the model management entity can take different processing based on different authentication results, which can improve the reliability of the operation of the first reasoning model.

When the model authentication entity and the model inference entity are deployed on different devices, the model management entity may request the model authentication entity for authentication, and determine whether the first inference model does not belong to the model inference entity according to the authentication result.

FIG. 3 is a schematic flowchart of a method 300 for model management provided by an embodiment of the present application.

Optionally, S301. A model management entity acquires a first inference model.

For example, the model management entity may send a model query request message to the model marketplace according to network status or operation requirements, where the model query request message is used to request to acquire the first reasoning model. The model marketplace may send the first inference model to the model management entity in response to the query request message, or send multiple models, and the model management entity determines the first inference model from the multiple models. Wherein, the model market has models trained by multiple suppliers, and the model market can be deployed in the model training entity shown in FIG. 1 .

Or, for another example, the model management entity may send a model training request message to the model training entity according to network status or operation requirements, where the model training request message is used to request the model training entity to perform training to generate the first inference model. The model training entity performs model training according to the model training request message to generate a first reasoning model. The model training entity sends the first inference model to the model management entity in response to the model training request message.

In addition, the model management entity and the model training entity can be deployed on the same operator platform.

S302, the model management entity sends the first provider information to the model certification entity, and correspondingly, the model certification entity receives the first provider information from the model management entity.

The first provider information indicates that the first inference model is provided by the first model provider. Information about the first provider For description, refer to the description of the first provider information in step S201 in the method 200, and for simplicity, details are not repeated here.

For example, the model management entity sends a model certification request message to the model certification entity, where the model certification request message is used to request certification of the first reasoning model, and the model certification request message carries the first provider information.

Optionally, in S303, the model management entity sends the second provider information and the first reasoning model to the model certification entity, and correspondingly, the model certification entity receives the second provider information and the first reasoning model from the model management entity.

The second provider information is used to indicate the equipment provider on which the first reasoning model is to be deployed, and the equipment provider is a second manufacturer or a second operator. For the description about the device provider and the first reasoning model, refer to the description of the device provider in step S202 of the method 200, and for the sake of brevity, details are not repeated here.

For example, the model management entity sends a model certification request message to the model certification entity, where the model certification request message carries the second provider information and the first reasoning model.

In a possible implementation, the model authentication request message may also carry at least one of the following information: identification information of the first inference model, version information of the first inference model, performance information of the first inference model, and - Usage information for an inference model. For a description of the above information, reference may be made to the description of the method 200, and for simplicity, details are not repeated here.

Optionally, in S304, the model authentication entity determines, according to the first inference model and the first provider information, that the provider of the first inference model is the first inference model.

For the description about whether the model certification entity determines whether the first inference model is provided by the first model provider, refer to the description in step S210 in the method 200, and for the sake of brevity, details are not repeated here.

Optionally, the model certification entity determines that the first reasoning model is provided by the first model provider, and the method 300 may perform steps S305 to S313.

S305. The model certification entity compares the first model provider and the equipment provider to generate a comparison result.

For example, the model certification entity may determine whether the first model provider and the device provider are consistent.

For the manner in which the model verification entity performs comparison to generate a comparison result, reference may be made to the description of step S202 in the method 200 , and details are not repeated here for brevity.

Optionally, when the comparison result shows that the first model provider and the device provider are inconsistent, the method 300 may perform step S306.

Optionally, at S306, the model verification entity determines the verification result according to the comparison result and performance information.

For another example, the model certification entity may send an evaluation data request message to the model reasoning entity, where the evaluation data request message is used to request the first data and the second data, and the evaluation data request message may include type information and condition information. In response to the evaluation data request message, the model reasoning entity sends an evaluation data response message to the model certification entity, where the evaluation data response message carries the first data and the second data. Among them, the type information may refer to the data types of the first data and the second data, and the condition information refers to the conditions that the first data and the second data need to meet. For example, the condition information may include standard conditions, time conditions, and area conditions. Model The reasoning entity sends the first data and the second data satisfying the condition to the model verification entity.

The manner in which the model certification entity determines the performance information according to the first data and the second data, and determines whether the first inference model is certified according to the performance information can refer to the description of step S205 in the method 200, and for the sake of brevity, details are not repeated here.

S307, the model authentication entity sends the authentication result to the model management entity, and correspondingly, the model management entity receives the authentication result from the model authentication entity.

For example, in response to the model certification request message, the model certification entity sends a model certification response message to the model management entity, where the model certification response message carries the certification result. For a description of the authentication result, reference may be made to the description of step S204 in the method 200, and for simplicity, details are not repeated here.

Optionally, in S308, the model authentication entity sends the authentication result to the model inference entity, and correspondingly, the model inference entity receives the authentication result from the model authentication entity.

For example, the model authentication entity sends a model authentication pass notification message to the model reasoning entity, and the model authentication pass notification message carries the authentication result.

Optionally, in S309, the model certification entity sends reason information to the model management entity, and correspondingly, the model management entity receives the reason information from the model certification entity.

The cause information is used to indicate the cause of the authentication result. For the description of the cause information, refer to the description of step S206 in the method 200, and for the sake of brevity, details are not repeated here.

For example, the model authentication response message carries the cause information.

When the authentication result indicates that the first reasoning model is authenticated, the method 300 may execute step S310.

Optionally, at S310, the model management entity sends a model deployment message to the model inference entity, and correspondingly, the model inference entity receives the model deployment message from the model management entity.

The model deployment message is used to deploy the first inference model. The model deployment message may carry at least one of the following information: identification information of the first reasoning model, version information of the first reasoning model, the first reasoning model, usage information of the first reasoning model, and the authentication result.

The model inference entity may determine that the authentication of the first inference model is successful according to the consistency of the authentication results in the model deployment message and the model authentication notification message. For example, the model inference entity stores the corresponding relationship between the model identifier and/or model version and the authentication result through the notification message according to the model authentication, and the model inference entity determines the stored corresponding authentication result and model according to the model identifier and/or version in the model deployment message. The authentication results in the deployment message are consistent, that is, it is determined that the authentication of the first reasoning model is successful. Furthermore, the model reasoning entity can deploy the first reasoning model.

Optionally, the model inference entity sends a model deployment response to the model management entity in response to the model deployment message, where the model deployment response is used to indicate the deployment status of the first inference model.

The model management entity can use two methods to handle the situation that the authentication result indicates that the first reasoning model fails to pass the authentication, and the two methods will be described below.

Method a:

Including steps S311 to S312.

Optionally, S311. The model management entity sends at least one of the following information to the model training entity: authentication result, cause information, and first adjustment information.

For related descriptions, refer to the description of step S207 in the method 200, and for the sake of brevity, details are not repeated here.

For example, the model management entity sends a model optimization request message to the model training entity, and the model optimization request message carries at least one of the following information: authentication result, cause information, and first adjustment information.

In a possible implementation manner, the model optimization request message also carries at least one of the following information: information of the equipment provider, identification information of the model, information of model performance or version information of the model.

Optionally, S312. The model training entity determines a second reasoning model.

For the manner in which the model training entity determines the second inference model, reference may be made to the description of step S208 in the method 200, and details are not repeated here for brevity.

For example, the model training entity sends a model optimization response message to the model management entity in response to the model optimization request message, where the model optimization response message carries the second inference model.

In a possible implementation, the model optimization response message also carries model performance information of the second inference model and usage information of the second inference model, and the usage information of the second inference model can be used to indicate the For the running condition, the model performance information is used to describe the performance that the second inference model can achieve when running using the running condition indicated by the information.

Method b:

Step S313 is included.

S313. The model management entity sends the first inference model and the second adjustment information to the model inference entity. Correspondingly, the model inference entity receives the first inference model and the second adjustment information from the model management entity.

The second adjustment information indicates that the reasoning model should be adjusted.

For example, the model management entity may send a model deployment request message to the model inference entity, where the model deployment request message carries the first inference model and the second adjustment information.

In a possible implementation manner, the model deployment request message further carries at least one of the following information: identification information, version information, model performance information, or usage information of the first inference model.

It should be understood that the adjustment or retraining of the first inference model by the model inference entity may refer to the adjustment or retraining of the inference model by the model training function module in the model inference entity.

When the model certification entity and the model reasoning entity are different functional modules on the same device, or when the model certification entity integrates the functions of model certification and operation, the model management network element sends a model deployment message to the model certification entity, and the model certification entity can After the authentication result is determined, it is determined whether the inference model can be run according to the authentication result.

FIG. 4 is a schematic flowchart of a method 400 for model management provided by an embodiment of the present application.

Optionally, S401. The model management entity acquires an inference model.

For the manner in which the model management entity acquires the inference model, refer to the description in step S301 in the method 300, and for simplicity, details are not repeated here.

S402, the model management entity sends the first provider information to the model certification entity, and correspondingly, the model certification entity receives the first provider information from the model management entity.

For the description of the first provider information, refer to the description of the first provider information in step S201 in the method 200, and for the sake of brevity, details are not repeated here.

For example, the model management entity sends a model deployment message to the model inference entity, where the model deployment message is used to deploy an inference model, and the model deployment message carries the first provider information.

Optionally, in S403, the model management entity sends the reasoning model to the model certification entity, and correspondingly, the model certification entity receives the reasoning model from the model management entity.

For example, the model deployment message carries the reasoning model.

In a possible implementation manner, the model deployment message may also carry at least one of the following information: identification information of the inference model, version information of the inference model, model performance information of the inference model, and use information of the inference model. For a description of the above information, reference may be made to the description of the method 200, and for simplicity, details are not repeated here.

Optionally, S404. The model certification entity determines, according to the inference model and the first provider information, that the first model provider is the inference model provider.

The description about whether the inference model is provided by the first model provider by the model certification entity can refer to the description in step S210 in the method 200, and for the sake of brevity, details are not repeated here.

Optionally, the model certification entity determines that the inference model is provided by the first model provider, and the method 400 may perform steps S405 to S408.

S405. The model certification entity compares the first model provider and the equipment provider to generate a comparison result.

Optionally, when the comparison result shows that the first model provider and the device provider are inconsistent, the method 400 may perform step S406.

Optionally, at S406, the model verification entity determines the verification result according to the comparison result and performance information.

For example, the model certification entity determines whether the reasoning model is certified according to the performance information, and the performance information is used to indicate the accuracy of the reasoning model.

For the manner in which the model certification entity judges whether the inference model has passed the certification based on the performance information, refer to the description of step S205 in the method 200, and for simplicity, details are not repeated here.

S407, the model authentication entity sends the authentication result to the model management entity, and correspondingly, the model management entity receives the authentication result from the model authentication entity.

The authentication result indicates whether the inference model can run in the model inference entity.

For example, in response to the model deployment request message, the model authentication entity sends a model deployment response message to the model management entity, where the model deployment response message carries the authentication result. For a description of the authentication result, reference may be made to the description of step S204 in the method 200, and for simplicity, details are not repeated here.

Optionally, in S408, the model certification entity sends reason information to the model management entity, and correspondingly, the model management entity receives the reason information from the model certification entity.

For example, the model deployment response message carries the cause information.

Furthermore, the model authentication entity and the model management entity can process the inference model according to the authentication result:

When the authentication result indicates that the inference model can be run in the model inference entity, the model authentication entity can notify the model inference entity to run the inference model through an internal interface.

When the authentication result indicates that the inference model cannot be run in the model inference entity, the model management entity can use the two methods described in method 300, method a and method b, to handle the situation that the inference model fails to pass the authentication, for For the sake of simplicity, it will not be repeated here.

FIG. 5 and FIG. 6 are schematic structural diagrams of a possible communication device provided by an embodiment of the present application. These communication devices can be used to implement the functions of the model authentication entity and the model management entity in the above method embodiments, and therefore can also realize the beneficial effects of the above method embodiments. In the embodiment of the present application, the communication device may be a model authentication entity or a model management entity, or a module (such as a chip) applied to the model authentication entity or the model management entity.

As shown in FIG. 5 , a communication device 500 includes a processing module 510 and a transceiver module 520 . The communication device 500 is configured to realize the functions of the model authentication entity and the model management entity in the method embodiment shown in FIG. 2 above. Alternatively, the communication device 500 may include a module for realizing any function or operation of the model authentication entity and the model management entity in the method embodiment shown in FIG. or any combination thereof.

When the communication device 500 is used to implement the function of the model authentication entity in the method embodiment shown in FIG. 2 , the transceiver module 520 is used to receive the first provider information from the model management entity. Provided by the model provider; the processing module 510 is used to compare the first model provider and the device provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the inference model; the processing module 510 is used to An authentication result of the inference model is sent to the model management entity based on the comparison result, and the authentication result indicates whether the inference model can run in the model inference entity.

More detailed descriptions about the processing module 510 and the transceiver module 520 can be directly obtained by referring to the relevant descriptions in the method embodiment shown in FIG. 2 , and will not be repeated here.

When the communication device 500 is used to realize the function of the model management entity in the method embodiment shown in FIG. The first model provider provides; the transceiving module 520 is also used to receive the authentication result of the first inference model from the model authentication entity, and the authentication result indicates whether the first inference model can run in the model inference entity.

As shown in FIG. 6 , the communication device 600 includes a processor 610 and an interface circuit 620 . The processor 610 and the interface circuit 620 are coupled to each other. It can be understood that the interface circuit 620 may be a transceiver or an input-output interface. Optionally, the communication device 600 may further include a memory 630 for storing instructions executed by the processor 610 or for storing The input data required to execute the instruction or store the data generated by the processor 610 after executing the instruction.

When the communication device 600 is used to implement the method shown in FIG. 2 or FIG. 4 , the processor 610 is used to implement the functions of the above-mentioned processing module 510 , and the interface circuit 620 is used to implement the functions of the above-mentioned transceiver module 520 .

When the communication device 600 is used to implement the method shown in FIG. 2 or FIG. 4 , the communication device 600 includes a processor 610 and an interface circuit 620 . The processor 610 and the interface circuit 620 are coupled to each other. It can be understood that the interface circuit 620 may be a transceiver or an input-output interface. Optionally, the communication device 600 may further include a memory 630 for storing instructions executed by the processor 610 or storing input data required by the processor 610 to execute the instructions or storing data generated after the processor 610 executes the instructions.

When the above-mentioned communication device is a chip applied to a model certification entity, the model certification entity chip implements the function of the model certification entity in the above method embodiment. The model certification entity chip receives information from other modules (such as radio frequency modules or antennas) in the model certification entity, and the information is sent to the model certification entity by the model management entity; or, the model certification entity chip sends information to other modules in the model certification entity Modules (such as radio frequency modules or antennas) send information that is sent by the model authentication entity to the model management entity.

When the above-mentioned communication device is a chip applied to a model management entity, the model management entity chip implements the function of the model management entity in the above method embodiment. The model management entity chip receives information from other modules in the model management entity (such as radio frequency modules or antennas), and the information is sent to the model management entity by the model authentication entity; or, the model management entity sends information to other modules in the model management entity (such as a radio frequency module or an antenna) to send information, which is sent by the model management entity to the model certification entity.

It can be understood that the processor in the embodiments of the present application can be a central processing module (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor can be a microprocessor, or any conventional processor.

In the embodiment of the present application, memory can be random access memory (Random Access Memory, RAM), flash memory, read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable Programmable read-only memory (Erasable PROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art . An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium can be located in the ASIC. In addition, the ASIC can be located in a network device or a terminal device. Certainly, the processor and the storage medium may also exist in the network device or the terminal device as discrete components.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer programs or instructions. When the computer program or instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a terminal device, or other programmable devices. said computer program or The commands may be stored in or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server integrating one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a DVD; and it may also be a semiconductor medium, such as a solid state disk (solid state disk, SSD).

In each embodiment of the present application, if there is no special explanation and logical conflict, the terms and/or descriptions between different embodiments are consistent and can be referred to each other, and the technical features in different embodiments are based on their inherent Logical relationships can be combined to form new embodiments.

It should be understood that in this embodiment of the application, the numbers "first", "second"... are only used to distinguish different objects, such as different network devices, and do not limit the scope of the embodiment of this application. Examples are not limited to this.

It should also be understood that in this application, "when", "if" and "if" all mean that the entity will make corresponding processing under certain objective circumstances, and it is not a limited time, nor does it require the entity to realize when There must be an action of judgment, and it does not mean that there are other restrictions.

It should also be understood that in each embodiment of the present application, "A corresponds to B" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B may also be determined according to A and/or other information.

It should also be understood that the term "and/or" in this article is only an association relationship describing associated objects, indicating that there may be three relationships, for example, A and/or B may indicate: A exists alone, and A and B exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

The meaning of the expression similar to "the item includes one or more of the following: A, B, and C" appearing in this application, unless otherwise specified, usually means that the item can be any of the following: A; B ;C;A and B;A and C;B and C;A,B and C;A and A;A,A and A;A,A and B;A,A and C,A,B and B;A , C and C; B and B, B, B and B, B, B and C, C and C; C, C and C, and other combinations of A, B and C. The above is an example of the three elements of A, B and C to illustrate the optional items of the project. When the expression is "the project includes at least one of the following: A, B, ..., and X", it is in the expression When there are more elements, then the applicable entries for this item can also be obtained according to the aforementioned rules.

It can be understood that the various numbers involved in the embodiments of the present application are only for convenience of description, and are not used to limit the scope of the embodiments of the present application. The size of the serial numbers of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic.

Claims

A method for model management, characterized in that the method comprises:

The model certification entity receives first provider information from the model management entity, the first provider information indicating that the reasoning model is provided by the first model provider;

The model authentication entity compares the first model provider and the device provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the inference model;

The model authentication entity sends an authentication result of the inference model to the model management entity based on the comparison result, and the authentication result indicates whether the inference model can run in the model inference entity.
The method of claim 1, characterized in that:

The result of the comparison is that the first model provider is inconsistent with the device provider, the accuracy of the reasoning model is less than a first threshold, and the authentication result indicates that the reasoning model cannot run in the model reasoning entity ;or,

The result of the comparison is that the first model provider is inconsistent with the equipment provider, the accuracy of the reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the reasoning model can be used in the model run in the inference entity.
The method according to claim 1 or 2, wherein, before the model certification entity compares the first model provider and the equipment provider, the method further comprises:

the model certification entity receives the inference model from the model management entity;

The model certification entity determines that the first model provider is the provider of the inference model.
The method of claim 3, wherein the model certification entity determining that the first model provider is the provider of the inference model comprises:

The model certification entity determines that the first format and the second format are consistent; or,

the model certification entity determines that the second model provider agrees with the first model provider; or,

the model certification entity determines that the first format is consistent with the second format, and that the second model provider is consistent with the first model provider;

Wherein, the first format is the format of the reasoning model, the second format is determined according to the first provider information, and the second model provider is determined according to the reasoning model.
The method according to any one of claims 1 to 4, further comprising:

The model authentication entity sends cause information to the model management entity, the cause information indicating the reason for the authentication result.
The method according to claim 1, wherein the comparison result is that the first model provider is consistent with the device provider, and the authentication result indicates that the reasoning model can be used in the model reasoning entity run.
The method according to any one of claims 1 to 6, wherein, before the model certification entity compares the first model provider and the equipment provider, the method further comprises:

The model certification entity receives second provider information from the model management entity, the second provider information indicating the device provider.
A method for model management, characterized in that the method comprises:

The model management entity sends the first provider information to the model certification entity, and the first provider information indicates that the first pushed The theoretical model is provided by the first model provider;

The model management entity receives an authentication result of the first inference model from the model authentication entity, the authentication result indicating whether the first inference model can run in the model inference entity.
The method of claim 8, wherein:

The first model provider is inconsistent with the device provider of the model inference entity, the accuracy of the first inference model is less than a first threshold, and the authentication result indicates that the first inference model cannot be used in the model running in the inference entity; or,

The first model provider is inconsistent with the device provider, the accuracy of the first reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the first reasoning model can reason in the model running in the entity.
The method according to claim 8 or 9, wherein the method further comprises:

The model management entity sends the first inference model to the model certification entity.
The method according to any one of claims 8 to 10, further comprising:

The model management entity receives reason information from the model certification entity, the reason information indicating a reason for the certification result.
The method according to any one of claims 8 to 11, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further comprises:

The model management entity sends at least one of the following information to the model training entity: the authentication result, cause information and first adjustment information, wherein the cause information indicates the reason for obtaining the authentication result, and the first The adjustment information instructs the model training entity to adjust the first inference model;

The model management entity receives a second inference model from the model training entity, and the second inference model is adjusted based on the first inference model.
The method according to any one of claims 8 to 11, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further comprises:

The model management entity sends the first inference model and second adjustment information to the model inference entity, and the second adjustment information instructs the model inference entity to adjust the first inference model.
The method according to claim 8, wherein the first model provider is consistent with the equipment provider of the model inference entity, and the authentication result indicates that the first inference model can be used in the model inference entity run in.
The method according to any one of claims 8 to 14, wherein before receiving the certification result of the first reasoning model from the model certification entity, the method further comprises:

The model management entity sends second provider information to the model certification entity, where the second provider information indicates a device provider of the model inference entity.
A method for model management, characterized in that the method comprises:

The model management entity sends first provider information to a model certification entity, and the model certification entity receives the first provider information from the model management entity, the first provider information indicating that the first reasoning model is provided by the first provided by the model provider;

The model authentication entity compares the first model provider with the device provider of the model inference entity to generate a comparison result, wherein the model inference entity is an entity that will run the first inference model;

The model authentication entity sends the first inference model to the model management entity based on the comparison result An authentication result, the model management entity receives the authentication result from the model authentication entity, the authentication result indicating whether the first inference model can run in the model inference entity.
The method of claim 16, wherein,

The comparison result is that the first model provider is inconsistent with the equipment provider, the accuracy of the first reasoning model is less than a first threshold, and the authentication result indicates that the first reasoning model cannot be used in the run in the model inference entity; or,

The comparison result is that the first model provider is inconsistent with the equipment provider, the accuracy of the first reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the first reasoning model can Run in the model inference entity.
The method according to claim 1 or 2, wherein, before the model certification entity compares the first model provider and the equipment provider, the method further comprises:

the model management entity sends the first inference model to the model certification entity, and the model certification entity receives the first inference model from the model management entity;

The model certification entity determines that the first model provider is a provider of the first inference model.
The method of claim 18, the model certification entity determining that the first model provider is the provider of the first inference model, comprising:

The model certification entity determines that the first format and the second format are consistent; or,

the model certification entity determines that the second model provider agrees with the first model provider; or,

the model certification entity determines that the first format is consistent with the second format, and that the second model provider is consistent with the first model provider;

Wherein, the first format is the format of the first inference model, the second format is determined according to the first provider information, and the second model provider is determined according to the first inference model of.
The method according to any one of claims 16 to 19, further comprising:

The model certification entity sends reason information to the model management entity, and the model management entity receives the reason information from the model certification entity, the reason information indicating the reason for the certification result.
The method according to any one of claims 16 to 20, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further comprises:

The model management entity sends at least one of the following information to the model training entity, and the model training entity receives at least one of the following information from the model management entity: the authentication result, reason information and first adjustment information, wherein the reason information indicates the reason for obtaining the authentication result, and the first adjustment information indicates that the model training entity adjusts the first inference model;

The model training entity sends a second inference model to the model management entity, and the model management entity receives the second inference model from the model training entity, the second inference model is based on the first inference model Adjusted.
The method according to any one of claims 16 to 20, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the method further comprises:

The model management entity sends the first inference model and the second adjustment information to the model inference entity, and the model inference entity receives the first inference model and the second adjustment information from the model management entity , the second adjustment information instructs the model inference entity to adjust the first inference model.
The method according to claim 16, wherein the comparison result is that the first model provider is consistent with the equipment provider, and the authentication result indicates that the first inference model can be used for reasoning in the model running in the entity.
The method according to any one of claims 16 to 23, wherein, before the model certification entity sends the certification result of the first inference model to the model management entity based on the comparison result, the Methods also include:

The model management entity sends second provider information to the model certification entity, and the model certification entity receives the second provider information from the model management entity, the second provider information indicating that the model The device provider for the inference entity.
A communication device for model management, characterized by comprising:

A transceiver module, configured to receive first provider information from the model management entity, the first provider information indicating that the reasoning model is provided by the first model provider;

A processing module, configured to compare the first model provider with a device provider of a model reasoning entity to generate a comparison result, wherein the model reasoning entity is an entity that will run the reasoning model;

The transceiving module is further configured to send an authentication result of the inference model to the model management entity based on the comparison result, the authentication result indicating whether the inference model can run in the model inference entity.
The communication device according to claim 25, characterized in that:

The comparison result is that the first model provider and the equipment provider are inconsistent, the accuracy of the inference model is less than a first threshold, and the authentication result indicates that the inference model cannot be used in the model inference entity run; or,

The result of the comparison is that the first model provider is inconsistent with the equipment provider, the accuracy of the reasoning model is greater than or equal to the first threshold, and the authentication result indicates that the reasoning model can be used in the model run in the inference entity.
The communication device according to claim 25 or 26, wherein the transceiver module is also used for:

receiving the inference model from the model management entity;

The processing module is also used to:

It is determined that the first model provider is the provider of the inference model.
The communication device according to claim 27, wherein the processing module is specifically used for:

determining that the first format and the second format are consistent; or,

determining that the second model provider is consistent with said first model provider; or,

determining that the first format is consistent with the second format, and that the second model provider is consistent with the first model provider;

Wherein, the first format is the format of the reasoning model, the second format is determined according to the first provider information, and the second model provider is determined according to the reasoning model.
The communication device according to any one of claims 25 to 28, wherein the transceiver module is also used for:

sending cause information to the model management entity, the cause information indicating the reason for the authentication result.
The communication device according to claim 25, wherein the comparison result is that the first model provider is consistent with the equipment provider, and the authentication result indicates that the inference model can be used for inference entities in the model Zhongyun OK.
The communication device according to any one of claims 25 to 30, wherein the transceiver module is also used for:

Receive second provider information from the model management entity, the second provider information indicating the device provider.
A communication device for model management, characterized by comprising:

A processing module, configured to generate first provider information, where the first provider information is used to indicate that the first reasoning model is provided by a first model provider;

A transceiver module, configured to send the first provider information to a model certification entity;

The transceiving module is further configured to receive an authentication result of the first inference model from the model authentication entity, the authentication result indicating whether the first inference model can run in the model inference entity.
The communication device according to claim 32, wherein the first model provider is inconsistent with the equipment provider of the model inference entity, the accuracy of the first inference model is less than a first threshold, and the authentication A result indicates that the first inference model is not capable of running in the model inference entity; or,

The first model provider is inconsistent with the device provider, the accuracy of the first inference model is greater than or equal to the first threshold, and the authentication result indicates that the first inference model can reason in the model running in the entity.
The communication device according to claim 32 or 33, wherein the transceiver module is also used for:

sending the first inference model to the model certification entity.
The communication device according to any one of claims 32 to 34, wherein the transceiver module is also used for:

Reason information is received from the model certification entity, the reason information indicating a reason for the certification result.
The communication device according to any one of claims 32 to 35, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the transceiver module is further used for:

Send at least one of the following information to the model training entity: the authentication result, reason information and first adjustment information, wherein the reason information indicates the reason for obtaining the authentication result, and the first adjustment information indicates the The model training entity adjusts the first inference model;

A second inference model is received from the model training entity, the second inference model being adjusted based on the first inference model.
The communication device according to any one of claims 32 to 36, wherein the authentication result indicates that the first inference model cannot run in the model inference entity, and the transceiver module is further used for:

Sending the first inference model and second adjustment information to a model inference entity, where the second adjustment information instructs the model inference entity to adjust the first inference model.
The communication device according to claim 32, wherein the first model provider is the same as the equipment provider of the model inference entity, and the authentication result indicates that the first inference model can be used for inference in the model running in the entity.
The communication device according to any one of claims 32 to 38, wherein the transceiver module is also used for:

Sending second provider information to the model authentication entity, the second provider information indicating a device provider of the model inference entity.
A communication device, characterized in that it includes at least one processor, and the at least one processor is used to execute a computer program stored in a memory, so that the device implements the communication system described in claims 1 to 7 or claims 8 to 15. The method of any one, or the method of any one of claims 16 to 24.
A computer-readable storage medium, characterized in that it includes a computer program, and when the computer program is run on a computer, it causes the computer to execute the computer according to any one of claims 1 to 7 or claims 8 to 15. The method described in the present invention, or the method described in any one of claims 16 to 24.
A computer program product, characterized in that, when the computer reads and executes the computer program product, the computer executes the method according to any one of claims 1 to 7, or the method according to any one of claims 8 to 15 A method according to any one of claims 16 to 24.
A communication system for model management, characterized by comprising the communication device according to any one of claims 25 to 31 and the communication device according to any one of claims 32 to 39.