CN115131603A

CN115131603A - Model processing method and device, storage medium and electronic equipment

Info

Publication number: CN115131603A
Application number: CN202210586640.XA
Authority: CN
Inventors: 曹佳炯; 丁菁汀
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-09-30

Abstract

The specification discloses a model processing method, a model processing device, a storage medium and an electronic device, wherein the method comprises the following steps: the method includes deploying an initial image processing model to at least one client in a target transaction scene, obtaining at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data, performing model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model, and then performing transaction processing on the target transaction scene based on the image processing model.

Description

Model processing method and device, storage medium and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a model processing method and apparatus, a storage medium, and an electronic device.

Background

With the rapid development of computer technology, the application of image processing to images to be processed in actual transaction scenes such as image recognition, image classification, image segmentation and image privacy protection by adopting an image processing model is more and more extensive, and before the image processing model is used, the image processing model needs to be trained;

in the training process of the image processing model, training data are collected, the image processing model is trained offline based on the training data, and after the training is finished, the trained image processing model is deployed to an actual transaction scene for image processing.

Disclosure of Invention

The specification provides a model processing method, a model processing device, a storage medium and an electronic device, and the technical scheme is as follows:

in a first aspect, the present specification provides a model processing method applied to a service platform, the method including:

deploying an initial image processing model to at least one client in a target transaction scenario, obtaining at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data;

performing model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model;

and performing transaction processing on the target transaction scene based on the image processing model.

In a second aspect, the present specification provides a model processing method, applied to a client, the method including:

acquiring an initial image processing model from a service platform, and deploying the initial image processing model to a target transaction scene;

performing model processing on input image data based on the initial image processing model to obtain model output data;

generating an image processing sample pair based on the input image data and the model output data, sending the image processing sample pair to a service platform, so that the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, and performs transaction processing on the target transaction scene based on the image processing model.

In a third aspect, the present specification provides a model processing apparatus comprising:

a sample acquisition module for deploying an initial image processing model to at least one client in a target transaction scenario, acquiring at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data; the initial image processing model is an image processing model generated by offline training aiming at the online transaction scene model;

a model generation module, configured to perform model calibration processing on the initial image processing model based on the at least one image processing sample pair, and generate an image processing model;

and the model updating module is used for carrying out transaction processing on the target transaction scene based on the image processing model.

In a fourth aspect, the present specification provides a model processing apparatus comprising:

the model deployment module is used for acquiring an initial image processing model from a service platform and deploying the initial image processing model to a target transaction scene;

the model processing module is used for carrying out model processing on input image data based on the initial image processing model to obtain model output data;

and the sample sending module is used for generating an image processing sample pair based on the input image data and the model output data, sending the image processing sample pair to a service platform, so that the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, and performs transaction processing on the target transaction scene based on the image processing model.

In a fifth aspect, the present specification provides a computer storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to carry out the above-mentioned method steps.

In a sixth aspect, the present specification provides a computer program product having stored thereon a plurality of instructions adapted to be loaded by a processor and to carry out the method steps as described above.

In a seventh aspect, this specification provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

The technical scheme provided by some embodiments of the present description brings beneficial effects at least including:

in one or more embodiments of the present specification, a service platform performs model commissioning by deploying an initial image processing model to at least one client in a target transaction scenario, where the client performs image processing on input image data in the target transaction scenario based on the initial image processing model to obtain model output data, and based on the model output data, the service platform may collect at least one image processing sample pair for the initial image processing model from the client, then perform model calibration processing on the initial image processing model using the image processing sample pair to generate an image processing model, and then perform transaction processing on the target transaction scenario based on the image processing model. The image processing model after model calibration can resist the distribution difference between training data and actual on-line data, has good model robustness, realizes the optimization of the model processing process, and effectively enhances the stability and the scene generalization capability of the model after the on-line target transaction scene.

Drawings

In order to more clearly illustrate the technical solutions in the present specification or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic view of a scenario of a model processing system provided herein;

FIG. 2 is a schematic flow diagram of another model processing method provided herein;

FIG. 3 is a schematic flow diagram of another model processing method provided herein;

FIG. 4 is a schematic flow diagram of another model processing method provided herein;

FIG. 5 is a schematic flow diagram of another model processing method provided herein;

FIG. 6 is a schematic diagram of a model processing apparatus provided in the present specification;

FIG. 7 is a schematic structural diagram of a sample acquisition module provided herein;

FIG. 8 is a schematic diagram of a model generation module provided herein;

fig. 9 is a schematic structural diagram of a structural adjustment unit provided in the present specification;

FIG. 10 is a schematic view of another model processing apparatus provided in the present specification;

fig. 11 is a schematic structural diagram of an electronic device provided in this specification;

fig. 12 is a schematic structural diagram of another electronic device provided in the present specification;

FIG. 13 is a schematic diagram of the operating system and user space provided in this specification;

FIG. 14 is an architectural diagram of the android operating system of FIG. 13;

FIG. 15 is an architectural diagram of the IOS operating system of FIG. 13.

Detailed Description

The technical solutions in the present specification will be clearly and completely described below with reference to the accompanying drawings in the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present application, it is noted that, unless explicitly stated or limited otherwise, "including" and "having" and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may alternatively include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The specific meaning of the above terms in this application will be understood to be a specific case for those of ordinary skill in the art. Further, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

The present application will be described in detail with reference to specific examples.

Please refer to fig. 1, which is a schematic view of a scenario of a model processing system provided in the present specification. As shown in FIG. 1, the model processing system may include at least a client cluster and a service platform 100.

The client cluster may include at least one client, as shown in fig. 1, and specifically includes a client 1 corresponding to a user 1, clients 2 and … corresponding to a user 2, and a client n corresponding to a user n, where n is an integer greater than 0.

Each client in the client cluster may be a communication-enabled electronic device including, but not limited to: wearable devices, handheld devices, personal computers, tablet computers, in-vehicle devices, smart phones, computing devices or other processing devices connected to a wireless modem, and the like. Electronic devices in different networks may be called different names, such as: user equipment, access terminal, subscriber unit, subscriber station, mobile station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user equipment, cellular telephone, cordless telephone, Personal Digital Assistant (PDA), electronic device in a 5G network or future evolution network, and the like.

The service platform 100 may be a separate server device, such as: rack, blade, tower or cabinet type server equipment, or hardware equipment with stronger computing power such as a workstation and a large computer; the server cluster can also be a server cluster formed by a plurality of servers, each server in the service cluster can be formed in a symmetrical mode, wherein each server has the equivalent function and the equivalent status in a transaction link, and each server can provide services for the outside independently, and the independent service can be understood as the assistance without other servers.

In one or more embodiments of the present specification, the service platform 100 and at least one client in the client cluster may establish a communication connection, and complete data interaction in the model processing process based on the communication connection, such as online transaction data interaction, for example, if the service platform 100 may transmit the model data of the initial image processing model to the at least one client based on the model processing method of the present specification, the client may deploy the initial image processing model into the target transaction scene; if the client side can perform image processing on the input image data in the target transaction scene based on the initial image processing model to obtain model output data, the client side can transmit image processing sample pairs containing the input image data and the model output data to the service platform through communication connection; for another example, the service platform may transmit the image processing model after calibrating the model based on the image processing sample to a number of clients, the clients may formally bring the image processing model online into a target transaction scenario, and so on.

It should be noted that the service platform 100 establishes a communication connection with at least one client in the client cluster for interactive communication through a network, where the network may be a wireless network including but not limited to a cellular network, a wireless local area network, an infrared network, or a bluetooth network, and the wired network includes but not limited to an ethernet network, a Universal Serial Bus (USB), or a controller area network. In one or more embodiments of the specification, data (e.g., object compressed packets) exchanged over a network is represented using techniques and/or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.

The embodiment of the model processing system provided in this specification and the model processing method in one or more embodiments belong to the same concept, and the execution subject corresponding to the model processing method in one or more embodiments of the specification may be the service platform 100 described above; the execution subject corresponding to the model processing method according to one or more embodiments of the specification may also be an electronic device corresponding to a client, and is specifically determined based on an actual application environment. The embodiment of the model processing system, which embodies the implementation process, can be seen in the following method embodiments, and is not described herein again.

Based on the scene diagram shown in fig. 1, the following describes in detail a model processing method provided in one or more embodiments of the present specification.

Referring to fig. 2, a flow chart of a model processing method, which can be implemented by a computer program and can run on a background investigation apparatus based on von neumann architecture, is provided for one or more embodiments of the present disclosure. The computer program may be integrated into the application or may run as a separate tool-like application. The model processing means may be a service platform.

Specifically, the model processing method includes:

s102: deploying an initial image processing model to at least one client in a target transaction scenario, obtaining at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data.

In an actual transaction scene, further processing of image data generated by the transaction scene is often performed based on an image processing model of machine learning. The image processing model may be applied in different scenes based on different machine vision tasks, that is, the image processing model may be a neural network suitable for different machine vision tasks, for example: further, initial image processing models trained based on different machine vision tasks may be deployed in corresponding target transaction scenarios, initial image processing models trained based on image recognition tasks may be deployed in image recognition transaction scenarios, initial image processing models trained based on image privacy protection tasks may be deployed in image privacy protection transaction scenarios, initial image processing models trained based on image classification tasks may be deployed in image classification transaction scenarios, and so on.

It can be understood that machine learning generally has data distribution differences between offline labeled training data and online transaction data, and in this application: the method comprises the steps of collecting image sample data under a corresponding affair scene as offline image training data in advance, carrying out corresponding label labeling to obtain label training data, carrying out model training on a model based on machine learning by adopting the part of label training data, and training a corresponding image processing model after the condition that an offline model training end condition is met, so as to obtain an initial image processing model.

Illustratively, an initial image processing model may be understood as a trained image processing model that satisfies an off-line model ending training condition. In an actual target transaction scene, an initial image processing model is deployed online and applied to an actual online transaction scene, and usually in an online transaction processing stage after the model is online, the fact that the initial image processing model for offline training and the image data distribution of online service are greatly different can be found, and the initial image processing model can be seriously overfitted by offline training data, so that the output effect of image processing is unstable undoubtedly after the model is deployed online, the processing performance of the model is sharply reduced, and the model processing effect in a daily transaction scene is poor. Such problems can be solved by executing the model processing method referred to in this specification.

Further, the initial image processing model may be created based on fitting of one or more of a Convolutional Neural Network (CNN) model, Deep Neural Network (DNN) model, Recurrent Neural Network (RNN), model, embedding (embedding) model, Gradient Boosting Decision Tree (GBDT) model, Logistic Regression (LR) model, and other machine learning models.

Schematically, a service platform builds an initial machine learning model aiming at a target transaction scene in advance on line, the initial machine learning model is set to adapt to the target transaction scene based on image processing tasks such as an image recognition task, an image classification task, an image privacy protection task and the like, model on-line training is carried out on the initial machine learning model by adopting on-line image sample data, a trained target image processing model is obtained when an on-line model finishing condition is met, then the target image processing model is used as an initial image processing model, and the initial image processing model is deployed on line;

illustratively, the service platform may deploy an initial image processing model to at least one client associated with a target transaction scene, so that the client processes input image data in the target transaction scene based on the initial image processing model to obtain model output data;

optionally, the service platform may construct an online transaction test environment including at least one client, where the online transaction test environment includes a target number (e.g., 100) of clients as a model test device, and after the initial image processing model is calibrated by the online training model, the generated image processing model may be actually deployed on a large scale online, for example, the generated image processing model is distributed to all clients associated with the target transaction scene.

Further, the client receiving the initial image processing model may apply the initial image processing model to a target transaction scene for image processing, where input image data may be generated in the target transaction scene, the number of input images is input data for the initial image processing model, the client may input the input image data into the initial image processing model for image processing, and the initial image processing model may output model output data. Namely, processing the input image data in the target transaction scene based on the initial image processing model to obtain model output data.

It can be understood that the data type of the model output data is determined based on the machine vision task, for example, if the machine vision task is an image recognition task, the model output data is an image recognition result; for example, if the machine vision task is an image privacy protection task, the output data of the model is desensitized images; for example, if the machine vision task is an image classification task, the model output data is an image classification result; for example, if the machine vision task is an image segmentation task, the model output data is the image segmentation result, and so on. The specific type of the model output data is not limited herein, and is determined based on the actual transaction situation.

Further, the service platform may instruct the client to collect at least one set of input image data and model output data of the initial image processing model, and the set of input image data and model output data is an image processing sample pair. The number of image processing sample pairs may be plural.

Optionally, the service platform may obtain at least one image processing sample pair from the client.

Optionally, the plurality of clients may also actively collect image processing sample pairs and then send the image processing sample pairs to the service platform, at this time, the service platform may receive the image processing sample pairs sent by at least one client, where the image processing sample pairs include input image data and model output data.

S104: and performing model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model.

The purpose of the model calibration processing is to enable the initial image processing model to adapt to an online target transaction scene, have certain robustness, have better robustness for resisting online image input data (such as online transaction image data of a user), and improve the model processing effect after the model is online. The model performance difference of the new target transaction scene after the initial image processing model is online is that the characteristic space of the model character under the line and the characteristic space corresponding to the new scene on the line have domain difference, and the domain difference is embodied in the characteristic difference such as image quality, image position and posture and the like. Therefore, the model processing method according to one or more embodiments of the present disclosure may be executed to perform some model calibration on the feature space where the model features are located, so as to adapt to the feature space of the online new target transaction scenario.

In one possible implementation, a part of the model network layer of the initial image processing model may be re-encoded and trained based on at least one image processing sample, so that model features (typically, feature vectors) extracted by the re-encoded and trained model in the model processing process may be adapted into a feature space of an online new target transaction scene.

Optionally, the model architecture parameters of at least one model network layer of the initial image processing model may be adjusted, and then retraining is performed on the adjusted initial image processing model by using the at least one image processing sample until a model end training condition is met, so as to generate the image processing model.

In a possible implementation manner, a model network layer structure is newly added to the initial image processing model, then retraining is performed on the newly added initial image processing model by using the at least one image processing sample, the retraining usually focuses on performing structure parameter updating on the newly added model network layer structure, and the structure parameters of the rest network layers are kept unchanged until a model end training condition is met, so as to generate the image processing model.

In one or more embodiments of the present specification, the model end training condition may include, for example, a value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, and the like. The specific model ending training condition may be determined based on actual conditions, and is not described herein again.

S106: and performing transaction processing on the target transaction scene based on the image processing model.

It will be appreciated that after the model calibration process is performed on the initial image processing model based on the at least one image processing sample, and an image processing model is generated, the image processing model may then be brought online and fit into the target transaction scenario.

Schematically, the service platform can deploy the image processing model to a large batch of clients associated with the target transaction scene, so that the clients can accurately process input image data generated on a line in the target transaction scene based on the image processing model to obtain model output data with good processing effect;

the service platform is used for identifying the online image processing model of the image identification transaction scene, then carrying out image identification on the image transaction data in the target transaction scene based on the image processing model, and outputting an image identification result; taking a machine vision task as an image classification task as an example, the image processing model is used for image classification, the service platform processes the online image processing model on the image classification transaction scene, then performs image classification on the image transaction data in the target transaction scene based on the image processing model, and outputs an image classification result; and taking the machine vision task as an image privacy protection task, wherein the image processing model is used for image privacy protection, the service platform processes the online image processing model on the image privacy protection transaction scene, then performs image desensitization on the image transaction data in the target transaction scene based on the image processing model, outputs a desensitized image, and the like.

In one or more embodiments of the present description, a service platform performs model commissioning by deploying an initial image processing model to at least one client in a target transaction scenario, where the client performs image processing on input image data in the target transaction scenario based on the initial image processing model to obtain model output data, and based on this, the service platform may collect at least one image processing sample pair for the initial image processing model from the client, then perform model calibration processing on the initial image processing model using the image processing sample pair to generate an image processing model, and then perform transaction processing on the target transaction scenario based on the image processing model. The image processing model after model calibration can resist the distribution difference between training data and actual on-line data, has good model robustness, realizes the optimization of the model processing process, and effectively enhances the stability and the scene generalization capability of the model after the on-line target transaction scene.

Referring to fig. 3, fig. 3 is a schematic flow chart diagram of another embodiment of a model processing method according to one or more embodiments of the present disclosure. Specifically, the method comprises the following steps:

s202: deploying an initial image processing model to at least one client in a target transaction scenario, obtaining at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data;

for details, reference may be made to the method steps in other embodiments of the present disclosure, which are not described herein again.

S204: checking and screening the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair;

the verification screening process may be understood as: and verifying the input result of the image processing samples on the service platform, and distinguishing an image processing normal sample pair which accords with the expectation from an image processing normal sample pair which does not accord with the expectation by judging whether each group of input image data and model output data accord with the expectation, namely whether the model output data output by the initial image processing model aiming at the input image data accords with the expectation.

In a possible implementation, an image processing specification for an initial image processing model may be set, and the image processing specification is used to measure whether the model output data meets the expectation of the machine vision task, and different machine vision tasks correspond to different image processing specifications, such as an image recognition task corresponding to the image recognition output specification, and the image recognition output specification is used to measure or even check the image recognition effect of the model output data of the initial image processing model; if the image classification task corresponds to the image classification output specification, the image classification output specification is used for measuring or even checking the image classification effect of the model output data of the initial image processing model; if the image classification task corresponds to the image privacy protection task, the image privacy protection task is used for measuring or even checking the image desensitization effect of the model output data of the initial image processing model, namely judging whether the hidden model output data after the image desensitization meets the privacy protection requirement or not, and the like.

Specifically, the service platform may perform a verification screening process on at least one pair of image processing samples based on the set image processing specification to determine at least one pair of image processing normal samples matching the image processing specification and at least one pair of image processing normal samples not matching the image processing specification.

Illustratively, the service platform can call an expert service, and verify, screen and process each group of image processing samples through the expert based on the image processing standard to detect whether the model output data in the image processing samples meet the image processing standard, and determine the image processing normal sample pairs meeting the image processing standard and the image processing abnormal sample pairs not meeting the image processing standard.

It can be understood that the initial image processing model is deployed to at least one client, the initial image processing model is subjected to trial operation in a target transaction scene, in the process, data of the initial image processing model is collected to perform sample accumulation, namely, image processing samples, then at least one image processing normal sample pair and at least one image processing abnormal sample pair can be obtained through checking and screening processing, and then the model can be subjected to structure adjustment processing based on the samples. The method meets the actual on-line deployment requirement, and improves the robustness of the model and the generalization capability of the scene.

S206: performing structural adjustment processing on the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair to obtain a reference image processing model;

in a possible implementation manner, the service platform may evaluate the performance of the initial image processing model in real time based on the number distribution of the image processing normal sample pairs and the image processing abnormal samples, to perform structural adjustment on the initial image processing model based on the performance evaluation condition, and then perform sample training on the adjusted model. The method comprises the following specific steps:

1. the service platform may determine performance evaluation parameters for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair.

The performance evaluation parameters are used for measuring the performance of the initial image processing model in the current target transaction scene, the fit degree of the initial image processing model and the target transaction scene can be quantified through the performance evaluation parameters, and the performance evaluation parameters are calculated at least based on the number of image processing normal sample pairs and image processing abnormal sample pairs.

Alternatively, the performance evaluation parameter may be based on a quantitative ratio of the image processing normal sample pairs and the image processing abnormal sample pairs, and a larger quantitative ratio indicates a poorer model performance.

Alternatively, the performance evaluation parameter may be a ratio of a first number of image processing exception sample pairs to a second number of total sample pairs (i.e., the number of all image processing normal sample pairs and image processing exception sample pairs), where a larger number ratio indicates a poorer model performance.

Illustratively, the service platform may obtain, in real time or periodically, a first number of pairs of image processing abnormal samples and a second number of pairs of the image processing normal samples; and determining a model run time of the initial image processing model; the model running time is the time length of the initial image processing model deployed to the client.

Illustratively, the service platform then determines a sample accumulation rate based on the first number, the second number, and the model runtime, taking the sample accumulation rate as a performance evaluation parameter for the initial image processing model;

the sample accumulation rate conforms to the following equation:

v＝(N ₁ / ₂ )/

where v is the sample accumulation rate, N ₁ Is said first number, N ₂ For the second number, t is the model runtime.

As can be appreciated, the service platform, by monitoring the cumulative rate of the initial image processing model: counting the number N1 of samples of a data set (such as a processing abnormal sample pair) which does not meet the requirement and the number N2 of samples of a data set (such as a processing normal sample pair) which meets the requirement, and acquiring the trial operation time t corresponding to the part of samples, namely the operation time of the initial image processing model on the client, and calculating to obtain the sample accumulation rate, wherein the sample accumulation rate can intuitively feed back the fit degree of the initial image processing model and a new target transaction scene, and the larger the sample accumulation rate is, the worse the performance of the current initial image processing model in the new target transaction scene is. I.e., to perform different degrees of structural adjustment based on the implementation performance evaluation parameters.

2. The service platform determines a model network adjustment layer aiming at the initial image processing model based on the performance evaluation parameters;

alternatively, a specific type of model network adjustment layer in the initial image processing model may be determined based on the performance evaluation parameters, for example, the model network adjustment layer may be an input layer, a characterization layer, an encoding layer, a decoding layer, and the like.

Optionally, a newly added model network adjustment layer for the initial image processing model may be determined based on the performance evaluation parameters, for example, the newly added model network adjustment layer may be an input layer, a characterization layer, an encoding layer, a decoding layer, and the like.

Different reference evaluation ranges can be set, the reference evaluation ranges correspond to different network structure adjustment strategies, and the network structure adjustment strategies at least comprise network layer adjustment modes (such as a newly added network layer and an adjusted original network layer), network layer types and network layer numbers. And acquiring a network structure adjustment strategy corresponding to the target evaluation range by judging the target evaluation range in which the performance evaluation parameter falls, and then performing structure adjustment processing based on the network structure adjustment strategy. And if a certain network structure adjustment strategy indicates that the structural parameters of the original input layer are optimized, adjusting the indicated structural parameters of the original input layer. And if a network structure adjustment strategy indicates a newly added model network adjustment layer (such as a coding layer), connecting the indicated newly added model network adjustment layer to the initial image processing model.

Here, the specific model configuration adjustment method is not limited, and may be set based on the actual application.

In a specific implementation scenario, the service platform may obtain at least one evaluation reference threshold for the performance evaluation parameter, and then determine at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter;

the number of the evaluation reference threshold values may be plural, and the evaluation reference threshold value is a threshold value or a critical value set for the performance evaluation parameter.

Illustratively, the evaluation reference threshold may comprise at least a first reference threshold and a second reference threshold, in particular determining at least one model network adjustment layer for the initial image processing model, as follows:

if the performance evaluation parameter is smaller than the first reference threshold, determining a first number of model network adjustment layers;

determining a second number of model network adjustment layers if the performance evaluation parameter is greater than or equal to the first reference threshold and less than or equal to the second reference threshold;

and if the performance evaluation parameter is larger than the second reference threshold, determining a third number of model network adjustment layers.

Optionally, the model network adjusting layer may be a model network layer in the initial image processing model, or may be a model network layer that needs to be added on the basis of an original network structure of the initial image processing model. The model network layer type can be a fit of one or more of several network layer types corresponding to the machine learning model involved in the related art, such as a model network can be a fully connected layer (FC layer)

For example, assume that the first reference threshold is denoted as v1 and the second reference threshold is denoted as v2(v1< v 2).

If the performance evaluation parameter-sample cumulative rate v is less than v1, the model network adjustment layer for the initial image processing model is determined to be a first number, e.g., the first number may be one.

If the performance evaluation parameter-sample cumulative rate v is between v1 and v2, then the model network adjustment layer for the initial image processing model is determined to be a second number. Such as two, for example.

If the performance evaluation parameter-sample cumulative rate v is greater than v2, then the model network adjustment layer for the initial image processing model is determined to be a third number. Such as the third number may be three.

It can be understood that the performance evaluation parameter feeds back the matching degree of the current initial image processing model and the new target transaction scene, and it can be understood that the higher the value of the performance evaluation parameter, the lower the matching degree is generally, and based on this, a different number of model network adjustments are required to improve the matching degree of the current model.

3. And carrying out structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model.

It is to be understood that, after determining the model network adjustment layer, the initial image processing model is subjected to the structural adjustment processing based on the corresponding structural adjustment processing manner.

Illustratively, the initial image processing model may be a model built from a codec structure involved in the meter vision task, the initial image processing model including at least an image coding network (e.g., an image Encoder), the above-mentioned model network adjustment layer may be a fully connected layer,

schematically, the service platform adds at least one model network adjustment layer into the initial image processing model to obtain a reference image processing model, and in the specific implementation: at least one fully-connected layer may be connected to the image coding network in the initial image processing model, e.g. after connecting several fully-connected layers to the image coding network, so that a reference image processing model is obtained. The reference image processing model after structure adjustment is subjected to model training by using the sample pairs so as to adjust the structure parameters.

S208: and performing model fine adjustment processing on the reference image processing model by adopting the image processing normal sample pair to generate an image processing model.

It can be understood that, in this specification, only the reference image processing model after model structure adjustment is adapted to a new scene, and parameters of the aforementioned model network adjustment layer are generally updated in a later training stage for the reference image processing model, and parameters of the rest of the model network adjustment layer are kept unchanged;

illustratively, taking a reference image processing model as an encoder-decoder structure as an example, parameters are usually updated only for a model network adjustment layer (such as a full connection layer) adjusted in an image coding network, while parameters of other model layers in the image coding network are kept unchanged.

It can be understood that the image processing normal sample pair is an image processing normal sample pair which is obtained after the verification screening processing is performed on a plurality of image processing sample pairs and accords with the expectation, and the image processing normal sample pair comprises reference input image data and reference model output data.

In a possible implementation manner, the pair of image processing abnormal samples may also be incorporated into the model fine tuning process, and the reference image processing model is subjected to model training based on the image processing abnormal sample labeled with the negative example by labeling the pair of image processing abnormal samples as the negative example, so as to obtain a trained image processing model.

1. And performing feature extraction on the reference model output data through the reference image processing model to obtain image reference features.

Illustratively, taking a reference image processing model as an encoder-decoder structure as an example, the reference image processing model may include at least an image decoding network (such as a decoder) and an image encoding network (an encoder);

the service platform performs feature extraction on the reference model output data through a reference image processing model to obtain image reference features, which may specifically be:

the service platform carries out noise adding processing on the reference model output data and then obtains the processed reference model output data;

and the service platform performs feature extraction on the reference model output data through an image decoding network in a back propagation mode to obtain image reference features.

In the normal sample of image processing, after gaussian noise is added to the reference model output data, the reference input image data and the reference model output data a that satisfy the image processing specification (such as privacy security specification) are fitted back through an image decoding network (such as a decoder) to obtain a corresponding image reference feature f, and the above process may be represented as:

f＝decoder ^-1 (a)

wherein f denotes an image reference feature, a denotes reference model output data, decoder ^-1 () Representing the decoding process of the image decoding network.

2. And obtaining training sample data based on the reference input image data and the image reference characteristics.

It can be understood that the reference input image data and the image reference features f are combined to form training sample data of the image coding network

3. And performing model training on the reference image processing model based on the training sample data and the reference model output data to obtain a trained image processing model.

Illustratively, the reference image processing model includes at least an image encoding network and an image decoding network, and the training sample data includes reference input image data and the image reference features;

in a feasible implementation manner, model training is performed on the reference image processing model based on training sample data to obtain a trained image processing model, which may specifically be:

3.1, in the training process of the image coding network, the service platform carries out network coding training on the image coding network based on reference input image data and image reference characteristics, and determines the coding output characteristics of the image coding network aiming at the reference input image data; namely, inputting reference input image data into a reference image processing model, carrying out forward propagation training through an image coding network of the reference image processing model, then outputting coding output characteristics aiming at the reference input image data, then calculating coding loss based on the image reference characteristics and the coding output characteristics by adopting backward propagation training, and adjusting model parameters based on the coding loss until the training is finished.

Schematically, a service platform performs network parameter updating training on a model network adjusting layer in an image coding network by using a coding loss function based on reference input image data and the image reference characteristics to obtain the trained image coding network; in the specific implementation: inputting reference input image data into a model to obtain coding output characteristics through an image coding network, then calculating coding loss between the coding output characteristics and the image reference characteristics according to a coding loss function, and adjusting parameters of a model network adjusting layer (such as a full connection layer) based on the coding loss, wherein the parameters comprise a weight value and a threshold value of the model network adjusting layer, for example, until the image coding network of the whole reference image processing model reaches a coding network end training condition to finish training, the model coding network converges.

Wherein the coding loss function satisfies the following formula:

and f' is the coding output characteristic of the image coding network aiming at the reference input image data.

Optionally, the coding loss function takes the image reference feature f and the coding output feature f' as function inputs, and calculates a distance (e.g. a euclidean distance) as the decoding loss.

Further, the process of model training the reference image processing model includes a training process of an image coding network and a training process of an image decoding network, and the following explanations are given to the training process of the image decoding network as follows:

3.2, in the training process of each round of image coding network, the coding output characteristic f 'of the reference input image data is used as the input of the image decoding network based on each round of image coding network, the image coding network carries out image decoding processing on the coding output characteristic f' to output decoding output data I ', then the decoding loss is calculated based on the reference model output data I and the decoding output data I', the image decoding network is trained based on the decoding loss until the image decoding network of the whole reference image processing model reaches the coding network end training condition to complete training, and the model decoding network converges. At this time, the trained image processing model can be obtained.

Schematically, the service platform performs network decoding training on the image decoding network by adopting a decoding loss function based on the encoding output characteristic and the image reference characteristic to obtain a trained image decoding network;

the decoding loss function satisfies the following formula:

wherein, the Loss B is decoding Loss, I is the reference model output data, and I' is the decoding output data of the image decoding network.

Optionally, the decoding loss function takes the reference model output data I and the decoding output data I' as function inputs, and calculates a negative direction distance (e.g. euclidean distance) as the decoding loss.

And 3.3, determining a model finishing training condition meeting the image processing model to obtain the trained image processing model.

The process of performing model training on the reference image processing model may include a training process of an image coding network and a training process of an image decoding network, and the training process of the image coding network and the training process of the image decoding network both reach a model end training condition to obtain the trained image processing model.

S210: and performing transaction processing on the target transaction scene based on the image processing model.

Referring to fig. 4, fig. 4 is a schematic flow chart of another embodiment of a model processing method according to one or more embodiments of the present disclosure.

In order to better understand the model processing method designed in this specification, the following explains that the present specification relates to the model processing method by taking a target transaction scene as a privacy desensitization transaction scene and an image processing model as an image desensitization model as an example, and a part of technical description not related to this embodiment may refer to one or more model processing method embodiments, specifically:

in recent years, biometric identification technology has been widely used in the production and life of people. Technologies such as face-brushing payment, face entrance guard, face attendance and face station entry all need to rely on biometric identification. However, as the biometric technology is more and more widely used, the safety problem is more and more prominent. Biometric systems often require the collection, processing, transmission, and storage of biometric information of a user for the purpose of biometric authentication. Each of these links may cause the leakage of biological information (strong privacy information) of the user, and may cause further property and information loss of the user. Therefore, the protection of the biological privacy information of the user is also the privacy desensitization affair scene which is usually involved, the privacy desensitization affair scene can involve desensitization treatment on a biological identification image by using an image processing model to obtain a desensitization image, and the desensitization image does not have the privacy characteristics of the user completely visually, so that the purpose of privacy protection is achieved.

S302: deploying the initial image processing model to at least one client in the target transaction scenario.

It will be appreciated that the initial image processing model is an initial image desensitization model.

Schematically, a service platform builds an initial machine learning model for a privacy desensitization affair scene online in advance, the initial machine learning model is set to adapt to the privacy desensitization affair scene based on an image privacy protection task, model offline training is carried out on the initial machine learning model by adopting offline image sample data, a trained target image processing model, namely a target image desensitization model, is obtained when an offline model finishing condition is met, then the target image desensitization model is used as the initial image desensitization model, and the initial image desensitization model is deployed online;

the privacy protection can be understood as desensitization at a visual/semantic level of the biometric image, and filtering out privacy information.

Illustratively, the service platform may deploy an initial image desensitization model to at least one client associated with a target transaction scene, so that the client processes input image data in the target transaction scene based on the initial image desensitization model to obtain model output data;

s304: obtaining, from a client, sample processing encrypted data for an initial image processing model, the sample processing encrypted data being encrypted data of at least one image processing sample pair, the image processing sample pair comprising input image data and model output data;

transaction scene data accumulation: aiming at a new privacy desensitization transaction scene, firstly, a plurality of clients corresponding to a service platform are deployed, then an initial image desensitization model is on-line, and a period of trial operation is started based on the initial image desensitization model, wherein the initial image desensitization model is used in the period, and does not usually have a cold start characteristic due to objective factors, so that the privacy desensitization transaction scene cannot be directly used for a long time.

Encrypting and uploading sample data: in the commissioning phase (for example, 24-48 hours), the client performs image desensitization on a biological identification original image of a user as input image data by using an initial image desensitization model, and then obtains a privacy protection image, namely model output data; the method comprises the steps that a biological identification original image and a privacy protection image of a client user are used as an image processing sample pair, and encryption is carried out at the same time (so as to ensure basic privacy security) to obtain sample processing encrypted data; the sample processing encrypted data is encrypted data of at least one image processing sample pair;

obtaining encrypted data: the plurality of clients can upload the encrypted sample processing encrypted data to the service platform, and the service platform can obtain the sample processing encrypted data aiming at the initial image processing model from the clients.

It can be understood that, in the trial operation stage (for example, 24 to 48 hours), since trial operation is performed on the initial image desensitization model, since objective factors are small in data interaction flow and short in operation time, a conventional encrypted privacy protection method is used to ensure privacy security, and after the image processing model subsequently generated by the service platform is finished cold start, a privacy protection method based on deep learning can be deployed, that is, a formal privacy desensitization transaction scene of the image processing model is on-line;

cold start problem in one or more embodiments of the present description, a problem of deploying a model for privacy protection to a new scenario and adapting to the new scenario is referred to;

small samples in one or more embodiments of the present disclosure, means that only a small amount of original image data exists in compliance with the privacy protection performance requirement in the case of a new privacy desensitization transaction scenario, that is, the sample processing encrypted data.

S306: decrypting the encrypted data of the sample processing to obtain at least one image processing sample, and performing model calibration processing on the initial image processing model based on the at least one image processing sample to generate an image processing model;

1. checking and screening at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair;

the verification screening process may be understood as: and checking the input result of the image processing samples on the service platform, and distinguishing the image processing normal sample pairs which accord with the expectation from the image processing normal sample pairs which do not accord with the expectation by judging whether each group of input image data and model output data accord with the expectation, namely whether the model output data output by the initial image processing model aiming at the input image data accord with the expectation.

The verification screening process can also be understood as data annotation: data decryption is carried out at a server, the image processing sample pair comprises an original image and a corresponding privacy protection image, and whether the privacy protection image meets the privacy protection requirement or not is judged; distinguishing a data set which meets the privacy protection as an image processing normal sample pair and a data set which does not meet the privacy protection requirement as an image processing abnormal sample pair;

2. and performing structure adjustment processing on the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair to obtain a reference image processing model.

The service platform can evaluate the performance of the initial privacy desensitization model in real time based on the number distribution condition of the image processing normal sample pairs and the image processing abnormal samples, so as to perform structure adjustment on the initial privacy desensitization model based on the performance evaluation condition and then perform sample training on the adjusted model. For example, it may be to calculate the cumulative rate.

And (3) calculating the accumulative rate: counting the number N1 of the samples of the abnormal image processing samples which do not meet the privacy protection requirement, the number N2 of the samples of the normal image processing samples which meet the privacy protection requirement, and the time t of trial operation corresponding to the collected samples, and then calculating to obtain the sample accumulation rate, wherein the larger the rate is, the worse the performance of the current model for privacy protection in a new scene is.

The service platform determines a model network adjustment layer aiming at the initial privacy desensitization model based on the sample accumulation rate;

illustratively, the evaluation reference threshold may comprise at least a first reference threshold and a second reference threshold, in particular determining at least one model network adjustment layer for the initial image processing model,

optionally, the model network adjustment layer may be a model network layer in the initial privacy desensitization model, or may be a model network layer that needs to be added on the basis of an original network structure of the initial image processing model. The model network layer type can be a fit of one or more of several network layer types corresponding to the machine learning model involved in the related art, such as a model network can be a fully connected layer (FC layer)

If the performance evaluation parameter-sample cumulative rate v is less than v1, then determining the model network adjustment layers for the initial privacy desensitization model to be a first number, such as one, followed by two FC layers in an image encoding network (such as an image Encoder Encoder);

if the performance evaluation parameter-sample cumulative rate v is between v1 and v2, then the model network adjustment layer for the initial privacy desensitization model is determined to be a second number. For example, the second number may be two, and may be two FC layers after the image coding network (e.g., image coder Encoder);

if the performance evaluation parameter-sample cumulative rate v is greater than between v2, then the model network adjustment layer for the initial privacy desensitization model is determined to be a third number. For example, the third number may be three, and may be three FC layers after an image encoding network (e.g., an image Encoder Encoder)

Further, the service platform carries out structure adjustment processing on the initial privacy desensitization model based on the model network adjustment layer to obtain a reference privacy desensitization model

Illustratively, the initial privacy-desensitization model may be a model constructed from a codec structure involved in the meter vision task, the initial privacy-desensitization model including at least an image coding network (e.g., an image Encoder), and the model network adjustment layer may be a fully-connected layer.

3. And the service platform then adopts the image processing normal sample to carry out model fine tuning processing on the reference privacy desensitization model, and generates a privacy desensitization model.

Reference may be made to the model fine tuning processing procedure related to other method embodiments in this specification, and details are not repeated here.

S308: carrying out model data encryption on the image processing model to obtain model encrypted data;

it can be understood that after the image processing model, namely the privacy desensitization model, is trained, the privacy desensitization model is encrypted by using an encryption algorithm to obtain model encryption data. To complete cold start deployment in a new scenario.

S310: and sending the model encrypted data to at least one client so that the client performs transaction processing on the image processing model decrypted based on the model encrypted data.

It can be understood that, for a large number of clients (such as all clients) deployed in a new scene, the trained privacy desensitization model is subjected to model encryption, the encrypted service platform is issued to the corresponding client to perform formal online deployment of the model, model decryption is performed on model encrypted data on the client to obtain the privacy desensitization model, and then the privacy desensitization model is utilized to perform privacy protection, so that better privacy protection performance can be obtained on the corresponding new scene.

In a specific implementation scenario, the service platform may instruct the plurality of clients to input the privacy image in the privacy desensitization transaction scenario into the image desensitization model, and perform visual/semantic desensitization on the privacy image by using the image desensitization model to filter out the privacy information, thereby outputting the transaction desensitization image.

The privacy image may be a user image, a biometric image (such as a facial image, a fingerprint image, an extremity image), a photograph, a communication record image, a conference image, a biometric image, and the like, which relate to privacy security, and the specific type of the privacy image may be set based on an actual transaction scene, which is not limited herein.

The transaction desensitization image is output after the privacy information is filtered out by desensitizing the privacy image at a visual/semantic level through the image desensitization model.

In one or more embodiments of the present description, a service platform performs model commissioning by deploying an initial image processing model to at least one client in a privacy-desensitized transaction scenario, where the client performs image processing on input image data in a target transaction scenario based on the initial image processing model to obtain model output data, and based on this, the service platform may collect at least one image processing sample pair for the initial image processing model from the client, then perform model calibration processing on the initial image processing model using the image processing sample pair to generate an image processing model, and then perform transaction processing on the privacy-desensitized transaction scenario based on the image processing model. The image processing model after model calibration can resist the distribution difference between training data and actual on-line data, has good model robustness, realizes the optimization of the model processing process, and effectively enhances the stability and the scene generalization capability of the model after the on-line privacy desensitization transaction scene; and the cold start phenomenon after the initial image processing model of offline training is online is solved, the new scene cold start of recoding training of the small sample model from the client based on the commissioning phase is realized, the privacy information can be prevented from being leaked in the privacy desensitization affair scene, and the privacy protection performance is improved.

Referring to fig. 5, a flow chart of a model processing method, which can be implemented by a computer program and can run on a background investigation apparatus based on von neumann architecture, is provided for one or more embodiments of the present disclosure. The computer program may be integrated into the application or may run as a separate tool-like application. The model processing means may be a client.

S402: acquiring an initial image processing model from a service platform, and deploying the initial image processing model to a target transaction scene;

according to some embodiments, a service platform builds an initial machine learning model for a target transaction scene online in advance, the initial machine learning model is set to adapt to the target transaction scene based on image processing tasks such as an image recognition task, an image classification task, an image privacy protection task and the like, model offline training is performed on the initial machine learning model by using offline image sample data, a trained target image processing model is obtained when an offline model ending condition is met, then the target image processing model is used as an initial image processing model, and the initial image processing model is deployed online;

illustratively, the service platform may deploy the initial image processing model to at least one client associated with the target transaction scenario. At the moment, the client can obtain an initial image processing model from the service platform, and the client processes input image data in a target transaction scene based on the initial image processing model to obtain model output data;

optionally, by constructing an online transaction test environment including at least one client, the clients including a target number (e.g., 100) in the online transaction test environment are used as model test devices, and after the initial image processing model is calibrated by the online training model, the generated image processing model may be actually deployed on a large scale online, for example, the generated image processing model is distributed to all clients associated with the target transaction scene.

S404: performing model processing on input image data based on the initial image processing model to obtain model output data;

further, the client may apply the initial image processing model to a target transaction scene for image processing, where input image data may be generated in the target transaction scene, the number of input images is input data for the initial image processing model, the client may input the input image data to the initial image processing model for image processing, and the initial image processing model may output model output data. Namely, processing the input image data in the target transaction scene based on the initial image processing model to obtain model output data.

S406: generating an image processing sample pair based on the input image data and the model output data, sending the image processing sample pair to a service platform, so that the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, and performs transaction processing on the target transaction scene based on the image processing model.

It is to be understood that the client collects at least one set of input image data and model output data of the initial image processing model, and the set of input image data and model output data is a pair of image processing samples. The number of image processing sample pairs may be plural.

Illustratively, the client may also actively collect image processing sample pairs and then send the image processing sample pairs to the service platform, and at this time, the service platform may receive at least one image processing sample pair sent by the client, where the image processing sample pairs include input image data and model output data.

Further, the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample to generate an image processing model, and after the image processing model is generated, the image processing model can be online and adapted to a target transaction scene.

It can be understood that the service platform can deploy the image processing model to a large number of clients associated with the target transaction scene, and at this time, the clients can accurately process input image data generated on a line in the target transaction scene based on the image processing model to obtain model output data with a good processing effect.

The model processing apparatus provided in the present specification will be described in detail below with reference to fig. 6. It should be noted that, the model processing apparatus shown in fig. 6 is used for executing the method of one or more embodiments of the model processing method shown in this specification, for convenience of description, only the part related to this specification is shown, and specific technical details are not disclosed, please refer to the embodiment of one or more embodiments of the model processing method shown in this specification.

Referring to fig. 6, a schematic structural diagram of a model processing apparatus of the present specification is shown. The model processing apparatus 1 may be implemented as all or a part of a user terminal by software, hardware, or a combination of both. According to some embodiments, the model processing apparatus 1 comprises a model processing module 11, a model processing module 12 and a model processing module 13, and is specifically configured to:

a sample acquiring module 11, configured to deploy an initial image processing model to at least one client in a target transaction scenario, and acquire at least one image processing sample pair for the initial image processing model from the client, where the image processing sample pair includes input image data and model output data; the initial image processing model is an image processing model generated by offline training aiming at the online transaction scene model;

a model generation module 12, configured to perform model calibration processing on the initial image processing model based on the at least one image processing sample pair, and generate an image processing model;

and the model updating module 13 is configured to perform transaction processing on the target transaction scene based on the image processing model.

Optionally, as shown in fig. 7, the sample acquiring module 11 includes:

the offline training unit 111 is used for performing model offline training on the target transaction scene to obtain a trained reference image processing model;

a model deployment unit 112, configured to deploy the reference image processing model as an initial image processing model to at least one client in a target transaction scene, so that the client processes input image data in the target transaction scene based on the initial image processing model to obtain model output data;

a sample obtaining unit 113, configured to receive at least one image processing sample pair sent by the client, where the image processing sample pair includes the input image data and the model output data.

Optionally, as shown in fig. 8, the model generating module 12 includes:

the verification screening unit 121 is configured to perform verification screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair;

a structure adjusting unit 122, configured to perform structure adjustment processing on the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples, so as to obtain a reference image processing model;

and a model fine-tuning unit 123, configured to perform model fine-tuning processing on the reference image processing model by using the image processing normal sample pair, so as to generate an image processing model.

Optionally, the verification screening unit 121 is specifically configured to:

and performing verification screening processing on the at least one image processing sample pair based on the image processing specification to determine at least one image processing normal sample pair matching the image processing specification and at least one image processing normal sample pair not matching the image processing specification.

Optionally, as shown in fig. 9, the structure adjusting unit 122 is configured to:

a parameter determination subunit 1221 configured to determine a performance evaluation parameter for the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples;

a structure adjustment subunit 1222 to determine a model network adjustment layer for the initial image processing model based on the performance evaluation parameters;

the structure adjusting subunit 1222 is further configured to perform structure adjustment processing on the initial image processing model based on the model network adjusting layer, so as to obtain a reference image processing model.

Optionally, the parameter determining subunit 1221 is specifically configured to

Acquiring a first number of the image processing abnormal sample pairs and a second number of the image processing normal sample pairs;

determining a model run time of the initial image processing model;

determining a sample accumulation rate based on the first quantity, the second quantity, and the model run time, the sample accumulation rate serving as a performance evaluation parameter for the initial image processing model;

the sample accumulation rate conforms to the following equation:

v＝(N ₁ /N ₂ )/t

Optionally, the structure adjusting subunit 1222 is specifically configured to:

obtaining at least one evaluation reference threshold for the performance evaluation parameter;

determining at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter;

the configuration adjustment subunit 1222 is further configured to: and adding the at least one model network adjusting layer into the initial image processing model to obtain a reference image processing model.

Optionally, the at least one evaluation reference threshold includes at least a first reference threshold and a second reference threshold, and the structure adjustment subunit 1222 is specifically configured to:

determining a third number of model network adjustment layers if the performance evaluation parameter is greater than the second reference threshold.

Optionally, the initial image processing model at least includes an image coding network, the model network adjusting layer is a fully connected layer, and the structure adjusting subunit 1222 is specifically configured to: and connecting the at least one full connection layer with the image coding network to obtain a reference image processing model.

Optionally, the image processing normal sample pair includes reference input image data and reference model output data, and the model fine-tuning unit 123 is configured to:

performing feature extraction on the reference model output data through the reference image processing model to obtain image reference features;

obtaining training sample data based on the reference input image data and the image reference characteristics;

and performing model training on the reference image processing model based on the training sample data and the reference model output data to obtain a trained image processing model.

Optionally, the reference image processing model at least includes an image decoding network, and the model fine-tuning unit 123 is specifically configured to: noise adding processing is carried out on the reference model output data to obtain the processed reference model output data;

and performing feature extraction on the reference model output data by adopting a back propagation mode through the image decoding network to obtain image reference features.

Optionally, the image processing model at least includes an image coding network and an image decoding network, the training sample data includes reference input image data and the image reference feature, and the model fine-tuning unit 123 is configured to:

performing network coding training on the image coding network based on the reference input image data and the image reference features, and determining coding output features of the image coding network for the reference input image data;

performing network decoding training on the image decoding network based on the encoded output features and the reference model output data;

and determining a model finishing training condition meeting the image processing model to obtain the trained image processing model.

Optionally, the model fine tuning unit 123 is configured to: performing network parameter updating training on a model network adjusting layer in the image coding network by adopting a coding loss function based on the reference input image data and the image reference characteristics to obtain the trained image coding network;

the coding loss function satisfies the following formula:

wherein Loss A is coding Loss, f is an image reference characteristic, and f' is a coding output characteristic of the image coding network for the reference input image data.

Optionally, the model fine tuning unit 123 is configured to: performing network decoding training on the image decoding network by adopting a decoding loss function based on the encoding output characteristics and the reference model output data to obtain the trained image decoding network;

the decoding loss function satisfies the following formula:

and the Loss B is decoding Loss, the I is the output data of the reference model, and the I' is the decoding output data of the image decoding network.

Optionally, the apparatus 1 is specifically configured to:

obtaining, from the client, sample processing encrypted data for the initial image processing model, the sample processing encrypted data being encrypted data of at least one image processing sample pair;

the performing transaction processing on the target transaction scene based on the image processing model comprises:

carrying out model data encryption on the image processing model to obtain model encrypted data;

and sending the model encrypted data to at least one client so that the client performs transaction processing on the image processing model decrypted based on the model encrypted data.

Optionally, the target transaction scene is a privacy desensitization transaction scene, and the image processing model is an image desensitization model; the device 1 is particularly configured to:

and inputting the privacy image in the privacy desensitization transaction scene into the image desensitization model, and outputting a transaction desensitization image.

It should be noted that, when the model processing apparatus provided in the foregoing embodiment executes the model processing method, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the functions described above. In addition, the model processing apparatus and the model processing method provided in the above embodiments belong to the same concept, and details of implementation processes thereof are referred to in the method embodiments and are not described herein again.

The above-mentioned serial numbers are for description purposes only and do not represent the merits of the embodiments.

In this specification, a service platform deploys an initial image processing model to at least one client in a target transaction scene to perform model commissioning, the client performs image processing on input image data in the target transaction scene based on the initial image processing model to obtain model output data, based on which the service platform may collect at least one image processing sample pair for the initial image processing model from the client, then performs model calibration processing on the initial image processing model by using the image processing sample pair to generate an image processing model, and then may perform transaction processing on the target transaction scene based on the image processing model. The image processing model after model calibration can resist the distribution difference between training data and actual on-line data, has good model robustness, realizes the optimization of the model processing process, and effectively enhances the stability and the scene generalization capability of the model after an on-line target transaction scene.

Referring to fig. 10, a schematic structural diagram of another model processing apparatus of the present specification is shown. The model processing means 2 may be implemented as all or part of a user terminal by software, hardware or a combination of both. According to some embodiments, the model processing apparatus 2 includes a sample obtaining module 21, a model generating module 22, and a model updating module 23, and is specifically configured to:

a sample obtaining module 21, configured to deploy an initial image processing model to at least one client in a target transaction scene, obtain at least one image processing sample pair for the initial image processing model from the client, where the image processing sample pair includes input image data and model output data; the initial image processing model is an image processing model generated by off-line training aiming at the on-line transaction scene model;

a model generation module 22, configured to perform a model calibration process on the initial image processing model based on the at least one image processing sample pair, and generate an image processing model;

and the model updating module 23 is configured to perform transaction processing on the target transaction scene based on the image processing model.

The above serial numbers are for description only and do not represent the merits of the embodiments.

In this specification, a service platform deploys an initial image processing model to at least one client in a target transaction scene to perform model commissioning, the client performs image processing on input image data in the target transaction scene based on the initial image processing model to obtain model output data, based on which the service platform may collect at least one image processing sample pair for the initial image processing model from the client, then performs model calibration processing on the initial image processing model by using the image processing sample pair to generate an image processing model, and then may perform transaction processing on the target transaction scene based on the image processing model. The image processing model after model calibration can resist the distribution difference between training data and actual on-line data, has good model robustness, realizes the optimization of the model processing process, and effectively enhances the stability and the scene generalization capability of the model after the on-line target transaction scene.

The present specification further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executing the model processing method according to the embodiment shown in fig. 1 to 5, and a specific execution process may refer to specific descriptions of the embodiment shown in fig. 1 to 5, which are not described herein again.

The present application further provides a computer program product, where at least one instruction is stored, and the at least one instruction is loaded by the processor and executes the model processing method according to the embodiment shown in fig. 1 to 5, and a specific execution process may refer to specific descriptions of the embodiment shown in fig. 1 to 5, which is not described herein again.

Please refer to fig. 11, which is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 11, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002. The communication bus 1002 is used to implement connection communication among these components. The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others. Processor 1001 may include one or more processing cores, among other things. The processor 1001 connects various parts throughout the server 1000 using various interfaces and lines, and performs various functions of the server 1000 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or a combination of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1001, but may be implemented by a single chip.

The Memory 1005 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 may be used to store an instruction, a program, code, a set of codes, or a set of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 11, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an application program.

In the electronic device 1000 shown in fig. 11, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; and the processor 1001 may be configured to call an application stored in the memory 1005, and specifically perform the following operations:

In one embodiment, the processor 1001, when executing the deploying of the initial image processing model to at least one client in an online transaction scenario, obtains at least one image processing sample pair for the initial image processing model from the client, specifically executes the following steps:

performing model offline training on a target transaction scene to obtain a trained reference image processing model;

deploying the reference image processing model as an initial image processing model to at least one client in a target transaction scene, so that the client processes input image data in the target transaction scene based on the initial image processing model to obtain model output data;

receiving at least one image processing sample pair sent by the client, wherein the image processing sample pair comprises the input image data and the model output data.

In an embodiment, when the processor 1001 performs the model calibration process on the initial image processing model based on the at least one image processing sample to generate the image processing model, specifically performs the following steps:

checking and screening the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair;

performing structural adjustment processing on the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair to obtain a reference image processing model;

and performing model fine adjustment processing on the reference image processing model by adopting the image processing normal sample pair to generate an image processing model.

In an embodiment, when the processor 1001 performs the verification and screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and at least one image processing abnormal sample pair, the following steps are specifically performed:

In an embodiment, when the processor 1001 performs the structure adjustment processing on the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples to obtain a reference image processing model, the following steps are specifically performed:

determining performance evaluation parameters for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair;

determining a model network adjustment layer for the initial image processing model based on the performance evaluation parameters;

and carrying out structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model.

In one embodiment, when the processor 1001 determines the performance evaluation parameter for the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples, the following steps are specifically performed:

determining a model run time of the initial image processing model;

the sample accumulation rate conforms to the following equation:

v＝(N ₁ /N ₂ )/t

In one embodiment, the processor 1001, when executing the determining of the model network adjustment layer for the initial image processing model based on the performance evaluation parameter, specifically executes the following steps:

the performing structure adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model, including:

and adding the at least one model network adjusting layer into the initial image processing model to obtain a reference image processing model.

In one embodiment, the processor 1001, in executing the at least one evaluation reference threshold, comprises at least a first reference threshold and a second reference threshold,

when determining at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter, specifically performing the following steps:

In an embodiment, the initial image processing model at least includes an image coding network, the model network adjustment layer is a fully connected layer, and when the processor 1001 adds the at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model, the following steps are specifically performed:

and connecting the at least one full connection layer with the image coding network to obtain a reference image processing model.

In one embodiment, the image processing normal sample pair includes reference input image data and reference model output data, and the processor 1001 specifically performs the following steps when performing model fine-tuning processing on the reference image processing model by using the image processing normal sample pair to generate an image processing model:

performing feature extraction on the output data of the reference model through the reference image processing model to obtain image reference features;

In one embodiment, the reference image processing model at least includes an image decoding network, and when the processor 1001 performs the feature extraction on the reference model output data through the reference image processing model to obtain an image reference feature, the following steps are specifically performed:

noise adding processing is carried out on the reference model output data to obtain the processed reference model output data;

In one embodiment, the image processing model at least includes an image coding network and an image decoding network, the training sample data includes reference input image data and the image reference features, and the processor 1001 specifically performs the following steps when performing the model training on the reference image processing model based on the training sample data and the reference model output data to obtain a trained image processing model:

and determining a model ending training condition meeting the image processing model to obtain the trained image processing model.

In one embodiment, when performing the network coding training on the image coding network based on the reference input image data and the image reference features, the processor 1001 specifically performs the following steps:

performing network parameter updating training on a model network adjusting layer in the image coding network by adopting a coding loss function based on the reference input image data and the image reference characteristics to obtain the trained image coding network;

the coding loss function satisfies the following formula:

In one embodiment, when the processor 1001 performs the network decoding training on the image decoding network based on the encoded output features and the reference model output data, specifically performs the following steps:

performing network decoding training on the image decoding network by adopting a decoding loss function based on the coding output characteristics and the reference model output data to obtain the trained image decoding network;

the decoding loss function satisfies the following formula:

In one embodiment, the processor 1001, when executing the obtaining of the at least one image processing sample pair for the initial image processing model from the client, specifically performs the following steps:

In one embodiment, the target transaction scenario is a privacy-desensitized transaction scenario, and the image processing model is an image desensitization model; when executing the transaction processing on the target transaction scene based on the image processing model, the processor 1001 specifically executes the following steps:

Referring to fig. 12, a block diagram of an electronic device according to an exemplary embodiment of the present application is shown. The electronic device in the present application may comprise one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected by a bus 150.

Processor 110 may include one or more processing cores. The processor 110 connects various parts within the overall electronic device using various interfaces and lines, and performs various functions of the electronic device 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120 and calling data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-programmable gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.

The Memory 120 may include a Random Access Memory (RAM) or a read-only Memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium. The memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, and the like), instructions for implementing various method embodiments described below, and the like, and the operating system may be an Android (Android) system, including a system based on Android system depth development, an IOS system developed by apple, including a system based on IOS system depth development, or other systems. The data storage area may also store data created by the electronic device during use, such as phone books, audio and video data, chat log data, and the like.

Referring to fig. 13, the memory 120 may be divided into an operating system space and a user space, wherein the operating system is run in the operating system space, and the native and third-party applications are run in the user space. In order to ensure that different third-party application programs can achieve a better operation effect, the operating system allocates corresponding system resources for the different third-party application programs. However, the requirements of different application scenarios in the same third-party application program on system resources are different, for example, in a local resource loading scenario, the third-party application program has a higher requirement on the disk reading speed; in the animation rendering scene, the third-party application program has a high requirement on the performance of the GPU. The operating system and the third-party application program are independent from each other, and the operating system cannot sense the current application scene of the third-party application program in time, so that the operating system cannot perform targeted system resource adaptation according to the specific application scene of the third-party application program.

In order to enable the operating system to distinguish a specific application scenario of the third-party application program, data communication between the third-party application program and the operating system needs to be opened, so that the operating system can acquire current scenario information of the third-party application program at any time, and further perform targeted system resource adaptation based on the current scenario.

Taking an operating system as an Android system as an example, programs and data stored in the memory 120 are as shown in fig. 14, and a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360, and an application layer 380 may be stored in the memory 120, where the Linux kernel layer 320, the system runtime library layer 340, and the application framework layer 360 belong to an operating system space, and the application layer 380 belongs to a user space. The Linux kernel layer 320 provides underlying drivers for various hardware of the electronic device, such as a display driver, an audio driver, a camera driver, a bluetooth driver, a Wi-Fi driver, power management, and the like. The system runtime library layer 340 provides a main feature support for the Android system through some C/C + + libraries. For example, the SQLite library provides support for a database, the OpenGL/ES library provides support for 3D drawing, the Webkit library provides support for a browser kernel, and the like. Also provided in the system runtime library layer 340 is an Android runtime library (Android runtime), which mainly provides some core libraries that can allow developers to write Android applications using the Java language. The application framework layer 360 provides various APIs that may be used in building an application, and developers may build their own applications by using these APIs, such as activity management, window management, view management, notification management, content provider, package management, session management, resource management, and location management. At least one application program runs in the application layer 380, and the application programs may be native application programs carried by the operating system, such as a contact program, a short message program, a clock program, a camera application, and the like; or a third-party application developed by a third-party developer, such as a game application, an instant messaging program, a photo beautification program, and the like.

Taking an operating system as an IOS system as an example, programs and data stored in the memory 120 are shown in fig. 15, and the IOS system includes: a Core operating system Layer 420(Core OS Layer), a Core Services Layer 440(Core Services Layer), a Media Layer 460(Media Layer), and a touchable Layer 480(Cocoa Touch Layer). The kernel operating system layer 420 includes an operating system kernel, drivers, and underlying program frameworks that provide functionality closer to hardware for use by program frameworks located in the core services layer 440. The core services layer 440 provides system services and/or program frameworks, such as a Foundation framework, an account framework, an advertisement framework, a data storage framework, a network connection framework, a geographic location framework, a motion framework, and so forth, as required by the application. The media layer 460 provides audiovisual related interfaces for applications, such as graphics image related interfaces, audio technology related interfaces, video technology related interfaces, audio video transmission technology wireless playback (AirPlay) interfaces, and the like. Touchable layer 480 provides various common interface-related frameworks for application development, and touchable layer 480 is responsible for user touch interaction operations on the electronic device. Such as a local notification service, a remote push service, an advertising framework, a game tool framework, a messaging User Interface (UI) framework, a User Interface UIKit framework, a map framework, and so forth.

In the framework shown in FIG. 15, the framework associated with most applications includes, but is not limited to: a base framework in the core services layer 440 and a UIKit framework in the touchable layer 480. The base framework provides many basic object classes and data types, provides the most basic system services for all applications, and is UI independent. While the class provided by the UIKit framework is a basic library of UI classes for creating touch-based user interfaces, iOS applications can provide UIs based on the UIKit framework, so it provides an infrastructure for applications for building user interfaces, drawing, processing and user interaction events, responding to gestures, and the like.

The Android system may be referred to as a mode and a principle for implementing data communication between the third-party application program and the operating system in the IOS system, and details are not repeated herein.

The input device 130 is used for receiving input instructions or data, and the input device 130 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used for outputting instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 are touch display screens for receiving touch operations of a user on or near the touch display screens by using any suitable object such as a finger, a touch pen, and the like, and displaying user interfaces of various applications. Touch displays are typically provided on the front panel of an electronic device. The touch display screen may be designed as a full-face screen, a curved screen, or a profiled screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, and a combination of a special screen and a curved screen, which is not limited in this specification.

In addition, those skilled in the art will appreciate that the configurations of the electronic devices illustrated in the above-described figures do not constitute limitations on the electronic devices, which may include more or fewer components than illustrated, or some components may be combined, or a different arrangement of components. For example, the electronic device further includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a bluetooth module, and other components, which are not described herein again.

In this specification, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is an operating system of the electronic device. The operating system may be an android system, an IOS system, or another operating system, which is not limited in this specification.

The electronic device of this specification may further have a display device mounted thereon, and the display device may be various devices that can implement a display function, for example: a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a Liquid Crystal Display (LCD), a Plasma Display Panel (PDP), and the like. A user may utilize a display device on the electronic device 101 to view information such as displayed text, images, video, and the like. The electronic device may be a smartphone, a tablet computer, a gaming device, an AR (Augmented Reality) device, an automobile, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as an electronic watch, an electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, an electronic garment, or the like.

In the electronic device shown in fig. 12, where the electronic device may be a terminal, the processor 110 may be configured to call an application program stored in the memory 120, and specifically perform the following operations:

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. A model processing method is applied to a service platform and comprises the following steps:

2. The method of claim 1, the deploying an initial image processing model to at least one client in an online transaction scenario, obtaining at least one image processing sample pair for the initial image processing model from the client, comprising:

3. The method of claim 1, the model calibration processing of the initial image processing model based on the at least one image processing sample, generating an image processing model, comprising:

checking and screening the at least one image processing sample pair to obtain at least one image processing normal sample pair and at least one image processing abnormal sample pair;

4. The method of claim 3, wherein the performing a verification screening process on the at least one image processing sample pair to obtain at least one image processing normal sample pair and at least one image processing abnormal sample pair comprises:

and performing verification screening processing on the at least one image processing sample pair based on the image processing specification to determine at least one image processing normal sample pair matched with the image processing specification and at least one image processing normal sample pair not matched with the image processing specification.

5. The method of claim 3, wherein performing a structural adjustment process on the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples to obtain a reference image processing model, comprises:

determining performance evaluation parameters for the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples;

6. The method of claim 5, said determining performance evaluation parameters for the initial image processing model based on the pair of image processing normal samples and the pair of image processing abnormal samples, comprising:

determining a model run time of the initial image processing model;

the sample accumulation rate conforms to the following equation:

v＝(N ₁ /N ₂ )/t

7. The method of claim 5, the determining a model network adjustment layer for the initial image processing model based on the performance evaluation parameters, comprising:

8. The method of claim 7, the at least one evaluation reference threshold comprising at least a first reference threshold and a second reference threshold,

said determining at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter comprises:

9. The method of claim 7, the initial image processing model comprising at least an image coding network, the model network adjustment layer being a fully connected layer,

adding the at least one model network adjustment layer into the initial image processing model to obtain a reference image processing model, comprising:

10. The method of claim 3, the image processing normal sample pair comprising reference input image data and reference model output data,

the step of performing model fine-tuning processing on the reference image processing model by using the image processing normal sample pair to generate an image processing model comprises the following steps:

11. The method of claim 10, the reference image processing model comprising at least an image decoding network,

the extracting the features of the reference model output data through the reference image processing model to obtain the image reference features comprises:

12. The method of claim 10, the image processing model comprising at least an image encoding network and an image decoding network, the training sample data comprising reference input image data and the image reference features,

performing model training on the reference image processing model based on the training sample data and the reference model output data to obtain a trained image processing model, including:

13. The method of claim 12, the network coding training the image coding network based on the reference input image data and the image reference features, comprising:

the coding loss function satisfies the following formula:

14. The method of claim 12, the network decoding training the image decoding network based on the encoded output features and the reference model output data, comprising:

performing network decoding training on the image decoding network by adopting a decoding loss function based on the encoding output characteristics and the reference model output data to obtain the trained image decoding network;

the decoding loss function satisfies the following formula:

15. The method of claim 1, the obtaining at least one image processing sample pair for the initial image processing model from the client, comprising:

16. The method of any of claims 1-15, the target transaction scenario being a privacy-desensitized transaction scenario, the image processing model being an image desensitization model;

17. A model processing method is applied to a client, and comprises the following steps:

18. A model processing apparatus, the apparatus comprising:

a sample acquisition module for deploying an initial image processing model to at least one client in a target transaction scenario, acquiring at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data; the initial image processing model is an image processing model generated by off-line training aiming at the on-line transaction scene model;

19. A model processing apparatus, the apparatus comprising:

20. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps according to any of claims 1-16 or 17.

21. A computer program product having stored at least one instruction for being loaded by said processor and for performing the method steps according to any of claims 1-16 or 17.

22. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-16 or 17.