CN115170840B

CN115170840B - Data processing system, method and electronic equipment

Info

Publication number: CN115170840B
Application number: CN202211092426.5A
Authority: CN
Inventors: 吴飞; 吕喆奇; 王峰; 杨红霞; 况琨
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-09-08
Filing date: 2022-09-08
Publication date: 2022-12-23
Anticipated expiration: 2042-09-08
Also published as: CN115170840A

Abstract

The application provides a data processing system, a data processing method and electronic equipment. A parameter generation model is deployed on cloud-side equipment in the system, and a lightweight model is deployed on end-side equipment. And the end-side equipment transmits sample data to the cloud-side equipment, wherein the sample data comprises an image sample or a behavior data sample. The cloud side equipment inputs the sample data into the parameter generation model to obtain a first parameter of the lightweight model, and sends the first parameter to the end side equipment, so that the end side equipment performs image recognition or behavior prediction on the sample data based on the model parameter of the lightweight model, and the model parameter of the lightweight model comprises the first parameter. In the data processing system, the cloud-side parameter generation model can dynamically update the dynamic parameters of the lightweight model on the end-side equipment according to the sample data input by the end-side equipment in real time, so that the end-side equipment can adjust the end-side lightweight model in time according to the change of the end-side scene data, and the generalization capability of the lightweight model of the end-side equipment is improved.

Description

Data processing system, method and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing system, a method, and an electronic device.

Background

In recent years, with the progress and development of mobile-end devices and lightweight neural networks, it has become practical to deploy deep learning models on mobile-end devices. For example, a lightweight computer vision model may be deployed on a smart security camera for facial recognition. For another example, a lightweight recommendation model may be deployed on a smartphone for e-commerce product recommendations.

The lightweight model can theoretically achieve good performance on the mobile terminal device, but when the models are deployed in practical application, the following problems exist: model performance degrades in scenarios where data changes rapidly. For example, in a computer vision task, the light and angle of an image input to a security camera are constantly changing, and for example, in a recommendation task, a user may generate different preferences in a short time.

Disclosure of Invention

Aspects of the present application provide a data processing system, a method and an electronic device, so as to solve the problem that a lightweight model deployed in a mobile terminal device at present has performance degradation in a scene where data changes rapidly.

In a first aspect, an embodiment of the present application provides a data processing system, including: the cloud side equipment is provided with a parameter generation model, and the end side equipment is provided with a lightweight model;

the end-side device is used for sending sample data acquired by the end-side device in real time to the cloud-side device, wherein the sample data comprises an image sample or a behavior data sample;

the cloud side equipment is used for inputting the sample data into the parameter generation model to obtain a first parameter of the lightweight model; sending the first parameter to the end-side device;

the end-side device is further used for carrying out image recognition or behavior prediction on the sample book data based on the model parameters of the lightweight model;

the model parameters of the lightweight model comprise the first parameters, and the first parameters are dynamic parameters.

In a second aspect, an embodiment of the present application provides a data processing method, which is applied to a cloud-side device, where a parameter generation model is deployed on the cloud-side device, the parameter generation model is used to generate a first parameter of a lightweight model, the first parameter is a dynamic parameter, the lightweight model is deployed on an end-side device, and the lightweight model is used for image recognition or behavior prediction; the method comprises the following steps:

receiving sample data acquired in real time from the end-side device, wherein the sample data comprises an image sample or a behavior data sample;

inputting the sample data into the parameter generation model to obtain a first parameter of the lightweight model;

and sending the first parameter to the end-side equipment so that the end-side equipment performs image recognition or behavior prediction on the sample data based on the model parameter of the lightweight model, wherein the model parameter of the lightweight model comprises the first parameter.

In a third aspect, an embodiment of the present application provides a data processing method, which is applied to an end-side device, where a lightweight model is deployed on the end-side device, and the lightweight model is used for image recognition or behavior prediction; the method comprises the following steps:

sending sample data acquired by the end-side equipment in real time to cloud-side equipment, wherein the sample data comprises an image sample or a behavior data sample;

receiving a first parameter from the cloud-side equipment, wherein the first parameter is a dynamic parameter of the lightweight model generated by a parameter generation model deployed on the cloud-side equipment based on the sample data;

and performing image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model, wherein the model parameters of the lightweight model comprise the first parameters.

In a fourth aspect, an embodiment of the present application provides a data processing apparatus, including:

the receiving module is used for receiving sample data acquired in real time from the end-side equipment, wherein the sample data comprises an image sample or a behavior data sample;

the processing module is used for inputting the sample data into a parameter generation model to obtain a first parameter of the lightweight model;

a sending module, configured to send the first parameter to the end-side device, so that the end-side device performs image recognition or behavior prediction on the sample data based on a model parameter of the lightweight model, where the model parameter of the lightweight model includes the first parameter.

In a fifth aspect, an embodiment of the present application provides a data processing apparatus, including:

the cloud side equipment comprises a sending module, a receiving module and a processing module, wherein the sending module is used for sending sample data acquired by the cloud side equipment in real time to the cloud side equipment, and the sample data comprises an image sample or a behavior data sample;

the receiving module is used for receiving a first parameter from the cloud-side equipment, wherein the first parameter is a dynamic parameter of a lightweight model generated by a parameter generation model deployed on the cloud-side equipment based on the sample data;

and the processing module is used for carrying out image identification or behavior prediction on the sample data based on the model parameters of the lightweight model, wherein the model parameters of the lightweight model comprise the first parameters.

In a sixth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; the memory is used for storing program instructions; the processor is arranged to call program instructions in the memory to perform the data processing method according to the second aspect, or to perform the data processing method according to the third aspect.

In a seventh aspect, the present application provides a computer readable storage medium storing a computer program, which when executed by a processor, causes the processor to execute the data processing method according to the second aspect, or execute the data processing method according to the third aspect.

In an eighth aspect, embodiments of the present application provide a computer program product, which includes a computer program/instructions, when executed by a processor, to cause the processor to execute the data processing method according to the second aspect, or to execute the data processing method according to the third aspect.

In the data processing system provided by the embodiment of the application, a parameter generation model is deployed on the cloud-side device, and a lightweight model is deployed on the end-side device. The method comprises the steps that the end-side equipment sends sample data collected by the end-side equipment in real time to the cloud-side equipment, wherein the sample data comprises image samples or behavior data samples. The cloud side equipment inputs the sample data into the parameter generation model to obtain a first parameter of the lightweight model, and sends the first parameter to the end side equipment, so that the end side equipment performs image recognition or behavior prediction on the sample data based on the model parameter of the lightweight model, wherein the model parameter of the lightweight model comprises the first parameter, and the first parameter is a dynamic parameter. In the data processing system, the cloud-side parameter generation model can dynamically update the dynamic parameters of the lightweight model on the end-side equipment according to sample data input by the end-side equipment in real time, so that the end-side equipment can adjust the lightweight model on the end side in time according to the change of the scene data on the end side, and the generalization capability of the lightweight model of the end-side equipment is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a data processing system according to an embodiment of the present application;

fig. 7 is a schematic data processing diagram of a parameter stabilizer according to an embodiment of the present application;

fig. 8 is a schematic flowchart of a data processing method according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

The following first briefly describes the terminology used in the embodiments of the present application.

First, the Multi-Layer Perceptron (MLP) is generalized by the Perceptron Learning Algorithm (PLA), which is mainly characterized by multiple neuron layers, and thus is also called Deep Neural Networks (DNN). The multilayer perceptron has at least three layers, including an input layer, an intermediate layer (hidden layer), and an output layer, each node in each layer being connected to each node in an adjacent layer.

Second, a Gated Recurrent Unit (GRU) is a commonly used gated recurrent neural network, which introduces the concept of reset gate and update gate to modify the hidden layer emphasis calculation mode in the recurrent neural network. Resetting gates helps to capture short-term dependencies in the time series, and updating gates helps to capture long-term dependencies in the time series. The addition of the gate control unit enables the gate control recurrent neural network to better capture the dependence of larger time step distance in the time sequence.

Third, distillation (distillation), which compresses a complex model into a small model, can firstly spend a great deal of effort in a training phase to train a complex model, and then generate a smaller model with a smaller computational cost in a deployment phase, while maintaining a certain network prediction performance.

And fourthly, the lightweight model is a network model which can be used at the mobile terminal side, and compared with the network model deployed at the cloud end, the computation amount and the parameter number of the model are reduced. Lightweight computer vision models include, for example, mobileNet, efficientNet, ghostNet, etc., and lightweight recommendation models include, for example, DIN, SASRec, GRU4Rec. Lightweight target detection models include, for example, yolox-nano, and the like. In the embodiment of the present application, the lightweight model deployed on the end side mainly includes the computer vision model and the recommendation model described above.

Fifth, a Meta Network (MN), is an end cloud collaboration framework.

Sixth, a Metabuilder (MG), a network deployed on the cloud that can generate dynamic model parameters for the lightweight on-premise model. In embodiments of the present application, the meta-generator is also referred to as a parameter generation model, and the meta-generator includes MLPs, which can be used to generate model parameters of the lightweight model on the end.

Seventh, a Meta Stabilizer (MS) is a module that is deployed on the cloud and can stabilize MG training and improve MG performance. In the embodiment of the present application, the meta stabilizer is also referred to as a parameter stabilizer, and the cloud stabilizer finally determines the MLP in the meta generator by training a plurality of sub MLPs, and the specific principle is described later.

Eighth, a feed forward operation, i.e., a model prediction process. In the embodiment of the application, the lightweight model parameters on the end-side device include static parameters and dynamic parameters, and under a cloud cooperation framework, the optimal static parameters of the lightweight model on the end-side device and the optimal model parameters of a parameter generation model (i.e., a meta-generator) on the cloud-side device can be obtained through a training process. In the inference/prediction process, the end-side equipment uploads the acquired test data to the cloud side, and the cloud side obtains dynamic parameters of the lightweight model through the parameter generation model. And then, updating the lightweight model by the end-side equipment, performing feed-forward operation on current test data of the end side based on the static parameters and the latest dynamic parameters of the lightweight model, and outputting a prediction result.

Currently, conventional research on deep neural networks focuses primarily on using larger models to improve model performance. However, it is impractical to deploy these large models on mobile end devices due to memory and computational limitations. With the continuous development of the lightweight neural network, it is a necessary trend to deploy a lightweight model on a mobile terminal. However, lightweight models on the end-side device are difficult to adapt to rapidly changing data scenarios. For example, a smart security camera obtains image input under different lighting conditions and at different angles. Alternatively, in recommending system tasks, the user may have different types of preferences for a short period of time.

In this regard, real-time training, real-time model distillation, or other knowledge migration methods of lightweight models on the end-side device using local sample data may be considered, but because of the limited amount of data, lightweight models on the end are easily trapped in overfitting. Moreover, in a scenario of computer vision application or the like, each end-side device cannot obtain tagged data even in a short time. Furthermore, the end-side device training requires back-propagation, which consumes more computational resources and energy on the end-side, making it difficult to be practically applied. In summary, training lightweight models on end-side devices is not feasible in practical applications.

Based on the analysis, considering an end cloud collaboration framework based on the MN, a parameter generation model is deployed on the cloud-side device, and the input of the parameter generation model may be a part of data acquired by the end-side device in real time based on an application scene, for example, the input of the model may be a first frame image captured by a camera within several minutes or several tens of minutes, or several tens of product sequences recently clicked by a user on the e-commerce application App. The output of the parametric generation model is the model parameters of the lightweight model on the end-side device. After the end-side equipment receives the model parameters of the lightweight model sent by the cloud-side equipment, the lightweight model is updated in time so as to be responsible for reasoning on the end-side equipment in a short time in the future. According to the scheme, the cloud side dynamically changes scene data based on the end side, and the parameters of the end side lightweight model are updated in real time, so that the end side lightweight model can have a good prediction or recognition effect when dealing with a scene with fast data change.

It should be noted that the model parameters (particularly, the dynamic parameters of the lightweight model) of the lightweight model deployed on the end-side device can be generated by the parameter generation model through a feed-forward operation, which is computationally efficient and can implement near-real-time model update.

Further, in order to improve the stability of the parameter generation model, it is also considered to deploy a parameter stabilizer on the cloud-side device, and the functional principle of the parameter stabilizer is as follows: by training a plurality of parameter generators (corresponding to the MLP part in the parameter generation model in the following embodiments), self-similar weighting is performed on the plurality of parameter generators, so as to obtain a self-corrected parameter generator, and the new parameter generator is used to predict the model parameters of the lightweight model on the end-side device.

It should be noted that the cloud-side device predicts model parameters of the lightweight model on the end-side device, where the model parameters are dynamic parameters in the lightweight model, and may be parameters of the last layers of the lightweight model.

First, a system architecture according to the present invention will be described below.

Fig. 1 is a schematic diagram of a data processing system according to an embodiment of the present application. As shown in fig. 1, the present embodiment provides a data processing system including: cloud side device 101 and end side device 102, cloud side device 101 with end side device 102 communication connection through the end cloud link.

In this embodiment, the end-side device 102 specifically refers to a data production device, including but not limited to a smart phone, a portable computer, a tablet computer, a wearable device, an intelligent household appliance, an intelligent security device, and the like. The end-side device 102 is responsible for the production, collection, and uploading of various types of data.

In this embodiment, the cloud-side device 101 may be a central cloud device of a distributed architecture, or may be an edge cloud device of a distributed architecture. The cloud-side device 101 provides super-strong computing and storage capabilities, and can generate model parameters of the lightweight model on the end-side device 102 based on sample data uploaded by the end-side device 102, so that the end-side device 102 updates the model parameters of the lightweight model in real time, and the performance of the lightweight model on the end-side device 102 is improved.

In order to facilitate understanding of the execution interaction between the devices in the system architecture of the present embodiment, the following describes an execution interaction process between the devices with reference to several specific application scenarios.

In one possible scenario, the cloud-side device is a central cloud device or an edge cloud device of the e-commerce platform. The cloud-side device can acquire behavior data of a user on the e-commerce platform from the end-side device (e.g., a smartphone), for example, the user browses a plurality of products within a certain time. The cloud side equipment generates model parameters of the lightweight model on the end side equipment based on the behavior data, and timely synchronizes the model parameters to the end side equipment so that the end side equipment can update the lightweight model, and the prediction effect of the end side lightweight model on the user behavior is improved.

It should be understood that the behavior data of the user changes in real time, the cloud-side device receives a series of behavior data collected by the end-side device in real time, model parameters of the end-side lightweight model are generated based on the behavior data and are issued to the end-side device, and the prediction effect of the end-side lightweight model on the current user behavior is improved.

In a possible scenario, the cloud-side device is a central cloud device or an edge cloud device of the intelligent security platform. The cloud-side device may obtain image data of a monitored scene from an end-side device (e.g., a security camera), for example, the camera acquires a group of image sequences, the group of image sequences includes a plurality of image frames, and the camera transmits a first frame image in the image sequences to the cloud-side device. The cloud side device generates model parameters of a lightweight model on the end side device based on the first frame image data, and synchronizes the model parameters to the end side device in time, so that the end side device updates the lightweight model of the cloud side device, and the detection effect of the lightweight model on a target object (such as a vehicle) in the image is improved.

It should be understood that the illumination and weather conditions of the monitored scene change in real time, the cloud-side device receives a real-time image sent by the end-side device, generates model parameters of the end-side lightweight model based on the real-time image, and sends the model parameters to the end-side device, so that the detection effect of the end-side lightweight model on the target object in the current image is improved.

In one possible scenario, the cloud-side device is a central cloud device or an edge cloud device of the image processing platform. The cloud-side device may obtain image data taken by a user from an end-side device (e.g., a smartphone), for example, a group of image sequences is acquired by a camera of the smartphone, the group of image sequences includes a plurality of image frames, and the smartphone transmits a first frame image in the image sequences to the cloud-side device. The cloud-side device generates model parameters of a lightweight model on the end-side device based on the first frame image, and synchronizes the model parameters to the end-side device in time, so that the end-side device updates the lightweight model thereof, and the recognition effect (e.g., expression recognition) of the lightweight model on a target object (e.g., a person's face) in the image is improved.

It should be understood that the shooting environment of the user changes in real time, the cloud-side device receives a real-time image sent by the end-side device, generates model parameters of the end-side lightweight model based on the real-time image, and sends the model parameters to the end-side device, so that the recognition effect of the end-side lightweight model on the target object in the current image is improved.

It should be noted that, for any of the above scenarios, after the cloud-side device completes generation of the model parameters and sends the end-side model parameters to the end-side device, the cloud-side device should delete the personal data and the privacy data related to the user in time for privacy protection.

In the embodiment of the present application, the function of the lightweight model of the end-side device is not particularly limited, and the lightweight model may be a lightweight model having other functions in addition to the functions of behavior prediction, vehicle detection, face recognition, and the like, which are exemplified in the above-described scenarios.

The technical solutions provided in the embodiments of the present application are described in detail below with specific embodiments. It should be noted that the technical solutions provided in the embodiments of the present application may include part or all of the following contents, and several specific embodiments below may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is a schematic diagram of a data processing system according to an embodiment of the present application. As shown in fig. 2, the data processing system includes: the cloud side equipment is provided with a parameter generation model, and the end side equipment is provided with a lightweight model. The method comprises the steps that the end-side equipment sends sample data acquired by the end-side equipment in real time to the cloud-side equipment, wherein the sample data comprises image samples or behavior data samples. The cloud side equipment inputs the sample data into the parameter generation model to obtain a first parameter of the lightweight model. The first parameters of the lightweight model are the dynamic parameters of the lightweight model, also referred to as variable parameters. The cloud side equipment sends the first parameter to the end side equipment so that the end side equipment updates the first parameter of the lightweight model, and the end side equipment can perform image recognition or behavior prediction on the sample data based on the updated lightweight model. Optionally, the peer-to-peer device may perform image recognition or behavior prediction on the sample data and other sample data acquired by the peer-to-peer device within a short period of time (within a preset certain time period) based on the updated lightweight model.

It should be noted that the model parameters of the lightweight model include a second parameter in addition to the first parameter, and the second parameter is a static parameter of the lightweight model, which is also referred to as a fixed parameter.

The end-side model parameters obtained by the model compression strategy are usually model parameters obtained by the cloud-side device based on sample data of different end-side devices, that is, the cloud-side device is trained by using full data. For a particular peer-side device, the data distribution on the peer-side is likely to change in the future, which will result in a degradation of the predictive performance of the peer-side model. Because real-time training data cannot be acquired on the end-side device, the existing model compression strategy cannot solve the problem.

The technical scheme provided by the embodiment is different from a model compression strategy under the cooperation of end cloud, the cloud side equipment receives sample data sent by the end side equipment, the sample data is test data acquired by the end side equipment in real time, such as a first frame of an image sequence or a click sequence, the cloud side equipment generates a model based on pre-trained parameters, model parameters of a lightweight model of the end side equipment are dynamically updated, the performance of the lightweight model of the end side equipment in each time period is improved, and therefore the defect that the model generalization capability of the lightweight model of the end side equipment is insufficient due to parameter limitation is overcome. In the processing process, the end-side device only needs to provide sample data to the cloud-side device, and the cloud-side device predicts the latest first parameter (specifically, the optimal first parameter under data distribution of the current end-side device) for the end-side device, so that the end-side device performs image recognition or behavior prediction on the current test data based on the end-side model parameter (including the second parameter and the latest first parameter).

Based on the data processing system shown in fig. 2, the data processing system will be described in more detail below with reference to different lightweight models on the end side.

In one embodiment, the lightweight models deployed on the end-side devices include a first lightweight model, the first lightweight model being used for image recognition. Correspondingly, the parameter generation model deployed on the cloud-side device comprises a first parameter generation model, and the first parameter generation model is used for generating a first parameter of the first lightweight model. The first parameter of the first lightweight model is a dynamic parameter of the model.

Fig. 3 is a schematic diagram of a data processing system according to an embodiment of the present application. As shown in fig. 3, in the data processing system of the present embodiment:

the end-side device sends sample data to the cloud-side device, where the sample data is an image sample in this embodiment. In particular, the image sample is the first image in a series of consecutive image frames acquired by the end-side device, such as the image sequence shown in fig. 3x ₀ To x _n The end-side device only sends the first frame image x in the image sequence to the cloud-side device ₀ 。

In consideration of the fact that the shooting environment and the shooting angle of the continuous image frames are not changed basically, the end-side device only needs to transmit the first frame image of the continuous image frames, and the computing load of the cloud-side device is reduced.

After the cloud side equipment receives the image sample, the image sample is input into a first parameter generation model deployed on the cloud side equipment, and a first parameter of a first lightweight model is obtained. And the cloud side equipment issues the first parameter of the first lightweight model to the end side equipment so that the end side equipment updates the first parameter of the first lightweight model.

And the end-side equipment identifies or detects a target object for each frame of image in the image sequence input into the first lightweight model based on the model parameters of the first lightweight model, and outputs an image identification or detection result. The model parameters of the first lightweight model comprise first parameters of the first lightweight model received from the cloud-side equipment, and second parameters of the first lightweight model.

As an example, referring to fig. 3, the first parameter generation model includes an encoder header (encoder sock), a first multi-layer perceptron MLP (i.e., MLP1 of fig. 3), and a second MLP (i.e., MLP2 of fig. 3). The input end of the encoder head is the input end of the first parameter generation model, the output end of the encoder head is connected with the input end of the first MLP, the output end of the first MLP is connected with the input end of the second MLP, and the output of the second MLP is the output of the first parameter generation model. Specifically, the cloud-side device inputs the image sample into an encoder head of the first parameter generation model, and then the image sample is subjected to data processing of the first MLP and the second MLP in sequence to obtain a first parameter of the first lightweight model.

It should be noted that the first MLP includes a linear layer, and the first MLP can linearly map one embedded vector to another embedded vector with the same dimension. The second MLP, which is different from the first MLP and includes two linear layers and a non-linear activation function, can convert the embedded vector into model parameters (e.g., the first parameters of the end-side lightweight model), which function as a decoder.

On the basis of the above example, the model structure of the first parametric generative model may be further extended, depending on which layer parameters of the first lightweight model are variable on the end-side device. As an example, the peer-side device may send the hyper-parameters to the cloud-side device

The hyperparameter is used to indicate a total number of layers of the dynamic parameters of the first lightweight model. Most of the time, if the appropriate hyper-parameter is selected

The performance of the first parameter generation model can be improved.

Illustratively, fig. 4 is a schematic diagram of a data processing system provided in an embodiment of the present application. As shown in fig. 4, the first lightweight model of the end-side device includes 1 input layer, 2 hidden layers, and 1 output layer, where the last 3 layers of the first lightweight model, i.e., 2 hidden layers and 1 output layer, have dynamic parameters, i.e., the total number of layers of the dynamic parameters is 3. Accordingly, based on the first parameter generation model shown in fig. 3, a total of 3 MLPs can be extended on the cloud-side device, each MLP including one MLP1 and one MLP2,3 MLP sharing one encoder header.

The first parameter output by the MLP2 of the 1 st MLP shown in fig. 4 corresponds to the 1 st hidden layer of the first lightweight model, the first parameter output by the MLP2 of the 2 nd MLP corresponds to the 2 nd hidden layer of the first lightweight model, and the first parameter output by the MLP2 of the 3 rd MLP corresponds to the output layer of the first lightweight model. Namely, the first parameter output by the 1 st way MLP2 is taken as the parameter of the 1 st hidden layer of the first lightweight model, the first parameter output by the 2 nd way MLP2 is taken as the parameter of the 2 nd hidden layer of the first lightweight model, and the first parameter output by the 3 rd way MLP2 is taken as the parameter of the output layer of the first lightweight model.

In summary, it can be known from the above examples that the model parameters of the first lightweight model include a first parameter of the P1 layer and second parameters of the M1-P1 layers, the second parameters are static parameters, and both P1 and M1-P1 are positive integers. Accordingly, the first MLP includes P1 first MLPs, and the second MLP includes P1 second MLPs. Referring to FIG. 4, M1 is 4 and P1 is 3.

In the above example, the first parameter generation model of the cloud-side device outputs multiple paths of first parameters, and each path of first parameter corresponds to one dynamic parameter layer of the first lightweight model of the end-side device.

In one embodiment, the lightweight model deployed on the end-side device includes a second lightweight model, the second lightweight model being used for behavioral prediction. Correspondingly, the parameter generation model deployed on the cloud-side device comprises a second parameter generation model, and the second parameter generation model is used for generating a first parameter of a second lightweight model. The first parameters of the second lightweight model are the dynamic parameters of the model.

Illustratively, fig. 5 is a schematic diagram of a data processing system provided in an embodiment of the present application. As shown in fig. 5, in the data processing system of the present embodiment:

and the end-side equipment sends sample data to the cloud-side equipment, wherein the sample data is a behavior data sample in the embodiment. Specifically, the behavior data sample is a series of behavior data of a target application program of the user on the end-side device, such as a click sequence m shown in fig. 5 ₀ To m _n And the end-side device sends the click sequence to the cloud-side device.

After the cloud side equipment receives the behavior data samples, the behavior data samples are input into a second parameter generation model deployed on the cloud side equipment, and first parameters of a second lightweight model are obtained. And the cloud side equipment issues the first parameter of the second lightweight model to the end side equipment so that the end side equipment updates the first parameter of the second lightweight model.

It should be noted that the behavior data sample includes a plurality of behavior data, such as a plurality of click behaviors shown in fig. 5, and the behavior data corresponding to each click behavior includes, for example, product information clicked and browsed by the user.

The method for inputting the behavior data sample into the second parameter generation model by the cloud-side device specifically includes: and the cloud side equipment inputs the plurality of behavior data into the second parameter generation model together.

And the end-side equipment predicts the future behavior of the user based on the model parameters of the second lightweight model and the click sequence input into the second lightweight model, and outputs a behavior prediction result. Wherein the model parameters of the second lightweight model comprise first parameters of the second lightweight model received from the cloud-side device, and second parameters of the second lightweight model.

As an example, referring to fig. 5, the second parametric generative model includes a gated recurrent neural network GRU and a third MLP (i.e., MLP2 of fig. 5). The input end of the GRU is the input end of the second parameter generation model, the output end of the GRU is connected with the input end of the third MLP, and the output end of the third MLP is the output end of the second parameter generation model. Specifically, the cloud side device inputs the behavior data samples into a GRU of the second parameter generation model, and then performs data processing through a third MLP to obtain a first parameter of the second lightweight model.

It should be noted that the third MLP of the present embodiment is similar to the second MLP of the previous embodiment, and the third MLP also includes two linear layers and a non-linear activation function, and the third MLP can convert the embedded vector into a model parameter (e.g., the first parameter of the end-side lightweight model), and its function is equivalent to a decoder.

On the basis of the above example, the model structure of the second parametric generation model may be further extended, depending on which layer parameters of the second lightweight model on the end-side device are variable. As an example, the end-side device may send a super-parameter to the cloud-side device, the super-parameter indicating a total number of layers of the dynamic parameters of the second lightweight model.

Fig. 6 is a schematic diagram of a data processing system according to an embodiment of the present application. As shown in fig. 6, the second lightweight model of the end-side device includes 1 input layer, 1 hidden layer and 1 output layer, where the last 2 layers of the second lightweight model, i.e. the hidden layer and the output layer, have dynamic parameters, i.e. the total number of dynamic parameters is 2. Accordingly, based on the second parameter generation model shown in fig. 5, a total of 2 processing modules, each including one GRU and one MLP2, may be extended on the cloud-side device.

Fig. 6 shows that the first parameter output by the MLP2 in the 1 st processing module corresponds to the hidden layer of the second lightweight model, and the first parameter output by the MLP2 in the 2 nd processing module corresponds to the output layer of the second lightweight model. Namely, the first parameter output by the MLP2 in the 1 st processing module is used as the parameter of the hidden layer of the second lightweight model, and the first parameter output by the MLP2 in the 2 nd processing module is used as the parameter of the output layer of the second lightweight model.

In summary, the model parameters of the second lightweight model include the first parameters of the P2 layer and the second parameters of the M2-P2 layer, and P2 and M2-P2 are positive integers. Accordingly, the GRU includes P2 GRUs, and the third MLP includes P2 third MLPs. Referring to fig. 6, M2 is taken as 3 and P2 is taken as 1.

In the above example, the second parameter generation model of the cloud-side device outputs multiple paths of first parameters, and each path of first parameter corresponds to one dynamic parameter layer of the second lightweight model of the end-side device.

The parameter generation model shown in the above embodiment, such as the first parameter generation model and the second parameter generation model, may dynamically update the model parameters of the lightweight model on the end-side device according to the samples of different environments where the end-side device is located, so as to improve the generalization capability of the lightweight model on the end-side device, that is, improve the performance of the lightweight model on the end-side device.

The above embodiment shows a parameter generation model, whose loss function can be defined as:

in the formula (I), the compound is shown in the specification,

some sample data (such as some image frame or some behavior data) representing the input parameter generation model, T takes 0 to T, T represents the duration of an image sequence or click sequence,

representing multiple image sequences or multiple click sequences.

A parameter generation model representing a cloud side;

representing a parametric generative model

The first parameter of the output (i.e. the first parameter of the lightweight model);

a second parameter representing a lightweight model;

representing a lightweight model of the end side;

indicating sample data

After the lightweight model is input, the lightweight model outputs a prediction result;

representing sample data

Corresponding labeling results;

is a loss function of model training, such as a binary cross entropy loss function, and the like, where the model training includes training a model parameter of a cloud-side parameter generation model and a second parameter of an end-side lightweight model;

representing the hyper-parameters for the adjustment training,

the positive number is, for example, 0.9, 1, 2, etc.

It should be noted that, for image recognition scenes, it is generally possible to use

A positive number less than 1, for example 0.9, is set for the purpose of: for an image sequence of duration T, the importance of the image frames in the image sequence decreases over time. By the setting, the prediction effect of the parameter generation model on the lightweight model parameters of the short-time scene can be improved.

Considering that the parameter generation model often generates large oscillation in the training process and the parameter generation effect of the lightweight model on the equipment on the opposite side of the parameter generation model is sometimes unstable, a parameter stabilizer is designed and deployed on the cloud-side equipment to improve the performance of the parameter generation model of the cloud-side equipment.

In one embodiment, a parameter stabilizer is further deployed on the cloud-side device, and the parameter stabilizer is used for optimizing the target layer parameters of the target MLP in the parameter generation model. The target layer of the target MLP may be any layer of the target MLP.

The optimization process of the parameter stabilizer of the cloud-side device on the target layer parameters of the target MLP specifically includes: acquiring target layer parameters of a plurality of sub-MLPs, wherein the plurality of sub-MLPs are used for constructing a target MLP; and performing similarity calculation and weighted summation processing on the target layer parameters of the plurality of sub MLPs to obtain the target layer parameters of the target MLP. Wherein the target layer parameters include weight parameters of the target layer. Optionally, the target layer parameters further include bias parameters of the target layer.

Optionally, the parameter stabilizer of the cloud-side device performs similarity calculation and weighted summation processing on the target layer parameters of the multiple sub MLPs to obtain the target layer parameters of the target MLP, and specifically includes:

generating a first matrix according to the target layer parameters of the plurality of sub MLPs; transposing the first matrix to obtain a second matrix; similarity calculation and weighted summation are carried out on the parameters of the first matrix and the second matrix, and the weight of the target layer parameters of the multiple sub MLPs is determined; and obtaining the target layer parameters of the target MLP based on the weights of the target layer parameters of the plurality of sub MLPs and the target layer parameters of the plurality of sub MLPs.

For better understanding of the parameter stabilizer, the processing procedure thereof will be described in detail with reference to a specific example.

As an example, referring to fig. 3, a parameter stabilizer deployed on the cloud-side device is used to optimize the target layer parameters of MLP2 in the first parameter generation model. The target layer parameters include any layer parameter of MLP2 in the first parametric generative model.

Fig. 7 is a schematic data processing diagram of a parameter stabilizer according to an embodiment of the present application. In this embodiment, based on the cloud-side device described in fig. 3, the first parameter generation model of the cloud-side device includes an encoder head, MLP1, and MLP2. A process for optimizing MLP2 in a first parametric generative model, comprising: the parameters of each layer in MLP2 are optimized. For ease of understanding, fig. 7 illustrates a scheme for optimizing the parameters of the first layer (the gray layer in fig. 7) of MLP2. As shown in fig. 7, optimizing the first layer parameters of MLP2 specifically includes:

step 1, first, n sub-MLPs 2 are constructed, such as the MLP2 in fig. 7 ₁ , MLP2 ₂ ,…, MLP2 _n And acquiring the first-layer parameters of the n sub-MLPs 2 through sample data training.

Step 2, generating a first matrix according to the first layer parameters of the n sub MLPs 2

。

Wherein the content of the first and second substances,

，

represents MLP2 ₁ The first layer parameters of (a) are,

represents MLP2 ₂ The first layer parameters of (a) are,

represents MLP2 _n The first layer parameter of (1).

Step 3, aiming at the first matrix

Transposing to obtain a second matrix

。

And 4, carrying out similarity calculation and weighted summation on the parameters of the first matrix and the second matrix, and determining the weight of the first-layer parameters of the n sub-MLPs 2.

As an example, step 4 specifically includes:

step 41, multiplying the first matrix and the second matrix to obtain an n x n self-similarity matrix, and recording the self-similarity matrix as

。

Step 42, pair

Summing up by rows, resulting in a weight vector p = { p } of n x 1 ₁ ,p ₂ ,…,p _n }。

Wherein p is ₁ Represents MLP2 ₁ Weight value of the first layer parameter of (1), p ₂ Represents MLP2 ₂ Of the first layer parameter, p _n Represents MLP2 _n The weight value of the first-level parameter of (1).

In this step, normalization processing needs to be performed on the weight vector p, which can refer to the following formula:

wherein i, j is ∈ [1,n ]],

Is positive, e.g.

1 is taken.

And 5, determining the first layer parameters of the MLP2 based on the weights of the first layer parameters of the n sub-MLPs 2 and the first layer parameters of the n sub-MLPs 2.

Specifically, the first layer parameter of MLP2 can be determined by the following formula

：

Based on the steps, similarity calculation and weighted summation processing are carried out on the first layer parameters of the n sub-MLPs 2, so that the first layer parameters of the MLP2 are obtained.

It should be noted that, in this embodiment, the first layer parameter of MLP2 includes a weight parameter in addition to the weight parameter

In addition, bias parameters are included

. In some embodiments, the bias parameters of the MLP2 may also be obtained by performing similarity calculation and weighted summation processing on the bias parameters of the n sub-MLPs 2.

Similarly, other layer parameters in MLP2 of the first parametric generative model may be optimized.

As an example, referring to fig. 5, a parameter stabilizer deployed on the cloud-side device is used to optimize the target layer parameters of MLP2 in the second parameter generation model. The target layer parameters include any layer parameter of MLP2 in the second parameter generation model.

The optimization procedure for any layer parameter of MLP2 in the second parameter generation model is similar to the above-described embodiment, and can be referred to the above-described embodiment, and is not expanded here.

The parameter stabilizer shown in the above embodiments may be used to reduce oscillation of the parameter generation model during training, and improve performance of the parameter generation model.

In the embodiment of the present application, in addition to providing a data processing system, a data processing method is also provided, and the data processing method is described below with reference to the accompanying drawings.

Fig. 8 is a schematic flowchart of a data processing method according to an embodiment of the present application. The data processing method provided by the embodiment relates to an interaction process of cloud-side equipment and end-side equipment. The cloud-side equipment is deployed with a parameter generation model, and the end-side equipment is deployed with a lightweight model. The parameter generation model is used for generating a first parameter of the lightweight model, the first parameter is a dynamic parameter, and the lightweight model is used for image recognition or behavior prediction.

As shown in fig. 8, the data processing method includes:

step 801, the end-side device sends sample data acquired by the end-side device in real time to the cloud-side device.

Wherein the sample data comprises an image sample or a behavior data sample.

Step 802, the cloud side equipment inputs the sample data into a parameter generation model to obtain a first parameter of the lightweight model.

Step 803, the cloud side device sends the first parameter to the end side device.

And 804, the end-side equipment performs image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model.

Wherein the model parameters of the lightweight model include a first parameter.

In one embodiment, the end-side device performs image recognition or behavior prediction on the sample data based on the first and second parameters of the lightweight model. It should be noted that the first parameter here is a first parameter that is updated by the cloud side device in real time based on current sample data.

The first parameter is a dynamic parameter of the lightweight model, and the second parameter is a static parameter of the lightweight model.

In one embodiment, the peer side device may perform image recognition or behavior prediction on the sample data and other sample data acquired by the peer side device within a short period of time (within a preset certain period of time) based on the second parameter and the updated first parameter.

In an optional embodiment of this embodiment, if the sample data is an image sample, the image sample is a first frame of a continuous image frame acquired by the end-side device; the lightweight model comprises a first lightweight model, and the first lightweight model is used for image recognition; the parameter generating model comprises a first parameter generating model used for generating first parameters of the first lightweight model. Inputting the sample data into a parameter generation model to obtain a first parameter of the lightweight model, wherein the parameter generation model comprises the following steps: and inputting the image sample into a first parameter generation model to obtain a first parameter of the first lightweight model.

In an optional embodiment of this embodiment, the first parameter generation model includes an encoder head, a first multi-layer perceptron MLP, and a second MLP; inputting the image sample into a first parameter generation model to obtain a first parameter of a first lightweight model, wherein the method comprises the following steps: and inputting the image sample into the head part of an encoder of the first parameter generation model, and sequentially carrying out data processing on the first MLP and the second MLP to obtain a first parameter of the first lightweight model.

In an optional embodiment of this embodiment, the model parameters of the first lightweight model include a first parameter of a P1 layer and a second parameter of an M1-P1 layer, the second parameter is a static parameter, and both P1 and M1-P1 are positive integers; accordingly, the first MLP includes P1 first MLPs and the second MLP includes P1 second MLPs.

In an optional embodiment of this embodiment, if the sample data is a behavior data sample, the behavior data sample is a series of behavior data of a target application program of the user on the end-side device; the lightweight model comprises a second lightweight model, and the second lightweight model is used for behavior prediction; the parameter generation model comprises a second parameter generation model used for generating first parameters of a second lightweight model; inputting the sample data into a parameter generation model to obtain a first parameter of the lightweight model, wherein the parameter generation model comprises the following steps: and inputting the behavior data sample into a second parameter generation model to obtain a first parameter of a second lightweight model.

In an optional embodiment of this embodiment, the second parameter generation model includes a gated recurrent neural network GRU and a third MLP; inputting the behavior data sample into a second parameter generation model to obtain a first parameter of a second lightweight model, wherein the method comprises the following steps: and inputting the behavior data sample into a GRU of a second parameter generation model, and then performing data processing of a third MLP to obtain a first parameter of a second lightweight model.

In an optional embodiment of this embodiment, the model parameters of the second lightweight model include a first parameter of a P2 layer and a second parameter of an M2-P2 layer, where P2 and M2-P2 are positive integers; accordingly, the GRU includes P2 GRUs, and the third MLP includes P2 third MLPs.

In an optional embodiment of this embodiment, a parameter stabilizer is further deployed on the cloud-side device, and the parameter stabilizer is used for optimizing a target layer parameter of a target MLP in the parameter generation model; the optimization process of the parameter stabilizer on the target layer parameters of the target MLP comprises the following steps: acquiring target layer parameters of a plurality of sub-MLPs, wherein the plurality of sub-MLPs are used for constructing a target MLP; performing similarity calculation and weighted summation processing on the target layer parameters of the plurality of sub MLPs to obtain target layer parameters of the target MLP; wherein the target layer parameters include weight parameters of the target layer.

In an optional embodiment of this embodiment, the obtaining the target layer parameter of the target MLP by performing similarity calculation and weighted summation processing on the target layer parameters of the multiple sub-MLPs includes: generating a first matrix according to the target layer parameters of the plurality of sub MLPs; transposing the first matrix to obtain a second matrix; similarity calculation and weighted summation are carried out on the parameters of the first matrix and the second matrix, and the weight of the target layer parameters of the multiple sub MLPs is determined; and obtaining the target layer parameters of the target MLP based on the weights of the target layer parameters of the plurality of sub MLPs and the target layer parameters of the plurality of sub MLPs.

In the data processing method shown in the embodiment of the application, the cloud-side device receives sample data sent by the end-side device, and the sample data can be an image sample or a behavior data sample acquired by the end-side device in real time. The cloud side equipment inputs the sample data into the parameter generation model, generates a first parameter of the lightweight model, and sends the first parameter to the end side equipment, so that the end side equipment performs image recognition or behavior prediction on the sample data based on the model parameter of the lightweight model, and the model parameter of the lightweight model comprises the first parameter. The method is suitable for a model prediction process, the cloud-side parameter generation model can dynamically update the dynamic parameters of the lightweight model on the end-side equipment according to sample data input by the end-side equipment in real time, so that the end-side equipment can timely adjust the lightweight model according to the change of the scene data of the end side, and the generalization capability of the lightweight model of the end-side equipment is improved.

In the embodiment of the application, besides a data processing method, a data processing device is also provided. Fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 9, the data processing apparatus 900 includes: a receiving module 901, a processing module 902 and a sending module 903.

A receiving module 901, configured to receive sample data acquired in real time from an end-side device, where the sample data includes an image sample or a behavior data sample;

the processing module 902 is configured to input the sample data into a parameter generation model to obtain a first parameter of the lightweight model;

a sending module 903, configured to send the first parameter to the end-side device, so that the end-side device performs image recognition or behavior prediction on the sample data based on a model parameter of the lightweight model, where the model parameter of the lightweight model includes the first parameter.

In an optional embodiment of this embodiment, if the sample data is an image sample, the image sample is a first frame of consecutive image frames acquired by the end-side device; the lightweight model comprises a first lightweight model, and the first lightweight model is used for image recognition; the parameter generation model comprises a first parameter generation model used for generating first parameters of the first lightweight model;

the processing module 902 is specifically configured to input the image sample into the first parameter generation model to obtain a first parameter of the first lightweight model.

In an optional embodiment of this embodiment, the first parameter generation model includes an encoder header, a first multi-layer perceptron MLP, and a second MLP; the processing module 902 is specifically configured to input the image sample into an encoder head of the first parameter generation model, and then sequentially perform data processing on the first MLP and the second MLP to obtain a first parameter of the first lightweight model.

In an optional embodiment of this embodiment, the model parameters of the first lightweight model include a first parameter of a P1 layer and a second parameter of an M1-P1 layer, where the second parameter is a static parameter, and P1 and M1-P1 are both positive integers; correspondingly, the first MLP includes P1 first MLPs, and the second MLP includes P1 second MLPs.

In an optional embodiment of this embodiment, if the sample data is a behavior data sample, the behavior data sample is a series of behavior data of a target application program of the user on the end-side device; the lightweight model comprises a second lightweight model, and the second lightweight model is used for behavior prediction; the parameter generation model comprises a second parameter generation model used for generating a first parameter of the second lightweight model;

the processing module 902 is specifically configured to input the behavior data sample into the second parameter generation model to obtain a first parameter of the second lightweight model.

In an optional embodiment of this embodiment, the second parameter generation model includes a gated recurrent neural network GRU and a third MLP; the processing module 902 is specifically configured to:

and inputting the behavior data sample into a GRU of the second parameter generation model, and obtaining a first parameter of the second lightweight model through data processing of the third MLP.

In an optional embodiment of this embodiment, the model parameters of the second lightweight model include a first parameter of a P2 layer and a second parameter of an M2-P2 layer, where P2 and M2-P2 are positive integers; correspondingly, the GRU includes P2 GRUs, and the third MLP includes P2 third MLPs.

In an optional embodiment of this embodiment, a parameter stabilizer is further deployed on the cloud-side device, and the parameter stabilizer is configured to optimize a target layer parameter of a target MLP in the parameter generation model;

the optimization process of the parameter stabilizer on the target layer parameters of the target MLP comprises the following steps: acquiring target layer parameters of a plurality of sub-MLPs, wherein the plurality of sub-MLPs are used for constructing the target MLP; performing similarity calculation and weighted summation processing on the target layer parameters of the plurality of sub-MLPs to obtain the target layer parameters of the target MLP; wherein the target layer parameters include weight parameters of the target layer.

In an optional embodiment of this embodiment, the obtaining, by the parameter stabilizer, the target layer parameter of the target MLP by performing similarity calculation and weighted summation processing on the target layer parameters of the multiple sub-MLPs includes: generating a first matrix according to the target layer parameters of the plurality of sub MLPs; transposing the first matrix to obtain a second matrix; performing similarity calculation and weighted summation on the parameters of the first matrix and the second matrix, and determining the weight of the target layer parameters of the plurality of sub-MLPs; and obtaining the target layer parameters of the target MLP based on the weights of the target layer parameters of the plurality of sub MLPs and the target layer parameters of the plurality of sub MLPs.

The data processing apparatus 900 of this embodiment may be deployed on a cloud-side device or a cloud server, and may dynamically update model parameters of a lightweight model on the end-side device according to samples of different environments where the end-side device is located, so as to improve the generalization capability of the lightweight model of the end-side device.

In the embodiment of the present application, a data processing apparatus is further provided, and fig. 10 is a schematic structural diagram of the data processing apparatus provided in the embodiment of the present application. As shown in fig. 10, the data processing apparatus 1000 includes: a sending module 1001, a receiving module 1002 and a processing module 1003.

The sending module 1001 is configured to send, to the cloud-side device, sample data acquired by the end-side device in real time, where the sample data includes an image sample or a behavior data sample; a receiving module 1002, configured to receive a first parameter from the cloud-side device, where the first parameter is a dynamic parameter of a lightweight model generated based on the sample data by a parameter generation model deployed in the cloud-side device; a processing module 1003, configured to perform image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model, where the model parameters of the lightweight model include the first parameter.

The data processing apparatus 1000 of this embodiment may be deployed on the end-side device, and may update the first parameter of the lightweight model deployed locally in time according to the first parameter of the lightweight model sent by the cloud-side device, so as to improve the generalization capability of the model.

It should be noted that in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and only for distinguishing the various operations, and the sequence number itself does not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 11, the electronic apparatus includes: a memory 1101 and a processor 1102. A memory 1101 for storing computer programs and may be configured to store other various data to support operations on the cloud-side device. The processor 1102 is coupled to the memory 1101, and configured to execute the computer program in the memory 1101 to implement the technical solution of the cloud-side device in the foregoing method embodiment, which is similar to the implementation principle and the technical effect and is not described herein again.

Optionally, as shown in fig. 11, the electronic device further includes: firewall 1103, load balancer 1104, communications component 1105, power component 1106, and other components. Only some of the components are schematically shown in fig. 11, and it is not meant that the electronic device includes only the components shown in fig. 11.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 12, the electronic apparatus includes: a memory 1201 and a processor 1202. A memory 1201 for storing a computer program and may be configured to store other various data to support operations on the end-side device. The processor 1202 is coupled to the memory 1201, and is configured to execute the computer program in the memory 1201, so as to implement the technical solution of the end-side device in the foregoing method embodiment, which is similar to the implementation principle and the technical effect, and details are not described here again.

Optionally, as shown in fig. 12, the electronic device further includes: firewall 1203, load balancer 1204, communications component 1205, power component 1206, and other components. Only some of the components are schematically shown in fig. 12, and the electronic device is not meant to include only the components shown in fig. 12.

The embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the technical solution of the cloud-side device in the foregoing method embodiments.

The present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the technical solution of the end-side device in the foregoing method embodiments.

An embodiment of the present application provides a computer program product, which includes a computer program/instruction, and when the computer program/instruction is executed by a processor, the processor is enabled to implement the technical solution of the cloud device in the foregoing method embodiments.

The present application provides a computer program product, which includes a computer program/instruction, and when the computer program/instruction is executed by a processor, the computer program/instruction causes the processor to implement the technical solution of the end-side device in the foregoing method embodiments.

The embodiment of the application provides a chip, including: the processing module can execute the technical scheme of the cloud device in the method embodiment. Optionally, the chip further includes a storage module (e.g., a memory), the storage module is configured to store instructions, the processing module is configured to execute the instructions stored by the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solution of the cloud device in the foregoing method embodiment.

The embodiment of the present application provides a chip, including: and the processing module and the communication interface are used for executing the technical scheme of the end-side equipment in the method embodiment. Optionally, the chip further includes a storage module (e.g., a memory), where the storage module is configured to store instructions, and the processing module is configured to execute the instructions stored by the storage module, and execute the instructions stored in the storage module, so that the processing module executes the technical solution of the end-side device in the foregoing method embodiment.

The Storage may be an Object Storage Service (OSS).

The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The communication component is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

The power supply assembly provides power for various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing system, comprising: the cloud side equipment is provided with a parameter generation model, and the end side equipment is provided with a lightweight model;

the cloud side equipment is used for inputting the sample data into the parameter generation model to obtain multiple paths of first parameters, and each path of first parameter corresponds to one dynamic parameter layer of the lightweight model of the end side equipment; the cloud side equipment is further used for sending the multi-path first parameters to the end side equipment; the parameter generation model comprises a plurality of data processing paths, each data processing path is used for outputting a path of first parameter, and the number of the data processing paths is the same as that of the dynamic parameter layers of the lightweight model;

the end-side equipment is further used for carrying out image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model;

the model parameters of the lightweight model comprise the multipath first parameters and second parameters, the first parameters are dynamic parameters of the lightweight model, and the second parameters are fixed parameters of the lightweight model.

2. The system of claim 1, wherein if the sample data is an image sample, the image sample is a first frame of consecutive image frames acquired by the end-side device;

the lightweight model comprises a first lightweight model, and the first lightweight model is used for image recognition; the parameter generation model comprises a first parameter generation model used for generating first parameters of the first lightweight model;

the cloud side equipment inputs the sample data into the parameter generation model to obtain a first parameter of the lightweight model, and the method specifically comprises the following steps: and inputting the image sample into the first parameter generation model to obtain a first parameter of the first lightweight model.

3. The system of claim 2, wherein the first parametric generative model comprises an encoder head, a first multi-layer perceptron MLP, and a second MLP; the cloud-side device inputs the image sample into the first parameter generation model to obtain a first parameter of the first lightweight model, and the method specifically includes:

and inputting the image samples into an encoder head of the first parameter generation model, and sequentially performing data processing of a first MLP and a second MLP to obtain a first parameter of the first lightweight model.

4. The system of claim 3, wherein the model parameters of the first lightweight model comprise P ₁ First parameter of layer and M ₁ -P ₁ A second parameter of the layer, the second parameter being a static parameter, P ₁ And M ₁ -P ₁ Are all positive integers, M ₁ Is greater than P ₁ A positive integer of (d);

correspondingly, the first MLP comprises P ₁ A first MLP, the second MLP comprising P ₁ A second MLP.

5. The system according to claim 1, wherein if the sample data is a behavior data sample, the behavior data sample is a series of behavior data of a target application program of a user on the end-side device;

the lightweight model comprises a second lightweight model, and the second lightweight model is used for behavior prediction; the parameter generation model comprises a second parameter generation model used for generating a first parameter of the second lightweight model;

the cloud side equipment inputs the sample data into the parameter generation model to obtain a first parameter of the lightweight model, and the method specifically comprises the following steps: and inputting the behavior data sample into the second parameter generation model to obtain a first parameter of the second lightweight model.

6. The system according to claim 5, characterized in that the second parametric generative model comprises a gated recurrent neural network (GRU) and a third MLP; the cloud-side device inputs the behavior data sample into the second parameter generation model to obtain a first parameter of the second lightweight model, and the method specifically includes:

7. The system of claim 6, wherein the model parameters of the second lightweight model comprise P ₂ First parameter of layer and M ₂ -P ₂ Second parameter of layer, P ₂ And M ₂ -P ₂ Is a positive integer, M ₂ Is greater than P ₂ A positive integer of (d);

accordingly, the GRU packetDraw P ₂ A GRU, the third MLP comprising P ₂ A third MLP.

8. The system according to any one of claims 1 to 7, wherein a parameter stabilizer is further deployed on the cloud-side device, and the parameter stabilizer is configured to optimize target layer parameters of a target MLP in the parameter generation model;

the optimization process of the parameter stabilizer of the cloud side device on the target layer parameters of the target MLP comprises the following steps:

obtaining target layer parameters of a plurality of sub-MLPs, wherein the plurality of sub-MLPs are used for constructing the target MLP;

performing similarity calculation and weighted summation processing on the target layer parameters of the plurality of sub-MLPs to obtain the target layer parameters of the target MLP;

wherein the target layer parameters include weight parameters of the target layer.

9. The system according to claim 8, wherein the parameter stabilizer of the cloud-side device obtains the target layer parameters of the target MLP by performing similarity calculation and weighted summation processing on the target layer parameters of the plurality of sub-MLPs, and specifically includes:

generating a first matrix according to the target layer parameters of the plurality of sub MLPs;

transposing the first matrix to obtain a second matrix;

performing similarity calculation and weighted summation on the parameters of the first matrix and the second matrix, and determining the weight of the target layer parameters of the plurality of sub-MLPs;

and obtaining the target layer parameters of the target MLP based on the weights of the target layer parameters of the plurality of sub MLPs and the target layer parameters of the plurality of sub MLPs.

10. The data processing method is characterized by being applied to cloud-side equipment, wherein a parameter generation model is deployed on the cloud-side equipment, the parameter generation model is used for generating a first parameter of a lightweight model, the first parameter is a dynamic parameter, the lightweight model is deployed on end-side equipment, and the lightweight model is used for image recognition or behavior prediction;

the method comprises the following steps:

receiving sample data acquired in real time from the end-side equipment, wherein the sample data comprises an image sample or a behavior data sample;

inputting the sample data into the parameter generation model to obtain multiple paths of first parameters, wherein each path of first parameters corresponds to one dynamic parameter layer of the lightweight model of the end-side equipment; the parameter generation model comprises a plurality of data processing paths, each data processing path is used for outputting a path of first parameters, and the number of the data processing paths is the same as that of the dynamic parameter layers of the lightweight model;

and sending the multi-path first parameters to the end-side equipment so that the end-side equipment performs image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model, wherein the model parameters of the lightweight model comprise the multi-path first parameters and second parameters, and the second parameters are fixed parameters of the lightweight model.

11. The method of claim 10, wherein if the sample data is an image sample, the image sample is a first frame of consecutive image frames acquired by the end-side device;

inputting the sample data into the parameter generation model to obtain a first parameter of the lightweight model, wherein the method comprises the following steps: and inputting the image sample into the first parameter generation model to obtain a first parameter of the first lightweight model.

12. The method according to claim 11, wherein the first parametric generative model comprises an encoder head, a first multi-layer perceptron MLP, and a second MLP; inputting the image sample into the first parameter generation model to obtain a first parameter of the first lightweight model, including:

and inputting the image sample into an encoder head of the first parameter generation model, and sequentially performing data processing of a first MLP and a second MLP to obtain a first parameter of the first lightweight model.

13. The method of claim 12, wherein the model parameters of the first lightweight model include P ₁ First parameter of layer and M ₁ -P ₁ A second parameter of the layer, the second parameter being a static parameter, P ₁ And M ₁ -P ₁ Are all positive integers, M ₁ Is greater than P ₁ A positive integer of (d);

14. The method according to claim 10, wherein if the sample data is a behavior data sample, the behavior data sample is a series of behavior data of a target application program of a user on the end-side device;

inputting the sample data into the parameter generation model to obtain a first parameter of the lightweight model, wherein the method comprises the following steps: and inputting the behavior data sample into the second parameter generation model to obtain a first parameter of the second lightweight model.

15. The method of claim 14, wherein the second parametric generative model comprises a gated recurrent neural network GRU and a third MLP; inputting the behavior data sample into the second parameter generation model to obtain a first parameter of the second lightweight model, including:

16. The method of claim 15, wherein the model parameters of the second lightweight model include P ₂ First parameter of layer and M ₂ -P ₂ Second parameter of the layer, P ₂ And M ₂ -P ₂ Is a positive integer, M ₂ Is greater than P ₂ A positive integer of (d);

correspondingly, the GRU comprises P ₂ A GRU, the third MLP comprising P ₂ A third MLP.

17. The method according to any one of claims 10 to 16, wherein a parameter stabilizer is further deployed on the cloud-side device, and the parameter stabilizer is used for optimizing target layer parameters of a target MLP in the parameter generation model; the optimization process of the parameter stabilizer on the target layer parameters of the target MLP comprises the following steps:

acquiring target layer parameters of a plurality of sub-MLPs, wherein the plurality of sub-MLPs are used for constructing the target MLP;

wherein the target layer parameters comprise weight parameters of a target layer.

18. The method as claimed in claim 17, wherein the obtaining the target layer parameters of the target MLP by performing similarity calculation and weighted summation on the target layer parameters of the plurality of sub-MLPs comprises:

transposing the first matrix to obtain a second matrix;

performing similarity calculation and weighted summation on the parameters of the first matrix and the second matrix, and determining the weight of the target layer parameters of the plurality of sub MLPs;

19. The data processing method is characterized by being applied to end-side equipment, wherein a lightweight model is deployed on the end-side equipment and used for image recognition or behavior prediction; the method comprises the following steps:

receiving multiple paths of first parameters from the cloud side equipment, wherein the multiple paths of first parameters are dynamic parameters of the lightweight model generated by a parameter generation model deployed on the cloud side equipment based on the sample data; each path of first parameter corresponds to one dynamic parameter layer of the lightweight model of the end-side device, the parameter generation model comprises a plurality of paths of data processing paths, each path of data processing path is used for outputting one path of first parameter, and the number of the data processing paths is the same as that of the dynamic parameter layers of the lightweight model;

and performing image recognition or behavior prediction on the sample data based on the model parameters of the lightweight model, wherein the model parameters of the lightweight model comprise the multipath first parameters and second parameters, and the second parameters are fixed parameters of the lightweight model.

20. An electronic device, comprising: a memory and a processor;

the memory is to store program instructions;

the processor is for invoking program instructions in the memory to perform a data processing method as claimed in any one of claims 10 to 19.