CN114462082A - Model construction method and device, storage medium and computer equipment - Google Patents

Model construction method and device, storage medium and computer equipment Download PDF

Info

Publication number
CN114462082A
CN114462082A CN202210066155.XA CN202210066155A CN114462082A CN 114462082 A CN114462082 A CN 114462082A CN 202210066155 A CN202210066155 A CN 202210066155A CN 114462082 A CN114462082 A CN 114462082A
Authority
CN
China
Prior art keywords
data
model
training
shared
shared data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210066155.XA
Other languages
Chinese (zh)
Inventor
袁煜明
王峰
闫晨旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yancheng Matrix Operation Management Co ltd
Original Assignee
Yancheng Matrix Operation Management Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yancheng Matrix Operation Management Co ltd filed Critical Yancheng Matrix Operation Management Co ltd
Priority to CN202210066155.XA priority Critical patent/CN114462082A/en
Publication of CN114462082A publication Critical patent/CN114462082A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0609—Buyer or seller confidence or verification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Bioethics (AREA)
  • Business, Economics & Management (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a model building method and device, a storage medium and computer equipment, wherein a training model is built, the training model is uploaded to a first block chain node, a plurality of E-commerce platforms located at a second block chain node download the training model from the first block chain node, the training model is pre-trained to obtain a pre-trained target training model, local data stored by the E-commerce platforms are calculated by using the target training model to obtain output data, the output data are screened to obtain shared data, the shared data are uploaded to the first block chain node, and the training model is subjected to parameter updating by using the shared data to obtain the shared model. The method and the device realize the collaborative modeling on the premise of protecting the internal privacy of the E-commerce platform, effectively eliminate a large amount of redundant data and malicious data, improve the model optimization efficiency and the data utilization rate of the whole block chain system, and avoid the waste of computing resources caused by repeated training.

Description

Model construction method and device, storage medium and computer equipment
Technical Field
The invention relates to the technical field of computers, in particular to a model construction method and device, a storage medium and computer equipment.
Background
In recent years, cross-border e-commerce plays an important role in aspects of stable foreign trade, stable enterprise, promotion of digital transformation and the like, and with the continuous expansion of the scale of the cross-border e-commerce, the generation of data is exponentially increased, and the traditional data analysis model cannot gradually adapt to the existing huge data scale. Meanwhile, with the increasing strictness of privacy protection and supervision, data transmission and collaborative calculation among the e-commerce platforms become no longer feasible. Due to the large number of e-commerce platforms and the complex trust situation, there may be some malicious e-commerce platforms disguised by an attacker. The malicious e-commerce platform can induce the end user to be connected to the malicious e-commerce platform, and secretly collect a large amount of sensitive privacy data collected in other e-commerce platforms, so that the user privacy data is leaked. In addition, the malicious e-commerce platform may also tamper user data of other e-commerce platforms, and the calculation results of the other e-commerce platforms are affected by the tampered 'dirty' data. Therefore, a computing node from an unknown management domain needs to be subjected to reputation evaluation and dynamic authorization and management so as to avoid the malicious e-commerce platform from being added into the collaborative computing.
Disclosure of Invention
In view of the above problems, the invention provides a model construction method and device, a storage medium and computer equipment.
According to a first aspect of the present invention, there is provided a model construction method, the method comprising:
building a training model, and uploading the training model and preset model hyper-parameter information to the first block chain node;
downloading the training model from the first block chain link points by a plurality of E-commerce platforms positioned at second block chain nodes, and pre-training the training model to obtain a pre-trained target training model;
calculating local data stored by the E-commerce platform by using the target training model to obtain output data;
screening the local data according to the output data to obtain shared data; the shared data is partial local data of a plurality of E-commerce platforms;
uploading the shared data to the first block chain node, and updating parameters of the training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the e-commerce platform.
Optionally, the constructing a training model includes:
determining a neural network framework of the training model, and taking the neural network framework as the model framework;
initializing model parameters of the training model, and setting model hyper-parameter information; the model hyper-parameter information comprises Dropout hyper-parameters and parameter uploading thresholds.
Optionally, the pre-training the training model includes:
setting the Dropout hyper-parameter for the training model;
inputting the local data stored by the E-commerce platform into the training model in batches for pre-training.
Optionally, the screening the output data to obtain shared data includes:
calculating a variance value of the output data using the following formula:
Figure BDA0003480366660000021
wherein M is the number of pre-training rounds; the r is the average value of the output data of multiple times of training; the p (omega) represents output data of the current training round; the alpha is a variance value and is used as an uncertainty quantitative value of the local data of the batch;
and screening the output data by using the parameter uploading threshold value to obtain the output data with the variance value higher than the parameter uploading threshold value as target data, and taking the target data and the variance value corresponding to the target data as shared data.
Optionally, the uploading the shared data to the first blockchain node, and performing parameter update on the training model by using the shared data to obtain a shared model includes:
uploading the shared data to the first block chain node, and performing credit evaluation on the e-commerce platforms by using the shared data to obtain credit values of the e-commerce platforms;
calculating the updating parameters of the training model by using the credit values corresponding to the E-commerce platforms and the data quantity of the shared data, and specifically calculating by using the following formula:
Figure BDA0003480366660000031
wherein t is the number of training rounds; omegat+1Updating parameters of the training model; n iskThe data volume of the shared data corresponding to the kth e-commerce platform; a is saidkUpdating the weight value for the credit value of the kth E-commerce platform;
and updating the parameters of the training model by using the updating parameters to obtain the sharing model.
Optionally, the uploading the shared data to the first blockchain node, and performing reputation evaluation on the e-commerce platform by using the shared data includes:
uploading the shared data of the E-commerce platforms to the first block chain node, and counting the shared data to obtain the data volume of the shared data corresponding to the E-commerce platforms;
and calculating a total uncertainty value of the shared data by using the data volume and the variance of the shared data, and taking the total uncertainty value as a reputation value of the E-commerce platform.
Optionally, the method further comprises:
encrypting the shared data of the E-commerce platforms by using a public key to generate encrypted data;
the E-commerce platform downloads the encrypted data from the first block link point, and decrypts the encrypted data by using a private key to obtain shared data of a plurality of E-commerce platforms;
the E-commerce platform optimizes the shared data of the E-commerce platforms to the target training model, and predicts local data stored by the E-commerce platform by using the optimized target training model; and/or the presence of a gas in the gas,
and the E-commerce platform acquires the sharing model from the first block link point, and predicts local data stored by the E-commerce platform by using the sharing model.
According to a second aspect of the present invention, there is provided a model building apparatus comprising:
a model building apparatus, characterized in that the apparatus comprises:
the training model building module is used for building a training model and uploading the training model and preset model hyper-parameter information to the first block chain node;
the model pre-training module is used for downloading the training models from the first block chain link points by a plurality of E-commerce platforms positioned at second block chain nodes, and pre-training the training models to obtain target training models completing the pre-training;
the output data calculation module is used for calculating the local data stored by the E-commerce platform by using the target training model to obtain output data;
the shared data screening module is used for screening the local data according to the output data to obtain shared data; the shared data is partial local data of a plurality of E-commerce platforms;
the shared model generation module is used for uploading the shared data to the first block chain node, and updating parameters of the training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the e-commerce platform.
According to a third aspect of the present invention, a computer-readable storage medium is proposed, on which a computer program is stored which, when being executed by a processor, carries out the steps of the model building method according to any one of the first aspect of the present invention.
According to a fourth aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the model construction method according to any one of the first aspect of the present invention when executing the computer program.
According to the model building method and device, the storage medium and the computer equipment, the training model is built, the training model is uploaded to the first block chain node, the training model is downloaded from the first block chain node by a plurality of E-commerce platforms located at the second block chain node, the training model is pre-trained to obtain a pre-trained target training model, local data stored by the E-commerce platforms are calculated by the target training model to obtain output data, the output data are screened to obtain shared data, the shared data are uploaded to the first block chain node, and the training model is subjected to parameter updating by the shared data to obtain the shared model. According to the method, the training model is built, the plurality of E-commerce platforms train the training model to obtain uncertainty data with large variance, the uncertainty data of the plurality of E-commerce platforms are counted to serve as shared data, the training model is subjected to parameter updating to obtain the shared model, the accuracy of the shared model is greatly improved, malicious E-commerce organizations with a large amount of redundant data and dirty data are effectively eliminated to participate in data sharing and collaborative calculation, and the model optimization efficiency of the whole block chain system is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating a model building method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a model building apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a model building apparatus according to another embodiment of the present invention;
fig. 4 shows a physical structure diagram of a computer device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiment of the present invention provides a model building method, an application and a block chain system, where the block chain system includes a plurality of block chain nodes, and as shown in fig. 1, the method at least includes the following steps S101 to S105:
and S101, constructing a training model, and uploading the training model and preset model hyper-parameter information to a first block chain node.
The model construction process in the embodiment of the invention is performed on a blockchain system, the blockchain system comprises a plurality of blockchain nodes, wherein a first blockchain node can be a central node used for training model release and data transmission and is generally served by a unified financial institution or a management department, a second blockchain node is a general name of the blockchain nodes corresponding to a plurality of e-commerce platforms in the blockchain system, and the second blockchain node can interact with the first blockchain node so that the e-commerce platforms participate in data sharing and collaborative calculation in the blockchain system. In general, the block link point may be a computer device requiring a certain storage space, such as a computer, a mobile phone, etc.
In an optional embodiment of the invention, a training model is constructed, specifically, a neural network framework of the training model can be determined, and the neural network framework is used as a model framework; initializing model parameters of the training model, and setting model hyper-parameter information; the model hyper-parameter information comprises Dropout hyper-parameters and parameter uploading thresholds.
The training model is a model for predicting local data of the e-commerce platform, for example, the training model may be a financing model for financing prediction of capital circulation data of the e-commerce platform, an analysis model for sales condition prediction of sales data of the e-commerce platform, and the like, which is not limited in the present invention.
Firstly, determining a needed neural network architecture according to a specific prediction type of a training model, setting Dropout hyper-parameters and parameter uploading thresholds after model parameter initialization is completed, and then uploading a network structure and the hyper-parameters to a first block chain node for all E-commerce platforms participating in data sharing and collaborative computing to obtain. The hyper-parameters are predetermined training variables before the model starts to train, and the parameter uploading threshold is a data screening basis for subsequently updating parameters of the training model.
And S102, downloading training models from the first block chain link points by a plurality of E-commerce platforms positioned at the second block chain nodes, and pre-training the training models to obtain the pre-trained target training models.
In an embodiment of the present invention, each e-commerce platform may download a unified training model from the first blockchain node at its corresponding second blockchain node.
In an optional embodiment of the invention, the training model is pre-trained, and specifically, a Dropout hyper-parameter can be set for the training model; inputting local data stored by the E-commerce platform into the training model in batches for pre-training.
And after the E-commerce platform acquires the training model, setting a uniform Dropout hyper-parameter for the training model, dividing local data into batches, inputting the batches into the training model for pre-training, and obtaining a target training model after each round of pre-training is completed.
And step S103, calculating local data stored in the E-commerce platform by using the target training model to obtain output data.
Since the local data is subjected to multiple rounds of calculation in batches in step S102, an output result of calculating the local data of the batch by the target training model after each round of pre-training is completed can be obtained.
Step S104, screening the local data according to the output data to obtain shared data; the shared data is part of local data of a plurality of e-commerce platforms.
In an optional embodiment of the present invention, the screening the output data to obtain the shared data includes: the variance value of the output data is calculated using the following formula:
Figure BDA0003480366660000071
wherein M is the number of pre-training rounds; r is the average value of the output data of multiple times of training; p (ω) represents the output data of the current round of training; alpha is a variance value and is used as an uncertainty quantized value of the local data of the batch; and screening the local data by using the parameter uploading threshold value to obtain the local data with the variance value higher than the parameter uploading threshold value as target data, and taking the target data and the variance value corresponding to the target data as shared data.
That is to say, all local data are uploaded to the first block chain node according to the requirement of the parameter uploading threshold, the local data with the variance value higher than the parameter uploading threshold are uploaded as uncertain data together with the variance value, the local data with the variance value lower than the parameter uploading threshold are left in the local, and the local data are not uploaded and do not participate in the next round of model pre-training. And taking the local data uploaded to the first block chain node as shared data so as to update parameters of a subsequent sharing model and share data of a plurality of E-commerce platforms.
Since the local target training model is updated according to the previous round of pre-training of the local data, therefore, the method has good screening value for a large amount of repeated local data, the output result of redundant data after multiple rounds of input tends to be stable, so the variance value is very small and has no training and uploading value any more, for abnormal data or dirty data with extremely low value, the output data fluctuation of the target training model becomes extremely unstable due to under-fitting, the variance of the output data will be very large, such data will be outside the filtering range, data exceeding the filtering threshold will be filtered out at this step, therefore, uncertainty analysis plays a very important role in screening data, on one hand, the uploaded data amount is reduced, and on the other hand, the storage pressure of a block chain system and the waste of computational resources of model repeated training are also reduced.
Step S105, uploading shared data to a first block chain node, and updating parameters of a training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the E-commerce platform.
In an optional embodiment of the invention, the shared data is used for updating parameters of the training model, the shared data is firstly uploaded to the first block chain node, and the credit evaluation is performed on the e-commerce platforms by using the shared data to obtain the credit values of the e-commerce platforms.
The credit evaluation is carried out on the e-commerce platform by using the shared data, and the credit evaluation specifically comprises the steps of carrying out statistics on the shared data to obtain the data volume of the shared data corresponding to the e-commerce platform; and calculating a total uncertainty value of the shared data by using the data volume and the variance of the shared data, and taking the total uncertainty value as a reputation value of the E-commerce platform.
That is to say, after the first block chain system acquires the shared data uploaded by the e-commerce platforms, the first block chain system performs statistics on the shared data, calculates the data volume of the shared data corresponding to the e-commerce platforms, and calculates the total uncertainty value, namely the total variance, of the shared data according to the data volume of the shared data and the corresponding variance value, and is also used for representing the credit value of the e-commerce platforms.
After the credit values of the e-commerce platforms are obtained, calculating the updating parameters of the training model by using the credit values corresponding to the e-commerce platforms and the data volume of the shared data, and specifically calculating by using the following formula:
Figure BDA0003480366660000091
wherein t is the number of training rounds; omegat+1Updating parameters for the training model; n iskThe data volume of the shared data corresponding to the kth e-commerce platform; alpha is alphakUpdating the weight value for the credit value of the kth E-commerce platform; training model using updated parametersAnd updating parameters to generate a sharing model.
In the embodiment of the invention, after the credit values and the data volumes corresponding to a plurality of e-commerce platforms are collected by the first block chain node, parameter updating can be performed on the shared model obtained in the previous round by using the credit values corresponding to the e-commerce platforms and the data volume of the shared data, the e-commerce platforms with high uncertain values have higher weight ratio in the current round of aggregation according to the uncertain value of the shared data, namely the credit value, as the weight value of the model aggregation, the data volume can occupy higher data weight in the parameter updating process for the e-commerce platforms with high credit values, the utilization rate of the shared data is improved, the shared model for completing the new round of parameter updating is obtained, and each e-commerce platform can download to replace a local target training model for predicting local data.
In an optional embodiment of the present invention, the shared data of a plurality of e-commerce platforms may be encrypted by using a public key to generate encrypted data; downloading the encrypted data from the chain link points of the first block by the e-commerce platform, and decrypting the encrypted data by using a private key to obtain shared data of a plurality of e-commerce platforms; and the E-commerce platform optimizes the shared data of the E-commerce platforms to the target training model, and predicts local data stored by the E-commerce platform by using the optimized target training model.
In another optional embodiment of the present invention, the e-commerce platform may further obtain a shared model from the first block link point, and use the shared model to predict local data stored by the e-commerce platform.
That is to say, after each e-commerce platform can obtain shared data of multiple e-commerce platforms through the first block link point, the shared data is used to optimize the local target training model to obtain a target training model with higher accuracy, or the shared model after completing parameter updating can be directly obtained, the local data is predicted by using the shared model, and the accuracy of data prediction is improved.
In addition, the e-commerce platform can check historical credit values of other e-commerce platforms to determine whether to download the shared data of the e-commerce platform when selecting to download the shared data, and the first block link point can also select whether to optimize the parameters of the shared model according to the shared data according to the historical credit values of the e-commerce platform, so that the e-commerce platform with a large amount of redundant data and dirty data is eliminated from participating in data sharing and collaborative calculation, and the model optimization efficiency of the whole block link system is improved.
The embodiment of the invention provides a model construction method and device, a storage medium and computer equipment, wherein a training model is constructed, the training model is uploaded to a first block chain node, a plurality of E-commerce platforms positioned at a second block chain node download the training model from the first block chain node, the training model is pre-trained to obtain a pre-trained target training model, the target training model is used for calculating local data stored by the E-commerce platforms to obtain output data, the output data is screened to obtain shared data, the shared data is uploaded to the first block chain node, the training model is subjected to parameter updating by using the shared data to obtain a shared model, malicious E-commerce mechanisms with a large amount of redundant data and dirty data are effectively eliminated to participate in data sharing and collaborative calculation, and collaboration is realized on the premise of ensuring that the internal privacy of the E-commerce platform is not leaked, the data utilization rate and the model accuracy are jointly improved, the model optimization efficiency of the whole block chain system is improved, and the waste of computing resources caused by repeated training is avoided.
Further, as a specific implementation of fig. 1, an embodiment of the present invention provides a model building apparatus, as shown in fig. 2, the apparatus may include: a training model building module 210, a model pre-training module 220, an output data calculation module 230, a shared data screening module 240, and a shared model generation module 250.
The training model building module 210 may be configured to build a training model, and upload the training model and preset model hyper-parameter information to the first blockchain node.
The model pre-training module 220 may be configured to download training models from the first block link points by a plurality of e-commerce platforms located at the second block link nodes, and pre-train the training models to obtain a target training model for completing the pre-training.
The output data calculation module 230 may be configured to calculate local data stored in the e-commerce platform by using the target training model to obtain output data.
The shared data screening module 240 may be configured to perform screening processing on the local data according to the output data to obtain shared data; the shared data is part of local data of a plurality of e-commerce platforms.
The shared model generating module 250 may be configured to upload shared data to the first block chain node, and perform parameter update on the training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the E-commerce platform.
Optionally, as shown in fig. 3, the model building apparatus provided in the embodiment of the present invention may further include: the shared resource acquisition module 260.
The shared resource obtaining module 260 may be configured to encrypt shared data of multiple e-commerce platforms by using a public key to generate encrypted data; downloading the encrypted data from the chain link points of the first block by the e-commerce platform, and decrypting the encrypted data by using a private key to obtain shared data of a plurality of e-commerce platforms; the e-commerce platform optimizes the shared data of the e-commerce platforms to a target training model, and predicts local data stored by the e-commerce platform by using the optimized target training model; and/or the e-commerce platform acquires a sharing model from the first block link point, and local data stored by the e-commerce platform is predicted by using the sharing model.
Optionally, the training model building module 210 may be further configured to determine a neural network framework of the training model, and use the neural network framework as the model framework; initializing model parameters of the training model, and setting model hyper-parameter information; the model hyper-parameter information comprises Dropout hyper-parameters and parameter uploading thresholds.
Optionally, the model pre-training module 220 may also be configured to set a Dropout hyper-parameter for the training model; inputting local data stored by the E-commerce platform into the training model in batches for pre-training.
Optionally, the shared data filtering module 240 may be further configured to calculate a variance value of the output data by using the following formula:
Figure BDA0003480366660000111
wherein M is the number of pre-training rounds; r is the average value of the output data of multiple times of training; p (ω) represents the output data of the current round of training; alpha is a variance value and is used as an uncertainty quantized value of the local data of the batch; and screening the local data by using the parameter uploading threshold value to obtain the local data with the variance value higher than the parameter uploading threshold value as target data, and taking the target data and the variance value corresponding to the target data as shared data.
Optionally, the sharing model generating module 250 may be further configured to upload the shared data to the first block chain node, and perform reputation evaluation on the e-commerce platforms by using the shared data to obtain reputation values of the e-commerce platforms; calculating the updating parameters of the training model by using the credit values corresponding to the E-commerce platforms and the data quantity of the shared data, and specifically calculating by using the following formula:
Figure BDA0003480366660000121
wherein t is the number of training rounds; omegat+1Updating parameters for the training model; n iskThe data volume of the shared data corresponding to the kth e-commerce platform; alpha is alphakUpdating the weight value for the credit value of the kth E-commerce platform; and updating the parameters of the training model by using the updated parameters to obtain a shared model.
Optionally, the sharing model generating module 250 may be further configured to upload shared data of multiple e-commerce platforms to the first block chain node, and count the shared data to obtain a data amount of the shared data corresponding to the e-commerce platform; and calculating a total uncertainty value of the shared data by using the data volume and the variance of the shared data, and taking the total uncertainty value as a reputation value of the E-commerce platform.
It should be noted that other corresponding descriptions of the functional modules related to the model building apparatus provided in the embodiment of the present invention may refer to the corresponding description of the method shown in fig. 1, and are not described herein again.
Based on the method shown in fig. 1, correspondingly, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the model building method according to any of the embodiments.
Based on the above embodiments of the method shown in fig. 1 and the apparatus shown in fig. 3, an embodiment of the present invention further provides an entity structure diagram of a computer device, as shown in fig. 4, the computer device may include a communication bus, a processor, a memory, and a communication interface, and may further include an input/output interface and a display device, where the functional units may complete communication with each other through the bus. The memory stores computer programs, and the processor is used for executing the programs stored in the memory and executing the steps of the model building method described in the above embodiment.
It is clear to those skilled in the art that the specific working processes of the above-described systems, devices, modules and units may refer to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, further description is omitted here.
In addition, the functional units in the embodiments of the present invention may be physically independent of each other, two or more functional units may be integrated together, or all the functional units may be integrated in one processing unit. The integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.
Those of ordinary skill in the art will understand that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computing device (e.g., a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention when the instructions are executed. And the aforementioned storage medium includes: u disk, removable hard disk, Read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disk, and other various media capable of storing program code.
Alternatively, all or part of the steps of implementing the foregoing method embodiments may be implemented by hardware (such as a computing device, e.g., a personal computer, a server, or a network device) associated with program instructions, which may be stored in a computer-readable storage medium, and when the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method according to the embodiments of the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments can be modified or some or all of the technical features can be equivalently replaced within the spirit and principle of the present invention; such modifications or substitutions do not depart from the scope of the present invention.

Claims (10)

1. A model building method is applied to a block chain system, and comprises the following steps:
building a training model, and uploading the training model and preset model hyper-parameter information to the first block chain node;
downloading the training model from the first block chain link points by a plurality of E-commerce platforms positioned at second block chain nodes, and pre-training the training model to obtain a pre-trained target training model;
calculating local data stored by the E-commerce platform by using the target training model to obtain output data;
screening the local data according to the output data to obtain shared data; the shared data is partial local data of a plurality of E-commerce platforms;
uploading the shared data to the first block chain node, and updating parameters of the training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the e-commerce platform.
2. The method of claim 1, wherein the constructing a training model comprises:
determining a neural network framework of the training model, and taking the neural network framework as the model framework;
initializing model parameters of the training model, and setting model hyper-parameter information; the model hyper-parameter information comprises Dropout hyper-parameters and parameter uploading thresholds.
3. The method of claim 2, wherein the pre-training the training model comprises:
setting the Dropout hyper-parameter for the training model;
inputting the local data stored by the E-commerce platform into the training model in batches for pre-training.
4. The method according to claim 1, wherein the local data is filtered according to the output data to obtain shared data; the shared data is part of local data of a plurality of E-commerce platforms, and comprises the following steps:
calculating a variance value of the output data using the following formula:
Figure FDA0003480366650000021
wherein M is the number of pre-training rounds; the r is the average value of the output data of multiple times of training; the p (omega) represents output data of the current training round; the alpha is a variance value and is used as an uncertainty quantitative value of the local data of the batch;
and screening the local data by using the parameter uploading threshold value to obtain the local data with the variance value higher than the parameter uploading threshold value as target data, and taking the target data and the variance value corresponding to the target data as shared data.
5. The method of claim 4, wherein uploading the shared data to the first blockchain node, and performing parameter update on the training model using the shared data to obtain a shared model comprises:
uploading the shared data to the first block chain node, and performing credit evaluation on the e-commerce platforms by using the shared data to obtain credit values of the e-commerce platforms;
calculating the updating parameters of the training model by using the credit values corresponding to the E-commerce platforms and the data quantity of the shared data, and specifically calculating by using the following formula:
Figure FDA0003480366650000022
wherein t is the number of training rounds; omegat+1Updating parameters of the training model; n iskThe data volume of the shared data corresponding to the kth e-commerce platform; a is saidkUpdating the weight value for the credit value of the kth E-commerce platform;
and updating the parameters of the training model by using the updating parameters to obtain the sharing model.
6. The method of claim 5, wherein uploading the shared data to the first blockchain node, and using the shared data to perform a reputation evaluation on the e-commerce platform comprises:
uploading the shared data of the E-commerce platforms to the first block chain node, and counting the shared data to obtain the data volume of the shared data corresponding to the E-commerce platforms;
and calculating a total uncertainty value of the shared data by using the data volume and the variance of the shared data, and taking the total uncertainty value as a reputation value of the E-commerce platform.
7. The method of any one of claims 1-6, further comprising:
encrypting the shared data of the E-commerce platforms by using a public key to generate encrypted data;
the E-commerce platform downloads the encrypted data from the first block link point, and decrypts the encrypted data by using a private key to obtain shared data of a plurality of E-commerce platforms;
the E-commerce platform optimizes the shared data of the E-commerce platforms to the target training model, and predicts local data stored by the E-commerce platform by using the optimized target training model; and/or the presence of a gas in the gas,
and the E-commerce platform acquires the sharing model from the first block link point, and predicts local data stored by the E-commerce platform by using the sharing model.
8. A model building apparatus, characterized in that the apparatus comprises:
the training model building module is used for building a training model and uploading the training model and preset model hyper-parameter information to the first block chain node;
the model pre-training module is used for downloading the training models from the first block chain link points by a plurality of E-commerce platforms positioned at second block chain nodes, and pre-training the training models to obtain target training models completing the pre-training;
the output data calculation module is used for calculating the local data stored by the E-commerce platform by using the target training model to obtain output data;
the shared data screening module is used for screening the local data according to the output data to obtain shared data; the shared data is partial local data of a plurality of E-commerce platforms;
the shared model generation module is used for uploading the shared data to the first block chain node, and updating parameters of the training model by using the shared data to obtain a shared model; the sharing model is used for predicting local data of the e-commerce platform.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the model building method according to any one of claims 1 to 7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the model building method according to any one of claims 1 to 7 when executing the computer program.
CN202210066155.XA 2022-01-20 2022-01-20 Model construction method and device, storage medium and computer equipment Pending CN114462082A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210066155.XA CN114462082A (en) 2022-01-20 2022-01-20 Model construction method and device, storage medium and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210066155.XA CN114462082A (en) 2022-01-20 2022-01-20 Model construction method and device, storage medium and computer equipment

Publications (1)

Publication Number Publication Date
CN114462082A true CN114462082A (en) 2022-05-10

Family

ID=81409897

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210066155.XA Pending CN114462082A (en) 2022-01-20 2022-01-20 Model construction method and device, storage medium and computer equipment

Country Status (1)

Country Link
CN (1) CN114462082A (en)

Similar Documents

Publication Publication Date Title
CN104424354B (en) The method and system of generation model detection abnormal user behavior is operated using user
US20220043920A1 (en) Blockchain-based secure federated learning
Kondratenko et al. Multi-criteria decision making for selecting a rational IoT platform
CA3080050A1 (en) Training tree-based machine-learning modeling algorithms for predicting outputs and generating explanatory data
CN111325417B (en) Method and device for realizing privacy protection and realizing multi-party collaborative updating of business prediction model
TWI724896B (en) Method and device for constructing relational network based on privacy protection
Vanli et al. Sequential nonlinear learning for distributed multiagent systems via extreme learning machines
CN112039702B (en) Model parameter training method and device based on federal learning and mutual learning
CN113486584B (en) Method and device for predicting equipment failure, computer equipment and computer readable storage medium
WO2019137049A1 (en) Prediction method and apparatus based on information sharing, electronic device and computer storage medium
CN113887845B (en) Extreme event prediction method, device, equipment and storage medium
CN111914516B (en) Method, device, equipment and storage medium for generating network data prediction sequence
Fu et al. GRAP: Grey risk assessment based on projection in ad hoc networks
CN112232833A (en) Lost member customer group data prediction method, model training method and model training device
CN112073126A (en) Method and device for ordering network node importance
CN110825589A (en) Anomaly detection method and device for micro-service system and electronic equipment
Akter et al. Edge intelligence-based privacy protection framework for iot-based smart healthcare systems
CN113689270A (en) Method for determining black product device, electronic device, storage medium, and program product
Ricardo et al. Developing machine learning and deep learning models for host overload detection in cloud data center
CN115174237B (en) Method and device for detecting malicious traffic of Internet of things system and electronic equipment
CN114462082A (en) Model construction method and device, storage medium and computer equipment
Yang et al. Neural network based min-entropy estimation for random number generators
US20210342691A1 (en) System and method for neural time series preprocessing
CN115249175A (en) Self-attention sequence recommendation method, system and equipment integrating time information
AU2021218217A1 (en) Systems and methods for preventative monitoring using AI learning of outcomes and responses from previous experience.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination