CN113486378A - System for realizing data set construction processing based on federal learning and generation method thereof - Google Patents
System for realizing data set construction processing based on federal learning and generation method thereof Download PDFInfo
- Publication number
- CN113486378A CN113486378A CN202110833141.1A CN202110833141A CN113486378A CN 113486378 A CN113486378 A CN 113486378A CN 202110833141 A CN202110833141 A CN 202110833141A CN 113486378 A CN113486378 A CN 113486378A
- Authority
- CN
- China
- Prior art keywords
- data
- data set
- module
- unit
- encryption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 title claims abstract description 20
- 238000010276 construction Methods 0.000 title claims abstract description 14
- 238000004458 analytical method Methods 0.000 claims abstract description 20
- 238000007726 management method Methods 0.000 claims description 22
- 238000012544 monitoring process Methods 0.000 claims description 12
- 238000013500 data storage Methods 0.000 claims description 6
- 238000013524 data verification Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000005856 abnormality Effects 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Abstract
The invention discloses a system for realizing data set construction processing based on federal learning and a generation method thereof, wherein the system for realizing data set construction processing based on federal learning comprises a data set management unit, a data set analysis unit, a data set encryption unit and a data set summarization unit; the data set management unit is used for receiving and uploading data; the data set analysis unit is used for analyzing data; the invention has the beneficial effects that: the system for realizing the data set construction processing based on the federal learning and the generation method thereof improve the safety of the access process of the data set, simplify the steps of the encryption process and the decryption process of the data set, shorten the access time and improve the use experience of users.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a system for realizing data set construction processing based on federal learning and a generation method thereof.
Background
Federal learning (federal learning) is a new artificial intelligence basic technology, which was proposed by google in 2016, and the design goal is to develop efficient machine learning among multiple parties or multiple computing nodes on the premise of guaranteeing information security during big data exchange, protecting terminal data and personal data privacy and guaranteeing legal compliance. The federal study needs a large amount of data sets in the training and model evaluation stages, but the security risk of the data sets in the actual use process is too large, the encryption and decryption processes of the data sets are too tedious, and a large amount of time is wasted.
Disclosure of Invention
The invention aims to provide a system for realizing data set construction processing based on federal learning and a generation method thereof, so as to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme: the system for realizing data set construction processing based on federal learning comprises a data set management unit, a data set analysis unit, a data set encryption unit and a data set summarization unit;
the data set management unit is used for receiving and uploading data;
the data set analysis unit is used for analyzing data;
the data set encryption unit is used for encrypting data;
the data set summarizing unit is used for storing data.
Preferably, the data set management unit comprises a data reading module, a data acquisition module and a data uploading module;
the data reading module can read historical data;
the data acquisition module is used for collecting newly acquired data;
the data uploading module is used for uploading historical data and newly acquired data.
Preferably, the data set analysis unit comprises a data receiving module and a data classification module;
the data receiving module is used for receiving data;
the data classification module is used for classifying and processing data.
Preferably, the data set encryption unit comprises a data encryption module, a data verification module, a secret key management module and a data decryption module;
the data encryption module is used for encrypting data;
the data verification is used for verifying the user identity;
the secret key management module is used for issuing a secret key token;
the data decryption module is used for decrypting the encrypted data.
Preferably, the data set summarizing unit comprises a data storage module, a data backup module and a data monitoring module, and the data storage module is in bidirectional electric connection with the data backup module;
the data monitoring module is used for monitoring the stored data and giving an abnormal alarm.
Preferably, the data acquisition module transmits data to the data server through a data transmission link, the input end of the data acquisition module is connected with the output end of the production equipment, and the output end of the data acquisition module is electrically connected with an RS232 serial port.
Preferably, the data set management unit, the data set analysis unit, the data set encryption unit and the data set summarization unit are all electrically connected with the data set processing system.
The invention also provides a generation method for realizing data set construction processing based on federal learning, which comprises the following steps:
s1, reading historical data, acquiring new data and uploading the data through a data set management unit;
s2, receiving data through a data set analysis unit, and then classifying the received data;
s3, encrypting the classified data through the data set encryption unit, then carrying out identity verification, and sending a key token for decryption after passing the verification;
s4, storing the decrypted data through the data set summarizing unit, backing up the data, monitoring the data and alarming for abnormality
Compared with the prior art, the invention has the beneficial effects that: the system for realizing the data set construction processing based on the federal learning and the generation method thereof improve the safety of the access process of the data set, simplify the steps of the encryption process and the decryption process of the data set, shorten the access time and improve the use experience of users.
Drawings
FIG. 1 is a block diagram of the overall structure of the present invention;
FIG. 2 is a block diagram of a data set management unit of the present invention;
FIG. 3 is a block diagram of a data set analysis unit according to the present invention;
FIG. 4 is a block diagram of a data set encryption unit according to the present invention;
FIG. 5 is a block diagram of a data set summary unit structure according to the present invention;
FIG. 6 is a flow chart of the method of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-6, the present invention provides a technical solution: the method comprises the following steps: the system for realizing data set construction processing based on federal learning comprises a data set management unit, a data set analysis unit, a data set encryption unit and a data set summarization unit;
the data set management unit is used for receiving and uploading data;
the data set analysis unit is used for analyzing data; it should be appreciated that the data set analysis unit may be operable to receive data uploaded by the data management unit and then analyze the data;
the data set encryption unit is used for encrypting data; it should be understood that the data set encryption unit may be configured to receive the data analyzed by the data set analysis unit and then encrypt the data;
the data set summarizing unit is used for storing data; it should be understood that the data set summary unit may be configured to receive the data encrypted by the data set encryption unit and then store the data.
Preferably, the data set management unit comprises a data reading module, a data acquisition module and a data uploading module;
the data reading module can read historical data;
the data acquisition module is used for collecting newly acquired data;
the data uploading module is used for uploading historical data and newly acquired data.
Preferably, the data set analysis unit comprises a data receiving module and a data classification module;
the data receiving module is used for receiving data;
the data classification module is used for classifying and processing data.
Preferably, the data set encryption unit comprises a data encryption module, a data verification module, a secret key management module and a data decryption module;
the data encryption module is used for encrypting data;
the data verification is used for verifying the user identity;
the secret key management module is used for issuing a secret key token;
the data decryption module is used for decrypting the encrypted data.
Preferably, the data set summarizing unit comprises a data storage module, a data backup module and a data monitoring module, and the data storage module is in bidirectional electric connection with the data backup module;
the data monitoring module is used for monitoring the stored data and giving an abnormal alarm.
Preferably, the data acquisition module transmits data to the data server through a data transmission link, the input end of the data acquisition module is connected with the output end of the production equipment, and the output end of the data acquisition module is electrically connected with an RS232 serial port.
Preferably, the data set management unit, the data set analysis unit, the data set encryption unit and the data set summarization unit are all electrically connected with the data set processing system.
The invention also provides a generation method for realizing data set construction processing based on federal learning, which comprises the following steps:
s1, reading historical data, acquiring new data and uploading the data through a data set management unit;
s2, receiving data through a data set analysis unit, and then classifying the received data;
s3, encrypting the classified data through the data set encryption unit, then carrying out identity verification, and sending a key token for decryption after passing the verification;
and S4, storing the decrypted data through the data set summarizing unit, backing up the data, and monitoring and alarming for abnormality.
In the description of the present invention, it is to be understood that the terms "coaxial", "bottom", "one end", "top", "middle", "other end", "upper", "one side", "top", "inner", "front", "center", "both ends", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention.
Furthermore, the terms "first", "second", "third", "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, whereby the features defined as "first", "second", "third", "fourth" may explicitly or implicitly include at least one such feature.
In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "disposed," "connected," "secured," "screwed" and the like are to be construed broadly, e.g., as meaning fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; the terms may be directly connected or indirectly connected through an intermediate, and may be communication between two elements or interaction relationship between two elements, unless otherwise specifically limited, and the specific meaning of the terms in the present invention will be understood by those skilled in the art according to specific situations.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. The system for realizing data set construction processing based on federal learning is characterized by comprising a data set management unit, a data set analysis unit, a data set encryption unit and a data set summarization unit;
the data set management unit is used for receiving and uploading data;
the data set analysis unit is used for analyzing data;
the data set encryption unit is used for encrypting data;
the data set summarizing unit is used for storing data.
2. The system for implementing a data set building process based on federated learning of claim 1, wherein: the data set management unit comprises a data reading module, a data acquisition module and a data uploading module;
the data reading module can read historical data;
the data acquisition module is used for collecting newly acquired data;
the data uploading module is used for uploading historical data and newly acquired data.
3. The system for implementing a data set building process based on federated learning of claim 1, wherein: the data set analysis unit comprises a data receiving module and a data classification module;
the data receiving module is used for receiving data;
the data classification module is used for classifying and processing data.
4. The system for implementing a data set building process based on federated learning of claim 3, wherein: the data set encryption unit comprises a data encryption module, a data verification module, a secret key management module and a data decryption module;
the data encryption module is used for encrypting data;
the data verification is used for verifying the user identity;
the secret key management module is used for issuing a secret key token;
the data decryption module is used for decrypting the encrypted data.
5. The system for implementing a data set building process based on federated learning of claim 4, wherein: the data set summarizing unit comprises a data storage module, a data backup module and a data monitoring module, wherein the data storage module is in bidirectional electric connection with the data backup module;
the data monitoring module is used for monitoring the stored data and giving an abnormal alarm.
6. The system for implementing a data set building process based on federated learning of claim 5, wherein: the data acquisition module transmits data to the data server through a data transmission link, the input end of the data acquisition module is connected with the output end of the production equipment, and the output end of the data acquisition module is electrically connected with an RS232 serial port.
7. The system for implementing a data set building process based on federated learning of claim 6, wherein: the data set management unit, the data set analysis unit, the data set encryption unit and the data set summarization unit are all electrically connected with the data set processing system.
8. The method for generating a system for implementing a data set construction process based on federal learning as claimed in any one of claims 1 to 7, comprising the steps of:
s1: reading historical data, acquiring new data and uploading the data through a data set management unit;
s2: receiving data through a data set analysis unit, and then classifying the received data;
s3: the classified data are encrypted through a data set encryption unit, then identity verification is carried out, and a key token is sent for decryption after the verification is passed;
s4: and storing the decrypted data through the data set summarizing unit, backing up the data, and monitoring and alarming for abnormality of the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833141.1A CN113486378A (en) | 2021-07-22 | 2021-07-22 | System for realizing data set construction processing based on federal learning and generation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833141.1A CN113486378A (en) | 2021-07-22 | 2021-07-22 | System for realizing data set construction processing based on federal learning and generation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113486378A true CN113486378A (en) | 2021-10-08 |
Family
ID=77942140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110833141.1A Pending CN113486378A (en) | 2021-07-22 | 2021-07-22 | System for realizing data set construction processing based on federal learning and generation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113486378A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104731A (en) * | 2019-11-19 | 2020-05-05 | 北京集奥聚合科技有限公司 | Graphical model full-life-cycle modeling method for federal learning |
| CN111695701A (en) * | 2020-06-12 | 2020-09-22 | 上海富数科技有限公司 | System for realizing data set construction processing based on federal learning and construction generation method thereof |
| WO2021022707A1 (en) * | 2019-08-06 | 2021-02-11 | 深圳前海微众银行股份有限公司 | Hybrid federated learning method and architecture |
| CN112434334A (en) * | 2020-11-25 | 2021-03-02 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and storage medium |
-
2021
- 2021-07-22 CN CN202110833141.1A patent/CN113486378A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021022707A1 (en) * | 2019-08-06 | 2021-02-11 | 深圳前海微众银行股份有限公司 | Hybrid federated learning method and architecture |
| CN111104731A (en) * | 2019-11-19 | 2020-05-05 | 北京集奥聚合科技有限公司 | Graphical model full-life-cycle modeling method for federal learning |
| CN111695701A (en) * | 2020-06-12 | 2020-09-22 | 上海富数科技有限公司 | System for realizing data set construction processing based on federal learning and construction generation method thereof |
| CN112434334A (en) * | 2020-11-25 | 2021-03-02 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108681966B (en) | Information supervision method and device based on block chain | |
| CN102036231B (en) | Network architecture security system for Internet of Things and security method thereof | |
| CN108009830A (en) | Products in circulation tracking and system based on block chain | |
| CN109547477A (en) | A kind of data processing method and its device, medium, terminal | |
| CN111127254B (en) | Marketing and distribution through data management system | |
| CN106209385A (en) | A kind of electrical network detection device based on multiple authentication manipulation authority | |
| CN107370601A (en) | A kind of intelligent terminal, system and method for integrating a variety of safety certifications | |
| CN111625870A (en) | Block chain based security audit method, system and storage medium | |
| CN207283595U (en) | User power utilization data analysis integrated system based on power information collection | |
| CN112134899A (en) | Factory terminal control double identity authentication method based on vision and network security | |
| CN113343309B (en) | Natural person database privacy security protection method and device and terminal equipment | |
| CN1655505A (en) | Bank cashier safety system and method | |
| CN113793158A (en) | Textile inspection detection information query method and system | |
| CN113486378A (en) | System for realizing data set construction processing based on federal learning and generation method thereof | |
| CN115600624B (en) | Entity seal storage management system and method based on RFID technology | |
| KR102428829B1 (en) | Remote meter reading system with security module applied | |
| CN109726576A (en) | A kind of computer encryption system using RFID identification system | |
| CN115952477A (en) | User data protection system of safety computer | |
| CN109101828A (en) | A kind of network is made house calls platform customer information stocking system | |
| He et al. | Smart grid nontechnical loss detection based on power gateway consortium blockchain | |
| CN109088731A (en) | A kind of Internet of Things cloud communication means and its device | |
| CN106204405A (en) | A kind of high security school bus information processing system | |
| CN202853732U (en) | Anti-cheating supervising detection system of hand-held electronic price computing scale | |
| CN202750117U (en) | SD cipher card based internet of things health medical service system | |
| CN112822185A (en) | User service system capable of establishing standardized and automatic service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |