CN115766159A - Private data processing method and device and electronic equipment - Google Patents
Private data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN115766159A CN115766159A CN202211388990.1A CN202211388990A CN115766159A CN 115766159 A CN115766159 A CN 115766159A CN 202211388990 A CN202211388990 A CN 202211388990A CN 115766159 A CN115766159 A CN 115766159A
- Authority
- CN
- China
- Prior art keywords
- segmentation model
- data
- private data
- processing
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application discloses a private data processing method, a device and electronic equipment, which relate to the technical field of data security, and the method comprises the following steps: responding to a processing signal of private data, and encrypting the private data based on the first segmentation model to obtain encrypted data; transmitting the encrypted data to a cloud device so that the cloud device decrypts the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data; and acquiring a processing result of the privacy data from the cloud equipment, and executing corresponding application operation based on the processing result of the privacy data. By the method, the problem that information is easy to leak in the privacy data processing process can be effectively solved, and the technical effect of effectively improving the information security in the privacy data processing process is achieved.
Description
Technical Field
The application relates to the field of data security, in particular to a private data processing method and device and electronic equipment.
Background
With the development and large-scale application of the internet of things technology, the problems of personal privacy and data protection are increasingly highlighted. Especially in smart homes, the application of internet of things devices generally requires processes such as user identity authentication and voice image recognition, which widely relates to security problems in processes of transmission and processing of private data such as user personal identity and personal biological characteristic information.
In the related technology, the transmission and processing process of the private data of the household Internet of things equipment mainly depends on the self-carried encryption function of a data transmission protocol between the Internet of things equipment and the cloud equipment and is ensured through the encryption algorithms of the Internet of things equipment end and the cloud equipment end, and the public encryption protocols and encryption algorithms are easy to intercept and crack in a targeted manner due to the openness of the public encryption protocols and encryption algorithms, so that the encryption protocols and algorithms are invalid, and finally the problem that the private data are easy to leak is caused.
Therefore, it is an urgent need to provide a method for effectively improving the security of transmission and processing of private data.
Disclosure of Invention
The method aims to solve the problems, namely the technical problem that the privacy data are easy to leak in the processing process. The application provides a privacy data processing method and device and electronic equipment.
According to an aspect of the application, a private data processing method based on a neural network model is provided, the neural network model includes a first segmentation model and a second segmentation model, the first segmentation model is arranged in an internet of things device, the second segmentation model is arranged in a cloud device, and the method is applied to the internet of things device and includes:
responding to a processing signal of private data, and encrypting the private data based on the first segmentation model to obtain encrypted data;
transmitting the encrypted data to a cloud device so that the cloud device decrypts the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data;
and acquiring a processing result of the private data from the cloud equipment, and executing corresponding application operation based on the processing result of the private data.
In one embodiment, the neural network model is obtained by training a lightweight deep neural network DNN structure based on a deep separable convolution method and then preprocessing the lightweight deep neural network DNN structure.
In an embodiment, the neural network model includes an input layer, M layers of convolutional layers, and an output layer, and the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capability of the internet of things device and/or the cloud device, where the first segmentation model includes the input layer and the first K (K is greater than or equal to 1 and less than or equal to M-1) layers of convolutional layers, and the second segmentation model includes a K +1 th layer of convolutional layer to the output layer.
In one embodiment, the encrypting the private data based on the first segmentation model to obtain encrypted data includes:
inputting the privacy data into an input layer of the first segmentation model, and extracting a feature matrix of the privacy data based on the first K convolutional layers in the first segmentation model;
and encrypting the characteristic matrix based on a preset encryption algorithm in the first segmentation model to obtain encrypted data.
In one embodiment, after extracting the feature matrix of the private data and before encrypting the feature matrix based on a preset encryption algorithm, the method further includes:
performing feature matrix compression extraction on the feature matrix based on the pooling layers corresponding to the front K layers of convolution layers;
the encrypting the feature matrix based on the preset encryption algorithm includes: and encrypting the feature matrix after compression and extraction based on a preset encryption algorithm.
According to another aspect of the embodiments of the present application, there is provided another private data processing method based on a neural network model, where the neural network model includes a first partition model and a second partition model, the first partition model is provided in an internet of things device, the second partition model is provided in a cloud device, and the method is applied to the cloud device, and includes:
receiving encrypted data transmitted by the Internet of things equipment, wherein the encrypted data is obtained by responding to a processing signal of private data by the Internet of things equipment and encrypting the private data based on the first segmentation model;
decrypting the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data;
and transmitting the processing result of the privacy data to the Internet of things equipment so that the Internet of things equipment executes corresponding application operation based on the processing result of the privacy data.
In one embodiment, the decrypting the encrypted data based on the second segmentation model and processing the private data according to the decryption result includes:
inputting the encrypted data into the second segmentation model, and decrypting the encrypted data based on a preset decryption algorithm in the second segmentation model to obtain a decryption result, wherein the decryption result is a feature matrix of the private data extracted from the first K layers of convolutional layers of the first segmentation model;
and training the decryption result from the convolution layer of the (K + 1) th layer to an output layer based on the second segmentation model to obtain a processing result of the private data.
According to another aspect of the application, a private data processing apparatus based on a neural network model is provided, the neural network model includes a first segmentation model and a second segmentation model, the first segmentation model is provided in an internet of things device, the second segmentation model is provided in a cloud device, the apparatus is applied to the internet of things device, and includes:
a model encryption module configured to encrypt the private data based on the first segmentation model in response to a processing signal of the private data to obtain encrypted data;
the transmission processing module is configured to transmit the encrypted data to a cloud device, so that the cloud device decrypts the encrypted data based on the second segmentation model, and processes the private data according to a decryption result to obtain a processing result of the private data;
the acquisition module is configured to acquire a processing result of the private data from the cloud device and execute corresponding application operation based on the processing result of the private data.
According to another aspect of the present application, there is provided another private data processing apparatus based on a neural network model, the neural network model includes a first partition model and a second partition model, the first partition model is provided in an internet of things device, the second partition model is provided in a cloud device, the apparatus is applied to the cloud device, and includes:
the receiving module is used for receiving encrypted data transmitted by the Internet of things equipment, wherein the encrypted data is obtained by encrypting the private data based on the first segmentation model by the Internet of things equipment in response to a processing signal of the private data;
the decryption processing module is arranged for decrypting the encrypted data based on the second segmentation model and processing the private data according to a decryption result to obtain a processing result of the private data;
the data transmission module is configured to transmit the processing result of the private data to the internet of things device, so that the internet of things device executes corresponding application operation based on the processing result of the private data.
According to yet another aspect of the application, an electronic device is provided, comprising a memory in which a computer program is stored and a processor arranged to execute the method of processing private data, or the further method of processing private data, by the computer program.
As can be appreciated, according to the private data processing method, the private data processing apparatus and the electronic device provided by the application, the private data is encrypted based on the first segmentation model in response to a processing signal of the private data, so as to obtain encrypted data; transmitting the encrypted data to a cloud device so that the cloud device decrypts the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data; and acquiring a processing result of the privacy data from the cloud equipment, and executing corresponding application operation based on the processing result of the privacy data. By the method, the problem that the privacy data are easy to leak in the processing process can be effectively solved, and the technical effect of effectively improving the safety in the privacy data processing process is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a diagram of a hardware environment according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a private data processing method according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a neural network model according to an embodiment of the present application;
FIG. 4 is a schematic illustration of pruning a DNN structure in an embodiment of the present application;
fig. 5 is a schematic flowchart of another private data processing method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a private data processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of another privacy data processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Before introducing the embodiments of the present application, the technical background of the embodiments of the present application is explained first: in the related technology, the transmission and processing process of private data, such as identity data, of the home Internet of Things device is mainly guaranteed by an Internet of Things device side and a cloud data encryption algorithm, and the transmission process often depends on the self-carried encryption functions of transmission protocols such as HTTP (Hyper Text Transfer Protocol), MQTT (Message queue Telemetry transmission), NB-IoT (Narrow Band Internet of Things), and the like. Due to the openness of the encryption protocol and the encryption algorithm, targeted interception and cracking can be easily caused, so that the encryption protocol and the encryption algorithm are invalid. In addition, because the data processing capacity of the internet of things equipment is limited, the data is often uploaded to the cloud to complete the implementation of complex intelligent application, in the process, the identity data of the user, such as personal image data and personal voice data, is usually stored in a simple symmetric encryption or source data format, and the cloud serves as a centralized data management and service response party.
In view of the above disadvantages and shortcomings, the application provides a private data processing method, device and electronic device, which are data encryption and processing methods based on a neural network model, and a lightweight and flexible network segmentation model thereof can be widely applied to application occasions of intelligent household appliance internet of things devices with different computing capabilities. The characteristic data extraction, compression and encryption are carried out on the related personal privacy data by utilizing the neural network segmentation model in the Internet of things equipment in the Internet of things data such as voice recognition and image processing processes, the characteristic data decryption and training are carried out by utilizing the neural network segmentation model in the cloud end equipment, the privacy data are processed, the Internet of things equipment end further completes corresponding application operation according to a processing result, the random hash and the irreversibility of data in the middle layer of the neural network model are utilized in the process, and the agnosticity of the network model structure of the divided neural network model is realized, so that the privacy data security closed loop of the whole data transmission and processing process is completed, and the security of the user privacy data in the transmission and processing processes is effectively improved.
According to an aspect of an embodiment of the present application, there is provided a private data processing method. The privacy data processing method can be widely applied to development scenes of full-House intelligent digital control application such as Smart Home, smart Home equipment ecology, smart Home (Intelligence House) ecology and the like.
Optionally, in this embodiment, the private data processing method may be applied to a hardware environment composed of an IOT (Internet of Things) home appliance device end and a cloud end as shown in fig. 1, and includes an Internet of Things device 110 and a cloud end device 120, where the Internet of Things device 110 and the cloud end device 120 are connected through a network. It is to be understood that the network may include, but is not limited to, at least one of: wired networks, wireless networks. The wired network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, local area networks, which may include, but are not limited to, at least one of the following: WIFI (Wireless Fidelity ), bluetooth.
Different from the related art, in the internet of things device 110 and the cloud device 120 in this embodiment, a first segmentation model and a second segmentation model of a neural network model are respectively deployed, as shown in fig. 1, the first segmentation model includes an input layer (not shown), a convolution layer (K layer), and a pooling layer (K layer), and the second segmentation model includes a convolution layer (K +1 layer), a pooling layer (K +1 layer), and an output layer, where after receiving user voice image data, an internet of things device (i.e., an IOT end-side device) converts the user voice image data into encoded data and inputs the encoded data into the first segmentation model, extracts feature (map) matrix data using the convolution layer, performs compression and extraction processing via the pooling layer, further encrypts the processed feature matrix data, and transmits the encoded data to the cloud device via an edge gateway route. The whole process completes the personal information safety closed loop of the whole process of data transmission, storage and application in the equipment end and the cloud end of the Internet of things, and effectively guarantees the ciphertext characteristics and data concealment of user sound and image information in the transmission, storage and application processes.
The scene schematic diagram of the present application is briefly described above, and details of the method, the apparatus and the electronic device for processing private data provided in the embodiment of the present application are described below by taking the internet of things device 110 and the cloud device 120 applied in fig. 1 as an example.
Referring to fig. 2, fig. 2 is a schematic flow chart of a private data processing method according to an embodiment of the present disclosure, and a private data processing method based on a neural network model is provided, where the neural network model includes a first segmentation model and a second segmentation model, the first segmentation model is disposed in an internet of things device, and the second segmentation model is disposed in a cloud device, and the method is applied to the internet of things device, and includes steps S201 to S203.
In one embodiment, the lightweight neural network model is adopted to be suitable for some marginal internet of things equipment with limited computing power, and the overall applicability of the scheme is improved. Specifically, the neural network model is obtained by training a lightweight deep neural network DNN structure based on a deep separable convolution method and then preprocessing the lightweight deep neural network DNN structure.
In this embodiment, a Depth-wise separable convolution (Depth-wise convolution) method is adopted, a convolution Kernel (Kernel) type and an activation function are designed, and a lightweight DNN (Deep Neural Networks) Neural network structure is trained, so that the Neural network structure can at least meet one or a combination of the following lightweight indexes: (1) a defined network width W and depth D, as shown in the neural network model structure in fig. 3; (2) operand bit widths such as neuron node weight values and activation functions; (3) calculating the required computing power and response time by the DNN network; (4) DNN identification accuracy and omission factor; (5) the DNN model generalization index and other hyper-parameter requirements and the like.
In this embodiment, the preprocessing method for the DNN structure of the lightweight deep neural network may include Pruning (Pruning), setting a threshold for the Weights (Weights) of the neural network neural node by Pruning the trained model, and Pruning off the connection of the neural node with a lighter weight. And searching the model again to find out redundant nodes without input and output, and pruning branches connected with the redundant nodes. And (4) carrying out secondary training on the trimmed model, and finely adjusting the weight value of each node on the premise of meeting the indexes. The model after pruning can greatly reduce the calculated amount and the occupancy rate of the memory unit, and save the calculation resources of the edge side, wherein the pruning steps and the effect are respectively pictures before pruning, in the middle process of pruning and after pruning from left to right as shown in fig. 4, wherein the middle process from before pruning to the middle process of pruning is the construction of a weight threshold, and the middle process from pruning to the middle process of pruning is the pruning of a redundant node. In some embodiments, in addition to pruning the model, other preprocessing methods may be included, such as sparse training.
The embodiment sets the neural network model as a lightweight DNN structure, can be applied to intelligent household appliances and IoT terminal equipment systems with limited computational power, can be universally applied to GPU and ARM systems or CPU, DSP, PFGA and MCU systems with smaller parallel computing power, and is characterized by comprising a structured memory and concurrent addition and multiplication computing units and a basic system structure for realizing neural network operation.
In one embodiment, the computing processing capacity of the internet of things equipment is considered when the neural network model is segmented, so that the processing efficiency of the private data is effectively improved. Specifically, the neural network model comprises an input layer, M layers of convolutional layers and an output layer, the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capacity of the internet of things device and/or the cloud device, the first segmentation model comprises the input layer and the front K (K is more than or equal to 1 and is less than or equal to M-1) layers of convolutional layers, and the second segmentation model comprises a K +1 th layer of convolutional layers and is connected to the output layer.
It can be understood that the more convolutional layers in the neural network, the more features can be trained, wherein the minimum is 2 layers, that is, only 1 layer of convolutional layer is set in the first segmentation model and the second segmentation model, respectively, in practical application, a person skilled in the art may combine the prior art to adaptively set the number M of convolutional layers, for example, set M according to the complexity of the voice image data, and may set the value range of M between 20 and 60.
Illustratively, the neural network model is segmented according to the computing processing capability of the internet of things end-side device, i.e. the parallel computing processing capability (shown in fig. 3), and the segmentation principle for the second neural network may consider one or a combination of the following parameters: (1) calculating the size of a quick Memory space (Memory font) of an edge side calculation unit occupied by the model of the time (2) required by the equipment side of the Internet of things; (3) data irreducibility indexes such as data hashing degrees and randomness entropy coefficients and the like. (4) The present embodiment is a neural network model that includes pooling layers, and also takes into account the effect of feature map data pooling (maximum pooling) and the downsampling compression ratio.
In some embodiments, during the segmentation process for the DNN network model, customized segmentation may be performed according to model structures of different kinds of neural networks. For example, the order and composition of convolution kernel convolution operation, pooling operation, activation function operation such as Sigmoid and ReLU, and operation such as output layer Softmax normalization function can be flexibly realized, and customized segmentation method and strategy can be performed according to the structures of AlexNet, leNet, googleNet, resNet and other ASR models such as RNN, LSTM, GAN and Bert network based on Transformer coding. The present embodiment is not particularly limited to the segmentation process of the neural network model.
In the embodiment, a middle layer data extraction mode is provided for a lightweight DNN network structure and a training method designed for limited computing power and memory space of intelligent household appliances and IoT equipment, wherein the model segmentation mode realizes customized segmentation and flexible deployment of a neural network model according to computing power resource ratio of an equipment end and a cloud end.
Step S201, responding to a processing signal of the private data, and encrypting the private data based on the first segmentation model to obtain encrypted data.
In this embodiment, the private data may be data information related to user identity information, such as face recognition, voice recognition, fingerprint recognition, and the like. In some embodiments, the private data may also be other data information, and those skilled in the art may set the private data accordingly in combination with the actual application.
Specifically, the internet of things equipment identifies corresponding private data when processing data, and responds to the processing information number of the corresponding private data, and executes encryption operation of the private data according to the first segmentation model. It will be appreciated that in response to indicating a condition or state on which an executed operation depends, one or more of the operations executed may be in real time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.
In one embodiment, encrypting the private data based on the first segmentation model in step S201 to obtain encrypted data may include the following steps:
inputting the private data into an input layer of the first segmentation model, and extracting a feature matrix of the private data based on a top K-layer convolutional layer in the first segmentation model;
and encrypting the characteristic matrix based on a preset encryption algorithm in the first segmentation model to obtain encrypted data.
It should be noted that, a person skilled in the art may set the preset encryption algorithm by combining the prior art and practical application, for example, in this embodiment, a symmetric encryption algorithm is used to encrypt the feature matrix, specifically, an encryption key generated by an encryption chip is used to encrypt the feature matrix extracted by the convolutional layer, where the encryption key pair includes an encryption key and a decryption key, and the encryption key is used to encrypt in the first segmentation model and decrypt in the second segmentation model.
Illustratively, the internet of things equipment converts private data into coded data and inputs the coded data into a first segmentation model, then extracts and compresses a characteristic matrix of the coded data by using a convolution layer and a pooling layer in the first segmentation model, and encrypts the characteristic matrix based on an encryption key generated by an encryption chip to obtain encrypted data.
In one embodiment, in order to reduce the amount of calculation, a pooling layer is also added to the neural network model. Specifically, after the feature matrix of the private data is extracted and before the feature matrix is encrypted based on a preset encryption algorithm, the method may further include the following steps:
performing feature matrix compression extraction on the feature matrix based on the pooling layers corresponding to the front K layers of convolutional layers;
the encrypting the feature matrix based on the preset encryption algorithm includes: and encrypting the feature matrix after compression and extraction based on a preset encryption algorithm.
In this embodiment, the pooling layer is used to perform further dimensionality reduction on the information extracted by the convolutional layer, reduce the amount of computation, and enhance the invariance of the feature map matrix, so as to increase the robustness in the aspects of image offset, rotation, and the like.
In this embodiment, the feature matrix data encrypted through the previous K segments of DNN network training is sent and stored in the cloud through the edge side gateway and the route, and waits for the cloud service and the call of the recognition application function interface. The user data which are stored in the cloud and wait to be called comprise feature map data (namely a feature map matrix and a feature matrix) which are encrypted twice by a previous K-segment DNN and an encryption chip and an encryption key generated by the encryption chip, the source data format and the features of the user data cannot be restored by the cloud or a user side (namely an Internet of things equipment side), the data have irrecoverability and irreversibility, and the possibility that the private data are cracked in the transmission process is effectively reduced.
Step S202, the encrypted data are transmitted to a cloud device, so that the cloud device decrypts the encrypted data based on the second segmentation model, and processes the private data according to a decryption result to obtain a processing result of the private data.
In one implementation, after data encryption based on the first segmentation model is completed by the internet of things device, an application request is initiated to the cloud device, and the encrypted data is carried in the application request to transmit the encrypted data. And after receiving the encrypted data, the cloud equipment decrypts and restores the encrypted data by using the second segmentation model, trains the characteristic matrix, and outputs a result, namely a processing result of the private data, wherein the processing of the private data can be face recognition and the like.
Specifically, when the IoT terminal device calls data through the cloud application interface, the cloud server extracts encrypted data currently transmitted or stored by the internet of things device, decrypts the encrypted data once and restores the encrypted data to obtain a feature map matrix, and then the feature matrix is input to a DNN module embedded in the cloud or the application terminal to complete processing of the remaining DNNs (K +1 layer to output layer), wherein the DNN module comprises a K +1 layer to output layer DNN network model. And finally, performing characterization output of the recognition result through a Softmax normalization model of an output layer, completing cloud-to-end data exchange through gateway routing, sending the recognition result to end-side equipment, and completing realization of user identity recognition or voice and image recognition functions, wherein the complete process can be combined with the process shown in figure 1.
Step S203, obtaining a processing result of the private data from the cloud device, and executing a corresponding application operation based on the processing result of the private data.
In this embodiment, the internet of things device obtains the processing result of the cloud for the private data through a gateway route (based on the internet of things TLS security transport layer protocol), wherein the processing result based on the private data executes the corresponding application operation, and after the face recognition passes, the intelligent door lock is automatically turned on and off, taking the intelligent door lock as an example.
In the embodiment, the neural network model is divided and respectively deployed at the cloud end and the IoT equipment end, cooperation and step-by-step calculation between the cloud end and the end are realized by extracting characteristic data of the neural network intermediate layer of the IoT equipment end, ciphertext characteristics and irreversibility of the characteristic data of the intermediate layer of the neural network are applied, and symmetric encryption operation of a bottom layer encryption chip is realized, so that collection and multi-layer encryption processing of user voice and image information are realized, data encryption and protection of the user data in the whole process of transmission, storage and application are realized, the technical problem of easy leakage in the privacy data processing process is effectively solved, and the security of privacy data processing is improved.
Referring to fig. 5, fig. 5 is a schematic flowchart of another private data processing method based on a neural network model according to an embodiment of the present disclosure, where the neural network model includes a first partition model and a second partition model, the first partition model is disposed in an internet of things device, the second partition model is disposed in a cloud device, and the method is applied to the cloud device, and the method includes steps S501-S503.
Step S501, receiving encrypted data transmitted by Internet of things equipment, wherein the encrypted data is obtained by responding to a processing signal of private data by the Internet of things equipment and encrypting the private data based on the first segmentation model;
step S502, decrypting the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data;
step S503, transmitting the processing result of the privacy data to the Internet of things equipment so that the Internet of things equipment executes corresponding application operation based on the processing result of the privacy data.
In one embodiment, the neural network model is obtained by training a lightweight deep neural network DNN structure based on a deep separable convolution method and then preprocessing the lightweight deep neural network DNN structure.
In an embodiment, the neural network model includes an input layer, M layers of convolutional layers, and an output layer, and the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capability of the internet of things device and/or the cloud device, where the first segmentation model includes the input layer and the first K (K is greater than or equal to 1 and less than or equal to M-1) layers of convolutional layers, and the second segmentation model includes a K +1 th layer of convolutional layer to the output layer.
In one embodiment, the decrypting the encrypted data based on the second segmentation model and processing the private data according to the decryption result includes:
inputting the encrypted data into the second segmentation model, and decrypting the encrypted data based on a preset decryption algorithm in the second segmentation model to obtain a decryption result, wherein the decryption result is a feature matrix of the private data extracted from the first K layers of convolutional layers of the first segmentation model;
and training the decryption result from the K +1 th convolution layer to an output layer based on the second segmentation model to obtain a processing result of the private data.
Correspondingly, the encrypted data may be encrypted data obtained after pooling operation is performed on the encrypted data by the first segmentation model, and a corresponding pooling layer may also be set in the second segmentation model to perform pooling operation on the feature matrix data, which is not described in this embodiment again.
In this embodiment, a data encryption implementation method for a lightweight DNN neural network designed for intelligent home appliances and internet of things terminal devices uses the feature that computing resources on the edge side of the internet of things device are limited to perform batch encryption uploading and storage on user pictures and voice data in a dedicated data encryption algorithm and distributed neural network computing manner, and flexibly allocates computing loads of a cloud end and an IoT device end on the basis of ensuring intelligent application and computing of the cloud end and the edge side, so that user personal data can complete various application calls on the premise of information security, such as identity authentication and implementation of image and voice recognition algorithms. The data security solution has a wide application prospect in the future of the intelligentization and miniaturization trend of the terminal equipment of the internet of things and the introduction of the AIoT technology.
It should be noted that, implementation principle portions that are not described in the method provided in the present embodiment have been described in detail in the foregoing embodiment, and the same technical effects can be achieved.
The embodiment of the application correspondingly still provides a privacy data processing apparatus based on neural network model, neural network model includes first segmentation model and second segmentation model, first segmentation model sets up in thing networking devices, the second segmentation model sets up in high in the clouds equipment, the device is applied to thing networking devices, as shown in fig. 6, the device includes:
a model encryption module 61 configured to encrypt the private data based on the first segmentation model in response to a processing signal of the private data, resulting in encrypted data;
a transmission processing module 62 configured to transmit the encrypted data to a cloud device, so that the cloud device decrypts the encrypted data based on the second segmentation model, and process the private data according to a decryption result to obtain a processing result of the private data;
an obtaining module 63, configured to obtain a processing result of the private data from the cloud device, and execute a corresponding application operation based on the processing result of the private data.
In one embodiment, the neural network model is obtained by training a lightweight deep neural network DNN structure based on a deep separable convolution method and then preprocessing the lightweight deep neural network DNN structure.
In an embodiment, the neural network model includes an input layer, M convolutional layers, and an output layer, and the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capability of the internet of things device and/or the cloud device, where the first segmentation model includes the input layer and the first K (K is greater than or equal to 1 and less than or equal to M-1) convolutional layers, and the second segmentation model includes a K +1 th convolutional layer and an output layer.
In one embodiment, the model cryptographic module comprises:
an input extraction unit configured to input the privacy data into an input layer of the first segmentation model and extract a feature matrix of the privacy data based on a top K-layer convolution layer in the first segmentation model;
and the encryption unit is used for encrypting the characteristic matrix based on a preset encryption algorithm in the first segmentation model to obtain encrypted data.
In one embodiment, the model cryptographic module further comprises:
a compression extraction unit configured to perform feature matrix compression extraction on the feature matrix based on the pooling layers corresponding to the front K convolutional layers;
the encryption unit is specifically configured to encrypt the feature matrix after compression and extraction based on a preset encryption algorithm.
It should be noted that, the apparatus provided in this embodiment can correspondingly implement all the method steps implemented by the internet of things device in the method embodiments, and can achieve the same technical effects, and detailed descriptions of the same parts and beneficial effects as those in the method embodiments are omitted here.
The embodiment of the application correspondingly still provides another kind of privacy data processing apparatus based on neural network model, neural network model includes first segmentation model and second segmentation model, first segmentation model sets up in thing networking device, the second segmentation model sets up in high in the clouds equipment, the device is applied to high in the clouds equipment, as shown in fig. 7, the device includes:
a receiving module 71, configured to receive encrypted data transmitted by an internet of things device, where the encrypted data is obtained by encrypting, by the internet of things device, privacy data based on the first segmentation model in response to a processing signal of the privacy data;
a decryption processing module 72 configured to decrypt the encrypted data based on the second segmentation model and process the private data according to a decryption result to obtain a processing result of the private data;
a data transmission module 73 configured to transmit the processing result of the private data to the internet of things device, so that the internet of things device executes a corresponding application operation based on the processing result of the private data.
In one embodiment, the decryption processing module includes:
the input decryption unit is used for inputting the encrypted data into the second segmentation model and decrypting the encrypted data based on a preset decryption algorithm in the second segmentation model to obtain a decryption result, and the decryption result is a feature matrix of the private data extracted by the first K layers of convolution layers of the first segmentation model;
and the processing unit is used for training the decryption result from the convolution layer of the (K + 1) th layer to an output layer based on the second segmentation model to obtain a processing result of the private data.
In one embodiment, the neural network model is obtained by training a lightweight deep neural network DNN structure based on a deep separable convolution method and then preprocessing the lightweight deep neural network DNN structure.
In an embodiment, the neural network model includes an input layer, M convolutional layers, and an output layer, and the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capability of the internet of things device and/or the cloud device, where the first segmentation model includes the input layer and the first K (K is greater than or equal to 1 and less than or equal to M-1) convolutional layers, and the second segmentation model includes a K +1 th convolutional layer and an output layer.
It should be noted that, the apparatus provided in this embodiment can correspondingly implement all the method steps implemented by the cloud device in the foregoing method embodiment, and can achieve the same technical effect, and detailed descriptions of the same parts and beneficial effects as those of the method embodiment are not repeated here.
Correspondingly, an electronic device according to an embodiment of the present application is further provided, as shown in fig. 8, and includes a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the private data processing method or the another private data processing method through the computer program.
It should be noted that, the electronic device provided in this embodiment can correspondingly implement all the method steps implemented by the internet of things device or the cloud device in the method embodiment, and can achieve the same technical effect, and details of the same parts and beneficial effects as those of the method embodiment are not described herein again.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer.
In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
In the description of the embodiments of the present application, the term "and/or" merely indicates an association relationship describing an associated object, and indicates that three relationships may exist, for example, a and/or B, and may indicate: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" means any combination of any one or more of a plurality, for example, including at least one of a, B, and may mean any one or more elements selected from the group consisting of a, B, and C communication. Further, the term "plurality" means two or more unless specifically stated otherwise.
The foregoing is only a preferred embodiment of the present application and it should be noted that, as will be apparent to those skilled in the art, numerous modifications and adaptations can be made without departing from the principles of the present application and such modifications and adaptations are intended to be considered within the scope of the present application.
Claims (10)
1. A privacy data processing method based on a neural network model is characterized in that the neural network model comprises a first segmentation model and a second segmentation model, the first segmentation model is arranged in Internet of things equipment, the second segmentation model is arranged in cloud end equipment, and the method is applied to the Internet of things equipment and comprises the following steps:
responding to a processing signal of private data, and encrypting the private data based on the first segmentation model to obtain encrypted data;
transmitting the encrypted data to a cloud device so that the cloud device decrypts the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data;
and acquiring a processing result of the privacy data from the cloud equipment, and executing corresponding application operation based on the processing result of the privacy data.
2. The method of claim 1, wherein the neural network model is obtained by training a lightweight Deep Neural Network (DNN) structure based on a deep separable convolution method and then preprocessing the DNN structure.
3. The method of claim 2, wherein the neural network model comprises an input layer, M convolutional layers, and an output layer, and the first segmentation model and the second segmentation model are obtained by segmenting the neural network model according to the computing processing capability of the internet of things device and/or the cloud device, wherein the first segmentation model comprises the input layer and the first K (1 ≦ K ≦ M-1) convolutional layers, and the second segmentation model comprises the K +1 th convolutional layer to the output layer.
4. A method according to any of claims 1-3, wherein said encrypting the private data based on the first segmentation model resulting in encrypted data comprises:
inputting the private data into an input layer of the first segmentation model, and extracting a feature matrix of the private data based on a top K-layer convolutional layer in the first segmentation model;
and encrypting the characteristic matrix based on a preset encryption algorithm in the first segmentation model to obtain encrypted data.
5. The method according to claim 4, wherein after extracting the feature matrix of the private data and before encrypting the feature matrix based on a preset encryption algorithm, the method further comprises:
performing feature matrix compression extraction on the feature matrix based on the pooling layers corresponding to the front K layers of convolutional layers;
the encrypting the feature matrix based on the preset encryption algorithm includes: and encrypting the feature matrix after compression and extraction based on a preset encryption algorithm.
6. A private data processing method based on a neural network model is characterized in that the neural network model comprises a first segmentation model and a second segmentation model, the first segmentation model is arranged in Internet of things equipment, the second segmentation model is arranged in cloud end equipment, and the method is applied to the cloud end equipment and comprises the following steps:
receiving encrypted data transmitted by the Internet of things equipment, wherein the encrypted data is obtained by responding to a processing signal of private data by the Internet of things equipment and encrypting the private data based on the first segmentation model;
decrypting the encrypted data based on the second segmentation model, and processing the private data according to a decryption result to obtain a processing result of the private data;
and transmitting the processing result of the privacy data to the Internet of things equipment so that the Internet of things equipment executes corresponding application operation based on the processing result of the privacy data.
7. The method according to claim 6, wherein the decrypting the encrypted data based on the second segmentation model and processing the private data according to the decryption result comprises:
inputting the encrypted data into the second segmentation model, and decrypting the encrypted data based on a preset decryption algorithm in the second segmentation model to obtain a decryption result, wherein the decryption result is a feature matrix of the private data extracted from the first K layers of convolutional layers of the first segmentation model;
and training the decryption result from the K +1 th convolution layer to an output layer based on the second segmentation model to obtain a processing result of the private data.
8. The utility model provides a privacy data processing apparatus based on neural network model, which characterized in that, neural network model includes first segmentation model and second segmentation model, first segmentation model sets up in thing networking equipment, the second segmentation model sets up in high in the clouds equipment, thing networking equipment is applied to the device, include:
a model encryption module configured to encrypt the private data based on the first segmentation model in response to a processing signal of the private data to obtain encrypted data;
the transmission processing module is configured to transmit the encrypted data to a cloud device so that the cloud device decrypts the encrypted data based on the second segmentation model, and process the private data according to a decryption result to obtain a processing result of the private data;
the acquisition module is configured to acquire a processing result of the private data from the cloud device and execute corresponding application operation based on the processing result of the private data.
9. The utility model provides a privacy data processing apparatus based on neural network model, its characterized in that, neural network model includes first segmentation model and second segmentation model, first segmentation model sets up in thing networking equipment, the second segmentation model sets up in high in the clouds equipment, the device is applied to high in the clouds equipment, includes:
the receiving module is used for receiving encrypted data transmitted by the Internet of things equipment, wherein the encrypted data is obtained by encrypting the private data based on the first segmentation model by the Internet of things equipment in response to a processing signal of the private data;
the decryption processing module is arranged for decrypting the encrypted data based on the second segmentation model and processing the private data according to a decryption result to obtain a processing result of the private data;
and the data transmission module is used for transmitting the processing result of the private data to the Internet of things equipment so as to enable the Internet of things equipment to execute corresponding application operation based on the processing result of the private data.
10. An electronic device, comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is configured to execute the private data processing method of any one of claims 1 to 5, or the private data processing method of any one of claims 6 or 7, by the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211388990.1A CN115766159A (en) | 2022-11-08 | 2022-11-08 | Private data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211388990.1A CN115766159A (en) | 2022-11-08 | 2022-11-08 | Private data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115766159A true CN115766159A (en) | 2023-03-07 |
Family
ID=85357343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211388990.1A Pending CN115766159A (en) | 2022-11-08 | 2022-11-08 | Private data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115766159A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116669018A (en) * | 2023-07-28 | 2023-08-29 | 陕西通信规划设计研究院有限公司 | Data processing method and device based on Internet of things communication |
| CN117155569A (en) * | 2023-10-30 | 2023-12-01 | 天清数安(天津)科技有限公司 | Privacy calculation method and system for fine-tuning pre-training model |
-
2022
- 2022-11-08 CN CN202211388990.1A patent/CN115766159A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116669018A (en) * | 2023-07-28 | 2023-08-29 | 陕西通信规划设计研究院有限公司 | Data processing method and device based on Internet of things communication |
| CN116669018B (en) * | 2023-07-28 | 2023-10-13 | 陕西通信规划设计研究院有限公司 | Data processing method and device based on Internet of things communication |
| CN117155569A (en) * | 2023-10-30 | 2023-12-01 | 天清数安(天津)科技有限公司 | Privacy calculation method and system for fine-tuning pre-training model |
| CN117155569B (en) * | 2023-10-30 | 2024-01-09 | 天清数安(天津)科技有限公司 | Privacy calculation method and system for fine-tuning pre-training model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mishra et al. | Delphi: A cryptographic inference system for neural networks | |
| CN110750801B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| Wang et al. | Privacy-preserving federated learning for internet of medical things under edge computing | |
| CN115766159A (en) | Private data processing method and device and electronic equipment | |
| CN111783130A (en) | Data processing method and device for privacy protection and server | |
| CN112231742B (en) | Model joint training method and device based on privacy protection | |
| CN109743727A (en) | Big data collection method is perceived based on the efficient secret protection that mist calculates | |
| Lyu et al. | FORESEEN: Towards differentially private deep inference for intelligent Internet of Things | |
| CN113542228A (en) | Data transmission method and device based on federal learning and readable storage medium | |
| CN111935168A (en) | Industrial information physical system-oriented intrusion detection model establishing method | |
| CN112597542A (en) | Target asset data aggregation method and device, storage medium and electronic device | |
| Moradi et al. | A real-time biometric encryption scheme based on fuzzy logic for IoT | |
| CN209881821U (en) | Intelligent identity recognition system based on 5G network | |
| Shankar et al. | Multiple share creation with optimal hash function for image security in wsn aid of ogwo | |
| CN112270415A (en) | Training data preparation method, device and equipment for encrypted machine learning | |
| Liu et al. | ESA-FedGNN: Efficient secure aggregation for federated graph neural networks | |
| CN115766160A (en) | Data processing method and device | |
| CN115983275A (en) | Named entity identification method, system and electronic equipment | |
| CN116796338A (en) | Online deep learning system and method for privacy protection | |
| CN111814165B (en) | Image privacy protection method based on deep neural network middle layer | |
| CN103327363A (en) | System and method for realizing control over video information encryption on basis of semantic granularity | |
| CN114463063A (en) | Data processing method and related device | |
| Yang et al. | A distributed federated transfer learning framework for edge optical network | |
| Yao et al. | A novel privacy-preserving neural network computing approach for E-Health information system | |
| CN116663064B (en) | Privacy protection neural network prediction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |