WO2021228148A1

WO2021228148A1 - Feature extraction method for protecting personal data privacy, model training method, and hardware

Info

Publication number: WO2021228148A1
Application number: PCT/CN2021/093367
Authority: WO
Inventors: 杨成平; 赵凯
Original assignee: 支付宝(杭州)信息技术有限公司
Priority date: 2020-05-14
Filing date: 2021-05-12
Publication date: 2021-11-18
Also published as: CN111553320A; CN114419712A; CN111553320B

Abstract

Provided in the embodiments of the present description are a feature extraction method for protecting personal data privacy, a model training method, and hardware. The feature extraction method comprises: obtaining a multi-frame image sequence in which a sample object is presented; using nonlinear conversion as an encryption method, performing feature representation on the obtained multi-frame image sequence, and obtaining initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, feature data of the sample object presented in the multi-frame image sequence being personal data of the sample object; and performing integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object, wherein the target encrypted feature data can be used as model training data.

Description

Feature extraction method, model training method and hardware for protecting personal data privacy

Technical field

This document relates to the field of data processing technology, in particular to a feature extraction method, model training method and hardware for protecting personal data privacy.

Background technique

Deep learning models have been used more and more widely by virtue of their ability to process information mechanically. Face recognition is a common business form in the field of deep learning. The principle of face recognition is based on a deep learning model to approximate the facial features of the user to be identified with the facial features of the sample, so as to determine the identity of the user to be identified. Obviously, there is a risk of leakage of sample facial features that belong to personal data, and it cannot effectively protect privacy.

In view of this, there is an urgent need for a technical solution that can protect the privacy of personal data in the field of deep learning.

Summary of the invention

The purpose of the embodiments of this specification is to provide a feature extraction method, model training method, and hardware for protecting personal data privacy, which can protect personal data privacy in the field of deep learning.

In order to achieve the above objectives, the embodiments of this specification are implemented as follows: In the first aspect, a feature extraction method for protecting personal data privacy is provided, including: acquiring a multi-frame image sequence presenting sample objects; using nonlinear transformation as an encryption method , Performing feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the sample object的 personal data; integrated learning of the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, to obtain the target encrypted feature data corresponding to the sample object.

In a second aspect, a model training method for protecting personal data privacy is provided, which includes: obtaining a multi-frame image sequence presenting sample objects; using nonlinear conversion as an encryption method to perform feature representation on the obtained multi-frame image sequence, Obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the personal data of the sample object; the sample object corresponds to the multi-frame image sequence Perform integrated learning on the initial encrypted feature data of the sample object to obtain the target encrypted feature data corresponding to the sample object; train a preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.

In a third aspect, a feature extraction device for protecting privacy data includes: an image sequence acquisition module that acquires a multi-frame image sequence presenting sample objects; a feature encryption representation module that uses nonlinear conversion as an encryption method to perform The feature representation of the multi-frame image sequence is performed to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the personal data of the sample object; feature integration The learning module performs integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted feature data corresponding to the sample object.

In a fourth aspect, there is provided an electronic device including: a memory, a processor, and a computer program stored on the memory and capable of being run on the processor, the computer program being executed by the processor: obtaining a sample present A multi-frame image sequence of the object; using nonlinear conversion as an encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the multi-frame image sequence The characteristic data of the sample object presented by the image sequence belongs to the personal data of the sample object; the integrated learning is performed on the initial encrypted characteristic data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted characteristic data corresponding to the sample object.

In a fifth aspect, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the following steps are implemented: Obtain a multi-frame image sequence presenting a sample object Use nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, wherein the sample object presented by the multi-frame image sequence The characteristic data of belongs to the personal data of the sample object; the integrated learning is performed on the initial encrypted characteristic data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted characteristic data corresponding to the sample object.

In a sixth aspect, a model training device for protecting personal data privacy is provided, including: an image sequence acquisition module, which acquires a multi-frame image sequence presenting sample objects; a feature encryption representation module, which uses nonlinear conversion as an encryption method to obtain Performing feature representation on the multi-frame image sequence to obtain initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the personal data of the sample object; The feature integrated learning module performs integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object to obtain the target encrypted feature data corresponding to the sample object; the model training module is based on the target encrypted feature corresponding to the sample object The model classification labels of the data and sample objects, and the preset learning model is trained.

In a seventh aspect, there is provided an electronic device including: a memory, a processor, and a computer program stored in the memory and capable of being run on the processor, the computer program being executed by the processor: obtaining a sample A multi-frame image sequence of the object; using nonlinear conversion as an encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the multi-frame image sequence The characteristic data of the sample object presented by the image sequence belongs to the personal data of the sample object; the integrated learning is performed on the initial encrypted characteristic data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted characteristic data corresponding to the sample object; The target encrypted feature data corresponding to the sample object and the model classification label of the sample object are trained on a preset learning model.

In an eighth aspect, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the following steps are implemented: Obtain a multi-frame image sequence presenting a sample object Use nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, wherein the sample object presented by the multi-frame image sequence The feature data of the sample object belongs to the personal data of the sample object; the integrated learning is performed on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted feature data corresponding to the sample object; based on the corresponding sample object The target encrypted feature data and the model classification label of the sample object, and the preset learning model is trained.

The solution of the embodiment of this specification adopts a non-linear conversion encryption method to perform feature encryption extraction on a multi-frame image sequence presenting a sample object to obtain the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object. The frame image sequence is integrated with the initial encrypted feature data to obtain high-level target encrypted feature data. Since the entire scheme relies on encrypted image feature data, if the encrypted image feature data is sampled, if leakage occurs, the personal data of the sample object will not be exposed, thus achieving privacy protection. At the same time, the target encrypted feature data is obtained by integrating the initial encrypted feature data of the multi-frame image sequence, which can effectively correct the loss caused by the image feature encryption, and can obtain better model performance when it is subsequently used for model training.

Description of the drawings

In order to more clearly explain the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some of the embodiments described in the embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Fig. 1 is a schematic flowchart of a feature extraction method provided by an embodiment of the specification.

Fig. 2 is a schematic flow chart of a model training method provided in an embodiment of this specification.

Fig. 3 is a schematic structural diagram of a feature extraction device provided by an embodiment of the specification.

Fig. 4 is a schematic structural diagram of a model training device provided by an embodiment of this specification.

Fig. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the specification.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of this specification.

As mentioned above, the principle of face recognition is based on the deep learning model to approximate the facial features of the user to be identified and the facial features of the sample to determine the identity of the user to be identified. Among them, the training of the deep learning model relies on sample face images. These sample facial images belong to the user's personal data, and keeping the bottom will risk privacy leakage. To this end, this document aims to propose a technical solution that can protect personal data privacy in the field of deep learning.

Fig. 1 is a flowchart of a feature extraction method for protecting personal data privacy according to an embodiment of this specification. The method shown in FIG. 1 may be executed by the corresponding device below, and includes: step S102, acquiring a multi-frame image sequence presenting a sample object.

Specifically, in this step, a multi-frame image sequence can be intercepted from the video presenting the sample object. For example, video shooting of a sample object is performed through the camera of the terminal device, and a multi-frame image sequence showing the sample object is cut out according to a preset frame rate.

Step S104, using nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence, where the feature data of the sample object presented in the multi-frame image sequence belongs to Personal data of sample subjects.

Based on the nonlinear conversion, the ratio of the change of the output value (initial encrypted feature data) to the change of the corresponding input value (multi-frame image sequence) is not a constant conversion, which has an encryption effect.

In practical applications, the way to achieve nonlinear conversion is not unique, and this specification does not specifically limit it. As an example introduction: In this step, a local sensitive hash algorithm may be used to perform hash conversion on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence.

Alternatively, a convolutional neural network model can also be used to perform feature encryption extraction on a multi-frame image sequence. Among them, the convolutional neural network model may include: a convolutional layer, which performs convolution processing on the obtained multi-frame image sequence to obtain an output feature set of the convolutional layer; a pooling layer, based on a maximum pooling algorithm and/or an average pooling The algorithm is to perform pooling processing on the output feature set of the convolutional layer to obtain the output feature set of the pooling layer; the fully connected layer converts the output feature set of the pooling layer into initial encrypted feature data of a specified dimension. Obviously, by inputting the obtained multi-frame image sequence into the convolutional neural network model, the initial encrypted feature data corresponding to the sample object output by the convolutional neural network model can be obtained.

Step S106: Perform integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object to obtain target encrypted feature data corresponding to the sample object.

Among them, ensemble learning is an existing machine learning method. It itself completes the learning task by constructing and combining multiple individual learners. Individual learners are usually generated from training data by an existing learning algorithm, such as C4.5 decision algorithm, BP neural network algorithm, etc. At this time, the ensemble contains only the same type of individual learners, such as "decision tree integration" The whole is a decision tree, and the "neural network ensemble" is all a neural network. Such an ensemble is "homogeneous." Individual learners in homogeneous integration are also called "base learners". The corresponding learning algorithm is called "base learning algorithm". An ensemble can also include different types of individual learners, for example, a decision tree and a neural network are included at the same time. Such an ensemble is called "heterogeneous". The individual learners in heterogeneous ensemble are generated by different learning algorithms. At this time, there is no base learning algorithm, which is often called "component learner" or directly called individual learner. In the embodiments of this specification, individual learners can be assembled through conventional collective learning strategies (for example, averaging method, voting method, and learning method) to realize the selection and integration of initial encrypted feature data, so as to obtain higher-level target encrypted feature data.

It should be understood that the initial encrypted feature data and/or target encrypted feature data obtained in this step can be used as model training data. Therefore, it is only necessary to reserve samples of the initial encrypted feature data and/or the target encrypted feature data. The previously obtained multi-frame image sequence can be deleted after the feature representation is completed, that is, "burn after use".

The feature method in the embodiment of this specification adopts a non-linear conversion encryption method to perform feature encryption extraction on a multi-frame image sequence presenting a sample object to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object. The multi-frame image sequence is integrated with the initial encrypted feature data to obtain high-level target encrypted feature data. Since the entire scheme relies on encrypted image feature data, if the encrypted image feature data is sampled, if leakage occurs, the personal data of the sample object will not be exposed, thus achieving privacy protection. At the same time, the target encrypted feature data is obtained by integrating the initial encrypted feature data of the multi-frame image sequence, which can effectively correct the loss caused by the image feature encryption, and can obtain better model performance when it is subsequently used for model training.

In the following, the method of the embodiment of this specification will be introduced as an example in combination with actual application scenarios.

In this application scenario, it is used to obtain the feature data required for face recognition. The process includes the following steps:

Step 1: Take a video shot of the face of the sample object, and intercept a multi-frame face image sequence showing the face of the sample object from the video.

Step 2: Using nonlinear conversion as the encryption method, perform feature expression on the obtained multi-frame face image sequence to obtain the initial encrypted feature data.

Step 3: Delete the multi-frame face image sequence of the sample object intercepted from the video.

Step 4: Store the initial encrypted feature data corresponding to the sample object in a feature database in association.

Step 5: Perform integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object to obtain the target encrypted feature data corresponding to the sample object.

Step 6: Associatively store the target encrypted feature data corresponding to the sample object to the feature database.

Based on the above process, in the subsequent process of training the user identification model, the initial encrypted feature data and/or target encrypted feature data corresponding to the object can be sampled from the feature library, and the user identification model can be trained.

Correspondingly, the embodiment of this specification also provides a model training method for protecting personal data privacy. Fig. 2 is a flowchart of a model training method according to an embodiment of this specification. The method shown in Figure 2 can be executed by the following corresponding devices, including:

Step S202: Obtain a multi-frame image sequence presenting sample objects.

Step S204, using nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence, where the feature data of the sample object presented in the multi-frame image sequence belongs to Personal data of the sample subject.

Step S206: Perform integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object to obtain target encrypted feature data corresponding to the sample object.

Step S208, training a preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.

In the specific training process, the target encrypted feature data corresponding to the sample object is used as the input data of the preset learning model, and the model classification label corresponding to the sample user is preset as the output data of the learning model. After the target encrypted feature data is input to the preset learning model, the training result given by the preset learning model can be obtained. This training result is the predicted classification result of the preset learning model for the sample user, and may be different from the true value classification result indicated by the model classification label of the sample user. The embodiment of this specification can calculate the error value between the predicted classification result and the true value classification result based on the loss function derived by the maximum likelihood estimation, and for the purpose of reducing the error value, the parameters in the preset learning model can be calculated Adjust (for example, the weight value of the underlying vector) to achieve the training effect.

The model training method in the embodiment of this specification adopts a non-linear conversion encryption method to perform feature encryption extraction on a multi-frame image sequence presenting a sample object to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, and then adopt an integrated learning method Integrate the initial encrypted feature data of the multi-frame image sequence to obtain high-level target encrypted feature data. Since the entire scheme relies on encrypted image feature data, if the encrypted image feature data is sampled, if leakage occurs, the personal data of the sample object will not be exposed, thus achieving privacy protection. At the same time, the target encrypted feature data is obtained by integrating the initial encrypted feature data of the multi-frame image sequence, which can effectively correct the loss caused by the image feature encryption, and after training the preset learning model, better model performance can be obtained.

It should be understood that the pre-trained learning model can be used for prediction and identification, so as to provide data support for related business decisions.

For example, the preset learning model in the embodiment of this specification can be applied to a face payment service. In the process of user identity verification of face payment, a multi-frame image sequence showing the payment object to be verified can be collected; then, the same non-linear conversion as described above is used as an encryption method to characterize the multi-frame image sequence of the payment object Indicates that the initial encrypted feature data of the payment object is obtained. In the same way, perform integrated learning on the initial encrypted feature data of the payment object, obtain the target encrypted feature data of the payment object, and input the target encrypted feature data of the payment object into the preset learning model, and the preset learning model determines whether the payment object is Authorized users (target users) for payment. Finally, the recognition result of the preset learning model is used to determine whether to initiate facial payment.

Corresponding to the above-mentioned feature extraction method, the embodiment of this specification also provides a feature extraction device for protecting private data. 3 is a schematic diagram of the structure of the feature extraction device 300 of the embodiment of this specification, including: an image sequence acquisition module 310, which acquires a multi-frame image sequence presenting sample objects; a feature encryption representation module 320, which uses a nonlinear transformation as an encryption method to The obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the individual of the sample object Data; a feature integrated learning module 330, which performs integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, to obtain the target encrypted feature data corresponding to the sample object.

The feature extraction device of the embodiment of this specification adopts a non-linear conversion encryption method to perform feature encryption extraction on a multi-frame image sequence presenting a sample object to obtain the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, and then through integrated learning The method integrates the initial encrypted feature data of the multi-frame image sequence to obtain high-level target encrypted feature data. Since the entire scheme relies on encrypted image feature data, if the encrypted image feature data is sampled, if leakage occurs, the personal data of the sample object will not be exposed, thus achieving privacy protection. At the same time, the target encrypted feature data is obtained by integrating the initial encrypted feature data of the multi-frame image sequence, which can effectively correct the loss caused by the image feature encryption, and can obtain better model performance when it is subsequently used for model training.

Optionally, the feature encryption representation module 320 specifically inputs the obtained multi-frame image sequence into a preset convolutional neural network model to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence. Here, the convolutional neural network model includes: a convolutional layer, which performs convolution processing on the obtained multi-frame image sequence to obtain an output feature set of the convolutional layer; a pooling layer, which is based on a maximum pooling algorithm and/or an average value The pooling algorithm performs pooling processing on the output feature set of the convolutional layer to obtain the output feature set of the pooling layer; the fully connected layer converts the output feature set of the pooling layer into initial encrypted feature data of a specified dimension.

Optionally, the feature encryption representation module 320 may also perform a hash conversion on the obtained multi-frame image sequence based on a locally sensitive hash algorithm to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object.

Optionally, the feature extraction device 300 of the embodiment of the present specification may further include: a storage module for associative storage of the sample object, the corresponding initial encrypted feature data and/or the target encrypted feature data.

Optionally, the feature extraction device 300 of the embodiment of the present specification may further include: a deletion module, which deletes the obtained multi-frame image sequence after obtaining the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object.

Corresponding to the above-mentioned feature extraction method, the embodiment of this specification also provides a model training device for protecting the privacy of personal data. 4 is a schematic diagram of the structure of the model training device 400 according to the embodiment of this specification, including: an image sequence acquisition module 410, which acquires a multi-frame image sequence presenting sample objects; and a feature encryption representation module 420, which uses a nonlinear transformation as an encryption method to The obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the feature data of the sample object presented in the multi-frame image sequence belongs to the individual of the sample object Data; feature integration learning module 430, which performs integrated learning on the initial encrypted feature data corresponding to the multi-frame image sequence of the sample object, to obtain the target encrypted feature data corresponding to the sample object; model training module 440, based on the corresponding sample object The target encryption feature data and the model classification label of the sample object are used to train the preset learning model.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present specification. Please refer to FIG. 5, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. Among them, the memory may include memory, such as high-speed random access memory (Random-Access Memory, RAM), or may also include non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required by other services.

The processor, network interface, and memory can be connected to each other through an internal bus. The internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnection standard) bus or an EISA (Extended) bus. Industry Standard Architecture, extended industry standard structure) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one bidirectional arrow is used to indicate in FIG. 5, but it does not mean that there is only one bus or one type of bus.

Memory, used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include memory and non-volatile memory, and provide instructions and data to the processor.

Optionally, the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to form the above-mentioned feature extraction device on a logical level. The processor executes a program stored in the memory, and is specifically configured to perform the following operations: acquiring a multi-frame image sequence presenting a sample object.

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object.

Perform integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object.

Or, the processor reads the corresponding computer program from the non-volatile memory to the memory and then runs it to form the aforementioned model training device on a logical level. The processor executes a program stored in the memory, and is specifically configured to perform the following operations: acquiring a multi-frame image sequence presenting a sample object.

Training the preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.

The above-mentioned feature extraction method disclosed in the embodiment shown in FIG. 1 of this specification or the model training method disclosed in the embodiment shown in FIG. 2 may be applied to a processor or implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of this specification can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

It should be understood that the electronic device of the embodiment of this specification can realize the function of the above-mentioned feature extraction apparatus in the embodiment shown in FIG. 1 or realize the function of the above-mentioned model training apparatus in the embodiment shown in FIG. 2. Since the principle is the same, this article will not go into details.

Of course, in addition to the software implementation, the electronic equipment in this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc. That is to say, the execution body of the following processing flow is not limited to each logic unit. It can also be a hardware or logic device.

In addition, the embodiment of this specification also proposes a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions. Wherein, the above instructions, when executed by a portable electronic device that includes multiple application programs, can cause the portable electronic device to execute the method of the embodiment shown in FIG. Image sequence.

Alternatively, the above instructions, when executed by a portable electronic device that includes multiple application programs, can cause the portable electronic device to execute the method of the embodiment shown in FIG. Image sequence.

Perform integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted feature data corresponding to the sample object.

Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system or a computer program product. Therefore, this specification may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The above are only examples of this specification, and are not intended to limit this specification. For those skilled in the art, this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims of this specification. In addition, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this document.

Claims

A feature extraction method for protecting personal data privacy, including:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Perform integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object.
The method according to claim 1,

Using nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, including:

The obtained multi-frame image sequence is input to a preset convolutional neural network model to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence; wherein, the convolutional neural network model includes:

A convolutional layer, performing convolution processing on the obtained multi-frame image sequence to obtain an output feature set of the convolutional layer;

The pooling layer performs pooling processing on the output feature set of the convolutional layer based on the maximum pooling algorithm and/or the average pooling algorithm to obtain the output feature set of the pooling layer;

The fully connected layer converts the output feature set of the pooling layer into the initial encrypted feature data of the specified dimension.
According to the method described in claim 1,

Using nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, including:

Based on a locally sensitive hash algorithm, hash conversion is performed on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence.
The method according to any one of claims 1-3, further comprising:

The sample object, the corresponding initial encrypted feature data and/or the target encrypted feature data are stored in association.
The method according to any one of claims 1-3, further comprising:

After the initial encrypted feature data of the sample object corresponding to the multi-frame image sequence is obtained, the obtained multi-frame image sequence is deleted.
A model training method for protecting personal data privacy, including:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Performing integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object;

Training the preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.
The method according to claim 6, further comprising:

After the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, the obtained multi-frame image sequence is deleted.
A feature extraction device for protecting private data, including:

The image sequence acquisition module acquires a multi-frame image sequence presenting sample objects;

The feature encryption representation module uses nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the multi-frame image sequence The presented characteristic data of the sample object belongs to the personal data of the sample object;

The feature integrated learning module performs integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain the target encrypted feature data corresponding to the sample object.
An electronic device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Perform integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object.
A computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, the following steps are implemented:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Perform integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object.
A model training device for protecting personal data privacy, including:

The image sequence acquisition module acquires a multi-frame image sequence presenting sample objects;

The feature encryption representation module uses nonlinear conversion as the encryption method to perform feature representation on the obtained multi-frame image sequence to obtain the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, wherein the multi-frame image sequence The presented characteristic data of the sample object belongs to the personal data of the sample object;

A feature integrated learning module, which performs integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object, to obtain the target encrypted feature data corresponding to the sample object;

The model training module trains the preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.
An electronic device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Performing integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object;

Training the preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.
An electronic device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor:

Obtain a multi-frame image sequence presenting sample objects;

Using nonlinear conversion as the encryption method, the obtained multi-frame image sequence is characterized, and the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object is obtained, wherein the sample object presented by the multi-frame image sequence The characteristic data belongs to the personal data of the sample object;

Performing integrated learning on the initial encrypted feature data of the multi-frame image sequence corresponding to the sample object to obtain target encrypted feature data corresponding to the sample object;

Training the preset learning model based on the target encrypted feature data corresponding to the sample object and the model classification label of the sample object.