CN108038543B - Expectation and anti-expectation deep learning method and neural network system - Google Patents
Expectation and anti-expectation deep learning method and neural network system Download PDFInfo
- Publication number
- CN108038543B CN108038543B CN201711001729.0A CN201711001729A CN108038543B CN 108038543 B CN108038543 B CN 108038543B CN 201711001729 A CN201711001729 A CN 201711001729A CN 108038543 B CN108038543 B CN 108038543B
- Authority
- CN
- China
- Prior art keywords
- expected
- label
- deep learning
- expectation
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to an expectation-anti-expectation deep learning method and a neural network system, wherein 2 deep learning neural networks are constructed through an expectation label and an anti-expectation label of output data, and the deep learning neural network corresponding to the expectation label and the deep learning neural network corresponding to the anti-expectation label are respectively trained, so that the problem that the deep learning neural network is in structure fluctuation when output data with opposite attributes exist is avoided, and the training reliability of the deep learning neural network is improved.
Description
Technical Field
The invention relates to the technical field of deep learning neural networks, in particular to an expectation and anti-expectation deep learning method and a neural network system.
Background
The deep learning neural network is trained through input data and output data. If some input data can generate output data with opposite attributes under different scenes (which is possible, if other condition data influencing the output result is not included in the input data, different output data can be generated due to the change of other condition data), the structure of the deep learning neural network is fluctuated in the training process, the training of the deep learning neural network is not facilitated, and the use of the deep learning neural network is also not facilitated, because the credibility of the output data cannot be known.
For example, the deep learning neural network determines whether a person is a male, and inputs facial features of different persons, and the person with the same facial features may be the male or the female, which may cause turbulence of the structure of the deep learning neural network during training.
Disclosure of Invention
Based on this, it is necessary to provide a expectation-inverse deep learning method and a neural network system for the problem of generating a structural oscillation of a deep learning neural network when there is output data with opposite attributes.
A expectation-expectation and anti-expectation deep learning method comprises the following steps: acquiring an expected label and an anti-expected label of output data in training data; wherein the anti-desired tag is a tag having an attribute opposite to that of the desired tag; initializing a deep learning neural network corresponding to the expected label to obtain an expected deep learning neural network, and initializing a deep learning neural network corresponding to an anti-expected label to obtain an anti-expected deep learning neural network; and respectively training the expected deep learning neural network and the anti-expected deep learning neural network.
The deep learning method comprises the following steps of respectively training the expected deep learning neural network and the anti-expected deep learning neural network: carrying out unsupervised training on the expected deep learning neural network and the inverse expected deep learning neural network respectively through input data in the training data; acquiring input data corresponding to an output label consistent with an expected label from training data, taking the input data as input and 1 as expected output, performing supervised training on the expected deep learning neural network, acquiring input data corresponding to an output label inconsistent with the expected label and an anti-expected label from the training data, taking the input data as input and 0 as expected output, and performing supervised training on the expected deep learning neural network; the method includes the steps of obtaining input data corresponding to an output label which is consistent with an anti-expectation label from training data, using the input data as input, using 1 as expected output, performing supervised training on an anti-expectation deep learning neural network, obtaining input data corresponding to an output label which is inconsistent with both the expectation label and the anti-expectation label from the training data, using the input data as input, using 0 as expected output, and performing supervised training on the anti-expectation deep learning neural network.
The deep learning method comprises the following steps of acquiring input data corresponding to an output label consistent with an expected label from training data, taking the input data as input and 1 as expected output, and performing supervised training on the expected deep learning neural network, wherein the steps comprise: acquiring first output data consistent with an expected label and first input data corresponding to the first output data from training data; screening data of a first preset proportion from each first input data to obtain each corresponding second input data; and taking each second input datum as an input, taking the first preset proportion as an expected output, and carrying out supervised training on the expected deep learning neural network.
The deep learning method comprises the following steps of acquiring input data corresponding to an output label consistent with an anti-expectation label from training data, taking the input data as input and 1 as expected output, and performing supervised training on the anti-expectation deep learning neural network, wherein the steps comprise: taking input data corresponding to an output label consistent with an anti-expectation label in the training data as third input data; screening out data of a second preset proportion from each third input data to obtain each corresponding fourth input data; and taking each fourth input data as input, taking the corresponding second preset proportion as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
After the deep learning method trains the expected deep learning neural network and the inverse expected deep learning neural network respectively, the deep learning method further comprises the following steps: and calculating the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label.
The deep learning method comprises the following steps of calculating the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label: inputting the input data into the expected deep learning neural network to obtain the output data of the expected deep learning neural network; inputting the input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network; and obtaining the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label according to the output data of the expected deep learning neural network and the output data of the anti-expected deep learning neural network.
The deep learning method comprises the following steps of obtaining the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label according to the output data of the expected deep learning neural network and the output data of the anti-expected deep learning neural network: taking output data of the expected deep learning neural network as a probability A that the output belongs to an expected label, and taking output data of the inverse expected deep learning neural network as a probability B that the output belongs to an inverse expected label; recording the credibility F of the output corresponding to the input data, which belongs to the expected label or the anti-expected label, as (A + B)/2; the output corresponding to the input data has a reliability of F × a for the expected tag, a reliability of F × B for the opposite tag, and a reliability of 1-F for the other tags.
A neural network system, comprising: the label acquisition module is used for acquiring an expected label and an anti-expected label of output data in the training data; wherein the anti-desired tag is a tag having an attribute opposite to that of the desired tag; the initialization module is used for initializing the deep learning neural network corresponding to the expected tag to obtain an expected deep learning neural network, and initializing the deep learning neural network corresponding to the anti-expected tag to obtain an anti-expected deep learning neural network; and the training module is used for respectively training the expected deep learning neural network and the anti-expected deep learning neural network.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, implements the expectation and anti-expectation deep learning methods.
A computer apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the expectation and anti-expectation deep learning methods when executing the program.
According to the expectation-anti-expectation deep learning method and the neural network system, 2 deep learning neural networks are constructed through the expectation label and the anti-expectation label of the output data, the deep learning neural network corresponding to the expectation label and the deep learning neural network corresponding to the anti-expectation label are trained respectively, the problem that the deep learning neural network is in structure swing when output data with opposite attributes exist is avoided, and the reliability of deep learning neural network training is improved.
Drawings
FIG. 1 is a flow diagram of a method for expectation and anti-expectation deep learning according to one embodiment;
fig. 2 is a schematic structural diagram of a neural network system according to an embodiment.
Detailed Description
The technical solution of the present invention will be explained below with reference to the accompanying drawings.
As shown in FIG. 1, the present invention provides a expectation-expectation and anti-expectation deep learning method, which may include the following steps:
s1, acquiring an expected label and an anti-expected label of output data in the training data; wherein the anti-desired tag is a tag having an attribute opposite to that of the desired tag;
for example, the expectation label "man" and the anti-expectation label "woman" of the acquired output data.
S2, initializing the deep learning neural network corresponding to the expected label to obtain an expected deep learning neural network, and initializing the deep learning neural network corresponding to the anti-expected label to obtain an anti-expected deep learning neural network;
initializing an input format of a deep learning neural network corresponding to the expected label into an input data format in the training data; the input format of the deep learning neural network corresponding to the anti-expectation label is also initialized to the input data format in the training data. The output format of the desired deep learning neural network is initialized to a number between 0 and 1, and if the output is 1, the output is a desired label, if the output is 0, the output is not a desired label, and if the output is a number between 0 and 1, the output may be a desired label. The output format of the anti-expectation deep learning neural network is initialized to a number between 0 and 1, and when the output is 1, the output is an anti-expectation label, when the output is 0, the output is not an anti-expectation label, and when the output is a number between 0 and 1, the output is possibly an anti-expectation label. Meanwhile, the configuration information of the existing similar deep learning neural network (the preset configuration information comprises the preset number of layers, the preset number of nodes on each layer and the preset weight value connected with each network) is obtained and used as the configuration information of the deep learning neural network corresponding to the expected label to configure the deep learning neural network corresponding to the expected label, and meanwhile, the configuration information is also used as the configuration information of the deep learning neural network corresponding to the anti-expected label to configure the deep learning neural network corresponding to the anti-expected label.
For example, 2 deep learning neural networks corresponding to the expectation label "man" and the anti-expectation label "woman" are initialized, which are called the expectation label "man" deep learning neural network and the anti-expectation label "woman" deep learning neural network.
And S3, respectively training the expected deep learning neural network and the anti-expected deep learning neural network.
In one embodiment, this step may be implemented as follows:
s3.1, performing unsupervised training on the expected deep learning neural network and the inverse expected deep learning neural network respectively through input data in the training data;
for example, the expectation label 'man' deep learning neural network and the anti-expectation label 'woman' deep learning neural network are respectively subjected to unsupervised training through the input data face images in the training data. It is worth noting that the same input data set can be used for unsupervised training of the expectation deep learning neural network and the inverse expectation deep learning neural network.
S3.2, acquiring input data corresponding to an output label consistent with an expected label from the training data, taking the input data as input, taking 1 as expected output, carrying out supervised training on the expected deep learning neural network, acquiring input data corresponding to an output label inconsistent with the expected label and an anti-expected label from the training data, taking the input data as input, taking 0 as expected output, and carrying out supervised training on the expected deep learning neural network;
further, input data corresponding to an output label consistent with an expected label in the training data can be used as first input data; screening first input data with a first preset proportion (assumed to be P%, P is a real number between 0 and 100) from the first input data to obtain each corresponding second input data (namely, reserving P% of data in the first input data, and emptying the rest of data to obtain each corresponding second input data); and taking each second input datum as an input, taking the first preset proportion as an expected output, and carrying out supervised training on the expected deep learning neural network.
It can be understood that "obtaining input data corresponding to an output label corresponding to an anti-expected label from training data, using the input data as an input, using 0 as an expected output, and performing supervised training on the expected deep learning neural network" is not performed because input data corresponding to an output label corresponding to an anti-expected label may be similar to input data corresponding to an output label corresponding to an expected label (for example, facial features of some men and women are very similar), thereby resulting in similar input data, different output labels being generated in the same deep learning neural network, and further negatively affecting the training effect of the expected deep learning neural network.
For example, acquiring an output label (the expected output is 1) consistent with the expected label 'man' from the training data and a face image corresponding to the output label to perform supervised training on the expected label 'man' deep learning neural network; acquiring an output label (the expected output is 0) inconsistent with an expected label 'man' and an anti-expected label 'woman' from training data and corresponding input data (such as an animal head portrait) of the output label and the anti-expected label 'woman', and performing supervised training on the expected label 'man' deep learning neural network; acquiring an output label (the expected output is 1) consistent with an expected label 'man' and a corresponding input data face image thereof from training data, reserving 60% of the input data face image (randomly selecting, uniformly selecting or selecting the part needing to be reserved according to a certain preset mode), removing the rest 40% of the input data face image from the image (randomly selecting, uniformly selecting or selecting the part needing to be reserved according to a certain preset mode), then taking the processed input data face image as a new input data face image, changing the output data 1 into the new output data 60%, namely 0.6, and carrying out supervised training on the expected label 'man' deep learning neural network.
And S3.3, acquiring input data corresponding to an output label consistent with the anti-expectation label from the training data, taking the input data as input, taking 1 as expected output, carrying out supervised training on the anti-expectation deep learning neural network, acquiring input data corresponding to an output label inconsistent with both the expectation label and the anti-expectation label from the training data, taking the input data as input, taking 0 as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
Further, input data corresponding to an output label consistent with an anti-expectation label in the training data can be used as third input data; screening out data with a second preset proportion (assumed to be P%, P is a real number between 0 and 100) from each third input data to obtain each corresponding fourth input data (namely, reserving P% of data in each third input data, and emptying the rest of data to obtain each corresponding fourth input data); and taking each fourth input data as input, taking the corresponding second preset proportion as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
It can be understood that "obtaining input data corresponding to an output label consistent with an expected label from training data, using the input data as an input, using 0 as an expected output, and performing supervised training on the anti-expectation deep learning neural network" is not performed because input data corresponding to an output label consistent with an expected label may be similar to input data corresponding to an output label consistent with an anti-expectation label (for example, facial features of some men and women are very similar), thereby resulting in similar input data, different output labels being generated in the same deep learning neural network, and further negatively affecting the training effect of the anti-expectation deep learning neural network.
For example, acquiring an output label (the expected output is 1) consistent with the anti-expectation label 'woman' from the training data and a face image corresponding to the output label to perform supervised training on the anti-expectation label 'woman' deep learning neural network; obtaining an output label (the expected output is 0) inconsistent with the expected label 'man' and the anti-expected label 'woman' from the training data and corresponding input data (such as an animal head portrait) to carry out supervised training on the expected label 'woman' deep learning neural network.
Acquiring an output label (the expected output is 1) which is consistent with an anti-expectation label 'woman' and a corresponding input data face image thereof from training data, reserving 60% of parts in the input data face image (randomly selecting, uniformly selecting or selecting parts needing to be reserved according to a certain preset mode), removing the rest 40% of parts from the image (randomly selecting, uniformly selecting or selecting parts needing to be reserved according to a certain preset mode), taking the processed input data face image as a new input data face image, changing output data 1 into new output data 60%, namely 0.6, and performing supervised training on the anti-expectation label 'woman' deep learning neural network.
Further, after the expected deep learning neural network and the inverse expected deep learning neural network are trained respectively, the reliability that the output corresponding to the input data belongs to the expected label and the reliability that the output corresponding to the input data belongs to the inverse expected label can be calculated. Specifically, the input data can be input into the expected deep learning neural network to obtain the output data of the expected deep learning neural network; inputting the input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network; and obtaining the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label according to the output data of the expected deep learning neural network and the output data of the anti-expected deep learning neural network.
For example, an input data face image is obtained, the input data face image is input into the expected deep learning neural network, output data of the expected deep learning neural network is obtained, the output data is a number between 0 and 1, and the closer to 1, the higher the probability that the output is an expected label is; inputting input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network, wherein the output data is a number between 0 and 1, and the closer to 1, the higher the probability that the output is the anti-expectation label is.
When calculating the reliability, the output data of the expected deep learning neural network can be taken as the probability (marked as A) that the output belongs to the expected label, and the output data of the inverse expected deep learning neural network can be taken as the probability (marked as B) that the output belongs to the inverse expected label; if the output corresponding to the input data belongs to the credibility (denoted as F) of the expected tag or the anti-expected tag, F is (A + B)/2; the output corresponding to the input data has a reliability of F × a for the expected tag, a reliability of F × B for the opposite tag, and a reliability of 1-F for the other tags.
For example, if the person is a man or a woman, the probability that the person is a man is F × a, the probability that the person is a man is F × B as the reliability of the person being a man, and the probability that the person is a woman is F × B as the reliability of the person being a woman; the probability of belonging to other tags is 1-F, which is the confidence that the person is neither a male nor a female.
For example, the deep learning neural network determines whether a person is a male, the avatar image is input into the training data, and if the output data in the training data is a male, the deep learning neural network corresponding to the "male" label is used for training, for example, output 1 represents a male, and output 0 represents a non-male (for example, an animal); if the output data in the training data is female, training by using a deep learning neural network corresponding to a 'female' label, for example, the output uses a number between 0 and 1 to represent the possibility of being female, 1 represents female, and 0 represents not female (for example, animal); if the output data in the training data is irrelevant to men and women, such as animals, the deep learning neural network corresponding to the label of the man and the deep learning neural network corresponding to the label of the woman are trained simultaneously.
When the deep learning neural network is used, a piece of data is input to the deep learning neural network corresponding to the 'man' label and the deep learning neural network corresponding to the 'woman' label for simultaneous calculation, and if the deep learning neural network corresponding to the 'man' label outputs 0.8; if the deep learning neural network output corresponding to the label of the 'woman' is 0.5, the reliability of the output of the 'man' or the 'woman' is (0.8+0.5)/2 is 65%, the reliability of the output of the 'man' or the 'woman' is 1-65% or 35%, the reliability of the output of the 'man' is 65% or 0.8% or 52%, the reliability of the output of the 'woman' is 65% or 0.5% or 32.5%, and the reliability of the output of the 'man' is relatively higher, so the 'man' is judged.
According to the method, 2 deep learning neural networks are constructed through expected labels and anti-expected labels of output data, and then if the output data in training data are consistent with the expected labels, the deep learning neural networks corresponding to the expected labels are trained; if the output data in the training data is consistent with the anti-expectation label, training a deep learning neural network corresponding to the anti-expectation label; if the output data in the training data is not related to the attribute (certainly, the output data is also not related to the anti-expectation label), the 2 deep learning neural networks corresponding to the expectation label and the anti-expectation label are trained at the same time. Training to obtain 2 deep learning neural networks, inputting input data into the 2 deep learning neural networks simultaneously in application to obtain 2 output data, and then integrating the 2 output data to obtain the reliability of the output data belonging to expected labels and the reliability of the output data belonging to anti-expected labels, so that the problem that the deep learning neural networks have structure fluctuation when output data with opposite attributes exist is avoided, and the reliability of deep learning neural network training is improved.
As shown in fig. 2, the present invention also provides a neural network system, which may include:
a label obtaining module 10, configured to obtain an expected label and an anti-expected label of output data in training data; wherein the anti-desired tag is a tag having an attribute opposite to that of the desired tag;
for example, the expectation label "man" and the anti-expectation label "woman" of the acquired output data.
The initialization module 20 is configured to initialize a deep learning neural network corresponding to the expected tag to obtain an expected deep learning neural network, and initialize a deep learning neural network corresponding to an anti-expected tag to obtain an anti-expected deep learning neural network;
initializing an input format of a deep learning neural network corresponding to the expected label into an input data format in the training data; the input format of the deep learning neural network corresponding to the anti-expectation label is also initialized to the input data format in the training data. The output format of the desired deep learning neural network is initialized to a number between 0 and 1, and if the output is 1, the output is a desired label, if the output is 0, the output is not a desired label, and if the output is a number between 0 and 1, the output may be a desired label. The output format of the anti-expectation deep learning neural network is initialized to a number between 0 and 1, and when the output is 1, the output is an anti-expectation label, when the output is 0, the output is not an anti-expectation label, and when the output is a number between 0 and 1, the output is possibly an anti-expectation label. Meanwhile, the configuration information of the existing similar deep learning neural network (the preset configuration information comprises the preset number of layers, the preset number of nodes on each layer and the preset weight value connected with each network) is obtained and used as the configuration information of the deep learning neural network corresponding to the expected label to configure the deep learning neural network corresponding to the expected label, and meanwhile, the configuration information is also used as the configuration information of the deep learning neural network corresponding to the anti-expected label to configure the deep learning neural network corresponding to the anti-expected label.
For example, 2 deep learning neural networks corresponding to the expectation label "man" and the anti-expectation label "woman" are initialized, which are called the expectation label "man" deep learning neural network and the anti-expectation label "woman" deep learning neural network.
And the training module 30 is used for respectively training the expected deep learning neural network and the anti-expected deep learning neural network.
In one embodiment, the training module may implement deep learning neural network training in the following manner:
s3.1, performing unsupervised training on the expected deep learning neural network and the inverse expected deep learning neural network respectively through input data in the training data;
for example, the expectation label 'man' deep learning neural network and the anti-expectation label 'woman' deep learning neural network are respectively subjected to unsupervised training through the input data face images in the training data. Description of the drawings: unsupervised training of the expected deep learning neural network and the inverse expected deep learning neural network can be performed using the same input data set.
S3.2, acquiring input data corresponding to an output label consistent with an expected label from the training data, taking the input data as input, taking 1 as expected output, performing supervised training on the expected deep learning neural network, acquiring input data corresponding to an output label inconsistent with the expected label and an anti-expected label from the training data, taking the input data as input, taking 1 as expected output, and performing supervised training on the expected deep learning neural network;
further, input data corresponding to an output label consistent with an expected label in the training data can be used as first input data; screening first input data with a first preset proportion (assumed to be P%, P is a real number between 0 and 100) from the first input data to obtain each corresponding second input data (namely, reserving P% of data in the first input data, and emptying the rest of data to obtain each corresponding second input data); multiplying the first output data consistent with the expected label by the first preset proportion to obtain second output data; and taking each second input datum as an input, taking the first preset proportion as an expected output, and carrying out supervised training on the expected deep learning neural network.
It can be understood that "obtaining input data corresponding to an output label matching an anti-expectation label from training data, using the input data as input, using 1 as expected output, and performing supervised training on the expected deep learning neural network" is not performed because input data corresponding to an output label matching an anti-expectation label may be similar to input data corresponding to an output label matching an expected label (for example, facial features of some men and women are very similar), thereby resulting in similar input data, different output labels being generated in the same deep learning neural network, and further negatively affecting the training effect of the expected deep learning neural network.
For example, acquiring an output label (the expected output is 1) consistent with the expected label 'man' from the training data and a face image corresponding to the output label to perform supervised training on the expected label 'man' deep learning neural network; acquiring an output label (the expected output is 0) inconsistent with an expected label 'man' and an anti-expected label 'woman' from training data and corresponding input data (such as an animal head portrait) of the output label and the anti-expected label 'woman', and performing supervised training on the expected label 'man' deep learning neural network; acquiring an output label (the expected output is 1) consistent with an expected label 'man' and a corresponding input data face image thereof from training data, reserving 60% of the input data face image (randomly selecting, uniformly selecting or selecting the part needing to be reserved according to a certain preset mode), removing the rest 40% of the input data face image from the image (randomly selecting, uniformly selecting or selecting the part needing to be reserved according to a certain preset mode), then taking the processed input data face image as a new input data face image, changing the output data 1 into the new output data 60%, namely 0.6, and carrying out supervised training on the expected label 'man' deep learning neural network.
And S3.3, acquiring input data corresponding to an output label consistent with the anti-expectation label from the training data, taking the input data as input, taking 1 as expected output, carrying out supervised training on the anti-expectation deep learning neural network, acquiring input data corresponding to an output label inconsistent with both the expectation label and the anti-expectation label from the training data, taking the input data as input, taking 1 as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
Further, input data corresponding to an output label consistent with an anti-expectation label in the training data can be used as third input data; screening third input data with a second preset proportion (assumed to be P%, P is a real number between 0 and 100) from the third input data to obtain each corresponding fourth input data (namely, reserving P% of data in the third input data, and emptying the rest of data to obtain each corresponding fourth input data); and taking each fourth input data as input, taking the corresponding second preset proportion as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
It can be understood that "obtaining input data corresponding to an output label consistent with an expected label from training data, using the input data as input, and using 1 as expected output, and performing supervised training on the anti-expectation deep learning neural network" is not performed because input data corresponding to an output label consistent with an expected label may be similar to input data corresponding to an output label consistent with an anti-expectation label (for example, facial features of some men and women are very similar), thereby resulting in similar input data, different output labels being generated in the same deep learning neural network, and further negatively affecting the training effect of the anti-expectation deep learning neural network.
For example, acquiring an output label (the expected output is 1) consistent with the anti-expectation label 'woman' from the training data and a face image corresponding to the output label to perform supervised training on the anti-expectation label 'woman' deep learning neural network; obtaining an output label (the expected output is 0) inconsistent with the expected label 'man' and the anti-expected label 'woman' from the training data and corresponding input data (such as an animal head portrait) to carry out supervised training on the expected label 'woman' deep learning neural network.
Acquiring an output label (the expected output is 1) which is consistent with an anti-expectation label 'woman' and a corresponding input data face image thereof from training data, reserving 60% of parts in the input data face image (randomly selecting, uniformly selecting or selecting parts needing to be reserved according to a certain preset mode), removing the rest 40% of parts from the image (randomly selecting, uniformly selecting or selecting parts needing to be reserved according to a certain preset mode), taking the processed input data face image as a new input data face image, changing output data 1 into new output data 60%, namely 0.6, and performing supervised training on the anti-expectation label 'woman' deep learning neural network.
Further, after the expected deep learning neural network and the inverse expected deep learning neural network are trained respectively, the reliability that the output corresponding to the input data belongs to the expected label and the reliability that the output corresponding to the input data belongs to the inverse expected label can be calculated. Specifically, the input data can be input into the expected deep learning neural network to obtain the output data of the expected deep learning neural network; inputting the input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network; and obtaining the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label according to the output data of the expected deep learning neural network and the output data of the anti-expected deep learning neural network.
For example, an input data face image is obtained, the input data face image is input into the expected deep learning neural network, output data of the expected deep learning neural network is obtained, the output data is a number between 0 and 1, and the closer to 1, the higher the probability that the output is an expected label is; inputting input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network, wherein the output data is a number between 0 and 1, and the closer to 1, the higher the probability that the output is the anti-expectation label is.
When calculating the reliability, the output data of the expected deep learning neural network can be taken as the probability (marked as A) that the output belongs to the expected label, and the output data of the inverse expected deep learning neural network can be taken as the probability (marked as B) that the output belongs to the inverse expected label; if the output corresponding to the input data belongs to the credibility (denoted as F) of the expected tag or the anti-expected tag, F is (A + B)/2; the output corresponding to the input data has a reliability of F × a for the expected tag, a reliability of F × B for the opposite tag, and a reliability of 1-F for the other tags.
For example, if the person is a man or a woman, the probability that the person is a man is F × a, the probability that the person is a man is F × B as the reliability of the person being a man, and the probability that the person is a woman is F × B as the reliability of the person being a woman; the probability of belonging to other tags is 1-F, which is the confidence that the person is neither a male nor a female.
For example, the deep learning neural network determines whether a person is a male, the avatar image is input into the training data, and if the output data in the training data is a male, the deep learning neural network corresponding to the "male" label is used for training, for example, output 1 represents a male, and output 0 represents a non-male (for example, an animal); if the output data in the training data is female, training by using a deep learning neural network corresponding to a 'female' label, for example, the output uses a number between 0 and 1 to represent the possibility of being female, 1 represents female, and 0 represents not female (for example, animal); if the output data in the training data is irrelevant to men and women, such as animals, the deep learning neural network corresponding to the label of the man and the deep learning neural network corresponding to the label of the woman are trained simultaneously.
When the deep learning neural network is used, a piece of data is input to the deep learning neural network corresponding to the 'man' label and the deep learning neural network corresponding to the 'woman' label for simultaneous calculation, and if the deep learning neural network corresponding to the 'man' label outputs 0.8; if the deep learning neural network output corresponding to the label of the 'woman' is 0.5, the reliability of the output of the 'man' or the 'woman' is (0.8+0.5)/2 is 65%, the reliability of the output of the 'man' or the 'woman' is 1-65% or 35%, the reliability of the output of the 'man' is 65% or 0.8% or 52%, the reliability of the output of the 'woman' is 65% or 0.5% or 32.5%, and the reliability of the output of the 'man' is relatively higher, so the 'man' is judged.
According to the method, 2 deep learning neural networks are constructed through expected labels and anti-expected labels of output data, and then if the output data in training data are consistent with the expected labels, the deep learning neural networks corresponding to the expected labels are trained; if the output data in the training data is consistent with the anti-expectation label, training a deep learning neural network corresponding to the anti-expectation label; if the output data in the training data is not related to the attribute (certainly, the output data is also not related to the anti-expectation label), the 2 deep learning neural networks corresponding to the expectation label and the anti-expectation label are trained at the same time. Training to obtain 2 deep learning neural networks, inputting input data into the 2 deep learning neural networks simultaneously in application to obtain 2 output data, and then integrating the 2 output data to obtain the credibility of the output data belonging to the expected label and the credibility of the output data belonging to the anti-expected label.
The neural network system of the present invention corresponds to the expectation-inverse deep learning method of the present invention, and the technical features and the advantages thereof described in the embodiments of the expectation-inverse deep learning method are all applicable to the embodiments of the neural network system, which is hereby stated.
Further, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is characterized in that the program is executed by a processor to implement the expectation and anti-expectation deep learning method. The expectation-and-expectation-counteractive deep learning method implemented by the program when the program is executed by the processor is the same as the above-mentioned embodiments of the expectation-and-expectation-counteractive deep learning method, and the details are not repeated here.
Further, the present invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the expectation-and-anti-expectation deep learning method when executing the program. The expectation-inverse-expectation-deep learning method implemented when the processor executes the program is the same as the embodiment of the expectation-inverse-expectation-deep learning method, and is not described herein again.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An expectation-inverse-expectation deep learning method is characterized by comprising the following steps of:
acquiring an expected label and an anti-expected label of output data in training data; wherein the anti-expectation label is a label with an attribute opposite to that of the expectation label, the input data in the training data is image data, the expectation label is of a first image category, and the anti-expectation label is of a second image category;
initializing a deep learning neural network corresponding to the expected label to obtain an expected deep learning neural network, and initializing a deep learning neural network corresponding to an anti-expected label to obtain an anti-expected deep learning neural network;
carrying out unsupervised training on the expected deep learning neural network and the inverse expected deep learning neural network respectively through input data in the training data;
acquiring input data corresponding to an output label consistent with an expected label from training data, taking the input data as input and 1 as expected output, performing supervised training on the expected deep learning neural network, acquiring input data corresponding to an output label inconsistent with the expected label and an anti-expected label from the training data, taking the input data as input and 0 as expected output, and performing supervised training on the expected deep learning neural network;
acquiring input data corresponding to an output label consistent with an anti-expectation label from training data, taking the input data as input and 1 as expected output, performing supervised training on the anti-expectation deep learning neural network, acquiring input data corresponding to an output label inconsistent with both the expectation label and the anti-expectation label from the training data, taking the input data as input and 0 as expected output, and performing supervised training on the anti-expectation deep learning neural network;
a trained expected deep learning neural network for predicting a probability that image data belongs to a first image class and an anti-expected deep learning neural network for predicting a probability that image data belongs to a second image class are obtained.
2. The expectation-anti-expectation deep learning method according to claim 1, wherein the step of obtaining input data corresponding to an output label consistent with an expected label from training data, using the input data as input, using 1 as expected output, and performing supervised training on the expectation deep learning neural network further comprises:
taking input data corresponding to an output label consistent with an expected label in the training data as first input data;
screening data of a first preset proportion from each first input data to obtain each corresponding second input data;
and taking each second input data as input, taking the corresponding first preset proportion as expected output, and carrying out supervised training on the expected deep learning neural network.
3. The expectation-and-anti-expectation deep learning method according to claim 1, wherein the step of obtaining input data corresponding to an output label consistent with an anti-expectation label from training data, using the input data as input and 1 as expected output, and performing supervised training on the anti-expectation deep learning neural network further comprises:
taking input data corresponding to an output label consistent with an anti-expectation label in the training data as third input data;
screening out data of a second preset proportion from each third input data to obtain each corresponding fourth input data;
and taking each fourth input data as input, taking the corresponding second preset proportion as expected output, and carrying out supervised training on the anti-expectation deep learning neural network.
4. The expectation-versus-expectation-deep learning method according to claim 1, further comprising the following steps after training the expectation-deep learning neural network and the expectation-versus-expectation-deep learning neural network respectively:
and calculating the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label.
5. The expectation-versus-expectation-deep learning method according to claim 4, wherein the step of calculating the credibility that the output corresponding to the input data belongs to the expectation label and the credibility that the output corresponding to the input data belongs to the expectation-versus-expectation label comprises:
inputting the input data into the expected deep learning neural network to obtain the output data of the expected deep learning neural network;
inputting the input data into the anti-expectation deep learning neural network to obtain output data of the anti-expectation deep learning neural network;
and obtaining the credibility that the output corresponding to the input data belongs to the expected label and the credibility that the output corresponding to the input data belongs to the anti-expected label according to the output data of the expected deep learning neural network and the output data of the anti-expected deep learning neural network.
6. The expectation-and-expectation-inverse deep learning method according to claim 5, wherein the step of obtaining the reliability that the output corresponding to the input data belongs to the expectation label and the reliability that the output corresponding to the input data belongs to the expectation label according to the output data of the expectation-and-expectation-inverse deep learning neural network comprises:
taking output data of the expected deep learning neural network as a probability A that the output belongs to an expected label, and taking output data of the inverse expected deep learning neural network as a probability B that the output belongs to an inverse expected label;
recording the credibility F of the output corresponding to the input data, which belongs to the expected label or the anti-expected label, as (A + B)/2;
the output corresponding to the input data has a reliability of F × a for the expected tag, a reliability of F × B for the opposite tag, and a reliability of 1-F for the other tags.
7. A neural network system, comprising:
the label acquisition module is used for acquiring an expected label and an anti-expected label of output data in the training data; wherein the anti-expectation label is a label with an attribute opposite to that of the expectation label, the input data in the training data is image data, the expectation label is of a first image category, and the anti-expectation label is of a second image category;
the initialization module is used for initializing the deep learning neural network corresponding to the expected tag to obtain an expected deep learning neural network, and initializing the deep learning neural network corresponding to the anti-expected tag to obtain an anti-expected deep learning neural network;
the training module is used for respectively carrying out unsupervised training on the expected deep learning neural network and the anti-expected deep learning neural network through input data in the training data; acquiring input data corresponding to an output label consistent with an expected label from training data, taking the input data as input and 1 as expected output, performing supervised training on the expected deep learning neural network, acquiring input data corresponding to an output label inconsistent with the expected label and an anti-expected label from the training data, taking the input data as input and 0 as expected output, and performing supervised training on the expected deep learning neural network; acquiring input data corresponding to an output label consistent with an anti-expectation label from training data, taking the input data as input and 1 as expected output, performing supervised training on the anti-expectation deep learning neural network, acquiring input data corresponding to an output label inconsistent with both the expectation label and the anti-expectation label from the training data, taking the input data as input and 0 as expected output, and performing supervised training on the anti-expectation deep learning neural network; a trained expected deep learning neural network for predicting a probability that image data belongs to a first image class and an anti-expected deep learning neural network for predicting a probability that image data belongs to a second image class are obtained.
8. The neural network system of claim 7, wherein the training module is further configured to take input data corresponding to an output label in the training data that is consistent with an expected label as the first input data; screening data of a first preset proportion from each first input data to obtain each corresponding second input data; and taking each second input data as input, taking the corresponding first preset proportion as expected output, and carrying out supervised training on the expected deep learning neural network.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the expectation-versus-expectation-deep learning method of any one of claims 1 to 6.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the expectation-versus-expectation-deep learning method of any one of claims 1 to 6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711001729.0A CN108038543B (en) | 2017-10-24 | 2017-10-24 | Expectation and anti-expectation deep learning method and neural network system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711001729.0A CN108038543B (en) | 2017-10-24 | 2017-10-24 | Expectation and anti-expectation deep learning method and neural network system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108038543A CN108038543A (en) | 2018-05-15 |
| CN108038543B true CN108038543B (en) | 2021-01-22 |
Family
ID=62093191
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711001729.0A Active CN108038543B (en) | 2017-10-24 | 2017-10-24 | Expectation and anti-expectation deep learning method and neural network system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038543B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101782652A (en) * | 2010-02-08 | 2010-07-21 | 上海和为科技有限公司 | Indoor positioning system based on RFID technology |
| CN103729459A (en) * | 2014-01-10 | 2014-04-16 | 北京邮电大学 | Method for establishing sentiment classification model |
| CN104850890A (en) * | 2015-04-14 | 2015-08-19 | 西安电子科技大学 | Method for adjusting parameter of convolution neural network based on example learning and Sadowsky distribution |
| CN104992142A (en) * | 2015-06-03 | 2015-10-21 | 江苏大学 | Pedestrian recognition method based on combination of depth learning and property learning |
| CN105260742A (en) * | 2015-09-29 | 2016-01-20 | 深圳大学 | Unified classification method for multiple types of data and system |
| US9443192B1 (en) * | 2015-08-30 | 2016-09-13 | Jasmin Cosic | Universal artificial intelligence engine for autonomous computing devices and software applications |
| CN106570148A (en) * | 2016-10-27 | 2017-04-19 | 浙江大学 | Convolutional neutral network-based attribute extraction method |
| CN106682694A (en) * | 2016-12-27 | 2017-05-17 | 复旦大学 | Sensitive image identification method based on depth learning |
| CN106815579A (en) * | 2017-01-22 | 2017-06-09 | 深圳市唯特视科技有限公司 | A kind of motion detection method based on multizone double fluid convolutional neural networks model |
| CN106845529A (en) * | 2016-12-30 | 2017-06-13 | 北京柏惠维康科技有限公司 | Image feature recognition methods based on many visual field convolutional neural networks |
| CN107077734A (en) * | 2015-02-06 | 2017-08-18 | 松下知识产权经营株式会社 | Determining method and program |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9418331B2 (en) * | 2013-10-28 | 2016-08-16 | Qualcomm Incorporated | Methods and apparatus for tagging classes using supervised learning |
| KR20160112186A (en) * | 2015-03-18 | 2016-09-28 | 삼성전자주식회사 | Method and apparatus for event-based learning in neural network |
-
2017
- 2017-10-24 CN CN201711001729.0A patent/CN108038543B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101782652A (en) * | 2010-02-08 | 2010-07-21 | 上海和为科技有限公司 | Indoor positioning system based on RFID technology |
| CN103729459A (en) * | 2014-01-10 | 2014-04-16 | 北京邮电大学 | Method for establishing sentiment classification model |
| CN107077734A (en) * | 2015-02-06 | 2017-08-18 | 松下知识产权经营株式会社 | Determining method and program |
| CN104850890A (en) * | 2015-04-14 | 2015-08-19 | 西安电子科技大学 | Method for adjusting parameter of convolution neural network based on example learning and Sadowsky distribution |
| CN104992142A (en) * | 2015-06-03 | 2015-10-21 | 江苏大学 | Pedestrian recognition method based on combination of depth learning and property learning |
| US9443192B1 (en) * | 2015-08-30 | 2016-09-13 | Jasmin Cosic | Universal artificial intelligence engine for autonomous computing devices and software applications |
| CN105260742A (en) * | 2015-09-29 | 2016-01-20 | 深圳大学 | Unified classification method for multiple types of data and system |
| CN106570148A (en) * | 2016-10-27 | 2017-04-19 | 浙江大学 | Convolutional neutral network-based attribute extraction method |
| CN106682694A (en) * | 2016-12-27 | 2017-05-17 | 复旦大学 | Sensitive image identification method based on depth learning |
| CN106845529A (en) * | 2016-12-30 | 2017-06-13 | 北京柏惠维康科技有限公司 | Image feature recognition methods based on many visual field convolutional neural networks |
| CN106815579A (en) * | 2017-01-22 | 2017-06-09 | 深圳市唯特视科技有限公司 | A kind of motion detection method based on multizone double fluid convolutional neural networks model |
Non-Patent Citations (3)
| Title |
|---|
| Understanding the Neural Control of Ingestive Behaviors:Helping to Separate Cause from Effect with Dehydration Associated Anorexia;Alan G.Watts;《Hormones and Behavior》;20020525;第37卷(第4期);第261-283页 * |
| 基于人脸视频信息的脉象分析;张晴晴等;《暨南大学学报(自然科学与医学版)》;20161117;第37卷(第5期);第430-435页 * |
| 基于迁移学习与深度卷积特征的图像标注方法研究;宋光慧;《中国博士学位论文全文数据库 信息科技辑》;20170815(第8期);第I138-83页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108038543A (en) | 2018-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6692488B2 (en) | Method and active learning system for training a neuron network | |
| CN109697434B (en) | Behavior recognition method and device and storage medium | |
| CN107680589B (en) | Voice information interaction method, device and equipment | |
| Guo et al. | Deep convolution feature learning for health indicator construction of bearings | |
| EP3975117A1 (en) | Image segmentation method and apparatus, and training method and apparatus for image segmentation model | |
| US11568212B2 (en) | Techniques for understanding how trained neural networks operate | |
| KR20180055070A (en) | Method and device to perform to train and recognize material | |
| KR102119057B1 (en) | Method for learning fracture diagnosis model and apparatus for the same | |
| WO2017158575A1 (en) | Method and system for processing a task with robustness to missing input information | |
| CN109522970B (en) | Image classification method, device and system | |
| JP5448758B2 (en) | Information processing apparatus, information processing method, information processing system, program, and data structure | |
| JP6158882B2 (en) | Generating device, generating method, and generating program | |
| CN111316290B (en) | System and method for interactive representation learning migration through deep learning of feature ontologies | |
| JP6107531B2 (en) | Feature extraction program and information processing apparatus | |
| CN112613349A (en) | Time sequence action detection method and device based on deep hybrid convolutional neural network | |
| Mirikharaji et al. | Deep auto-context fully convolutional neural network for skin lesion segmentation | |
| JP2007530088A (en) | Object-specific segmentation | |
| US20230252271A1 (en) | Electronic device and method for processing data based on reversible generative networks, associated electronic detection system and associated computer program | |
| Yang et al. | 3d topology-preserving segmentation with compound multi-slice representation | |
| CN110809768A (en) | Data cleansing system and method | |
| CN113688882B (en) | Training method and device for continuous learning neural network model with enhanced memory | |
| CN108038543B (en) | Expectation and anti-expectation deep learning method and neural network system | |
| Xiao et al. | Self-explanatory deep salient object detection | |
| CN114266920A (en) | Deep learning image classification method and system based on knowledge driving | |
| CN113158970A (en) | Action identification method and system based on fast and slow dual-flow graph convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |