CN112819715A - Data recovery method, network training method, related device and electronic equipment - Google Patents
Data recovery method, network training method, related device and electronic equipment Download PDFInfo
- Publication number
- CN112819715A CN112819715A CN202110127932.2A CN202110127932A CN112819715A CN 112819715 A CN112819715 A CN 112819715A CN 202110127932 A CN202110127932 A CN 202110127932A CN 112819715 A CN112819715 A CN 112819715A
- Authority
- CN
- China
- Prior art keywords
- person
- image data
- feature
- neural network
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012549 training Methods 0.000 title claims abstract description 42
- 238000011084 recovery Methods 0.000 title claims description 6
- 238000013528 artificial neural network Methods 0.000 claims abstract description 81
- 238000000605 extraction Methods 0.000 claims abstract description 71
- 238000012545 processing Methods 0.000 claims abstract description 65
- 230000009467 reduction Effects 0.000 claims abstract description 16
- 238000002372 labelling Methods 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 11
- 239000000126 substance Substances 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000001514 detection method Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000011176 pooling Methods 0.000 description 4
- 230000003796 beauty Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002087 whitening effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a data restoration method, a network training method, a related device and electronic equipment, and relates to the field of artificial intelligence such as computer vision and deep learning. The specific implementation scheme is as follows: acquiring first person image data, wherein the first person image data is person image data subjected to image processing; performing feature extraction on the first human image data based on a target neural network to obtain a first human feature, wherein the first human feature is a human feature of the first human image data before image processing; and acquiring second person image data matched with the first person characteristic. According to the technology of the application, the problem of low data reduction accuracy in the data reduction technology is solved, and the data reduction accuracy is improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and specifically relates to a data recovery method, a network training method, a related device and electronic equipment.
Background
With the rapid development of image processing technology, the figure images uploaded in the figure test scene at present may be images subjected to makeup, facial beautification or processing, which brings many problems to skin detection and artificial intelligence face detection in medical and beauty industries, identity recognition in financial industries and the like. For example, in the artificial intelligence face measurement in the medical and beauty industry, if the figure image uploaded by the user is an image shot by the user in a makeup state or an image automatically beautified by a camera, the skin quality problem does not exist in the result detected by the system.
The original appearance of the user in the character image can be restored through a data restoration technology, and in the related art, the original appearance of the user in the character image is usually restored through processes of stretching, buffing and the like of the human face in the character image.
Disclosure of Invention
The disclosure provides a data recovery method, a network training method, a related device and electronic equipment.
According to a first aspect of the present disclosure, there is provided a data restoring method, including:
acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
performing feature extraction on the first human image data based on a target neural network to obtain a first human feature, wherein the first human feature is a human feature of the first human image data before image processing;
and acquiring second person image data matched with the first person characteristic.
According to a second aspect of the present disclosure, there is provided a network training method, including:
acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is a person feature of the third person image data before image processing;
acquiring fourth person image data matched with the second person characteristics;
determining first difference information of the fourth human image data and target image data;
updating parameters of the target neural network based on the first difference information.
According to a third aspect of the present disclosure, there is provided a data reduction apparatus including:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring first person image data which is subjected to image processing;
a first feature extraction module, configured to perform feature extraction on the first human image data based on a target neural network to obtain a first human feature, where the first human feature is a human feature of the first human image data before image processing;
and the second acquisition module is used for acquiring second person image data matched with the first person characteristics.
According to a fourth aspect of the present disclosure, there is provided a network training apparatus, comprising:
the third acquisition module is used for acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
the second feature extraction module is used for performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is a person feature of the third person image data before image processing;
the fourth acquisition module is used for acquiring fourth character image data matched with the second character characteristics;
the first determining module is used for determining first difference information of the fourth human image data and the target image data;
an updating module for updating the parameter of the target neural network based on the first difference information.
According to a fifth aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any one of the methods of the first aspect or to perform any one of the methods of the second aspect.
According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform any one of the methods of the first aspect or to perform any one of the methods of the second aspect.
According to a seventh aspect of the present disclosure, there is provided a computer program product capable of performing any one of the methods of the first aspect or performing any one of the methods of the second aspect when the computer program product is run on an electronic device.
According to the technology of the application, the problem that the data reduction accuracy is low in the data reduction technology is solved, and the data reduction accuracy is improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic flow chart diagram of a data reduction method according to a first embodiment of the present application;
FIG. 2 is a block diagram of a data reduction of first human image data based on a target neural network;
FIG. 3 is a schematic flow chart diagram of a network training method according to a second embodiment of the present application;
FIG. 4 is a schematic diagram of a data reduction device according to a third embodiment of the present application;
FIG. 5 is a schematic diagram of a network training apparatus according to a fourth embodiment of the present application;
FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
First embodiment
As shown in fig. 1, the present application provides a data reduction method, including the following steps:
step S101: first person image data, which is image-processed person image data, is acquired.
In the embodiment, the data restoration method relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and can be widely applied to a plurality of scenes such as skin detection in medical and beauty industries, artificial intelligence side faces and identity recognition in financial industries.
In practical use, the data restoring method according to the embodiment of the present application may be executed by the data restoring apparatus according to the embodiment of the present application. The data restoring device of the embodiment of the application can be configured in any electronic equipment to execute the data restoring method of the embodiment of the application. The electronic device may be a server or a terminal, and is not limited herein.
The first human image data may be image data of a target image, and the target image may include image data of a human figure, which may include at least face image data of the human figure. Of course, the first human image data may further include image data such as a style template of the human figure, an overlay headwear border, and additional elements added to the human figure. In the following embodiments, the first human image data will be described by taking human face image data as an example.
The first person image data may include person attribute image data and person line image data, the person attribute image data may be a color RGB image data stream, and the person line image data may be a line image data stream. The line image data may be image data obtained by performing a painting extraction on an RGB image.
In addition, the first person image data is person image data subjected to image processing, and the image processing may include face buffing processing, face liquefaction processing, face whitening processing, head-mounted frame wearing processing, style conversion processing, and the like. The image processing software can be used for processing the person image, and the image processing function of the camera can also be used for processing the person image during shooting.
The first person image data may be obtained in a variety of manners, for example, a person image captured by a camera, where a face of a person image in the person image may be a face in a makeup state, and in such a scene, the person image may be regarded as an image subjected to face whitening processing. The figure image can also be an image subjected to face beautifying processing in the shooting process, and the face beautifying processing can be face buffing processing, face liquefying processing or head ornament frame wearing processing and the like. Accordingly, the data of the person image, which is the data of the first person image, may be obtained.
For another example, a personal image may be downloaded from a network or a personal image pre-stored in the acquiring device, and the personal image may be an image processed image, and accordingly, data of the personal image may be acquired, and the data of the personal image may be the first personal image data.
Step S102: and performing feature extraction on the first human image data based on a target neural network to obtain a first human feature, wherein the first human feature is a human feature of the first human image data before image processing.
In this step, the target neural network may be a convolutional neural network, and feature extraction may be performed on the first human image data by using the convolutional neural network to obtain a first human feature, where the first human feature may be a human feature of the first human image data before the image processing.
For example, if the first person image data is the person image data subjected to face beautification, the first person feature is the person feature of the first person image data before face beautification, that is, after face beautification is performed. And if the first person image data is the person image data subjected to the style conversion processing, the first person feature is the person feature of the first person image data in the normal shooting style.
The first person feature may include a person attribute feature and a person line feature, the person attribute feature may be a feature extracted based on the person attribute image data, and the person line feature may be a feature extracted based on the person line image data.
The character attribute features may include features such as contrast, brightness and shape of the human face, and the character line features may include link features of the character lines, such as whether the links are consistent.
The target neural network may include at least two convolutional layer modules, which are respectively used for performing feature extraction on the face attribute image data to obtain face attribute features, and performing feature extraction on the face line image data to obtain face line features.
In order to obtain the first human feature by using the target neural network, the target neural network needs to be trained to train initial values of the features, that is, to train human features of the image-processed human image data before the image processing, so as to restore the original appearance of the human face in the human image.
Step S103: and acquiring second person image data matched with the first person characteristic.
The second personal image data may be image data of the first personal image data before image processing, which is data of the image of the person at the time of the true value of the image pixel before image processing.
The second person image data matching the first person feature may be acquired in two ways, and the first person image data matching the first person feature may be acquired from a database in which person attribute image data corresponding to different person attribute features are stored.
Specifically, the person attribute image data corresponding to the person attribute feature may be acquired from the database, and the second person image data may be constructed based on the person line feature and the person attribute image data.
The second method can be used for up-sampling the figure of the character attribute characteristics to obtain character attribute image data, and the second character image data can be constructed based on the character line characteristics and the character attribute image data.
In this embodiment, feature extraction is performed on the first human image data through a target neural network to obtain a first human feature, where the first human feature is a human feature of the first human image data before the image processing; and acquiring second person image data matched with the first person features. Therefore, the character features of the image-processed character image data before the image processing can be obtained based on the target neural network, the original appearance of the human face in the character image can be restored based on the character features, and compared with the restoration processing of stretching, grinding and the like of the human face in the character image, the data restoration accuracy can be improved. Correspondingly, the human image is subjected to skin detection, face detection or identity recognition and the like based on the second human image data, so that the detection precision can be improved.
Optionally, the first person image data includes first person attribute image data and first person line image data, and the step S102 specifically includes:
performing feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and performing feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
In this embodiment, the first human image data may include first human attribute image data and first human line image data, the target neural network may include two convolutional layer modules, referring to fig. 2, fig. 2 is a schematic frame diagram of data reduction of the first human image data based on the target neural network, and as shown in fig. 2, the target neural network includes an RGB convolutional layer module and a line convolutional layer module.
Feature extraction can be performed on the first person attribute image data based on the RGB convolutional layer module to obtain first person attribute features, and feature extraction can be performed on the first person line image data based on the line convolutional layer module to obtain first person line features.
The first person line image data may be image data obtained by performing a drawing extraction on the first person attribute image data.
The first person characteristic may include a first person attribute characteristic and a first person line characteristic, the first person attribute characteristic may include characteristics of contrast, brightness, shape and the like of the human face, and the first person line characteristic may include a link characteristic of the person line, such as whether the links are consistent.
In the embodiment, the first human attribute feature and the first human line feature are respectively extracted through the two convolutional layer modules based on the target neural network, so that the first human attribute feature and the first human line feature can be identified when line joints in the first human line data are inconsistent and restored, and the accuracy of data restoration can be further improved.
Optionally, the performing, based on the target neural network, feature extraction on the first person attribute image data to obtain a first person attribute feature, and performing feature extraction on the first person line image data based on the target neural network to obtain a first person line feature includes:
performing feature extraction on the first person attribute image data based on a target neural network to obtain a first feature; feature extraction is carried out on the first human line image data based on the target neural network, and second features are obtained;
fusing the first characteristic and the second characteristic to obtain a target characteristic;
and carrying out feature classification on the target features to obtain first person attribute features and first person line features.
In this embodiment, as shown in fig. 2, the target neural Network further includes a generation candidate Network (RPN) layer and a Rol pooling layer, where the dimension of the first feature may include a plurality of features that can respectively represent dimensions such as contrast, brightness, and shape of a human face, the first feature may be processed by the RPN layer and then input to the Rol pooling layer, the second feature is also input to the Rol pooling layer, and the Rol pooling layer may merge the first feature and the second feature and perform feature classification to obtain a first human attribute feature and a first human line feature.
Thereafter, as shown in fig. 2, the person attribute image data corresponding to the first person attribute feature may be acquired, and the second person image data may be constructed based on the first person line feature and the person attribute image data.
In this embodiment, the first feature and the second feature are fused to obtain a target feature, and the target feature is subjected to feature classification to obtain a first person attribute feature and a first person line feature, so that the accuracy of feature extraction can be improved.
Second embodiment
As shown in fig. 3, the present application provides a network training method, including the following steps:
step S301: acquiring training sample data, wherein the training sample data comprises third person image data and target image data, and the third person image data is person image data subjected to image processing.
Step S302: performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is a person feature of the third person image data before image processing;
step S303: acquiring fourth person image data matched with the second person characteristics;
step S304: determining first difference information of the fourth human image data and target image data;
step S305: updating parameters of the target neural network based on the first difference information.
The purpose of this embodiment is to train a target neural network to train initial values of features, that is, to train human features of image-processed human image data before the image processing, so as to restore the original appearance of the human face in the human image.
The training sample data may include a plurality of pieces of third personal image data and target image data corresponding to the third personal image data, and the target image data may be image data of the third personal image data before image processing. For example, the person image of the character image a may include an original person image and a person image of the original person image subjected to image processing, the original person image is not subjected to any image processing, the target image data may be data of the original person image, and the third person image data may be data of the person image of the original person image subjected to image processing.
Feature extraction may be performed on the third person image data based on a target neural network to obtain a second person feature, where the second person feature may be a person feature of the third person image data before the image processing.
And acquiring fourth person image data matched with the second person characteristics, wherein the acquisition mode of the fourth person image data is similar to that of the second person image data, and the details are not repeated here.
Then, the fourth human image data may be compared with the target image data to determine first difference information between the fourth human image data and the target image data, where the first difference information between the fourth human image data and the target image data may be determined by using an existing or new distance calculation method.
Finally, the parameters of the target neural network may be updated based on the first difference information until the first difference information reaches a minimum, at which point the target neural network may be trained.
In this embodiment, the target neural network is trained to train an initial value of the feature, that is, to train the character feature of the image data of the character subjected to the image processing before the image processing, so as to restore the original face of the human face in the character image.
Optionally, the training sample data further includes a character labeling feature of the target image data, and before the step S305, the method further includes:
determining second difference information of the second character characteristics and the character marking characteristics;
the step S305 specifically includes:
updating a parameter of the target neural network based on the first difference information and the second difference information.
In this embodiment, the training sample data may further include a character tagging feature of the target image data, and when training sample data is prepared, an existing or new feature extraction network may be used to perform feature extraction on the target image data to obtain a character tagging feature of the target image data, where the character tagging feature represents a character feature in the target image data.
In order to improve the accuracy and speed of the target neural network training, the existing or new distance calculation method can be adopted to determine the second difference information of the second character features and the character labeling features.
Thereafter, parameters of the target neural network may be updated based on the first difference information and the second difference information. Specifically, a sum of the first difference information and the second difference information may be calculated, and based on the sum, the parameters of the target neural network may be updated until the sum reaches a minimum, at which point the training of the target neural network is completed.
In this embodiment, the accuracy and speed of the target neural network training can be improved by determining the second difference information of the second character feature and the character labeling feature, and updating the parameter of the target neural network based on the first difference information and the second difference information.
Optionally, the third person image data includes second person attribute image data and second person line image data, and the step S302 specifically includes:
performing feature extraction on the second person attribute image data based on a target neural network to obtain second person attribute features, and performing feature extraction on the second person line image data based on the target neural network to obtain second person line features;
wherein the second character characteristics include the second character attribute characteristics and second character line characteristics.
In this embodiment, the target neural network may include an RGB convolutional layer module and a line convolutional layer module.
Feature extraction may be performed on the second character attribute image data based on the RGB convolutional layer module to obtain a second character attribute feature, and feature extraction may be performed on the second character line image data based on the line convolutional layer module to obtain a second character line feature.
The second character line image data may be image data obtained by performing a drawing extraction on the second character attribute image data.
The second person characteristic may include a second person attribute characteristic and a second person line characteristic, the second person attribute characteristic may include characteristics of contrast, brightness, shape and the like of the human face, and the second person line characteristic may include a link characteristic of the person line, such as whether the links are consistent.
In the embodiment, the second character attribute features and the second character line features are respectively extracted through the two convolutional layer modules based on the target neural network, so that the line connection positions in the second character line data can be identified when the line connection positions are inconsistent, an initial value model of the character lines can be trained, and the accuracy of data reduction is further improved.
Optionally, the dimension of the second person attribute feature is N, the person labeling feature includes a person attribute labeling feature and a person line labeling feature, the dimension of the person attribute labeling feature is M, both M and N are positive integers greater than 1, and N is equal to M; the determining second difference information of the second person characteristic and the person labeling characteristic comprises:
determining a first loss value of the feature of each dimension in the second person attribute features and the feature corresponding to the dimension in the person attribute labeling features for the feature of each dimension in the second person attribute features;
determining a second loss value of the second character line feature and the character line marking feature;
wherein the second difference information includes the first loss value and a second loss value.
In this embodiment, for the feature of each dimension in the second person attribute features, different loss functions may be used to determine the first loss value of the feature of the dimension in the second person attribute features and the first loss value of the feature corresponding to the dimension in the person attribute labeling features based on the data distribution characteristics of the feature of each dimension.
Specifically, for the additional elements added to the character image and the feature of the scene dimension, the following formula (1) may be adopted to determine the first loss value of the feature of the dimension in the second character attribute feature and the feature corresponding to the dimension in the character attribute labeling feature.
Wherein, in the above formula (1),
which represents the value of the first loss to be,
may represent additional elements added to the character and features of the scene dimensions in the second character attribute feature, Φl(x) Can represent the additional elements added to the character image in the character attribute labeling characteristics and the characteristics of the scene dimension
And x represents data of additional elements added to the character image and features of scene dimensions in the second character attribute feature and the character attribute labeling feature respectively, and l represents a low-dimensional loss function.
For the feature of the style template dimension, a first loss value of the feature of the dimension in the second person attribute feature and the feature corresponding to the dimension in the person attribute labeling feature may be determined by using the following formula (2).
Wherein, in the above formula (2),
represents a first loss value, dcThe function of the vector is represented by a function,
denotes a feature for the style template dimension in the second person attribute feature, Ψi(Φl(yNN(i) ) represents a feature for a wind template dimension in the person attribute labeling feature, M is the number.
And Y represents the data of the second character attribute feature and the character attribute labeling feature for the feature of the style template dimension, and l represents the low-dimensional loss function.
For the feature of the face brightness dimension, the following equation (3) may be adopted to determine a first loss value of the feature of the dimension in the second person attribute feature and the feature corresponding to the dimension in the person attribute labeling feature.
Wherein, in the above formula (3),
which represents the value of the first loss to be,
may represent a feature for the face brightness dimension in the second person attribute feature, and Γ (x) may represent a feature for the face brightness dimension in the person attribute mark-up feature, and
and x represents data of the feature in the second person attribute feature and the person attribute labeling feature for the face brightness dimension, respectively.
Further, a second loss value for the second character line feature and the character line marking feature may be determined based on an existing or new loss function. The second difference information includes a first loss value and a second loss value, and the second difference information may specifically be a sum of the first loss value and the second loss value obtained by the calculation.
In this embodiment, based on the data distribution characteristics of the features of each dimension, different loss functions are adopted to determine the first loss values of the features of the dimension in the second character attribute features and the features corresponding to the dimension in the character attribute labeling features, so as to obtain second difference information, thereby further improving the precision and speed of the target neural network training.
Third embodiment
As shown in fig. 4, the present application provides a data restoring apparatus 400, including:
a first obtaining module 401, configured to obtain first person image data, where the first person image data is person image data subjected to image processing;
a first feature extraction module 402, configured to perform feature extraction on the first human image data based on a target neural network to obtain a first human feature, where the first human feature is a human feature of the first human image data before image processing;
a second obtaining module 403, configured to obtain second person image data matched with the first person feature.
Optionally, the first person image data includes first person attribute image data and first person line image data, and the first feature extraction module 402 includes:
the feature extraction unit is used for performing feature extraction on the first person attribute image data based on a target neural network to obtain a first person attribute feature, and performing feature extraction on the first person line image data based on the target neural network to obtain a first person line feature;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
Optionally, the feature extraction unit is specifically configured to perform feature extraction on the first human attribute image data based on a target neural network to obtain a first feature; feature extraction is carried out on the first human line image data based on the target neural network, and second features are obtained; fusing the first characteristic and the second characteristic to obtain a target characteristic; and carrying out feature classification on the target features to obtain first person attribute features and first person line features.
The data restoring apparatus 400 provided in the present application can implement each process implemented by the embodiment of the data restoring method, and can achieve the same beneficial effects, and for avoiding repetition, the details are not repeated here.
Fourth embodiment
As shown in fig. 5, the present application provides a network training apparatus 500, comprising:
a third obtaining module 501, configured to obtain training sample data, where the training sample data includes third person image data and target image data, and the third person image data is person image data subjected to image processing;
a second feature extraction module 502, configured to perform feature extraction on the third personal image data based on a target neural network to obtain a second personal feature, where the second personal feature is a personal feature of the third personal image data before image processing;
a fourth obtaining module 503, configured to obtain fourth person image data matched with the second person feature;
a first determining module 504, configured to determine first difference information between the fourth human image data and the target image data;
an updating module 505, configured to update a parameter of the target neural network based on the first difference information.
Optionally, the training sample data further includes a character labeling feature of the target image data, and the apparatus further includes:
the second determining module is used for determining second difference information of the second character characteristic and the character marking characteristic;
the updating module is specifically configured to update a parameter of the target neural network based on the first difference information and the second difference information.
Optionally, the third person image data includes second person attribute image data and second person line image data, and the second feature extraction module 502 is specifically configured to perform feature extraction on the second person attribute image data based on a target neural network to obtain second person attribute features, and perform feature extraction on the second person line image data based on the target neural network to obtain second person line features; wherein the second character characteristics include the second character attribute characteristics and second character line characteristics.
Optionally, the dimension of the second person attribute feature is N, the person labeling feature includes a person attribute labeling feature and a person line labeling feature, the dimension of the person attribute labeling feature is M, both M and N are positive integers greater than 1, and N is equal to M; the second determining module is specifically configured to determine, for a feature of each dimension in the second person attribute features, a first loss value of a feature of the dimension in the second person attribute features and a feature corresponding to the dimension in the person attribute labeling features; determining a second loss value of the second character line feature and the character line marking feature; wherein the second difference information includes the first loss value and a second loss value.
The network training apparatus 500 provided in the present application can implement each process implemented by the embodiment of the network training method described above, and can achieve the same beneficial effects, and for avoiding repetition, the details are not repeated here.
There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.
FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the device 600 can also be stored. The calculation unit 601, the ROM602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, such as the data restoration method or the network training method. For example, in some embodiments, the data reduction method or the network training method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM602 and/or the communication unit 609. When loaded into RAM603 and executed by the computing unit 601, a computer program may perform one or more steps of the data recovery method described above, or perform one or more steps of the network training method described above. Alternatively, in other embodiments, the computing unit 601 may be configured by any other suitable method (e.g., by means of firmware) to perform a data reduction method or a network training method.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more editing languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (17)
1. A method of data reduction, comprising:
acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
performing feature extraction on the first human image data based on a target neural network to obtain a first human feature, wherein the first human feature is a human feature of the first human image data before image processing;
and acquiring second person image data matched with the first person characteristic.
2. The method of claim 1, wherein the first human image data comprises first human attribute image data and first human line image data, and the feature extracting the first human image data based on a target neural network to obtain a first human feature comprises:
performing feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and performing feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
3. The method of claim 2, wherein the performing feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features and performing feature extraction on the first person line image data based on the target neural network to obtain first person line features comprises:
performing feature extraction on the first person attribute image data based on a target neural network to obtain a first feature; feature extraction is carried out on the first human line image data based on the target neural network, and second features are obtained;
fusing the first characteristic and the second characteristic to obtain a target characteristic;
and carrying out feature classification on the target features to obtain first person attribute features and first person line features.
4. A network training method, comprising:
acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is a person feature of the third person image data before image processing;
acquiring fourth person image data matched with the second person characteristics;
determining first difference information of the fourth human image data and target image data;
updating parameters of the target neural network based on the first difference information.
5. The method of claim 4, wherein the training sample data further comprises person labeling features of the target image data, and before updating parameters of the target neural network based on the first difference information, the method further comprises:
determining second difference information of the second character characteristics and the character marking characteristics;
the updating the parameter of the target neural network based on the first difference information comprises:
updating a parameter of the target neural network based on the first difference information and the second difference information.
6. The method of claim 5, wherein the third person image data includes second person attribute image data and second person line image data, and the feature extraction of the third person image data based on the target neural network to obtain the second person feature comprises:
performing feature extraction on the second person attribute image data based on a target neural network to obtain second person attribute features, and performing feature extraction on the second person line image data based on the target neural network to obtain second person line features;
wherein the second character characteristics include the second character attribute characteristics and second character line characteristics.
7. The method of claim 6, wherein the dimension of the second person attribute feature is N, the person labeling features comprise a person attribute labeling feature and a person line labeling feature, the dimension of the person attribute labeling feature is M, M and N are positive integers greater than 1, and N is equal to M; the determining second difference information of the second person characteristic and the person labeling characteristic comprises:
determining a first loss value of the feature of each dimension in the second person attribute features and the feature corresponding to the dimension in the person attribute labeling features for the feature of each dimension in the second person attribute features;
determining a second loss value of the second character line feature and the character line marking feature;
wherein the second difference information includes the first loss value and a second loss value.
8. A data recovery apparatus comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring first person image data which is subjected to image processing;
a first feature extraction module, configured to perform feature extraction on the first human image data based on a target neural network to obtain a first human feature, where the first human feature is a human feature of the first human image data before image processing;
and the second acquisition module is used for acquiring second person image data matched with the first person characteristics.
9. The apparatus of claim 8, wherein the first person image data comprises first person attribute image data and first person line image data, the first feature extraction module comprising:
the feature extraction unit is used for performing feature extraction on the first person attribute image data based on a target neural network to obtain a first person attribute feature, and performing feature extraction on the first person line image data based on the target neural network to obtain a first person line feature;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
10. The apparatus according to claim 9, wherein the feature extraction unit is specifically configured to perform feature extraction on the first human attribute image data based on a target neural network to obtain a first feature; feature extraction is carried out on the first human line image data based on the target neural network, and second features are obtained; fusing the first characteristic and the second characteristic to obtain a target characteristic; and carrying out feature classification on the target features to obtain first person attribute features and first person line features.
11. A network training apparatus comprising:
the third acquisition module is used for acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
the second feature extraction module is used for performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is a person feature of the third person image data before image processing;
the fourth acquisition module is used for acquiring fourth character image data matched with the second character characteristics;
the first determining module is used for determining first difference information of the fourth human image data and the target image data;
an updating module for updating the parameter of the target neural network based on the first difference information.
12. The apparatus of claim 11, wherein the training sample data further comprises character annotation features of the target image data, the apparatus further comprising:
the second determining module is used for determining second difference information of the second character characteristic and the character marking characteristic;
the updating module is specifically configured to update a parameter of the target neural network based on the first difference information and the second difference information.
13. The apparatus according to claim 12, wherein the third person image data includes second person attribute image data and second person line image data, and the second feature extraction module is specifically configured to perform feature extraction on the second person attribute image data based on a target neural network to obtain second person attribute features, and perform feature extraction on the second person line image data based on the target neural network to obtain second person line features; wherein the second character characteristics include the second character attribute characteristics and second character line characteristics.
14. The apparatus of claim 13, wherein the dimension of the second person attribute feature is N, the person labeling feature comprises a person attribute labeling feature and a person line labeling feature, the dimension of the person attribute labeling feature is M, M and N are positive integers greater than 1, and N is equal to M; the second determining module is specifically configured to determine, for a feature of each dimension in the second person attribute features, a first loss value of a feature of the dimension in the second person attribute features and a feature corresponding to the dimension in the person attribute labeling features; determining a second loss value of the second character line feature and the character line marking feature; wherein the second difference information includes the first loss value and a second loss value.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3 or to perform the method of any one of claims 4-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-3 or to perform the method of any one of claims 4-7.
17. A computer program product for performing the method of any one of claims 1-3 or performing the method of any one of claims 4-7 when the computer program product is run on an electronic device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110127932.2A CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110127932.2A CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112819715A true CN112819715A (en) | 2021-05-18 |
| CN112819715B CN112819715B (en) | 2024-04-05 |
Family
ID=75860310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110127932.2A Active CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819715B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
| CN111275784A (en) * | 2020-01-20 | 2020-06-12 | 北京百度网讯科技有限公司 | Method and device for generating image |
| AU2020101122A4 (en) * | 2020-06-25 | 2020-07-30 | A, Tamilselvi MS | Cnn model based approach towards breast cancer features detection from ultra sound images |
| CN111753770A (en) * | 2020-06-29 | 2020-10-09 | 北京百度网讯科技有限公司 | Person attribute identification method and device, electronic equipment and storage medium |
| CN112232425A (en) * | 2020-10-21 | 2021-01-15 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
-
2021
- 2021-01-29 CN CN202110127932.2A patent/CN112819715B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
| CN111275784A (en) * | 2020-01-20 | 2020-06-12 | 北京百度网讯科技有限公司 | Method and device for generating image |
| AU2020101122A4 (en) * | 2020-06-25 | 2020-07-30 | A, Tamilselvi MS | Cnn model based approach towards breast cancer features detection from ultra sound images |
| CN111753770A (en) * | 2020-06-29 | 2020-10-09 | 北京百度网讯科技有限公司 | Person attribute identification method and device, electronic equipment and storage medium |
| CN112232425A (en) * | 2020-10-21 | 2021-01-15 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 杨瞻远: "基于深度神经网络的图像超分辨率重构算法研究", 《中国优秀硕士学位论文全文数据库》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112819715B (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190325200A1 (en) | Face image processing methods and apparatuses, and electronic devices | |
| US10599914B2 (en) | Method and apparatus for human face image processing | |
| CN113327278B (en) | Three-dimensional face reconstruction method, device, equipment and storage medium | |
| CN113343826B (en) | Training method of human face living body detection model, human face living body detection method and human face living body detection device | |
| CN113033566B (en) | Model training method, recognition method, device, storage medium, and program product | |
| CN115049799B (en) | Method and device for generating 3D model and virtual image | |
| CN114723888B (en) | Three-dimensional hair model generation method, device, equipment, storage medium and product | |
| CN113177449B (en) | Face recognition method, device, computer equipment and storage medium | |
| CN113177472A (en) | Dynamic gesture recognition method, device, equipment and storage medium | |
| CN114549710A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN112802037A (en) | Portrait extraction method, device, electronic equipment and storage medium | |
| CN113221771A (en) | Living body face recognition method, living body face recognition device, living body face recognition equipment, storage medium and program product | |
| CN113962845B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN114549728A (en) | Training method of image processing model, image processing method, device and medium | |
| CN114120413A (en) | Model training method, image synthesis method, device, equipment and program product | |
| CN113379877A (en) | Face video generation method and device, electronic equipment and storage medium | |
| CN113177466A (en) | Identity recognition method and device based on face image, electronic equipment and medium | |
| CN115393488B (en) | Method and device for driving virtual character expression, electronic equipment and storage medium | |
| CN114140320B (en) | Image migration method and training method and device of image migration model | |
| CN113781653B (en) | Object model generation method and device, electronic equipment and storage medium | |
| CN112819715B (en) | Data recovery method, network training method, related device and electronic equipment | |
| CN112560848B (en) | Training method and device for POI (Point of interest) pre-training model and electronic equipment | |
| CN115082298A (en) | Image generation method, image generation device, electronic device, and storage medium | |
| CN113903071A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN113657248A (en) | Training method and device for face recognition model and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |