CN113065592A - Image classification method and device, electronic equipment and storage medium - Google Patents
Image classification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113065592A CN113065592A CN202110346171.XA CN202110346171A CN113065592A CN 113065592 A CN113065592 A CN 113065592A CN 202110346171 A CN202110346171 A CN 202110346171A CN 113065592 A CN113065592 A CN 113065592A
- Authority
- CN
- China
- Prior art keywords
- classification
- image
- target
- feature
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000013528 artificial neural network Methods 0.000 claims abstract description 45
- 238000000605 extraction Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 12
- 238000012360 testing method Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 16
- 238000002372 labelling Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 abstract description 12
- 239000013598 vector Substances 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure provides an image classification method, an apparatus, an electronic device, and a storage medium, the image classification method including: inputting the images to be classified into a trained neural network for feature extraction; obtaining target classification central features corresponding to the images to be classified based on the extracted image features and original classification central features; the original classification center features are obtained by training and learning of the neural network based on sample images; and classifying the images to be classified based on the target classification center characteristics to obtain a classification result. According to the embodiment of the application, the single detection model can adaptively detect the image tampered by multiple tampering modes, and the applicability of the detection model is improved.
Description
Technical Field
The present disclosure relates to the field of computer vision technologies, and in particular, to an image classification method and apparatus, an electronic device, and a storage medium.
Background
With the continuous progress of the deep forgery tampering technology, the deep forgery detection method is provided with great challenges. For example, a Generic Adaptive Networks (GAN) can generate counterfeit data with extremely high fidelity, thereby achieving the effect of falsifying and falsifying.
Due to the increasing richness of the counterfeiting method, the difficulty of identifying various types of counterfeit images by using a single detection model is increased. Furthermore, the variance in the level of the tamper method will also result in greater variability within the tamper category dataclass. Therefore, it is important how to adaptively detect counterfeit images in multiple falsification modes by using a single detection model.
Disclosure of Invention
The embodiment of the disclosure at least provides an image classification method and device, electronic equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides an image classification method, including:
inputting the images to be classified into a trained neural network for feature extraction;
obtaining target classification central features corresponding to the images to be classified based on the extracted image features and original classification central features; the original classification center features are obtained by training and learning of the neural network based on sample images;
and classifying the images to be classified based on the target classification center characteristics to obtain a classification result.
In the embodiment of the disclosure, after the image features of the image to be classified are extracted, the original classification center features are shifted according to the extracted image features to obtain the target classification center features, so that the adjusted target classification center features are more suitable for the classification of the current image to be classified. That is, the offset of the current image to be classified to the original classification center feature is generated according to the feature of the current image to be classified, so that the self-adaptive adjustment of the classification center feature is realized, different images to be classified are met, and the accuracy of image classification is improved.
According to the first aspect, in a possible implementation manner, the obtaining, based on the extracted image features and original classification center features, target classification center features corresponding to the image to be classified includes:
obtaining the offset of the target classification central feature relative to the original classification central feature based on the extracted image feature;
and obtaining the target classification central feature based on the offset and the original classification central feature.
In the embodiment of the disclosure, the offset of the target classification center feature relative to the original classification center feature is obtained according to the extracted image feature, and then the target classification center feature is obtained based on the offset and the original classification center feature.
According to the first aspect, in one possible implementation, the original classification center features include original real sample class center features and original tampered sample class center features; the target classification central features comprise target real sample class central features and target tampering sample class central features;
classifying the image to be classified based on the target classification center features to obtain a classification result, wherein the classification result comprises the following steps:
classifying the images to be classified based on the target real sample class central feature and the target tampering sample class central feature to obtain the classification result.
In the embodiment of the disclosure, the original classification center comprises an original real sample class center feature and an original tampered sample class center feature; the images to be classified are classified according to the two classification center features, and the accuracy of the test can be further improved.
According to the first aspect, in a possible implementation manner, the obtaining, based on the extracted image features and original classification center features, target classification center features corresponding to the image to be classified includes:
respectively obtaining a first offset of the target real sample class central feature relative to the original real sample classification central feature and a second offset of the target tampered sample class central feature relative to the original tampered sample class central feature on the basis of the extracted image features;
and adding the first offset and the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset and the original tampered sample class center feature to obtain the target tampered sample class center feature.
In the embodiment of the disclosure, a first offset relative to an original real sample class center and a second offset relative to an original tampered sample class center are predicted at the same time, and element-by-element addition is performed on feature dimensions to obtain an adjusted classification center, so that the adaptability of the obtained target real sample class center feature and the target tampered sample class center feature is improved.
According to the first aspect, in a possible implementation manner, the classifying the image to be classified based on the target classification center feature to obtain a classification result includes:
respectively comparing the extracted image features with the target real sample class central features and the target tampering sample class central features to respectively obtain a first difference comparison result between the image to be classified and the target real sample class central features and a second difference comparison result between the image to be classified and the target tampering sample class central features;
and classifying the images to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.
According to the first aspect, in a possible implementation manner, the classifying the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result includes:
and determining the classification corresponding to the difference comparison result with the minimum difference in the first difference comparison result and the second difference comparison result as the classification result corresponding to the image to be classified.
According to a first aspect, in a possible implementation, the image to be classified comprises a test image, the method further comprising:
acquiring a labeling result corresponding to the test image;
and adjusting parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain the tested neural network.
In the embodiment of the disclosure, after the classification result is obtained, the parameters of the neural network are adjusted according to the obtained classification result, so that the performance of the neural network can be further improved, and the reliability of subsequent image detection is further improved.
In a second aspect, an embodiment of the present disclosure provides an image classification apparatus, including:
the extraction module is used for inputting the images to be classified into the trained neural network for feature extraction;
the calculation module is used for obtaining target classification center features corresponding to the images to be classified based on the extracted image features and the original classification center features; the original classification center features are obtained by training and learning of the neural network based on sample images;
and the classification module is used for classifying the images to be classified based on the target classification center characteristics to obtain a classification result.
According to the second aspect, in a possible implementation, the calculation module is specifically configured to:
obtaining the offset of the target classification central feature relative to the original classification central feature based on the extracted image feature;
and obtaining the target classification central feature based on the offset and the original classification central feature.
According to the second aspect, in one possible implementation, the original classification center features include original real sample class center features and original tampered sample class center features; the target classification central features comprise target real sample class central features and target tampering sample class central features;
the classification module is specifically configured to:
classifying the images to be classified based on the target real sample class central feature and the target tampering sample class central feature to obtain the classification result.
According to the second aspect, in a possible implementation, the calculation module is specifically configured to:
respectively obtaining a first offset of the target real sample class central feature relative to the original real sample classification central feature and a second offset of the target tampered sample class central feature relative to the original tampered sample class central feature on the basis of the extracted image features;
and adding the first offset and the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset and the original tampered sample class center feature to obtain the target tampered sample class center feature.
According to the second aspect, in a possible implementation, the classification module is specifically configured to:
respectively comparing the extracted image features with the target real sample class central features and the target tampering sample class central features to respectively obtain a first difference comparison result between the image to be classified and the target real sample class central features and a second difference comparison result between the image to be classified and the target tampering sample class central features;
and classifying the images to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.
According to the second aspect, in a possible implementation, the classification module is specifically configured to:
and determining the classification corresponding to the difference comparison result with the minimum difference in the first difference comparison result and the second difference comparison result as the classification result corresponding to the image to be classified.
According to a second aspect, in a possible embodiment, the image to be classified comprises a test image; the device further comprises:
the acquisition module is used for acquiring a labeling result corresponding to the test image;
and the adjusting module is used for adjusting parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain the tested neural network.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the image classification method according to the first aspect.
In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the image classification method according to the first aspect.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 illustrates a flowchart of an image classification method provided by an embodiment of the present disclosure;
FIG. 2 is a flow chart of another image classification method provided by the embodiments of the present disclosure;
FIG. 3 is a flowchart illustrating a method for determining a target classification center according to an embodiment of the present disclosure;
FIG. 4a is a schematic diagram illustrating a process of classifying an image of a to-be-classified graph according to an original classification center feature provided by an embodiment of the present disclosure;
FIG. 4b is a schematic diagram illustrating a process of determining a target classification center feature according to an embodiment of the present disclosure;
FIG. 4c is a schematic diagram illustrating another process for determining a target classification center feature provided by the embodiment of the present disclosure;
FIG. 5 is a flowchart illustrating a method for classifying an image to be classified to obtain a classification result according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a neural network provided by an embodiment of the present disclosure;
FIG. 7 is a flowchart illustrating another method for classifying an image to be classified to obtain a classification result according to an embodiment of the present disclosure;
FIG. 8 is a flow chart illustrating a further image classification method provided by an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of an image classification apparatus provided in an embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of another image classification apparatus provided in the embodiment of the present disclosure;
fig. 11 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Deep Fake (Deep Fake) technique is also known in the industry as artificial intelligence face changing technique, which can digitally process images or videos to imitate a specific person, and generate numerous false audio and video contents, making it difficult to distinguish the true information from the false information. Due to the potential social hazard of deep forged content, the detection and the authenticity identification of the forged content are very necessary.
The existing tampering and counterfeiting means comprise face changing, five sense organs changing, face attribute changing, full-image simulation generation and the like, and the difference of the tampered images generated by various tampering means is large, so that the difference in the tampered data is large, and difficulty is brought to model detection.
In addition, when the tampering mode is unknown, the stable effect in various tampering data is difficult to achieve by using a single detection model. Therefore, it is important how to adaptively detect counterfeit images in multiple falsification modes by using a single detection model.
Based on the research, the present disclosure provides an image classification method, which includes inputting an image to be classified into a trained neural network for feature extraction; then, based on the extracted image features and original classification center features, obtaining target classification center features corresponding to the images to be classified; the original classification center features are obtained by training and learning of the neural network based on sample images; and classifying the images to be classified based on the target classification center characteristics to obtain a classification result. Therefore, for different images to be classified, corresponding target classification center features can be obtained, and then the images to be classified are classified based on the target classification center features, so that the current model can adapt to different types of images to be classified.
To facilitate understanding of the present embodiment, first, an image classification method disclosed in the embodiments of the present disclosure is described in detail, where an execution subject of the image classification method provided in the embodiments of the present disclosure is generally an electronic device with certain computing capability, and the electronic device includes, for example: a terminal device, which may be a mobile device, a user terminal, a handheld device, a computing device, a vehicle device, a wearable device, or the like, or a server or other processing device. In some possible implementations, the image classification method may be implemented by a processor calling computer readable instructions stored in a memory.
Referring to fig. 1, a flowchart of an image classification method provided in an embodiment of the present disclosure is shown, where the image classification method includes the following steps S101 to S103:
and S101, inputting the image to be classified into the trained neural network for feature extraction.
For example, the image to be classified is an image to be detected for authenticity, and may be a single image, or a certain frame of image obtained after decoding the target video, which is not limited herein.
It should be understood that a large number of sample images with labels are input into the neural network for training and learning, and a trained neural network can be obtained. Wherein the plurality of sample images may include a number of real sample class images and a number of tampered sample class images. Thus, the original classification center features used for classifying the images to be classified can be obtained through training of a large number of sample images.
S102, obtaining target classification center features corresponding to the images to be classified based on the extracted image features and original classification center features; wherein the original classification center features are obtained by training and learning of the neural network based on sample images.
Illustratively, the offset of the target classification center feature relative to the original classification center feature can be obtained based on the extracted image features; and then adding the offset and the original classification center feature to obtain the new classification center feature. Thus, the obtained target can be adapted to the classification of the current image to be classified.
S103, classifying the images to be classified based on the target classification center features to obtain a classification result.
For example, the extracted image features of the image to be classified may be compared with the target classification center features, and if the comparison result is within a preset range, it may be determined that the image to be classified is a real image or a tampered image. The specific comparison and classification method will be described in detail later.
In the embodiment of the disclosure, after the image features of the image to be classified are extracted, the original classification center features are shifted according to the extracted image features to obtain the target classification center features, so that the adjusted target classification center features are more suitable for the classification of the current image to be classified. That is, the offset of the current image to be classified to the original classification center feature is generated according to the feature of the current image to be classified, so that the self-adaptive adjustment of the classification center feature is realized, different images to be classified are met, and the accuracy of image classification is improved.
In some embodiments, to further improve the image detection accuracy, the original classification center features include original real sample class center features and original tampered sample class center features. That is, the technical scheme provided by the present disclosure may be implemented by a neural network, and in the process of training the neural network, training may be performed according to the authenticity of the input sample image, so as to obtain the original real sample class center feature and the original tampered sample class center feature, respectively. Therefore, referring to fig. 2, a flowchart of another image classification method provided in the embodiment of the present disclosure is shown, where the image classification method includes the following steps S201 to S203:
s201, inputting the image to be classified into the trained neural network for feature extraction.
Step S201 is similar to step S101, and is not described herein again.
S202, obtaining a target real sample class central feature corresponding to the image to be classified based on the extracted image feature and the original real sample class central feature, and obtaining a target tampering sample class central feature corresponding to the image to be classified based on the extracted image feature and the original tampering sample class central feature.
Illustratively, as shown in FIG. 3, step S202 may include the following S2021-2022:
s2021, based on the extracted image features, respectively obtaining a first offset of the target real sample class central feature relative to the original real sample classification central feature, and obtaining a second offset of the target tampered sample class central feature relative to the original tampered sample class central feature.
For example, the first offset and the second offset of the target tampered sample class center feature relative to the original tampered sample class center feature may be obtained according to the following formula and the currently extracted image feature, respectively.
Wherein x is the extracted graphImage features; [ WIwN;]is a classification center feature; tau is a scale factor; byCorresponding to different classification center features.
In particular, in a neural network, the offset may be derived by two fully-connected layers of the neural network. Wherein the two connection layers respectively have a normalization processing function (SoftMax) and a Linear rectification function (ReLU). The ReLU, also called modified linear unit, is an activation function commonly used in artificial neural networks, and usually refers to a nonlinear function represented by a ramp function and its variants.
And S2022, adding the first offset and the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset and the original tampered sample class center feature to obtain the target tampered sample class center feature.
Exemplarily, an image feature is obtained by performing feature extraction on an image to be classified through a trained neural network, the image feature is a high-dimensional feature, and a schematic diagram of a two-dimensional plane formed after dimension reduction is shown in fig. 4a, where w indicated by a is wrRepresenting the characteristics of the classification center of the original real sample class, w indicated by BfRepresenting the classification center feature of the original tampered sample class, "+" representing the image to be classified with the classification result as the tampered image, and "·" representing the image to be classified with the classification result as the real image.
Illustratively, the t-SNE algorithm may be employed to map high-dimensional features to a two-dimensional plane. Wherein t-SNE is a machine learning algorithm for dimension reduction. In particular, t-SNE is a nonlinear dimension reduction algorithm, and is suitable for carrying out visualization on high-dimensional data by reducing dimensions to 2-dimension or 3-dimension.
S203, classifying the images to be classified based on the target real sample class central feature and the target tampering sample class central feature to obtain the classification result.
For example, referring to fig. 4a, an original real sample class classification center feature a and an original tampered sample class classification center feature B determine an original classification surface S, and if an image to be classified is classified according to the original real sample class classification center feature a and the original tampered sample class classification center feature B, an image to be classified X, which is originally a real image, is classified as a tampered image, and an image to be classified Y, which is originally a tampered image, is classified as a real image, so that the image to be classified X and the image to be classified Y are not accurately classified.
However, after the steps of the scheme are adopted, referring to fig. 4B, first, a first offset and a second offset are respectively determined according to the extracted image features of the image X to be classified, then, a target real sample class classification center feature a1 and a target tampered sample class classification center feature B1 are obtained based on the first offset and the second offset, and a target classification surface S1 is re-determined by the target real sample class classification center feature a1 and the target tampered sample class classification center feature B1, as can be seen from fig. 4B, after the original real sample class classification center feature a and the original tampered sample class classification center feature B are adjusted, the classification result of the image X to be classified is a real image, that is, the classification result at this time is accurate.
Similarly, referring to fig. 4c, first, a first offset and a second offset are respectively determined according to the extracted image features of the image Y to be classified, then, a target real sample class classification center feature a2 and a target tampered sample class classification center feature B2 are obtained based on the first offset and the second offset, and a target classification surface S2 is re-determined by the target real sample class classification center feature a2 and the target tampered sample class classification center feature B2, as can be seen from fig. 4c, after the original real sample class classification center feature a and the original tampered sample class classification center feature B are adjusted, the classification result of the image Y to be classified is a tampered image, that is, the classification result at this time is accurate, and the applicability of the neural network is improved.
Illustratively, as shown in fig. 5, in some embodiments, S203 for the above step may include the following S2031 to S2032:
s2031, comparing the extracted image features with the target real sample class central features and the target tampered sample class central features respectively to obtain a first difference comparison result between the image to be classified and the target real sample class central features and a second difference comparison result between the image to be classified and the target tampered sample class central features respectively.
S2032, classifying the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.
Exemplarily, referring to fig. 6, after an image (a) to be classified is input to a neural network, feature extraction is performed, and a target real sample class central feature and a target tampered sample class central feature can be obtained according to the extracted image features; then, comparing the images to be classified with the central features of the target real sample class and the central features of the target tampered sample class respectively to obtain a first comparison result and a second comparison result; and determining the classification result of the current image to be classified according to the comparison result.
For example, if the comparison difference result between the current image to be classified and the center feature of the target real sample class is smaller, it indicates that the probability that the current image to be classified is a real image is higher; on the contrary, if the comparison difference result between the current image to be classified and the target tampered sample class center feature is smaller, it is indicated that the probability that the current image to be classified is the tampered image is larger. Therefore, in some embodiments, the classification corresponding to the difference comparison result indicating the smallest difference in the first difference comparison result and the second difference comparison result may be determined as the classification result corresponding to the image to be classified.
In other embodiments, referring to fig. 7, regarding step S203, when classifying the image to be classified based on the target real sample class center feature and the target tampered sample class center feature to obtain a classification result, the following steps S203a to 203b may be included:
s203a, respectively calculating cosine similarities between the extracted image features and the target real sample class central features and between the extracted image features and the target tampered sample class central features, and respectively obtaining a first calculation result between the image to be classified and the real sample class central features and a second calculation result between the image to be classified and the target tampered sample class central features.
The cosine similarity, also called cosine similarity, is evaluated by calculating the cosine value of the included angle between two vectors. Cosine similarity maps vectors into a vector space, such as the most common two-dimensional space, according to coordinate values.
S203b determines the classification corresponding to the larger calculation result of the first calculation result and the second calculation result as the classification result corresponding to the image to be classified.
Wherein, the cosine similarity measures the similarity between two vectors by measuring the cosine value of the included angle of the two vectors. The cosine value of the 0-degree angle is 1, and the cosine value of any other angle is not more than 1; and its minimum value is-1. The cosine of the angle between the two vectors thus determines whether the two vectors point in approximately the same direction. When the two vectors have the same direction, the cosine similarity value is 1; when the included angle of the two vectors is 90 degrees, the value of the cosine similarity is 0; the cosine similarity has a value of-1 when the two vectors point in completely opposite directions. The result is independent of the length of the vector, only the pointing direction of the vector. Cosine similarity is commonly used in the positive space, and therefore gives values between-1 and 1.
In this embodiment, the larger the settlement result is, the larger the similarity between the image feature and the classification center feature is (the smaller the difference in indication is), and therefore, the classification corresponding to the larger calculation result of the first calculation result and the second calculation result is determined as the classification result corresponding to the image to be classified. For example, if the first calculation result is larger, determining that the current image to be classified is a real image; and if the second calculation result is larger, determining that the current image to be classified is a tampered image.
In some embodiments, the image to be classified comprises a test image; referring to fig. 8, a flowchart of another image classification method provided in an embodiment of the present disclosure is different from the method in fig. 1, and the method further includes the following steps S104 to S105:
and S104, acquiring a labeling result corresponding to the test image.
And S105, adjusting parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain the tested neural network.
Referring to fig. 6 again, it can be understood that, in the case that the images to be classified are test images, each test image has a corresponding labeling result, and the labeling result can be a real image or a tampered image. In order to enable the neural network to provide the detection accuracy while being suitable for the images tampered by different tampering modes, the parameters of the neural network can be adjusted according to the classification result obtained by the method and the labeling result of the test image. That is, after the test image is classified to obtain the classification result, the classification result and the labeling result can be compared, if the labeling result is a real image and the decomposition result is a real image, the current classification result is accurate, otherwise, the current classification result is inaccurate, and the parameters of the neural network need to be further adjusted until the classification result is consistent with the labeling result, so that the performance of the neural network is improved.
For example, the loss L may be determined according to the following formula, based on which parameters of the neural network may be adjusted.
L(x，y)＝-logP(Y＝y|x)
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same technical concept, an image classification device corresponding to the image classification method is further provided in the embodiment of the present disclosure, and as the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the image classification method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.
Referring to fig. 9, there is shown a schematic diagram of an image classification apparatus 500 according to an embodiment of the present disclosure, the image classification apparatus includes:
the extraction module 501 is configured to input the image to be classified into the trained neural network for feature extraction;
a calculating module 502, configured to obtain a target classification center feature corresponding to the image to be classified based on the extracted image feature and the original classification center feature; the original classification center features are obtained by training and learning of the neural network based on sample images;
and a classification module 503, configured to classify the image to be classified based on the target classification center feature to obtain a classification result.
In a possible implementation, the calculation module 502 is specifically configured to:
obtaining the offset of the target classification central feature relative to the original classification central feature based on the extracted image feature;
and obtaining the target classification central feature based on the offset and the original classification central feature.
In one possible implementation, the original classification center features include original real sample class center features and original tampered sample class center features; the target classification central features comprise target real sample class central features and target tampering sample class central features;
the classification module 503 is specifically configured to:
classifying the images to be classified based on the target real sample class central feature and the target tampering sample class central feature to obtain the classification result.
In a possible implementation, the calculation module 502 is specifically configured to:
respectively obtaining a first offset of the target real sample class central feature relative to the original real sample classification central feature and a second offset of the target tampered sample class central feature relative to the original tampered sample class central feature on the basis of the extracted image features;
and adding the first offset and the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset and the original tampered sample class center feature to obtain the target tampered sample class center feature.
In a possible implementation, the classification module 503 is specifically configured to:
respectively comparing the extracted image features with the target real sample class central features and the target tampering sample class central features to respectively obtain a first difference comparison result between the image to be classified and the target real sample class central features and a second difference comparison result between the image to be classified and the target tampering sample class central features;
and classifying the images to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.
In a possible implementation, the classification module 503 is specifically configured to:
and determining the classification corresponding to the difference comparison result with the minimum difference in the first difference comparison result and the second difference comparison result as the classification result corresponding to the image to be classified.
In one possible embodiment, referring to fig. 10, the image to be classified comprises a test image; the device further comprises:
an obtaining module 504, configured to obtain an annotation result corresponding to the test image;
and an adjusting module 505, configured to perform parameter adjustment on the trained neural network according to the obtained classification result and the labeling result of the test image, so as to obtain a tested neural network.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
Based on the same technical concept, the embodiment of the disclosure also provides an electronic device. Referring to fig. 11, a schematic structural diagram of an electronic device 700 provided in the embodiment of the present disclosure includes a processor 701, a memory 702, and a bus 703. The memory 702 is used for storing execution instructions and includes a memory 7021 and an external memory 7022; the memory 7021 is also referred to as an internal memory and temporarily stores operation data in the processor 701 and data exchanged with an external memory 7022 such as a hard disk, and the processor 701 exchanges data with the external memory 7022 via the memory 7021.
In this embodiment, the memory 702 is specifically configured to store application program codes for executing the scheme of the present application, and is controlled by the processor 701 to execute. That is, when the electronic device 700 is operated, the processor 701 and the memory 702 communicate with each other through the bus 703, so that the processor 701 executes the application program code stored in the memory 702, thereby executing the method described in any of the foregoing embodiments.
The Memory 702 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 701 may be an integrated circuit chip having signal processing capabilities. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It is to be understood that the illustrated structure of the embodiment of the present application does not specifically limit the electronic device 700. In other embodiments of the present application, the electronic device 700 may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the image classification method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product bears a program code, and instructions included in the program code may be used to execute steps of the image classification method in the foregoing method embodiments, which may be referred to specifically for the foregoing method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. An image classification method, comprising:
inputting the images to be classified into a trained neural network for feature extraction;
obtaining target classification central features corresponding to the images to be classified based on the extracted image features and original classification central features; the original classification center features are obtained by training and learning of the neural network based on sample images;
and classifying the images to be classified based on the target classification center characteristics to obtain a classification result.
2. The method according to claim 1, wherein the obtaining of the target classification center feature corresponding to the image to be classified based on the extracted image feature and the original classification center feature comprises:
obtaining the offset of the target classification central feature relative to the original classification central feature based on the extracted image feature;
and obtaining the target classification central feature based on the offset and the original classification central feature.
3. The method according to claim 1 or 2, wherein the original classification center features comprise original real sample class center features and original tampered sample class center features; the target classification central features comprise target real sample class central features and target tampering sample class central features;
classifying the image to be classified based on the target classification center features to obtain a classification result, wherein the classification result comprises the following steps:
classifying the images to be classified based on the target real sample class central feature and the target tampering sample class central feature to obtain the classification result.
4. The method according to claim 3, wherein the obtaining of the target classification center feature corresponding to the image to be classified based on the extracted image feature and the original classification center feature comprises:
respectively obtaining a first offset of the target real sample class central feature relative to the original real sample classification central feature and a second offset of the target tampered sample class central feature relative to the original tampered sample class central feature on the basis of the extracted image features;
and adding the first offset and the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset and the original tampered sample class center feature to obtain the target tampered sample class center feature.
5. The method according to claim 3 or 4, wherein the classifying the image to be classified based on the target classification center feature to obtain a classification result comprises:
respectively comparing the extracted image features with the target real sample class central features and the target tampering sample class central features to respectively obtain a first difference comparison result between the image to be classified and the target real sample class central features and a second difference comparison result between the image to be classified and the target tampering sample class central features;
and classifying the images to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.
6. The method according to claim 5, wherein the classifying the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result comprises:
and determining the classification corresponding to the difference comparison result with the minimum difference in the first difference comparison result and the second difference comparison result as the classification result corresponding to the image to be classified.
7. The method according to any one of claims 1-6, wherein the image to be classified comprises a test image, the method further comprising:
acquiring a labeling result corresponding to the test image;
and adjusting parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain the tested neural network.
8. An image classification apparatus, characterized in that the apparatus comprises:
the extraction module is used for inputting the images to be classified into the trained neural network for feature extraction;
the calculation module is used for obtaining target classification center features corresponding to the images to be classified based on the extracted image features and the original classification center features; the original classification center features are obtained by training and learning of the neural network based on sample images;
and the classification module is used for classifying the images to be classified based on the target classification center characteristics to obtain a classification result.
9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the image classification method of any of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the image classification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110346171.XA CN113065592A (en) | 2021-03-31 | 2021-03-31 | Image classification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110346171.XA CN113065592A (en) | 2021-03-31 | 2021-03-31 | Image classification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113065592A true CN113065592A (en) | 2021-07-02 |
Family
ID=76565140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110346171.XA Pending CN113065592A (en) | 2021-03-31 | 2021-03-31 | Image classification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113065592A (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150071528A1 (en) * | 2013-09-11 | 2015-03-12 | Digitalglobe, Inc. | Classification of land based on analysis of remotely-sensed earth images |
| CN105184309A (en) * | 2015-08-12 | 2015-12-23 | 西安电子科技大学 | Polarization SAR image classification based on CNN and SVM |
| WO2016171923A1 (en) * | 2015-04-21 | 2016-10-27 | Alibaba Group Holding Limited | Method and system for identifying a human or machine |
| CN106096561A (en) * | 2016-06-16 | 2016-11-09 | 重庆邮电大学 | Infrared pedestrian detection method based on image block degree of depth learning characteristic |
| US20170140300A1 (en) * | 2015-11-18 | 2017-05-18 | Honda Motor Co., Ltd. | Classification apparatus, robot, and classification method |
| CN108229341A (en) * | 2017-12-15 | 2018-06-29 | 北京市商汤科技开发有限公司 | Sorting technique and device, electronic equipment, computer storage media, program |
| CN108229267A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object properties detection, neural metwork training, method for detecting area and device |
| CN108960260A (en) * | 2018-07-12 | 2018-12-07 | 东软集团股份有限公司 | A kind of method of generating classification model, medical image image classification method and device |
| WO2019061661A1 (en) * | 2017-09-30 | 2019-04-04 | 平安科技（深圳）有限公司 | Image tamper detecting method, electronic device and readable storage medium |
| US20190108423A1 (en) * | 2017-10-06 | 2019-04-11 | Mitsubishi Electric Research Laboratories, Inc. | System and method for image comparison based on hyperplanes similarity |
| CN110197218A (en) * | 2019-05-24 | 2019-09-03 | 绍兴达道生涯教育信息咨询有限公司 | Thunderstorm gale grade forecast classification method based on multi-source convolutional neural networks |
| CN110210535A (en) * | 2019-05-21 | 2019-09-06 | 北京市商汤科技开发有限公司 | Neural network training method and device and image processing method and device |
| CN111368342A (en) * | 2020-03-13 | 2020-07-03 | 众安信息技术服务有限公司 | Image tampering identification model training method, image tampering identification method and device |
| CN112381775A (en) * | 2020-11-06 | 2021-02-19 | 厦门市美亚柏科信息股份有限公司 | Image tampering detection method, terminal device and storage medium |
-
2021
- 2021-03-31 CN CN202110346171.XA patent/CN113065592A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150071528A1 (en) * | 2013-09-11 | 2015-03-12 | Digitalglobe, Inc. | Classification of land based on analysis of remotely-sensed earth images |
| WO2016171923A1 (en) * | 2015-04-21 | 2016-10-27 | Alibaba Group Holding Limited | Method and system for identifying a human or machine |
| CN105184309A (en) * | 2015-08-12 | 2015-12-23 | 西安电子科技大学 | Polarization SAR image classification based on CNN and SVM |
| US20170140300A1 (en) * | 2015-11-18 | 2017-05-18 | Honda Motor Co., Ltd. | Classification apparatus, robot, and classification method |
| CN106096561A (en) * | 2016-06-16 | 2016-11-09 | 重庆邮电大学 | Infrared pedestrian detection method based on image block degree of depth learning characteristic |
| CN108229267A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object properties detection, neural metwork training, method for detecting area and device |
| WO2019061661A1 (en) * | 2017-09-30 | 2019-04-04 | 平安科技（深圳）有限公司 | Image tamper detecting method, electronic device and readable storage medium |
| US20190108423A1 (en) * | 2017-10-06 | 2019-04-11 | Mitsubishi Electric Research Laboratories, Inc. | System and method for image comparison based on hyperplanes similarity |
| CN108229341A (en) * | 2017-12-15 | 2018-06-29 | 北京市商汤科技开发有限公司 | Sorting technique and device, electronic equipment, computer storage media, program |
| CN108960260A (en) * | 2018-07-12 | 2018-12-07 | 东软集团股份有限公司 | A kind of method of generating classification model, medical image image classification method and device |
| CN110210535A (en) * | 2019-05-21 | 2019-09-06 | 北京市商汤科技开发有限公司 | Neural network training method and device and image processing method and device |
| CN110197218A (en) * | 2019-05-24 | 2019-09-03 | 绍兴达道生涯教育信息咨询有限公司 | Thunderstorm gale grade forecast classification method based on multi-source convolutional neural networks |
| CN111368342A (en) * | 2020-03-13 | 2020-07-03 | 众安信息技术服务有限公司 | Image tampering identification model training method, image tampering identification method and device |
| CN112381775A (en) * | 2020-11-06 | 2021-02-19 | 厦门市美亚柏科信息股份有限公司 | Image tampering detection method, terminal device and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| SUSUMU ITOH: "A machine learning approach to reducing image coding artifacts", 《2017 25TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO)》, 26 October 2017 (2017-10-26), pages 14851489 * |
| 祁亨年: "基于多类支持向量机的遥感图像分类及其半监督式改进策略", 《复旦学报(自然科学版)》, no. 05, 25 October 2004 (2004-10-25), pages 781 - 784 * |
| 钟辉: "基于卷积神经网络的图像拼接篡改检测算法", 《吉林大学学报(工学版)》, vol. 50, no. 04, 12 July 2019 (2019-07-12), pages 1428 - 1434 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109948397A (en) | A kind of face image correcting method, system and terminal device | |
| Yan et al. | Multi-scale difference map fusion for tamper localization using binary ranking hashing | |
| CN103839042A (en) | Human face recognition method and human face recognition system | |
| CN112257808A (en) | Integrated collaborative training method and device for zero sample classification and terminal equipment | |
| Subedi et al. | [Retracted] Feature Learning‐Based Generative Adversarial Network Data Augmentation for Class‐Based Few‐Shot Learning | |
| Chen et al. | Invariant leaf image recognition with histogram of Gaussian convolution vectors | |
| CN113011387A (en) | Network training and human face living body detection method, device, equipment and storage medium | |
| CN111275070B (en) | Signature verification method and device based on local feature matching | |
| CN112116592A (en) | Image detection method, training method, device and medium of image detection model | |
| CN115690803A (en) | Digital image recognition method and device, electronic equipment and readable storage medium | |
| Wyzykowski et al. | Multiresolution synthetic fingerprint generation | |
| Chi et al. | A novel local human visual perceptual texture description with key feature selection for texture classification | |
| CN113065592A (en) | Image classification method and device, electronic equipment and storage medium | |
| Soukup et al. | Robust object recognition under partial occlusions using NMF | |
| CN113822292B (en) | Vehicle characteristic information storage method and device, computer equipment and storage medium | |
| CN115439733A (en) | Image processing method, image processing device, terminal equipment and computer readable storage medium | |
| Sanin et al. | K-tangent spaces on Riemannian manifolds for improved pedestrian detection | |
| CN114925765A (en) | Construction method, device, equipment and storage medium of antagonism integrated classification model | |
| CN114067401A (en) | Target detection model training and identity verification method and device | |
| David et al. | Authentication of Vincent van Gogh’s work | |
| Kang et al. | Fast representation based on a double orientation histogram for local image descriptors | |
| Backes et al. | Texture classification using fractal dimension improved by local binary patterns | |
| Zheng et al. | Fast discriminative stochastic neighbor embedding analysis | |
| CN112766320A (en) | Classification model training method and computer equipment | |
| Yang et al. | A robust scheme for copy detection of 3D object point clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |