WO2020001196A1 - Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur - Google Patents

Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur Download PDF

Info

Publication number
WO2020001196A1
WO2020001196A1 PCT/CN2019/087570 CN2019087570W WO2020001196A1 WO 2020001196 A1 WO2020001196 A1 WO 2020001196A1 CN 2019087570 W CN2019087570 W CN 2019087570W WO 2020001196 A1 WO2020001196 A1 WO 2020001196A1
Authority
WO
WIPO (PCT)
Prior art keywords
vector
classification
center
distance
sample
Prior art date
Application number
PCT/CN2019/087570
Other languages
English (en)
Chinese (zh)
Inventor
陈岩
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2020001196A1 publication Critical patent/WO2020001196A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions

Definitions

  • the present application relates to the field of computer technology, and in particular, to an image method, an electronic device, and a computer-readable storage medium.
  • Deep learning algorithms require a large number of training images.
  • engineers formulate screening criteria and screen a large number of images to obtain training images based on the screening criteria.
  • an image processing method an electronic device, and a computer-readable storage medium are provided.
  • An image processing method includes:
  • Clustering the sample vector according to the number of classifications to obtain the clustering center and classification vector corresponding to each classification; level
  • the similarity between the clustering center and the classification vector in each classification is detected, and the training image corresponding to the classification vector whose similarity is greater than the first threshold is used as the first type of training image for the classification.
  • An electronic device includes a memory and a processor.
  • the memory stores a computer program.
  • the processor causes the processor to perform the following operations:
  • Clustering the sample vector according to the number of classifications to obtain a clustering center and a classification vector corresponding to each classification;
  • the similarity between the clustering center and the classification vector in each classification is detected, and the training image corresponding to the classification vector whose similarity is greater than the first threshold is used as the first type of training image for the classification.
  • a computer-readable storage medium stores a computer program thereon.
  • the computer program is executed by a processor, the following operations are implemented:
  • Clustering the sample vector according to the number of classifications to obtain a clustering center and a classification vector corresponding to each classification;
  • the similarity between the clustering center and the classification vector in each classification is detected, and the training image corresponding to the classification vector whose similarity is greater than the first threshold is used as the first type of training image for the classification.
  • the image processing method, electronic device, and computer-readable storage medium provided in the embodiments of the present application can filter training images of each category according to the sample vectors output by the activation layer of the neural network, which can improve the filtering efficiency of the training image.
  • FIG. 1 is a flowchart of an image processing method in one or more embodiments.
  • FIG. 2 is a flowchart of performing clustering processing on sample vectors in one or more embodiments.
  • FIG. 3 is a flowchart of performing clustering processing on sample vectors in another or more embodiments.
  • FIG. 4 is a schematic diagram of a clustering process of sample vectors in one or more embodiments.
  • FIG. 5 is a flowchart of an image processing method in one or more embodiments.
  • FIG. 6 is a flowchart of an image processing method in another embodiment.
  • FIG. 7 is a structural block diagram of an image processing apparatus in one or more embodiments.
  • FIG. 8 is a schematic diagram of an internal structure of an electronic device in one or more embodiments.
  • FIG. 9 is a schematic diagram of an image processing circuit in one or more embodiments.
  • first”, “second”, and the like used in this application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.
  • the first image may be referred to as a second image, and similarly, the second image may be referred to as a first image. Both the first image and the second image are images, but they are not the same image.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment. As shown in FIG. 1, an image processing method includes operations 102 to 106. among them:
  • a training image is input to a neural network, and a sample vector output by an activation layer of the neural network is obtained.
  • the training image may be an image stored locally on the electronic device, or an image downloaded from the network by the electronic device.
  • a large number of training images are needed for training.
  • Neural network refers to a computing model composed of a large number of nodes (neurons) connected to each other.
  • the neural network may be CNN (Convolutional Neural Network, Convolutional Neural Network), DNN (Deep Neural Network, Deep Neural Network), RNN (Recurrent Neural Network, Recurrent Neural Network), etc., and is not limited thereto.
  • a neural network generally includes an input layer, a hidden layer, and an output layer; the input layer is used to receive the input of the image; the hidden layer is used to process the received image; the output layer is used to output the processing result of the image processing.
  • the hidden layers of the neural network may include a convolutional layer, an activation layer, a pooling layer, and a fully connected layer.
  • the sample vector refers to a vector composed of feature values in a feature map output by a neural network activation layer according to a preset rule after a training image is input into a neural network.
  • the electronic device can input the training image into the neural network, and the electronic device can obtain a sample vector composed of the feature values in the feature map according to the rules according to the feature map output by the activation layer in the neural network.
  • Operation 104 Perform clustering processing on the sample vector according to the number of classifications to obtain a clustering center and a classification vector corresponding to each classification.
  • the number of classifications refers to the number of scene classifications that the trained neural network can use for recognition.
  • the number of classifications can also refer to the number of scene classifications in the training image.
  • the classification can be landscape, beach, blue sky, green grass, snow, night, dark, backlight, sunrise / sunset, fireworks, spotlight, indoor, macro, text document, portrait, baby, cat, dog , Food, etc. are not limited to this.
  • Clustering refers to the process by which training images are divided into multiple classifications composed of similar scene classifications.
  • the electronic device can use a partitioning method such as K-MEANS (hard clustering) algorithm or K-MEDOIDS (center point) algorithm, and a hierarchical method such as BIRCH (Balanced Iterative Reducing and Clustering Using Hierarchies).
  • K-MEANS hard clustering
  • K-MEDOIDS center point
  • BIRCH Bit Iterative Reducing and Clustering Using Hierarchies
  • clustering algorithm and other algorithms, such as graph theory clustering method, perform clustering processing on training images.
  • the cluster center refers to the center vector of each classification, and the distance between the cluster center in the classification and the classification vector in the classification is the smallest.
  • a classification vector refers to a sample vector corresponding to each classification. For example, there are sample vectors A, B, C, and D. If the classification is M and N, the electronic device performs cluster processing on the sample vectors. Sample vectors A and B form classification M, and sample vectors C and D form classification N. , The classification vectors corresponding to
  • the electronic device performs cluster processing on the sample vector according to the required number of classifications, and can obtain a classification vector and a clustering center corresponding to each classification.
  • Operation 106 Detect the similarity between the clustering center and the classification vector in each classification, and use the training image corresponding to the classification vector whose similarity is greater than the first threshold as the first type of training image for classification.
  • the electronic device may determine the similarity between the cluster center and the classification vector by calculating the distance between the cluster center and the classification vector; the greater the distance, the more similar the classification vector is to the cluster center of the classification. The smaller the distance, the greater the similarity between the classification vector and the clustering center of the classification.
  • the electronic device can set corresponding similarity for different distance values in advance, obtain the distance value between the cluster center and the classification vector through detection, and obtain the corresponding similarity according to the distance value as the similarity between the cluster center and the classification vector.
  • the first threshold can be set according to the requirements of the actual application.
  • the first threshold may be 70%, 80%, 90%, etc., and is not limited thereto.
  • the first type of training image refers to a training image with the same classification that can be used to train a neural network that can recognize the classification, that is, the first type of training image can be used as a positive sample in neural network training.
  • the electronic device can obtain the clustering center of each classification and the classification vector in the classification, detect the similarity between each classification vector in the classification and the clustering center of the classification, and obtain the training image corresponding to the classification vector whose similarity is greater than the first threshold. This training image is used as the first type of training image in this classification.
  • the present application by inputting a training image to a neural network, obtaining a sample vector output from an activation layer of the neural network, and performing cluster processing on the sample vector according to the number of classifications, to obtain a clustering center and a classification vector corresponding to each classification. Detecting the similarity between the clustering center and the classification vector in each classification, and using the training image corresponding to the classification vector whose similarity is greater than the first threshold value as the first type of training image for classification can improve the efficiency of filtering the training image.
  • the process of obtaining a sample vector output by the activation layer of the neural network in the provided image processing method includes: obtaining a sample vector output by the penultimate activation layer in the neural network.
  • the activation layer of the neural network is a layer for performing a function change on the feature map obtained through the convolution layer according to the activation function.
  • the electronic device can obtain a sample vector of the output of the activation layer in the neural network.
  • the electronic device can obtain a sample vector output by the penultimate activation layer in the neural network. For example, when the training image E is input into the neural network, the output of each activation layer in the neural network is out (1), out (2), ..., out (k), and k is the number of activation layers in the neural network. Then, the electronic device can obtain the vector output by the penultimate activation layer in the neural network, that is, out (k-1), as the sample vector corresponding to the training image E.
  • the electronic device obtains the sample vector output from the penultimate activation layer in the neural network, which can increase the dimension of the sample vector and facilitate the differentiation of different training images.
  • the sample vector is clustered according to the number of classifications to obtain the clustering center corresponding to each classification.
  • classification vector detecting the similarity between the clustering center and the classification vector in each classification, and using the training image corresponding to the classification vector whose similarity is greater than the first threshold as the first type of training image for classification, can improve the efficiency of training image screening.
  • a process of performing clustering processing on a sample vector according to the number of classifications in the provided image processing method includes operations 202 to 206. among them:
  • a classification number of center vectors are configured according to the classification number.
  • the electronic device configures the number of classification center vectors according to the number of classifications, and each classification corresponds to a center vector. Specifically, the electronic device may randomly select the classified number of center vectors according to the classified number. In one embodiment, the electronic device may also obtain a preset number of classified center vectors.
  • Operation 204 Adjust each center vector according to the distance between the sample vector and each center vector.
  • the distance between the sample vector and each center vector can be detected using the distance formula.
  • the distance formula may be a Manhattan distance formula, a Euclidean distance formula, a relative entropy formula, and the like, and is not limited thereto.
  • the electronic device can detect the distance between the sample vector and each center vector according to the distance formula, use the classification corresponding to the center vector with the smallest sample vector distance as the classification of the sample vector, and center the center according to the distance between each sample vector and the center vector in the classification. The vector is adjusted.
  • Operation 206 Obtain the adjusted center vector as the classified cluster center.
  • the electronic device obtains the adjusted center vector as the clustering center of the classification, and among the sample vectors, the sample vector with the smallest distance from the clustering center of the classification is the classification vector in the classification.
  • the electronic device configures the number of classified center vectors according to the number of classifications, adjusts each center vector according to the distance between the sample vector and each center vector, and obtains the adjusted center vector as the clustering center of the classification, and can obtain the clustering center corresponding to each classification.
  • the process of adjusting each center vector according to the distance between the sample vector and each center vector includes operations 302 to 310. among them:
  • Operation 302 Configure a classified number of center vectors according to the classified number.
  • the classification corresponding to the center vector with the smallest distance between the sample vectors is used as the classification of the sample vectors.
  • the electronic device After the electronic device detects the distance between the sample vector and each center vector, and classifies the sample vector with the smallest distance from the center vector into a classification, the distance between all the sample vectors and the center vector in the classification is the lowest.
  • Operation 306 Adjust the center vector according to the distance between each sample vector and the center vector in the classification.
  • the electronic device adjusts the center vector so that the distance between all the sample vectors and the center vector in the classification is the smallest.
  • the electronic device may use a square error to construct an objective function.
  • the objective function is the sum of the squared differences of the distances of all sample vectors in the classification to the clustering centers of the classification. The distance of the vector is the smallest, so the objective function needs to be smaller.
  • the electronic device can obtain the partial derivative of the objective function to obtain the update function of the center vector, and adjust the center vector according to the update function and the distance between each sample vector and the center vector in the classification.
  • Operation 308 Repeat the classification of the center vector with the smallest distance from the sample vector as the sample vector according to the adjusted center vector, and adjust the center vector according to the distance between each vector and the center vector in the classification.
  • the adjusted center vector changes, so the distance between each sample vector and the adjusted center vector also changes.
  • the electronic device uses the adjusted center vector as the new center vector of each classification, re-detects the distance between each sample vector and the center vector, and uses the classification corresponding to the center vector with the smallest sample vector distance as the new classification of the sample vector, and Adjust the center vector according to the distance between each sample vector and the center vector in the classification.
  • a final center vector is obtained as a clustering center for classification.
  • the preset number of times can be set according to the requirements of the actual application, which is not limited here.
  • the electronic device can obtain the center vector obtained as the cluster center of each classification when the number of adjustments of the center vector exceeds a preset number of times.
  • the sample vector with the smallest distance from the cluster center is the classification in the classification. vector.
  • the electronic device may also obtain the adjusted center vector as the clustering center when the distance between the adjusted center vector and the last adjusted center vector is smaller than the first distance value.
  • the electronic device may also obtain the center vector as the clustering center of the classification when the distance between all the sample vectors and the center vector in the classification is smaller than the second distance value.
  • FIG. 4 is a schematic diagram of a clustering process of sample vectors in an embodiment.
  • the vectors F, G, H, I, and J are sample vectors corresponding to the training images F, G, H, I, and J, respectively; as shown in (b) of FIG. 4,
  • the electronic device can randomly configure two center vectors X and Y; the electronic device can detect the distance between each sample vector F, G, H, I, J and the center vector X and Y, as shown in FIG.
  • the distance between the sample vector F, G, H and the center vector X is less than the distance from the center vector Y, and the distance between the sample vector I, J and the center vector X is greater than the distance from the center vector Y, then The sample vectors F, G, and H are classified into the classification corresponding to the center vector X, and the sample vectors I and J are classified into the classification corresponding to the center vector Y.
  • the electronic device After the classification is completed, the electronic device The vectors F, G, and H adjust the center vector X, and the adjusted center vector is X1.
  • the center vector is adjusted according to the sample vectors I and J, and the adjusted center vector is Y1.
  • the electronic device needs to re-test the sample.
  • the distances between the vectors F, G, H, I, J and the center vectors X1, Y1 are shown in (e) in Figure 4.
  • the sample vectors H and The distance of the heart vector Y1 is smaller than the distance from the center vector X1, and the sample vector H is re-classified into the classification corresponding to the center vector Y1.
  • the electronic device can continue to repeatedly calculate (c) and FIG. 4 in FIG. 4
  • the process of (d) in 4 adjusts the center vector until the number of adjustments exceeds a preset number.
  • the center vectors Xn and Yn are the center vectors obtained by adjusting n times. When n is greater than a preset number, the electronic device can obtain Xn and Yn as the clustering centers of classification, respectively.
  • the electronic device adjusts the center vector according to the classification corresponding to the center vector with the smallest distance from the sample vector as the sample vector, adjusts the center vector according to the distance between each sample vector and the center vector in the classification, and readjusts the center vector according to the adjusted center vector.
  • the number of adjustments exceeds a preset number, the final center vector is obtained as the clustering center of the classification, and the clustering center corresponding to each classification can be obtained.
  • the process of adjusting each center vector according to the distance between the sample vector and each center vector includes operations 502 to 504. among them:
  • the electronic device can determine the similarity between the cluster center and the classification vector by calculating the distance between the cluster center and the classification vector; the larger the distance, the smaller the similarity between the classification vector and the cluster center of the classification, and the smaller the distance, The greater the similarity between the classification vector and the clustering center of the classification; the electronic device can set the similarity for different distance values in advance.
  • the training image corresponding to the sample vector whose similarity between the cluster centers is less than the second threshold is used as the classified second type training image.
  • the second threshold may be set according to the requirements of the actual application, and may be, for example, 10%, 20%, 30%, and the like, without being limited thereto.
  • the electronic device detects the similarity between each classification vector in the classification and the clustering center of the classification, obtains a training image corresponding to the classification vector whose similarity is less than the second threshold, and uses the training image as the second type of training image in the classification.
  • the second type of training image may be a training image used to reduce the error rate of the neural network that can be trained to recognize the classification, that is, the second type of training image may be used as a negative sample in the training of the neural network.
  • the electronic device detects the similarity between the cluster center of each classification and the sample vector, and uses the training image corresponding to the sample vector whose similarity to the cluster center is less than the second threshold as the second type of training image for classification, which can improve the training image screening efficiency.
  • the process of adjusting each center vector according to the distance between the sample vector and each center vector includes operations 602 to 604. among them:
  • Operation 602 Detect the distance between the cluster center and the classification vector in each classification.
  • the distance calculation formula may be a European-style distance calculation formula, a standard European-style distance calculation formula, a Manhattan distance calculation formula, a cosine distance calculation formula, and the like are not limited thereto.
  • the electronic device can detect the distance between the cluster center and the classification vector in each classification according to the distance calculation formula. The larger the distance, the smaller the similarity between the classification vector and the cluster center of the classification, and the smaller the distance, the greater the similarity between the classification vector and the cluster center of the classification.
  • the process of detecting the distance between the clustering center and the classification vector in each classification in the provided image processing method further includes: detecting a distance between the clustering center and the classification vector in each classification by using a European distance calculation formula.
  • the electronic device can obtain the European distance calculation formula as Where d ij represents the distance between the vector i and the vector j. x ik represents the k-th eigenvalue in the vector i, x jk represents the k-th eigenvalue in the vector j, and n represents the number of eigenvalues in the vector.
  • the electronic device can obtain the clustering center in the classification and the classification vector of the classification, and use the European-style distance calculation formula to substitute the clustering center and each feature value in the classification vector into the distance calculation formula to obtain the clustering center and the classification vector distance.
  • Operation 604 Use the training image corresponding to the classification vector whose distance is less than the preset distance as the first type of training image in the classification.
  • the preset distance can be set according to actual application requirements, which is not limited here.
  • the electronic device uses the clustering center and the classification vector in each classification to use the training image corresponding to the classification vector whose distance is less than the preset distance as the first type of training image in the classification.
  • the electronic device can obtain the correspondence of each classification from a large number of training images.
  • the first type of training images can improve the filtering efficiency of training images.
  • an image processing method is provided, and specific operations for implementing the method are as follows:
  • Neural networks are convolutional neural networks, deep neural networks, recurrent neural networks, and so on.
  • a neural network generally includes an input layer, a hidden layer, and an output layer; the input layer is used to receive the input of the image; the hidden layer is used to process the received image; the output layer is used to output the processing result of the image processing.
  • the hidden layers of the neural network may include a convolutional layer, an activation layer, a pooling layer, and a fully connected layer.
  • the electronic device can input the training image into the neural network, and the electronic device can obtain a sample vector composed of the feature values in the feature map according to the rules according to the feature map output by the activation layer in the neural network.
  • the electronic device obtains a sample vector output by the penultimate activation layer in the neural network.
  • the output of each activation layer in the neural network is out (1), out (2), ..., out (k), and k is the number of activation layers in the neural network.
  • the electronic device can obtain the vector output by the penultimate activation layer in the neural network, that is, out (k-1), as the sample vector corresponding to the training image E.
  • the electronic device obtains the sample vector output by the penultimate activation layer in the neural network, which can improve the dimension of the sample vector and facilitate the discrimination of different training images.
  • the electronic device performs cluster processing on the sample vector according to the number of classifications to obtain a clustering center and a classification vector corresponding to each classification.
  • the number of classifications refers to the number of scene classifications that the trained neural network can use for recognition.
  • Scene classification can be scenery, beach, blue sky, green grass, snow, night, dark, backlight, sunrise / sunset, fireworks, spotlight, indoor, macro, text document, portrait, baby, cat, dog, food, etc. this.
  • the electronic device performs cluster processing on the sample vector according to the required number of classifications, and can obtain a classification vector and a clustering center corresponding to each classification.
  • the electronic device configures the number of classified center vectors according to the number of classifications, adjusts each center vector according to the distance between the sample vector and each center vector, and obtains the adjusted center vector as the clustering center for classification.
  • the electronic device configures the number of classification center vectors according to the number of classifications, and each classification corresponds to a center vector.
  • the electronic device may randomly select the classified number of center vectors according to the classified number.
  • the distance between the sample vector and each center vector can be detected using the distance formula.
  • the distance formula may be a Manhattan distance formula, a Euclidean distance formula, a relative entropy formula, and the like, and is not limited thereto.
  • the electronic device obtains the adjusted center vector as the clustering center of the classification, and among the sample vectors, the sample vector with the smallest distance from the clustering center of the classification is the classification vector in the classification.
  • the electronic device configures the number of classification center vectors according to the number of classifications; the classification corresponding to the center vector with the smallest distance from the sample vector is used as the classification of the sample vector; and the center vector is adjusted according to the distance between each sample vector and the center vector in the classification; According to the adjusted center vector, the classification with the center vector having the smallest distance from the sample vector as the sample vector is repeatedly performed, and the operation of adjusting the center vector according to the distance between each vector and the center vector in the classification is performed; when the number of adjustments exceeds a preset number of times, Obtain the final center vector as the clustering center for classification.
  • the electronic device may also obtain the adjusted center vector as the clustering center when the distance between the adjusted center vector and the last adjusted center vector is smaller than the first distance value. In one embodiment, the electronic device may also obtain the center vector as the clustering center of the classification when the distance between all the sample vectors and the center vector in the classification is smaller than the second distance value.
  • the electronic device detects the similarity between the clustering center and the classification vector in each classification, and uses the training image corresponding to the classification vector whose similarity is greater than the first threshold as the first type of training image for classification.
  • the electronic device can obtain the clustering center of each classification and the classification vector in the classification, detect the similarity between each classification vector in the classification and the clustering center of the classification, and obtain the training image corresponding to the classification vector whose similarity is greater than the first threshold. This training image is used as the first type of training image in this classification.
  • the electronic device detects the distance between the cluster center and the classification vector in each classification, and uses the training image corresponding to the classification vector whose distance is less than a preset distance as the first type of training image in the classification.
  • the distance calculation formula may be a European-style distance calculation formula, a standard European-style distance calculation formula, a Manhattan distance calculation formula, a cosine distance calculation formula, and the like are not limited thereto.
  • the preset distance can be set according to actual application requirements.
  • the electronic device uses the clustering center and the classification vector in each classification to use the training image corresponding to the classification vector whose distance is less than the preset distance as the first type of training image in the classification.
  • the electronic device detects the similarity between the cluster center of each classification and the sample vector, and uses the training image corresponding to the sample vector whose similarity to the cluster center is less than the second threshold as the second type of training image for classification.
  • the second threshold can be set according to the requirements of the actual application.
  • the electronic device detects the similarity between each classification vector in the classification and the clustering center of the classification, obtains a training image corresponding to the classification vector whose similarity is less than the second threshold, and uses the training image as the second type of training image in the classification.
  • the electronic device uses a European-style distance calculation formula to detect the distance between the cluster center and the classification vector in each classification.
  • the electronic device can obtain the Euclidean distance calculation formula, and obtain the cluster center in the classification and the classification vector of the classification, and use the Euclidean distance calculation formula to substitute the cluster center and each feature value in the classification vector into the distance calculation formula to obtain The distance between the cluster center and the classification vector.
  • FIG. 7 is a structural block diagram of an image processing apparatus according to an embodiment.
  • an image processing apparatus includes a vector acquisition module 720, a cluster processing module 740, and an image determination module 760. among them:
  • a vector acquisition module 720 is configured to input a training image to a neural network, and obtain a sample vector output from an activation layer of the neural network.
  • a clustering processing module 740 is configured to perform clustering processing on the sample vector according to the number of classifications to obtain a clustering center and a classification vector corresponding to each classification.
  • the image determination module 760 is configured to detect the similarity between the cluster center and the classification vector in each classification, and use the training image corresponding to the classification vector whose similarity is greater than the first threshold as the first type of training image for classification.
  • the vector acquisition module 720 may be further configured to acquire a sample vector output by the penultimate activation layer in the neural network.
  • the cluster processing module 740 may be further configured to configure the number of classification center vectors according to the number of classifications, adjust each center vector according to the distance between the sample vector and each center vector, and obtain the adjusted center vector as the classified cluster. center.
  • the distance processing module 740 may be further configured to use the classification corresponding to the center vector with the smallest distance to the sample vector as the classification of the sample vector, adjust the center vector according to the distance between each sample vector and the center vector in the classification, and adjust according to the adjustment.
  • the subsequent center vector repeatedly performs the classification using the center vector with the smallest distance from the sample vector as the sample vector, and adjusts the center vector according to the distance between each sample vector and the center vector in the classification. When the number of adjustments exceeds a preset number, the operation is obtained. The final center vector is used as the classification cluster center.
  • the image determination module 760 may be further configured to detect the similarity between the cluster center of each classification and the sample vector, and use the training image corresponding to the sample vector whose similarity with the cluster center is less than the second threshold as the classification.
  • the second type of training images may be further configured to detect the similarity between the cluster center of each classification and the sample vector, and use the training image corresponding to the sample vector whose similarity with the cluster center is less than the second threshold as the classification. The second type of training images.
  • the image determination module 760 may be further configured to detect the distance between the cluster center and the classification vector in each classification, and use the training image corresponding to the classification vector whose distance is less than a preset distance as the first type of training image in the classification.
  • the image determination module 760 may be further configured to detect the distance between the cluster center and the classification vector in each classification by using a European distance calculation formula.
  • the image processing apparatus obtains a sample vector output by an activation layer of a neural network by inputting a training image to a neural network, and performs cluster processing on the sample vector according to the number of classifications to obtain a clustering center and classification corresponding to each classification.
  • Vector to detect the similarity between the clustering center and the classification vector in each classification, and use the training image corresponding to the classification vector whose similarity is greater than the first threshold as the first type of training image for classification, which can improve the filtering efficiency of the training image.
  • each module in the above image processing apparatus is for illustration only. In other embodiments, the image processing apparatus may be divided into different modules as needed to complete all or part of the functions of the above image processing apparatus.
  • Each module in the image processing apparatus may be implemented in whole or in part by software, hardware, and a combination thereof.
  • the above-mentioned modules may be embedded in the hardware in or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • FIG. 8 is a schematic diagram of the internal structure of the electronic device in one embodiment.
  • the electronic device includes a processor, a memory, and a network interface connected through a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the memory is used to store data, programs, and the like. At least one computer program is stored on the memory, and the computer program can be executed by a processor to implement the image processing method applicable to the electronic device provided in the embodiments of the present application.
  • the memory may include a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the computer program can be executed by a processor to implement an image processing method provided by each of the following embodiments.
  • the internal memory provides a cached operating environment for operating system computer programs in a non-volatile storage medium.
  • the network interface may be an Ethernet card or a wireless network card, and is used to communicate with external electronic devices.
  • the electronic device may be a mobile phone, a computer, a tablet computer, or a personal digital assistant or a wearable device.
  • each module in the image processing apparatus provided in the embodiments of the present application may be in the form of a computer program.
  • the computer program can be run on a terminal or a server.
  • the program module constituted by the computer program can be stored in the memory of the terminal or server.
  • the computer program is executed by a processor, the operations of the method described in the embodiments of the present application are implemented.
  • An embodiment of the present application further provides a computer-readable storage medium.
  • One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image processing method.
  • a computer program product containing instructions that, when run on a computer, causes the computer to perform an image processing method.
  • An embodiment of the present application further provides an electronic device.
  • the above electronic device includes an image processing circuit, and the image processing circuit may be implemented by using hardware and / or software components, and may include various processing units defining an ISP (Image Signal Processing) pipeline.
  • FIG. 9 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 9, for ease of description, only aspects of the image processing technology related to the embodiments of the present application are shown.
  • the image processing circuit includes an ISP processor 940 and a control logic 950.
  • the image data captured by the imaging device 910 is first processed by the ISP processor 940, which analyzes the image data to capture image statistical information that can be used to determine and / or one or more control parameters of the imaging device 910.
  • the imaging device 910 may include a camera having one or more lenses 912 and an image sensor 914.
  • the image sensor 914 may include a color filter array (such as a Bayer filter).
  • the image sensor 914 may obtain light intensity and wavelength information captured with each imaging pixel of the image sensor 914, and provide a set of raw data that may be processed by the ISP processor 940. Image data.
  • the sensor 920 may provide parameters (such as image stabilization parameters) of the acquired image processing to the ISP processor 940 based on the interface type of the sensor 920.
  • the sensor 920 interface may use a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the foregoing interfaces.
  • SMIA Standard Mobile Imaging Architecture
  • the image sensor 914 may also send the original image data to the sensor 920, and the sensor 920 may provide the original image data to the ISP processor 940 based on the interface type of the sensor 920, or the sensor 920 stores the original image data in the image memory 930.
  • the ISP processor 940 processes the original image data pixel by pixel in a variety of formats.
  • each image pixel may have a bit depth of 9, 10, 12, or 14 bits, and the ISP processor 940 may perform one or more image processing operations on the original image data and collect statistical information about the image data.
  • the image processing operations may be performed with the same or different bit depth accuracy.
  • the ISP processor 940 may also receive image data from the image memory 930.
  • the sensor 920 interface sends the original image data to the image memory 930, and the original image data in the image memory 930 is then provided to the ISP processor 940 for processing.
  • the image memory 930 may be a part of a memory device, a storage device, or a separate dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.
  • DMA Direct Memory Access
  • the ISP processor 940 may perform one or more image processing operations, such as time-domain filtering.
  • the processed image data may be sent to the image memory 930 for further processing before being displayed.
  • the ISP processor 940 receives processing data from the image memory 930 and performs image data processing on the processing data in the original domain and in the RGB and YCbCr color spaces.
  • the image data processed by the ISP processor 940 may be output to the display 970 for viewing by the user and / or further processed by a graphics engine or a GPU (Graphics Processing Unit).
  • the output of the ISP processor 940 can also be sent to the image memory 930, and the display 970 can read image data from the image memory 930.
  • the image memory 930 may be configured to implement one or more frame buffers.
  • the output of the ISP processor 940 may be sent to an encoder / decoder 960 to encode / decode image data.
  • the encoded image data can be saved and decompressed before being displayed on the display 970 device.
  • the encoder / decoder 960 may be implemented by a CPU or a GPU or a coprocessor.
  • the statistical data determined by the ISP processor 940 may be sent to the control logic 950 unit.
  • the statistical data may include image information of the image sensor 914 such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, and lens 912 shading correction.
  • the control logic 950 may include a processor and / or a microcontroller that executes one or more routines (such as firmware). The one or more routines may determine the control parameters of the imaging device 910 and the ISP processing according to the received statistical data. Parameters of the controller 940.
  • control parameters of the imaging device 910 may include sensor 920 control parameters (such as gain, integration time for exposure control, image stabilization parameters, etc.), camera flash control parameters, lens 912 control parameters (such as focus distance for focusing or zooming), or these A combination of parameters.
  • ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 912 shading correction parameters.
  • the image processing method in FIG. 9 can be used to implement the foregoing image processing method.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM), which is used as external cache memory.
  • RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR, SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR dual data rate SDRAM
  • SDRAM enhanced SDRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé de traitement d'images, comprenant les étapes suivantes : entrer des images d'apprentissage dans un réseau neuronal et obtenir des vecteurs d'échantillon émises par une couche d'activation du réseau neuronal ; regrouper les vecteurs d'échantillon en fonction du nombre de classifications, et obtenir des centres de regroupement et des vecteurs de classification correspondant aux classifications ; détecter la similarité entre les centres de regroupement et les vecteurs de classification dans les classifications, et utiliser les images d'apprentissage correspondant aux vecteurs de classification dont la similarité est supérieure à une première valeur de seuil comme première classe d'images d'apprentissage des classifications.
PCT/CN2019/087570 2018-06-28 2019-05-20 Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur WO2020001196A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810686074.3 2018-06-28
CN201810686074.3A CN109002843A (zh) 2018-06-28 2018-06-28 图像处理方法和装置、电子设备、计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2020001196A1 true WO2020001196A1 (fr) 2020-01-02

Family

ID=64600760

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/087570 WO2020001196A1 (fr) 2018-06-28 2019-05-20 Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN109002843A (fr)
WO (1) WO2020001196A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112668635A (zh) * 2020-12-25 2021-04-16 浙江大华技术股份有限公司 图像归档方法、装置、设备及计算机存储介质
CN114092920A (zh) * 2022-01-18 2022-02-25 腾讯科技（深圳）有限公司 一种模型训练的方法、图像分类的方法、装置及存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109002843A (zh) * 2018-06-28 2018-12-14 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN109948734B (zh) * 2019-04-02 2022-03-29 北京旷视科技有限公司 图像聚类方法、装置及电子设备
CN110210535B (zh) * 2019-05-21 2021-09-10 北京市商汤科技开发有限公司 神经网络训练方法及装置以及图像处理方法及装置
CN111445304A (zh) * 2020-02-26 2020-07-24 平安科技（深圳）有限公司 信息推荐方法、装置、计算机设备及存储介质
CN111444366B (zh) * 2020-04-10 2024-02-20 Oppo广东移动通信有限公司 图像分类方法、装置、存储介质及电子设备
CN111598012B (zh) * 2020-05-19 2021-11-12 恒睿(重庆)人工智能技术研究院有限公司 一种图片聚类管理方法、系统、设备及介质
CN113408670B (zh) * 2021-08-17 2021-12-10 深圳电通信息技术有限公司 图像分类方法和装置
CN113920373A (zh) * 2021-10-29 2022-01-11 平安银行股份有限公司 一种对象分类方法、装置、终端设备及存储介质
CN114494797A (zh) * 2022-01-27 2022-05-13 北京百度网讯科技有限公司 用于训练图像检测模型的方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104992191A (zh) * 2015-07-23 2015-10-21 厦门大学 基于深度学习的特征和最大置信路径的图像分类方法
WO2017055878A1 (fr) * 2015-10-02 2017-04-06 Tractable Ltd. Étiquetage semi-automatique d'ensembles de données
CN107122712A (zh) * 2017-03-27 2017-09-01 大连大学 基于卷积神经网络和双向局部特征聚合描述向量的掌纹图像识别方法
CN107609541A (zh) * 2017-10-17 2018-01-19 哈尔滨理工大学 一种基于可变形卷积神经网络的人体姿态估计方法
CN108121781A (zh) * 2017-12-18 2018-06-05 辽宁师范大学 基于高效样本选取与参数优化的相关反馈图像检索方法
CN109002843A (zh) * 2018-06-28 2018-12-14 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102494987A (zh) * 2011-12-13 2012-06-13 天津卓朗科技发展有限公司 球墨铸铁中显微粒子的自动分类评级方法
CN103093235B (zh) * 2012-12-30 2016-01-20 北京工业大学 一种基于改进距离核主成分分析的手写体数字识别方法
CN103473275A (zh) * 2013-08-23 2013-12-25 中山大学 一种采用多特征融合的图像自动标注方法和系统
CN104765768B (zh) * 2015-03-09 2018-11-02 深圳云天励飞技术有限公司 海量人脸库的快速准确检索方法
CN107562742B (zh) * 2016-06-30 2021-02-05 江苏苏宁云计算有限公司 一种图像数据处理方法及装置
CN106203525A (zh) * 2016-07-18 2016-12-07 戎巍 电子设备及其应用的图像处理方法及系统
CN107729928B (zh) * 2017-09-30 2021-10-22 百度在线网络技术（北京）有限公司 信息获取方法和装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104992191A (zh) * 2015-07-23 2015-10-21 厦门大学 基于深度学习的特征和最大置信路径的图像分类方法
WO2017055878A1 (fr) * 2015-10-02 2017-04-06 Tractable Ltd. Étiquetage semi-automatique d'ensembles de données
CN107122712A (zh) * 2017-03-27 2017-09-01 大连大学 基于卷积神经网络和双向局部特征聚合描述向量的掌纹图像识别方法
CN107609541A (zh) * 2017-10-17 2018-01-19 哈尔滨理工大学 一种基于可变形卷积神经网络的人体姿态估计方法
CN108121781A (zh) * 2017-12-18 2018-06-05 辽宁师范大学 基于高效样本选取与参数优化的相关反馈图像检索方法
CN109002843A (zh) * 2018-06-28 2018-12-14 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112668635A (zh) * 2020-12-25 2021-04-16 浙江大华技术股份有限公司 图像归档方法、装置、设备及计算机存储介质
CN114092920A (zh) * 2022-01-18 2022-02-25 腾讯科技（深圳）有限公司 一种模型训练的方法、图像分类的方法、装置及存储介质

Also Published As

Publication number Publication date
CN109002843A (zh) 2018-12-14

Similar Documents

Publication Publication Date Title
WO2020001196A1 (fr) Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur
WO2019233297A1 (fr) Procédé de construction d'un ensemble de données, terminal mobile et support de stockage lisible
US11138478B2 (en) Method and apparatus for training, classification model, mobile terminal, and readable storage medium
WO2019233263A1 (fr) Procédé de traitement vidéo, dispositif électronique, et support d'enregistrement lisible par ordinateur
WO2020001197A1 (fr) Procédé de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur
US11178324B2 (en) Focusing method and device, electronic device and computer-readable storage medium
WO2019233393A1 (fr) Procédé et appareil de traitement d'image, support de stockage et dispositif électronique
US10896323B2 (en) Method and device for image processing, computer readable storage medium, and electronic device
US11882357B2 (en) Image display method and device
WO2019233394A1 (fr) Procédé et appareil de traitement d'image, support de stockage et dispositif électronique
WO2020259179A1 (fr) Procédé de mise au point, dispositif électronique et support d'informations lisible par ordinateur
WO2019233266A1 (fr) Procédé de traitement d'image, support de stockage lisible par ordinateur et dispositif électronique
US11233933B2 (en) Method and device for processing image, and mobile terminal
WO2019233341A1 (fr) Procédé et appareil de traitement d'images, support d'informations lisible par ordinateur, et dispositif électronique
WO2019237887A1 (fr) Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur
WO2019233392A1 (fr) Procédé et appareil de traitement d'image, dispositif électronique et support d'informations lisible par ordinateur
CN110276767A (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
WO2019233271A1 (fr) Procédé de traitement d'image, support d'informations lisible par ordinateur et dispositif électronique
US20220189029A1 (en) Semantic refinement of image regions
WO2019223594A1 (fr) Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile
US11301711B2 (en) Systems and methods for enhancing real-time image recognition
WO2019233260A1 (fr) Procédé et appareil d'envoi d'informations de publicité, support d'informations, et dispositif électronique
WO2019223513A1 (fr) Procédé de reconnaissance d'image, dispositif électronique et support de stockage
CN108848306B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
WO2023125750A1 (fr) Procédé et appareil de débruitage d'image, et support de stockage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19825385

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19825385

Country of ref document: EP

Kind code of ref document: A1